# Port Dedicated DM E2E Cases to Premium

## Overview

Port 5 DM (Data Migration) e2e test cases from `test/e2e/cases/dedicated/dm/` to `test/e2e/cases/premium/dm/`, adapting them from the dedicated DM API (`/api/v1/orgs/{org_id}/projects/{project_id}/clusters/{cluster_id}/dataflow/dm/`) to the premium DM API (`/api/v2025/cloud/v1beta2/tidbs/{tidbId}/premiumMigrations`).

## Source Cases (Dedicated)

| File | Case Constructor | Description |
|------|-----------------|-------------|
| `dm_existing_migration_smoke.go` | `NewDMFullMigrationSmokeCase` | Full migration (dump + load): create → run → finish → delete |
| `dm_incremental_migration_smoke.go` | `NewDMIncrementalMigrationSmokeCase` | Incremental migration: create → run → pause → resume → delete |
| `dm_job_lifecycle_smoke.go` | `NewDMJobLifecycleSmokeCase` | Full lifecycle: create → list → run → get → pause → resume → delete |
| `dm_precheck_smoke.go` | `NewDMPrecheckSmokeCase` | Precheck: submit → wait → verify results |
| `dm_scale_smoke.go` | `NewDMScaleSmokeCase` | Scale: create → run → scale spec → verify → delete |

## Analysis

### API Differences: Dedicated vs Premium DM

| Aspect | Dedicated DM | Premium DM |
|--------|-------------|------------|
| **Swagger** | `api/dm.swagger.json` | `api/premium_dm.ui.swagger.json` |
| **Path prefix** | `/api/v1/orgs/{org_id}/projects/{project_id}/clusters/{cluster_id}/dataflow/dm/` | `/api/v2025/cloud/v1beta2/tidbs/{tidbId}/` |
| **Path params** | `org_id`, `project_id`, `cluster_id` | `tidbId` (cluster ID only) |
| **Job entity** | `DmJob` (with `SubTasks[].Stage`) | `PremiumMigration` (with `SubTasks[].Stage`, `State`) |
| **Job ID field** | `id` | `migrationId` |
| **Create body** | `name`, `mode`, `importMode`, `specID`, `clusterUser/Password`, `sources[].connProfileID`, `precheckID`, `fullInstanceMigrate` | `displayName`, `mode`, `importMode`, `sources[]`, `target.user/password` |
| **Source model** | `DmSource` with `ConnProfile` or `ConnProfileID` + `BaRules` | `PremiumDMSource` with inline `ConnProfile` + `PremiumMigrationRules` |
| **Operate job** | `PUT .../jobs/{id}/status` with `Operation` (Pause/Resume) | `POST .../premiumMigrations/{id}:pause` / `:resume` (separate endpoints) |
| **Scale job** | `POST .../jobs/{id}/scale-up-down` with `toSpecID` | **NOT available** in premium API |
| **Precheck** | `POST .../dm/pre-check` | `POST .../premiumMigrationPrechecks` |
| **Job modes** | `full`, `incremental`, `all` | `MODE_UNSPECIFIED`, `ALL`, `INCREMENTAL` (no "FULL" — only `ALL`) |
| **Import modes** | `logical`, `physical` | `IMPORTMODE_LOGICAL`, `IMPORTMODE_PHYSICAL` |
| **Sub-task stages** | `Running`, `Paused`, `Failed`, `Finished`, etc. (lowercase) | `RUNNING`, `PAUSED`, `FAILED`, `FINISHED` (uppercase enum) |
| **ConnProfile** | Separate conn-profile registration API | Inline in source, or validate via `validateSourceConnection` |
| **Cluster entity** | `TiDBDedicatedEntity` (has ProjectID) | `TiDBPremiumEntity` (no ProjectID) |
| **Dependency** | `TiDBDedicatedRequirement` | `TiDBPremiumRequirement` |

### Key Adaptations Required

1. **No Scale API**: Premium DM does not have a scale endpoint → **Skip porting `dm_scale_smoke.go`**.
2. **No separate ConnProfile API**: Premium DM uses inline `ConnProfile` in source or `validateSourceConnection` → No need for `ConnProfileClientWrapper`.
3. **No `full` mode**: Premium DM only supports `ALL` and `INCREMENTAL` → The "full migration" case maps to `ALL` mode (which includes dump + load + sync; we can delete the job after the load phase completes, similar to dedicated "full").
4. **No `specID`**: Premium DM create body doesn't have `specID` → Removed from helpers.
5. **No `precheckID` in create body**: Premium DM create doesn't reference precheck → Precheck is done independently.
6. **Separate pause/resume endpoints**: Instead of a single `OperateJob(op)` method, need separate `PauseJob` / `ResumeJob` methods.
7. **State vs Sub-task stage**: Premium has both `State` (top-level: CREATING, RUNNING, PAUSED, FAILED, DELETING) and `SubTasks[].Stage` per sub-task.

### Cases to Port (4 of 5)

| Dedicated Case | Premium Case | Notes |
|---------------|-------------|-------|
| `dm_existing_migration_smoke.go` | `dm_full_migration_smoke.go` | Use `ALL` mode instead of `full`; use inline ConnProfile; no specID |
| `dm_incremental_migration_smoke.go` | `dm_incremental_migration_smoke.go` | Use `INCREMENTAL` mode; separate pause/resume endpoints |
| `dm_job_lifecycle_smoke.go` | `dm_job_lifecycle_smoke.go` | Use `INCREMENTAL` mode; adapt list/get/pause/resume |
| `dm_precheck_smoke.go` | `dm_precheck_smoke.go` | Adapt precheck body; use PremiumMigration precheck API |
| ~~`dm_scale_smoke.go`~~ | **SKIP** | Premium DM has no scale API |

## Execution Steps

### Step 1: Generate Premium DM API Client

Add `generate-premium-dm-api-client` target to `test/e2e/Makefile`:

```makefile
.PHONY: generate-premium-dm-api-client
generate-premium-dm-api-client: $(SWAGGER_BIN)
	@echo "Generating Premium DM Go client SDK using swagger..."
	@rm -rf $(PREMIUM_CONSOLE_API_CLIENT_DIR)/dm
	@mkdir -p $(PREMIUM_CONSOLE_API_CLIENT_DIR)/dm
	$(SWAGGER_BIN) generate client \
		-f ../../api/premium_dm.ui.swagger.json \
		-A premiumDm \
		--tags=PremiumMigrationService \
		--target=$(PREMIUM_CONSOLE_API_CLIENT_DIR)/dm \
		--client-package=client \
		--model-package=model
```

Also add it to the `generate-api-client` dependency list.

Run: `cd test/e2e && make generate-premium-dm-api-client`

**Verification**: `ls test/e2e/internal/tidbcloud/premium/consoleapi/dm/client/` should exist with generated code.

### Step 2: Update Premium `singleton_client.go`

Add the premium DM client singleton:

```go
import (
    dmclient "github.com/tidbcloud/dataflow-service-e2e/internal/tidbcloud/premium/consoleapi/dm/client"
)

var premiumDMClient *dmclient.PremiumDm

func doOnce() {
    transport := newTransportWithBearerToken()
    bizChangefeedClient = cdcclient.New(transport, nil)
    premiumDMClient = dmclient.New(transport, nil)
}

func PremiumDMClient() *dmclient.PremiumDm {
    once.Do(doOnce)
    return premiumDMClient
}
```

**Verification**: `cd test/e2e && go build ./...`

### Step 3: Create Premium DM Client Wrapper

Create `test/e2e/internal/tidbcloud/premium/consoleapi/dm_client_wrapper.go`:

A wrapper providing high-level methods around the generated client:
- `CreateMigration(cluster, body)` → returns migrationId
- `GetMigration(cluster, migrationId)` → returns PremiumMigration
- `ListMigrations(cluster, pageSize, pageToken)` → returns list
- `DeleteMigration(cluster, migrationId)`
- `PauseMigration(cluster, migrationId)`
- `ResumeMigration(cluster, migrationId)`
- `CreatePrecheck(cluster, body)` → returns precheckId
- `GetPrecheck(cluster, precheckId)` → returns PremiumMigrationPrecheck
- `CancelPrecheck(cluster, precheckId)`

All methods take `*TiDBPremiumEntity` and use `cluster.ClusterID` as the `tidbId` path parameter.

**Verification**: `cd test/e2e && go build ./internal/tidbcloud/premium/consoleapi/...`

### Step 4: Create Premium DM Validator

Create `test/e2e/cases/premium/common/dm_validator.go`:

Analogous to `dedicated/common/dm_validator.go` but using:
- `TiDBPremiumEntity` instead of `TiDBDedicatedEntity`
- `PremiumMigrationState` and `PremiumDMSubTaskStage` enums
- Premium DM client wrapper

Methods:
- `IsMigrationInState(cluster, id, expectedState)` — check top-level state
- `IsMigrationRunning(cluster, id)` — state == RUNNING
- `IsMigrationPaused(cluster, id)` — state == PAUSED
- `IsMigrationDeleted(cluster, id)` — not found in list
- `WaitForSubTaskStage(cluster, id, expectedStage, timeout)` — poll sub-task stage
- `IsPrecheckDone(cluster, precheckId)` — status != RUNNING

**Verification**: `cd test/e2e && go build ./cases/premium/common/...`

### Step 5: Create Premium DM Test Helpers

Create `test/e2e/cases/premium/dm/test_helpers.go`:

Constants and builder functions adapted for premium DM API:
- Mode constants: `PremiumDMModeAll`, `PremiumDMModeIncremental`
- Import mode constants: `PremiumDMImportModeLogical`, `PremiumDMImportModePhysical`
- `GenerateDMJobName(prefix)` — same as dedicated
- `BuildPremiumDMSource(mysql)` — build `PremiumDMSource` with inline ConnProfile
- `BuildCreateMigrationBody(name, mode, importMode, cluster, mysql, migrationRules)` — build create body
- `BuildPrecheckBody(name, mode, importMode, cluster, mysql)` — build precheck body

**Verification**: `cd test/e2e && go build ./cases/premium/dm/...`

### Step 6: Port DM Test Cases (4 cases)

#### 6a: `dm_full_migration_smoke.go`
Adapted from `dedicated/dm/dm_existing_migration_smoke.go`:
- Use `TiDBPremiumRequirement` / `TiDBPremiumEntity`
- Use `ALL` mode (equivalent to dedicated `full`)
- No separate conn profile step — inline in source
- No `specID` or `precheckID` in create body (precheck is run independently before create)
- Wait for sub-task stage `FINISHED`
- Delete and verify

#### 6b: `dm_incremental_migration_smoke.go`
Adapted from `dedicated/dm/dm_incremental_migration_smoke.go`:
- Use `TiDBPremiumRequirement` / `TiDBPremiumEntity`
- Use `INCREMENTAL` mode
- Separate `PauseMigration` / `ResumeMigration` calls
- Verify stays running (Still check)

#### 6c: `dm_job_lifecycle_smoke.go`
Adapted from `dedicated/dm/dm_job_lifecycle_smoke.go`:
- Full lifecycle: create → list → run → get details → pause → resume → delete
- Verify mode and state via `GetMigration`

#### 6d: `dm_precheck_smoke.go`
Adapted from `dedicated/dm/dm_precheck_smoke.go`:
- Submit precheck, wait for completion, verify items

**Verification**: `cd test/e2e && go build ./cases/premium/dm/...`

### Step 7: Register Cases in Premium Registry

Update `test/e2e/internal/case_driver/cases_registry_premium.go`:
- Import `premium/dm` package
- Register all 4 cases for AWS, GCP, Azure, AliCloud
- Add to case groups: `awsSmoke`, `gcpSmoke`, `azureSmoke`, `alicloudSmoke` (DM precheck + full migration)
- Add to `awsWeeklyRegression`, etc.

**Verification**: `cd test/e2e && go build ./...`

### Step 8: Update Makefile Generate Target

Add `generate-premium-dm-api-client` to the `generate-api-client` dependencies in `test/e2e/Makefile`.

**Verification**: `make -n generate-api-client` should list the new target.

### Step 9: Final Build Verification

```bash
cd test/e2e && go build ./...
```

## Files Created/Modified

| Action | File | Description |
|--------|------|-------------|
| Modified | `test/e2e/Makefile` | Add `generate-premium-dm-api-client` target |
| Generated | `test/e2e/internal/tidbcloud/premium/consoleapi/dm/` | Generated swagger client |
| Modified | `test/e2e/internal/tidbcloud/premium/consoleapi/singleton_client.go` | Add DM client singleton |
| Created | `test/e2e/internal/tidbcloud/premium/consoleapi/dm_client_wrapper.go` | DM API wrapper |
| Created | `test/e2e/cases/premium/common/dm_validator.go` | DM validation helpers |
| Created | `test/e2e/cases/premium/dm/test_helpers.go` | Test constants and builders |
| Created | `test/e2e/cases/premium/dm/dm_full_migration_smoke.go` | Full migration test |
| Created | `test/e2e/cases/premium/dm/dm_incremental_migration_smoke.go` | Incremental migration test |
| Created | `test/e2e/cases/premium/dm/dm_job_lifecycle_smoke.go` | Lifecycle test |
| Created | `test/e2e/cases/premium/dm/dm_precheck_smoke.go` | Precheck test |
| Modified | `test/e2e/internal/case_driver/cases_registry_premium.go` | Register DM cases |

## Risk Assessment

- **Low risk**: Only creating new test code; no changes to production service code.
- **API model mismatch risk**: The premium DM API models differ significantly from dedicated. The generated swagger client will define the exact types. Test case logic must be adapted to match.
- **`ALL` mode mapping**: Dedicated "full" mode does dump+load then finishes. Premium "ALL" mode does dump+load+sync (continuous). The full migration test may need to check for the load phase completion differently (e.g., wait for sub-task step to be `SYNC` or stage `FINISHED` after pausing).