# Dataflow Service E2E Module

This Go module tests user-visible Dataflow Service flows against real TiDB Cloud dependencies. Keep four contracts intact: deterministic case selection, correct dependency assignment, isolated cleanup, and a traceable report.

Read the parent [repository guidance](../../AGENTS.md) first. This file records development contracts; the closest case, validator, dependency, or client wrapper remains authoritative.

## Safety

- Keep `mustCleanup: true` for normal runs. The driver releases provisioned dependencies, and every case must delete API resources it creates.
- `config.local*.yaml`, `run_test.log`, and `e2e-report.xml` are local-only and may contain sensitive data.
- Cloud runs mutate resources. Confirm workflow inputs and obtain authorization before dispatching one.

## Layout

| Area | Sources |
|---|---|
| Entry and teardown | `run_test.go`, `run_dedicated_test.go`, `run_premium_test.go`, `run_essential_test.go` |
| Selection, provisioning, execution, cleanup | `internal/case_driver/` |
| Scenarios and reusable assertions | `cases/`, `cases/{dedicated,premium,essential}/common/` |
| Config, API clients, CI adaptation | `internal/config/`, `internal/tidbcloud/`, `script/ci_inputs_to_env_config.js` |
| Commands and client generation | `Makefile` |

Dedicated covers backup, changefeed, CRRF, DM, and import; Premium covers changefeed, DM, and import. Select a suite from the architecture and available client wrappers—matching package names do not imply parity.

The canonical GitHub Actions entrypoint is `.github/workflows/e2e-test.yaml`.
Its `service_plan` input maps `dedicated` to the Dedicated registry,
`essential-v2` and `premium` to the Premium registry, and `starter` and
`essential-v1` to the Essential v1 registry. The Starter and Essential v2
values are suite aliases; they do not parameterize the real cluster service
plan. The legacy `run-premium-e2e` Make target remains a compatibility alias,
and `run-essential-e2e` dispatches the Essential v1 alias.

Essential v1 covers AWS and AliCloud Serverless clusters with explicit
`servicePlan=Essential`, `minRcu=2000`, and `maxRcu=4000` dependency
dimensions. Use `run_essential_test.go` and `RegisterEssentialCases`. Cases
use the Premium-aligned flat domain packages `backup`, `changefeed`, `dm`,
`export`, and `import_data`. Each logical case has a descriptively named file
in the domain package; keep substantive case-specific validators in matching
validator files, but call meaningful shared validators directly instead of
adding forwarding wrappers. Do not add nested per-case packages or
compatibility constructor wrappers. Console/OpenAPI differences stay hidden
behind normalized Essential facades. The
registry exposes core smoke groups plus provider weekly, new-region, domain,
negative, cancellation, and PrivateLink groups.
Snapshot Restore remains implemented but is intentionally unregistered until a
scheduled fixture can guarantee an automatic backup; PiTR restore is
Console-only, and an explicit OpenAPI selection fails fast.

## Harness Contracts

### Lifecycle and selection

`TestMain` accepts `-env`, `-domain`, `-api-mode`, `-config-file`, and `-async-deps`. `InitConfig` merges `config.common.yaml` with the selected environment file, requires and projects the selected suite plan's entry from `suitePlanRegions` into the runtime region accessors, then applies the domain and API-mode overrides. Tier registries filter and group cases by the effective domain.

- `cases2Run` selects exact, case-sensitive names; `caseGroup2Run` accepts one or more comma-separated group names. Both may be set at once — their cases combine as a union, with duplicates run once. Leaving both empty is an error.
- Dependencies provision before execution. With `async-deps=true`, a case starts when its dependencies are ready; otherwise all provisioning completes first.
- Suite teardown waits for all provisioning goroutines to finish before its final dependency inventory cleanup, including when asynchronous provisioning is enabled.
- Cases are parallel subtests, have a 40-minute dependency wait, and do not have a stable registry/log order: the driver iterates a map and calls `t.Parallel()`.
- When `mustCleanup` is true, dependency release is registered through each subtest's `t.Cleanup` before `t.Parallel()` and dependency waiting. This covers successful cases, `FailNow`, dependency failures, and recovered panics without changing the business result when release fails.

### Cases and validators

Each `E2ECase` must have a unique, case-sensitive `Name`, correct domain tag (`cdc`, `br`, `crrf`, `dm`, `import`, or Essential's `export`), precise dependency requirements, and a `Run` method. Embed `cases.BaseCase`; do not reimplement assignment, typed accessors, logging, or step control.

Essential cases declare their dependency map directly in each case. Keep the
Essential service plan/provider/region and Kafka/MySQL network, authentication,
provider, and region dimensions visible at the call site; do not hide them
behind boolean or network-selector requirement helpers. Public Kafka/MySQL
fixtures remain provider- and region-independent, while PrivateLink fixtures
match the Essential cluster provider and region. The harness initializes
`config.GlobalConfig` before registration; dependency declarations should use
it directly rather than adding nil fallbacks that create an unsupported path.

| Helper | Use |
|---|---|
| `Step` | One-shot action; fail immediately on error |
| `RetryDo` | Retriable API action or idempotent cleanup |
| `Expect` | Poll for an eventual state |
| `Still` | Prove an expected state stays stable |

A validator returns `nil` only after its expected condition holds. Keep reusable API calls and status checks in a tier/domain `common` validator, not unrelated cases.

### Dependencies

Requirements and entities match by type and dimensions. Shared requirements receive the partition's shared entity; dedicated requirements consume a distinct entity and are only for resources unsafe to share concurrently. Each entity records the completion state of cases actually assigned to it. When a case completes, only its assigned entities are inspected; a driver-created entity is queued for parallel deletion after all its recorded users complete. Existing entities (`IsCreatedByDriver() == false`) and unassigned entities are left for suite teardown. A single cleanup loop prevents concurrent duplicate deletion, and suite teardown waits for queued cleanup before retrying failures from its remaining inventory. With `mustCleanup=false`, the driver preserves the existing no-cleanup behavior. Never create an untracked resource directly in a case.

To add a dependency type, add models under `internal/model/`, a `dependency_*.go` implementation (provisioning, matching, cleanup), registration in `NewDependenciesManager`, configuration support, and focused allocator/dependency tests. Supported types include Dedicated, Premium, and Serverless TiDB; Kafka, MySQL, Pulsar; and S3, GCS, and Azure Blob Storage.

## Change Workflow

### Cases

1. Choose the tier, domain, API mode, provider variants, and closest existing case.
2. Define expected transitions and failures in a focused validator.
3. Embed `BaseCase`, declare only the dependencies needed, and register each supported variant with tier-specific `addCases`.
4. Deliberately add variants to smoke, regression, or new-region groups; `all` and provider-wide groups are automatic.
5. Install retryable, partial-creation-safe deferred cleanup before creating a case-owned resource; run `gofmt` on changed Go files. Essential Import CSV uses the authoritative `<database>.<table>.csv` object name and validates the real target schema and row count.

### Tests, configuration, and clients

- Do not add unit tests under `test/e2e/`. Validate changes through `gofmt`, static review, and an explicitly authorized E2E run; E2E framework code is exercised by the scenario suite.
- Preserve the common-plus-environment config overlay. The tracked common/environment profiles use `suitePlanRegions` to keep Dedicated, Premium, and Essential region support separate; do not flatten those entries into one region list. Keep workflow inputs, `ci_inputs_to_env_config.js`, config fields, Make arguments, and suite flags aligned; use an ignored `CONFIG_FILE=/absolute/path/...` override for authorized targeted runs.
- Generated models in `internal/tidbcloud/**/model/` are never hand-edited. Change the generator input, run `make generate-api-client`, then review and update dependent wrappers, validators, and tests.

## Verification

Run from this module unless stated otherwise.

```bash
gofmt -w <changed-go-files>

# User-authorized cloud runs
make dedicated-e2e-test-local DOMAIN=<domain> API_MODE=<mode> CONFIG_FILE=/absolute/path/to/config.local.yaml
make premium-e2e-test-local DOMAIN=<domain> API_MODE=<mode> CONFIG_FILE=/absolute/path/to/config.local.staging.yaml
```

The non-local CI targets write `run_test.log` and `e2e-report.xml`; local
targets write directly to the terminal. CI runs from the repository root
through the unified `make run-e2e` target; set
`SERVICE_PLAN=dedicated`, `SERVICE_PLAN=premium`, or
`SERVICE_PLAN=essential-v1` as needed. `make run-premium-e2e` remains a
compatibility alias and `make run-essential-e2e` is the Essential convenience
alias. The unified workflow accepts explicit
`dedicated_cluster_ids`, `premium_cluster_ids`, and `serverless_cluster_ids`.

## E2E Behavior Skill Sync

For a behavior change to `run_*.go`, `cases/**`, `internal/**`, the E2E `Makefile`, `Dockerfile`, `script/**`, `config*.yaml`, module files, or an E2E workflow, review `../../../../kb/skills/dataflow-service-e2e/`.

- Update `SKILL.md` for trigger, primary-workflow, safety-boundary, or command changes.
- Update `references/framework.md` for lifecycle, config, selection, dependencies, execution, cleanup, reports, or CI; update `references/case-authoring.md` for case, validator, registration, grouping, or dependency-authoring changes.
- Keep registries as the case inventory authority. In the task summary, record changed skill files or why documented behavior did not change.
- Before handoff, verify the skill entry and both references exist, links resolve, and focused Go tests pass. Include any skill update in the same handoff even when committed separately.