# Motivation
Provide an efficient framework for control plane e2e tests automation of CDC-related features on the cloud.
1. Easy runs framework in Dataflow Service repo with
    - Triggered by Makefile cmd
    - Triggered by GitHub Action
2. Nice report for troubleshooting
3. Easy grouping cases for different scenarios
4. Easy for the developer to setup the test environment when developing features  
5. Support running cases in different environment.
...


# Framework overview

1. Case abstraction
```golang
type E2ECase interface {
    Name() string
    // declare case dependency requirements, such as what kinds of kafka or tidb cluster are needed
    DeclareDependencyRequirements() map[string]DependencyRequirement
    // provide method for case driver to assign dependency entity for the case
    FulfillDependencyRequirement(requirementName string, entity DependencyEntity)
    SetT(t *testing.T)
    SetLogger(lgr *log.CaseLogger)
    // the steps of cases
    Run()
}


```
2. Case driver abstraction
    - Automatically tidb clusters management
    - Automatically downstreams management (TBD)
```golang
type CaseDriver interface {
    // according to cases' tidb cluster and cdc downstream requirements, combine with existed tidb clusters and cdc downstreams, cases dirver will provision enough tidb clusters and cdc downstreams and assign to the cases
    SetUp(tidbClusterIds []string, cdcDownstreams []*CDCDownstream)
    // run all cases 
    Run(t *testing.T)
    // cleanup the tidb clusters and cdc downstreams which provisioned by case driver
    TearDown()
}

```
3. Common steps abstraction, provide 4 basic functions to assemble case steps for easy maintainance.
```golang
// do sth with retry
func (k *BaseCase) RetryDo(action string, doF func() error, attempts uint, delay time.Duration) *BaseCase {}
// expect reach a specific status after do sth
func (k *BaseCase) Expect(status string, statusF func() error, attempts uint, delay time.Duration) *BaseCase {}
// expect status can be stable for a certain period
func (k *BaseCase) Still(status string, statusF func() error, attempts uint, delay time.Duration) *BaseCase {}
// allow user to define steps which can not easy to implement by above 3 functions
func (k *BaseCase) Step(name string, stepF func() error) *BaseCase {}

```
4. Cases management, registry and group cases.
5. Configuration management, support configuration for different environments, offering fine-grained control for cases running.
6. Case specific logger, logs with cases specific prefix to make it easy to trouble shooting when running cases parallelly.
7. If you want add a new dependency, you need to implement following interfaces. After that, you need to register the `Dependency` to `DependenciesManager`. For more detail, you can refer to the implementation of `KafkaDependency` and `TiDBDedicatedDependency`
```golang
// DependencyCharacteristic defined what kinds dependency you want.
type DependencyCharacteristic interface {
    Type() string
    // Dimensions are the details characteristic of the dependency
    Dimensions() map[string]string
}

// DependencyRequirement is for the case to declare it's dependency requirement
type DependencyRequirement interface {
    DependencyCharacteristic
    // IsDedicated means the case want a dedicated dependency entity, won't share with other cases
    IsDedicated() bool
}

// DependencyEntity is a concrete dependency entity which can be assigned to cases
type DependencyEntity interface {
    DependencyCharacteristic
    // IsCreatedByDriver means the dependency entity is created by the case driver, case driver will destroy it after case running done
    IsCreatedByDriver() bool
}

// Dependency is about management of the same type of DependencyEntity
type Dependency interface {
    Type() string
    GetExistingUsableEntities() ([]DependencyEntity, error)
    ProvisionEntities(requirementStats entitiesStats) ([]DependencyEntity, error)
    Delete(entities ...DependencyEntity) error
}

```

# How to write a case?
1. Copy `./config.common.yaml` to `./config.local.yaml`, make change to `./config.local.yaml` according to the comment. The tracked `config.common.yaml` and `config.{dev,staging,prod}.yaml` files use `suitePlanRegions`. These files are shared by Dedicated, Premium, and Essential suites; the suite selects only its own plan's regions at runtime. Pass the chosen file through `CONFIG_FILE`, for example `CONFIG_FILE=$(pwd)/config.local.staging.yaml make dedicated-e2e-test-local`.
2. Go `./cases/changefeed` (if you case not belong to this product, please create a new dir under `./cases`) , create a go file for you case, like xxx_case.log, implement following methods.
```golang
type XXXCase struct {
    BaseCase // by embed BaseCase you can reusing the common steps abstraction functions
}
func NewXXXCase() E2ECase {
    return &XXXCase{}
}

func (k *XXXCase) Name() string {
    return "XXXCaseName"
}

func (k *XXXCase) TiDBClusterRequirements() map[string]TiDBClusterRequirement {
    panic("implement me")
}

func (k *XXXCase) CDCDownstreamRequirements() map[string]CDCDownstreamRequirement {
    panic("implement me")
}

func (k *XXXCase) Run() {
    panic("implement me")
}

```
3. Comon utils/steps
    - you can find some common steps in `./cases/common`, if not found, please add to it.
    - you can to call api, refer to: 1) cdc console api `./internal/tidbcloud/consoleapi/cdc_client_wrapper.go`; 2) cluster console api `./internal/tidbcloud/consoleapi/cluster_non_gen/client.go`
    - if you want to write sample data to tidb cluster, please refer to `/internal/sample_data/write.go`
4. Logging, please use `k.Logger()` for logging.
5. Registry, you should registry your case in `./internal/case_mgnt/driver/case_registry.go`, let case driver found you case. You may also need to put case in some special group.
6. Run case, you shoud add you case to the `config.local.yaml` and execute command `make e2e-test-local` at the root dir of this repository
```yaml
...
cases2Run:
  - XXXCaseName

```

# How to run case in CI?

1. Go to https://github.com/tidbcloud/dataflow-service/actions
2. Choose workflow 'Run E2E Test Cases', click 'Run workflow', input parameters and run.

## Cloud credential placeholders

Tracked E2E configuration contains placeholders instead of static cloud access
credentials. Supply the corresponding values through environment variables:

| Configuration | Environment variable |
|---|---|
| S3 access key | `S3_ACCESS_KEY` |
| S3 secret key | `S3_SECRET_KEY` |
| Date-integration Azure Blob SAS | `AZURE_SAS_TOKEN` |

The Dedicated and Premium GitHub workflows map these variables from repository
secrets `E2E_S3_AK`, `E2E_S3_SK`, and
`E2E_AZURE_SAS_TOKEN`. Local runs may export the runtime variables in the
invoking shell; do not write their values into tracked or ignored configuration
files.

# Tips for development efficiency

## TiDB Cloud E2E authentication

The configured `tidbCloudToken` is used first. When it is empty, the Go E2E
runner logs in through the existing Puppeteer helper. In GitHub Actions, the
workflow selects the environment-specific secrets and exports the selected
values as generic `INPUT_TIDBCLOUD_USER` and `INPUT_TIDBCLOUD_PASSWORD`
variables. The existing CI config-preparation script writes those values into
the generated environment config consumed by the Go runner. The source
secrets are `TIDBCLOUD_STG_LOGIN_EMAIL`/`TIDBCLOUD_STG_LOGIN_PASSWD` for
`staging`, and `TIDBCLOUD_PROD_LOGIN_EMAIL`/`TIDBCLOUD_PROD_LOGIN_PASSWD` for
`prod`. Email/password authentication uses the official Puppeteer Docker image
in both local runs and GitHub Actions; neither path requires Puppeteer, Chrome,
or browser libraries on the host. Do not commit generated credentials or print
them in logs.

Case names do not select an E2E suite. For example, dispatch
`awsEssentialV1CDCInvalidParamsSmoke` with `service_plan: essential-v1` (or
`SERVICE_PLAN=essential-v1` through the Make target); the workflow does not
infer the suite from the case name.

When developing cases, it will invovle many times code changes and cases rerun. Please manually configure following fields in you config file `config.local.yaml` to shorter the case runnning time.

```yaml
# manual set tidb cloud console token instead of letting case dirver do it automatically (may not very stable)
tidbCloudToken: "add token here"
# prepare a cluster instead of provisioned by case driver every time
tidbDedicatedClusterIds: ["add cluster id here"]
orgID: "add org id here" 
projectID: "add project id here" 
cases2Run:
  - "you case name"
```