# Execution and architecture reference

## Contents

- [Verified scope](#verified-scope)
- [Repository responsibilities](#repository-responsibilities)
- [End-to-end execution flow](#end-to-end-execution-flow)
- [Go runner lifecycle](#go-runner-lifecycle)
- [Action semantics](#action-semantics)
- [GitHub Actions integration](#github-actions-integration)
- [Configuration groups](#configuration-groups)
- [Important implementation constraints](#important-implementation-constraints)
- [Troubleshooting path](#troubleshooting-path)
- [Common first questions](#common-first-questions)

## Verified scope

Runner and workflow facts were verified against
[`tidbcloud/tidb-management-service@ff5c78ac9`](https://github.com/tidbcloud/tidb-management-service/tree/ff5c78ac952849735b14a93ecabcceec4933ea33)
on 2026-07-15. Open the linked current-main files when a later runner change could affect the answer.

## Repository responsibilities

### APITestGen

- `scenario_testcases/`: reusable scenario YAML source consumed by scenario workflows.
- `skills/`: project workflows for framework onboarding, authoring, and review.
- `llm-scenario-generator/`: LLM-driven exploration and raw scenario generation.
- `log-to-reusable/`: conversion logic for turning exploration transitions into reusable steps.
- `.github/workflows/nl2scenario.yaml`: manual natural-language exploration/generation workflow.

The authoring/generation utilities do not execute committed scenario YAML in CI. Execution is owned by the other repository.

### tidb-management-service

The runner and workflow knowledge is included here so an agent can answer without cloning another repository.

| Responsibility | GitHub source |
|---|---|
| Test entry, setup, execution, summary, teardown | [`Scenario_suite_test.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/Scenario_suite_test.go) |
| Environment/config-file loading | [`testmain_test.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/testmain_test.go), [`functional-test/pkg/config/setup.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/pkg/config/setup.go) |
| YAML/JSON models and parsers | [`model/generationsteps.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/model/generationsteps.go) |
| Placeholder substitution and action construction | [`actionbuilder.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/actionbuilder.go) |
| HTTP/command execution, retries, CEL saves | [`action.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/action.go) |
| Embedded YAML loader | [`testdata/fs.go`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/cases/openapidedicated/autogencase/testdata/fs.go) |
| Make target and JUnit output | [`functional-test/e2e-commands.mk`](https://github.com/tidbcloud/tidb-management-service/blob/main/functional-test/e2e-commands.mk) |
| Dedicated workflow | [`e2e-openapi-dedicated-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-openapi-dedicated-scenario.yaml) |
| BYOC workflow | [`e2e-openapi-byoc-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-openapi-byoc-scenario.yaml) |
| Essential workflow | [`e2e-openapi-essential-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-openapi-essential-scenario.yaml) |
| Essential V2 workflow | [`e2e-openapi-essential-v2-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-openapi-essential-v2-scenario.yaml) |
| NextGen global workflow | [`e2e-openapi-nextgen-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-openapi-nextgen-scenario.yaml) |
| NextGen regional workflow | [`e2e-regionalapi-nextgen-scenario.yaml`](https://github.com/tidbcloud/tidb-management-service/blob/main/.github/workflows/e2e-regionalapi-nextgen-scenario.yaml) |

## End-to-end execution flow

1. A scenario workflow accepts environment, provider, region, tenant/project, auth, config, and one or more YAML filenames.
2. `Prepare matrix` converts the provider-specific comma-separated filenames into one matrix job per YAML file.
3. The job installs the provider/runtime tools required by command steps and configures cloud identity.
4. `Fetch scenario cases yaml files` copies or downloads the selected APITestGen YAML into `functional-test/cases/openapidedicated/autogencase/testdata/`.
5. The workflow exports `OPEN_API_DEDICATED_*` values, including the selected filename and `READ_CASE_FROM_FS`.
6. `Run scenario cases` runs:

   ```sh
   make GO_TEST_RUN_ARGS='TestScenario' GO_TEST_TESTIFY_METHOD='.*' \
     test-openapidedicated-autogencase
   ```

7. The Make target invokes `gotestsum`, runs only the `TestScenario` Go test, allows all testify suite methods, and writes `functional-test/e2e-tests.xml`.
8. `test-summary/action` reads the JUnit file and the workflow aggregates the matrix results.

Copying must happen before `go test`: `testdata/fs.go` uses `//go:embed *.yaml`, so the selected file is compiled into the test binary.

## Go runner lifecycle

### Configuration and initial placeholders

`TestMain` configures Viper with the `OPEN_API_DEDICATED` prefix. Environment variables override an optional YAML config file.

`TestScenario` creates the suite and seeds placeholders including:

- `open_api_key`, `project_id`, `org_id`, and `tenant_id`;
- `region`, `region_id`, and `cloud_provider`;
- optional auth-role keys;
- provider credentials exposed by the workflow;
- optional Azure SAS tokens;
- optional generated cluster/shared-pool IDs.

If `region` is empty, the current runner supplies a provider-specific default. Do not rely on this when the scenario contract requires a specific region.

### SetupSuite

`SetupSuite`:

1. validates the selected YAML, project, provider, region, API key, and database DSNs for ordinary hosts;
2. initializes the v1beta1 Dedicated client used by teardown;
3. opens the result and generator databases and creates both DAOs;
4. loads steps from the generator DB when `read_case_from_fs` is false, otherwise loads the selected embedded YAML;
5. parses every request, expectation, and non-null save JSON before executing any step;
6. prepares provider clients, including AWS cleanup support.

Database initialization is currently unconditional. Even with filesystem YAML and `save_result_to_db=false`, a full local run still needs reachable tester and generator DSNs.

### Test_001_ExecuteActions

The suite iterates the `generationSteps` slice in its existing order. Filesystem YAML is appended in YAML sequence and is not sorted by `order`; keep sequence and `order` aligned.

For each step, the runner:

1. logs the current step and placeholder map;
2. builds an HTTP or command action;
3. executes the action and its expectation with bounded retries;
4. fails immediately unless `continue_on_unexpected` is true;
5. copies successful saved values into the scenario placeholder map;
6. appends a result record.

### Summary and teardown

`Test_999_Summary` annotates result records and writes them only when `save_result_to_db` is true. The database connection and DAO were already created during setup.

`TearDownSuite` runs unless `skip_tear_down` is true. It treats every placeholder key beginning with `cluster` as a cluster ID and attempts v1beta1 deletion plus a v1beta2 fallback. AWS also deletes rejected VPC endpoints from the configured test VPC; other provider cleanup remains scenario-owned.

## Action semantics

- Supported executable request types are `http`, `cmd_aws`, `cmd_gcp`, `cmd_azure`, `cmd_mysqlsh`, and `cmd_aliyun`. SDK constants exist in the model but are not built by `ActionBuilder`.
- Relative HTTP paths become `https://{host}{scenario.context.base_path}{request.path}`. Absolute URLs bypass this assembly.
- `scenario.context.host` overrides the environment host; `.env.` patterns are resolved from the configured environment.
- HTTP auth supports digest, bearer-token headers, and customized `X-Auth-Content` behavior. Every HTTP step needs a compatible non-null auth object in the current builder.
- `path_params`, `body_params`, and `args_params` map request placeholders to scenario placeholder keys.
- A top-level request-body `displayName` receives a timestamp suffix at runtime.
- Command placeholders are not expanded inside arguments beginning with `--`; keep a dynamic flag value in the following array item.
- HTTP expectations use `status_code` plus optional boolean CEL over `body`. Command expectations use `exit_code` plus optional boolean CEL over parsed `output`.
- Total attempts are `max_retries + 1`; the delay budget is `max_retries * delay_after` seconds.
- Saves happen only after a successful action. HTTP save CEL exposes `body` and `saved_placeholder`; command save CEL currently exposes the misspelled `saved_palceholder`.
- Saved CEL values must reduce to string, integer, boolean, or double so the runner can store them as strings.
- `continue_on_unexpected` allows later steps to run, but a failed action does not produce its response/output saves.

## GitHub Actions integration

- Dedicated, BYOC, Essential, NextGen global, and NextGen regional workflows currently fetch scenario YAML from APITestGen `main`.
- Essential V2 checks out APITestGen with the `apitestgen_branch` input, so it can validate YAML from an unmerged branch.
- `fetch_from_apitestgen_repo=true` drives both the fetch/copy step and `OPEN_API_DEDICATED_READ_CASE_FROM_FS=true`.
- When filesystem loading is disabled, setup selects generation steps from the generator DB by `generation_id` instead of loading the selected embedded file. At the verified revision, that branch does not initialize `scenarioContext`, while HTTP action construction dereferences it; do not treat DB-backed HTTP execution as a working path without re-checking or fixing the runner.
- Matrix filenames must exactly match files under `scenario_testcases/`.
- External automation may dispatch these workflows through the GitHub REST API and wrap each trigger script with restricted audit logging. Treat the script structure as an operational pattern only; never copy literal credentials or environment-specific values from an automation host. See [runtime-configuration.md](runtime-configuration.md).

## Configuration groups

Use the workflow files as the exact current source for parameter names, control flow, and secret bindings. Never copy literal environment or credential values from them. The runner consumes these groups:

- target: `ENV`, `HOST`, `PROJECT_ID`, `TENANT_ID`, `CLOUD_PROVIDER`, `REGION`;
- auth: `API_KEY`, optional `AUTH_ROLE`, `ACCOUNT_TOKEN`, or login URL/email/password;
- case source: `READ_CASE_FROM_FS`, `SCENARIO_YAML`, and DB-backed `GENERATION_ID`;
- persistence: `DB_API_TESTER_DSN`, `DB_API_GENERATOR_DSN`, `SAVE_RESULT_TO_DB`, `EXECUTION_ID`;
- lifecycle: `SKIP_TEAR_DOWN`, optional generated cluster/shared-pool IDs;
- provider integrations: AWS profile/credentials, AliCloud credentials, and Azure SAS values.

Prefix every key with `OPEN_API_DEDICATED_` when using environment variables.

## Important implementation constraints

- The APITestGen YAML must exist in testdata before Go compilation because of `go:embed`.
- YAML sequence, not the numeric `order` field, determines execution order.
- `request`, `expect`, and non-null `save` are JSON strings embedded in YAML and are parsed during suite setup.
- A zero or missing meaningful expectation can turn a scenario into a transport-only smoke test; assert the requested behavior with status/exit and CEL.
- A non-zero command exits before the current runner can evaluate a non-zero `exit_code` expectation. Treat new command cases as success-exit assertions unless the runner changes.
- A failed continued step cannot seed downstream placeholders.
- Keys beginning with `cluster` are reserved for actual cluster IDs because teardown consumes them.
- Automatic teardown does not replace explicit cleanup for backups, endpoints, buckets, IAM objects, BYOC resources, or other child resources.
- `save_result_to_db=false` disables inserts, not database initialization.
- The DB-backed source branch currently has inconsistent prerequisites: validation still requires `scenario_yaml`, and HTTP action construction expects a scenario context that the branch does not set.

## Troubleshooting path

Start with the failing matrix YAML filename, then inspect in this order:

1. `Prepare matrix`: confirm the exact filename and provider-specific input.
2. `Fetch scenario cases yaml files`: confirm the intended APITestGen branch and inspect the copied file.
3. Environment: confirm host, environment, provider, region, tenant/project, auth, DSNs, `READ_CASE_FROM_FS`, and `SCENARIO_YAML`.
4. Setup parse errors: validate YAML and embedded `request`, `expect`, and `save` JSON.
5. Placeholder errors: search for `DEBUG: do step` and inspect `saved_placeholder` producer/consumer order.
6. HTTP failures: use the logged URL, method, request body, response status/body, headers, and `x-debug-trace-id`.
7. Command failures: confirm tool installation, provider login/OIDC, argument expansion, JSON output, and command name/type mapping.
8. Results: inspect `functional-test/e2e-tests.xml`; if persistence is enabled, inspect `scenario_result_records` through the approved database access path.

## Common first questions

### Which repository is the source of truth?

APITestGen owns committed reusable YAML. `tidb-management-service` owns execution semantics and GHA wiring. An API's OpenAPI/proto definitions in `tidb-management-service` own the interface contract.

### Does the runner execute `order` numerically?

No. It logs `order` but executes the slice in YAML order. Keep values unique and increasing because humans and review tools use them to reason about the lifecycle.

### Can I validate a YAML without credentials?

Yes, run the repository checker for static validation. A full runner replay is live and requires environment credentials plus both database DSNs in the current implementation.

### Why was my new file not found?

Confirm the workflow fetched the exact filename before `go test`. For local replay, copy the YAML into testdata before compiling because the loader reads embedded files.

### Will teardown remove everything?

No. It automatically targets cluster-prefixed placeholders and limited AWS rejected endpoints. Write explicit cleanup steps for all other resources and respect child-before-parent deletion order.