# Wedge: Premium Reserved

Codename: **controlplane9** | Repo: [tidbcloud/controlplane9](https://github.com/tidbcloud/controlplane9)

Status: DRAFT | Date: 2026-04-28 | Phase-2 lead: @cloud-cp-lead *(placeholder)*

Derives from: [20260425-design-cp9-detail.md](20260425-design-cp9-detail.md) | Research input: Premium Reserved Feishu wiki pull (PM spec + billing spec, 2026-04-28; working extract not committed here) | Supersedes the historical Premium Autoscaler wedge (autoscaler M1 shipped 4/16; remaining cp9 wedge work folds into 07).

> Drift callout: [20260413-design-cp9-stakeholder-guide.md](20260413-design-cp9-stakeholder-guide.md) and [20260417-design-cp9-landing-page.md](20260417-design-cp9-landing-page.md) still cite Premium Autoscaler as the wedge. See [Out of scope and follow-ups](#out-of-scope-and-follow-ups).

> 2026-05-08 update: issue #106 refactors the cp9 env-creation contract. The persona/template and lifecycle snippets below are historical 2026-04 context until replaced by the unified `cp9.yaml` and Tilt-centered lifecycle v3 in [20260506-design-cp9-env-creation-refactor.md](20260506-design-cp9-env-creation-refactor.md). Crossplane and its provider become normal `live` services; staging CRD dump and special `crossplane_install` are no longer valid. Network/tunnel setup is optional for the 07 narrow wedge; local IAM setup is required.

> Historical command-surface note (2026-06-01): this wedge design predates the command cleanup train. Its command examples may mention removed ideas such as `sync`, `onboard`, `config`, `persona`, `crd`, `migrate`, `test`, `cost`, `reaper`, `stub-gen`, `overlay`, `verify`, `feature-cycle`, `topology`, or `up --persona` / `up --discover` / `up --profile` / `up --remote`. Current runnable surfaces are `setup`, `doctor`, `up`, `status`, `logs`, `deploy`, `destroy`, direct `pause`, and `db dump-init`.

---

## TL;DR

Premium Reserved is the new narrowest wedge. Customer buys **purchased vCPU**; backend maps it to a fixed TiDB / TiKV / TiProxy / PD / optional-worker topology via a sizing calculator (per [PM spec](https://pingcap.feishu.cn/wiki/WgSFwgUbyiq6pakXuC1cL2NunAd)); pricing is per-vCPU, billing keys off the cluster status state machine (per [billing spec](https://pingcap.feishu.cn/wiki/IpSUwMBrSiPdn9kzmTkcHZVynee)). 07 ships when an engineer can take the Reserved OpenAPI create/update change from spec to passing E2E inside a per-session cp9 environment, without touching `staging-seed-ng-0711`.

```
                                Today (shared dev)              After 07 ships (cp9 wedge)
                                ----------------------          --------------------------------
  Cara writes code              local                           local
  Build + unit test             Makefile                        Makefile (unchanged)
  Deploy for E2E                staging-seed-ng-0711            Tilt dev loop generated by cp9
                                + 4 sibling teams sharing       (per-session, isolated)
  Customer dataplane            shared cluster pool             OpenAPI create/update provisions
                                                                tagged EKS + tidb-operator + TiDB
  Reserved API                  staging openapi                 in-cluster tidb-mgmt-portal +
                                                                tidb-mgmt-service + nextgen-global
  Billing pipeline              staging metering + billing-svc  metering + billing-svc stubs
                                                                (REAL after create+update green)
  Cleanup                       hope nobody touched it          cp9 destroy: exit 0 only on
                                                                empty post-scan
  Time-to-iterate               hours (queue + flakiness)       minutes (after first cold-start)
  Blast radius on mistake       4 other teams                   nobody else
```

Host decision for this wedge: cp9 stays **all-in k3d**. Local k3d is the default daily-dev host. EC2+k3d and tunnel/network setup are opt-in follow-ups, not required for M1. The customer dataplane remains the real production-provisioned EKS path exercised by OpenAPI create/update.

Dev plan, anchored to the [Global Control 2026-04-23 status](https://pingcap.feishu.cn/wiki/CsT9wM3ziizz9kkwDN4cMHkBnHh):

- **5/19** — integration testing.
- **5/26** — preview release.

---

## The wedge thesis

Premium Reserved is the right Phase-2 lead because it's a **strict superset** of the cp9 capability surface and a forcing function on the normal Create TiDB Cluster + Update TiDB Cluster + billing-event triad that the autoscaler wedge under-tested.

It needs:

- Real cluster-lifecycle services (`cluster-ng-service`, `dataplane-manager`) running the production Create TiDB Cluster + Update TiDB Cluster paths.
- Dataplane provisioning exercised as the side effect of those API calls: EKS, network, IAM, tidb-operator, and TiDB are reconciled by CP service code, not pre-created or special-cased by cp9.
- Real status state machine (`CREATING → AVAILABLE → MODIFYING → AVAILABLE`) emitting cluster-event payloads.
- Real OpenAPI / nextgen-global-service / billing-event boundary — Reserved is a new SKU at the OpenAPI layer routed through the existing Premium tier.
- Real cloud resources tagged + reapable (customer dataplane EKS, EBS, IAM, security groups).
- A new vCPU-driven topology calculator added to `cluster-service-ng` `gen_component.go` that produces the spec's mapping-table shapes.

If cp9 works for Premium Reserved, it works for the rest of Phase-2. Inverse: any cp9 gap that blocks Reserved would block them too.

**Concrete wedge target: normal create/update API paths, AWS only.** The validation payloads come from the PM spec, but cp9 does not model Reserved business flows. cp9 boots the CP environment; E2E tests call the normal OpenAPI Create TiDB Cluster path with `purchased_vcpu=56` and the normal OpenAPI Update TiDB Cluster path with `purchased_vcpu=104`. Both produce real TiDB clusters whose topologies match the mapping table, both emit billing events that match the billing spec's status table.

**What 07 does NOT prove.** Multi-cloud (Azure / Alicloud — Phase-2 follow); customer trust-boundary features (CMEK / private link / RDS migration / S3 import); TiFlash; worker autoscaling validation (06 surface); migration flows; the customer-side sizing-calculator UI.

**Phase-2 exit, Reserved-specific.** @cloud-cp-lead's team migrates the Reserved OpenAPI create/update feature delivery from `staging-seed-ng-0711` to cp9 with the new flow. After ≥ 2 consecutive cp9 sessions across ≥ 2 engineers run T1 + T2 green, the soak gate per [05 Migration §3](20260425-design-cp9-detail.md#migration-from-01) clears for Reserved — feeding the broader Phase-2 subtractive cutover.

---

## Host and tenancy decision

This section folds the critical decision from [08 host-tenancy refactor](20260430-design-cp9-host-tenancy-refactor.md) into the 07 wedge itself so 07 is reviewable on its own.

### Decision

07 uses a **tiered k3d host model**:

| Mode | Role | Tenancy | Status for 07 |
|---|---|---|---|
| `local-k3d` | Default daily-dev path on the developer Mac | One cp9 k3d cluster per developer session | Required |
| `remote-ec2-k3d` | Opt-in escape hatch for constrained laptops, large sessions, or remote development | One tagged EC2 instance running one k3d cluster per developer session | Follow-up, not M1 |

`local-k3d` remains the default because the wedge optimizes for quick code-deploy-verify loops, local debugging, and single-developer blast radius. `remote-ec2-k3d` preserves the same Kubernetes distribution and cp9 lifecycle while moving the host compute off the laptop. The host choice must not change the `source` / `live` / `stub` service graph, the OpenAPI create/update paths, the customer dataplane provisioning path, or the E2E assertions.

### Dataplane invariant

07 validates normal OpenAPI create/update paths provisioning real EKS + tidb-operator + TiDB resources through production CP service code. That dataplane remains a service-side side effect, not a cp9 bootstrap capability and not a separate persona requirement. T1/T2 prove the wedge only if this production provisioning path stays intact.

### Config and implementation contract

Unified `cp9.yaml` may select the host explicitly:

```yaml
name: cp9-cara-rs1

host:
  mode: local-k3d            # default

# Opt-in escape hatch after issue #106 substrate integration:
# host.mode: remote-ec2-k3d
```

Implementation shape after issue #106:

- Keep local k3d lifecycle explicit for the daily-dev path.
- Let Terraform substrate own IAM/metadb support infra for M1.
- Let Terraform substrate own remote EC2 placement and support infra later when `host.mode: remote-ec2-k3d`.
- Route the change-deploy-verify loop through the generated Tiltfile after cp9 bootstrap.
- Keep reaper coverage for Terraform-created and service-created AWS resources.
- Do not require network/tunnel or add other host modes under 07 M1.
- Do not change the customer dataplane provisioning path under 07.

Rollback is one local-config change: use `host.mode: local-k3d`. Destroy semantics still follow [05's two-stage guarantee](20260425-design-cp9-detail.md#cp9-destroy--two-stage-guarantee): local/session cleanup first, tag-scoped cloud reaper second.

---

## Premium Reserved `cp9.yaml` shape

> 2026-05-07 update: standalone `topologies/premium-reserved.yaml` and `cp9.local.yaml` are superseded by the issue #106 unified `cp9.yaml` model. Premium Reserved should use `source` for services under active code change, `live` for critical-path real app-delivery services such as Crossplane / provider / dataplane-manager, and `stub` for generated or recording stubs.
>
> 2026-05-09 update: issue #106 defines the Premium Reserved persona extension contract as `cp9.premium-reserved.example.yaml`, the `premium.tfvars` Terraform var file, and the `premium-reserved` Argo/kustomize env in `app-delivery`. The shared persona name is the business key across those artifacts. For the 07 narrow wedge, the persona owner manually derives those artifacts from staging Pulumi, existing Terraform, and app-delivery/ArgoCD evidence; future cp9 can add tooling to make that extension easier. The durable target is still `auto-deploy/terraform/aws/cp9-substrate/premium.tfvars`; the in-repo copy proves the M1 contract first.

```yaml
apiVersion: cp9.tidbcloud.com/v1alpha2
kind: Session
name: cp9-cara-rs1

host:
  mode: local-k3d                 # default; remote-ec2-k3d is opt-in

target:
  cloud: aws
  region: us-west-2
  aws:
    account_allowlist:
      - "123456789012"
    iam:
      profile: cp9
      role_arn: arn:aws:iam::123456789012:role/cp9-developer

terraform:
  repo: ../auto-deploy
  ref: main
  path: terraform/aws/cp9-substrate
  var_file: terraform/aws/cp9-substrate/premium.tfvars
  state: local

appDelivery:
  # cp9-owned Argo-style inputs. Registries and bases are shared; each
  # persona owns one dedicated env under envs/<persona>/.
  repo: ./personas/app-delivery
  ref: main
  env: premium-reserved

tilt:
  mode: up

services:
  source:
    tidb-mgmt-portal:
      repo: ../tidb-management-service/tidb-mgmt-portal
      ref: main # override with a feature branch when testing in-flight service changes
      build:
        main: cmd/tidb-mgmt-portal/main.go
      app:
        registry: registry/tidb-mgmt-portal.yaml
        env: premium-reserved
    tidb-mgmt-service:
      repo: ../tidb-management-service/tidb-mgmt-service
      ref: main # override with a feature branch when testing in-flight service changes
      build:
        main: cmd/tidb-mgmt-service/main.go
      app:
        registry: registry/tidb-mgmt-service.yaml
        env: premium-reserved
    cluster-ng-server:
      repo: ../cluster-service-ng
      ref: main # override with a feature branch when testing in-flight service changes
      build:
        main: cmd/cluster_ng_server/main.go
      app:
        registry: registry/cluster-ng-server.yaml
        env: premium-reserved
      # Current implementation keeps local debug env in the persona app-delivery
      # env patch instead of a loose repo-root override file.

  live:
    crossplane:
      app:
        name: nextgen-crossplane
        registry: registry/crossplane.yaml
        env: premium-reserved
    upbound-aws-provider:
      app:
        registry: registry/upbound-aws-provider.yaml
        env: premium-reserved
    dataplane-manager:
      app:
        registry: registry/dataplane-manager.yaml
        env: premium-reserved

  stub:
    account-svc: {}
    metrics-service: {}
    metering:
      kind: metering-recorder
    billing-svc:
      kind: billing-recorder
```

There is intentionally no separate dataplane requirement. cp9 stops at bringing up CP services, support infra, live app-delivery dependencies, metadb support, and stubs. The customer dataplane is created only when the production OpenAPI create/update request runs through `cluster-ng-server` and `dataplane-manager`.

`metering` and `billing-svc` both ship as permissive stubs by default. The actual billing pipeline is **metering → billing-svc**, not status-event → billing — `metering` ([tidbcloud/metering](https://github.com/tidbcloud/metering)) collects cluster-lifecycle metadata + cloud-usage data periodically and publishes to billing API; `billing-svc` consumes it and builds invoices via PM-designed pricing formulas. The `SendClusterEventToUNC` path is user notification, not billing data — do not anchor billing assertions on it.

The metering stub records every published payload (vCPU-by-component, duration, cluster status) to `.cp9/<name>/stubs/metering/payloads.jsonl`. T3 reads this file. **Flip criterion** — see Per-service audit § billing-svc + metering.

---

## Architecture deviation: CP layer collapse

In production, TiDB Cloud separates **Global CP** (one cluster) from **Regional CP** (one per product line per cloud per region). Premium Reserved spans both: `tidb-mgmt-portal` + `tidb-mgmt-service` + `nextgen-global-service` are global; `cluster-ng-service` + `dataplane-manager` are regional (Premium Regional CP, `staging-seed-ng-0711`).

In cp9, all `source`, `live`, and `stub` services run in **one k3d host cluster**. The default host is local k3d on the developer Mac; `remote-ec2-k3d` keeps the same k3d shape on a tagged EC2 instance. The regional/global distinction collapses either way. This applies to every persona; the autoscaler wedge surfaced it first, and 07 carries it forward unchanged.

**Implication.** Hidden assumptions of the form "I'm in regional VPC, so dataplane is reachable on a private link" or "my regional IAM role grants me X" become invalid.

**Audit class** — same three preconditions from [05 C3](20260425-design-cp9-detail.md#c3--cluster-lifecycle-service-preconditions) apply: externally-configurable, tag-complete, topology-agnostic.

Cross-cluster auth between cp9 k3d services and the customer dataplane EKS is **cluster-ng-service's responsibility** as part of its production provisioning logic. cp9 makes no modifications. If a service fails to reach the dataplane in cp9, that's a topology-agnostic audit gap on cluster-ng-service, not a cp9 gap.

---

## Day in the life — Cara ships a Reserved sizing-calculator fix

A concrete walkthrough. Cara is on @cloud-cp-lead's team, owns the topology calculator landing in `cluster-ng-service`. Each step calls out which capability is doing the work.

### Step 1: Configure (one-time per repo / session)

Cara has `tidb-management-service`, `cluster-service-ng`, `app-delivery`, and `auto-deploy` checked out as siblings. She copies `cp9.premium-reserved.example.yaml` to `cp9.yaml`, then sets session `name: cp9-cara-rs1`. The committed example defaults source refs to `main` so first-run dogfood works from clean checkouts; she changes them only when testing in-flight service branches.

```bash
aws sso login --profile cp9
cp9 setup       # validates cp9-developer role
cp9 doctor      # pre-flight: docker, kubectl, k3d, tilt, local AWS/IAM, M-series Mac
                # optional remote-ec2-k3d adds EC2, ssh, and network checks later
```

**Capability:** local IAM + AWS creds reachable + validated. Owner identity = Cara's SSO email. The unified config loader validates `cp9.yaml` against JSON Schema and resolves the Premium Reserved `source` / `live` / `stub` graph. A tunnel is not required for the 07 narrow wedge.

### Step 2: `cp9 up`

```bash
cp9 up          # first run: boots the k3d CP host + source/live/stub services
                # the real customer dataplane cold-start happens in Step 4
```

Lifecycle note: this paragraph used to reference 05's `crossplane_install`, staging `crd_install`, standalone `metadb_plan`, and `config_gen` states. Issue #106 replaces that flow with lifecycle v3: config load -> local IAM ready -> Terraform substrate -> selected local k3d host ready -> service plan -> context materialization -> generated Tiltfile -> `tilt up` / `tilt ci`. Crossplane and provider installation happen through app-delivery `live` service manifests, CRDs install with their owning service manifests, and Tilt owns build/apply/readiness after cp9 handoff.

Differences from the autoscaler wedge:

- `source` services replace the old source-built REAL category; `live` services cover critical-path app-delivery dependencies such as Crossplane, the AWS provider, and dataplane-manager.
- CRDs install with their owning app-delivery service manifests instead of a staging CRD dump.
- `metadb_seed` mirrors the **nextgen-tier metadb schema** (`staging_us_west_2_ng_0711`) — different from the autoscaler wedge.
- generated stub inputs include `metering` and `billing-svc` (06 also stubbed billing-svc but did not exercise the metering path); Tilt builds and deploys those stubs, and the stubs record metering payloads + computed invoice rows to session files.

`cp9 status` shows: cp9 bootstrap green, Terraform substrate green, selected host green, and Tilt resources for all `source`, `live`, and `stub` services healthy.

**Audit gate per service.** This is when **tag-complete** matters most — Step 4 (create) will create real cloud resources via the prod path. Every one of those must carry `cp9-owner` + `cp9-session` for Step 6 destroy to pass.

### Step 3: Iterate

Cara edits `cluster-service-ng/domain/server/gen_component.go` — adding the vCPU-driven topology calculator. Local Makefile validates:

```bash
cd ../cluster-service-ng
make dev        # build + lint + unit tests
```

Tests pass. She wants to E2E:

Tilt sees the file change and rebuilds/redeploys the `cluster-ng-service` resource. If Cara wants an explicit manual trigger, `cp9 deploy cluster-ng-service` is a compatibility wrapper around `tilt trigger cluster-ng-service`.

Pod restarts in cp9 with her change.

### Step 4: Run E2E (OpenAPI create at 56 vCPU)

```bash
go test ./test/e2e -run TestPremiumReserved_CreateTopology/Create56
```

What happens, end-to-end:

1. The E2E test calls the normal Create TiDB Cluster OpenAPI on `tidb-mgmt-portal`; portal forwards to `tidb-mgmt-service`.
2. `tidb-mgmt-service/app/openapi/nextgen/tidb/create.go` routes Premium → `nextgen-global-service` via gRPC, including the new `purchased_vcpu` field.
3. `nextgen-global-service/domain/service/cluster/create.go` calls `cluster-ng-service` `CreateTiDBCluster` with the Reserved payload.
4. `cluster-ng-service/domain/server/gen_component.go` runs Cara's vCPU-driven calculator — for `purchased_vcpu=56`, emits the 8c-band topology: `2× tidb8c16g + 3× tikv8c32g + 2× tiproxy2c4g + 3× pd4c8g`.
5. DAG (`PlanCreateTiDBCluster`) renders the TiDBCluster CR via `pkg/template/`, applies through `dataplane-manager` → real tidb-operator on the customer dataplane EKS.
6. Cluster phase walks `CREATING → AVAILABLE`. The `metering` stub polls the customer dataplane (via cluster-ng-service-wired auth), captures cluster status + topology + vCPU-by-component, and publishes a payload to its stub file. The `billing-svc` stub consumes the metering payload and writes a computed invoice row (`56 vCPU × billable duration × unit_price`) to its stub file. (The `SendClusterEventToUNC` user-notification call also fires but is not on the billing path.)

The test asserts: rendered TiDBCluster CR matches the PM spec mapping-table row for 56 vCPU; metering stub payloads include vCPU-by-component totaling 56 with status `Active`; billing-svc stub invoice row applies the per-vCPU rate to the Active duration only (Creating not billable); `cp9 status --json` surfaces the current session resource/cost reporting state while tag audits prove AWS resources carry `cp9-owner` / `cp9-session`.

If something fails: host-cluster logs come from Tilt (`tilt logs` directly or `cp9 logs` as a compatibility wrapper). Customer dataplane diagnostics remain selected-host / dataplane kubeconfig diagnostics once cluster-ng-service has created the real dataplane EKS through the production path.

### Step 5: Update to 104 vCPU

```bash
go test ./test/e2e -run TestPremiumReserved_UpdateTopology/Update56To104
```

End-to-end:

1. The E2E test reuses the cluster created through the normal create path, then calls the normal Update TiDB Cluster OpenAPI through `tidb-mgmt-portal` → `tidb-mgmt-service` → `nextgen-global-service` → `cluster-ng-service`.
2. `cluster-ng-service/domain/server/update_tidbcluster.go` recomputes via Cara's calculator. For `purchased_vcpu=104` (16c band entry), topology is `2× tidb16c32g + 3× tikv16c64g + 2× tiproxy2c4g + 1× tidb-worker-4c8g + 1× tikv-worker-4c16g + 3× pd4c8g`.
3. Cluster phase walks `AVAILABLE → MODIFYING → AVAILABLE`. The reshape is non-trivial: 8c→16c node-spec swap, +2 worker types appear. tidb-operator handles the rolling apply.
4. metering stub publishes a new payload (vCPU-by-component totaling 104 with status flipping `Modifying → Active`). billing-svc stub appends invoice rows: cluster-level billable through `Modifying`, node-level billable only when each node is `Active` (per the billing-spec status table).

The test asserts: final topology = 104-vCPU mapping-table row; metering payloads cover the `Modifying` window (cluster billable, nodes billable only when Active) and the post-update `Active@104vCPU` window; billing-svc invoice rows reflect the per-vCPU rate change at the Active boundary.

### Step 6: `cp9 destroy`

```bash
cp9 destroy     # two-stage; ~3-8 min for full customer-dataplane cleanup
```

Stage 1 deletes k3d, drops Cara's Zero session DB, removes `.cp9/cp9-cara-rs1/`. Stage 2 ([C4 reaper](20260425-design-cp9-detail.md#c4--force-purge-reaper-tag-scoped-multi-cloud)) scans every in-scope AWS region for `cp9-owner=cara@pingcap.com,cp9-session=cp9-cara-rs1`, deletes in dependency-safe order, exits 0 only on empty.

EBS volumes attached to terminated EKS instances follow the same retry pattern as 06.

---

## Per-service cp9-readiness audit

Authoritative v1alpha2 M1 audit artifact: [20260509-premium-reserved-v1alpha2-readiness-audit.md](20260509-premium-reserved-v1alpha2-readiness-audit.md).

2026-05-09 status:

- The cp9-owned Premium Reserved local-M1 harness is now accepted after issue `#119` / PR `#134`: local-k3d only, no mandatory tunnel, and no EKS creation during cp9 bootstrap.
- The real source/live service audit is still **not** closed. The current FAIL/BLOCKED state is tracked in the linked artifact above and the owning-repo remediation issues it references.
- The historical 2026-05-06 app-delivery-only pass below remains useful as source evidence, but it is no longer the authoritative final state.

This section is the C3 audit tracker for the Premium Reserved wedge. Audit per [05 C3 contract](20260425-design-cp9-detail.md#c3--cluster-lifecycle-service-preconditions): externally configurable, tag-complete, and topology-agnostic.

The #82 implementation train is closed. This audit remains open for acceptance and adoption: it records concrete app-delivery IaC evidence and the owning-repo gaps that must be resolved before any service can move from `FAIL` to `PASS`.

Status semantics:

- `PASS`: audit evidence is attached or linked, and no remaining service-side change is required for this gate.
- `FAIL`: audit evidence is missing, or a real compatibility gap is known.
- `BLOCKED`: intentionally not yet flipped to REAL because an earlier wedge gate must finish first.

Gate definitions:

| Gate | Meaning |
|---|---|
| Externally configurable | Every environment-specific dependency must be configurable without code changes: service endpoints, metadb/RDS DSN, AWS account/region, provider config refs, registry prefix, feature flags, IAM role, and credentials. |
| Tag-complete | Every direct or indirect cloud-resource create path must propagate `cp9-owner` and `cp9-session`. If a service creates no cloud resources in this path, that must be proven with code-search/runtime evidence. |
| Topology-agnostic | The service must work when Global CP and Regional CP are co-located in one cp9 k3d session while customer dataplane EKS is created only through the normal OpenAPI create/update flow. |

App-delivery audit pass:

- Date: 2026-05-06
- Repo snapshot: `tidbcloud/app-delivery@db355fc840`
- Scope: app-delivery IaC only. This pass does not prove service application code paths.

Key finding: app-delivery has enough surface to describe the current staging topology, but the checked-in staging manifests are still environment-specific. They pin staging DNS names, RDS hosts, AWS account IDs, IAM roles, regions, registry prefixes, provider configs, public domains, and static resource tags. The audit therefore stays at `FAIL` for all `source` and `live` services until owning repos provide either cp9-specific overrides or a documented template path that cp9 can drive per session.

Summary:

| Service | Owning repo | Deploy surface | 07 role | Externally configurable | Tag-complete | Topology-agnostic | Current blocker |
|---|---|---|---|---|---|---|---|
| `tidb-mgmt-portal` | `tidbcloud/tidb-management-service` | `tidb-mgmt-portal/` | global, source | `FAIL` | `FAIL` | `FAIL` | app-delivery pins staging account/IPIP dependencies and legacy dedicated-global-service compatibility endpoint; no app-code proof yet for "no cloud create path". |
| `tidb-mgmt-service` | `tidbcloud/tidb-management-service` | `tidb-mgmt-service/` + `nextgen-global-service/` | global, source | `FAIL` | `FAIL` | `FAIL` | app-delivery pins RDS, KMS, Secrets Manager, SQS, billing/core/dataflow/account dependencies, and LoadBalancer/IRSA AWS assumptions. |
| `cluster-ng-service` | `tidbcloud/cluster-service-ng` | `staging/dedicated-cluster-ng-service/cluster-ng-server/envs/staging-us-west-2-ng-0711/` | regional, source | `FAIL` | `FAIL` | `FAIL` | app-delivery pins region/account/RDS/registry/metering buckets/IAM roles; RBAC can create Crossplane AWS resources, but `cp9-owner` / `cp9-session` propagation is not proven. |
| `dataplane-manager` | `tidbcloud/cluster-service-ng` | `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/` | regional controller, live | `FAIL` | `FAIL` | `FAIL` | app-delivery enables resource tagging but only records static staging tags; provider config, account list, DNS, role ARN, region, and node-pool topology are environment-pinned. |
| `metering` | `tidbcloud/metering` | `app-delivery/dedicated-metering-server/` | STUB now -> REAL later | `BLOCKED` | `BLOCKED` | `BLOCKED` | Must stay STUB until T1 + T2 are green on two consecutive cp9 sessions across two engineers, then pass this audit before flipping REAL. |
| `billing-svc` | `TBD` | `staging/billing-svc/` | STUB now -> REAL later | `BLOCKED` | `BLOCKED` | `BLOCKED` | Must stay STUB until metering has already flipped REAL and stabilized, then pass this audit before flipping REAL. |

### 1. tidb-mgmt-portal *(global, source)*

**What it does.** OpenAPI + UI shell — entry for customer / console requests. Routes by SKU to mgmt-service.

**Today's deploy shape.** Deployment under `tidb-mgmt-portal/`; image from staging ECR; ConfigMap input in `tidb-mgmt-portal/envs/staging/config.yaml`; kustomize namespace `tidb-management-service`.

**cp9 transformation.** Build from source; remove ECR pin; rewrite account/ACL/IPIP/mgmt-service/compat endpoints to cp9-selected in-cluster services or stubs; expose via k3d ingress. **No Reserved-specific code change** — the new SKU surfaces through existing OpenAPI endpoints, but the proto field passes through.

| Gate | Status | Evidence / blocker |
|---|---|---|
| Externally configurable | `FAIL` | `tidb-mgmt-portal/envs/staging/config.yaml` pins `tidbMgmtService.address` to `kubernetes:///tidb-mgmt-service:10001`, `accountService.address` / `aclAddress` to `*.staging-ms:10000`, `ipip.address` to `ipip-svc.staging-ms:8080`, and `compatApi.address` to `kubernetes:///dedicated-global-service:10000`. |
| Tag-complete | `FAIL` | app-delivery does not show a portal cloud-resource create path, but this is not enough for `PASS`; application-code evidence is still required to prove the Premium Reserved path does not create or mutate cloud resources. |
| Topology-agnostic | `FAIL` | The manifest uses fixed staging namespaces and compatibility endpoints. The portal must prove it can use cp9-selected service endpoints in the collapsed Global CP + Regional CP k3d topology without manual patching. |

App-delivery IaC evidence:

- `tidb-mgmt-portal/envs/staging/config.yaml`: staging account service, ACL service, IPIP service, and dedicated-global-service compatibility endpoints are embedded in the ConfigMap input.
- `tidb-mgmt-portal/envs/staging/kustomization.yaml`: namespace is `tidb-management-service`; image is a staging ECR release tag.

Required remediation before `PASS`:

- Add or document cp9-driven overrides for account/ACL/IPIP/mgmt-service/compat endpoints.
- Capture application-code evidence that the portal does not create cloud resources in the Premium Reserved create/update path, or enumerate all create paths and required tags.
- Link a cp9 smoke through `tidb-mgmt-portal` showing no cp9-specific code path or manual portal patching.

### 2. tidb-mgmt-service *(global, source - primary code-change site)*

**What it does.** Hosts `nextgen-global-service` (Premium tier router), `dedicated-global-service`, `serverless-global-service`. Receives mgmt-portal requests; routes by SKU to the appropriate global service; the global service then calls regional cluster-ng / cluster-service / dataplane-manager.

**Today's deploy shape.** `tidb-mgmt-service/` and `nextgen-global-service/`; staging manifests pin AWS RDS, KMS, Secrets Manager, SQS, Auth0, billing/core/dataflow/account dependencies, and IRSA role configuration.

**cp9 transformation.** Build from source. Substitute RDS → cp9 metadb (per-session, schema mirrored from staging). Substitute regional gRPC URLs → in-cluster `cluster-ng-service.cp9.svc.cluster.local`.

Code change: `nextgen-global-service/domain/service/cluster/create.go` extends to accept `purchased_vcpu` + workload pattern; `tidb-mgmt-service/app/openapi/nextgen/tidb/create.go` propagates the proto field; the nextgen-global update handler handles the Reserved branch on update.

| Gate | Status | Evidence / blocker |
|---|---|---|
| Externally configurable | `FAIL` | `tidb-mgmt-service/envs/staging/config.yaml` pins RDS host/database, KMS CMK ARN, account/dataflow/core/billing staging endpoints, JSM SQS URL/region, and claim URL host. `nextgen-global-service/envs/staging/config.yaml` separately pins MySQL, Secrets Manager ARN, Auth0 domains/client IDs, KMS/PCA ARNs, certificate bucket, and provider identity mapping. |
| Tag-complete | `FAIL` | app-delivery manifests include AWS-facing configuration and IRSA roles but do not prove whether management-service application code creates cloud resources or propagates `cp9-owner` / `cp9-session`. Application-code audit is still required. |
| Topology-agnostic | `FAIL` | Both global services assume staging namespaces and AWS-backed dependencies. `tidb-mgmt-service/base/service.yaml` also creates an AWS LoadBalancer service by default, which is not topology-neutral for a local cp9 k3d session. |

App-delivery IaC evidence:

- `tidb-mgmt-service/envs/staging/config.yaml`: RDS host, KMS ARN, SQS URL, `*.staging-ms` dependencies, and billing endpoint are static staging values.
- `nextgen-global-service/envs/staging/config.yaml`: RDS host, Secrets Manager ARN, Auth0 domains, KMS/PCA ARNs, certificate bucket, and supplement provider identities are static staging values.
- `tidb-mgmt-service/envs/staging/serviceaccount.yaml`: IRSA role is `arn:aws:iam::385595570414:role/tidb-management-service.common-setup.staging`.
- `tidb-mgmt-service/base/service.yaml`: service type is `LoadBalancer` with AWS NLB annotations.

Required remediation before `PASS`:

- Add or document cp9-driven overrides for RDS/metadb, regional service endpoints, account/core/dataflow/billing dependencies, Auth0/proxy settings, event/SQS targets, and AWS secret/KMS/PCA values.
- Prove application code has no cloud-resource create path for Premium Reserved create/update, or enumerate every create path and tag injection point.
- Link T1/T2 create+update evidence through the normal OpenAPI path, not a cp9-only shortcut.

### 3. metering + billing-svc *(global, STUB now → REAL after flip criterion)*

**What they do.** Two-stage pipeline.

- **`metering`** ([tidbcloud/metering](https://github.com/tidbcloud/metering)) periodically collects cluster-lifecycle metadata (status, topology, vCPU-by-component) + cloud-usage data and publishes to billing API. This is the load-bearing data feed for invoices.
- **`billing-svc`** consumes metering output, applies the [billing spec](https://pingcap.feishu.cn/wiki/IpSUwMBrSiPdn9kzmTkcHZVynee) PM-designed pricing formulas (Total vCPU × duration × unit_price; per-status billable/not-billable), writes invoice rows to RDS, emits to SQS for downstream finance / Salesforce.

The `SendClusterEventToUNC` path on `tidb-mgmt-service` is **user notification**, not billing data — it lives under the billing-svc API surface for historical reasons. Wedge 07 does not anchor billing assertions on it.

**Today's deploy shape.**

- `metering` — repo [tidbcloud/metering](https://github.com/tidbcloud/metering); deployment manifest under `app-delivery/dedicated-metering-server/` (`instance/env/{dev,staging,prod}-us-west-2/` + `network/`). Regional service.
- `billing-svc` — 3 Deployments (billing2-api, billing2-worker, dispatcher) under `staging/billing-svc/`; RDS `nightly-cbp-dev-staging…`; IAM role `arn:aws:iam::470438934515:role/dbaas-dev-role`; SQS `billing-report-staging`; Salesforce sandbox creds in `staging/billing-svc/envs/staging/config.properties`.

**cp9 transformation today (STUB).** Both services stubbed.

- `metering` stub records each periodic-collect payload (vCPU-by-component, duration, status) to `.cp9/<name>/stubs/metering/payloads.jsonl`.
- `billing-svc` stub accepts metering input and writes computed invoice rows (per-vCPU rate × billable duration) to `.cp9/<name>/stubs/billing-svc/invoices.jsonl`.

T3 reads both files.

**Flip criterion (STUB -> source/live).** Sequenced; both gates required:

1. **T1 + T2 green on two consecutive cp9 sessions across two engineers** (cluster-lifecycle stable enough to introduce metering).
2. **Flip metering out of STUB first.** Build `metering` from source or run it as a `live` app-delivery service, point it at the per-session cp9 cluster + dataplane EKS. Verify metering payloads in cp9 match what staging metering emits for an equivalent cluster shape.
3. **Flip billing-svc out of STUB second.** Build billing-svc from source or run it as a `live` app-delivery service, swap RDS -> cp9 MySQL with billing schema migrated (Open Q2), mock SQS (e.g., LocalStack) and Salesforce sandbox (no-op HTTP). T3 re-asserts invoice rows match the spec's pricing formula end-to-end.

| Gate | Status | Evidence / blocker |
|---|---|---|
| Externally configurable | `BLOCKED` | `metering` must prove its polling target, credential source, and billing publish destination are externally configurable before REAL. `billing-svc` must prove RDS/MySQL, SQS, Salesforce, and IAM/credential inputs are externally configurable before REAL. |
| Tag-complete | `BLOCKED` | Current STUBs write local JSONL only. REAL flip needs either proof that neither runtime creates cloud resources in the Reserved path, or explicit tag propagation for any created resources. |
| Topology-agnostic | `BLOCKED` | REAL flip must prove metering can poll the per-session dataplane EKS and billing-svc can consume metering payloads from the collapsed cp9 topology without production-only DNS/VPC/IAM assumptions. |

Required remediation before `PASS`:

- File owning-repo issues / PRs if metering dataplane poll target, cloud-usage target, billing endpoint, billing DB, SQS, Salesforce, or IAM dependency is hardcoded.
- Link acceptance evidence: metering REAL payloads match staging payload shape for an equivalent cluster, then billing REAL invoice rows match the PM pricing formula with downstream side effects mocked or safely redirected for cp9.

### 4. cluster-ng-service *(regional, source - primary code-change site)*

**What it does.** Owns the regional cluster lifecycle. DAG-based workflow engine (`PlanCreateTiDBCluster`, `PlanUpdateTiDBCluster`); component generator builds tidb-operator-flavored CRs from request specs; metering writer emits cluster events.

**Today's deploy shape.** Staging AWS env under `staging/dedicated-cluster-ng-service/cluster-ng-server/envs/staging-us-west-2-ng-0711/`. Deploy contains `cluster-ng-server` (main), `executor` (DAG runner), and supporting services. Per-env config pins cloud provider, region, RDS host, dataplane account ID, ECR registry prefix, DPM/config-center DNS, metering S3 buckets, and IAM roles.

**cp9 transformation.** Pin to the `staging-us-west-2-ng-0711` env. Substitute regional RDS → cp9 metadb. Substitute dataplane account, region, registry, DPM/config-center DNS, S3 metering, and IAM role through cp9-driven config.

Code change (the main 07 surface):
- `api/cluster-service-ng/v1beta/services.proto` — add the Reserved sizing request fields to `CreateClusterRequest` and `UpdateClusterRequest`; Open Q1 decides whether this is only `purchased_vcpu` or a richer input struct.
- `api/cluster-service-ng/v1beta/resources.proto` — add `BillingMode {ON_DEMAND, RESERVED}` enum on `Cluster`.
- `domain/server/gen_component.go` — new path `genReservedTopology(purchased_vcpu, pattern, qps, data_size_gb) → []Component` matching the PM spec sizing logic (TiKV by data + by QPS; TiDB by QPS with -30% large-cluster credit; TiProxy by total TiDB vCPU; PD spec by TiKV node count; workers in 16c band only).
- `domain/server/create_tidbcluster.go`, `update_tidbcluster.go` — branch on `BillingMode == RESERVED`; call new generator.
- `domain/workflow/dag/node.go` — keep the normal `PlanCreateTiDBCluster` / `PlanUpdateTiDBCluster` DAGs; add no Reserved-specific DAG.
- `pkg/storage/mysql/schema/schema.go` + `pkg/storage/mysql/migrate/db/migrate_2026042800x.go` — `BillingMode` + `PurchasedVcpu` columns.

| Gate | Status | Evidence / blocker |
|---|---|---|
| Externally configurable | `FAIL` | Staging config pins `cloudProvider: aws`, `region: us-west-2`, ECR registry, default dataplane account `790636686964`, RDS host/database, `dataPlaneManagerAddr`, `configCenterAddr`, metering S3 buckets, and metering writer role ARN. The variant template has placeholders, but the committed staging env is not a cp9 session-driven config surface. |
| Tag-complete | `FAIL` | `cluster-ng-server` has broad RBAC over Crossplane AWS APIs (`aws.crossplane.io`, `eks.aws.crossplane.io`, `ec2.aws.crossplane.io`, `iam.aws.crossplane.io`, etc.). No app-delivery evidence proves `cp9-owner` / `cp9-session` are added to all direct and Crossplane-created resources. |
| Topology-agnostic | `FAIL` | The service assumes fixed regional service DNS (`executor.cluster-ng-service`, `dataplane-manager.dataplane-manager`, `config-center.devops`) plus region/account-specific AWS resources. cp9 must prove this can run inside the collapsed k3d control plane while still driving the per-session dataplane EKS. |

App-delivery IaC evidence:

- `staging/dedicated-cluster-ng-service/cluster-ng-server/envs/staging-us-west-2-ng-0711/config.yaml`: pins account, region, registry, RDS, DPM/config-center DNS, S3 buckets, and IAM role.
- `staging/dedicated-cluster-ng-service/cluster-ng-server/envs/staging-us-west-2-ng-0711/serviceaccount.yaml`: IRSA role is `arn:aws:iam::385595570414:role/staging-us-west-2-ng-0711-dataplane-manager`.
- `staging/dedicated-cluster-ng-service/cluster-ng-server/variants/staging-aws/rbac.yaml`: grants broad Crossplane AWS API permissions.

Required remediation before `PASS`:

- Add or document cp9-driven overrides for target account, region, RDS/metadb, registry, DPM address, config-center address, S3 metering, and IAM role.
- Produce a tag propagation map from Premium Reserved request entry to all created AWS resource classes: EKS, node groups, EC2/EBS/ENI/SG, IAM role/policy, ELB, S3/SQS/SNS/DynamoDB/Lambda if applicable.
- Link T1 create and T2 update evidence followed by `cp9 status` and `cp9 destroy` proving all tagged resources are discoverable by `cp9-owner` / `cp9-session`.

### 5. dataplane-manager / nextgen-regional-control-plane support graph *(regional, live)*

**What it does.** The Premium regional CP host runs many sub-apps. The wedge needs the smallest app-delivery support graph, with `dataplane-manager` as the live regional controller that reconciles customer dataplane infrastructure through Crossplane.

**Today's deploy shape.** 26 sub-apps under `staging/nextgen-regional-control-plane/`; each has its own ApplicationSet across env variants. The `dataplane-manager` AWS staging env is `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/`.

**cp9 transformation.** Deploy the minimal-9 set:

| Sub-app | Why required |
|---|---|
| `meta-namespace` | Bootstrap namespaces. |
| `dataplane-manager-crds` + `dataplane-manager` | Cluster infrastructure CRs (Dataplane, AppBox, VPC, LoadBalancer …) + the controller that reconciles them. |
| `gateway-api-crds` + `gateway-crds` | Kubernetes Gateway API + Kong CRDs. |
| `external-secrets-operator` + `cluster-secretstore` | Sync cloud creds into k8s. |
| `crossplane` + `upbound-aws-provider` | C2 capability. |
| `cert-manager` + `issuers` | TLS for ingress + service. |
| `kong-ingress-controller` | API gateway. |
| `aws-load-balancer-controller` *(AWS only)* | NLB / ALB provisioning. |

Skipped (non-blocking): `ticdc-service`, `traffic-replay`, `victoria-metrics-operator`, `alidns-webhook` (non-AWS), `o11y-agent*`, `kube-prometheus`, `nextgen-metering-server`.

| Gate | Status | Evidence / blocker |
|---|---|---|
| Externally configurable | `FAIL` | `root-config.yaml` pins provider `eks`, logical/physical region, DNS account/role/zone/domain, PCA ARN, service role name, public access CIDRs, authorized subjects, data plane role name, and AWS account list. `ProviderConfig` uses IRSA and the service account pins the staging role ARN. |
| Tag-complete | `FAIL` | `resourceTaggerConfig.enabled: true` is present, and `resourceLabels` / kustomize labels include static `tags.tidbcloud.com/environment`, `servicetype`, and `usedby`, but there is no `cp9-owner` or `cp9-session` label/tag in app-delivery. `additionalCloudTags` only contains `ManagedBy: PingCAP`. |
| Topology-agnostic | `FAIL` | The controller config models a real regional seed/shoot topology with fixed DNS, public access CIDRs, provider config, account role, and node-pool catalog. cp9 must prove the minimal-9 subset works when the controller runs in the collapsed k3d control plane. |

App-delivery IaC evidence:

- `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/config/root-config.yaml`: region/account/DNS/role/PCA/service-role/node-pool/public-access inputs are staging-specific; `resourceTaggerConfig` is enabled but static tags lack cp9 owner/session values.
- `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/account/account-list.yaml`: dataplane account is `790636686964` with a fixed role name.
- `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/account/providerconfig.yaml`: provider config is `provider-aws` with IRSA credentials.
- `staging/nextgen-regional-control-plane/dataplane-manager/envs/aws-staging-us-west-2-ng-0711/kustomization.yaml`: kustomize labels are static staging/dedicated/seed values.

Required remediation before `PASS`:

- Add or document cp9-driven overrides for ProviderConfig, account list, DNS, PCA, service role, public access CIDRs, data plane role, node-pool catalog, and image registry.
- Extend or prove tag propagation so every managed resource gets `cp9-owner` and `cp9-session`, not only static staging tags.
- Link T1/T2 dataplane reconciliation evidence plus `cp9 destroy` Stage 2 finding zero surviving tagged resources after cleanup.

### Remediation / owner tracker

This table tracks owning-repo work required before Premium Reserved is declared accepted for broader use. It is intentionally separate from #82 implementation completion.

| Service | Owning repo | Owner | Required filing before `PASS` | Link |
|---|---|---|---|---|
| `tidb-mgmt-portal` | `tidbcloud/tidb-management-service` | `@cloud-cp-lead` | File issues / PRs for cp9 endpoint overrides and any portal app-code cloud-create/tag evidence gap. | `TBD` |
| `tidb-mgmt-service` | `tidbcloud/tidb-management-service` | `@cloud-cp-lead` | File issues / PRs for RDS/secret/KMS/SQS/service endpoint externalization and any direct untagged resource-create path. | `TBD` |
| `cluster-ng-service` | `tidbcloud/cluster-service-ng` | `@cloud-cp-lead` | File issues / PRs for cp9-driven regional/account/metadb/registry/IAM config and full resource tag propagation. | `TBD` |
| `dataplane-manager` | `tidbcloud/cluster-service-ng` | `@cloud-cp-lead` | File issues / PRs for cp9-driven provider/account/DNS/PCA/node-pool config and cp9 owner/session tag propagation. | `TBD` |
| `metering` | `tidbcloud/metering` | `@billing-lead` | File issues / PRs before the STUB -> REAL flip if any cp9 target is hardcoded. | `TBD` |
| `billing-svc` | `TBD` | `@billing-lead` | File issues / PRs before the STUB -> REAL flip if any cp9 target is hardcoded. | `TBD` |

The audit is complete only when each `source` and `live` service above has all three gates at `PASS`, every required service-side remediation is filed in the owning repo and linked here, and any `metering` / `billing-svc` flip out of STUB uses this section as its audit gate.

---

## E2E harness — full shape

Test files live in the cluster-service-ng repo (closest to the topology-calculator change) per [05 E2E harness](20260425-design-cp9-detail.md#e2e-test-harness):

- `cluster-service-ng/test/e2e/reserved_create_test.go` — `TestPremiumReserved_CreateTopology`
- `cluster-service-ng/test/e2e/reserved_update_test.go` — `TestPremiumReserved_UpdateTopology`

Required `cp9test` helpers (most reused from 06; one new):

| Helper | Purpose | Reuse / new |
|---|---|---|
| `NewClient(t)` | Connect to active cp9 session | reuse |
| `CreateTiDBCluster(t, req)` | Drives normal Create TiDB Cluster OpenAPI; caller passes the Reserved payload | reuse / thin wrapper |
| `UpdateTiDBCluster(t, clusterID, req)` | Drives normal Update TiDB Cluster OpenAPI with changed `purchased_vcpu` | reuse / thin wrapper |
| `GetTopology(t, clusterID)` | Reads rendered TiDBCluster CR; returns components map | new |
| `MeteringPayloads(t, clusterID)` | Reads `.cp9/<name>/stubs/metering/payloads.jsonl`, returns metering payloads matching cluster ID | new |
| `BillingInvoices(t, clusterID)` | Reads `.cp9/<name>/stubs/billing-svc/invoices.jsonl`, returns computed invoice rows matching cluster ID | new |
| `PollUntil(t, c, path, predicate, timeout)` | Polls with backoff | reuse |
| `JSONPathEquals(field, want)` | Equality predicate constructor | new (06 had GreaterThan only) |
| `RequireNoSLOBreach(t, c, clusterID)` | Asserts no synthetic SLO breach | reuse |
| `LastTraceID()` | For log correlation | reuse |

```go
// reserved_create_test.go — illustrative shape
func TestPremiumReserved_CreateTopology(t *testing.T) {
    cases := []struct {
        vcpu int
        want map[string]int   // component name → replica count
    }{
        {56,  map[string]int{"tidb": 2, "tikv": 3, "tiproxy": 2, "pd": 3}},
        {96,  map[string]int{"tidb": 4, "tikv": 6, "tiproxy": 2, "pd": 3}},
        {104, map[string]int{"tidb": 2, "tikv": 3, "tiproxy": 2, "tidb-worker": 1, "tikv-worker": 1, "pd": 3}},
        {208, map[string]int{"tidb": 4, "tikv": 6, "tiproxy": 2, "tidb-worker": 1, "tikv-worker": 1, "copr-worker": 1, "pd": 3}},
        {288, map[string]int{"tidb": 6, "tikv": 9, "tiproxy": 2, "tidb-worker": 1, "tikv-worker": 1, "copr-worker": 1, "pd": 3}},
    }
    c := cp9test.NewClient(t)
    for _, tc := range cases {
        cluster := c.CreateTiDBCluster(t, cp9test.TiDBClusterRequest{
            SKU: "premium-reserved",
            PurchasedVCPU: tc.vcpu,
            Pattern: "read",
            QPS: 10000,
            DataGB: 500,
        })
        got := c.GetTopology(t, cluster.ID)
        for comp, want := range tc.want {
            cp9test.RequireEqual(t, got[comp], want, fmt.Sprintf("vcpu=%d component=%s", tc.vcpu, comp))
        }
        // metering should publish at least one payload with vCPU-by-component summing to tc.vcpu
        cp9test.RequireMeteringVCPUTotal(t, c.MeteringPayloads(t, cluster.ID), tc.vcpu)
    }
}
```

**Cleanup:** tests don't call delete APIs. The session's `cp9 destroy` handles teardown and runs the force-purge stage internally — same [05 invariant](20260425-design-cp9-detail.md#cp9-destroy--two-stage-guarantee) as 06.

**Cold-start budget:** first OpenAPI create in a cp9 session may wait 15-30 min while production lifecycle code provisions the underlying EKS, network, IAM, tidb-operator, and TiDB resources. Later cases run in the same session and may reuse whatever that production path leaves warm. CI runs all Reserved E2E tests in one session before destroy.

### Wedge test additions

Four integration tests beyond the wedge E2E itself, each closing a silent-regression channel. T1 + T2 are the wedge thesis; T3 + T4 are 07-specific tests over and above 06's 4-test set.

| Test | Repo | What it asserts |
|---|---|---|
| **T1 — Reserved create produces spec-compliant topology** | cluster-service-ng | Table-driven over the 5 mapping-table rows (56, 96, 104, 208, 288 vCPU). `JSONPathEquals` on the rendered TiDBCluster CR for replica count + node spec per component. Catches regression in the topology calculator. |
| **T2 — Reserved update reshapes topology + crosses MODIFYING** | cluster-service-ng | Create at 56, update `purchased_vcpu` to 104, then to 208 through the normal Update TiDB Cluster path. Asserts: phase transitions (`AVAILABLE → MODIFYING → AVAILABLE`) via cluster-agent reads; final topology matches the mapping-table row; metering payloads cover the `Modifying` window with the new vCPU-by-component shape. Catches regression in the update path or state-machine. |
| **T3 — Metering + billing pipeline matches spec** | cluster-service-ng (or cp9) | Create + update + delete; assert (a) metering stub payloads contain the right vCPU-by-component, status, and timestamps for each phase; (b) billing-svc stub invoice rows match the billing spec status table — Active billable; Creating non-billable until first Active; Modifying billed at cluster level only; Deleting billed through tombstone — with `Total vCPU × billable duration × unit_price`. Drives the STUB→REAL flip criterion. |
| **T4 — `cp9test` helper smoke tests** | cp9 | Per-helper unit/integration tests against fake servers, including the generic OpenAPI helpers (`CreateTiDBCluster`, `UpdateTiDBCluster`, `GetTopology`, `MeteringPayloads`, `BillingInvoices`, `JSONPathEquals`). Foundation for every Phase-2 team — buggy helpers mask real regressions. Continues 06's T4. |

T1 + T2 + T3 ship with the wedge E2E; T4 ships in cp9's regular CI alongside [05's invariant tests](20260425-design-cp9-detail.md#test-strategy).

---

## Phase-2 exit criteria — Reserved-specific

Inherits [05's success criteria](20260425-design-cp9-detail.md#success-criteria). Reserved-specific concretization tied to the meeting-note milestones:

| Date | Gate |
|---|---|
| **5/19** (integration testing) | T1 + T2 green on at least one engineer's cp9 session; `cp9.premium-reserved.example.yaml`, `premium.tfvars`, and `premium-reserved` app-delivery env committed or in linked PRs; metering + billing-svc still STUB; per-service audit started for cluster-ng-service. |
| **5/26** (preview release) | T1 + T2 + T3 green on **two** engineers' sessions across **two** consecutive runs each; metering flipped out of STUB; billing-svc flip pending or done (per criterion); per-service audits passed for the `source` and `live` graph; `cp9 destroy` exit-0 in CI >= 95% over a soak window. |

After 5/26 + a 4-week soak window of (1)-(4) clean, the soak gate per [05 Migration §3](20260425-design-cp9-detail.md#migration-from-01) clears for Reserved. Subtractive cutover (decommissioning the Reserved team's slice of `staging-seed-ng-0711`) can begin.

---

## Risks specific to the Reserved wedge

Inherits [05's risk table](20260425-design-cp9-detail.md#risks). Reserved-specific:

| Risk | Mitigation |
|---|---|
| Topology calculator placement (PM spec talks about a customer-side calculator UI; backend may also need to run the sizing logic) | Confirm the request shape with @cloud-cp-lead before T1 lands. If the UI sends only `purchased_vcpu`, the backend rebuilds the calc - Open Q1. |
| DAG insertion surface inside the normal create/update DAGs | Prototype the smallest insertion point in cluster-ng-service; keep the external path as Create TiDB Cluster / Update TiDB Cluster. Doc reflects the chosen approach. |
| Dataplane cold start after the first OpenAPI create (EKS + tidb-operator) dominates iteration time | Run the cases in one cp9 session so the production lifecycle path can reuse any warmed primitives it already supports; cp9 lifecycle is re-entrant; warm pool follow-up if friction forces it. |
| metering / billing-svc REAL flip slips past 5/26 | Flip is criterion-based, not date-based, and sequenced (metering REAL first, billing-svc REAL after). If cluster-lifecycle stays unstable, both stay STUB through preview; T3 still runs against the stubs and asserts the metering→billing payload contracts. |
| Worker-node update path (>100 vCPU) overlaps with autoscaler wedge | 07 deliberately asserts only the **static** mapping at create/update time; **does not validate worker autoscaling**. Worker autoscaling stays as a separate audit (06 surface). |
| Status state-machine emits unexpected events under failure (e.g., partial update aborts mid-DAG) | T3 asserts on the **happy path**. Failure-mode billing semantics are out of 07 scope; flag for follow-up after preview. |
| RDS substitution for cluster-ng-service introduces schema drift between cp9 and staging | `metadb_seed` must mirror the production `staging_us_west_2_ng_0711` schema; a schema-drift check in `cp9 doctor` catches divergence. |
| nextgen-global-service / mgmt-service hardcoded RDS or regional gRPC URLs | Externally-configurable audit gate; 5/19 gate requires this audit started for cluster-ng-service, 5/26 for the `source` and `live` graph. |

---

## Open questions

1. **`purchased_vcpu` request shape.** Single int field, or struct (`{vcpu, pattern, qps_hint, data_size}`)? PM spec talks about pattern + QPS + data size as calculator inputs; current cluster-ng-service request is RU-driven. Confirm with @cloud-cp-lead before the proto change lands.
2. **metering polling target + billing-svc REAL backing DB.** Two flip-related opens:
   - When `metering` goes REAL, what does it poll? In prod it polls the dataplane EKS + cloud usage APIs. In cp9, the customer dataplane EKS exists as the side effect of the normal create/update path; cloud-usage APIs are the dev account, scoped via `cp9-owner`/`cp9-session` tags. Confirm `metering` accepts both targets via config.
   - When `billing-svc` goes REAL, does cp9 use the dev billing RDS through an optional network/tunnel path (simpler, but per-developer concurrency risk on the shared DB), or does each session boot a local MySQL with billing schema migrated (cp9 ideal; requires schema-migration tooling in `cp9 sync`)? Recommend local; quantify the migration tooling work in a follow-up.
3. **Persona overlap with autoscaler wedge.** Reserved at >100 vCPU implicitly covers some of what 06's autoscaler wedge tested (worker mapping at create/update time). 06 covers the autoscaler control loop tuning HPA (different surface). They can coexist; 07 deliberately scopes out autoscaling. Revisit after 5/26 preview if either persona becomes redundant.
4. **DAG insertion.** The Reserved calculator should plug into the normal `PlanCreateTiDBCluster` / `PlanUpdateTiDBCluster` flow. Decision drives the cluster-ng audit scope. Sequence: prototype with cluster-ng owners during the 5/19 -> 5/26 window; pick before preview.
5. **OpenAPI fixture ownership.** Keep Reserved payload construction in the cluster-service-ng E2E tests, while cp9 exposes only generic session/test helpers. No Reserved-specific cp9 CLI surface.
6. **Billing-spec coverage in T3.** The status table covers Active / Modifying / Creating / Deleting / Importing / Maintaining / Restoring. T3 asserts on Active + Modifying + Deleting (the create + update + delete path). Importing / Maintaining / Restoring are out of 07 scope; document in a status-coverage matrix follow-up.

---

## Cross-reference index

Verifies 07 covers every C1–C5 contract from 05:

| Capability | Where 07 exercises it |
|---|---|
| C1 — Local IAM + AWS creds | Step 1 (`cp9 setup`, `cp9 doctor`) + issue #106 `iam_ready`. Tunnel/network is optional for 07 M1. |
| C2 — Crossplane + per-cloud creds | Issue #106 `live` service graph: Crossplane and provider manifests come from cp9-owned Argo envs, Tilt applies them, and Terraform outputs supply support infra and credentials. |
| C3 — Cluster-lifecycle service preconditions | Per-service audit section (5 services). Audit gate calls in Steps 1, 2, 4, 5. |
| C4 — Tag-scoped reaper | Step 6 (Stage 2 force-purge). Risks table (cluster-ng-service tag-complete). Exit criteria 5/26. |
| C5 — Persona / topology template loader | Replaced by unified `cp9.yaml` config load and service-plan resolution. |

Tag invariant exercised throughout (Step 2 deploy → Step 4 OpenAPI provisioning → Step 6 reaper). State-machine invariants covered by the [05 invariant tests](20260425-design-cp9-detail.md#test-strategy). PM and billing spec contents are not inlined here; the working Feishu extract lives in personal-doc and is not committed.

---

## Out of scope and follow-ups

**Out of scope.** TiFlash; CMEK; private-link / VPC-peering; tunnel/network setup for M1; customer-analog setup; non-AWS clouds (Azure / Alicloud — Phase-2 follow); the customer-side sizing-calculator UI; failure-mode billing semantics (partial update aborts, restore, importing, maintaining); autoscaling validation (06 surface); alternate host modes beyond `local-k3d` / follow-up `remote-ec2-k3d`; changes to the customer dataplane provisioning path.

**Follow-ups, ordered:**

1. **Per-service audit completion** for the `source` and `live` services. Owners assigned by @cloud-cp-lead + per-service team lead. AWS first.
2. **`cp9test` helpers shipped** in cp9 repo (`CreateTiDBCluster`, `UpdateTiDBCluster`, `GetTopology`, `MeteringPayloads`, `BillingInvoices`, `JSONPathEquals`). Implementing agent.
3. **DAG insertion decision** (Open Q4) — prototype, decide, document during 5/19 -> 5/26.
4. **metering STUB -> REAL flip** (per criterion). Targeted before 5/26.
5. **billing-svc STUB -> REAL flip** (per criterion, sequenced after metering REAL). Targeted on or after 5/26.
6. **Schema-migration tooling for cp9 metadb** - needed for billing-svc flip out of STUB if local-MySQL path picked (Open Q2). Implementing agent.
7. **Soak run + measurement** of cold-start budget — capture in success criterion #3 of [05](20260425-design-cp9-detail.md#success-criteria).
8. **Host/network follow-up** — make `local-k3d` explicit, improve k3d doctor/resource diagnostics, then implement `remote-ec2-k3d` and any required tunnel/network capability as opt-in escape hatches.
9. **Stakeholder + landing-page drift updates** — once 07 stabilizes, update their wedge references from Premium Autoscaler to Premium Reserved.
10. **(Done in this PR)** Port this wedge doc into [tidbcloud/controlplane9 docs/](https://github.com/tidbcloud/controlplane9/tree/main/docs).
11. **Cost narrative** from `cp9 status` / AWS cost evidence during soak. Feeds Phase-2 decommission.

---

## Self-consistency checks

Verify before approval:

- Unified `cp9.yaml` `source` entries map to audit sections: `tidb-mgmt-portal` -> section 1, `tidb-mgmt-service` -> section 2, `cluster-ng-server` -> section 4.
- Unified `cp9.yaml` `live` entries map to audit/support graph sections: Crossplane/provider and `dataplane-manager` -> audit section 5.
- Unified `cp9.yaml` `stub` entries include `billing-svc` -> audit section 3. `account-svc` and `metrics-service` are off-feature-path and intentionally not audited.
- Persona key consistency: `cp9.premium-reserved.example.yaml`, `terraform/aws/cp9-substrate/premium.tfvars`, and the app-delivery `premium-reserved` env are the three durable extension points.
- Per-service audit follows the user's 5 deployment-manifest paths (`tidb-mgmt-portal/`, `tidb-mgmt-service/`, `staging/billing-svc/`, `staging/dedicated-cluster-ng-service/`, `staging/nextgen-regional-control-plane/`).
- Day-in-the-life cluster sizes (56, 104) appear in T1 + T2 mapping rows.
- Billing-spec status table -> T3 assertions (Active / Modifying / Deleting paths covered; Creating / Importing / Maintaining flagged out of scope in Open Q6).
- 5/19 / 5/26 milestones from the meeting note → exit-criteria gates.
- Cross-reference index covers C1–C5; no capability missing.
- Host decision is self-contained: `local-k3d` default, `remote-ec2-k3d` follow-up, network/tunnel optional for M1, and customer dataplane provisioning unchanged.
- All open questions trace to either spec uncertainty, owner mapping, or 07-specific implementation choices; nothing dangling.
- Out-of-scope explicitly enumerates the deferred surfaces (TiFlash, CMEK, private-link, autoscaling, non-AWS) so readers don't expect them.
- 07 does not duplicate 05 or the specs; it cites them for capability contracts and PM/billing logic, and adds wedge-specific concretization.