# Design: starter-full Regional Graph Classification

Slock task: `#cp9:df920499`

GitHub issue: `#366`

Parent design:
[`20260608-design-cp9-starter-full-persona.md`](./20260608-design-cp9-starter-full-persona.md)

Status: Proposed for HITL review.

## Summary

`starter-full` imports the Starter/Essential v1 two-layer topology. This is not
the Premium/Essential v2 regional graph. In Starter/Essential v1, the regional
control plane is also the shared dataplane: shared TiDB pools, keyspace pools,
gateway processes, and the regional service run together in the regional EKS.

Real staging is source material and validation input only, not a runtime
success dependency. The physical staging contexts named by the historical
Starter/Essential v1 doc are the ground truth:

- global CP context: `staging-base-us-west-2`
- regional CP+DP context: `staging-us-east-1-f01`

The cp9 runtime must still be cp9-owned:

1. local k3d runs the global CP services needed for OpenAPI entry;
2. #364 remote EKS runs the Starter regional CP+DP;
3. #365 AWS-hosted o11y stub replaces shared staging observability;
4. cp9 generates or stubs secrets/config by default;
5. any unknown imported key fails closed before render/up.

## Source Material Reviewed

Ground truth:

- `historical_eng_docs.md` in `SabaPing/personal-doc`, section
  [`starter/essential v1 physical architecture`](https://github.com/SabaPing/personal-doc/blob/main/work/.archive/essential/historical_eng_docs.md#starteressential-v1-physical-architecture)
- same document section
  [`Repos`](https://github.com/SabaPing/personal-doc/blob/main/work/.archive/essential/historical_eng_docs.md#repos)
- same document section
  [`Real staging environment access`](https://github.com/SabaPing/personal-doc/blob/main/work/.archive/essential/historical_eng_docs.md#real-staging-environment-access)

Primary controlplane9 source:

- `docs/20260608-design-cp9-starter-full-persona.md`
- `docs/20260608-adr-starter-full-remote-eks-o11y.md`
- `cp9.starter-full.example.yaml`

External repo source references:

- `tidbcloud/aws-shared-provider/cmd`
- `tidbcloud/aws-shared-provider/cd/projects`
- `tidbcloud/tidb-management-service/serverless/main`
- `tidbcloud/tidb-management-service/main/tidb-mgmt-portal`
- `tidbcloud/tidb-management-service/main/tidb-mgmt-service`
- `tidbcloud/tidb-management-service/main/serverless-global-service`
- `tidbcloud/serverless-service`
- `tidbcloud/app-delivery/serverless-svc`
- `tidbcloud/app-delivery/serverless-global-service`
- `tidbcloud/app-delivery/serverless-regional-service`
- `tidbcloud/app-delivery/tidb-mgmt-portal`
- `tidbcloud/app-delivery/tidb-mgmt-service`
- `tidbcloud/aws-shared-cd/pool`

Do not use these Premium/Essential v2 directories as Starter regional graph
source:

- `app-delivery/staging/nextgen-regional-control-plane`
- `app-delivery/staging/dedicated-cluster-ng-service`

## Live staging read-only verification

On 2026-06-08, after SSO refresh, the staging graph was checked with read-only
Kubernetes metadata calls through a temporary kubeconfig generated from the
`dbaas-dev` AWS profile. The probe used these command classes only:

- `aws eks update-kubeconfig` for `staging-base-us-west-2` and `staging-us-east-1-f01`
- `kubectl get ns`
- `kubectl get deploy,statefulset,svc -A` filtered to Starter/Essential v1 terms
- `kubectl get crd` filtered to serverless, gateway, keyspace, pool, and TiDB CRDs

No secrets, configmaps, or object payloads were read.

Observed global CP context `staging-base-us-west-2`:

- namespaces include `staging-ms`, `tidb-management-service`, `tidb-admin`,
  `kong`, and shared infrastructure namespaces;
- `staging-ms` runs `serverless-svc-http-server`,
  `serverless-svc-grpc-server`, `serverless-svc-worker`, and
  `serverless-svc-br-worker`;
- `tidb-management-service` runs `tidb-mgmt-portal`,
  `tidb-mgmt-service-sts`, `serverless-global-service-sts`,
  `serverless-global-regional-proxy`, and supporting services;
- `dedicated-global-service` and `nextgen-global-service` also exist in this
  shared global CP namespace, but that only confirms shared staging contains
  multiple product lines. It does not make those Premium/Essential v2 services
  part of the Starter/Essential v1 source graph.

Observed regional CP+DP context `staging-us-east-1-f01`:

- namespaces include `tidb-management-service`, `tidb-admin`,
  `regional-infra`, `serverless-metering`, `keyspace-pool`,
  `encrypt-keyspace-pool`, `tidb-pool`, `vip-tidb-pool`,
  `tidb-serverless`, `tidb-worker`,
  `monitoring-serverless`, and many `free-tier-staging-*` tenant namespaces;
- `tidb-management-service` runs `new-serverless-regional-service`;
- `tidb-admin` runs `manager`, `gateway-controller`, `serverless-meta-server`,
  `meta-store`, `tidb-gateway-inner`, and `tidb-http-gateway`;
- `regional-infra` runs `regional-redis`;
- `serverless-metering` runs metering services and workers;
- `tidb-serverless` runs TiDB Operator, discovery components, PD/TiDB/TiKV
  shared-pool statefulsets, keyspace etcd groups, and `tikv-worker`;
- `tidb-worker` runs `scaler`;
- CRDs include `keyspaces.serverless.pingcap.com`,
  `gateways.gateway.serverless.pingcap.com`,
  `tidbautoscalings.serverless.pingcap.com`, TiDB operator CRDs, and
  observability agent CRDs.

This live check confirms the historical doc's Starter/Essential v1 shape:
global CP services are separate from a regional EKS that is both regional
control plane and shared dataplane. It also confirms that #367 must not start
from the Premium/Essential v2 `cluster-service-ng` or `dataplane-manager` graph.

## Fail-Closed Rule

Fail-closed rule: every imported config key must be resolved before render.

Every imported config key must be resolved by one of these classification
outcomes. This is not a product API type; it is the #367 render/doctor contract
for deciding whether a value is safe to use or must fail closed.

| Outcome | Meaning |
| --- | --- |
| Source | cp9 builds and runs the owning service code from a declared repo/ref. |
| Live | cp9 deploys a real component or uses a real dependency, with owner/risk/cleanup recorded. |
| Stub | cp9 serves the dependency locally, remotely, or in-process with a deterministic contract. |
| Generated | cp9 creates the value or resource per session and records it as evidence. |
| Not needed | The component is excluded from the first Starter/Essential v1 proof, with reason. |

Unknown inputs are fatal at doctor/render/up time. Empty strings are allowed only
when the owning service documents that empty disables an off-path integration.

## Regional Service Graph

### Source

| Component | Runtime location | Why it is in scope |
| --- | --- | --- |
| `serverless-service` HTTP/gRPC/worker | local k3d global CP | Accepts Starter/Essential v1 OpenAPI work and owns worker regional config, o11y lookup, cluster proxy lookup, and pool-policy sync. |
| `tidb-mgmt-portal` | local k3d global CP | Historical global CP repo set includes the portal. Keep only if OpenAPI routing/proof needs it; otherwise classify as Not needed in #367 render. |
| `tidb-mgmt-service` | local k3d global CP | Global OpenAPI/core service entry and integration point for Starter/Essential v1 service APIs. |
| `serverless-global-service` | local k3d global CP | Global serverless control service and regional proxy path. It must point to the cp9 regional service, not staging. |
| `serverless-regional-service` | remote EKS regional CP+DP | Regional companion service deployed in real staging as `new-serverless-regional-service`; it carries cloud/region/kubernetes/infra-cluster-table and regional RPC behavior. |
| `aws-shared-provider` `manager` | remote EKS regional CP+DP | Manages shared TiDB pools, keyspace pools, pool capacity, TiDB image/version/template config, node selectors, and autoscale/pool decisions. |
| `aws-shared-provider` `meta-server` | remote EKS regional CP+DP | Bridges region/kube cluster metadata to the shared PD/etcd backend and RDS-secret-backed infra metadata. |
| `aws-shared-provider` `tidb-gateway` | remote EKS regional CP+DP | Public/inner/privatelink SQL gateway for token-to-TiDB routing, backend health, traffic metrics, and session behavior. |
| `aws-shared-provider` `tidb-http-gateway` | remote EKS regional CP+DP | HTTP gateway path for Starter/Essential v1 endpoints where required by the OpenAPI train. |
| `aws-shared-provider` `gateway-controller` | remote EKS regional CP+DP | Reconciles gateway CRDs and gateway pod annotations for endpoint routing. |
| `aws-shared-provider` `tidb-worker` | remote EKS regional CP+DP | Runs shared TiDB worker jobs against the same regional substrate. Keep on the first proof only if create/delete/endpoint paths require it. |
| `encrypt-keyspace-pool` | remote EKS regional CP+DP | Mirrors the f01 encrypted keyspace activation pool as a separate namespace/config surface from `keyspace-pool`. |

### Live

Live exceptions must name dependency, owner, risk, and cleanup.

Live does not mean shared staging. It means cp9 deploys the real component into
the cp9-owned remote EKS session or uses a read-only external artifact.

| Dependency | Owner | Risk | Cleanup |
| --- | --- | --- | --- |
| #364 remote EKS, nodegroups, VPC, ELB/NLB, and IAM substrate | cp9 session | Resource leak or over-broad IAM if owner/session scoping regresses. | `cp9 destroy` removes session-tagged AWS resources and proves zero survivors. |
| #365 AWS-hosted o11y stub Lambda/DynamoDB | cp9 session | Metrics proof is deterministic, not real workload telemetry. | `cp9 destroy` removes session-scoped Lambda/DynamoDB and evidence table. |
| Kubernetes CRDs and controllers for `aws-shared-provider` gateway/keyspace/autoscale paths | cp9 remote EKS session | CRD or controller drift can block create or cleanup. | Removed with remote EKS teardown. |
| PingCAP/ECR product images for TiDB, gateway, manager, meta-server, regional service | Owning service/release teams; cp9 records exact tags | Image tag drift or private registry auth can break proof. | No cloud resource cleanup; record tags in proof. |

Shared staging EKS, staging RDS, staging Redis, staging DynamoDB tables, staging
Route53 hosted zones, staging KMS/PCA material, staging kubeconfigs, and staging
client certs are not runtime Live exceptions for #367.

### Stub

| Dependency | Stub contract |
| --- | --- |
| Account, billing, dataflow, notification, service-plan, and Auth0 identity | Return deterministic identity/project/plan facts needed for Starter/Essential v1 create; do not test those product lines here. |
| Metering and traffic reporting | Stub unless Starter autoscale or endpoint proof requires a user-visible metering assertion. Gateway may emit to a cp9 stub endpoint. |
| Email/import/export/backup/BR/Lightning dependencies | Stub or disable unless the selected Starter/Essential v1 train explicitly runs those flows. |
| External eventbus/queue/object-store paths | Stub unless a source service refuses to start without them; then record a generated local endpoint. |

### Generated

Prefer Terraform for generated Starter regional resources and values whenever the
value is cloud infrastructure, Kubernetes substrate, cloud-service wiring, or
session-owned persistent state. Go code should orchestrate the lifecycle, render
source-owned application manifests, and validate fail-closed contracts; it should
not become the default place to hand-create infrastructure values that Terraform
can own.

| Input | Generated by cp9 |
| --- | --- |
| local k3d kubeconfig | Existing cp9 host lifecycle. |
| regional kubeconfig | #364 remote EKS output mounted into global CP and regional components that need an explicit kubeconfig. |
| o11y base URL, token, and evidence table | #365 AWS-hosted o11y stub output. |
| `serverless-service` regional config | cp9 renders region, `kubernetes`, `o11yUrl`, `clusterProxyUrl`, `k8sIngressUrl`, and any auth material from session outputs. |
| `serverless-regional-service` args/config | cp9 renders `--ns-prefix`, `--cloud-provider`, `--region`, `--infra-cluster-table`, `--kubernetes`, TiDB version, and service addresses. |
| manager pool config | cp9 renders pool namespaces (`tidb-pool`, `vip-tidb-pool`, `keyspace-pool`, `encrypt-keyspace-pool`, etc.), sizes, node selectors, TiDB image/version, priorities, VPA/autoscale booleans, and resource requests. |
| endpoint-routing inputs | cp9 renders gateway hostnames, service DNS, optional Route53 records, TLS secrets, and returned endpoint proof. |
| Redis, infra tables, and queue addresses | Prefer local/session-generated endpoints or deterministic stubs; never copy staging endpoints. |
| etcd/PD endpoints | cp9 remote EKS shared TiDB/PD runtime output or generated local substitute if source code allows. |
| client certs and TLS material | cp9-generated cert-manager/self-signed material for gateway, TiDB, worker, meta-server, and Redis client TLS. |
| RDS/MySQL/metadb facts | cp9-owned metadb and generated seed rows. Staging RDS values are forbidden. |
| KMS/PCA/secrets/API keys | cp9-generated/session-scoped material or disabled off-proof path. Staging ARNs and ciphertexts are forbidden. |

### Not Needed

| Component | Reason |
| --- | --- |
| `cluster-service-ng`, `dataplane-manager`, and `nextgen-global-service` | Premium/Essential v2 graph, not Starter/Essential v1. |
| `app-delivery/staging/nextgen-regional-control-plane` | Premium/Essential v2 app-delivery env, not Starter regional source. |
| `app-delivery/staging/dedicated-cluster-ng-service` | Dedicated/Premium cluster-ng env, not Starter regional source. |
| Alicloud and Azure providers | First `starter-full` train is AWS only. |
| `datacleaner`, `audit-log`, `ops-service`, `csi-node-taint`, `eks-qps`, `etcd-qps` | Regional support/ops utilities; exclude until a create/delete/endpoint proof requires them. |
| Real `o11y-agent`, scrape pipelines, and Victoria/Prometheus stacks | #365 supplies the query surface. Real telemetry is a later confidence train. |
| BR, Lightning, import/export pools | Not part of first create, endpoint, autoscale, or SQL proof unless Essential v1 research shows they are mandatory. |

## Config Classification

| Area | Classification | Notes |
| --- | --- | --- |
| o11y env/config | Generated plus upstream-hook-needed | Use #365 stub URL/token. Any service that hardcodes staging Prometheus or prepends `http://` to a host needs a config-only hook for full base URL and auth. |
| regional kubeconfigs | Generated | Use #364 remote EKS kubeconfig. Staging kubeconfig secrets are forbidden. |
| Redis | Stub or Generated | Gateway and regional services may require Redis/cache/API-key behavior; provide local/session Redis or a deterministic stub. Staging Redis is forbidden. |
| etcd / PD | Generated or in-session Live | Real architecture uses PD/etcd in the regional dataplane. cp9 must deploy/generate that endpoint in remote EKS or fail closed. |
| client certs | Generated | Manager/meta-server/tidb-worker/tidb-gateway certs must be session generated. |
| gateway | Source plus Live plus Generated | Run `tidb-gateway`, optional `tidb-http-gateway`, and `gateway-controller`; generate hosts, TLS, HAProxy/LB settings, and endpoint evidence. |
| keyspace | Source plus Generated | Use `aws-shared-provider` keyspace CRDs/controllers and generated pool seed facts. Keyspace ids/names come from the cp9 create flow. |
| pool | Source plus Generated | Use manager/pool code and generated namespaces/sizes/images/node selectors. Starter autoscale proof uses deterministic o11y responses against these pools. |
| endpoint-routing | Generated | `clusterProxyUrl`, `k8sIngressUrl`, gateway hostnames, and optional DNS records point to cp9 remote EKS services. Staging ELB/DNS is forbidden. |
| image registry/tags | Live exception | Pull exact product image tags from reviewed source/staging config as read-only input and record them. |
| RDS/MySQL | Generated | Use cp9-owned metadb and seed rows. Staging RDS is forbidden. |
| KMS/PCA | Generated or disabled | Use session-scoped keys/certs or disable off-proof paths. Staging KMS/PCA/ciphertexts are forbidden. |

## Upstream config hooks

These hooks are implementation inputs for #367 and later. They are
configuration-only; they must not bypass product logic.

| Owning repo | Hook | Contract |
| --- | --- | --- |
| `serverless-service` | o11y base URL and auth token/header | Accept a full `http://` or `https://` Prometheus base URL and optional token/header for #365. |
| `serverless-service` | regional endpoint-routing config | Allow cp9 to render `worker.regionConfig[].k8sConfig[]` from remote EKS outputs. |
| `tidb-management-service/serverless-global-service` | regional proxy target | Point regional proxy and region records at cp9 `serverless-regional-service`, not staging. |
| `tidb-management-service/tidb-mgmt-service` | Starter/Essential v1 service endpoints | Allow OpenAPI/core config to point at cp9 serverless-global/service endpoints. |
| `tidb-management-service/serverless/main` `serverless-regional-service` | regional service args/config | Accept cp9-rendered `kubernetes`, region, infra table, TiDB version, service URLs, and generated secrets. |
| `aws-shared-provider` `manager` | pool/keyspace/o11y/gateway config | Accept cp9-rendered pool namespaces, image tags, node selectors, remote-write/o11y, Redis, RDS/metadb, and cert paths. |
| `aws-shared-provider` `tidb-gateway` / `tidb-http-gateway` | endpoint-routing config | Accept cp9-generated Redis/TLS/infra table/kubeconfig/gateway mode/hosted-zone inputs. |
| `aws-shared-provider` `meta-server` / `tidb-worker` | certs, PD/etcd, RDS/metadb, kubeconfig | Accept cp9-generated cert paths and in-session backend endpoints. |
| `app-delivery` / Pulumi render sources | cp9 environment render | Render from Starter source directories and cp9-provided values; do not copy staging envs or other product-line directories. |

## Review Gate for #367

#367 may begin deploy work only after this classification is accepted or amended.
The first deploy slice should add a doctor/render check that enumerates every
regional input and fails closed if the input is still unknown, staging-sourced,
or missing owner/session cleanup evidence. When implementing generated regional
inputs, default to Terraform-owned outputs first; add Go generation only for
application render/orchestration gaps that Terraform cannot naturally own.