# Premium / Premium Reserved: from create-API RCU to dataplane machines

How a TiDB Cloud Premium-family create call turns into concrete AWS EC2 machines
(count and size), where each decision is made, and which knobs cp9 can turn
without touching upstream service source.

This was reverse-engineered from `cluster-service-ng` source and the cp9-stored
configs/seeds while investigating why a "minimal" Premium create on cp9 spun up
~22 machines. File:line references to upstream services use the service repo's
own paths (e.g. `cluster-service-ng: domain/server/gen_component.go`); cp9 paths
are repo-relative to `controlplane9`.

## TL;DR

- There are **two create paths** in the Premium family:
  - **`Premium` (RCU-driven)** — you pass `minRcu`/`maxRcu`; the machine **count and
    size are computed by `cluster-service-ng`** from RCU.
  - **`Premium_Reserved` (topology-driven)** — you pass a `reservedCapacityTemplateId`;
    the machine **count and size are read verbatim from a fixed-topology template**
    that cp9 seeds into the metadb.
- For the RCU path, the **per-component count floor is hardcoded** in
  `cluster-service-ng` (PD=3, PD_TSO=2, TiProxy≥2, TiKV floored to a multiple of 3
  with min 3, each worker=1, resource_manager=2, plus fixed RU-per-vCPU ratios).
  cp9's `nodeCountQuota` is already at/below these floors, so **cp9 config cannot
  reduce the RCU-path count** — only an upstream code change can.
- The **node size (the `4c8g` class → `c8g.xlarge`) IS reachable from cp9 config**:
  the RU→class table and the dataplane node pools are both cp9-owned. Introducing a
  smaller registered class (e.g. `2c4g`→`c8g.large`) and repointing the components
  is a cp9-only change (pending validation that the DB components run on it).
- The observed count is the **whole shared pool's** RCU sum (this cluster + other
  tenants on the pool), and per-component counts **ratchet up, never down**.
- The 3× `t3.medium` "appbox" is a **dataplane-manager** infra concept (the EKS
  admin nodegroup), not chosen by the RCU calculator.

## Service-type meaning

`cluster-service-ng` does not have a literal "Premium" branch; the product names map
onto capacity modes and shared-pool shape
(`cluster-service-ng: domain/workflow/action/send_metering_metadata.go`):

| Product name        | CapacityMode        | Shared pool shape                         |
| ------------------- | ------------------- | ----------------------------------------- |
| `Premium`           | `RCU_DRIVEN`        | RCU-driven shared pool, no shared TiProxy |
| `Essential`         | `RCU_DRIVEN`        | RCU-driven shared pool, with shared TiProxy |
| `Premium_Reserved`  | `TOPOLOGY_DRIVEN`   | fixed-topology pool from a template       |

## Path A — `Premium` (RCU-driven): API → machines

### 1. API entry
`CreateTiDBCluster` (`cluster-service-ng: domain/app/app.go`) requires a
`RequestUnit{Min,Max}` unless the mode is `TOPOLOGY_DRIVEN`. The domain service
`CreateTiDBCluster` (`domain/server/create_tidbcluster.go`) loads the tenant's
`RuConfig` (`create_sharedpool.go`, `ruConfigRepo.GetRuConfig`) and calls topology
generation.

### 2. Pool-wide RCU aggregation (why a single create can be "big")
`getRequestUnitTotal` (`domain/server/create_tidbcluster.go`) sums this cluster's
`Min`/`Max` with **all other clusters already on the same shared pool**. So the
machine count reflects the **whole pool's** RCU, not just this request. Counts are
reconciled by `compareNodeQuota`, which **only ratchets node counts up**.

### 3. The calculator — `gen_component.go`
For each component, **count is derived from `minRcu`, the node spec/class from
`maxRcu`**:

- Hardcoded RU-per-vCPU ratios (`cluster-service-ng: domain/server/gen_component.go`):
  `tiproxyPerCpuRu=15000`, `tidbPerCpuRu=5000`, `tikvPerCpuRu=2500`,
  `tiflashPerCpuRu=9000`. CREATING-phase init caps: TiKV 9, TiDB 6, TiProxy 2,
  TiFlash 1.
- Per-component count:
  - **TiKV**: `min / (cpu * 2500)`, then floored to a multiple of 3 with **min 3**.
  - **TiDB**: `max(min / (cpu * 5000), 1)`.
  - **TiProxy**: `max(min / (cpu * 15000), 2)` — **hard min 2**.
  - **PD = 3**, **PD_TSO = 2**, **PD_SCHEDULING = 2** — fixed.
  - **TiKV_worker / coprocessor_worker / TiDB_worker = 1** each — fixed.
  - **resource_manager = 2** — fixed, only if `enableResourceManager`.
  - **TiFlash write/compute**: `max(min / (cpu * 9000), 1)`, capped by its quota
    (cp9 sets the quota to 0 → no TiFlash).
- `buildNodeQuota` = `min(nodeCountQuota, calculated[, initCap if CREATING])`, then
  the per-component transform/floor, then the min-override.
- Node spec/class: `GetNodeSpec(component, maxRcu)`
  (`cluster-service-ng: domain/entity/utils.go`) walks the component's
  `NodeSpecRuRange` and picks the class whose `[min,max]` contains `maxRcu`.
- `GetNodeCountQuota` defaults to **60** if unset.

### 4. Worked example: `minRcu=5000, maxRcu=6000`, 4-CPU classes
A single fresh cluster emits roughly:

| Component            | Count | Why                                   |
| -------------------- | ----- | ------------------------------------- |
| TiKV                 | 3     | `5000/(4*2500)=0` → floor-to-3        |
| TiDB                 | 1     | `max(5000/(4*5000),1)`                |
| TiProxy              | 2     | hard min 2                            |
| PD / PD_TSO / PD_SCH | 3/2/2 | fixed                                 |
| TiKV/copro/TiDB worker | 1/1/1 | fixed                               |
| resource_manager     | 2     | fixed (if enabled)                    |
| TiFlash w/c          | 0/0   | quota 0 in cp9 config                 |

≈ **16-node floor** for one cluster. The measured **~19 `c8g.xlarge`** on cp9 is
this floor plus the pool-wide RCU sum (other tenants) and the ratchet-up behaviour.

### 5. The config that drives it (`server.ruConfig`)
The calculator reads a per-tenant `RuConfig`; with **no `ru_config` DB row**, it
uses the **service startup config** verbatim — which cp9 stores at
`personas/app-delivery/envs/premium/cluster-ng-server/config.yaml` under
`server.ruConfig`:

- `nodeCountQuota.<component>` — per-component **max** node cap. cp9 already sets
  these minimally (`tikv: 1`, `tidb: 1`, `pd: 1`, …).
- `components.<component>.<class>: {min,max}` — the RU→class lookup table (the
  **size** selector).

`cluster-service-ng: pkg/storage/mysql/ruconfig/ruconfig.go` converts the startup
`RuConfig` into the entity used by the calculator and returns it when the tenant has
no override row.

## Path B — `Premium_Reserved` (topology-driven): API → machines

You pass `reservedCapacityTemplateId`; the topology is **read verbatim** from a
`reserved_capacity_templates` row that **cp9 seeds**
(`personas/app-delivery/envs/premium/premium-metadb/10-nextgen-global-service-20260521.sql`).
Template `id=1` (the cp9 dogfood template) carries a `fixed_topology` JSON that
explicitly lists each component's `node_spec{cpu,memory_gi}` **and** `node_count`:

```
PD:3, TiKV:3, TiKV_worker:3, TiDB_worker:3, TiDB:2, TiProxy:2   (all cpu:4 / mem:8)
purchased_vcpu: 40
```

Because both the **size and count are explicit in a cp9-owned seed**, the reserved
path's machines are **fully cp9-controllable** with no service-code change: lower a
`node_count` to cut machines, lower a `node_spec` to shrink them.

## Class → AWS instance (dataplane node pools)

The component class string (`4c8g`) is mapped to a concrete AWS instance type by
**dataplane-manager**, not cluster-ng:

- cluster-ng builds a `nodeSpecMap` from dataplane-manager's node pools
  (`cluster-service-ng: domain/server/create_sharedpool.go`,
  `genNodeSpecMap` ← `ListNodePools`); a class that is not registered there fails
  create with `node spec ... not found`.
- The node pools (and their AWS instance types) are defined in the cp9-owned
  `personas/app-delivery/envs/premium/dataplane-manager/root-config.yaml`:
  - `nodePools`: `c8g.xlarge` (4 vCPU / 8 GiB) and `c8g.2xlarge` (8 vCPU / 16 GiB).
  - `adminNodeConfig` ("appbox"): `t3.medium`, `minSize: 1`, `maxSize: 3` — the EKS
    admin nodegroup (load-balancer controller, cert-manager, cluster-autoscaler,
    etc.), **independent of the RCU calculator**.
- Worker nodes are scaled by `cluster-autoscaler` to fit the pods the calculator
  schedules; idle workers scale to 0 (slowly — autoscaler cooldown), the appbox
  keeps ≥1.

## Where each lever lives

| Lever                                   | Location                                                                 | Owner            |
| --------------------------------------- | ------------------------------------------------------------------------ | ---------------- |
| RCU→class table, per-component quota    | `cluster-ng-server/config.yaml` `server.ruConfig.{components,nodeCountQuota}` | **cp9 config**   |
| Reserved-template topology (count+size) | `premium-metadb/10-nextgen-global-service-*.sql` `reserved_capacity_templates.fixed_topology` | **cp9 seed** |
| Worker node pools + AWS instance types  | `dataplane-manager/root-config.yaml` `nodePools`                         | **cp9 config**   |
| Appbox node size/min/max                | `dataplane-manager/root-config.yaml` `adminNodeConfig`                   | **cp9 config**   |
| RU-per-vCPU ratios; init caps           | `cluster-service-ng: gen_component.go` constants                         | service code     |
| Per-component floors (PD 3, TSO 2, TiKV mult-of-3 ≥3, TiProxy 2, workers 1, RM 2) | `cluster-service-ng: gen_component.go` | service code |
| Default quota = 60                      | `cluster-service-ng: domain/entity/utils.go`                            | service code     |
| Registering a brand-new class           | dataplane-manager node pools + `node_instance_groups`                    | cp9 (dataplane-manager config) |

## Reducing the minimal footprint

- **Count (RCU path): not reducible from cp9 config.** cp9's `nodeCountQuota` is
  already at the floors; the real minimum (~16 nodes/cluster) is the hardcoded
  per-component floors and RU ratios in `cluster-service-ng`. Lowering it needs an
  upstream change (e.g. configurable floors, or smaller fixed minimums for a "dev"
  profile). The only cp9-side count win is disabling `resource_manager` (−2) and
  keeping TiFlash quota at 0 (already done).
- **Count (reserved path): fully reducible from cp9 config** — edit the
  `fixed_topology` `node_count`s in the template seed.
- **Size: reducible from cp9 config (both paths)** — add a smaller node pool
  (`c8g.large`, 2 vCPU) to `dataplane-manager/root-config.yaml`, register a `2c4g`
  class, and point the components at it (RCU path: `components.*`; reserved path:
  the template `node_spec`). This roughly halves per-node cost. **Must be validated**
  that TiKV/PD/TiDB run acceptably on a 2 vCPU / 4 GiB node.
- **Idle warm pool**: workers scale to 0, leaving the EKS control plane + ≥1
  `t3.medium` appbox + the VPC/NAT network; that is the steady-state idle cost.

## Takeaways for the warm-pool work

- A "minimal" Premium **RCU** create is bounded below by service-code floors; cp9
  cannot make it arbitrarily small without an upstream change.
- The **reserved-template path** gives cp9 full, seed-level control of both count and
  size — relevant if the warm pool (and/or agent loops) can use a reserved template.
- The cleanest cp9-only reduction available today is **machine size** (a smaller
  registered node class), applicable to either path.