# Tenant-Aware Autoscale Policy Redesign

> **Document structure**: Part I is the **active design** (what's implemented or being implemented). Part II is **No Goal / Wrong Design** (explicitly not in this design + rejected alternatives). Part III is **Historical / Outdated** (prior PRs, removed code paths — kept only for traceability).
>
> Reading path: TL;DR → Part I (active contract) → Part II (to verify what's in/out of scope) → Part III (only when tracing earlier PRs).

---

## Part I: Active Design

### Terminology

| Term | Meaning (autoscaler internal) | Meaning (K8s HPA) |
|---|---|---|
| SCALE IN | Horizontal scale-in (NODE COUNT--) | HPA `Behavior.ScaleDown` |
| SCALE OUT | Horizontal scale-out (NODE COUNT++) | HPA `Behavior.ScaleUp` |
| SCALE UP | Vertical scale-up (CPU/MEM++) | (no HPA equivalent; cloud API) |
| SCALE DOWN | Vertical scale-down (CPU/MEM--) | (no HPA equivalent; cloud API) |

> Note: the autoscaler-internal SCALE IN/OUT naming is **reversed** vs. HPA `ScaleUp/Down`. Reviewers must map per the table above when reading HPA code.

### Previously Implemented

- TIDB SCALE IN/OUT supported via HPA
- TIPROXY SCALE IN/OUT supported via HPA
- TIKV_WORKER / TIDB_WORKER / COPROCESSOR_WORKER SCALE IN/OUT supported via HPA
- TIKV SCALE OUT supported via autoscaler; gated by `ru_configs.FeatureFlags.enable_autoscale_tikv_cpu_scale_out`
- TIKV DISK EXPAND supported via autoscaler; gated by `ru_configs.FeatureFlags.enable_autoscale_disk_expand`
- TIKV IOPS / THROUGHPUT supported via autoscaler; gated by `ru_configs.FeatureFlags.enable_autoscale_iops_throughput_expand`
- SHARED_POOL RCU quota adjustment; gated by `ru_configs.FeatureFlags.enable_scale_shared_pool_by_ru`
- TIDB GROUP RCU quota adjustment; gated by `ru_configs.FeatureFlags.enable_scale_tidb_group_by_ru`
- TIPROXY SCALE IN is stored on HPA `Behavior.ScaleDown`. In disabled mode, `ApplyComponentHpaAction` may seed that slot from `ru_configs.FeatureFlags.enable_autoscale_tiproxy_scale_in` (see §Authority when configs conflict).

### TL;DR

Previously the core configuration lived at the sharedpool / tidb_group level; default switches were hardcoded in config files, with no per-tenant customization, and storage was scattered across multiple interfaces (`SetHPAScaleBehavior` / `SetRuConfig(sharedpool)`). This redesign adds:

1. **DEFAULT / TENANT level** configuration management — affects newly created clusters by default
2. **TENANT level** configuration changes that fan out to existing sharedpool / cluster
3. A **unified SHAREDPOOL / CLUSTER configuration interface** — without modifying existing storage metadata; both the new unified interface and the legacy `SetHPAScaleBehavior` / `SetRuConfig(sharedpool)` paths remain available

**This iteration's scope**: only controls **on/off switches** (the 6 autoscale flags, including `enable_autoscale_tiproxy_scale_in`). **Thresholds / limits / steps / cooldowns** remain out of scope and continue to be read from their original sources (HPA template, scaler config YAML, etc.).

### Deployment Safety Control

The policy subsystem is disabled unless a region explicitly acknowledges
activation for all supported single-tenant pools:

```yaml
server:
  biz:
    autoscalePolicy:
      mode: disabled       # disabled | enforce
      allowAllTenants: false
```

- The zero value and an omitted stanza are `disabled`.
- `mode: enforce` is valid only with `allowAllTenants: true`; setting either
  field alone fails startup or remains disabled.
- In disabled mode, all AutoscalePolicy RPCs keep their wire surface but return
  `FailedPrecondition`. SharedPool creation uses the legacy tenant RuConfig
  whole row, and HPA workflows use the built-in per-component baseline without
  reading policy bundles. The one exception is TIPROXY ScaleDown: see
  §Authority when configs conflict. TIDB scale-down uses the conservative
  `MinChange` baseline in both modes. Legacy `SetRuConfig`, `SetHPAScaleBehavior`,
  and the normal HPA lifecycle remain active.
- `enforce` activates the existing policy implementation for every
  single-tenant SharedPool whose persisted provider tenant tag is available.
  A task tenant, when present, must match that authoritative tag. Pools carrying
  the `shared-multi-tenant` marker remain on the legacy path because a pool-level
  tenant policy has no unambiguous owner there. Tenant allowlists and shadow mode
  are not implemented by this control and must not be assumed from unknown YAML
  fields; they require ownership-aware request checks in a follow-up. Once
  enforce is enabled, tenant or bundle resolution errors fail the HPA action
  instead of falling back to a policy that may allow more scaling.
- Repositories and additive migrations are constructed in both modes. The gate
  controls behavior, not schema availability.

### Authority when configs conflict

HPA `spec.behavior.SelectPolicy` is the runtime store for worker / TIDB /
TIPROXY scale-in-out. `ru_configs.FeatureFlags` is the runtime store for TIKV /
RCU flags. Those two writers stay independent **except for one explicit
overlay**: in disabled mode, `ApplyComponentHpaAction` seeds TIPROXY
`ScaleDown` at HPA create time from `enable_autoscale_tiproxy_scale_in`, and
explicit `SetRuConfig(SHARED_POOL, ...)` writes that change that flag sync
existing TiProxy HPAs.

`AutoscalePolicy.IsEnforced()` chooses which config source
`ApplyComponentHpaAction` consults. **The same operator action has opposite
results on the two sides of this switch.** When two sources disagree, the first
matching row below wins:

| Priority | Condition | Source that wins | What loses |
|---|---|---|---|
| 1 | Explicit `SetRuConfig(SHARED_POOL, enable_autoscale_tiproxy_scale_in=...)` value change | The write path syncs existing TiProxy HPAs in that pool (true → `MinChange`, false → `Disabled`). | The previous live HPA ScaleDown slot. |
| 2 | Existing HPA `SelectPolicy` is non-nil | The live HPA value | ApplyComponentHpaAction desired values. `reconcileExistingHPA` never overwrites a set slot. |
| 3 | `IsEnforced()==true` (single-tenant) | TENANT bundle > DEFAULT bundle > `OriginalScale*` | FeatureFlags, YAML `config.RuConfig`, and the tenant RuConfig row. `applyTiProxyScaleInGate` is **not** called. |
| 4 | `IsEnforced()==false`, or enforce+multi-tenant (non-TIPROXY-ScaleDown) | Per-component `OriginalScale*` baseline, fill-nil-only | The policy bundle is **not** read. |

YAML `server.ruConfig.enableAutoscaleTiproxyScaleIn` is **not an HPA input**.
It is region-scoped process config. `GetRuConfigWithFound` keeps the same
fallback value as `GetRuConfig` while reporting whether the row exists, so
`applyTiProxyScaleInGate` can distinguish "missing row" from "persisted row
whose value matches YAML". Flipping YAML does not flip every TiProxy HPA in the
region. Disabled-mode `CreateSharedPool` also clears
`enable_autoscale_tiproxy_scale_in` when the resolved value comes from YAML
fallback. The overlay honors any persisted SHARED_POOL value that is
**explicitly true** — including a value that happens to equal YAML.

The **pool FeatureFlags flag** is a live explicit switch for TIPROXY ScaleDown
in disabled mode: `SetRuConfig(SHARED_POOL, ...)` writes that change
`enable_autoscale_tiproxy_scale_in` update the pool's existing TiProxy HPAs
immediately. Replacing the row with the same flag value has no HPA side effect,
and unrelated `ApplyComponentHpaAction` reconciles do not overwrite a non-nil
HPA `SelectPolicy`.

| Operator action | disabled (`IsEnforced()==false`) | enforce (`IsEnforced()==true`) |
|---|---|---|
| YAML `enableAutoscaleTiproxyScaleIn=true` | **No HPA effect** (region YAML is ignored by the overlay and not frozen into new pool rows) | Ignored for HPA |
| `SetRuConfig(pool, enable_autoscale_tiproxy_scale_in=true)` | Writes the pool flag and immediately syncs existing TiProxy HPAs in **that pool** to `ScaleDown=MinChange` if the previous persisted flag was nil / missing / false | Ignored for single-tenant HPA; enforce+multi-tenant follows the disabled path |
| `SetRuConfig(pool, enable_autoscale_tiproxy_scale_in=false)` | Writes the pool flag and immediately syncs existing TiProxy HPAs in **that pool** to `ScaleDown=Disabled` if the previous persisted flag was nil / missing / true | Ignored for single-tenant HPA; enforce+multi-tenant follows the disabled path |
| TENANT / DEFAULT bundle `TIPROXY SCALE_IN ENABLED` | No effect on HPA (bundle unread; RPC `FailedPrecondition`) | New or nil-slot TiProxy HPAs get `ScaleDown=MinChange` |
| `SetHPAScaleBehavior` | Writes HPA directly; later unrelated reconcile will not clobber a non-nil slot. | Writes HPA directly; later reconcile will not clobber |

This overlay is one-way FeatureFlags→HPA (`applyTiProxyScaleInGate` in
`domain/workflow/action/apply_hpa.go` for create-time seed, and
`syncTiProxyScaleInGateFromRuConfig` for explicit
`SetRuConfig(SHARED_POOL, ...)` flag-value changes), not the removed
bidirectional
`syncHPAFromRuConfig`. Enforce mode still does not read FeatureFlags for HPA.
See invariant 6.

### SHARED_POOL Policy Scope

- TIKV_WORKER / TIDB_WORKER / COPROCESSOR_WORKER SCALE IN/OUT
- TIKV SCALE OUT / DISK EXPAND / TIKV IOPS / THROUGHPUT
- SHARED_POOL RCU quota adjustment / TIDB GROUP RCU quota adjustment
- TIPROXY SCALE IN/OUT (only the **pool-shared** variant; see §Pool-shared TIPROXY below)

**Write dispatch at SHARED_POOL scope**: the bundle RPC write path (`applyPolicyPatchToTargets`) dispatches per component type to the correct storage point:

| Component category | Components | Storage point | Write mechanism |
|---|---|---|---|
| Worker (HPA-backed) | TIKV_WORKER, TIDB_WORKER, COPROCESSOR_WORKER | K8s HPA `spec.behavior` | `writeSharedPoolWorkersHPABehavior` → `SetHPAScaleBehavior` |
| TIKV / RCU / virtual (FeatureFlags-backed) | TIKV, SHARED_POOL, TIDB_GROUP | `ru_configs.FeatureFlags` | `writeRuConfigByOperation` → `SetRuConfig` |
| TIPROXY (pool-shared, `HasSharedTiProxy`) | TIPROXY | K8s HPA `spec.behavior` | `writeSharedPoolWorkersHPABehavior` → `SetHPAScaleBehavior` |

For POST (REPLACE) operations, both storage points are written: the HPA path for worker specs, and the FeatureFlags path to clear any prior overrides (even when the request has only worker specs — an all-nil FeatureFlags block is written to satisfy the REPLACE contract). PATCH operations only modify the storage point addressed by the request specs (merge semantics).

### CLUSTER Policy Scope

- TIDB SCALE IN/OUT
- TIPROXY SCALE IN/OUT (per-group variant — addressed by `groupId`)
  - **Read**: `GetAutoscalePolicy` / `GetEffectiveAutoscalePolicy` at TIDB_GROUP scope surfaces the per-group TIPROXY HPA value.
  - **Write**: Per-group TIPROXY at TIDB_GROUP scope is **accepted** — routes through `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`, same as TIDB. See §Pool-shared TIPROXY for the full scope×variant write matrix.

> Note: CLUSTER policy is currently stored entirely in HPA. There are no `ru_configs` table parameters for TIDB_GROUP scope, and no `ru_configs` rows are created for CLUSTER.

### Pool-shared TIPROXY

`entity.HasSharedTiProxy(sharedPool)` (i.e. `DeployType == "shared_tiproxy"`) denotes a pool that has exactly **one** TiProxy HPA serving the whole pool, rather than one per cluster. The shared TiProxy component is generated at sharedpool-create time (`domain/server/gen_component.go`) with `Id = FnvHash(sharedPoolID)`; the HPA label is `pingcap.com/group=db-<FnvHash(spID)>`, the HPA name is `tiproxygroup-<groupName>` (matches the per-group naming scheme). This single pool-level HPA is the canonical store for SHARED_POOL-scope TiProxy policy.

**Why SHARED_POOL scope (not CLUSTER)**: the pool-level HPA is genuinely pool-wide — there is one per pool, regardless of how many clusters it hosts. TIDB_GROUP scope is keyed by per-group `groupId`; per-cluster reads (`readRuntimeSpecsForCluster(groupId)`) use the K8s selector `group=db-<groupId>` and so do **not** match the pool-level HPA (its group label differs). SHARED_POOL scope is the only read scope that naturally surfaces it.

**Scope distinction**:

| Variant | HPA group label | Reads at |
|---|---|---|
| Per-group TIPROXY (non-shared pool, default) | `db-<tidbClusterID>` | TIDB_GROUP scope only (`groupId = clusterId`) |
| Pool-shared TIPROXY (`HasSharedTiProxy` pool) | `db-<FnvHash(spID)>` | SHARED_POOL scope only |

A pool may host clusters with their own per-group TiProxy HPA **and/or** a single pool-shared TiProxy HPA. Under the active design, only the latter is exposed at SHARED_POOL scope; per-group TiProxy stays CLUSTER-only.

**Cross-scope behavior (TIPROXY write path — topology-aware accept/reject)**:

Bundle RPCs (`CreateOrReplaceAutoscalePolicy`, `PatchAutoscalePolicy`, `SyncAutoscalePolicy`) accept TIPROXY spec writes at topology-correct scopes and reject cross-topology combinations via label selector mismatch. On the **read** side (Get / GetEffective), the scope×variant combinations produce the following observable outcomes:

| Pool topology | Operator action | Result |
|---|---|---|
| `!HasSharedTiProxy` (per-group TiProxy, default) | Get / GetEffective CLUSTER TIPROXY | Returns the per-group HPA value |
| `!HasSharedTiProxy` (per-group TiProxy, default) | Get / GetEffective SHARED_POOL TIPROXY | **Returns nothing**: the runtime read of SHARED_POOL does not surface any TIPROXY HPA in this topology (per-group TiProxy is CLUSTER-only). |
| `HasSharedTiProxy` (pool-shared TiProxy) | Get / GetEffective CLUSTER TIPROXY | **Returns nothing**: same selector mismatch — per-cluster CLUSTER reads do not surface the pool-shared HPA. |
| `HasSharedTiProxy` (pool-shared TiProxy) | Get / GetEffective SHARED_POOL TIPROXY | Returns the pool-shared HPA value (the only combination where SHARED_POOL + TIPROXY surfaces data). |

On the **write** side (CreateOrReplace / Patch / Sync), the four scope×variant combinations follow topology-aware accept/reject:

| Pool topology | Operator action | Result |
|---|---|---|
| `!HasSharedTiProxy` (per-group TiProxy, default) | Bundle RPC TIDB_GROUP TIPROXY | **Accepted** — routes through `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`, same as TIDB |
| `!HasSharedTiProxy` (per-group TiProxy, default) | Bundle RPC SHARED_POOL TIPROXY | **Rejected**: no pool-shared TIPROXY in this topology (per-group TiProxy is CLUSTER-only) |
| `HasSharedTiProxy` (pool-shared TiProxy) | Bundle RPC TIDB_GROUP TIPROXY | **Rejected**: pool-shared TIPROXY is SHARED_POOL-only (its `group=db-<FnvHash(spID)>` label does not match any per-cluster CLUSTER write) |
| `HasSharedTiProxy` (pool-shared TiProxy) | Bundle RPC SHARED_POOL TIPROXY | **Accepted** — routes through SHARED_POOL HPA write path |

> Two combinations are accepted (per-group TIPROXY at TIDB_GROUP scope, pool-shared TIPROXY at SHARED_POOL scope) and two are rejected as topology mismatches (per-group TIPROXY at SHARED_POOL scope, pool-shared TIPROXY at TIDB_GROUP scope). The rejected combinations fail naturally via label selector mismatch rather than an explicit component-type block.

**Rationale for topology-aware accept/reject**: bundle RPCs accept TIPROXY writes at the correct scope for each topology variant — per-group TIPROXY at TIDB_GROUP scope routes through `writeTidbGroupHPABehavior` (same as TIDB), and pool-shared TIPROXY at SHARED_POOL scope routes through the pool-scoped HPA write path. Cross-topology combinations (per-group TIPROXY at SHARED_POOL, pool-shared TIPROXY at CLUSTER) are rejected naturally via label selector mismatch rather than an explicit component-type block at the RPC entry, keeping the Fanout Eligibility matrix internally consistent.

**Storage**: HPA `spec.behavior.Scale{Up,Down}.SelectPolicy` on the `tiproxygroup-<db-FnvHash(spID)>` HPA is the runtime store. Same translation table as CLUSTER TIPROXY (including the Disabled→MinChange flip for SCALE_IN ENABLED). There is no SHARED_POOL-bundle override tier and no `autoscale_policies` row.

`ru_configs.FeatureFlags.enable_autoscale_tiproxy_scale_in` exists (added with the 6th autoscale flag). It is **not** a second runtime store for TIPROXY SCALE_IN. It is the disabled-mode overlay described in §Authority when configs conflict: create-time seed plus explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change sync so the flag can be turned off without relying on unrelated reconciles. Enforce-mode HPA writes still come from the bundle / `SetHPAScaleBehavior`, not from this flag.

**Write path (this iteration)**: at `CreateSharedPool` time, `ApplyComponentHpaAction` iterates `sharedPool.ComponentGroups`; when `HasSharedTiProxy` it includes the shared TiProxy component. Enforce mode calls `ResolveHPABehaviorFromBundle(tenantId, TIPROXY)` (DEFAULT+TENANT bundle → translation table → render into `tiproxy_group_hpa`). Disabled mode uses `OriginalScaleDownPolicy(TIPROXY)=Disabled`, then `applyTiProxyScaleInGate` may flip ScaleDown to `MinChange` from the pool FeatureFlags flag. TIPROXY is in the `EnableHAPComponent` allowlist, so this is not gated out. The post-create apply-hpa write is the only path that produces a non-default TIPROXY HPA value this iteration on the create path — modify-time writes go through bundle RPC / `SetHPAScaleBehavior` (see Open Question 17).

**No-goal this iteration**:

- No SHARED_POOL bundle override tier — the bundle table stays DEFAULT/TENANT only; the SHARED_POOL tier is the live HPA.
- No `apply_to_all_shared_pools` fanout for pool-shared TIPROXY (no row to fan out to).

### Core Configuration Mapping Table

| Dimension | Component | Storage point | Write path |
|---|---|---|---|
| SCALE_OUT / SCALE_IN (NODE_COUNT, CPU) | TIDB (per-group) | K8s HPA `spec.behavior.Scale{Up,Down}.SelectPolicy` (per-group) | On cluster create: via workflow `ApplyComponentHpaAction`. On cluster modify: bundle RPC TIDB_GROUP → `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`. |
| SCALE_OUT / SCALE_IN (NODE_COUNT, CPU) | TIPROXY (per-group) | K8s HPA `spec.behavior.Scale{Up,Down}.SelectPolicy` (per-group) | On cluster create: via workflow `ApplyComponentHpaAction`. On cluster modify: bundle RPC TIDB_GROUP → `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior` (same as TIDB). See §Pool-shared TIPROXY. |
| SCALE_OUT / SCALE_IN (NODE_COUNT, CPU) | TIDB_WORKER / TIKV_WORKER / COPROCESSOR_WORKER (per-pool) | K8s HPA `spec.behavior.Scale{Up,Down}.SelectPolicy` (per-pool) | On sharedpool create: via workflow `ApplyComponentHpaAction`. On sharedpool modify: write directly to HPA behavior. |
| SCALE_OUT / SCALE_IN (NODE_COUNT, CPU) | TIPROXY (pool-shared, `HasSharedTiProxy`) | K8s HPA `spec.behavior.Scale{Up,Down}.SelectPolicy` on `tiproxygroup-<db-FnvHash(spID)>` | On sharedpool create: via workflow `ApplyComponentHpaAction` (DEFAULT+TENANT bundle → translation). On sharedpool modify: bundle RPC SHARED_POOL → write directly to HPA behavior. See §Pool-shared TIPROXY. |
| SCALE_OUT (NODE_COUNT, CPU) | TIKV (per-pool, non-HPA) | `ru_configs.FeatureFlags.enable_autoscale_tikv_cpu_scale_out` | `SetRuConfig(sharedPool, ...)` |
| SCALE_OUT (DISK_SIZE, DISK) | TIKV | `ru_configs.FeatureFlags.enable_autoscale_disk_expand` | Same as above |
| SCALE_OUT (IOPS / THROUGHPUT, IO) | TIKV | `ru_configs.FeatureFlags.enable_autoscale_iops_throughput_expand` (same flag covers both axes) | Same as above |
| SCALE / NODE_SPEC / RCU | SHARED_POOL (logical component; TIKV_WORKER / TIDB_WORKER / COPROCESSOR_WORKER / TIKV) | `ru_configs.FeatureFlags.enable_scale_shared_pool_by_ru` | Same as above |
| SCALE / NODE_SPEC / RCU | TIDB_GROUP (logical component) | `ru_configs.FeatureFlags.enable_scale_tidb_group_by_ru` | Same as above |

**Notes**:

1. Combinations outside this table must be rejected by the interface on entry (InvalidArgument).
2. SHARED_POOL / TIDB_GROUP are logical components introduced by the autoscale policy surface; the underlying parameters are only `enable_scale_shared_pool_by_ru` / `enable_scale_tidb_group_by_ru` in `ru_configs.FeatureFlags`. **These two virtual components are first-class entries in the mapping table — the unified configuration interface must accept and translate them**, same as any other row in this table.
3. **TIPROXY appears in two rows**: per-group (row 2) and pool-shared (row 4) — covering the two HPA topologies. See §Pool-shared TIPROXY. A pool may carry both — per-group reads/writes are TIDB_GROUP-scope only; the pool-shared HPA is SHARED_POOL-scope only. **Bundle RPCs accept TIPROXY at topology-correct scopes**: per-group TIPROXY at TIDB_GROUP scope (through `writeTidbGroupHPABehavior`), pool-shared TIPROXY at SHARED_POOL scope (through pool-scoped HPA write); cross-topology combinations are rejected naturally via label selector mismatch.
4. **TIPROXY SCALE_IN also has a FeatureFlags flag** (`enable_autoscale_tiproxy_scale_in`). That flag is the disabled-mode overlay onto HPA ScaleDown (create-time seed plus explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change sync so the flag can be turned off), not a parallel runtime store. Who wins when it disagrees with the bundle / YAML / an existing HPA is §Authority when configs conflict. Enforce-mode create still follows rows 2 and 4 (bundle → HPA).

### Inheritance Chains

1. `DEFAULT → TENANT → SHARED_POOL`:
   - RuConfig.featureFlag merge happens at `CreateSharedPool` time
   - HPA parameters are merged at write time via workflow `ApplyComponentHpaAction`. Enforce mode uses the three-tier bundle priority for per-pool components (TIKV_WORKER / TIDB_WORKER / COPROCESSOR_WORKER) plus, when `HasSharedTiProxy`, the pool-shared TIPROXY component (see §Pool-shared TIPROXY). Disabled mode uses `OriginalScale*` and the TIPROXY FeatureFlags overlay — §Authority when configs conflict.
2. `DEFAULT → TENANT → CLUSTER`:
   - CLUSTER has no bundle row and no `ru_configs` row. The only **create-time** write path is `ApplyComponentHpaAction` triggered by `CreateCluster`, writing per-group HPA (TIDB / per-group TIPROXY). Enforce mode uses the same three-tier bundle priority; disabled mode uses the baseline plus the TIPROXY overlay from the parent pool's FeatureFlags. For modify-time writes, TIDB and per-group TIPROXY both go through bundle RPC TIDB_GROUP → `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`. The pool-shared TIPROXY HPA is **not** addressed by per-cluster CLUSTER reads/writes — see §Pool-shared TIPROXY.

- The policy bundle table `autoscale_policy_bundle_specs` is **only written for DEFAULT / TENANT**; SHARED_POOL / CLUSTER never write to this table.
- Runtime reads of SHARED_POOL / CLUSTER policies go through `readRuntimeSpecs` which reads HPA `spec.behavior` + `ru_configs.FeatureFlags` directly, **without reading** the bundle table.

### CreateSharedPool — Freezes FeatureFlags

1. Read server YAML `config.RuConfig` (defaults for the 6 autoscale flags).
2. Call `PriorityResolveForSharedPoolFromBundle(bundleRepo, tenantID, server_default)` which resolves DEFAULT → TENANT → server_default per-field in one pass. The helper reads BOTH the DEFAULT bundle row `(scope='default', subject_id='__default__')` and the TENANT bundle row `(scope='tenant', subject_id=T)` itself and applies the three-tier rule per field: `if tenant.field != nil → tenant.field; else if default.field != nil → default.field; else → server_default.field` (proto3 `optional bool` keeps `nil` and `false` distinguishable). nil bundleRepo / empty tenantID falls back to server-default-only semantics (defense in depth).
3. Write the merged result to `sharedpool.ru_configs.FeatureFlags` — **frozen**. Subsequent `SetRuConfig(tenant, ...)` will not push to this sharedpool's FeatureFlags row; only TENANT-scope PATCH / SYNC with `apply_to_all_shared_pools=true` will fan out. Disabled mode additionally clears `enable_autoscale_tiproxy_scale_in` when it merely echoes YAML, so region YAML cannot freeze into every new pool (see §Authority when configs conflict).
4. After `CreateSharedPool` completes, the workflow triggers `ApplyComponentHpaAction` for each per-pool component (TIKV_WORKER / TIDB_WORKER / COPROCESSOR_WORKER, plus the pool-shared TIPROXY when `HasSharedTiProxy`). Enforce mode calls `ResolveHPABehaviorFromBundle(tenantId, compType)`; disabled mode uses the built-in per-component baseline without reading policy bundles, then for TIPROXY overlays `enable_autoscale_tiproxy_scale_in` (see §Authority when configs conflict). The result is rendered into the HPA template and written to K8s.

The older `PriorityResolveForSharedPool(tenantSpec, server_default)` overload is kept for unit tests / callers that already have the tenant spec in hand; it only sees the TENANT row and would miss DEFAULT-scope policies set via `CreateOrReplaceAutoscalePolicy(SCOPE_DEFAULT, ...)`. New code paths MUST use `PriorityResolveForSharedPoolFromBundle`.

### CreateCluster — No `ru_configs`, HPA only

1. `CreateCluster` does not write any `ru_configs.FeatureFlags` (TIDB_GROUP scope has no `ru_configs` row, and no `ru_configs` row is created).
2. The `ApplyComponentHpaAction` workflow computes the desired `Scale{Up,Down}.SelectPolicy` for each per-group TIDB / TIPROXY HPA. Enforce mode calls `ResolveHPABehaviorFromBundle(tenantId, compType)` using **TENANT bundle > DEFAULT bundle > `OriginalScale*` baseline**; disabled mode uses the baseline directly without reading policy bundles, then for TIPROXY overlays `enable_autoscale_tiproxy_scale_in` from the parent shared pool's FeatureFlags (see §Authority when configs conflict).
3. **First-create**: the desired values are written into `templArgs["ScaleUpPolicy"] / ["ScaleDownPolicy"]`, and the template renders the K8s manifest.
4. **Existing HPA**: goes through `reconcileExistingHPA`, which **only fills nil slots**. Values previously written by explicit `SetHPAScaleBehavior` / `writeTidbGroupHPABehavior` are not overwritten by unrelated reconciles. Turning the disabled-mode TIPROXY flag on or off for existing HPAs is handled by the explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change path.
5. After `CreateCluster`, calling `CreateOrReplaceAutoscalePolicy(SCOPE_TIDB_GROUP, ...)` writes per-group **TIDB** and **TIPROXY** HPA directly via `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`. See §Pool-shared TIPROXY. This does **not** go through bundle merge, and does not trigger a reconcile.

### HPA Behavior Assembly Rule

Which source `ApplyComponentHpaAction` reads is gated by `AutoscalePolicy.IsEnforced()`. The two modes do **not** agree, and the same operator action can produce opposite HPA results — see §Authority when configs conflict before reading this table.

- **Disabled** (`IsEnforced()==false`) and enforce+multi-tenant: do **not** call this helper. Use `OriginalScale*` as the baseline. For TIPROXY ScaleDown only, `applyTiProxyScaleInGate` overlays `enable_autoscale_tiproxy_scale_in` at create time / nil-slot fill; explicit `SetRuConfig(SHARED_POOL, ...)` flag-value changes sync existing TiProxy HPAs.
- **Enforce** (single-tenant): `ResolveHPABehaviorFromBundle(ctx, bundleRepo, tenantId, compType)` is called once per `ApplyComponentHpaAction` reconcile cycle, applying the following three-tier priority to decide `Scale{Up,Down}.SelectPolicy`. FeatureFlags are **not** read.

  1. **TENANT bundle spec** (`autoscale_policy_bundle_specs`, `scope='tenant', subject_id=T`): if a spec exists for the `(compType, direction, NODE_COUNT, CPU)` tuple, translate per the translation table below.
  2. **DEFAULT bundle spec** (`scope='default'`): same.
  3. **`OriginalScale{Up,Down}(compType)` baseline**: hardcoded per-component defaults.

#### Translation Table (bundle state → SelectPolicy)

| Direction | Spec state | Resulting SelectPolicy | Rationale |
|---|---|---|---|
| SCALE_OUT | ENABLED | `MaxChangePolicySelect` (baseline) | Baseline is already MaxChange; ENABLED is a no-op (preserved for spec symmetry) |
| SCALE_OUT | DISABLED | `DisabledPolicySelect` | New capability: disable HPA scale-out for this component |
| SCALE_IN | ENABLED | per-component baseline, **except when baseline=Disabled, flip to `MinChangePolicySelect`** | Preserve per-component scale-in aggressiveness (TIDB=MinChange, workers=MaxChange, TIPROXY: Disabled → MinChange) |
| SCALE_IN | DISABLED | `DisabledPolicySelect` | Disable HPA scale-in |
| (no spec at any level) | n/a | `OriginalScaleUpPolicy()` / `OriginalScaleDownPolicy(compType)` | Legacy behavior before this redesign: "no policy → baseline" |

The **Disabled→MinChange flip** is the only special case. It only triggers for SCALE_IN when baseline=Disabled (TIPROXY). If an operator wants to keep Disabled, they must **omit** the SCALE_IN spec — writing ENABLED would flip to MinChange. "Keep baseline" and "explicit Disabled" are indistinguishable at the bundle layer.

> **Behavior change — TIDB scale-in baseline (MaxChange → MinChange).** v0.33.1's `desiredScaleDownPolicy(TIDB)` returned `MaxChangePolicySelect`. The consolidated `resolve.OriginalScaleDownPolicy(TIDB)` baseline is `MinChangePolicySelect`, and both disabled and enforce modes use that baseline for a freshly-created TIDB HPA when no explicit policy overrides it. This is an intentional deviation from v0.33.1 so TiDB scales in conservatively and the creation path agrees with `SetHPAScaleBehavior`. Existing HPAs are unaffected — `reconcileExistingHPA` only fills a *nil* `ScaleDown.SelectPolicy`, so an already-set `Max` is preserved.

### POST (Replace) / PATCH / Fanout

#### Fanout Eligibility (scope × operation × `apply_to_all_shared_pools`)

`apply_to_all_shared_pools=true` is only honored for the following combinations; everything else is rejected with `InvalidArgument`:

| Scope | Operation | `apply_to_all=false` (default) | `apply_to_all=true` |
|---|---|---|---|
| DEFAULT | POST / PATCH / SYNC | Writes DEFAULT policy bundle; takes effect on future `CreateSharedPool` | **Rejected** (would bypass every tenant override) |
| TENANT | POST (REPLACE) | Writes TENANT policy bundle; takes effect on future `CreateSharedPool` | **Rejected** (REPLACE fanout would wipe N sharedpools' FeatureFlags in one call) |
| TENANT | PATCH (MERGE) | Writes TENANT policy bundle | **Supported** (MERGE fanout is safe: only changes caller-specified fields) |
| TENANT | SYNC | n/a — SYNC always fans out | **Supported** (`apply_to_all=false` is treated as `true` at TENANT scope) |
| SHARED_POOL / CLUSTER | POST / PATCH | Writes the single target row; `apply_to_all` field has no effect at this scope | Accepted (no-op at single-target scopes) |
| SHARED_POOL / CLUSTER | SYNC | **Rejected** (single target — no "targets" concept) | **Rejected** |
| SHARED_POOL + pool-shared TIPROXY | POST / PATCH | **Accepted** at SHARED_POOL scope (routes through pool-scoped HPA write) | Accepted (no-op at single-target scopes) |
| CLUSTER + TIPROXY | POST / PATCH | **Accepted** for per-group TIPROXY (routes through `writeTidbGroupHPABehavior`); pool-shared TIPROXY is naturally excluded by label mismatch | Accepted (no-op at single-target scopes) |

### Field Semantics (passing nil values)

"Passing nil values" refers to a proto3 `optional bool` field being nil (i.e., the field is unset).

- Passing nil is only accepted by TENANT + PATCH: **currently a no-op** (the key is preserved in the merged bundle, not deleted).
  > **TODO**: per-key deletion is not yet implemented. When the `DeleteAutoscalePolicy` / `ClearAutoscalePolicyBundle` RPC lands (see Open Question #14), passing `STATE_UNSPECIFIED` in PATCH should delete the key and let the effective value fall back to DEFAULT.
- In POST, omitting a field already expresses "follow default", so POST rejects nil values.
- GET returns the current view; nil values are displayed as-is. GET effective resolves to DEFAULT.
- Passing nil with `apply_to_all_shared_pools=true`: fans out the DEFAULT effective value.

### DEFAULT Scope Specifics

- DEFAULT is the bottom of the inheritance chain; fields **cannot be deleted** (PATCH can override; POST `specs: []` is a no-op, not a wipe).
- The initial DEFAULT row is written via DB migration.
- When DEFAULT has no value, the fallback rule is: only Workers and TIDB enable SCALE OUT; everything else DISABLE.

### Async Fanout

For future cluster growth, async fanout will need to be supported, but it is not in scope for this iteration.

### Regional Runtime Guard (bypass interface only, no runtime effect)

This is for regions (e.g., Singapore) where machine capacity is tight and operators sometimes need to restrict scale-in across all clusters for a period, to prevent releasing capacity that can't be reacquired. This iteration only marks the interface but does not implement runtime enforcement.

### Metrics

1. **Write path**
   - `autoscale_policy_write_total{scope, operation, apply_to_all, result}` + duration histogram

2. **Fanout**
   - `autoscale_policy_fanout_targets_total{tenant_id, operation, status}` + duration (sync in-process this iteration; Phase 2 async adds inflight gauge)

3. **Bundle state**
   - `autoscale_policy_bundle_version{scope, subject_id, component}` gauge

> Note: `EvaluateAutoscalePolicy` and the 3 action-layer callers are not part of this iteration's scope. Runtime Guard is bypass-only. Metrics cover write / fanout / bundle only.

---

## Part II: No Goal / Wrong Design

> **This section lists design points explicitly NOT in this design, and designs that were tried and rejected.** Any PR proposing these should be rejected with a reference to this section.

### No Goal — Not in This Iteration

| Design point | Status | Reason / Followup |
|---|---|---|
| Runtime Guard runtime enforcement (actual scale-in blocking) | **No goal** | This iteration implements schema + RPC only; no enforcement path is wired up |
| Async fanout (job table / worker pool / `GetAsyncFanoutStatus`) | **No goal** | Phase 2; current sync is in-process |
| Explicit `ClearAutoscalePolicyBundle` RPC | **No goal** | Current POST `specs: []` is a no-op; real wipe is Phase 2 |
| `request_id` 24h TTL idempotency dedup table | **No goal** | MVP only logs `request_id`, no dedup enforcement |
| Success-path audit log | **No goal** | Only error-path logs emitted today |
| `shared_pool.tenant_id` column | **No goal** | MVP uses `tidbClusterRepo.List` to find pools; empty sharedpools (no clusters) are missed |
| SYNC RPC at SHARED_POOL / CLUSTER | **No goal** | Single target has no fanout semantics; rejected at boundary |
| Per-pool detailed policy metrics (>20 series / pool) | **No goal** | Cardinality explosion risk; metrics belong in autoscaler-engine, not csng |
| Ops Portal UI | **No goal** | Owned by portal team separately |
| Changing threshold / limit / step / cooldown source | **No goal** | Continues to be read from config / HPA template / scaler config |
| `params_patch` runtime effect (finer fields like `max_disk_size_gb` / `cooldown`) | **No goal** | MVP supports boolean flags only; needs `autoscale_policy_params` table later |
| Automatic shortage detector triggering Runtime Guard | **No goal** | Phase 2 |

### Wrong Design — Rejected or Found Incorrect

> Designs below have been **rejected** or **discovered to be wrong**. Any PR introducing these patterns must be rejected.

#### 1. Eager Bidirectional Sync (rejected)

**Why wrong**: `SetRuConfig(sharedPool)` writes `FeatureFlags` → `syncHPAFromRuConfig` recomputes HPA `Behavior` → `SetHPAScaleBehavior` → `syncFeatureFlagsFromHPA` writes back to FeatureFlags. Non-invertible (TIPROXY ScaleUp = `shared OR tidb` is not splittable), and amplifies bug surface (multiple review rounds attempted to mitigate resulting issues before the design was removed entirely).

**Removed helpers**: `syncHPAFromRuConfig`, `syncFeatureFlagsFromHPA`, `hpaToFeatureFlagsPatch`, `applyHPAScaleBehavior`, `syncMaterializedPoliciesFromHPA`, `DesiredScale{Down,Up}PolicyFromRuConfig`, SHARED_POOL writers in `materializeDeltaToAutoscalePolicies`.

**Active contract**: HPA and `FeatureFlags` are independent storage points, not a bidirectional derived pair. The one remaining FeatureFlags→HPA write is the disabled-mode TIPROXY ScaleDown overlay (create-time seed plus explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change sync). See Part I §Authority when configs conflict.

#### 2. Writing `autoscale_policies` for SHARED_POOL / CLUSTER (rejected)

**Why wrong**: 4 production write sites eagerly wrote SHARED_POOL / CLUSTER rows, which duplicated the independent-storage model and was prone to staleness; 3 readers (Evaluate / GetResolved / List) read from this table, which duplicated the direct read via `readRuntimeSpecs`.

**Active contract**: `autoscale_policies` is **dormant** for SHARED_POOL / CLUSTER. Schema, GORM model, and repo are retained for backward compatibility only; production neither reads nor writes. `ApplyMaterializedAutoscalePolicies` RPC is deprecated and rejects all callers at the service boundary (`option deprecated = true`).

#### 3. SCALE_IN flag and HPA sharing one FeatureFlags bit (coupling bug)

**Why wrong**: `enable_scale_tidb_group_by_ru` / `enable_scale_shared_pool_by_ru` originally controlled RCU quota *and* were read by `DesiredScale{Down,Up}PolicyFromRuConfig` to drive HPA `Behavior`. Operators wanting "allow RCU quota changes but block HPA scale-in" could not express that intent.

**Fix**: A decoupling change separated RCU flags from HPA `Behavior`. SCALE_IN goes through HPA `Behavior` (CPU axis); RCU flags control spec adjustments only (CPU/memory spec change).

#### 4. Ambiguous `autoscale_policies` key on SCALE_IN (related to the coupling bug)

**Old model**: `(SCALE_IN, NODE_COUNT, CPU)` was a unique key that two writers collided on (`syncNonCPUFeatureFlagsToAutoscalePolicies` and `DeriveAutoscalePoliciesFromHPABehavior`); `MaterializedFrom` decided the winner but semantics were confused.

**Fix**: With `autoscale_policies` dormant for SHARED_POOL / CLUSTER, the issue disappears automatically.

#### 5. `EvaluateAutoscalePolicy` path (not active in this iteration)

> ⚠️ **Wrong design warning**: the path that wires `EvaluateAutoscalePolicy` into the 3 action-layer callers (`ScaleSharedPool` / `CreateTiKVCapacityReservation` / RCU cron) is **not implemented in this iteration**. Production runtime decisions are still driven by hardcoded `SetRuConfig` / `SetHPAScaleBehavior` writes, not by reason-labeled Evaluate calls.

The reason taxonomy (5 raw + 4 classified) is documented for future implementation but does not currently produce these reasons in production.

#### 6. Deprecated RPCs (retained for wire compatibility, but rejected at boundary)

- `ApplyMaterializedAutoscalePolicies`: deprecated, rejects all callers at the service boundary, `option deprecated = true`. Body is reduced to a reject; no DB call. **This RPC is intentionally retained as a wire-compatibility stub** — any external callers (including the global bootstrap `SyncResourcePoolAutoscalePolicies`) receive a clear error pointing to the correct replacement API. Migration of external callers is tracked in Open Question 9.
- `SetHPAScaleBehavior` (legacy): kept for backward compat. TIPROXY HPA writes now go through bundle RPCs at topology-correct scopes (per-group TIPROXY at CLUSTER, pool-shared TIPROXY at SHARED_POOL). **Not** the recommended SRE emergency-brake entry — use the policy RPC instead.

### Implementation Invariants (must hold)

> Violating any of these is a regression.

1. **Two independent storage writers, with one explicit overlay**: `ru_configs.FeatureFlags` is written by `SetRuConfig(sharedPool, ...)` and policy RPC SHARED_POOL writes; HPA `Behavior` is written by `SetHPAScaleBehavior` and `ApplyComponentHpaAction` reconcile. There is **no bidirectional derivation** (the removed `syncHPAFromRuConfig` / `syncFeatureFlagsFromHPA` pair). The one remaining FeatureFlags→HPA write is the disabled-mode TIPROXY ScaleDown overlay in §Authority when configs conflict — create-time seed plus explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change sync, never HPA→FeatureFlags.
2. **Storage-point exclusivity**: `autoscale_policies` is not read or written for SHARED_POOL / CLUSTER; the 4 original write sites are no-op; `ApplyMaterializedAutoscalePolicies` rejects.
3. **`readRuntimeSpecs` is the sole read path**: SHARED_POOL / CLUSTER runtime reads go through HPA + `FeatureFlags`, not `autoscale_policies`.
4. **DEFAULT / TENANT writes only land in the bundle table**; never in `ru_configs.FeatureFlags` (except via fanout `apply_to_all=true`) or `autoscale_policies`.
5. **Fanout matrix boundary enforcement**: the `(scope, operation, apply_to_all_shared_pools)` triple is validated at the RPC entry; on rejection, return `InvalidArgument`.
6. **`ApplyComponentHpaAction` source depends on `IsEnforced()`**: enforce (single-tenant) calls `ResolveHPABehaviorFromBundle` (DEFAULT+TENANT re-merge) and **does not read** `FeatureFlags`. Disabled mode (and enforce+multi-tenant) does **not** read the bundle; it uses `OriginalScale*` and, for TIPROXY ScaleDown only, overlays `enable_autoscale_tiproxy_scale_in` at create time / nil-slot fill. The same operator action therefore has opposite HPA results on the two sides of the switch — §Authority when configs conflict is the contract. An existing non-nil `SelectPolicy` wins during unrelated reconciles.
7. **Pool-shared TiProxy is SHARED_POOL scope only**: per-cluster CLUSTER reads/writes must not address the pool-shared TIPROXY HPA (its `group=db-<FnvHash(spID)>` label differs from any per-cluster `groupId`); SHARED_POOL-scope reads do address it. See §Pool-shared TIPROXY.
8. **TIPROXY bundle RPC entry is topology-aware**: per-group TIPROXY accepted at TIDB_GROUP scope (routes through `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior`), pool-shared TIPROXY accepted at SHARED_POOL scope (routes through pool-scoped HPA write path), cross-topology combinations rejected (naturally via label selector mismatch). Read paths (Get / GetEffective at CLUSTER / SHARED_POOL scope) continue to surface only the HPA rows matching their selector — an empty result on read is the expected "no policy in scope" fallback, not an error. See §Pool-shared TIPROXY.
9. **Reason taxonomy is stable**: 5 raw + 4 classified reason values must not be extended without updating runbook alerts and dashboards.

---

## Part III: Historical / Outdated

> This section contains designs that have been replaced or removed. Kept only for tracing earlier PRs. Active contract is in Part I.

### Earlier Iterations

#### Earlier write-via-gate attempt (two superseded PRs)

An earlier "write through gate" attempt, where `AutoscalePolicyService` write paths passed through `EvaluateAutoscalePolicy` before syncing to `SetRuConfig` / `SetHPAScaleBehavior`.

**Problems**: gating on the write path introduced unnecessary latency; `SetRuConfig`'s REPLACE semantics collided with policy PATCH / SYNC MERGE / REPLACE_ALL semantics; eager sync between HPA and FeatureFlags became an impossible triangle.

**Replaced by this design**: the new model has no policy gating on the write path. Enforce-mode `ApplyComponentHpaAction` merges the bundle at write time, not via FeatureFlags. Disabled-mode TIPROXY ScaleDown overlays `enable_autoscale_tiproxy_scale_in` at create time, and explicit `SetRuConfig(SHARED_POOL, ...)` flag-value changes sync existing TiProxy HPAs so the flag can be turned off — that is a one-way FeatureFlags→HPA sync of one slot, not the removed bidirectional write-path gate. See Part I §Authority when configs conflict.

### Eager Bidirectional Sync (removed)

#### Old model (previously active, now removed)

```
SetRuConfig(sharedPool, X)
  ├─ writes FeatureFlags
  └─ syncHPAFromRuConfig
       ├─ calls SetHPAScaleBehavior
       │   ├─ writes HPA Behavior
       │   └─ syncFeatureFlagsFromHPA (writes FeatureFlags back)
       └─ calls materializeDeltaToAutoscalePolicies (writes autoscale_policies)
```

#### Removed helpers

- `pkg/autoscale/resolve/resolve.go::DesiredScale{Down,Up}PolicyFromRuConfig`
- `syncHPAFromRuConfig`, `syncFeatureFlagsFromHPA`, `hpaToFeatureFlagsPatch`, `applyHPAScaleBehavior`
- `syncMaterializedPoliciesFromHPA`
- `materializeDeltaToAutoscalePolicies` (SHARED_POOL writers)
- `pkg/autoscale/hpa_behavior/hpa_behavior.go::DeriveAutoscalePoliciesFromHPABehavior` (SHARED_POOL callers no-op)

#### Replacement

- Enforce-mode `ApplyComponentHpaAction` calls `ResolveHPABehaviorFromBundle` (TENANT > DEFAULT > `OriginalScale*` baseline) to compute HPA `Behavior` and does **not** read `FeatureFlags`.
- Disabled-mode `ApplyComponentHpaAction` uses `OriginalScale*` without reading the bundle, then for TIPROXY ScaleDown overlays `enable_autoscale_tiproxy_scale_in` at create time / nil-slot fill. See Part I §Authority when configs conflict — that overlay is not this bidirectional sync.
- `reconcileExistingHPA` **only fills nil slots**. Explicit `SetHPAScaleBehavior` / `writeTidbGroupHPABehavior` values are preserved across unrelated reconciles.
- `ru_configs.FeatureFlags` and HPA `Behavior` are **independent writers**, not a derived pair, except for the create-time TIPROXY overlay plus explicit `SetRuConfig(SHARED_POOL, ...)` flag-value-change sync.

### Review rounds before the eager-sync removal (four iterations)

Four review iterations occurred against the eager bidirectional sync model before the model was removed entirely. Each iteration attempted to mitigate a specific class of bug; the model was ultimately removed because the bugs were symptoms of a deeper design issue (independent storage points were being cross-derived).

#### Round 1 — drop TIKV from HPA scope

**Issue**: `syncHPAFromRuConfig` iterated `ComponentType_TIKV`, but `SetHPAScaleBehavior`'s `switch` had no TIKV case. TIKV calls fell through to `default` and returned `Unimplemented`; the caller logged and swallowed. `syncNonCPUFeatureFlagsToAutoscalePolicies` deliberately omitted the TIKV CPU row, so `SetRuConfig(sharedPool, ...)` with `enable_autoscale_tikv_cpu_scale_out=true` returned success without changing either live HPA or `autoscale_policies`.

**Resolution**: TIKV CPU row is no longer HPA-derived. TIKV per-pool autoscaling is not HPA-based; `DeriveAutoscalePoliciesFromHPABehavior` emits 0 TIKV rows; `hpaComponents` excludes TIKV.

#### Round 2 — eager sync failures are part of the write contract

**Issue**: `reconcileExistingHPA` only wrote `Behavior` if currently nil; a transient dataplane failure during eager sync would leave `ru_configs` + `autoscale_policies` updated while HPA stayed stale.

**Resolution**: eager sync became a hard requirement of the write contract; `syncHPAFromRuConfig` and `syncNonCPUFeatureFlagsToAutoscalePolicies` return errors instead of swallowing.

> Note: this fix became moot once eager sync was removed entirely.

#### Round 3 — suppress HPA→FeatureFlags back-sync on RuConfig-originated HPA patches

**Issue**: `syncHPAFromRuConfig` called `SetHPAScaleBehavior`, which itself called `syncFeatureFlagsFromHPA` afterwards — forming a loop. TIPROXY ScaleUp = `shared OR tidb` is non-invertible, so `hpaToFeatureFlagsPatch` set both flags to true, mutating a deliberate value.

**Resolution**: `applyHPAScaleBehavior` (used by paths 1 and 2) skips `syncFeatureFlagsFromHPA`, parameterized by `skipFeatureFlagsBackSync`. TIPROXY ScaleUp is not inverted; TIPROXY ScaleDown still inverts a single flag (`enable_scale_shared_pool_by_ru`).

#### Round 4 — `SetHPAScaleBehavior` runs the full post-sync on every successful path

**Issue**: two success paths (TIDB scale-down retry `updateTiDBHPAWithRetry`, empty-ops `opsForScaleBehavior`) did not run the full materialize + FeatureFlags back-sync, violating the "two storage points + derived view in lockstep" invariant.

**Resolution**: shared `setHPAScaleBehaviorImpl` with a single "successful path tail".

> Note: the "lockstep" assumption was overturned by the SCALE_IN/HPA decoupling change and the active design; eager sync was removed entirely; Round 4's code changes are no-op along with the removed helpers.

### Async Fanout (Phase 2 design, kept for traceability)

#### Persistent job table schema (Phase 2 proposed)

```sql
autoscale_policy_async_fanout_jobs
- job_id                       varchar(64)   PRIMARY KEY   -- UUID
- request_id                   varchar(64)                  -- idempotency key
- tenant_id                    varchar(64)   NOT NULL
- trigger_operation            varchar(16)   NOT NULL       -- 'PATCH' / 'SYNC'
- specs_hash                   varchar(64)   NOT NULL
- specs_json                   jsonb         NOT NULL
- target_pool_ids              jsonb         NOT NULL       -- snapshot at enqueue time
- status                       varchar(16)   NOT NULL       -- 'PENDING' / 'RUNNING' / 'COMPLETED' / 'PARTIAL' / 'FAILED'
- per_target_status            jsonb         NOT NULL DEFAULT '{}'
- target_count                 int           NOT NULL
- success_count                int           NOT NULL DEFAULT 0
- failure_count                int           NOT NULL DEFAULT 0
- started_at                   timestamp     NULL
- completed_at                 timestamp     NULL
- expire_at                    timestamp     NOT NULL       -- TTL (default 7d)
- created_at                   timestamp     NOT NULL DEFAULT NOW()
- updated_at                   timestamp     NOT NULL DEFAULT NOW()

INDEX (status, expire_at)
INDEX (request_id)
```

#### Worker pool (Phase 2 design)

- in-process goroutine pool, default 8 workers
- each worker: atomically claim one PENDING job → process snapshotted `target_pool_ids` sequentially (per-pool apply is sync, no internal concurrency to avoid HPA / FeatureFlags races) → record per-target status → set terminal status
- cross-job parallelism; within-job sequential
- ctx cancel: leave unfinished targets at intermediate state, job status = PARTIAL

#### Polling RPC (Phase 2 proposed)

```proto
rpc GetAsyncFanoutStatus(GetAsyncFanoutStatusRequest) returns (GetAsyncFanoutStatusResponse);

enum AutoscalePolicyAsyncFanoutStatus {
  ASYNC_FANOUT_STATUS_UNSPECIFIED = 0;
  PENDING = 1;
  RUNNING = 2;
  COMPLETED = 3;
  PARTIAL = 4;
  FAILED = 5;
}
```

#### Pin tests required before merge

- `TestAsyncFanoutThresholdBoundary`: N=threshold → sync; N=threshold+1 → async
- `TestAsyncFanoutEnqueueIdempotent`: same `request_id` → same `job_id`, no duplicate job
- `TestAsyncFanoutPerTargetPartialFailure`: mixed → PARTIAL
- `TestAsyncFanoutPerTargetAllFailed`: all failed → FAILED
- `TestAsyncFanoutWorkerCtxCancel`: shutdown mid-job → PARTIAL, no lost progress
- `TestGetAsyncFanoutStatus_Unknown`: NotFound
- `TestGetAsyncFanoutStatus_Expired`: NotFound after TTL
- `TestAsyncFanoutHonorsFanoutEligibilityMatrix`: DEFAULT scope + apply_to_all=true never enqueues

### SCALE_IN flag / HPA confusion (background)

The old `autoscale_policies` schema used the single key `(SCALE_IN, NODE_COUNT, CPU)` for every component. Two mechanisms wrote to this key:

1. `syncNonCPUFeatureFlagsToAutoscalePolicies` derived the row from `FeatureFlags` flags, with `MaterializedFrom = SHARED_POOL_OVERRIDE`
2. `DeriveAutoscalePoliciesFromHPABehavior` derived the row from HPA `Behavior.ScaleDown.SelectPolicy`, with `MaterializedFrom = HPA_BEHAVIOR`

`SpecToFeatureFlags` mapped policy RPC SCALE_IN specs onto the FeatureFlags path; `resolve.DesiredScale{Down,Up}PolicyFromRuConfig` read the same flags to compute HPA `Behavior`. The bidirectional sync treated these flags as a master switch for "TIDB / worker auto-scale enable" — which was wrong, since the flags only controlled RCU quota.

The decoupling fix path (follow-up):

1. Drop `syncNonCPUFeatureFlagsToAutoscalePolicies` derivation of SCALE_IN rows
2. Drop policy RPC → `SpecToFeatureFlags` → flag mapping for TIDB / TIDB_WORKER / TIKV_WORKER / COPROCESSOR_WORKER SCALE_IN
3. Route SCALE_IN patches at the policy RPC handler through `SetHPAScaleBehavior` (spec proto needs a `group_id` field)
4. `autoscale_policies` SCALE_IN key stays `(SCALE_IN, NODE_COUNT, CPU)`; HPA semantics encoded in `MaterializedFrom`
5. `desiredDown = resolve.DesiredScaleDownPolicyFromRuConfig` for TIDB / workers returns `DisabledPolicySelect` (HPA-side default), not derived from RCU flags

> This background is here only for tracing the historical decision; the active contract is in Part I §Core Configuration Mapping Table.

### Open Questions (with current status)

| # | Open Question | Status |
|---|---|---|
| 1 | `params_patch` storage | **Not implemented** (no goal); needs `autoscale_policy_params` table |
| 2 | HPA behavior lag on `apply_to_all` | **Not applicable** (active design uses lazy reconcile + reconcile-only-if-nil) |
| 3 | Success-path audit log | **Not implemented** (no goal) |
| 4 | `request_id` idempotency token TTL | **Not implemented** (no goal) |
| 5 | `apply_to_all_shared_pools` escalation path (confirmation token / IAM) | **Not implemented** (no goal) |
| 6 | SYNC `KEYS_ONLY` key list size cap | KEYS_ONLY implemented; size cap not implemented |
| 7 | `shared_pool.tenant_id` column | **Not implemented** (no goal) |
| 8 | TIDB_GROUP scope | **Resolved**: all 6 RPCs accept CLUSTER; CLUSTER has no `ru_configs`, uses per-group HPA |
| 9 | `SyncResourcePoolAutoscalePolicies` (bootstrap, external service) | **Resolved**: `ApplyMaterializedAutoscalePolicies` (this repo) is a wire-compatibility stub that rejects all callers with a clear error message. The external bootstrap caller must be migrated to `CreateOrReplaceAutoscalePolicy(SCOPE_SHARED_POOL)` — tracked as a separate migration task outside this repository. |
| 10 | `AutoscalePolicyFieldDiff.field_path` semantics | Currently only `"state"`; expand sub-paths when `params_patch` lands |
| 11 | Canonical-vs-effective Preview | MVP returns `Unimplemented` when `current_effective != current` or `proposed_effective != proposed`; per-target effective preview deferred |
| 12 | Async fanout (csng-side, large N) | **Not implemented** (no goal, Phase 2) |
| 13 | Back-fill migration for pre-proto3-optional shared pools | **Not implemented** (no goal) |
| 14 | Explicit `ClearAutoscalePolicyBundle` RPC | **Not implemented** (no goal) |
| 15 | Alert routing review for `autoscale_policy_*` rules | Runbooks team follow-up |
| 16 | Tenant-level removal of a single override value | **Resolved**: `state=INHERIT` in PATCH (TENANT scope) |
| 17 | Modify-time policy write path for pool-shared TIPROXY | **Resolved**: bundle RPC SHARED_POOL accepts TIPROXY component specs and writes directly to HPA behavior (same as worker components at SHARED_POOL scope). Per-group TIPROXY at TIDB_GROUP scope routes through `writeTidbGroupHPABehavior` → `SetHPAScaleBehavior` (same as TIDB). |
| 18 | `readRuntimeSpecsForSharedPool` filter | **Resolved (this PR)**: per-group TIDB/TIPROXY are filtered out (CLUSTER-only); the pool-shared TIPROXY is kept when `HasSharedTiProxy(sharedPool)` and its HPA group label matches `db-<FnvHash(spID)>`. See §Pool-shared TIPROXY. |

### Reviewer Guide

#### PRs that change a write path (`CreateOrReplaceAutoscalePolicy` / `PatchAutoscalePolicy` / `SyncAutoscalePolicy` / `SetRuConfig(SHARED_POOL, ...)` / `SetHPAScaleBehavior` / `readRuntimeSpecs`) must answer:

1. Writing the correct storage point: FeatureFlags and HPA `Behavior` are independent writers, **no bidirectional derivation**. The disabled-mode TIPROXY ScaleDown overlay is the documented exception in §Authority when configs conflict — do not reintroduce `syncHPAFromRuConfig`.
2. **No** write to `autoscale_policies` SHARED_POOL / CLUSTER rows
3. `readRuntimeSpecs` is the sole SHARED_POOL / CLUSTER read path
4. DEFAULT / TENANT writes only land in the bundle table
5. `reconcile-only-if-nil` guard preserved for every slot. Turning disabled-mode TIPROXY ScaleDown on/off from `enable_autoscale_tiproxy_scale_in` must happen on explicit `SetRuConfig(SHARED_POOL, ...)` flag-value changes, not unrelated reconciles.

#### PRs that change the policy RPC surface (proto) must:

1. Update `api/cluster-service-ng/v1beta/services.proto` and `resources.proto` (separate go.mod); run `make proto` to regen; do not hand-edit generated files
2. Update the Source-of-Truth Model and Implementation Invariants sections (wire-level contract is the doc, not just proto)
3. Update `cluster-service-ng/AGENTS.md` §AUTOSCALE POLICY MODULE entry
4. Add a pin test in `domain/server/autoscale_policy_bundle_test.go`

#### PRs that change resolve / HPA behavior derivation (`pkg/autoscale/resolve/resolve.go`) must:

1. Keep per-component hardcoded defaults (`OriginalScaleUpPolicy` / `OriginalScaleDownPolicy`) in sync with template defaults (`{{!arg|default:"..."}}`) — fallback tier in Part I §HPA Behavior Assembly Rule
2. Enforce-mode HPA `Behavior` is computed from the bundle table, not from `sharedpool.ru_configs.FeatureFlags`. Disabled-mode TIPROXY ScaleDown overlays `enable_autoscale_tiproxy_scale_in` at create time / nil-slot fill and syncs existing HPAs only on explicit `SetRuConfig(SHARED_POOL, ...)` flag-value changes — if you change that overlay, the sync trigger, or `IsEnforced()` gating, update §Authority when configs conflict and invariant 6 in the same PR.

#### PRs that touch a metric in this module must (see Part I §Metrics invariants):

1. Cardinality bound: per-pool metric ≤ 20 series / pool
2. Delete-path coverage: `DeleteLabelValues` / `DeletePartialMatch` in pool cancellation path
3. Lifetime scope documented in `Help`: "reflects autoscaler-engine's CURRENT view"
4. Source-of-truth cross-reference: `GetResolvedAutoscalePolicyBundle` is canonical
5. No functionality changes
6. Pin test
7. Runbook pairing (`autoscale_tikv_policy_dimension_enabled` family)

---

## Document Metadata

- **Baseline**: rewritten from the original 5018-line doc
- **Part I source**: based on the Feishu wiki ([Regional - csng]多租户 Autoscale Policy 技术设计方案)
- **Part II**: explicit no-goal and wrong-design markers
- **Part III**: historical / outdated reference
- **Code paths**: `pkg/autoscale/resolve/resolve.go` · `domain/server/autoscale_policy_runtime_specs.go` · `domain/server/autoscale_policy_bundle.go` · `domain/workflow/action/apply_hpa.go` · `domain/server/autoscale_runtime_guard.go` (bypass)