# tipoc-tv Standard Testcases

This directory stores reusable pressure scenario definitions for the TPC-E
securities simulation. The files are input contracts for repeatable benchmark
runs and management reports; they are not evidence that a run passed.

## Case directories and issue/PR mapping

New testcase requirements are grouped into the following directories:

| Directory | Scope | Mapping index |
| --- | --- | --- |
| [`placement-rule/`](placement-rule/) | Placement Rule cases | [`placement-rule/README.md`](placement-rule/README.md) |
| [`resource-control/`](resource-control/) | Resource Control cases | [`resource-control/README.md`](resource-control/README.md) |
| [`resource-unit-v3/`](resource-unit-v3/) | Resource Unit V3 cases | [`resource-unit-v3/README.md`](resource-unit-v3/README.md) |

Each case entry must record its GitHub issue and PR in the mapping table in
the corresponding directory. Use one row per case and keep the issue as the
requirement owner and the PR as the implementation owner. If a case has no PR
yet, leave the PR column as `TBD`; do not create an unlinked case file.

The standard mapping fields are:

| Field | Meaning |
| --- | --- |
| Case ID | Stable testcase identifier; do not rename after reports reference it |
| Issue | Requirement or acceptance-tracking issue |
| PR | Implementation or documentation PR; `TBD` before creation |
| Status | `planned`, `in-progress`, `ready`, or `archived` |
| Report | Related PoC report path or `TBD` |

The current scenario set captures the measured single-node `50G` pressure work
and makes it reusable for later local or larger-environment runs. A formal pass
still requires a fresh four-stage `init -> prepare -> run -> cleanup` attempt,
zero blocked statements, `statement_qps >= 10000`, and attempt-root guard
success.

## Files

| File | Purpose |
| --- | --- |
| `essential-matching-mix-10case.json` | Essential scenario matrix with ten reusable data and workload combinations for the write-heavy exchange-matching simulation load, hotspot sensitivity, and anonymized point-get profile comparison. Standard TPC-E remains the master canonical workload and is not counted as an essential matching case. |
| `securities-big-query-interference-10case.json` | RU-containment matrix using the canonical Big Query operation, generic runtime overrides, and operation-owned receipts while securities foreground traffic remains active. |
| `internet-big-query-interference.json` | Independently selectable Ride, Food, Mart, domain-pair, all-domain, and explicit-subset business-report interference cases with statement RU/latency evidence contracts. |
| `internet-ru-acceptance-combination.json` | E2E-ready paired Internet RU matrix for Big Query, indexed/missing-index, service-area hotspot, and Big Query repeatability with an exact config-diff allowlist. |
| `internet-missing-index-interference.json` | Paired indexed/missing Ride, Food, and Mart business lookups with exact plan/work and cleanup-restoration evidence. |
| `internet-service-area-hotspot.json` | Ride, Food, Mart, all two-domain subsets, and all-domain service-area 20/80 hotspot cases with deterministic compiled authority and per-domain distribution receipts. |
| `pressure-50g-single-node.json` | Standard scenario set for the measured single-node `50G` pressure runs, including continuous ramp, restore-isolated samples, ceiling check, and hotspot stress. |
| `securities-local-10g-diff-gate.yaml` | Local `10G` four-stage diff-gate spec for low-cost runtime validation. It follows the same TiUP/local object-store path as the local script family and is not pass evidence by itself. |
| `securities-tpce-read-only.yaml` | First-class `50G` securities read-service spec covering customer portal, trader workbench, and broker reporting traffic without mutating the foreground TPC-E state. |
| `tidb-x-premium-migration-poc.yaml` | TiDB X Premium migration POC case-manifest template. It declares preparation, required, and optional POC cases as product-supported, partial-product, manual-required, external-required, or not-supported without adding a fifth public stage. |

## Scenario Families

| Family | Use | Restore Policy | Pass Interpretation |
| --- | --- | --- | --- |
| `canonical_continuous_ramp` | Shows how throughput and latency evolve as concurrency rises without resetting state between steps. | Restore once before the group. | Useful trend evidence, but not restore-isolated capacity evidence. |
| `canonical_restore_isolated` | Compares individual concurrency levels from the same prepare-end BR baseline. | Restore before each step. | Intended basis for formal capacity curves once warmup and isolation receipts are implemented. |
| `canonical_ceiling_check` | Explores the current single-node ceiling above the initial ramp. | Restore once before the group in the measured sample set. | Identifies the best measured operating point and latency knee. |
| `hotspot_burst` | Stresses a declared hotspot workload profile. | Restore before the hotspot run. | Expected to expose contention; it is not a canonical pass sample. |
| `exchange_matching_low_concurrency` | Establishes the first low-concurrency exchange-matching baseline before the standard baseline row. | Restore before each essential case. | Compares business mix shape and low-pressure latency only. |
| `exchange_matching_baseline` | Measures the write-heavy exchange-matching mix at low concurrency before capacity tests. | Restore before each essential case. | Compares business mix shape, but does not reproduce protocol-level prepared-statement overhead. |
| `exchange_matching_medium_concurrency` | Raises exchange-matching pressure on the small dataset before the c512 capacity row. | Restore before each essential case. | Shows the moderate-concurrency latency and KV command transition. |
| `exchange_matching_capacity` | Raises exchange-matching concurrency to identify statement QPS, Select p999, Commit p999, and lock pressure limits. | Restore before each essential case. | Capacity evidence only after fresh attempt-root closeout and metric collection. |
| `hotspot_sensitivity` | Uses a declared hotspot profile to test contention and distribution skew inside the exchange-matching workload. | Restore before each essential case. | Expected to expose hotspot contention; it is not a canonical pass sample. |
| `point_get_hotset` | Uses a concentrated hotset to approximate point-get-heavy short-transaction behavior. | Restore before each essential case. | Best current point-read path check; protocol-level `general` parity remains out of scope. |
| `exchange_matching_scale_baseline` | Repeats the exchange-matching baseline shape at a larger dataset size. | Restore before each essential case. | Confirms that the baseline shape survives target-size scaling before higher pressure. |
| `exchange_matching_scale` | Raises exchange-matching pressure on the larger dataset. | Restore before each essential case. | Capacity and lock-pressure evidence only after fresh attempt-root closeout and metric collection. |
| `anonymous_profile_fit` | Uses supported workload-profile hotspot parameters to approximate an anonymized point-get short-transaction signal. | Restore before each essential case. | Best current executable fit; exact `general` parity requires a protocol-mode capability that the current pressure spec does not expose. |
| `anonymous_profile_fit_scale` | Repeats the anonymized profile fit on a larger dataset and driver pool. | Restore before each essential case. | Scale-fit evidence only; it must still report residual profile distance and protocol gaps. |

The anonymous profile ratio targets in `essential-matching-mix-10case.json`
intentionally total `0.91`. The remaining `0.09` is an anonymized residual for
SQL categories outside the disclosed Select, general, Update, Insert, Begin,
and Commit buckets. It must not be redistributed into the disclosed buckets or
expanded with private tenant details.

## Report Requirements

Every pressure testcase run must report:

1. target size and dataset binding
2. catalog table count, imported table count, and empty prepare table summary
3. source object count and CSV source bytes
4. restore policy and BR baseline receipt when applicable
5. warmup duration and measurement duration
6. concurrency, pressure worker count, and SQL pool size
7. executed statement count and measured statement QPS
8. blocked/error count and root cause summary
9. TiDB p99 latency and TiDB QPS
10. TiKV gRPC rate, write stall, and CPU where available
11. PD abnormal region or leader-balance symptoms where available
12. disk free-space range and network symptoms where available
13. workload profile and observed distribution summary
14. SQL type QPS and p99/p999 latency for Select, general, Update, Insert, Begin, and Commit where the monitoring surface exposes them
15. protocol command rates for StmtPrepare, StmtExecute, StmtClose, and Query, plus plan-cache hit and miss rates
16. TiKV command mix for get, batch_get, lock_keys, commit, and rollback
17. Distsql coprocessor p99 so point-get-like cases can prove whether they stayed on the point-read path
18. pressure-node CPU, memory, and network samples when runs use external pressure nodes

## Reuse Rules

Larger datasets and environment profiles reuse scenario ids, not measured
values. For `essential`, `premium`, or any other profile, bind a new target
size, machine topology, source object-store location, BR baseline, user metadata
path, CSV object target, and concurrency gradient. Do not ask users to provide
TiDB Global Sort internal URIs in the testcase; the import executor derives and
manages that path during `IMPORT INTO`. Reports may record the actual execution
evidence path used by TiDB, but that path is not a user input parameter. Do not
start a larger run from this file alone; open the matching task card and
evidence directory first.

Executable local specs such as `securities-local-10g-diff-gate.yaml` are
reusable diff-gate inputs rather than pass evidence. They should preserve the
real product chain and bounded machine assumptions, but they do not replace the
formal distributed `D10` target environment.

The YAML `pressure` section configures the public `tipoc run` stage. It is not
a fifth public stage and does not change the product front door.

## TiDB X Premium Migration POC Template

`tidb-x-premium-migration-poc.yaml` is a reusable case-declaration template for
the TiDB X Premium migration POC. It keeps the executable `tipoc` subset inside
the existing `init -> prepare -> run -> cleanup` flow and lets the compiled
manifest and run report show which POC cases are product-owned, partial, manual,
external, or unsupported.

The currently product-feasible required cases are:

1. `required-02-snapshot-import-index`, for the generated TPC-E source,
   TiDB `IMPORT INTO`, row readiness, and prepare-end ADD INDEX subset.
2. `required-03-statistics`, for statistics collection and validation evidence.
3. `required-04-schema-change`, for the frozen prepare-end ADD INDEX subset and
   declared action-reporting surface. Generic DDL execution remains a future
   action-harness capability.
4. `required-05-normal-load-stability`, for canonical TPC-E mixed read/write
   pressure and semantic proof.
5. `required-06-hotspot-observability`, for hotspot workload-profile evidence
   and reported run metrics, while alert timeliness and root-cause tracing
   remain external.
6. `required-07-ceiling-search`, for concurrency step evidence, QPS, latency,
   blocked statements, and imported bottleneck observations.

The template deliberately marks billing renewal, cloud scale control, failure
injection, continuous backup/restore, DM or migration catch-up, cutover, CDC,
archive/delete, TiFlash, audit, million database/table, and wide-table cases as
`manual_required`, `external_required`, or `not_supported`. TTL is marked
`partial_product` because `tipoc` can declare and plan TiDB TTL DDL, while
background TTL job observation, expired-row deletion evidence, and false-delete
validation remain external evidence. Those labels are part of the expected
product truth; they prevent a report from presenting external platform work as
a `tipoc` pass.

Every declared case includes an evidence requirement and a pass/fail gate. For
manual, external, or unsupported cases, the evidence requirement names the
manual packet, external packet, or gap record that must be attached before a POC
readout can treat the case as addressed.