# Roadmap Milestone Workload Bundles

Date: 2026-03-22
Status: Draft (Normative for roadmap economics and throughput gates)
Owner: Governance / Testing / Query Gateway / Query Broker

## 1. Scope

This document defines the workload-bundle format, execution rules, and evidence requirements used to close roadmap milestone-level economics gates and any roadmap checkpoint that explicitly reuses the same workload-bundle model.

Normative usage:
- roadmap delivery-milestone economics gates and the `2026-08-31` TiDB private-preview checkpoint economics gate MUST use the rules in this document;
- compatibility-profile economics gates that defer to roadmap workload bundles inherit the same rules;
- the `2026-08-31` platform-readiness checkpoint uses the specialized closeout package defined in Section `5.4` rather than a normal delivery-milestone economics bundle;
- the `2026-09-15` external-GA baseline prerequisite checkpoint uses the specialized prerequisite package defined in Section `5.5A` rather than a normal delivery-milestone economics bundle;
- the `2026-09-30` scalability checkpoint uses the specialized checkpoint-evidence package defined in Section `5.5` rather than a normal delivery-milestone economics bundle;
- the `2026-10-15` evidence-freeze checkpoint uses the specialized candidate-freeze package defined in Section `5.5B` rather than a normal delivery-milestone economics bundle;
- the October single-collection validation package inside the `2026-10-31` GA milestone uses the specialized package defined in Section `5.7` rather than the normal multi-tenant workload bundle;
- the `2026-12-15` Phase-4A foundation checkpoint uses the specialized foundation package defined in Section `5.7A`;
- the `2027-01-31` internal engineering qualification checkpoint uses the specialized package defined in Section `5.8`;
- the `2027-01-31` post-GA expansion checkpoint uses the specialized package defined in Section `5.9`;
- the `2027-02-28` external certification checkpoint uses the specialized certification-evidence package defined in Section `5.10` rather than a normal delivery-milestone economics bundle;
- this document governs workload construction, warm-up, sample windows, and evidence publication only;
- it does not redefine compatibility conformance suites or search-quality judging rules in `docs/testing/search-quality-evaluation.md`.

## 2. Bundle Format

Every milestone-closing workload bundle MUST publish one immutable manifest containing at least:
- `bundle_id`
- target roadmap milestone and gate names
- environment identifier and topology summary
- service revisions for Query Gateway, Query Broker, CCS, ingest path, and any phase-critical background-control services
- dataset / catalog snapshot identifiers
- tenant-mix definition
- query/write mix definition
- cache and warm-up policy
- sample-window definition
- enabled background workloads
- `profile_attribution` for every profile-scoped economics gate closed by the bundle, including workload-class ownership and metric-subset ownership
- optional `comparison_group_id` when the milestone gate requires enabled/disabled comparative evidence
- result-artifact manifest

The bundle manifest MUST be replayable. Re-running the same bundle against the same environment and data snapshot MUST produce the same declared traffic mix and the same measurement windows.

If a milestone-closing bundle is used to close profile-scoped economics inside a shared milestone, the manifest MUST make that attribution auditable without external interpretation.

### 2.1 Accepted Closeout Record

Every milestone or checkpoint closeout that depends on this document MUST publish one immutable accepted closeout record containing at least:
- roadmap milestone/checkpoint identifier
- accepted workload `bundle_id` set when workload bundles are part of the closure
- accepted benchmark-bundle version(s) when search-quality evidence is part of the closure
- accepted `comparison_group_id` values when comparative evidence is part of the closure
- accepted specialized checkpoint-package identifier(s) when closure also uses non-bundle evidence packages
- accepted frozen owner-document revision identifiers when the roadmap gate includes frozen architecture/spec ownership
- superseded run/package identifiers that were reviewed but not accepted for closeout

The accepted closeout record is the authoritative answer to “which evidence actually closed this gate”.

## 3. Execution Rules

### 3.1 Run Structure

Every milestone-closing run MUST contain these stages in order:
- environment validation
- warm-up
- steady-state measurement
- artifact export

Metrics used for milestone closure MUST come from the steady-state measurement stage only.

### 3.2 Warm-Up and Cache Policy

Warm-up policy MUST be explicit in the bundle manifest.

Minimum requirements:
- warm-up traffic MUST use the same declared workload mix as the measured run, unless the bundle explicitly declares a narrower bootstrap mix for cache priming;
- cache state at the start of the measured window MUST be declared as one of:
  - `cold`
  - `warm`
  - `restarted-but-replayed`
- if a milestone gate depends on cache-sensitive resolution or admission behavior, the measured window MUST begin only after the relevant caches and background readers reach the declared steady-state condition;
- if both cold and warm behavior are important, the bundle MUST publish both, but milestone closure uses the declared steady-state policy only.

### 3.3 Sample Windows

Each milestone-closing run MUST publish at least three non-overlapping steady-state sample windows.

Minimum sample-window rules:
- each window MUST be at least `10 minutes`;
- the total measured steady-state duration MUST be at least `30 minutes`;
- all reported percentile, throughput, and normalized-cost metrics MUST be computed both per-window and for the aggregate;
- a milestone gate fails if any required metric passes only in the aggregate while one or more declared steady-state windows violate the same threshold.

### 3.4 Background Work

If the milestone being closed claims behavior under compaction, GC, predictive scheduling, or admission control, the bundle MUST explicitly state whether those background systems are:
- disabled
- enabled but idle
- enabled and actively loaded

Milestone closure is valid only when the bundle state matches the milestone claim.

### 3.5 Comparative A/B Evidence

If a milestone gate requires proving regression or improvement versus a disabled baseline, the evidence set MUST publish a comparison group rather than one standalone bundle run.

Minimum comparison-group rules:
- the enabled and disabled runs MUST share the same dataset / catalog snapshot identifiers, tenant mix, query/write mix, cache policy class, and sample-window rules;
- the only intentional behavioral delta between the paired runs MUST be the declared control under test;
- the comparison artifact set MUST publish both per-run metrics and the computed delta used by the roadmap gate;
- if the paired runs cannot share the same environment revision, the evidence package MUST explain the unavoidable delta and why the comparison remains admissible.

### 3.6 Evidence Freshness and Rerun Rules

Accepted evidence from this document becomes stale for a given gate when any material behavior that can affect that gate changes after the accepted run and before milestone/checkpoint close.

Minimum invalidation triggers:
- admission-policy or quota-policy changes that alter effective throttling or fairness behavior
- routing-policy changes that alter request placement or the active background-work state
- predictive-scheduling / compaction / GC policy changes that alter the measured workload state
- any other roadmap-declared economics behavior change that would have made the accepted run a materially different experiment

When any invalidation trigger occurs, the affected gate MUST publish fresh admissible evidence and update the accepted closeout record rather than silently reusing the older run.

### 3.7 Cross-Checkpoint Reuse Matrix

| Artifact Family | Admissible Reuse | Mandatory Freshness Boundary | Explicit Non-Substitution Rule |
|---|---|---|---|
| milestone-closing workload bundle | later audit context or same-family comparison only | any Section `3.6` invalidation trigger | does not replace specialized checkpoint packages or benchmark bundles |
| benchmark bundle cross-referenced by this document | may be reused only while `docs/testing/search-quality-evaluation.md` allows it | any retrieval-affecting change that requires a fresh benchmark bundle | does not replace workload bundles or checkpoint packages |
| accepted August precheck artifact set | only the `2026-09-30` scalability checkpoint close and later audit context | any harness/instrumentation/ramp-shape change that makes the staged-ramp evidence materially different | does not replace the `2026-09-15` prerequisite package or the October single-collection package |
| `2026-09-15` prerequisite package | October dual-verdict audit context only when explicitly named by the closeout record | any material production-CDC, RCU-foundation, or formal-verification-baseline change before October close | does not replace the `2026-09-30` scalability package or October workload/package families |
| `2026-09-30` scalability package | October cardinality-preparedness audit context only when explicitly named by the closeout record | any material checkpoint-environment or threshold-affecting change before October close | does not replace the October multi-tenant workload bundle or October single-collection package |
| `2026-10-15` evidence-freeze package | `2026-10-31` verdict preparation and audit context only when explicitly named by the closeout record | any invalidation-trigger or defect-driven rerun that supersedes one of the frozen candidate artifact families | does not itself close the October dual verdict |
| `2026-12-15` Phase-4A foundation package | `2027-01-31` qualification entry and evidence reuse only | any claim-matrix, simulation-foundation, or primitive-readiness mutation after acceptance is prohibited unless the checkpoint is formally re-opened | does not replace the `2027-01-31` internal qualification package |
| `2027-01-31` internal qualification package | `2027-02-28` certification entry and evidence reuse only | no threshold/claim/package mutation is allowed after acceptance; certification may only reference the frozen set | cannot be redefined, relaxed, or superseded by the same-date Phase-4B expansion package |

## 4. Common Evidence Artifacts

Every milestone-closing workload bundle MUST publish:
- immutable bundle manifest
- raw request/response telemetry or equivalent aggregated trace export
- per-window metrics tables
- environment summary
- service configuration summary
- failure / retry summary
- cache and warm-up summary
- background-work summary
- normalized-cost export containing Query Gateway `RCU_estimated` and Query Broker `object-store touches`

The artifact manifest MUST name every file used for milestone closure.

## 5. Phase-Specific Bundle Requirements

### 5.1 Phase-1 Engineering-Validation Bundle (`2026-06-30`)

Required workload classes:
- federated write (`Remote DML` -> `wait_for_visible`)
- federated / primary read
- projected read for TiDB planner/executor integration validation
- metadata lookup and statistics reads

Required environment properties:
- single declared engineering-validation tenant set is acceptable;
- static connection/quota rails MAY be active;
- RCU token-bucket enforcement MUST NOT be required for bundle validity.

Required closure notes:
- the bundle MUST state whether semantic and hybrid retrieval runs share the same environment as the federated SQL runs or are published as a companion artifact set;
- the bundle MUST state the readiness-lane policy used for immediate read-after-write validation.
- if the bundle closes profile-scoped economics for MySQL or Elasticsearch in the same milestone, the manifest MUST publish the profile-attribution mapping used for those gates.

### 5.2 Phase-2 Public-Preview Bundle (`2026-07-31`)

Required workload classes:
- MySQL Profile-2 query mix
- Elasticsearch Profile-2 query mix
- preview-scope write traffic for delivered Phase-1/2 compatibility routes

Required tenant shape:
- at least one preview tenant cohort with declared steady-state traffic;
- profile-scoped economics closure for MySQL and Elasticsearch MUST identify the metric subset attributable to each profile's declared query mix.

Required closure notes:
- the accepted closeout record MUST name the accepted workload bundle for the public-preview close;
- the bundle MUST identify the minimum viable CCS-backed write-capacity authority state and the preview-grade ingest-hardening write routes exercised by the run.

### 5.3 TiDB Private-Preview Bundle (`2026-08-31`)

Required workload classes:
- TiDB projected / federated query mix
- flexible-filtering-index preview routes

Required notes:
- shadow metering is admissible for normalized-cost reporting in this checkpoint;
- the bundle MUST remain auditable as a checkpoint-local TiDB private-preview package rather than as part of the July public-preview milestone closure;
- the accepted closeout record MUST identify the accepted TiDB private-preview bundle and any superseded reruns.

### 5.4 Platform-Readiness Closeout Package (`2026-08-31`)

The same-date `2026-08-31` platform-readiness checkpoint is not a normal economics bundle. It MUST publish one specialized closeout package that binds the accepted readiness evidence together.

Required evidence:
- immutable checkpoint manifest with one package identifier
- accepted frozen owner-document revision identifiers for Query Gateway, Background Governance, and CDC Adapter
- accepted CDC preview-readiness package identifier
- accepted TiDB DDL/schema-sync readiness package identifier
- accepted background-governance-readiness package identifier
- accepted query/background CCS quota-authority readiness package identifier
- accepted RCU design-freeze evidence identifier
- accepted formal-verification-preparation evidence identifier
- accepted closeout record cross-reference naming any same-date TiDB private-preview bundle that was reviewed separately

Required checkpoint rules:
- this checkpoint package does not replace the TiDB private-preview workload bundle for TiDB economics closure;
- it exists to prove that the same-date platform prerequisites were reviewed and accepted as one coherent checkpoint close, rather than as disconnected sub-approvals.

### 5.5 Scalability Checkpoint Evidence Package (`2026-09-30`)

The `2026-09-30` checkpoint is a scalability checkpoint rather than a delivery-milestone economics gate. It MUST publish a specialized checkpoint evidence package focused on cardinality/ramp closure and MUST NOT be treated as an implicit economics bundle.

Required evidence:
- immutable checkpoint manifest with ramp-stage identifiers, final checkpoint environment summary, referenced harness/instrumentation revisions, and the accepted August precheck artifact-set identifier;
- staged-ramp summary covering the deferred smaller-scale prefix and descriptor runs, including adjacent-stage degradation calculations used by the roadmap checkpoint gate;
- final checkpoint metrics export for:
  - `GetCollectionMetadata` latency,
  - CCS memory / heap watermark,
  - `CollectionNameBinding` resolution-cache hit ratio,
  - post-warm unknown-name / negative-cache S3 penetration;
- raw observability artifacts needed to audit the declared thresholds, including cache, watermark, and request-path exports;
- explicit evidence-separation note showing how this checkpoint package differs from the later October single-collection validation package.

Required checkpoint rules:
- gateway-resolution caching and CCS watermark instrumentation MUST be enabled in the checkpoint environment because they are normative prerequisites in the roadmap gate;
- the package MUST include both staged-ramp and final-checkpoint evidence; a lone final `10^8` run is insufficient;
- this checkpoint package does not close delivery-milestone economics or search-quality gates unless a later delivery milestone explicitly reuses the same artifacts under its own rules.

### 5.5A External-GA Baseline Prerequisite Package (`2026-09-15`)

The `2026-09-15` checkpoint is a prerequisite package rather than a delivery-milestone economics bundle. It MUST publish one specialized package focused on production-CDC, RCU-foundation, and executed formal-verification-baseline closure before October hardening begins.

Required evidence:
- immutable checkpoint manifest with one prerequisite-package identifier;
- accepted production-grade external-truth CDC path evidence identifier;
- accepted RCU-foundation evidence identifier covering distributed token buckets, whale shielding, compaction/GC admission budgets, and the admission observability baseline;
- accepted executed-formal-verification-baseline evidence identifiers for the already-prepared stateless model-checking and property-based fault-testing campaigns;
- any reused `2026-08-31` platform-readiness sub-evidence identifiers that the checkpoint consumes;
- explicit reuse/separation note showing how this package differs from both the later `2026-09-30` scalability package and the October dual-verdict artifact families.

Required checkpoint rules:
- this package MAY be reused by the `2026-10-31` closeout only as an explicitly named prerequisite package in the accepted verdict record;
- it does not replace the `2026-09-30` scalability package;
- it does not replace the Phase-3 multi-tenant workload bundle, the October single-collection validation package, or the benchmark bundle required by the October external-GA minimum bar.

### 5.5B Phase-3 Evidence-Freeze Package (`2026-10-15`)

The `2026-10-15` checkpoint is a candidate-freeze package rather than a delivery-milestone economics bundle. It MUST publish one specialized package that freezes the candidate October artifact families early enough to preserve a real rerun buffer before the end-of-October closeout.

Required evidence:
- immutable checkpoint manifest with one evidence-freeze-package identifier;
- accepted candidate Phase-3 multi-tenant workload bundle identifier;
- accepted candidate benchmark-bundle identifier;
- accepted candidate October single-collection validation-package identifier;
- any accepted candidate predictive-scheduling comparison-group identifiers required by the later same-date platform-qualification verdict;
- explicit rerun rule note showing that only invalidation-triggered or defect-driven reruns may supersede the frozen candidates before `2026-10-31`.

Required checkpoint rules:
- this package does not itself close the `2026-10-31` external-GA minimum-bar or platform-qualification verdicts;
- it exists to freeze the candidate artifact set and preserve the remaining window for reruns, verdict preparation, and qualification hardening rather than for first-time evidence generation;
- the final October accepted closeout records MUST state whether each frozen candidate artifact was reused as-is or superseded by an admissible rerun.

### 5.6 Phase-3 Multi-Tenant GA-Readiness Bundle (`2026-10-31`)

Required workload classes:
- metadata search
- semantic search
- hybrid search
- profile-delivered compatibility traffic that affects retrieval behavior
- background-governance load with compaction / GC / predictive scheduling enabled when those systems are part of the claimed milestone closure

Required tenant shape:
- at least three quota tiers;
- at least one large-tenant cohort and one long-tail cohort;
- at least one noisy-neighbor stress slice where similarly provisioned tenants compete in the same sample window.

Required closure notes:
- the bundle MUST publish the admitted-throughput skew calculation used for the roadmap fairness gate;
- any retrieval-affecting route delivered in the same milestone MUST identify whether it is included in the search-quality benchmark bundle required by `docs/testing/search-quality-evaluation.md`.
- if the milestone closes profile-scoped economics for multiple profile lines, the manifest MUST publish the profile-attribution mapping used for each closed profile gate.
- if the milestone closes predictive-scheduling evidence, the evidence package MUST publish a paired enabled/disabled comparison group that satisfies Section `3.5`.
- the accepted closeout record for the milestone MUST name this multi-tenant bundle separately from the October single-collection validation package.

### 5.7 Phase-3 Single-Collection Validation Package (`2026-10-31`)

The October single-collection validation gate is distinct from the Phase-3 multi-tenant workload bundle and MUST publish its own specialized validation package.

Required evidence:
- immutable package manifest with one validation-package identifier
- declared environment/topology summary and dataset/storage-layout summary
- warm-up policy and steady-state measurement policy for the single-collection run
- raw telemetry and metrics exports for the declared single-collection throughput/latency thresholds
- explicit volume declaration for row count, metadata volume, and vector volume exercised by the run
- failure / retry / throttle summary for the validation run
- explicit evidence-separation note showing how this package differs from the milestone-closing multi-tenant workload bundle

Required checkpoint rules:
- the accepted closeout record for the `2026-10-31` milestone MUST name both the accepted multi-tenant workload bundle and the accepted October single-collection validation package;
- the single-collection package does not replace the multi-tenant workload bundle for economics/fairness closure.

### 5.7A Phase-4A Foundation Package (`2026-12-15`)

The `2026-12-15` checkpoint is a Phase-4A foundation checkpoint rather than the final qualification package. It MUST publish one specialized package that freezes the certification-critical qualification foundations before January execution begins.

Required evidence:
- immutable foundation manifest with one foundation-package identifier;
- accepted Phase-4A claim-matrix baseline artifact listing every certification-feeding claim and the evidence family expected at `2027-01-31`;
- accepted executable reference-model / deterministic fault-matrix / scenario-matrix artifact identifiers;
- accepted certification-critical primitive-readiness artifacts for control-plane sharding, storage-efficiency baseline, capacity-model structure, and qualification topology/environment definitions;
- explicit consumed-later map showing which artifacts January may reuse directly and which evidence families still require fresh January execution.

Required checkpoint rules:
- this package freezes the claim matrix and qualification-package shape before January execution starts;
- this package does not replace the final `2027-01-31` internal engineering qualification package;
- January closeout MUST explicitly state which accepted foundation artifacts it reused and which fresh January artifacts it added.

### 5.8 Phase-4A Internal Engineering Qualification Package (`2027-01-31`)

Required workload classes:
- the internal qualification scenarios required by `docs/governance/roadmap.md`
- shape-preserving read/write/background mixes for the accepted scale points

Required evidence:
- immutable qualification manifest with one qualification-package identifier;
- accepted frozen `2026-12-15` foundation-package identifier plus the claim-matrix baseline it froze;
- scenario summary for every accepted scale point, including environment/topology, workload/data shape, and whether the point contributes to mandatory Scenario A, mandatory Scenario B, or an intermediate trend point;
- raw metrics/artifacts for the mandatory Scenario A/B runs and any intermediate scale points used for trend-line, forecast-error, or confidence-bound calculation;
- failure/operations evidence exports for the soak, chaos, replay, backpressure, and recovery campaigns used by the checkpoint;
- extrapolation artifact set naming measured inputs, derating factors, confidence bounds, slope-bend checks, and any rejection note used during checkpoint close;
- reused-versus-new artifact map whenever the package references earlier phase evidence or shared benchmark/workload material.

Required checkpoint rules:
- the package MUST publish the accepted scale points used for forecast-error and confidence-interval calculation;
- the package MUST identify any derated acceptance note used to tolerate adjacent-point regression;
- the package MUST consume a previously accepted Phase-4A foundation package rather than redefining the claim matrix or qualification-package shape from zero during January closeout;
- extrapolated planning metrics MUST be tied back to measured scale points rather than synthetic-only projections;
- the package MUST make it possible to audit every accepted checkpoint-closing claim in the claim matrix without external interpretation;
- the package MUST identify which artifacts close the mandatory non-Samsung internal `50 TB` Scenario A/B evidence and which artifacts are only supporting trend points;
- the package MUST make clear that same-window continuation items are audit-only unless explicitly promoted into the accepted checkpoint-closing claim set;
- the package MUST distinguish directly measured planning data from extrapolated projections and MUST publish any rejection or derating note that affected checkpoint close;
- the accepted closeout record MUST name the accepted internal qualification package because this is the only `2027-01-31` checkpoint that feeds external certification.

### 5.9 Phase-4B Post-GA Expansion Closeout Package (`2027-01-31`)

Required evidence:
- accepted gated-production workload bundle identifiers for any delivered post-GA routes
- fresh benchmark-bundle identifier(s) required by `docs/testing/search-quality-evaluation.md`
- any required comparison-group identifiers for rerank or similar comparative evidence
- platform-expansion evidence identifiers for non-retrieval checkpoint families such as high-lag snapshot sync
- accepted closeout record proving that this same-date checkpoint is distinct from the internal engineering qualification checkpoint

Required checkpoint rules:
- this package MUST NOT redefine the accepted Phase-4A engineering-qualification thresholds;
- the package MAY reuse Phase-4A assets, but it must identify reused versus checkpoint-specific artifacts explicitly, including whether any reused asset came from the `2026-12-15` foundation package or the `2027-01-31` qualification package.

### 5.10 External Scale Certification Evidence Package (`2027-02-28`)

The `2027-02-28` checkpoint is a certification checkpoint rather than a delivery-milestone economics gate. It MUST publish a specialized certification evidence package that cross-references the accepted `2027-01-31` internal qualification package set (and any reused `2026-12-15` foundation artifacts named there) instead of reopening the Phase-4A engineering-qualification format.

Required evidence:
- immutable certification manifest with certified top-end claims, checkpoint date, partner scope, and referenced `2027-01-31` internal qualification package identifiers;
- partner environment summary, allowed environment deltas relative to the accepted internal qualification environment, and declared workload/data correspondence for each certified claim;
- per-claim pass/fail matrix showing which claims are closed by internal evidence reuse versus partner-window execution;
- raw certification artifacts for every partner-executed workload or validation run used by the checkpoint;
- comparison note explaining why any partner-only evidence remains within the already accepted internal qualification envelope.

Required certification rules:
- the package MUST NOT redefine or relax any accepted `2027-01-31` internal qualification threshold;
- any claim certified externally but not reproducible from the frozen internal evidence package MUST identify the missing internal reproducibility reason explicitly;
- if a partner-window run reuses an internal workload shape, the certification manifest MUST reference the original internal bundle and state only the admissible delta.

## 6. Failure Rules

A workload bundle run or checkpoint package MUST NOT be used for milestone/checkpoint closure if any of the following is true:
- the manifest/package is incomplete or mutable;
- the accepted closeout record is missing or does not identify the accepted evidence clearly;
- the measured window begins before the declared warm-up completion condition;
- required workload classes are missing;
- required tenant classes are missing;
- required background systems are in a different state than the claimed milestone closure;
- required artifacts for normalized-cost or throughput reporting are missing.

## 7. Relationship to Search-Quality Gates

This document governs economics and throughput workload bundles.

If a milestone or checkpoint also closes a search-quality gate:
- the retrieval benchmark bundle remains governed by `docs/testing/search-quality-evaluation.md`;
- the workload bundle or specialized checkpoint package defined here MUST cross-reference the benchmark bundle identifier used for the same milestone/checkpoint;
- any route whose delivery materially affects retrieval quality MUST appear in both evidence sets unless the roadmap explicitly declares why it is excluded and the gate explicitly accepts that exclusion.