# Clinic Statements Surface Policy

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "clinic-statements-surface-policy-2026-08-18",
  "problems": [
    {
      "problemId": "clinic-non-aws-statements-selected-as-load-bearing-evidence",
      "summary": "Automatic Clinic evidence planning selects Statements as load-bearing workload evidence even though non-AWS clusters do not configure that datasource, causing an avoidable provider failure and degraded replay.",
      "status": "optimized_verified",
      "applicability": "Automatic Clinic evidence acquisition for workload discovery or attribution when the frozen target does not establish an AWS Statements datasource.",
      "issueKeys": ["TCID-12986"],
      "sourceRefs": ["https://tidb.atlassian.net/browse/TCID-12986#problem:clinic-non-aws-statements-selected-as-load-bearing-evidence"],
      "signalTextIncludesAny": ["clinic_datasource_not_configured", "statement-interval aggregation was unavailable", "non-aws statements datasource"],
      "observedThrough": "2026-08-18",
      "effectiveAt": "2026-08-18T06:23:01Z",
      "verifiedAt": "2026-08-18T17:21:21.557Z",
      "attemptedAction": "Classify clinic_data_proxy.statements as optional supplemental evidence: it may corroborate a candidate established by TopSQL or Slow Query, while empty, unconfigured, or failed results remain visible but do not trigger transport retry, query repair, partial acquisition, or handoff.",
      "evidenceUrl": "https://tidb.atlassian.net/browse/TCID-12986?focusedCommentId=23357171"
    }
  ]
}
-->

- Owner: Auto-oncall Evidence Agent and Provider Executor
- Source: TCID-12986 replay `9858179a-14cd-4c19-a664-f1cca1bc305f`
- Implementation status: deployed to staging
- Effectiveness status: optimized_verified

## Bounded problem

The GCP replay selected `clinic_data_proxy.statements` to rank interval CPU
work. That datasource was not configured, so the query failed before it could
answer the workload-attribution question and made the scenario partial. The
same replay obtained representative Slow Query evidence, and the repaired GCP
TopSQL source can provide aggregate digest CPU and execution count.

The frozen Evidence Agent context contains the exact organization and cluster
IDs but does not establish an AWS Statements datasource. Automatic planning
therefore cannot safely distinguish the one environment where Statements may
exist from non-AWS environments where it does not.

## Candidate change

- Keep Statements available as one narrowly bounded supplemental query only
  after TopSQL or Slow Query has established the workload candidate.
- Have the Provider Executor mark Statements query coverage as `supplemental`.
  Preserve its compact artifact and query event, but exclude an empty,
  unconfigured, or failed result from query repair, scenario completion,
  evidence-acquisition degradation, and human handoff.
- Use TopSQL for aggregate workload distribution and Slow Query for exact
  execution, plan, scan, wait, memory, and spill evidence.
- Preserve the narrower unresolved attribution boundary when neither stable
  surface can answer it; Statements alone cannot close that boundary.

## Verification evidence

The initial hard-prohibition candidate was superseded before rollout. On
2026-08-18, the revised supplemental policy passed the full TypeScript build and
882/882 tests. Focused tests prove that the Clinic executor marks Statements as
supplemental, materialization keeps its failed coverage and compact artifact
without an error or partial handoff, and execution reporting retains the failed
query event and HTTP attempt without producing a failed provider system event
or degraded acquisition. A focused executor test also proves that a Statements
HTTP 500 is attempted only once. After Agent9 Clinic added generic `sql_query`
and Agent9 Wetech added `statements`, the managed Clinic Tool now classifies the
actual read-only SQL surface and emits `evidenceRole=supplemental`; Agent9
materialization applies the same non-blocking completion and error semantics to
both providers.

A branch-local `mode=read` replay of TCID-12986 completed on 2026-08-18 as run
`268bd887-9fa2-4d4e-bc8d-f025cd574acf`. The generated SQL query was named
`topsql-workload-discovery` but actually referenced `clinic_data_proxy.statements`;
the runner classified it from the SQL surface rather than trusting the queryId.
That query failed with `clinic_sql_partition_filter_missing`, remained visible
as failed supplemental coverage, and produced no handoff error or same-round
query repair. Both scenarios completed, overall `evidenceAcquisition=complete`,
`providerCapabilityVerified=true`, and `failureCodes=[]`. The final
`human_required` decision was for unresolved workload identity and causality,
not the supplemental Statements failure. Staging availability remains pending.

After rebasing the candidate onto the multi-target Clinic baseline, a second
branch-local `mode=read` replay completed as run
`38812153-6a97-4b78-9ea9-8a95b0a7e218` with HTTP 200. Its Agent again named a
`clinic_data_proxy.statements` query `topsql-cpu-ranking`; deterministic SQL
surface classification still marked it failed supplemental. The workload
handoff completed 22 of 22 primary objectives with `errors=[]`, and the failed
supplemental query caused zero query-repair turns. The whole replay nevertheless
had another problem: query-latency round 2 failed independently with
`clinic_slow_query_digest_invalid`, so overall evidence acquisition was
`degraded` and this record remains a candidate rather than verified. Read mode
left DB, Jira, evaluation publication, and Feishu writes `not_requested`.

Source PR [#971](https://github.com/tidbcloud/aisupport/pull/971) and
app-delivery PR [#77512](https://github.com/tidbcloud/app-delivery/pull/77512)
were merged. Staging converged to image
`f2c832c6a482e649c8d3496a94a7769b73794cf0` with one Ready replica, zero pod
restarts, and HTTP 200 from both `/healthz` and `/readyz`.

The deployed same-cutoff `mode=read` replay of TCID-12986 completed with HTTP
200 as run `3cac7aff-e189-4c71-a830-aa2d574942cc`. The Evidence Agent used Slow
Query and bounded metrics and did not select Statements as load-bearing
evidence. All 30 Provider queries succeeded, evidence acquisition and response
generation completed, and `failureCodes=[]`. The final `human_required`
decision came from the separate need for human review of query changes and
incomplete statement-to-TiKV attribution, not from Statements availability or
a Provider failure.

The later staging `mode=write` replay
`b975f4a1-22a3-4cb6-8069-87779f626a02` preserved that policy on the same
historical ticket and deployed code. It used Slow Query, typed TopSQL, and
bounded metrics without selecting Statements as load-bearing evidence. All 41
Clinic query events succeeded, evidence acquisition was `complete`, and
`failureCodes=[]`. Publication updated the existing Jira internal comment
`23357171`; readback confirmed the same comment ID and `public=false` without
creating a duplicate or customer-visible comment.

## Unresolved gaps

Statements availability is still not known from the frozen Clinic target. Both
the deterministic batch helper and Agent9 generic SQL path therefore keep it
permissive but do not make its availability or result a completion requirement.
The write replay proves the source case no longer depends on Statements, but a
future case that actually attempts an unavailable supplemental Statements query
must still preserve that failed coverage without degrading acquisition.

## Next validation action

Monitor a later deployed case that actually attempts unavailable Statements and
confirm its supplemental failure remains visible without degrading acquisition
or triggering handoff.
