# Provider TiKV CPU Metric Selection

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "provider-tikv-cpu-metric-selection-2026-08-13",
  "problems": [
    {
      "problemId": "evidence-agent-native-grouping-switches-to-pod-metrics",
      "summary": "A Wetech refinement interpreted native metric grouping as Kubernetes pod/container discovery and selected a batch of native-pod CPU queries instead of the ordinary per-instance TiKV process CPU and quota evidence.",
      "status": "optimized_unverified",
      "applicability": "Evidence Agent acquisition for an ordinary TiKV CPU hotspot or CPU-capacity question through Clinic or Wetech, where per-instance TiKV process CPU and actual TiKV CPU quota should precede any explicitly requested Kubernetes quota, throttle, or cgroup boundary.",
      "issueKeys": ["APID-29259"],
      "sourceRefs": ["https://tidb.atlassian.net/browse/APID-29259#problem:native-pod-metric-selection"],
      "signalTextIncludesAny": ["container-cpu-by-native-pod", "process-cpu-by-native-instance", "native pod", "native grouping"],
      "observedThrough": "2026-08-14",
      "effectiveAt": "2026-08-13T16:21:27.000Z",
      "verifiedAt": null,
      "attemptedAction": "Give Clinic and Wetech Evidence Agents one provider-owned TiKV CPU reference: prefer per-instance process CPU plus actual TiKV CPU quota, define native grouping as labels of the selected metric family, and admit Kubernetes pod/container CPU families only for an explicit quota, throttle, or cgroup capacity-boundary question with independently known label scope.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/pull/897"
    }
  ]
}
-->

- Optimization ID: `provider-tikv-cpu-metric-selection-2026-08-13`
- Owner: auto-oncall-services Evidence Agent and Clinic/Wetech provider skills
- Source reference: APID-29259 replay `c5c3f32d-e039-4d09-89f4-82b5f1e87e61`
- Implementation status: merged and effective in staging
- Effective status: `optimized_unverified`

## Bounded problem

The APID-29259 refinement selected multiple pod/container CPU queries after an
earlier contract required cross-family queries to use bounded native grouping.
That wording was intended to keep every metric family on its own label
vocabulary. It did not authorize changing the requested TiKV CPU evidence into
a Kubernetes infrastructure investigation. The resulting query batch did not
establish the ordinary TiKV per-instance CPU ranking or compare it with TiKV's
actual CPU quota.

## Change

The Clinic and Wetech provider skills now give their Evidence Agents the same
small TiKV CPU reference: start with per-instance
`rate(process_cpu_seconds_total[2m])`, then compare it with
`tikv_server_cpu_cores_quota`. The provider skills and core contract define
native grouping as labels on the metric family already chosen.
Pod/container usage, request, limit, and throttle metrics remain available only
when the supplied question explicitly asks for a Kubernetes quota, throttle, or
cgroup capacity boundary and that family's own label scope is known.

The Grafana panel finder and its panel catalog were design references only. No
Investibook path, package, data file, or runtime dependency was added.

## Deployment and verification status

Contract tests verify that both provider skills carry the TiKV process
CPU/quota order, the pod/container admission condition, and the no-cross-label
mapping rule. Existing role-contract tests verify that Scenario Diagnostic
Agent still cannot load or operate a provider skill.

The implementation merged in
[aisupport PR #897](https://github.com/tidbcloud/aisupport/pull/897) as
`dc09e37c265b1c51ec0b2c376fafa4048710b74d`. The staging deploy workflow
[run 31716790562](https://github.com/tidbcloud/aisupport/actions/runs/31716790562)
completed successfully, and
[app-delivery PR #77263](https://github.com/tidbcloud/app-delivery/pull/77263)
merged the staging registration. The current staging image is the valid
successor `fb13c641f6e5defe650d7072dfb544dccf547bab`. After publishing the
Clinic provider skill to the existing skill PVC, the bundled, PVC, and
installed `clinic-api/SKILL.md` all read back as
`7c4524b18d47f982ab84c3d3ddada8687d3131c8781c11e505799bc8601bf016`.
The replacement Pod became Ready at `2026-08-13T16:21:27.000Z`; this is the
effective timestamp because it is the first observed staging boundary at which
both the image and PVC-overridden Clinic skill were active.

Post-change read-only evidence:

- APID-29259 run `1fb2d313-5133-41b8-8911-c077bdad2e02` passed this
  record's selection criterion: the Evidence Agent selected per-instance
  `process_cpu_seconds_total` and `tikv_server_cpu_cores_quota`, and did not
  substitute native-pod/container CPU metrics. Both selected families remained
  explicit unexplained-empty gaps, so this proves selection rather than CPU
  utilization or capacity.
- APID-29338 run `77e9c2c8-ac59-451a-a9bb-9777dae00fe9` and APID-29274 run
  `f60162a3-f935-4308-9df0-6d9461fa8461` preserved the bounded per-instance
  process-CPU path in adjacent Wetech controls. They are not substitutes for
  completing the APID-29259 publication lifecycle.

The bounded selection criterion has replay evidence, but the applicable Jira
internal comment refresh and readback have not completed. This record therefore
remains `optimized_unverified`, not `optimized_verified`.

## Unresolved gaps

This change reduces selection of an unrelated metric family. It does not define
whether a correctly scoped empty metric is expected or unexplained; that is a
separate evidence-acquisition problem. It also does not address the independent
APID-29274 comparison-window handoff label loss or Scenario evidence sensitive-
text false positive.

## Next validation action

Refresh the APID-29259 historical owner job's existing Jira internal comment
through the authorized write path, then read back the same comment ID and
verify `public=false` and the expected paired body. Keep the unrelated empty-
result coverage and APID-29274 defects outside this record's verification
claim.