# Core Handling Contract Convergence

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "core-handling-contract-convergence-2026-08-11",
  "problems": [
    {
      "problemId": "classification-evidence-required-duplicates-owned-routing-fields",
      "summary": "Scenario classification persisted an evidenceRequired boolean that duplicated resolutionMode and evidenceProvider. The duplicate ownership was removed and the new contract was verified structurally; a historical audit found no case where the obsolete field had actually contradicted those owners or blocked publication.",
      "status": "optimized_verified",
      "issueKeys": [],
      "sourceRefs": [
        "https://github.com/tidbcloud/aisupport/blob/main/auto-oncall-services/docs/AUTO_ONCALL_CORE_CONTRACT.md#problem:classification-evidence-required-duplicates-owned-routing-fields"
      ],
      "signalTextIncludesAny": [
        "evidenceRequired contradicts resolutionMode",
        "classification evidence requirement duplicates provider route",
        "provider execution depends on parallel boolean"
      ],
      "observedThrough": "2026-08-11",
      "effectiveAt": "2026-08-11T04:52:47Z",
      "verifiedAt": "2026-08-11T05:02:38.884Z",
      "attemptedAction": "Remove evidenceRequired and execution.required from new payloads; make resolutionMode own answer handling, evidenceProvider own executable routing, execution own actual Provider outcome, and reject structurally contradictory classifier combinations.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/pull/750"
    },
    {
      "problemId": "completed-evidence-assisted-answer-ineligible-for-publication",
      "summary": "Customer Reply rejected every evidence_assisted result from classification intent even when the selected Provider and final answer completed successfully.",
      "status": "optimized_unverified",
      "issueKeys": [],
      "sourceRefs": [
        "https://github.com/tidbcloud/aisupport/blob/main/auto-oncall-services/docs/AUTO_ONCALL_CORE_CONTRACT.md#problem:completed-evidence-assisted-answer-ineligible-for-publication"
      ],
      "signalTextIncludesAny": [
        "evidence_assisted customer reply rejected",
        "complete provider evidence cannot publish",
        "evidence_required publication skip"
      ],
      "observedThrough": "2026-08-11",
      "effectiveAt": "2026-08-11T04:52:47Z",
      "verifiedAt": null,
      "attemptedAction": "Admit evidence_assisted only when the selected Provider matches the execution record, evidence acquisition is complete and capability-verified with no failure code, and the normalized final answer has no clarification, requested context, or human handoff.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/actions/runs/31459617280"
    },
    {
      "problemId": "nonoperative-risk-text-blocks-reviewed-consulting-answers",
      "summary": "Generic production tokens and any non-empty riskFlags list blocked human-accepted consulting answers even when the request was advice, the draft stated its boundary, and no privileged support action was requested.",
      "status": "optimized_unverified",
      "issueKeys": ["APID-29261", "APID-29264", "APID-29304"],
      "sourceRefs": [
        "https://tidb.atlassian.net/browse/APID-29261#problem:nonoperative-production-risk-publication",
        "https://tidb.atlassian.net/browse/APID-29264#problem:self-service-configuration-advice-publication",
        "https://tidb.atlassian.net/browse/APID-29304#problem:version-guidance-publication"
      ],
      "signalTextIncludesAny": [
        "production_cluster forced human review",
        "production_excluded forced human review",
        "risk_flagged blocked reviewed consulting answer"
      ],
      "observedThrough": "2026-08-11",
      "effectiveAt": "2026-08-11T04:52:47Z",
      "verifiedAt": null,
      "attemptedAction": "Keep concrete privileged, destructive, sensitive, and severe-incident risks fail-closed while removing generic production-token and any-risk publication denial; retain final decision, normalized-draft safety, and the independent customer validator.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/actions/runs/31459617280"
    }
  ]
}
-->

- Optimization ID: `core-handling-contract-convergence-2026-08-11`
- Owner: auto-oncall-services
- Source reference: user-reviewed eight-ticket consulting corpus and the current
  staging classification/execution/publication contract
- Implementation status: merged and deployed to staging
- Effective status: one problem `optimized_verified`; two problems
  `optimized_unverified`

## Bounded capability

This change gives each core fact one owner from scenario classification through
customer publication. It removes the parallel evidence-requirement booleans,
keeps Provider selection independent from final human ownership, and evaluates
publication from the actual execution plus normalized final result.

It also makes risk handling proportional to the observed boundary. Ordinary
product guidance may mention production scope, upgrade behavior, configuration
side effects, or query correctness. Privileged support actions, destructive
mutations, sensitive disclosure, cross-org data, and severe active incidents
remain fail-closed.

## Implementation

- Add `docs/AUTO_ONCALL_CORE_CONTRACT.md` as the canonical field-ownership and
  stage-boundary document.
- Require agents to read it before modifying the core path.
- Remove `ScenarioClassification.evidenceRequired` and
  `execution.evidenceAcquisition.required` from new code and test payloads.
- Derive Provider execution from `evidenceProvider != none` so bounded read-only
  diagnosis may still support a `human_required` case.
- Validate legal `resolutionMode`, Provider, and `missingKeys` combinations at
  the LLM classifier boundary.
- Admit complete `evidence_assisted` results to the existing cohort, Bot
  Assignment, final-result, validator, idempotency, and Jira publication path.
- Preserve historical JSON tolerance by ignoring obsolete extra fields rather
  than emitting compatibility fields in new payloads.

## Verification status

Local type checking and the complete repository test suite pass. Contract tests
cover direct answers, complete evidence-assisted publication, degraded or
mismatched Provider rejection, non-operative risk flags, classifier combination
validation, historical execution parsing, diagnostics, evaluation, and runtime
problem signals.

Source PR [#748](https://github.com/tidbcloud/aisupport/pull/748) and
app-delivery PR
[#76771](https://github.com/tidbcloud/app-delivery/pull/76771) are merged.
Staging rolled out image
`922f7472c1b29c4fbf3466b8362d8adf9761ad0c` at
2026-08-11T04:52:47Z, with one updated and Ready replica. The live Pod has
Customer Reply enabled and Auto Coverage includes `direct_answer`,
`clarification_needed`, and `evidence_assisted`.

Post-change case verification used reviewed case APID-29309. Read replay
`54524c39-8423-41cd-896a-7b5f3b6a9d4c` and write replay
`83dba5c1-1280-4f0d-bf27-711fac922bd4` both classified it as
`feature_usage_or_capability/direct_answer` with Provider `none`, no missing
keys or risk flags, complete response and evidence execution, and no obsolete
parallel boolean. The comparison verdict was `ai_better` with no AI gap.

The write replay updated existing Jira internal comment `23453424` in place.
Readback of that exact comment confirmed `public=false` and the expected new
body. No duplicate or customer-visible comment was created.

That replay verifies the new payload and routing contract, but it does not show
that `evidenceRequired` caused the old case behavior. A subsequent historical
audit found 311 persisted classifications containing the obsolete field, with
zero `direct_answer + evidenceRequired=true` and zero
`evidence_assisted + evidenceRequired=false` combinations. The five persisted
Bot Assignment customer-reply decisions contained no `evidence_required`
denial. APID-29309's historical customer-reply denial was `feature_disabled`.
The verified status of the first problem is therefore limited to removal of the
duplicate field and consistent new payload ownership; it is not evidence of a
diagnosis-quality or publication-rate improvement.

Additional current-staging read replays did not produce a case that exercised
either remaining criterion completely:

- APID-29314 failed closed after a Scenario Classifier execution failure and
  therefore did not exercise evidence-assisted publication.
- APID-29200 produced an `ai_better` direct answer, but its current
  classification had no risk flag and the result still required clarification,
  so it did not exercise proportional non-hard-risk publication.
- APID-29317 and APID-29248 were stopped when the bounded candidate search was
  ended; incomplete runs are not verification evidence.

The earlier Feishu summary-table duplicate was repaired separately. The older
duplicate `public_candidate` and `public_answer` records were removed, a live
14-day DB-to-Base sync processed 420 ticket rows with zero failures, and all
eight summary stages updated with zero summary failures. Readback showed exactly
one record per Stage Key.

## Unresolved gaps

- No controlled Bot Assignment has verified a complete evidence-assisted public
  reply and its Jira `public=true` readback.
- The proportional non-operative-risk publication change still needs a
  post-change replay whose classification actually carries a non-hard risk
  flag.
- No suitable historical case was found that also satisfied every final-answer,
  Provider-execution, cohort, and publication-safety condition. The two
  remaining problems stay `optimized_unverified`; no status is inferred from a
  near match or an incomplete replay.

## Next validation action

Wait for a naturally eligible, non-duplicative staging case rather than forcing
a near match. For evidence-assisted publication, verify that the selected
Provider execution is complete and capability-verified, the final answer has no
clarification or human handoff, and the validator permits the normalized draft;
only an explicitly authorized controlled Bot Assignment may verify an actual
single Jira `public=true` write. Separately verify proportional risk handling
when a current classification genuinely carries a non-hard risk flag. Until
those exact criteria are exercised, keep both problems
`optimized_unverified`.
