# Diagnostic Scenario Partial Progress

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "diagnostic-scenario-partial-progress-2026-07-31",
  "problems": [
    {
      "problemId": "provider-query-timeout-discards-compact-scenario-evidence",
      "summary": "When a diagnostic scenario times out, completed compact provider evidence is retained only in trace artifacts while synthesis receives an empty bounded-failure packet.",
      "status": "optimized_unverified",
      "issueKeys": ["APID-29259", "APID-29231", "ST-2865"],
      "sourceRefs": [
        "https://tidb.atlassian.net/browse/APID-29259#problem:provider-query-scenario-timeout",
        "https://tidb.atlassian.net/browse/APID-29231#problem:provider-query-scenario-timeout",
        "https://tidb.atlassian.net/browse/ST-2865#problem:reserve-finalizer-discards-child-evidence"
      ],
      "signalTextIncludesAny": [
        "scenario timeout discards partial evidence",
        "bounded failure packet has no observations",
        "provider query aborted after compact result",
        "partial evidence not admitted to synthesis",
        "finalizer did not receive the real clinic result",
        "diagnostic_objective_gate_evidence_ref_unknown"
      ],
      "observedThrough": "2026-08-05",
      "effectiveAt": "2026-08-05T02:54:36Z",
      "verifiedAt": null,
      "attemptedAction": "Keep one thin schema-validated child evidence handoff refreshed during acquisition, reserve backend finalization time under the runner-enforced outer scenario cap, remove the reserve LLM finalizer, and deterministically convert the latest valid handoff into a bounded partial packet that preserves observations, counter-evidence, acquisition gaps and errors, and canonical artifact refs without causal, identity, or severity promotion.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/issues/531#issuecomment-5134153239"
    }
  ]
}
-->

- Optimization ID: `diagnostic-scenario-partial-progress-2026-07-31`
- Owner: auto-oncall-services
- Source references: deployed `APID-29259` and `APID-29231` scenario timeouts
- Implementation status: deployed to staging in image
  `8f890060857eeb08e42cc8d469978db0ceac902e`
- Effective status: `optimized_unverified`

## Bounded problem

The runner records streamed scenario activity in a failure trace but replaces
the synthesis handoff with an empty bounded-failure packet. This loses compact
facts already acquired before the timeout and can cause the final reply to ask
for an event window or instance that the same run already resolved.

## Candidate implementation

- Evaluate workload identity before horizontal mechanism queries.
- Allow at most one necessary alignment check before TopSQL, followed by one
  narrow Slow Query refinement when a generic wrapper requires it.
- Atomically refresh one strict
  `evidence/scenarios/<scenario-id>/child-evidence-handoff.json` after each
  compact query; the runner remains the only writer of the canonical progress
  checkpoint.
- Reserve 45-60 seconds for finalization and bound each provider query by the
  remaining pre-reserve deadline.
- On timeout, recover only the current-attempt, size-bounded, schema-valid child
  handoff. The backend converts it directly to a partial packet, preserves its
  factual observations, counter-evidence, acquisition gaps/errors and canonical
  refs, and never starts a reserve LLM finalizer or scans raw provider artifacts.
- Keep the scenario and unresolved gates bounded-failure. Partial evidence does
  not become negative evidence, identity proof, or a confirmed cause.

## Verification evidence

Local tests now induce complete and partial no-terminal handoffs, verify that no
reserve finalizer is started, and confirm that factual child observations reach
final synthesis as a bounded partial packet. This is contract evidence only;
source PR `tidbcloud/aisupport#613` and app-delivery PR `#76225` delivered the
implementation to staging. The live Deployment reached generation 265 with the
exact image at `2026-08-05T02:54:36Z`; the Pod was Ready with zero restarts and
both health endpoints returned HTTP 200. This proves runtime availability, not
the case-specific timeout outcome.

- ST-2865 replay `8be8f99b-1577-42d3-9d44-d51bbda33f46` preserved seven
  capacity observations and seven memory observations through a degraded
  packet into a materially improved bounded final answer. Its existing Jira
  comment `23432370` was refreshed and read back with `public=false`.
- APID-29259 replay `c1c2b634-7c05-4aef-baeb-e6cc175b67e8` reached
  `diagnostic_scenario_reserve_boundary_reached` but preserved the valid
  supplied-evidence handoff into the final answer. It did not preserve a new
  provider observation, so it is partial runtime evidence rather than complete
  acceptance.
- APID-29231 replay `c8735add-6469-4596-8bc7-a6c70ec84b9e` wrote seven provider
  observations to the child handoff, then failed with
  `diagnostic_scenario_packet_child_handoff_mismatch`; the final answer dropped
  those provider facts. Its Jira comment was intentionally left unchanged.

## Unresolved gaps

- The timeout path remains a bounded failure because the scenario coordinator
  did not complete semantic classification; preserving facts is not equivalent
  to manufacturing a diagnosis.
- `diagnostic_scenario_packet_child_handoff_mismatch` can still discard an
  otherwise valid provider handoff, as reproduced by APID-29231. The bounded
  problem therefore remains `optimized_unverified` despite the ST-2865 pass.

## Next validation action

Diagnose and correct the packet-versus-child-handoff mismatch without weakening
schema, artifact-ref, or semantic boundaries, then replay APID-29231 at the
same cutoff. Keep APID-29259 available as a reserve-boundary control and do not
refresh either APID Jira comment until provider facts survive final synthesis.