# Wetech TopSQL scope and HTTP diagnostics

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "wetech-topsql-scope-http-2026-09-07",
  "problems": [
    {
      "problemId": "topsql-query-cost-and-hidden-transport-failure",
      "summary": "Wetech TopSQL default scope is unnecessarily large and HTTP 0 hides transport failures.",
      "status": "optimized_unverified",
      "applicability": "Wetech Managed Tool TopSQL query planning and Dashboard transport diagnostics.",
      "issueKeys": [
        "APID-29470"
      ],
      "sourceRefs": [
        "https://tidb.atlassian.net/browse/APID-29470#problem:topsql-query-cost-and-hidden-transport-failure"
      ],
      "signalTextIncludesAny": [
        "TopSQL failed with HTTP 0",
        "TopSQL failed with HTTP 503"
      ],
      "observedThrough": "2026-09-07",
      "effectiveAt": "2026-09-07T13:18:37.160Z",
      "verifiedAt": null,
      "attemptedAction": "Wetech v58 and matching Skill installed; AOS 65232e57 is Ready with original analysis budgets. One bounded live top=5 canary returned managed_tool_result_unknown after 123,141 ms with no Provider data; keep unverified. No new full replay was run.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/actions/runs/34137649995"
    }
  ]
}
-->

Owner: AOS Wetech Managed Tool and provider query Skill. Source: operator request after APID-29470 isolated replay.

Default TopSQL to five entries in the executable query contract and Skill guidance. Preserve explicit limits up to 20 and the existing Slow Query default. Build on merged PR #1387 API-first session access; no new browser-first path. Preserve received HTTP status (including response-body failures); before a response exists, report null status plus timeout or network error. Record request duration independently of Tool/bootstrap time.

Validation on 2026-09-07: 54 focused transport/session/query tests passed. Existing v52 live checks used SaaS02, instance 10.107.41.28:10080: 10-minute top=10 failed with HTTP 0 after 85.802 seconds; 1-minute top=10 succeeded after 82.617 seconds (11 rows); 10-minute top=5 succeeded after 60.167 seconds (6 rows). These are total Tool durations and single-run comparisons, not HTTP-only measurements or proof of the 503 root cause. PD RSS, up and process-start metrics over 19:50-20:30 CST showed no restart or scrape outage.

Prepared immutable Wetech v54 package `eecc16ae2f187145db39fbf048ba678779ed22e355b076b336ee4676c8e1b00b` and bundle `963efaefc5854a2d08f37f2a1fe695159145faba2f29cf228beb0d6fd478d2d6`. These artifacts retain the API-first behavior from v53 / PR #1387. PR #1388 is merged and the package and matching Skill are deployed. The isolated replay below verifies real query execution and HTTP-only timing; default-five adoption must be judged from the submitted query parameters, because explicit higher limits remain supported.

## Deployment and isolated replay

AOS `eefdf77930d9c425299f1e7a1b21c32e3fc31afd` is Ready on image digest `sha256:f777036a6a3f42f76013867c4f4c7655443bf0fcdb461c112788ad7bc86226f4`. Release workflow 34125767050 and app-delivery PRs #79452/#79453 succeeded. Evidence Agent installation readback confirms Wetech v54 (`eecc16ae2f187145db39fbf048ba678779ed22e355b076b336ee4676c8e1b00b`). Agent9 loaded the AOS-owned bundle on both Ready replicas without an application-image change.

One isolated read replay completed at 2026-09-07T13:38:20.091Z after 1,182,931 ms (19m 42.931s): Job `aos-replay-apid-29470-a1926c51`, run `5aa523af-28ca-4bec-bf8e-6ae4270cc033`, answer fingerprint `5e39a78d896f8865a3d1ef05f48278da93bd002c2eb98707385e392c22c4d649`. Judge returned `ai_better`; response generation completed, evidence acquisition remained degraded. Full run, frozen read-publication, summary and logs were exported to `/home/nizheming/.cache/auto-oncall-services/replays/apid-29470-main-eefdf779-20260907`; the Job was deleted after collection. No Jira, DB or Base publication was performed.


Four first-round TopSQL calls produced three HTTP 200 results with API durations 31,416 / 11,758 / 8,075 ms and one transport timeout after 60,006 ms. The timeout correctly preserved null HTTP status and `errorClass=timeout`; there was no HTTP 0 or HTTP 503 in this run. The failed call included an instance filter (10.107.42.112:10080) and explicit top=10 for the original ten-minute window. Successful result counts were 21 / 11 / 21, so this replay does not establish default-five adoption: explicit larger limits still override the deployed default. The workload refinement subsequently exhausted its 79,762 ms remaining stage budget. Keep `optimized_unverified`: HTTP diagnostics are verified by real results, but planning still needs a bounded check of why ordinary ranking chose larger limits before another release/replay. The time-label mismatch in the final answer is a separate Router/input issue. Do not infer a 503 root cause or an OOM from these observations.


Follow-up after the same replay: the Skill now asks Evidence Agent to explicitly pass `limit: 5` for ordinary ranking and first instance queries, rather than offering omission or a generic larger coverage choice. The executable default and explicit-limit support stay unchanged. This follow-up is local and not yet published.


Local timeout follow-up: pinned Playwright Docker against SaaS02, instance 10.107.42.112:10080, epoch 1788547260–1788547860 and limit=5 reproduced a transport timeout after 62,451 ms with the 60-second cap. A serial experiment with a 120-second HTTP cap succeeded (HTTP 200, 6 rows, 14,393 ms). Because that successful response arrived inside 60 seconds and reused authentication state, this is not proof that doubling the timeout resolved the backend cause. Source now allows 120 seconds for TopSQL HTTP only, with a 180-second Managed Tool envelope and 185-second local simulator envelope. Other HTTP operations and analysis budgets are unchanged. Wetech v55 immutable artifacts and source pins are prepared (package `2a3f1013dbd6578610fbd2e859c71672cbc3d60e6af3ebb552a924debcd6890e`, bundle `7cde731a744b2391ebf64f88a86416d0751c2df47e6c7e28ea6effb2aa20796f`). All 1,168 unit tests and six isolated-runner checks passed, including a simulated 70-second successful HTTP response; loader pins validate against current source. The live Docker experiment used the identical final index.mjs bytes. Tool/Skill installation, deployment and one APID-29470 isolated replay remain required before claiming live verification. Artifacts: `/home/nizheming/.cache/auto-oncall-services/replays/apid-29470-topsql-local-timeout-20260907`.


Deployment correction: v55 failed Agent9 startup because its 180,000 ms Tool timeout exceeds the registry maximum of 120,000 ms. The old Ready replica remained serving and the AOS-owned loader configuration/annotation was rolled back to the v54 bundle. v56 restores the supported 120,000 ms Tool envelope and 125,000 ms local simulator envelope, retaining the 120,000 ms TopSQL HTTP cap. The immutable catalog excludes the invalid v55 manifest. Tool initialization consumes part of the outer 120-second budget; this does not promise an exclusive 120 seconds for HTTP. No Agent9 runtime limit or analysis budget is changed.


## Final deployment and bounded canary

The operator replaced the planned new full replay with one simple live query and publication of the existing frozen read result. Wetech v58 package `b368150113c070602eece0bbd953b1eb99a91882c776d02d92e522d0f8c599ec` preserves the 120-second HTTP/Tool caps. Both Ready Agent9 replicas loaded bundle `4e7ed6e66c4c8457211184c8d5ab49705c478109bad8f05f77b4f8c3735dc0ae`; the Evidence Agent installation is active at v58. After the rolling replicas converged, release 34137649995 attempt 2 succeeded. App-delivery PRs #79458/#79459 deployed AOS `65232e57b35ead1e3892b8d79e37edac0da19663`, image digest `sha256:df15cae51f122c838c5c3aed17b3388d645f504b6b271c8a542fdcd02e87b78e`. The Ready service returns HTTP 200 for health and retains pipeline/gap/worker budgets of 1,320,000 / 1,035,000 / 1,500,000 ms.

Exactly one Managed Tool canary used SaaS02, instance 10.107.42.112:10080, epoch 1788547260–1788547860 and explicit limit=5. Operation `op_92d5c439036a4ec289dac2357782b93d` in session `sess_fc6df7beae204b238604327a690ad1f7` ran from 15:26:03.106Z to 15:28:06.247Z on 2026-09-07: 123,141 ms total Tool duration. It returned `unknown_result` / `managed_tool_result_unknown` with `retryContract=do_not_retry`, no HTTP status, no HTTP-only duration and no query data. It was not retried. This is a failed canary, not proof of backend timeout, cache delay, OOM, successful default-five planning, or an exclusive 120-second HTTP allowance. The exact next investigation is the outer Managed Tool execution/result boundary for this operation; no extra timeout or budget changes were made in this validation step.

Private evidence: `/home/nizheming/.cache/auto-oncall-services/replays/apid-29470-v55-canary-publish-20260907/canary-summary.json` and `turns.json`.

The existing eefdf779/v54 frozen replay was then published unchanged to internal Jira comment `23815249`, DB run `150218`, evaluation and Base. Its three successful TopSQL samples and one historical 60-second timeout must not be attributed to the v58 canary. No further replay or query was run.
