# Wetech Middle East Dashboard Routing

<!-- auto-oncall-optimization-record
{
  "schemaVersion": "auto-oncall-optimization-record-v1",
  "optimizationId": "wetech-middle-east-dashboard-routing-2026-08-11",
  "problems": [
    {
      "problemId": "wetech-middle-east-dashboard-application-mismatch",
      "summary": "Middle East SaaS clusters were sent to the Game Dashboard P81 application, which returned Cluster Not Found before Slow Query or TopSQL could execute.",
      "status": "optimized_unverified",
      "issueKeys": ["APID-29296"],
      "sourceRefs": ["https://tidb.atlassian.net/browse/APID-29296#problem:wetech-middle-east-dashboard-application-mismatch"],
      "signalTextIncludesAny": ["Cluster Not Found", "dashboard session is not ready", "wetech_dashboard_session_unavailable"],
      "observedThrough": "2026-08-11",
      "effectiveAt": "2026-08-11T07:24:16Z",
      "verifiedAt": null,
      "attemptedAction": "Partition Middle East Dashboard URLs by their SaaS or Game P81 application, detect Cluster Not Found immediately, and try the configured standby application serially when primary session establishment fails.",
      "evidenceUrl": "https://github.com/tidbcloud/aisupport/tree/main/auto-oncall-services/docs/opts/2026-08-11-wetech-middle-east-dashboard-routing.md"
    }
  ]
}
-->

- Optimization ID: `wetech-middle-east-dashboard-routing-2026-08-11`
- Owner: auto-oncall-services
- Source: APID-29296 staging replay `34eff525-726b-4f5f-b0b0-c9c6325a6d6b`
- Implementation status: deployed to staging, diagnosis outcome not verified

## Bounded problem

The image-derived target was correctly normalized to
`wetech:middle-east-prod:SaaS`, and Prometheus queries succeeded. The Dashboard
configuration nevertheless selected the `game-gcp-sg` P81 application first
for that SaaS cluster. Its proxy returned `Cluster Not Found`, so the provider
waited for session readiness and never issued Slow Query or TopSQL requests.

The four Middle East Dashboard URLs were also copied onto every cluster even
though they represent separate SaaS and Game applications. Although the list
was documented as primary followed by standby, the executor selected only the
first URL.

## Attempted change

The seven SaaS-family clusters now use only the SaaS SG and JP Dashboard
applications. `gogame` and `gogame-xxljob` use only the Game SG and JP
applications. Dashboard session establishment detects `Cluster Not Found`
without waiting for the full login timeout and tries the next configured URL
strictly serially. Provider queries still run only once after a session is
ready.

## Verification status

Before implementation, browser checks from the staging runtime proved that
the Middle East `SaaS` proxy key fails on the Game SG application and returns
Slow Query rows on both SaaS applications. The `gogame` proxy key succeeds on
both Game applications. After implementation, the patched skill was copied
into a staging pod without replacing the installed skill. Its default `SaaS`
route returned nine Slow Query rows, and its default `gogame` route completed
successfully with an empty one-minute result. The Wetech contract suite passes
60 tests and the repository suite passes 734 tests.

The merged source commit `c444c2fd2fc41513e04cb4d259dfa362e469463d`
was deployed through aisupport workflow run `31468399884` and app-delivery PR
`#76791`. The live staging Pod became Ready at `2026-08-11T07:24:16Z` with
zero restarts, and both `/healthz` and `/readyz` returned HTTP 200.

Post-deployment read-only replay
`fa3d1f59-d259-4e6c-a0cf-976c61c0ea76` verified the repaired transport. The
query-execution scenario collected 100 Slow Query rows and 51 TopSQL rows from
the SaaS Dashboard origin `https://phkymwfdqy.pzero.perimeter81.com`; the
statement-summary request completed with an empty result rather than a session
failure. Its leading Slow Query row was the same workload later identified by
the human investigation: `analyze table ... with 256 buckets, 500 topn` on
`10.164.19.243:10080`, with `memory_max=46295458134` bytes and
`query_time=294.251770322` seconds.

## Unresolved gaps

The final replay draft did not improve. Although the Scenario Diagnostic Agent
received the exact SQL text, instance, memory, and duration, it still marked
workload identity unresolved. Final Synthesis then prioritized a TiKV-memory
`metric_semantics_unverified` signal, and the publication guard replaced the
case-specific evidence with the generic recording-rule/dashboard-semantics
checklist. One memory-scenario Slow Query request also returned HTTP 400, and a
round-two phase-decomposition plan used an invalid empty `fields` parameter;
neither failure invalidates the successfully materialized query-execution Slow
Query evidence.

This record remains `optimized_unverified`: the bounded Dashboard application
mismatch is fixed in staging, but the known case answer is not corrected and
its existing Jira internal comment was not refreshed.

## Next validation action

Treat the evidence-use/publication-guard behavior as a separate diagnosis
problem. After that problem is repaired, replay APID-29296 again and refresh
its existing Jira internal comment before promoting this problem to
`optimized_verified`.
