# OP Automation Contract

## Current Design

- `wetech_op_grafana.py browser-query-range` is the primary query path. The
  browser is an authentication bootstrap only; Grafana datasource and
  `query_range` requests use direct HTTP with exported session cookies.
- `wetech_op_grafana.py dashboard-query` is the TiDB Dashboard query path for
  TopSQL, SQL Statements, and Slow Query data.
- `wetech_op_grafana.py batch` is the auto-oncall evidence-child path. It
  validates one runner-bound plan, executes independent metric operations with
  a per-service-instance limit of five and independent Dashboard operations
  with a separate limit of two, and atomically refreshes one compact batch
  result. Declared `instanceFromQueryId` and `windowFromQueryId` dependencies
  wait for their source metric.
  Each Dashboard slot owns a stable browser profile; metric HTTP requests share
  exported cookies and serialize only browser-backed session refresh.
- When one Dashboard operation establishes that the Dashboard session is not
  ready, later Dashboard operations in the same batch record that bounded
  dependency failure immediately instead of repeating the same session wait.
  At most the two already-active Dashboard slots can observe that failure.
- The batch target is normalized from the runner-owned auth target reference,
  a plan target reference/object, or compatible query fields. Agent-facing
  operation aliases are normalized to the concrete provider commands before
  validation and execution.
- Cluster lookup resolves to a region Grafana host plus a concrete datasource
  selector before any PromQL request is sent.
- Dashboard cluster lookup resolves to a dashboard host plus the concrete
  TiDB Dashboard proxy cluster name.
- Dashboard proxy routing and Grafana datasource routing are independent.
  `dashboard_cluster_name` must never fall back to `datasource_name` or a
  Grafana datasource UID, even when their displayed names are identical.
- Standalone Dashboard origins use explicit `dashboard_direct: true` routing
  and direct `/dashboard/` plus `/api/...` paths. They do not require or infer
  a proxy cluster name. Multiple Dashboard URLs are ordered primary then
  standby entry points.
- `query-range` with a user-copied Grafana cURL is retained only as a
  last-resort fallback.
- The script never reads Chrome profiles.
- Existing P81 2FA codes come only from `scripts/get_verify_code.py`.

## Risks

- P81 2FA is a second factor; the current code must be fetched through the
  configured verify-code script and used only in memory.
- Raw browser cookies and authorization headers must not be printed or written
  to reports.
- Dashboard component log task-group search can cause unsafe PD memory growth.
  The skill must not expose or invoke it.
- Grafana datasource IDs are origin-local and are normally discovered from the
  logged-in Grafana API. A cluster inventory may carry `datasource_id` only
  when listing and individual datasource APIs are permission-denied, an exact
  proxy query has been live-verified on that origin, and no unverified standby
  origin shares the selector.
- Some standalone Grafana origins deny `/api/datasources`; in that case the
  datasource name must be derived from dashboard panel JSON and stored as an
  exact `datasource_name_regex`.
- Browser and direct cookie session caches expire; failed cached sessions must first try the
  shared P81 username/password plus verify-code script before asking for manual
  intervention.
- Dashboard aggregation pages can render cluster iframes without completing a
  usable Dashboard API session. Dashboard API queries must use the selected
  cluster's direct `/proxy/<cluster>/dashboard/#/...` app context.

## Target Contract

- Auto-oncall calls this helper with the Scenario Router's already selected
  canonical alias and exact region. The helper validates that inventory member
  and resolves transport configuration; it does not infer a ticket spelling or
  switch regions.
- `browser-query-range` owns browser authentication for Wetech OP clusters;
  datasource discovery and Prometheus queries must not depend on a live page or
  Playwright request context.
- Cluster lookup defaults to `asia-prod` when `--region` is omitted.
- A configured business alias resolves to its one canonical cluster entry only
  inside the selected region. Duplicate matches fail closed; aliases never
  trigger a cross-region fallback.
- A full `dashboard_cluster_name` is also an admissible exact target when it
  uniquely maps to one configured cluster entry; its configured canonical alias
  and region are used. Ambiguous names fail closed.
- Shared Grafana origins are common. Querying a cluster requires a concrete
  `datasource_uid`, `datasource_name_regex`, or a live-verified origin-local
  `datasource_id`; the Grafana default datasource is not a valid fallback for
  shared origins.
- It uses a skill-managed Playwright profile under
  `~/.cache/wetech-op-grafana/browser/`, not the user's Chrome profile.
- It may auto-fill shared P81/Grafana usernames and passwords.
- It fetches existing P81 2FA codes through `scripts/get_verify_code.py` by
  default, then fills the visible browser form.
- `--terminal-2fa` is an explicit manual fallback.
- It must not write 2FA codes to stdout, evidence files, compact reports, or
  browser diagnostics.
- It discovers a Prometheus datasource through Grafana's `/api/datasources`
  endpoint, then calls datasource proxy `query_range` directly.
- A configured `datasource_uid` or `datasource_id` bypasses datasource
  enumeration and is validated by the proxied Prometheus request itself.
- If `/api/datasources` returns permission denied after a successful Grafana
  login, `datasources` can fall back to `/api/search` plus dashboard JSON and
  report panel datasource names. `browser-query-range` resolves the selected
  name through the individual datasource API, then uses the UID proxy when
  available or the numeric ID proxy otherwise.
- `rebuild-datasource-map` logs in to each selected region Grafana, lists
  Prometheus datasources, matches them to selected cluster aliases, and writes
  `datasource_uid` only when a unique match is found and `--write` is passed.
- It persists only metric evidence and compact reports through the existing
  evidence path. Browser state and a mode-`0600` direct cookie cache stay under
  the private skill session directory and never enter reports or stdout. The
  cache also retains the browser User-Agent because P81 binds application
  authorization to that non-secret client metadata.
- `dashboard-query` opens the direct TiDB Dashboard app for the selected
  cluster, recovers P81 authorization failures through the standard password
  login path, signs in to Dashboard with the shared monitor account, and calls
  Dashboard APIs from that same browser origin.
- `dashboard-query` requires either an explicit inventory
  `dashboard_cluster_name`, an explicit `--dashboard-cluster-name` override,
  or `dashboard_direct: true` with a dedicated Dashboard origin. Missing or
  ambiguous Dashboard routing fails closed before opening a browser.
- Component log search is disabled. Do not call `/api/logs/taskgroup`, and do
  not fall back to a Grafana log datasource.
- `dashboard-query` persists raw Dashboard payloads only in `dashboard.json`;
  compact reports keep counts and previews and never include tokens, cookies,
  passwords, or raw authorization headers.
- Provider data commands emit one bounded `provider-result-v1` JSON object.
  Stdout contains only compact summaries and artifact paths. Raw payloads,
  complete plans, chart data, and detailed error responses remain in a
  run-scoped evidence directory and are not the default agent input.
- `workspace_evidence_report.json` uses the same compact provider boundaries
  and byte budget as stdout. Each write recompacts legacy report entries,
  retains at most ten recent entries and preview rows, and removes `payload`,
  `plans`, and complete plan fields. `dashboard.json` remains raw.
- Compact metric stdout and report entries preserve up to ten
  `series_summaries`. Each series retains decision labels including
  `instance`, `type`, job/component/store identity when returned, and numeric
  `min`, `max`, `mean`, `last`, `point_count`, and `series_count`. Metric
  summaries retain the ten highest series by numeric `max` so affected-instance
  selection does not depend on Prometheus response order. Report writes rebuild
  these fields from raw `metrics.json` instead of carrying forward legacy
  depth-truncated values.
- A compact metric label is scoped to its producing query. The Evidence Agent
  may reuse one exact equality selector only for the same metric family when
  that exact label key/value is present, copying the value verbatim without
  escaping punctuation. It must never construct a PromQL regex from prior
  label values. It uses a bounded native aggregation or `topk` for multiple
  entities or a different metric family and correlates returned series instead
  of guessing that display names or `instance`, `store`, `address`, `pod`,
  `node`, or `host` vocabularies are interchangeable. `instanceFromQueryId`
  for TopSQL is the only helper-owned cross-operation selector dependency.
- Dashboard APIs are not the first-pass 24h exploration path. Use Prometheus
  metrics to locate a hotspot time and instance, then query Dashboard APIs in a
  narrow window: TopSQL should normally use 10-15 minutes; SQL Statements and
  Slow Query should normally use 30-60 minutes.
- TopSQL requires exactly one explicit `--instance` selected from time-aligned
  metrics. It must not query the instance inventory and silently select the
  first item, and it must not fan out across instances. TopSQL defaults to
  `top=10`, `window=10s`, and an approximately 120-second total command
  budget. It sends one summary request and does not sleep or retry
  automatically. A timeout is surfaced immediately so the caller can narrow
  the time range or correct the selected instance before a new command.
  `--command-timeout` may shorten this budget but cannot raise it above 120
  seconds.
- The auto-oncall `batch` path rejects a TopSQL absolute range longer than 900
  seconds before browser startup or provider access. It never silently clamps
  a broad range because choosing the first, middle, or last segment could move
  the query away from the incident. A plan without an exact incident-aligned
  TopSQL subwindow must omit TopSQL and preserve the evidence gap. This guard
  does not shorten broader Prometheus alignment or 30-60 minute Slow Query
  requests.
- A TopSQL query may declare `instanceFromQueryId` and `windowFromQueryId`
  against a preceding TiDB instance-labelled metric in the same batch.
  The helper deterministically selects the highest metric series, reuses its
  exact instance, and centers a window of at most 900 seconds on that series'
  peak timestamp while keeping the window inside the runner-owned bounds. It
  does not require a second agent turn or a second batch call.
- TopSQL compact previews and record counts are derived from
  `summary.data`. Empty digest rows with `is_other=true` are the Dashboard
  "Other" aggregate, not missing data.

## Implementation Steps

- Add optional Playwright dependency and install guidance.
- Add duration parsing for last-window queries.
- Add browser login helpers for P81, verify-code script handoff, manual 2FA
  fallback, and Grafana login.
- Add Grafana datasource discovery and browser-authenticated query execution.
- Reuse existing metric summarization and evidence persistence.
- Add TiDB Dashboard session handling and API queries for TopSQL, SQL
  Statements, and Slow Query.

## Verification Checklist

- `list-clusters` still works without Playwright installed.
- `browser-query-range --help` works and reports optional dependency errors only
  when the command is executed without Playwright.
- `login-guide --cluster archit` resolves to `asia-prod` by default; passing
  `--region middle-east-prod` selects the Middle East production entry.
- Shared-Grafana clusters without a datasource selector fail before querying
  instead of using a default datasource.
- Standalone clusters may use an exact `datasource_name_regex` from dashboard
  panels when UID is not visible through the datasource API.
- Existing `query-range` cURL flow remains available.
- `browser-query-range` writes no cookies, authorization headers, passwords, or
  2FA codes to stdout, evidence files, or browser diagnostics.
- A valid direct cookie cache completes datasource discovery and `query_range`
  without importing or launching Playwright. A missing or rejected cache starts
  the browser only for authentication, closes it, and retries direct HTTP once.
- The CLI exposes no component log command and repository search finds no
  `/api/logs/taskgroup` request path.
- A first visible run can complete P81/Grafana login with
  `scripts/get_verify_code.py` filling 2FA.
- A later cached-session run can use `--headless`; if P81 reports
  authorization failure or shows the standard login form, the command can
  refresh the shared login automatically instead of failing immediately.
- `dashboard-query --kind statements`, `--kind slow-query`, and `--kind topsql`
  can run through the direct Dashboard app without relying on the aggregation
  page cluster picker.
- TopSQL nested `summary.data` rows are counted and previewed correctly,
  including `is_other=true` aggregate rows.
- TopSQL without `--instance` returns `topsql_instance_required` before browser
  startup; an explicit instance results in one `/api/topsql/summary` query with
  default `top=10`.
- A timed-out TopSQL summary is attempted exactly once. Its request timeout is
  bounded by the remaining `--command-timeout` budget, and no default 60-second
  retry sleep occurs.
- An auto-oncall batch TopSQL range longer than 900 seconds fails locally with
  `op_batch_topsql_window_too_wide` without starting a browser or calling the
  Dashboard; an exactly 900-second TopSQL range remains valid, and a one-hour
  Slow Query range remains valid.
- A batch using only the runner auth `targetReference` and agent-facing
  `metric`/`topsql` operations resolves the exact Wetech target, runs the metric
  first, and derives the dependent TopSQL instance and bounded hotspot window.
- Provider stdout remains within its byte budget and excludes `payload`,
  complete plan fields, and chart data. Success raw data and error details are
  recoverable from the returned run-scoped artifact paths.
- Rewriting a report containing legacy TopSQL `preview_rows[].plans` removes
  those plans while leaving the raw `dashboard.json` payload intact, and the
  resulting compact report stays within the provider byte budget.
- A multi-series metric result keeps readable instance/type labels and numeric
  extrema in both stdout and the compact report. A legacy report containing
  `<truncated:depth>` metric fields is repaired from raw `metrics.json` on the
  next metric write.

## Evidence Agent semantic repair

- Provider transport retries remain deterministic helper behavior.
- An executed-query semantic failure is returned to the same Evidence Agent
  thread. It may replace only the failed load-bearing objectives and can change
  query text, aggregation, labels, bounded window, step, ordering, projection,
  or supported operation while retaining the exact target and question.
- Each query can be repaired at most four times in the same acquisition round.
  An unchanged plan is never re-executed. The Agent stops when a repair is not
  load-bearing or has no bounded supported form.
- A successful query with zero matching rows remains an explicit gap but counts
  as complete execution. Only failed or incomplete queries make the batch
  partial.