# Diagnose Evidence Correlation MVP ExecPlan

This ExecPlan is a living document. Keep `Progress`, `Surprises & Discoveries`, `Decision Log`, and `Outcomes & Retrospective` up to date as work proceeds.

Reference: `PLANS.md` at repository root; this plan must be maintained according to it. This plan absorbed the original root-level diagnose design notes and is now the repository source of truth for the diagnose MVP history and future design notes.


## Purpose / Big Picture

After this change, a user can run `clinic diagnose <seed>` to search Loki logs for a free-text error seed, extract reusable anchors from matching log lines, correlate related evidence into incident candidates, and inspect why each incident was grouped. The command does not infer root cause. It prepares deterministic, explainable evidence so a human or later agent can continue the investigation.

The observable MVP behavior is:

    clinic diagnose "Request range exceeds bound" --cluster-id 123 --since 24h
    clinic diagnose "Request range exceeds bound" --cluster-id 123 --since 24h -o json

The default table output shows compact incident handles and summaries. JSON output includes `query_scope`, incident evidence, `score_reasons`, and warnings.


## Execution Protocol (Mandatory)

Implement this plan sequentially. Do not execute multiple subtasks at the same time, even if they look independent.

For every implementation subtask, use a fresh non-inherited subagent session. Give that subagent only this plan path, the current subtask, and the repository context needed for that subtask. The parent agent reviews the result, integrates any needed adjustments, runs the subtask validation commands, commits the subtask changes, and updates the `Progress` section before starting the next subtask.

Each subtask has one commit boundary. Do not start the next subtask until the previous subtask is reviewed, validated, committed, and reflected in `Progress`. If a subtask cannot be completed or committed cleanly, stop and resolve that subtask before moving on.

The operator prompt template for runtimes that require explicit delegation authorization is:

    Implement docs/2026-04-21-diagnose-plan.md using fresh non-inherited subagents per subtask, sequentially, with parent review/integration, one commit per subtask, and Progress updates after each subtask commit.

If subagents are unavailable in the execution environment, the current agent may execute subtasks directly, but it must preserve the same sequential order, review checkpoint, validation commands, one-commit-per-subtask boundary, and `Progress` updates.


## Progress

- [x] (2026-04-21 Asia/Shanghai) Converted the original root-level diagnose notes into this repository-compliant ExecPlan.
- [x] (2026-04-21 Asia/Shanghai) Subtask 1: Added the `diagnose <seed>` command skeleton, root registration, help text, and initial validation tests.
- [x] (2026-04-21 Asia/Shanghai) Subtask 2: Implemented diagnose scope resolution, safe seed Loki query construction, duration parsing, request retries, and validation tests.
- [x] (2026-04-21 Asia/Shanghai) Subtask 3: Implemented Loki response normalization, shared diagnose JSON types, generic field parsing, anchor extraction, deterministic evidence IDs, and extraction tests.
- [x] (2026-04-21 Asia/Shanghai) Subtask 4: Implemented deterministic evidence correlation, incident scoring, bridge safeguards, score reasons, incident IDs, and low-confidence warnings.
- [x] (2026-04-21 Asia/Shanghai) Subtask 5: Implemented table and JSON rendering contracts, command output wiring, no-result output, low-confidence output, raw-line scoping, and rendering tests.
- [x] (2026-04-21 Asia/Shanghai) Subtask 6: Updated command docs, README examples, legibility checks, and final verification.
- [x] (2026-04-21 Asia/Shanghai) Final review pass: fixed sub-threshold strong-bridge correlation bypass, restored shared `now-0s`/`now-0d` compatibility, and added focused regression tests.
- [x] (2026-04-21 Asia/Shanghai) Retired the root-level diagnose notes after folding unique post-MVP design notes into this plan.


## Surprises & Discoveries

- Observation: The original diagnose notes explicitly said fixed diagnosis categories are unnecessary and should not be the core abstraction.
  Evidence: The design notes say the durable object is the incident, not the error type, and warn against `RequestRangeExceedsBoundIncident`-style types.

- Observation: The existing `loki query` command defaults missing start/end to the latest 5 minutes, but `diagnose` must default to `--since 1h` when no time scope resolves.
  Evidence: `internal/cmd/flag_env.go` defines `defaultRangeFallbackWindow = 5 * time.Minute`; the original diagnose notes stated the diagnose MVP defaults to `--since 1h`.

- Observation: `scripts/check-agent-legibility.sh` currently validates a fixed command-family list that does not include `diagnose`.
  Evidence: The script has `families=(auth env cluster data metrics loki events raw)`.

- Observation: Subtask 1 quality review found that command-surface docs are stale while `diagnose` is only partially implemented.
  Evidence: The plan already assigns `docs/COMMAND_SURFACE.md`, README, legibility script, and final plan retrospective updates to Subtask 6, so the Subtask 1 commit records the parser/help skeleton and leaves docs updates for that doc-focused subtask.

- Observation: Diagnose tests that rely on missing `CLINIC_RANGE_START` and `CLINIC_RANGE_END` must clear those variables and avoid parallel execution.
  Evidence: Subtask 2 quality review reproduced failures by setting those env vars before running default-scope and partial-scope tests.

- Observation: Unbracketed scalar fields can be followed by free-text log messages, so scalar anchor extraction cannot treat the rest of the line as the field value.
  Evidence: The plan fixture `regionID=6500 Request range exceeds bound` must produce `region_id=6500` for the Subtask 4 bridge.

- Observation: Error-text extraction needs the diagnose seed in addition to raw Loki lines.
  Evidence: `Request range exceeds bound` is the seed and a planned strong anchor, but a body-only normalizer cannot know to preserve that exact seed phrase.

- Observation: Rendering must be tested through the command handler, not only through lower-level helpers.
  Evidence: Subtask 5 parent review found `newDiagnoseCommand` still called the discard-output compatibility wrapper, so helper-level rendering tests passed while `clinic diagnose <seed>` would have printed no output.

- Observation: Empty JSON slices must be initialized explicitly for the public diagnose JSON contract.
  Evidence: A no-result JSON regression test showed `incidents` and `warnings` encoded as `null` until `DiagnoseResult` initialized them as empty slices.

- Observation: The legibility script and command surface docs were still missing `diagnose`, and the README only documented Loki examples.
  Evidence: `scripts/check-agent-legibility.sh` used `families=(auth env cluster data metrics loki events raw)`, `docs/COMMAND_SURFACE.md` had no `## diagnose` section, and the README Loki section had no diagnose example.

- Observation: Command-surface raw-line scoping must mention weak evidence as well as primary evidence.
  Evidence: Subtask 6 review found that JSON may include raw lines under both `incidents[].evidence[].line` and `incidents[].weak_evidence[].line`.

- Observation: A sub-threshold strong bridge could bypass the intended bridge threshold through union order.
  Evidence: Final review found a case where evidence with both `conn_id` and `region_id` first joined a connection group through a 10-point `conn_id` edge, then pulled in region evidence through an 8-point `region_id` edge because the previous guard only blocked sub-threshold edges when both roots already had size greater than one.

- Observation: `ParseDuration` is stricter than existing `now-<duration>` time expressions.
  Evidence: The diagnose-specific non-positive duration rule belongs to `--since` and `--window`, while existing shared time flags previously accepted `now-0s` as equivalent to `now`.


## Decision Log

- Decision: Keep the MVP stateless and register only `clinic diagnose <seed>`.
  Rationale: Stateful investigation IDs require a local cache and lifecycle design. The MVP must not make run-local handles like `I1` look persistent.
  Date/Author: 2026-04-21 / Codex

- Decision: Put command code in `internal/cmd` and keep `internal/clinic` transport-only.
  Rationale: Existing architecture places command parsing, validation, request shaping, and rendering in `internal/cmd`; `internal/clinic` only normalizes HTTP transport.
  Date/Author: 2026-04-21 / Codex

- Decision: Split implementation across focused `internal/cmd/diagnose_*.go` helpers while registering the command through `internal/cmd/diagnose.go`.
  Rationale: The feature has separable scope resolution, extraction, correlation, and rendering logic. Smaller files reduce context load without changing package boundaries.
  Date/Author: 2026-04-21 / Codex

- Decision: Implement one initial Loki substring query for the seed and do not add secondary follow-up Loki queries in the MVP.
  Rationale: The original diagnose notes scoped follow-up queries to later phases; MVP next steps are hints only.
  Date/Author: 2026-04-21 / Codex

- Decision: Add a small public duration parser in `internal/timeexpr` if needed for `--since` and `--window`.
  Rationale: Existing parsing already understands `d` as a 24-hour day for `now-<duration>`, but the command needs duration parsing without duplicating that logic in `diagnose.go`.
  Date/Author: 2026-04-21 / Codex

- Decision: Treat `clinic diagnose help` as a free-text seed rather than a help alias.
  Rationale: `diagnose` accepts a required free-text seed, and `help` can be a legitimate search term. Help remains available through `-h` and `--help`.
  Date/Author: 2026-04-21 / Codex

- Decision: Reject non-positive diagnose durations and non-increasing explicit time ranges locally.
  Rationale: `--since 0s`, `--window 0s`, or `--start >= --end` would create zero-width or inverted Loki ranges that are not useful for evidence correlation.
  Date/Author: 2026-04-21 / Codex

- Decision: Keep `normalizeDiagnoseEvidence(body []byte)` as the body-only helper and add a seed-aware internal normalization path for command execution.
  Rationale: The plan names the body-only helper for tests and reuse, while the command path needs the seed to set exact `error_text` anchors when a line contains the user's search seed.
  Date/Author: 2026-04-21 / Codex

- Decision: Document `diagnose` alongside the Loki examples and treat the command-surface docs as part of the user-visible contract.
  Rationale: The new command is a Loki-backed evidence workflow, so placing the example near Loki keeps the discoverability path aligned with how users will find and run it.
  Date/Author: 2026-04-21 / Codex

- Decision: Gate sub-threshold correlation edges by exact strong anchor/value continuity.
  Rationale: Edges below the bridge threshold should extend an existing component only through a strong anchor/value already used by that component. Introducing a new strong anchor into a component is bridge behavior and must reach the bridge threshold.
  Date/Author: 2026-04-21 / Codex

- Decision: Preserve `now-0s` and `now-0d` for shared time expressions while keeping `ParseDuration` positive-only.
  Rationale: Diagnose duration flags need positive durations, but `now-0s` is an existing shared time-expression alias for `now` used by non-diagnose commands.
  Date/Author: 2026-04-21 / Codex


## Outcomes & Retrospective

Subtask 1 is complete. The repository now has a registered `clinic diagnose <seed>` skeleton with help text, seed validation, output validation, root registration coverage, and no Loki request path yet. Remaining MVP behavior starts in Subtask 2 with scope resolution and seed Loki query construction.

Subtask 2 is complete. `clinic diagnose <seed>` now resolves cluster and time scope, defaults to the latest hour when no range resolves, supports `--since` and `--around/--window`, rejects conflicting or partial time modes, builds one escaped Loki substring query, and fetches the raw `query_range` response through the existing retry helper. The command still does not normalize or render incidents; that starts in Subtask 3.

Subtask 3 is complete. The command now decodes Loki stream responses into deterministic evidence records, preserves stream labels and raw lines, extracts generic fields and anchors, sets seed-aware error text, deduplicates entries, and assigns stable `E1`, `E2`, and later evidence IDs. Correlation and rendering are still intentionally absent and remain assigned to Subtasks 4 and 5.

Subtask 4 is complete. Evidence correlation now groups deterministic incident candidates with explainable score reasons, bridge thresholds, weak-only safeguards, aggregate anchors, sorted display IDs, and low-confidence warnings. The command computes a `DiagnoseResult` after fetching and normalizing Loki evidence, but table and JSON rendering remain assigned to Subtask 5.

Subtask 5 is complete. `clinic diagnose <seed>` now renders compact table output by default and structured JSON through the shared JSON writer. No-result and low-confidence searches return success with explanatory output, table rendering avoids raw log lines, JSON keeps raw lines under evidence entries, and command-level tests cover the output path.

Subtask 6 is complete. The command surface docs now include `diagnose`, the legibility script recognizes the new family, and the README shows diagnose examples beside the Loki material. The plan now records the docs-only verification step as complete and preserves the stateless/no-root-cause contract in the user-facing references.

Final review fixes are complete. Correlation now prevents a strong-but-sub-threshold bridge from introducing a new anchor family into an existing incident, while still allowing same-anchor direct groups and bridge-threshold merges. Shared `now-<duration>` parsing again accepts zero-duration aliases for `now`, while `ParseDuration` remains positive-only for diagnose `--since` and `--window` validation.


## Context and Orientation

This repository builds `clinic`, a Go CLI for TiDB Cloud Clinic APIs. The binary entrypoint is `cmd/clinic/main.go`, which delegates to `internal/cmd.Run`. Top-level command routing lives in `internal/cmd/root.go`. Command help text lives in `internal/cmd/help.go` and is treated as a user-facing contract.

The command package uses Cobra through local helpers in `internal/cmd/cobra_helpers.go`. Most command families expose a `new<Family>Command` function, a `<family>CommandHandler`, and one or more `run<Command>` functions. The root command registers runtime-backed families in `defaultRootRunner().handlers`.

Runtime context is represented by `internal/cmd.RuntimeContext`. It contains the selected environment, selected profile, config, `Client`, verbose flag, and diagnostics writer. Cluster selection should use `requiredFlagOrEnvWithContext(ctx, "cluster-id", value, envClinicClusterID)` so the existing precedence remains: explicit flag, then `CLINIC_CLUSTER_ID`, then selected profile `cluster_id` when a profile is active.

Existing time parsing helpers are:

    internal/cmd/time_parse.go:
      parseFlexibleTimeToUnixNano
      parseTimeFlagToUnixNano
      parseOptionalTimeFlagToUnixNano

    internal/timeexpr/timeexpr.go:
      ParseNowExpressionToUnixNano

`parseFlexibleTimeToUnixNano` accepts `now`, `now-<duration>`, datetime strings, and epochs in seconds, milliseconds, or nanoseconds. The `internal/timeexpr` package supports `d` as a 24-hour day in `now-<duration>` expressions.

Existing Loki helpers in `internal/cmd/loki.go` should be reused:

    buildDefaultLokiQuery
    applyDefaultLokiSelectorLabels
    executeLokiQueryRangeRawWithRetry
    lokiMaxLimit
    lokiQueryRangeResponse and related response structs

Do not shell out to `clinic loki query`. `diagnose` must execute the request in-process with the existing fake-client test pattern.

The fake client used by command tests is `recordingClinicClient` in `internal/cmd/cluster_test.go`. It records requests and returns queued `clinicResponse` values. Tests can avoid extra cluster lookup API calls by building a `RuntimeContext` with a selected profile whose `ClusterID`, `OrgID`, and `ClusterDeployTypeV2` are populated; `resolveClusterReferenceByIDWithMetadata` then resolves from the selected profile.

Important terms for this plan:

- Seed: the required free-text positional argument passed to `clinic diagnose`. It is used as a Loki substring filter.
- Evidence: one normalized Loki log entry with stream labels, timestamp, raw line, parsed fields, and extracted anchors.
- Anchor: an identifier extracted from evidence, such as `conn`, `txnStartTS`, `regionID`, SQL text, table names, exact error text, store address, keyspace, or stack frame.
- Strong anchor: an anchor that may create a correlation edge by itself when close enough in time, such as `conn`, `txnStartTS`, `regionID`, normalized SQL, table name plus close time, or exact error text plus close time.
- Weak anchor: context that can support a group but must not create one by itself, such as component, instance, namespace, broad error tokens, or timestamp proximity alone.
- Incident: a deterministic display group of correlated evidence. `I1`, `I2`, and similar IDs are stable only within one invocation.
- Score reason: structured evidence explaining why two or more evidence items were grouped.


## Scope

In scope for this MVP:

- Register `clinic diagnose <seed>`.
- Support `--cluster-id`, `--start`, `--end`, `--since`, `--around`, `--window`, `--limit`, `-o table`, and `--output table|json`.
- Treat `<seed>` as required free text.
- Resolve cluster ID through the existing flag/env/profile precedence.
- Resolve time scope through exactly one effective mode: `--start/--end`, `--since`, or `--around/--window`.
- Query Loki with one seed substring filter using the default Loki selector labels.
- Normalize Loki stream results into evidence.
- Extract common TiDB/TiKV anchors without requiring fixed categories.
- Correlate evidence with deterministic, explainable scoring.
- Render compact table output by default and structured JSON with raw evidence lines only under evidence entries.
- Return exit code 0 when the search ran successfully, including no-result and low-confidence cases.

Out of scope for this MVP:

- Anchor-specific seed flags.
- Stateful subcommands, investigation cache, saved investigation IDs, or lifecycle states.
- Secondary Loki follow-up queries.
- Root-cause inference.
- AI/model-style reasoning.
- Hard-coded category types such as `RequestRangeExceedsBoundIncident`.
- Printing raw log lines in table output.


## Target User Contract

The command surface is:

    clinic diagnose <seed> [--cluster-id <id>] [--start <time>] [--end <time>] [--since <duration>] [--around <time>] [--window <duration>] [--limit <n>] [-o table|json]

`<seed>` is required. More than one positional argument should be rejected by normal flag parsing unless the user quotes the free text as one argument. Missing seed returns a validation error.

Time scope rules:

When `--since <duration>` is present, set end to `now` and start to `now - duration`. `--since` overrides `CLINIC_RANGE_START`, `CLINIC_RANGE_END`, and selected profile range defaults.

When `--around <time> --window <duration>` is present, parse `--around` with the same accepted formats as `--start` and `--end`; set start to `around - window/2` and end to `around + window/2`. `--around/--window` overrides `CLINIC_RANGE_START`, `CLINIC_RANGE_END`, and selected profile range defaults.

When neither `--since` nor `--around/--window` is present, resolve `--start` and `--end` independently through explicit flag, `CLINIC_RANGE_*`, and selected profile `range_*`. If both remain unresolved, default to `--since 1h`.

Reject partial or conflicting modes. Examples: `--since` with `--start`, `--since` with `--around`, `--around` without `--window`, `--window` without `--around`, and only one of `--start` or `--end` after fallback resolution.

`--limit` defaults to `100`, must be greater than `0`, and must be less than or equal to `lokiMaxLimit` from `internal/cmd/loki.go`.

Output rules:

Default `table` output prints incident handles, timestamps, components, selected anchors, score, evidence count, and concise next-step guidance. It must not print raw log lines.

`json` output uses stable snake_case field names. It includes `query_scope` for every successful search. It includes raw log lines only under `incidents[].evidence[].line`.

No matches is success. Table output should say no incidents were found and show the searched cluster/start/end. JSON output should include an empty `incidents` array and empty `warnings`.

Matching evidence with no group above the incident threshold is also success. Table output should say evidence was found but no incident passed the correlation threshold. JSON output should include a warning.


## Interfaces and Dependencies

Create or modify these files:

    internal/cmd/diagnose.go
      New command family, flag parsing, scope resolution orchestration, Loki request construction, and runDiagnoseSearch pipeline.

    internal/cmd/diagnose_types.go
      Evidence, EvidenceSource, Anchors, DiagnoseResult, QueryScope, Incident, ScoreReason, and ExpansionHint types with explicit JSON tags.

    internal/cmd/diagnose_extract.go
      Loki response parsing, evidence normalization, bracket/key-value field parsing, generic anchor extraction, SQL normalization, table extraction, and deterministic evidence ID assignment.

    internal/cmd/diagnose_correlation.go
      Weighted graph or union-find correlation, scoring constants, score reasons, weak evidence handling, deterministic incident sorting, and incident ID assignment.

    internal/cmd/diagnose_render.go
      Table output and JSON output.

    internal/cmd/diagnose_test.go
      Command, scope, query, extraction, correlation, rendering, and integration tests.

    internal/cmd/root.go
      Register `diagnose` in `defaultRootRunner().handlers`.

    internal/cmd/help.go
      Add `diagnose` to `rootHelpText` and add `diagnoseHelpText`.

    internal/cmd/help_test.go
      Add diagnose command help coverage.

    internal/cmd/root_test.go
      Add root registration coverage if not sufficiently covered by help tests.

    internal/timeexpr/timeexpr.go
      Add a narrow exported duration parser only if `diagnose` needs one for `--since` and `--window`.

    internal/timeexpr/timeexpr_test.go
      Add tests for the exported duration parser if it is added.

    docs/COMMAND_SURFACE.md
      Add the `diagnose` command-family section and update command-change obligations.

    scripts/check-agent-legibility.sh
      Add `diagnose` to the required command-family list.

    README.md
      Add a short user-facing diagnose example near the Loki examples and update notes if needed.

Preferred data model shape:

    type Evidence struct {
        ID        string            `json:"id"`
        Time      time.Time         `json:"time"`
        Component string            `json:"component,omitempty"`
        Instance  string            `json:"instance,omitempty"`
        Line      string            `json:"line"`
        Fields    map[string]string `json:"fields,omitempty"`
        Anchors   Anchors           `json:"anchors,omitempty"`
        Source    EvidenceSource    `json:"source"`
    }

    type EvidenceSource struct {
        Stream map[string]string `json:"stream,omitempty"`
    }

    type Anchors struct {
        ConnID      string   `json:"conn_id,omitempty"`
        TxnStartTS  string   `json:"txn_start_ts,omitempty"`
        RegionID    string   `json:"region_id,omitempty"`
        StoreAddr   string   `json:"store_addr,omitempty"`
        Keyspace    string   `json:"keyspace,omitempty"`
        SQL         string   `json:"sql,omitempty"`
        TableNames  []string `json:"table_names,omitempty"`
        ErrorText   string   `json:"error_text,omitempty"`
        StackFrames []string `json:"stack_frames,omitempty"`
    }

    type DiagnoseResult struct {
        QueryScope QueryScope `json:"query_scope"`
        Incidents  []Incident `json:"incidents"`
        Warnings   []string   `json:"warnings"`
    }

    type QueryScope struct {
        ClusterID string    `json:"cluster_id"`
        Seed      string    `json:"seed"`
        StartTime time.Time `json:"start"`
        EndTime   time.Time `json:"end"`
        LogQL     string    `json:"logql"`
        Limit     int       `json:"limit"`
    }

    type Incident struct {
        ID                string          `json:"id"`
        Seed              string          `json:"seed"`
        Status            string          `json:"status"`
        StartTime         time.Time       `json:"start"`
        EndTime           time.Time       `json:"end"`
        Summary           string          `json:"summary"`
        Score             float64         `json:"score"`
        ScoreReasons      []ScoreReason   `json:"score_reasons,omitempty"`
        Anchors           Anchors         `json:"anchors,omitempty"`
        EvidenceCount     int             `json:"evidence_count"`
        WeakEvidenceCount int             `json:"weak_evidence_count"`
        Evidence          []Evidence      `json:"evidence,omitempty"`
        WeakEvidence      []Evidence      `json:"weak_evidence,omitempty"`
        NextSteps         []ExpansionHint `json:"next_steps,omitempty"`
    }

    type ScoreReason struct {
        EvidenceIDs []string `json:"evidence_ids"`
        Anchor      string   `json:"anchor"`
        Value       string   `json:"value,omitempty"`
        Points      float64  `json:"points"`
        Description string   `json:"description"`
    }

    type ExpansionHint struct {
        Name        string `json:"name"`
        Description string `json:"description"`
        Query       string `json:"query,omitempty"`
    }

The implementer may add unexported helper types as needed, but JSON field names above are the public contract and should stay stable.


## Plan of Work

### Subtask 1: command skeleton, registration, and help

Scope: make `clinic diagnose` a registered top-level command family with seed validation, output flag validation, no Loki request yet, and user-facing help.

Files:

    Create: internal/cmd/diagnose.go
    Create: internal/cmd/diagnose_test.go
    Modify: internal/cmd/root.go
    Modify: internal/cmd/help.go
    Modify: internal/cmd/help_test.go
    Modify: internal/cmd/root_test.go if needed

Implementation details:

Define `diagnoseCommandHandler(ctx *RuntimeContext, args []string, stdout io.Writer) error` following the pattern used by `lokiCommandHandler` and `eventsCommandHandler`.

Define `newDiagnoseCommand(ctx *RuntimeContext, stdout io.Writer) *cobra.Command`. The MVP is a single stateless leaf-like command, so the command can parse its own args directly rather than adding subcommands. It should render `diagnoseHelpText` when called with no args only if no seed is present and the user used help; otherwise missing seed is a validation error.

Register `"diagnose": diagnoseCommandHandler` in `defaultRootRunner().handlers`.

Add `diagnose` to `rootHelpText` available commands.

Add `diagnoseHelpText` documenting:

    clinic [--env <env>] [--profile <name>] diagnose <seed> [flags]

Include flags, time modes, output formats, no-root-cause language, and examples.

Initial tests to add in `internal/cmd/diagnose_test.go`:

    TestDiagnoseRequiresSeed
    TestDiagnoseRejectsUnexpectedExtraArgs
    TestDiagnoseRejectsInvalidOutput
    TestDiagnoseHelpRenders

Update `TestCommandHelpRendersForEveryCommandFamily` in `internal/cmd/help_test.go` with a diagnose case. Update root help assertions so root help includes diagnose.

Acceptance:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run 'Diagnose|Help|Root' -v

Expected result: tests pass. At this stage, tests should only assert skeleton validation/help behavior; they should not require Loki requests or incident output.

Commit:

    git add internal/cmd/diagnose.go internal/cmd/diagnose_test.go internal/cmd/root.go internal/cmd/help.go internal/cmd/help_test.go internal/cmd/root_test.go
    git commit -m "feat(cmd): add diagnose command skeleton"


### Subtask 2: scope resolution and seed Loki query construction

Scope: implement cluster/time/limit/output scope resolution and construct one Loki query_range request for the seed. The command may still return an empty result after fetching.

Files:

    Modify: internal/cmd/diagnose.go
    Modify: internal/cmd/diagnose_test.go
    Modify: internal/timeexpr/timeexpr.go if needed
    Modify: internal/timeexpr/timeexpr_test.go if needed

Implementation details:

Create an unexported `diagnoseOptions` type containing seed, cluster ID flag, start/end flags, since, around, window, limit, and output.

Create an unexported `diagnoseScope` type or reuse `QueryScope` once Subtask 3 creates shared types. It must hold resolved cluster ID, seed, start/end as `time.Time`, LogQL query, and limit.

Resolve cluster ID with:

    requiredFlagOrEnvWithContext(ctx, "cluster-id", flagValue, envClinicClusterID)

Implement time resolution with a testable helper that accepts `now time.Time`, for example:

    resolveDiagnoseTimeScope(ctx, opts, now) (start time.Time, end time.Time, err error)

The production path calls it with `time.Now()`.

For `--since` and `--window`, prefer adding this helper in `internal/timeexpr/timeexpr.go`:

    func ParseDuration(value string) (time.Duration, error)

It should accept `time.ParseDuration` syntax plus `d` as a 24-hour day, reject empty values, and reject negative durations. Reuse the existing private day-unit logic instead of duplicating it in `internal/cmd`.

Build the seed LogQL with the existing default selector:

    selector, err := buildDefaultLokiQuery(ctx, resolvedClusterID)
    logql := selector + " |= " + strconv.Quote(seed)

Use `strconv.Quote` or equivalent `%q` formatting to escape the seed as a LogQL string literal. Do not use regex LogQL in the MVP.

Create the request:

    Method: GET
    Path: /data-proxy/loki/<cluster-id>/api/v1/query_range
    Query: query=<logql>, start=<unix ns>, end=<unix ns>, limit=<limit>

Use `executeLokiQueryRangeRawWithRetry(ctx, req)` to fetch raw body bytes.

Tests to add:

    TestDiagnoseBuildsSeedLokiQueryWithDefaultSince
    TestDiagnoseSinceOverridesEnvAndProfileRange
    TestDiagnoseUsesExplicitStartEnd
    TestDiagnoseUsesEnvAndProfileStartEndFallback
    TestDiagnoseDefaultsToSinceOneHourWhenNoRangeResolves
    TestDiagnoseAroundWindowBuildsCenteredRange
    TestDiagnoseRejectsConflictingTimeModes
    TestDiagnoseRejectsPartialTimeModes
    TestDiagnoseLimitDefaultAndBounds
    TestDiagnoseEscapesSeedInLogQL
    TestDiagnoseRetriesGatewayTimeoutThroughLokiHelper

Use `RuntimeContext{SelectedEnv: "staging", SelectedProfile: &config.ProfileRecord{ClusterID: "cluster-42", OrgID: "org-1", ClusterDeployTypeV2: "dedicated"}, Client: client}` to avoid cluster lookup unless a test explicitly needs fallback behavior.

Acceptance:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/timeexpr -v
    go test ./internal/cmd -run 'Diagnose.*(Scope|Query|Time|Since|Around|Window|Limit|Escape|Retry)' -v

Expected result: tests pass. The recorded Loki request should use nanosecond start/end values and the expected `query_range` path.

Commit:

    git add internal/cmd/diagnose.go internal/cmd/diagnose_test.go internal/timeexpr/timeexpr.go internal/timeexpr/timeexpr_test.go
    git commit -m "feat(cmd): resolve diagnose query scope"


### Subtask 3: Loki normalization and generic anchor extraction

Scope: parse raw Loki JSON into deterministic evidence items and extract anchors without diagnosis categories.

Files:

    Create: internal/cmd/diagnose_types.go
    Create: internal/cmd/diagnose_extract.go
    Modify: internal/cmd/diagnose.go
    Modify: internal/cmd/diagnose_test.go

Implementation details:

Move shared result structs into `diagnose_types.go` with explicit JSON tags as described in `Interfaces and Dependencies`.

Add `normalizeDiagnoseEvidence(body []byte) ([]Evidence, error)` in `diagnose_extract.go`. It should unmarshal `lokiQueryRangeResponse`, reject unexpected non-empty `ResultType` values that are not `streams`, and skip malformed entries with fewer than two Loki value fields.

For each Loki value, parse timestamp from nanosecond text into `time.Time` with `time.Unix(0, ns).UTC()`. Preserve the exact raw log line after trimming only trailing CR/LF. Preserve stream labels in `Evidence.Source.Stream`.

Populate `Evidence.Component` from stream label `component` first, then best-effort parsing from the line only if the label is absent. Populate `Evidence.Instance` from stream label `instance` first.

Parse generic fields from log lines:

- Bracket fields such as `[conn=281072972]`, `[regionID=6500]`, `[sql=select ...]`.
- Key-value fields such as `conn=281072972`, `txnStartTS=450000`, `regionID=6500`, `storeAddr=10.0.0.1:20160`, and `keyspaceName=foo`.
- Preserve unknown fields in `Evidence.Fields` instead of discarding them.

Extract anchors using generic rules:

- `conn`, `connID`, `conn_id` become `Anchors.ConnID`.
- `txnStartTS`, `txn_start_ts` become `Anchors.TxnStartTS`.
- `regionID`, `region_id`, `region` become `Anchors.RegionID` when the value is numeric.
- `storeAddr`, `store_addr` become `Anchors.StoreAddr`.
- `keyspaceName`, `keyspace` become `Anchors.Keyspace`.
- SQL is best-effort. Normalize whitespace, lowercase SQL keywords by lowercasing the whole string, and preserve placeholders. Do not try to parse full SQL grammar.
- Table names are best-effort from normalized SQL for simple `from <table>`, `join <table>`, `update <table>`, and `into <table>` forms.
- Error text is the seed if the raw line contains the seed, otherwise a short normalized error-looking substring when present. Do not create fixed categories.
- Stack frames are best-effort lines containing `.go:<line>` or function-looking frames.

Deduplicate evidence by stream-label stable key, timestamp, and exact raw line. Sort deduplicated evidence by time ascending, stable stream-label key ascending, then raw line ascending. Assign deterministic IDs `E1`, `E2`, `E3` after sorting.

Tests to add:

    TestDiagnoseNormalizesLokiStreamsIntoEvidence
    TestDiagnoseRejectsMalformedLokiJSON
    TestDiagnoseRejectsUnexpectedLokiResultType
    TestDiagnoseDeduplicatesAndAssignsStableEvidenceIDs
    TestDiagnoseExtractsBracketFields
    TestDiagnoseExtractsKnownKVAnchors
    TestDiagnosePreservesUnknownFields
    TestDiagnoseNormalizesSQLAndExtractsTables
    TestDiagnoseExtractsStackFramesBestEffort

Acceptance:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run 'Diagnose.*(Normalize|Evidence|Extract|Anchor|SQL|Stack|Loki)' -v

Expected result: tests pass. Evidence IDs are stable across repeated runs with the same fixture, and no test depends on a hard-coded error category.

Commit:

    git add internal/cmd/diagnose_types.go internal/cmd/diagnose_extract.go internal/cmd/diagnose.go internal/cmd/diagnose_test.go
    git commit -m "feat(cmd): extract diagnose evidence anchors"


### Subtask 4: deterministic correlation and incident scoring

Scope: group evidence into incident candidates with explainable score reasons and deterministic incident IDs.

Files:

    Create: internal/cmd/diagnose_correlation.go
    Modify: internal/cmd/diagnose.go
    Modify: internal/cmd/diagnose_test.go

Implementation details:

Implement correlation as a weighted graph or union-find over evidence nodes. A node is one `Evidence`. An edge exists only when the relationship is explainable through score reasons.

Use these starting scoring constants:

    same conn_id within window:        +10
    same txn_start_ts:                 +10
    same region_id:                    +8
    same normalized SQL:               +6
    same table name:                   +4
    same exact error text in window:   +4
    same instance as weak support:     +2
    broad error token as weak support: +1
    timestamps within 5 seconds:       +5
    timestamps within 60 seconds:      +2

Use these safeguards:

    minimum direct incident edge: 8
    minimum bridge edge: 12
    weak-only edge: never creates a group
    default correlation window: 5 minutes
    maximum incident time span: no more than the effective query window, and preferably bounded by the default correlation window for grouping decisions

Treat `conn_id`, `txn_start_ts`, `region_id`, normalized SQL, table names, and exact error text as strong anchors. Treat component, instance, namespace, broad words, and timestamp proximity as weak support only.

Bridge evidence can merge partial groups only when it has strong relationships to both sides and the bridge relationship reaches the bridge threshold. For example, a TiDB log with both `conn_id=281072972` and `region_id=6500` can connect a TiDB-side connection group to a TiKV-side region group if the combined score reasons meet the threshold.

Every score that affects grouping must produce a `ScoreReason` with evidence IDs, anchor name, value, points, and human-readable description. If the implementation cannot explain why evidence belongs together, it must not group that evidence.

Build incident aggregate anchors from the most frequent non-empty anchor values in grouped evidence. Set `Status` to `correlated`. Set `StartTime` and `EndTime` from grouped evidence. Generate `Summary` from the strongest anchors using careful language such as "shares conn_id and region_id anchors with nearby logs"; do not say "caused by".

Sort incidents before assigning IDs:

1. score descending,
2. evidence count descending,
3. start time ascending,
4. stable anchor key lexicographically.

Assign `I1`, `I2`, `I3` after sorting. IDs are display handles only.

Tests to add with focused fixtures:

    TestDiagnoseCorrelatesConnRegionAndSQL
    TestDiagnoseDoesNotMergeWeakOnlyEvidence
    TestDiagnoseBridgeEvidenceRequiresThreshold
    TestDiagnoseKeepsNoisyEvidenceSeparate
    TestDiagnoseRecordsScoreReasonsForGrouping
    TestDiagnoseSortsIncidentsDeterministically
    TestDiagnoseAssignsIncidentIDsAfterSorting
    TestDiagnoseLowConfidenceEvidenceReturnsWarning

Use a range-bound-style fixture with at least these entries:

    E1: TiDB seed error, conn=281072972, sql=S, time=15:44:59
    E2: TiKV log, regionID=6500, same error text, time=15:45:02
    E3: TiDB log, conn=281072972, regionID=6500, time=15:45:03
    E4: unrelated TiDB warning, same component only, time=15:45:04

Expected grouping: E1 and E3 group through `conn_id`; E2 joins through E3 with `region_id` and time proximity; E4 stays separate or weak. The incident must not claim root cause.

Acceptance:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run 'Diagnose.*(Correlate|Incident|Score|Bridge|Weak|Sort|Confidence)' -v

Expected result: tests pass. Score reasons are present for every grouping edge used by the fixture.

Commit:

    git add internal/cmd/diagnose_correlation.go internal/cmd/diagnose.go internal/cmd/diagnose_test.go
    git commit -m "feat(cmd): correlate diagnose evidence"


### Subtask 5: table and JSON rendering

Scope: implement the final user-visible output contracts and connect the complete pipeline.

Files:

    Create: internal/cmd/diagnose_render.go
    Modify: internal/cmd/diagnose.go
    Modify: internal/cmd/diagnose_test.go

Implementation details:

`runDiagnoseSearch` should now execute the deterministic pipeline:

    resolve scope
    build seed LogQL
    fetch raw Loki JSON
    normalize evidence
    extract anchors
    correlate incidents
    render table or JSON

Use `writeJSON` from `internal/cmd/root.go` for structured JSON output so HTML escaping behavior stays consistent with the rest of the CLI.

Default table output should be compact. For non-empty incidents, print at most 10 incidents. Include seed, incident count, ID, start time in RFC3339, component summary, selected anchors such as conn and region, evidence count, score, and one summary line. End with next-step guidance that only mentions registered capabilities:

    rerun with -o json to inspect score reasons and evidence
    use query_scope.logql with clinic loki query when raw logs are needed

Do not print commands for nonexistent stateful diagnose subcommands.

For no incidents and no evidence:

    no incidents found for "Request range exceeds bound"
    searched cluster=123 start=2026-04-20T14:44:59Z end=2026-04-20T16:44:59Z

For evidence below confidence threshold:

    evidence found for "Request range exceeds bound", but no incident passed the correlation threshold
    searched cluster=123 start=... end=...

JSON output must include:

    query_scope.cluster_id
    query_scope.seed
    query_scope.start
    query_scope.end
    query_scope.logql
    query_scope.limit
    incidents
    warnings

JSON output can include raw log lines, but only under `incidents[].evidence[].line` or `incidents[].weak_evidence[].line`. Do not duplicate raw lines in summaries, score reason descriptions, warnings, or query scope.

Tests to add:

    TestDiagnoseTableRendersIncidentHandles
    TestDiagnoseTableDoesNotPrintRawLogLines
    TestDiagnoseJSONIncludesQueryScope
    TestDiagnoseJSONIncludesScoreReasonsAndEvidence
    TestDiagnoseJSONKeepsRawLinesOnlyUnderEvidence
    TestDiagnoseNoResultsExitSuccess
    TestDiagnoseLowConfidenceOutputExitSuccess
    TestDiagnoseMalformedLokiResponseReturnsError
    TestDiagnoseClinicAPIErrorReturnsError

Acceptance:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run 'Diagnose' -v

Expected result: all diagnose tests pass. Table tests prove raw log lines are absent from table output. JSON tests prove raw lines are scoped to evidence entries.

Commit:

    git add internal/cmd/diagnose_render.go internal/cmd/diagnose.go internal/cmd/diagnose_test.go
    git commit -m "feat(cmd): render diagnose incidents"


### Subtask 6: docs, legibility, and full verification

Scope: update agent/user docs and run the repository verification ladder.

Files:

    Modify: docs/COMMAND_SURFACE.md
    Modify: scripts/check-agent-legibility.sh
    Modify: README.md
    Modify: docs/2026-04-21-diagnose-plan.md

Implementation details:

Add a `## diagnose` section to `docs/COMMAND_SURFACE.md` near the other command-family sections:

    ## diagnose

    - Source: `internal/cmd/diagnose.go` plus focused helpers `internal/cmd/diagnose_*.go`
    - Help text: `internal/cmd/help.go` (`diagnoseHelpText`)
    - Typical tests: `internal/cmd/diagnose_test.go`
    - Note: `diagnose` is a stateless evidence-correlation command. It does not infer root cause, create saved investigations, or register stateful subcommands in the MVP.
    - Note: table output is compact and does not print raw log lines; JSON output includes raw lines only under evidence entries.

Add `diagnose` to the `families=(...)` array in `scripts/check-agent-legibility.sh`.

Add a short README example near the Loki section:

    Diagnose correlated evidence:

    ./bin/clinic diagnose "Request range exceeds bound" --cluster-id 123456789 --since 24h
    ./bin/clinic diagnose "Request range exceeds bound" --cluster-id 123456789 --since 24h -o json

Document that diagnose does not infer root cause and table output omits raw log lines.

Update this plan's `Progress`, `Surprises & Discoveries`, `Decision Log`, and `Outcomes & Retrospective` after implementation. The update must be part of this subtask commit.

Validation commands:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run 'Diagnose|Help|Root' -v
    go test ./internal/timeexpr -v
    make legibility-check
    make test
    make verify

Expected results:

    go test ./internal/cmd -run 'Diagnose|Help|Root' -v
      PASS

    go test ./internal/timeexpr -v
      PASS

    make legibility-check
      agent legibility checks passed

    make test
      go test ./... -v completes successfully

    make verify
      legibility checks and tests complete successfully

Commit:

    git add docs/COMMAND_SURFACE.md scripts/check-agent-legibility.sh README.md docs/2026-04-21-diagnose-plan.md
    git commit -m "docs: document diagnose command surface"


## Concrete End-to-End Scenario

Use this fixture shape in tests to prove the command works end to end with the fake client:

    {
      "status": "success",
      "data": {
        "resultType": "streams",
        "result": [
          {
            "stream": {"component":"tidb","instance":"tidb-0"},
            "values": [
              ["1775181899000000000","[conn=281072972] [sql=select * from db1.t_ux_3 where id = ?] Request range exceeds bound"]
            ]
          },
          {
            "stream": {"component":"tikv","instance":"tikv-0"},
            "values": [
              ["1775181902000000000","regionID=6500 Request range exceeds bound"]
            ]
          },
          {
            "stream": {"component":"tidb","instance":"tidb-0"},
            "values": [
              ["1775181903000000000","[conn=281072972] [regionID=6500] retrying request"]
            ]
          },
          {
            "stream": {"component":"tidb","instance":"tidb-1"},
            "values": [
              ["1775181904000000000","background warning with no shared strong anchor"]
            ]
          }
        ]
      }
    }

A command invocation like this should produce one high-confidence incident and leave the weak warning out of the main group:

    clinic diagnose "Request range exceeds bound" --cluster-id cluster-42 --start 1775178299 --end 1775185499 -o json

Expected properties:

- Request path is `/data-proxy/loki/cluster-42/api/v1/query_range`.
- Request query contains `{cluster_env="staging", namespace="tidbcluster-42"} |= "Request range exceeds bound"`.
- JSON `query_scope.seed` is `Request range exceeds bound`.
- JSON `incidents[0].id` is `I1`.
- JSON `incidents[0].status` is `correlated`.
- JSON `incidents[0].anchors.conn_id` is `281072972`.
- JSON `incidents[0].anchors.region_id` is `6500`.
- JSON `incidents[0].score_reasons` includes reasons for `conn_id` and `region_id`.
- JSON does not state a root cause.


## Validation and Acceptance

The feature is accepted when these commands pass from the repository root:

    cd /Users/jujiajia/code/tidbcloud/clinic-cli
    go test ./internal/cmd -run Diagnose -v
    go test ./internal/cmd -run 'Help|Root' -v
    go test ./internal/timeexpr -v
    make legibility-check
    make verify

Behavioral acceptance:

- `clinic diagnose --help` documents the MVP command and flags.
- `clinic diagnose` without a seed returns a validation error.
- `clinic diagnose "seed" --cluster-id 123 --since 24h` sends one Loki query_range request with a safely escaped substring filter.
- `clinic diagnose "seed" --cluster-id 123 --start <time> --end <time>` respects explicit range flags.
- `clinic diagnose "seed" --cluster-id 123 --around <time> --window 10m` searches a centered window.
- Conflicting or partial time modes fail with clear validation errors.
- Empty Loki results return success with no incidents.
- Low-confidence evidence returns success with a warning.
- Correlated evidence returns deterministic incidents, score reasons, and display IDs.
- Table output does not print raw log lines.
- JSON output includes raw log lines only under evidence entries.
- Existing `clinic loki query` behavior remains unchanged.


## Idempotence and Recovery

All validation commands are safe to rerun. Fake-client tests should not contact external services.

Before each subtask, run `git status --short` and review any dirty files. Do not revert unrelated user changes. If the worktree contains changes in files touched by the current subtask, inspect them and integrate with them rather than overwriting them.

If a subtask test fails, stop within that subtask. Do not proceed to the next subtask or commit until the failure is understood and fixed.

If a commit command fails because unrelated files are dirty, stage only the files listed for the current subtask. Do not use `git reset --hard` or `git checkout --` to clean the tree.

If the diagnose implementation accidentally changes `clinic loki query` behavior, restore the previous behavior by updating the new diagnose-specific helper rather than weakening existing Loki tests.


## Artifacts and Notes

This plan is the diagnose implementation and design-history source of truth. If future work discovers a mismatch between this plan and current repository behavior, update `Surprises & Discoveries` and `Decision Log` before changing code.

Revision note: Subtask 6 completed the repository docs and legibility updates, so the plan now reflects the final docs-visible diagnose contract and the new command-family coverage in the legibility script.

The MVP intentionally emits hints, not follow-up commands for nonexistent features. Saved investigations, lifecycle states, and local cache paths belong to a separate post-MVP plan.

Post-MVP backlog notes preserved from the original design:

- Future work can add anchor-specific seed modes, but they can remain stateless and should not imply persistent investigation handles.
- Stateful investigation commands need saved investigation state first. Store the original query, evidence, anchors, and follow-up hints before adding lifecycle commands.
- If saved investigations are introduced, use explicit persistent IDs such as `diag-20260421-001`; do not make run-local handles like `I1`, `I2`, or `I3` look persistent across independent stateless runs.
- A future cache path may use `~/.config/clinic/investigations/<investigation-id>.json`, but hidden persistence is out of scope for the MVP and requires its own design.
- A future lifecycle can use explicit states such as `OPEN -> CORRELATED -> EXPLAINED -> ACTIONABLE -> CLOSED`, but only after saved investigation state exists.

Optional rule packs should be explainers, not categories. Incidents must still work without any explainer. A future interface can look like:

    type Explainer interface {
        Name() string
        Match(Incident) bool
        Explain(Incident) Explanation
    }

Example explainer names from the original design were `RegionSplitNearError`, `SameErrorInTiDBAndTiKV`, `DDLBeforeQuery`, `PanicStackPresent`, `DataProxyUnavailable`, and `StartupVersion`. These should produce helpful annotations only; the durable object remains the incident.

The durable principle is that `clinic diagnose` should not try to know every TiDB error. It should make unknown errors diagnosable by finding seed evidence, extracting reusable anchors, grouping related evidence, and returning follow-up hints through deterministic, testable rules.