# Expected Diagnosis: O11Y-2190994

## Root Cause

Compound root cause: Business read load surged 13.5x (SELECT time 199→2696 s/s), triggering EBS IOPS throttling on 58 volumes (peak 39.6%). db-tikv-19 became the hotspot TiKV instance (coprocessor P99 22.49s). Multiple `SELECT *` queries on `dh_active_receive_logs` table (40,865 executions, avg 4.77s) were the primary application-layer driver.

## Evidence Chain

1. **DB Time Model**: SELECT time 199→2696 s/s (+1253%) — read load surge as trigger
2. **EBS metrics**: 58 volumes >20% throttling, peak 39.6% — infrastructure bottleneck
3. **TiKV coprocessor**: db-tikv-19 P99 latency 22.49s at 03:05 UTC — hotspot instance
4. **Slow queries**: `dh_active_receive_logs` 40,865 exec, avg 4.77s — SQL anti-patterns (SELECT *, no index)
5. **Timeline**: TiKV spikes at 03:05, P999 alert at 03:07 — causal sequence confirmed

## Causality

Business read surge → dense Coprocessor requests to TiKV → EBS IOPS limit triggered → db-tikv-19 hotspot (coprocessor P99 22.49s) → TiDB P999 latency 5.77s → alert.

## Severity

High — customer-visible P999 spike to 5.77s (10x baseline); auto-recovered.

## Recommendations

1. Optimize `dh_active_receive_logs` query: replace `SELECT *`, add composite index
2. Upgrade EBS volumes to io2/gp3 with provisioned IOPS
3. Investigate 03:05-03:20 UTC business traffic surge origin
4. Deploy TiFlash for OLAP-style aggregate queries on `dh_*` tables

## Scoring Rubric

> Checklist for regression testing. The Agent must satisfy these to pass.

- [ ] Identifies EBS IOPS throttling as root cause contributor (58 volumes, >20%)
- [ ] Identifies SELECT time surge (+1253%) as application-layer root cause
- [ ] Identifies db-tikv-19 as hotspot TiKV instance by coprocessor latency
- [ ] Identifies `dh_active_receive_logs` (or similar) as top slow query table
- [ ] Provides causal chain linking read surge → EBS throttle → TiKV latency → P999 spike
- [ ] Recommends index creation and/or SELECT * optimization