# Cloud Storage Engine Grafana Dashboard

TiKV cluster monitoring dashboards, generated as Grafana JSON from Python code
using [grafanalib](https://github.com/weaveworks/grafanalib).

Please avoid manually modifying the generated `.json` files.

## Generate Dashboard JSON

The standard dashboard JSON and checksum are tracked and validated by
`scripts/check-dashboards`. Serverless JSON and checksums are generated by the
same command and ignored by Git. When present locally, their checksums are also
validated by `scripts/check-dashboards`.

Two generation options are available.

### Option 1: via the repo helper script (Docker)

```bash
# Generate all dashboards
./scripts/gen-tikv-details-dashboard
```

The script also runs `isort` and `black` before generation. Serverless outputs
are local artifacts and must not be committed.

### Option 2: via uv (no Docker required)

[uv](https://docs.astral.sh/uv/) manages the Python virtual environment and
dependencies.

#### Setup

```bash
cd metrics/grafana

# Create venv and install dependencies (first time only)
uv sync
```

#### Generate

```bash
# Format Python sources
.venv/bin/isort --profile black *.py
.venv/bin/black *.py

# Generate tracked JSON
.venv/bin/generate-dashboard -o tikv_details.json tikv_details.dashboard.py
python shared_resource_diagnosis.py -o shared_resource_diagnosis.json

# Optionally generate local serverless JSON
.venv/bin/generate-dashboard -o tikv_details_serverless.json tikv_details_serverless.dashboard.py
.venv/bin/generate-dashboard -o tikv_worker_serverless.json tikv_worker_serverless.dashboard.py

# Checksums (run from repo root to match the Docker-generated path prefix)
cd ../..
sha256sum ./metrics/grafana/tikv_details.json > metrics/grafana/tikv_details.json.sha256
sha256sum ./metrics/grafana/shared_resource_diagnosis.json > metrics/grafana/shared_resource_diagnosis.json.sha256
sha256sum ./metrics/grafana/tikv_details_serverless.json > metrics/grafana/tikv_details_serverless.json.sha256
sha256sum ./metrics/grafana/tikv_worker_serverless.json > metrics/grafana/tikv_worker_serverless.json.sha256
```

Commit the generated `tikv_details` and `shared_resource_diagnosis` JSON and
checksum files. The serverless JSON and checksum files are ignored local
outputs.

#### Adding Dependencies

```bash
uv add <package-name>
```

## Files

| File | Description |
|------|-------------|
| `common.py` | Shared helpers: `graph_panel`, `target`, `expr_sum_rate`, `Layout`, etc. |
| `tikv_details.dashboard.py` | TiKV Details main dashboard |
| `tikv_details.json` | Generated JSON — do not edit manually |
| `tikv_details.json.sha256` | SHA256 checksum of `tikv_details.json` (generated alongside the JSON) |
| `tikv_details_serverless.dashboard.py` | Serverless TiKV Details wrapper and template variables |
| `tikv_details_serverless.json` | Local serverless JSON — ignored by Git |
| `tikv_details_serverless.json.sha256` | Local checksum of `tikv_details_serverless.json` — ignored by Git and checked when present |
| `tikv_worker_serverless.dashboard.py` | Serverless TiKV Worker dashboard |
| `tikv_worker_serverless.json` | Local serverless TiKV Worker JSON — ignored by Git |
| `tikv_worker_serverless.json.sha256` | Local checksum of `tikv_worker_serverless.json` — ignored by Git and checked when present |
| `pyproject.toml` | Project metadata and dependency declarations |
| `shared_resource_diagnosis.py` | Clinic SharedScope Shared Resource One Page dashboard generator |
| `shared_resource_diagnosis.json` | Generated Clinic SharedScope dashboard template |
| `shared_resource_diagnosis.json.sha256` | SHA256 checksum of the generated dashboard |

## Shared Resource One Page

This is the source template for Clinic's SharedScope dashboard. Clinic injects
the hidden physical `tidb_cluster` routing variable and uses the hidden
`shared_resource_scope=resource_pool` marker to enable resource-pool metric
routing. The dashboard connects logical-cluster TiDB service health, query
latency, RU demand, and logical coprocessor response throughput with physical
TiKV CPU, IO, storage wait, unified read-pool, and keyspace-fairness signals.
Use `cluster_id`, `tikv_instance`, and `keyspace_id` for attribution and
drill-down after opening the dashboard through a Clinic Shared Resource link.

All-cluster and all-keyspace views use bounded Top-N queries where appropriate.
For high-cardinality production deployments, precompute the expensive overview
queries with recording rules rather than relying on dashboard Top-N alone.

## Keyspace fair read-pool panels

`tikv_details` includes a **Keyspace Fair Queue** section for attributing YATP
unified read-pool work to a keyspace. It is a TiKV/physical-cluster view: first
select the physical `tidb_cluster`, then narrow with `instance` and
`keyspace_id` as needed. `keyspace_id` is the numeric keyspace identifier, not
a tenant display name.

The panels are emitted only when the YATP keyspace-fair queue and its metrics
are enabled. A typical configuration is:

```toml
[readpool.unified]
use-tokio = false
enable-yatp-keyspace-metrics = true

[readpool.unified.keyspace-fairness]
enabled = true

[storage]
use-separated-scheduler-pool = true
```

The section shows poll CPU/wall-time rate, fair-queue slice wait duration,
queue depth, waited task slices, and active groups per TiKV. Use a single
keyspace and a bounded set of instances for drill-down. For a tenant overview,
aggregate by `keyspace_id` and apply a Top-N limit; do not render every
keyspace × instance combination in one panel.

These metrics identify the keyspace and TiKV instance consuming or waiting for
read-pool time. They do not identify an SQL digest. Correlate a selected time
range with TiDB TopSQL, statement summary, tracing, or exemplars rather than
adding SQL-digest labels to Prometheus metrics.