# RU Limit Inspection PromQL Reference

Use these as templates only when the bundled collector cannot be used or needs adaptation.
For the standard Dedicated/Classic path, `scripts/collect-rc-report.mjs` is the
executable source of truth for query definitions. Adapt labels to the dashboard profile and
keep query windows bounded. Prefer `query_range` for report data.

## Contents

- [Label Profiles](#label-profiles)
- [Resource Group Configuration](#resource-group-configuration)
- [RU/s Rate](#rus-rate)
- [RU Max-Per-Second](#ru-max-per-second)
- [Query Duration, QPS, and RC Wait](#query-duration-qps-and-rc-wait)
- [DDL and Analyze Progress](#ddl-and-analyze-progress)
- [Global Impact and Resource Usage](#global-impact-and-resource-usage)
- [Triggered TiKV Tail-Latency Queries](#triggered-tikv-tail-latency-queries)
- [Query Safety](#query-safety)

## Label Profiles

These templates implement `dedicated-classic-v1`. Do not adapt them to Starter/Essential or
Premium/BYOC by changing labels alone; those products require different metric and RU-limit
contracts.

Common label patterns:
- On-prem or Grafana dashboard: `k8s_cluster`, `tidb_cluster`, sometimes `job`.
- TiDB Cloud / O11Y: `tidb_cluster_id` plus `component="tidb"` or `component="tikv"`.
- RU-limit metrics may need labels inferred from `resource_manager_server_group_config`.
- Keep logical TiDB matchers separate from component TiDB matchers. Logical resource-group
  queries may include keyspace labels; process CPU metrics usually do not. Use
  `$tidb_component_matchers` for `process_cpu_seconds_total`.
- The bundled collector does not add `job` to logical TiDB resource-group queries by default.
  It adds default component matchers for CPU and TiKV tail queries: `job="tidb"` and
  `job=~".*tikv"`. Override them with `--tidb-extra-matcher`, `--tikv-extra-matcher`,
  `--tidb-component-matchers`, or `--tikv-matchers` when the datasource uses a different component label.
  Full component matcher overrides must preserve the exact selected cluster matchers. A full
  TiDB logical override must also preserve the selected keyspace matcher.
- Keyspace-aware dashboards may expose `keyspace_name`, `namespace`, or another keyspace label.
  Add that matcher to RU-limit and TiDB selectors when same-named resource groups can
  exist in more than one keyspace. Keep TiKV selectors cluster-scoped unless those TiKV metrics
  actually carry the keyspace label.

Use a normal chart step for broad 24h trends and a smaller step for RU-limit KPI max and
configuration history. For a 24h report, `step=240s` for normal charts and `20s` for
RU/configuration evidence is a reasonable starting point when scrape interval supports it. Do not
lower every query to the KPI step.

## Resource Group Configuration

```promql
max by (resource_group, type) (
  resource_manager_server_group_config{
    $cluster_matchers,
    type=~"ru_per_sec|priority|ru_capacity",
    resource_group=~"$target_resource_groups"
  }
)
```

## RU/s Rate

Use the standard Grafana `RU` panel scope, adjusted by refund counters. Define `$read_net_ru` as:

```promql
clamp_min(
  sum(rate(resource_manager_resource_unit_read_request_unit_sum{
    $cluster_matchers,
    type=~"|tp",
    resource_group=~"$report_resource_groups"
  }[1m])) by (resource_group)
  -
  (
    sum(rate(resource_manager_resource_unit_read_request_unit_refund_sum{
      $cluster_matchers,
      type=~"|tp",
      resource_group=~"$report_resource_groups"
    }[1m])) by (resource_group)
    or
    sum(rate(resource_manager_resource_unit_read_request_unit_sum{
      $cluster_matchers,
      type=~"|tp",
      resource_group=~"$report_resource_groups"
    }[1m])) by (resource_group) * 0
  ),
  0
)
```

Define `$write_net_ru` with the same expression using the write request/refund metrics, then use
this union-safe composition so a group with only one side is retained:

```promql
(($read_net_ru) or ($write_net_ru) * 0)
+
(($write_net_ru) or ($read_net_ru) * 0)
```

Compute `RU/s Max`, utilization, headroom, and over-limit duration from the range result.
The `type=~"|tp"` filter follows the standard Grafana `RU` panel used by this skill. Keep
background-task detail metrics separate unless the dashboard has those series and the user asks
for that evidence.

## RU Max-Per-Second

Define `$read_ru_max` as:

```promql
sum(resource_manager_resource_unit_read_request_unit_max_per_sec{
  $cluster_matchers,
  resource_group=~"$report_resource_groups"
}) by (resource_group)
```

Define `$write_ru_max` with the corresponding write metric and combine them without dropping a
one-sided resource group:

```promql
(($read_ru_max) or ($write_ru_max) * 0)
+
(($write_ru_max) or ($read_ru_max) * 0)
```

Use this for short-spike context only.

## Query Duration, QPS, and RC Wait

```promql
histogram_quantile(0.99,
  sum(rate(tidb_server_handle_query_duration_seconds_bucket{
    $tidb_matchers,
    resource_group=~"$report_resource_groups"
  }[1m])) by (le, resource_group)
)
```

```promql
sum(rate(tidb_executor_statement_total{
  $tidb_matchers,
  resource_group=~"$report_resource_groups"
}[1m])) by (resource_group)
```

```promql
histogram_quantile(0.99,
  sum(rate(resource_manager_client_request_success_bucket{
    $tidb_matchers,
    resource_group=~"$report_resource_groups"
  }[1m])) by (le, resource_group)
)
```

## DDL and Analyze Progress

```promql
sum(tidb_ddl_running_job_count{$tidb_matchers})
```

```promql
sum(tidb_ddl_waiting_jobs{$tidb_matchers})
```

```promql
sum(increase(tidb_statistics_auto_analyze_total{$tidb_matchers}[1m])) by (type)
```

```promql
histogram_quantile(0.95,
  sum(rate(tidb_statistics_auto_analyze_duration_seconds_bucket{$tidb_matchers}[1m])) by (le)
)
```

```promql
histogram_quantile(0.95,
  sum(rate(tidb_server_handle_query_duration_seconds_bucket{
    $tidb_matchers,
    sql_type="AnalyzeTable"
  }[1m])) by (le)
)
```

## Global Impact and Resource Usage

```promql
histogram_quantile(0.99,
  sum(rate(tidb_server_handle_query_duration_seconds_bucket{
    $tidb_matchers,
    sql_type!="internal"
  }[1m])) by (le)
)
```

```promql
sum(rate(tidb_executor_statement_total{$tidb_matchers}[1m]))
```

```promql
sum(rate(tidb_server_execute_error_total{$tidb_matchers}[1m]))
```

```promql
sum(rate(process_cpu_seconds_total{$tidb_component_matchers}[1m]))
```

```promql
sum(rate(process_cpu_seconds_total{$tikv_matchers}[1m]))
```

## Triggered TiKV Tail-Latency Queries

Run these only for short windows selected by the inspection trigger, normally up to 3 windows
with about 30 minutes of padding around warning-line utilization, over-limit, or candidate impact
points. Do not use the full inspection window for tail correlation unless the full window is the
trigger interval.

```promql
topk($tail_top_n,
  sum by (instance) (
    rate(tikv_grpc_request_source_duration_vec{
      $tikv_matchers,
      source=~".*stats.*"
    }[1m])
  )
)
```

```promql
topk($tail_top_n,
  histogram_quantile(0.99,
    sum(rate(tikv_grpc_msg_duration_seconds_bucket{
      $tikv_matchers,
      type=~"coprocessor"
    }[1m])) by (le, instance)
  )
)
```

```promql
topk($tail_top_n,
  sum by (instance) (
    rate(tikv_grpc_request_source_duration_vec{
      $tikv_matchers,
      source=~".*(ddl|DDL).*"
    }[1m])
  )
)
```

```promql
topk($tail_top_n,
  histogram_quantile(0.99,
    sum(rate(tikv_grpc_msg_duration_seconds_bucket{
      $tikv_matchers,
      type=~"kv_prewrite|kv_commit"
    }[1m])) by (le, instance, type)
  )
)
```

## Query Safety

- Keep time ranges and label filters narrow. On timeout or sample-limit errors, retry in bounded
  chunks, then increase step or narrow labels/Top-N instead of repeating the same query.
- Validate the generated query identity and result labels against the pinned profile before using
  the values in a conclusion.
- Summarize range results before loading them into model context.