---
name: pd-resource-control
description: |
  Reference handbook for PD resource-control incidents.
  Use when investigating RU throttling, token-bucket behavior, long
  Time_queued_by_rc latency, resource-group quota errors, runaway detection
  gaps, or configuration changes that do not take effect.
---

# PD Resource Control

Known RU, resource-group, and token-bucket patterns for PD on-call diagnosis.

Source note: adapted from the Feishu `PD oncall handbook` <https://pingcap.feishu.cn/wiki/E5KkwI6VJiFA4nkcaVEcDhWonqu>.

## When To Use This Skill

Use this skill when:

- SQL latency is explained by RU throttling or queuing
- `Exceeded resource group quota limitation` appears unexpectedly
- `Time_queued_by_rc` is long even though the group appears to have quota
- resource-group config changes do not take effect
- resource-control panels and behavior disagree

## First Checks

- Confirm whether the symptom is quota exhaustion, long queueing, wrong priority behavior, or wrong observability.
- Check whether workload is balanced across TiDB instances.
- Record whether the issue is in default group, a custom resource group, or internal SQL.
- Confirm whether the config was changed through the correct PD controller API.

## Known Cases

- Detailed reference: [`references/resource-control-known-cases.md`](references/resource-control-known-cases.md)

### Slow read request caused by lower priority under resource control

- Phenomenon: default-group read requests can become abnormally slow even when TiKV shows no obvious CPU or IO bottleneck.
- Workaround:
  - set all resource groups to the same priority
  - upgrade to fixed versions
- Fixed version:
  - `v8.5.4` / `v8.1.3` / `v7.5.8`
- Related oncall:
  - v7.5.3 slow point get `TICKET-8220`

### `Time_queued_by_rc` can stay high even when quota appears sufficient

- Phenomenon: request retries repeatedly reacquire tokens and accumulate queue time.
- Workaround: decrease `controller.LTBTokenRPCMaxDelay` from `1s` to `200ms`.
- Related oncall:
  - v7.5.5 high query latency despite available RU capacity `GTOC-8141`

### `controller.ltb-max-wait-duration` config change may not take effect

- Phenomenon: changing the config through the PD config API has no effect because the controller API is the supported path.
- Workaround: set it through `pd-ctl` or `curl` against the correct controller API path.
- Related oncall:
  - v8.5.3 config modify for `controller.ltb-max-wait-duration` did not work `TCOC-4651`

### Resource control can cause latency spikes under uneven TiDB workload

- Phenomenon: RU on a hot TiDB instance drops to zero while low-load TiDB instances still have equal quota.
- Workaround: keep workload distribution across TiDB instances balanced.
- Related oncall:
  - v7.5.6 RU usage anomaly `TICKET-8004`
  - v7.5.4 RU exhaustion analysis `ONCALL-7885` / `TICKET-6290`

### Query-limit actions do not support all action types correctly

- Phenomenon: some `query_limit` actions such as `cooldown`, `switch_group`, and `dry_run` are not recognized correctly in affected versions.
- Workaround: upgrade to fixed version.
- Fixed version:
  - `v8.5.2`
- Related oncall:
  - v8.5.1 resource control `action=cooldown` observation question `TICKET-5907` / `ONCALL-7743`

### Listing resource groups can panic PD

- Phenomenon: PD can panic with `json: unsupported value: NaN` while listing resource groups.
- Workaround:
  - delete unused resource groups
  - or upgrade to fixed version
- Fixed version:
  - `v7.1.4` / `v7.5.1`
- Related oncall:
  - v7.5.1 PD restarted for unknown reason `TCOC-2111`
  - v7.1.1 PD panicked in rotation `TCOC-1983`