---
name: pd-placement-rules
description: |
  Reference handbook for PD placement-rule and isolation-label incidents.
  Use when investigating placement-rule resets, SQL placement-policy interactions,
  location-label surprises, isolation-label gaps, or leader imbalance after
  large rule changes.
---

# PD Placement Rules

Known placement-rule, placement-policy, and isolation-label patterns for PD on-call diagnosis.

Source note: adapted from the Feishu `PD oncall handbook` <https://pingcap.feishu.cn/wiki/E5KkwI6VJiFA4nkcaVEcDhWonqu>.

## When To Use This Skill

Use this skill when:

- placement rules are reset, ignored, or overridden unexpectedly
- placement-rule in SQL interacts badly with PD-level rules
- location labels or isolation labels do not behave as expected
- leader or replica distribution looks odd right after rule changes

## First Checks

- Confirm whether the change came from `pd-ctl`, SQL placement policy, or both.
- Check current rule groups and whether any group uses `override`.
- Confirm whether `SURVIVAL_PREFERENCES` is present when SQL placement policy is expected to inherit cluster defaults.
- Record whether PD leader transfer happened near the incident.

## Known Cases

- Detailed reference: [`references/placement-rules-known-cases.md`](references/placement-rules-known-cases.md)

### Leader imbalance after placement-rule adjustment or large-scale TiKV failures

- Phenomenon: after a large placement-rule change or mass TiKV failures, PD may transfer many leaders and temporarily concentrate them on stores with smaller store IDs.
- Workaround: usually do nothing and let `balance-leader-scheduler` smooth the distribution over time.
- Related oncall:
  - v8.5.1 leader became significantly imbalanced after applying placement rules `TICKET-6623`

### PD leader transfer may reset placement rules

- Phenomenon: when TiKV labels configured through `pd-ctl` are referenced by placement rules, PD leader transfer can reset placement rules in affected versions.
- Workaround:
  - reapply placement rules after leader transfer
  - upgrade to patched versions
- Related oncall:
  - v8.1.2 placement rule reset after leader transfer `NAID-11507` / `GTOC-7779`

### Placement rules in SQL can override PD-level placement rules

- Phenomenon: when table-level placement policy is created through SQL, it can override cluster-level placement rules instead of inheriting them.
- Diagnostic commands:

```bash
tiup ctl:<version> pd config placement-rule rule-group show
tiup ctl:<version> pd config placement-rule show
```

- Workaround:
  - prefer placement rules in SQL on `v7.5+`
  - if mixing PD rules and SQL placement rules, understand override precedence explicitly
- Related oncall:
  - v6.5.7 asked whether placement-rule and placement rule in SQL can be mixed `TICKET-4474`

### Placement rules in SQL can silently drop location-label inheritance

- Phenomenon: if SQL placement policy does not set `SURVIVAL_PREFERENCES`, table-level placement can stop inheriting cluster-level location labels.
- Impact: two replicas can appear on the same physical host.
- Workaround: set `SURVIVAL_PREFERENCES`.
- Related oncall:
  - v7.5.0 replica-read and traffic-switching anomaly related to placement config `ONCALL-6944`

### Disabling placement rule can cause TiDB GC errors

- Phenomenon: if placement rules are disabled and TiDB later tries to delete table placement rules during GC, PD can return `StatusPreconditionFailed`.
- Workaround:
  - ignore if there is no functional impact
  - or re-enable placement rules
- Related oncall:
  - v6.5.12 TiDB logged `delete placement-rules failed` `TICKET-5867`

### Without isolation labels, two replicas can land on the same host

- Phenomenon: if isolation labels are not configured and space is tight, two replicas of one region can appear on different TiKV instances on the same physical host.
- Workaround: configure isolation labels.
- Related oncall:
  - v5.2.1 two replicas of one region were on the same host `ONCALL-6973`