# Titan Behavior Known Cases

Source note: adapted from the Feishu `Storage oncall handbook` <https://pingcap.feishu.cn/wiki/RNzIwCexFiA1uXkRCvzcsIocnVb>.

## Titan GC Causes Read and Write Slowdown

- Phenomenon:
  - Frequent Titan GC consumes IO resources and increases foreground read and write latency.
- Trigger / Preconditions:
  - Titan is enabled.
  - Titan GC runs frequently enough to compete with foreground workload for IO.
- Root cause:
  - Titan GC consumes disk bandwidth and background resources that would otherwise serve foreground reads and writes.
- Workaround:
  - Lower `max-background-gc=1` to reduce Titan GC worker concurrency.
  - Increase `discardable-ratio` if disk space allows, so GC runs less often.
- Related oncall:
  - v6.5.4 IO usage spike with Titan GC `GTOC-6955`

## Inaccurate Region Size Estimation in Titan

- Phenomenon:
  - Large regions may fail to split in time.
  - Space utilization across TiKV instances can become uneven.
- Trigger / Preconditions:
  - Titan is enabled and the affected release line includes inaccurate size-estimation behavior.
- Root cause:
  - Titan region-size estimation is inaccurate in the affected versions, which distorts split timing and store-balance decisions.
- Workaround:
  - Lower the region-size threshold carefully if the immediate problem is oversized regions.
  - Upgrade to `v8.5.4` when the dominant issue is uneven distribution improvement.
- Related oncall:
  - v8.5.3 store sizes severely imbalanced `TICKET-6721` / `ONCALL-8008`
  - v8.1.2 TiSpark panic caused by scanning a large region `NAID-11433` / `GTOC-7702`
  - v8.1.1 uneven space utilization across TiKV instances `NAID-10959` / `GTOC-7266`

## Titan Enabled Causes Severe Data Skew Across TiKV Instances

- Phenomenon:
  - TiKV stores become severely imbalanced after Titan is enabled, especially when wide-table data dominates.
- Trigger / Preconditions:
  - Titan is enabled.
  - The workload contains large amounts of wide-table data.
- Root cause:
  - Titan and RocksDB compact on different rhythms, which can amplify skew in physical data layout across stores.
- Workaround:
  - No generally effective workaround is documented in the source material.
- Affected version:
  - All supported versions with Titan enabled can be affected.