# [Known Issue] Memory leak in TiKV when async commit / 1PC overlaps with concurrent range scans ([tikv#19284](https://github.com/tikv/tikv/issues/19284))

Source note: adapted from GitHub issue [tikv#19284](https://github.com/tikv/tikv/issues/19284).

## Issue

- Link: [tikv#19284](https://github.com/tikv/tikv/issues/19284)
- Summary: when async commit or 1PC is enabled, overlapping range scans and writes can leak memory in TiKV's `ConcurrencyManager`, causing instance memory to keep growing until OOM.

## Phenomenon

- TiKV memory usage keeps increasing over time.
- Memory does not return to baseline even after the workload drops.
- In severe cases, the affected TiKV instance eventually OOMs or restarts.
- Heap profile can show a large amount of memory retained under `scheduler.concurrency_manager`.

## Root Cause

TiKV uses a global `ConcurrencyManager` to coordinate transaction concurrency for async commit and 1PC. This component maintains an in-memory lock table (`LockTable`) before locks are persisted to RocksDB.

Under normal conditions:

- locks are inserted into `LockTable` during prewrite
- after the lock is written to RocksDB, the in-memory entry should be released

The bug is triggered when all of the following happen at the same time:

- async commit or 1PC transactions are in prewrite
- concurrent range scans are running
- the scan range overlaps with the write key range

In this case, some in-memory lock entries are not released correctly, so `LockTable` retains memory indefinitely and causes a leak.

## Trigger Condition

- `tidb_enable_async_commit = ON` or `tidb_enable_1pc = ON`
- concurrent range scans exist, such as:
  - coprocessor range scans from SQL
  - `ScanLock` during GC
  - replica read or follower read
- read ranges overlap with the active write ranges

## Diagnostic Steps

### 1. Observe memory behavior

- Check whether TiKV memory keeps growing over time.
- Check whether memory remains high even after traffic falls.
- Check whether OOM or abnormal restart happens after long-running growth.

### 2. Check heap profile

- Collect a TiKV heap profile.
- Look for heavy memory usage under:
  - `scheduler.concurrency_manager`

### 3. Verify transaction settings when heap profile is unavailable

Check whether async commit or 1PC is enabled:

```sql
SHOW GLOBAL VARIABLES LIKE 'tidb_enable_async_commit';
SHOW GLOBAL VARIABLES LIKE 'tidb_enable_1pc';
```

If either feature is enabled, continue checking the workload pattern:

- range scans and writes run concurrently
- the read ranges overlap with the write key ranges

## Workaround

Disable async commit and 1PC to avoid the problematic code path:

```sql
SET GLOBAL tidb_enable_async_commit = OFF;
SET GLOBAL tidb_enable_1pc = OFF;
```

Notes:

- This fully mitigates the leak trigger.
- Transaction latency might increase after disabling these features.
- Evicting leaders does not solve the problem, because `ConcurrencyManager` is maintained at the TiKV instance level rather than per leader region.

## Fixed Version

The issue is fixed in:

- `v7.1.7`
- `v7.5.8`
- `v8.1.3`
- `v8.5.6`

Upgrade to one of the fixed versions whenever possible.