# TiDB extractor recovery — 2026-09-08

## Failure and scope

Live source before repair: `main@c7ff8f1`, image
`sha256:6814169426f924137d84cfc160efa71b0ffd55e8d1f733fa314cd78c55ec8959`.
NexusCRM execution `pingcap-nexuscrm-extract-d28fw` completed both Provider
exports but failed `manifest_table_coverage`: 149 of 380 selected tables had
schema artifacts and no Parquet. A read-only source check found all 149 empty.
Finance execution `pingcap-df-finance-extract-l64m4` completed six exports but
failed on three schema-only tables: `data_platform_load_target_lease`,
`monthly_accounts_receivable`, `ps_man_days_reject`.

The exact-coverage guard introduced in PR #12 incorrectly required a data file
for every table. Dumpling writes table schema independently of data and does not
write a data file for an empty table. See the [upstream writer](https://github.com/pingcap/tidb/blob/master/dumpling/export/writer.go)
and [Parquet type mapping](https://github.com/pingcap/tidb/blob/master/pkg/dumpformat/parquetfile/column_type.go).

Earlier NexusCRM runs additionally failed in the system domain with Provider
`FAILED` and `Export failed with unknown error`. The internal Provider cause
is still unknown; bounded retries mitigate transient failures but do not prove
that defect fixed.

## Repair contract

- Enable empty-table materialization and at most three terminal-failure export
  attempts for NexusCRM and Finance. Preserve the complete selected table set.
- Every attempt has a separate GCS prefix. The manifest lists only successful
  attempts; partial data from failed attempts is never admitted.
- A missing Parquet can be replaced by typed zero-row Parquet only after the
  exact selected export succeeds, the table schema artifact exists, dump
  metadata contains a completion marker and snapshot TSO, and an exact
  `SELECT 1 ... LIMIT 1` at that TSO proves zero rows. Row estimates and current
  emptiness are not acceptance evidence. Missing schema, expired snapshots,
  nonempty source tables, or unsupported types remain failures.
- Empty Parquet preserves full snapshot deletion semantics through the existing
  loader, including clearing stale final rows for tables that became empty.
- Cancellation is never retried. Persistent Provider failure still prevents
  manifest publication. No source table is excluded to obtain a green job.

## Acceptance and release receipts

The single authoritative release/acceptance record is the body of
[PR #14](https://github.com/pingcap-inc/pingcap-datapipelines/pull/14).
It binds the reviewed source SHA, immutable images, Cloud Run executions,
full manifest coverage, and downstream readback. Do not infer complete recovery
from an extractor success or a green no-op load alone.

Acceptance requires both full extract manifests with exact table coverage and
zero schema drift, then the corresponding full loads, table control receipts,
and raw/final checks. Old eight-table recovery does not satisfy this boundary.