---
title: Outbox Pattern Support for TiCDC
feature_id: ticdc-outbox-protocol
owner: Benjamin Meadowcroft
status: draft
target_release: Summer 2026
created: 2026-03-16
last_updated: 2026-03-25
---

# Outbox Pattern Support for TiCDC

## Executive Summary

This feature adds first-class transactional outbox support to TiCDC through a new `outbox-json` protocol and explicit dispatch configuration for keys, payloads, headers, and topic routing.

It primarily serves [PER-002 Application Developer](../../shared/personas.md#per-002-application-developer), [PER-001 Data Platform Engineer](../../shared/personas.md#per-001-data-platform-engineer), and [PER-003 SRE / Platform Operator](../../shared/personas.md#per-003-sre--platform-operator) workflows that need reliable event publication from committed business transactions without custom middleware.

The outcome is lower-friction migration from Debezium-style outbox pipelines, clearer operational behavior during rollout and recovery, and less application-side publish orchestration for teams adopting TiDB and TiCDC.

## Press Release

### PingCAP launches transactional outbox support in TiCDC

TiCDC `outbox-json` support helps application and platform teams migrate existing outbox pipelines with fewer custom adapters and more predictable event delivery behavior.

San Francisco, CA, March 25, 2026. PingCAP today announced first-class transactional outbox support for TiCDC through the new `outbox-json` output protocol. Teams can now write business rows and outbox events in the same transaction and use TiCDC to emit those committed events directly to downstream messaging systems without adding custom adapter steps between TiDB and the downstream systems.

"Customers adopting the outbox pattern want fewer moving parts between a committed transaction and a downstream event," said the PingCAP TiCDC team. "This release gives operators and application teams moving onto TiDB a cleaner migration path for their existing outbox-based downstream integrations."

### About Outbox Pattern Support for TiCDC

Outbox Pattern Support for TiCDC adds a dedicated protocol for committed outbox rows and configurable mapping for message keys, payloads, headers, and topic selection. The feature is designed for teams migrating from existing outbox CDC pipelines that already depend on stable event semantics and want to preserve transaction-boundary correctness while reducing custom glue code. It also gives operators a clearer rollout, validation, and recovery model than an adapter-heavy deployment.

### Availability

The initial release targets Kafka downstreams for GA support.

## FAQ

### What problem does this solve?

TiCDC can replicate row changes, but does not natively provide a clean, opinionated outbox path for message-first event rows. Teams currently add transformation layers or custom application logic to publish application-oriented events.

### Why now?

More customers are building CDC-first architectures and are blocked by the lack of a clean bridge between transactional writes and event publishing.

### Why this approach?

A dedicated outbox output protocol in TiCDC keeps consistency, ordering, and replication behavior in the same data-path instead of splitting it across app-side message buses.

### What is out of scope?

This feature does not add generic message transformation, stream processing, or downstream orchestration. It focuses on robust event extraction and routing from outbox rows.

---

## Jobs to Be Done

#### JTBD-001

> When [PER-002 Application Developer](../../shared/personas.md#per-002-application-developer) commits business data that must also drive downstream workflows, they want to write an outbox row in the same transaction and have TiCDC emit it directly, so they can build reliable event-driven workflows without XA or custom event transformation pipelines.

This job is about making transactional, application-oriented event publication a supported product path rather than an application-managed integration.

#### JTBD-002

> When [PER-001 Data Platform Engineer](../../shared/personas.md#per-001-data-platform-engineer) migrates an existing Debezium-style outbox pipeline, they want deterministic topic, key, payload, and header selection in TiCDC, so they can reduce custom middleware and complete rollout with lower operational risk.

This job is about predictable migration behavior and minimizing bespoke data-path components that have to be maintained separately from TiCDC.

## Problem and Context

Modern architectures commonly rely on Debezium-style outbox patterns to avoid dual-write inconsistencies. These patterns often require extra services or adapters to format and route event payloads.

TiCDC currently lacks a first-class outbox protocol, which increases migration cost and error surface area for customers moving to TiDB CDC-based systems.

This gap is especially painful when teams want:
- pass-through event rows,
- deterministic topic dispatch without custom transformation services,
- header propagation with minimal application changes.

The missing product path creates friction for both application teams that want transaction-bound event emission and operators who need a predictable rollout, observability, and recovery model.

## Goals and Non-Goals

### Goals

- **G-001**: Enable TiCDC to support the transactional outbox pattern as a first-class product workflow.
- **G-002**: Support pass-through emission of outbox rows as downstream messages.
- **G-003**: Support deterministic topic routing from row data.
- **G-004**: Support selected row columns as message headers.

### Non-Goals

- **NG-001**: Replication of DDL changes downstream is out of scope for this release.
- **NG-002**: General-purpose message transformation inside TiCDC is out of scope for this release.
- **NG-003**: Arbitrary payload rewriting inside TiCDC is out of scope for this release.
- **NG-004**: Downstream acknowledgment-driven updates back into outbox state are out of scope for this release; TiCDC will not write publication status back to source rows.
- **NG-005**: First-class duplicate suppression or idempotency enforcement is out of scope for this release; downstream consumers remain responsible for idempotent processing under existing TiCDC semantics.

## System Invariants

#### INV-001: Preserve Source Transaction Ordering

Outbox event emission must preserve source transaction ordering guarantees for externally visible effects.

Extends: [INV-GLOBAL-002](../../shared/system-invariants.md#inv-global-002-commit-visibility-follows-defined-ordering)

#### INV-002: Emit Events in Commit Order

When multiple outbox rows are committed, TiCDC must emit them in a way that remains consistent with source commit order.

Extends: [INV-GLOBAL-002](../../shared/system-invariants.md#inv-global-002-commit-visibility-follows-defined-ordering)

#### INV-003: No Silent Loss After Observation

No committed outbox message may be silently lost after TiCDC has observed it for delivery.

Extends: [INV-GLOBAL-001](../../shared/system-invariants.md#inv-global-001-acknowledged-writes-are-durable), [INV-GLOBAL-004](../../shared/system-invariants.md#inv-global-004-silent-corruption-is-unacceptable)

#### INV-004: Degraded States Stay Observable

Lag, routing failures, and downstream delivery disruption must remain observable through supported TiCDC operational surfaces.

Extends: [INV-GLOBAL-005](../../shared/system-invariants.md#inv-global-005-unhealthy-states-must-be-observable)

## User / Operator Scenarios

See [Scenarios.md](./Scenarios.md) for full scenario definitions.

Key scenario areas covered:
- Onboarding and first-time configuration: SCN-010
- Steady-state event emission and routing: SCN-001, SCN-002, SCN-003
- Failure and recovery behavior: SCN-006, SCN-007, SCN-008, SCN-011, SCN-012
- Rollout, migration, and decommission workflows: SCN-009, SCN-013

## Functional Requirements

#### FR-001: Outbox Output Protocol

Priority: P0
Supports: JTBD-001, SCN-001, SCN-006, SCN-007, SCN-012, SCN-013

TiCDC shall support an `outbox-json` protocol in which configured outbox rows are emitted as downstream events without requiring additional adaptation.

**Acceptance Criteria:**
- Given a changefeed configured with `protocol=outbox-json`, when an application commits an outbox row, then the operator can observe a downstream message emitted for that row using the configured outbox mapping.
- Given the changefeed is resumed after an interruption, when previously committed outbox rows remain pending, then the operator can observe delivery resume without silent loss of already observed rows.

#### FR-002: Configurable Message Key

Priority: P0
Supports: JTBD-001, JTBD-002, SCN-001, SCN-005, SCN-011

The feature shall let users select which outbox column populates the downstream message key.

**Acceptance Criteria:**
- Given a changefeed configured with `key-column=<column>`, when an outbox event is emitted, then the downstream message key matches the configured column value.
- Given the configured key column is null, when TiCDC evaluates the event, then the operator can observe a documented handling path instead of ambiguous downstream behavior.

#### FR-003: Configurable Message Payload

Priority: P0
Supports: JTBD-001, JTBD-002, SCN-001, SCN-004, SCN-012

The feature shall let users select which outbox column populates the downstream message payload and, for Kafka sinks, support the same large-message handling model already available to `canal-json` and `open-protocol`.

**Acceptance Criteria:**
- Given a changefeed configured with `value-column=<column>`, when an outbox event is emitted, then the downstream message payload matches the configured column value.
- Given an oversized payload is written to the configured value column, when TiCDC processes the row for a Kafka sink, then the operator can configure the same documented large-message handling options used by `canal-json` and `open-protocol` instead of being limited to an immediate changefeed failure on size alone.
- Given `large-message-handle-option=handle-key-only`, when the encoded outbox message exceeds the Kafka topic limit, then TiCDC emits handle-key-only output using the same operator-visible contract as the existing Kafka large-message path.
- Given `large-message-handle-option=claim-check` and a valid `claim-check-storage-uri`, when the encoded outbox message exceeds the Kafka topic limit, then TiCDC stores the large payload in external storage and emits a Kafka message that lets consumers locate and retrieve the full payload.
- Given `large-message-handle-compression` is enabled, when TiCDC evaluates an oversized outbox message for Kafka delivery, then TiCDC compresses the encoded message before comparing it with the Kafka message size limit.

#### FR-004: Column-Based Topic Dispatch

Priority: P0
Supports: JTBD-002, SCN-002, SCN-005, SCN-008, SCN-009

The feature shall optionally derive the downstream topic from a configured outbox row column.

**Acceptance Criteria:**
- Given dispatcher rule `{column:<column-name>}`, when an outbox row is emitted, then the downstream message is routed to the topic named by that column value.
- Given the configured topic value is invalid or the dispatcher is misconfigured, when TiCDC evaluates the row, then the operator can observe a routing failure or diagnostic signal instead of silent misrouting.

#### FR-005: Message Header Columns

Priority: P1
Supports: JTBD-001, JTBD-002, SCN-003, SCN-009, SCN-010

The feature shall let users configure outbox row columns that are emitted as downstream message headers.

**Acceptance Criteria:**
- Given configured header mappings, when an outbox row is emitted, then the downstream message includes headers derived from the configured columns.
- Given a new outbox changefeed is configured with header mappings, when the operator validates a test event, then the header output matches the declared mapping without additional transformation middleware.

## Non-Functional Requirements

#### NFR-001: Performance Parity with Existing JSON Protocols

Category: performance
Priority: P0
Shared References: [OBS-001](../../shared/observability-requirements.md#obs-001-health-and-progress-metrics)

The feature must keep throughput and lag within the same operational envelope expected for comparable `canal-json` workloads in the same environment.

**Acceptance Criteria:**
- Given a sustained outbox workload representative of existing CDC traffic, when operators compare throughput and lag against the current `canal-json` baseline in the same environment, then observed performance remains within the accepted non-regression threshold for release sign-off.

#### NFR-002: Kafka Compatibility for GA Scope

Category: compatibility
Priority: P0
Shared References: [COMP-005](../../shared/compatibility-guidelines.md#comp-005-preserve-stable-external-apis-and-contracts), [COMP-006](../../shared/compatibility-guidelines.md#comp-006-protect-protocol-compatibility)

The feature must support Kafka downstreams as the initial GA compatibility target for `outbox-json`.

**Acceptance Criteria:**
- Given a supported Kafka sink configuration, when operators enable `outbox-json`, then committed outbox rows are delivered with stable key, payload, header, and topic semantics without requiring an intermediate translation service.

#### NFR-003: Explicit Unsupported Compatibility Boundaries

Category: compatibility
Priority: P1
Shared References: [COMP-006](../../shared/compatibility-guidelines.md#comp-006-protect-protocol-compatibility), [COMP-010](../../shared/compatibility-guidelines.md#comp-010-document-unsupported-combinations)

The feature must document unsupported downstream combinations explicitly, including that Pulsar support is deferred from the initial release.

**Acceptance Criteria:**
- Given an operator reviews launch documentation for `outbox-json`, when they evaluate supported sinks for the initial release, then Kafka support is explicit and Pulsar is explicitly documented as deferred rather than implied.

#### NFR-004: Reuse Existing TiCDC Observability Surfaces

Category: observability
Priority: P0
Shared References: [OBS-001](../../shared/observability-requirements.md#obs-001-health-and-progress-metrics), [OBS-002](../../shared/observability-requirements.md#obs-002-error-visibility), [OBS-004](../../shared/observability-requirements.md#obs-004-alertable-failure-signals), [OBS-008](../../shared/observability-requirements.md#obs-008-operator-debuggable-state)

The feature must expose health, lag, and failure signals through existing TiCDC observability surfaces so operators can diagnose outbox behavior without inspecting internal state manually.

**Acceptance Criteria:**
- Given an outbox changefeed is healthy, when an operator inspects standard TiCDC telemetry, then they can confirm progress and lag using supported metrics or status surfaces.
- Given a routing or downstream delivery failure occurs, when the operator inspects TiCDC telemetry, then the failure is visible through supported error or status surfaces without requiring direct datastore inspection.

## Success Metrics

| Metric | Target |
|---|---|
| JTBD-001 time to first successful event | Operators can configure a new outbox changefeed and observe the first valid downstream event in under 30 minutes during guided rollout validation |
| JTBD-002 migration adapter reduction | Migration projects can remove at least one custom outbox transformation or publish adapter from the critical path when moving to TiCDC |
| Operator recovery visibility | Operators can identify routing or sink-delivery failures using standard TiCDC telemetry in under 10 minutes during validation exercises |
| Replication lag regression | No material lag regression versus the approved `canal-json` baseline in release validation |

## Risks

#### R-001: Dispatcher Misconfiguration

Priority: P1
Incorrect dispatcher configuration may misroute events or route them to invalid topics.
Mitigation: Provide explicit validation scenarios, documented configuration guidance, and operator-visible diagnostics for invalid routing.

#### R-002: Uneven Partition Distribution

Priority: P1
Single outbox tables or poor key selection may create uneven downstream partition distribution.
Mitigation: Validate key configuration guidance in migration scenarios and surface lag or hotspot symptoms through standard observability.

#### R-003: Large Payload Performance Degradation

Priority: P1
Large payload columns may reduce throughput or increase lag under sustained load.
Mitigation: Validate oversized payload handling and performance thresholds before release and document recommended workload boundaries.

## Open Questions

- [DEFERRED] Downstream acknowledgment back into outbox state: Deferred for this release. Out of scope per NG-004 because TiCDC does not write publication status back to source rows.
- [DEFERRED] First-class deduplication support: Deferred for this release. Out of scope per NG-005; the feature follows existing TiCDC delivery semantics and expects downstream idempotent processing.

## References

- Internal design notes for TiCDC output protocol configuration  
- Debezium outbox pattern reference materials  
- Prior TiCDC compatibility and protocol docs