# TiCDC Syncpoint Feature Documentation​ - Part I

## Syncpoint background ​

Users have the requirement to verify the data consistency of upstream and downstream without stopping synchronization in master-slave replication system like TiCDC. Online consistency check for dynamic data is very difficult. By leveraging the Snapshot function of TiDB, then TiCDC only needs to find the mapping from a set of upstream data snapshots to downstream data snapshots and make sure the data of snapshots are completely consistent. This means we can verify the consistency of historical snapshots from upstream and downstream, and approximately confirm the correctness of TiCDC. This replaces the problem of verifying dynamic data with the problem of verifying static data, reducing the overall technical difficulty.​

Syncpoint's capability is to maintain a set of upstream and downstream snapshot mapping relationships (in effect, TSOs) that theoretically guarantee that the data corresponding to the upstream snapshot and the data corresponding to the downstream snapshot are identical.

## Fundamentals of Syncpoint​

The purpose of Syncpoint is to maintain a set of upstream and downstream database snapshot mapping relationships, so that the upstream and downstream snapshot data are completely consistent. In order to achieve this goal, we discuss why the downstream data is not consistent with the upstream under normal circumstances. Firstly:​

* Upstream transactions are split into different TiCDC nodes by table, and transactions are written downstream concurrently between different TiCDC nodes, so the progress of data synchronization between tables is uneven.​
* Transactions in the table are concurrently written downstream according to certain rules, so the order of transactions cannot be guaranteed.

 In summary, the cause of downstream data inconsistency is actually caused by TiCDC  writing downstream concurrently.

The Syncpoint functionality coordinates each worker that writes to the downstream, and according to a user-specified period, sends a signal to each worker to notify them to pause on a unified TSO. The worker will stop writing after a signaled worker has written all transactions that precede a certain TSO. When all workers stop writing. In theory, the downstream data must be exactly the same as the historical snapshot corresponding to the upstream TSO. The flow chart of the whole process is as follows:

![syncpoint flowchart](/hc/article_attachments/7507814739095/syncpoint.jpg)

## Using Syncpoint

Create changefeed command:

```{.bash .copyable}
➜  ~ tiup cdc cli changefeed create --help
tiup is checking updates for component cdc ...
Starting component `cdc`: /Users/leoppro/.tiup/components/cdc/v6.1.0/cdc /Users/leoppro/.tiup/components/cdc/v6.1.0/cdc cli changefeed create --help
Create a new replication task (changefeed)
Usage:
  cdc cli changefeed create [flags]
Flags:
  -c, --changefeed-id string              Replication task (changefeed) ID
      --config string                     Path of the configuration file
      --cyclic-filter-replica-ids uints   (Experimental) Cyclic replication filter replica ID of changefeed (default [])
      --cyclic-replica-id uint            (Experimental) Cyclic replication replica ID of changefeed
      --cyclic-sync-ddl                   (Experimental) Cyclic replication sync DDL of changefeed (default true)
      --disable-gc-check                  Disable GC safe point check
  -h, --help                              help for create
      --no-confirm                        Don't ask user whether to ignore ineligible table
      --opts key=value                    Extra options, in the key=value format
      --schema-registry string            Avro Schema Registry URI
      --sink-uri string                   sink uri
      --sort-engine string                sort engine used for data sort (default "unified")
      --start-ts uint                     Start ts of changefeed
      --sync-interval duration            (Experimental) Set the interval for syncpoint in replication(default 10min) (default 10m0s)
      --sync-point                        (Experimental) Set and Record syncpoint in replication(default off)
      --target-ts uint                    Target ts of changefeed
      --tz string                         timezone used when checking sink uri (changefeed timezone is determined by cdc server) (default "SYSTEM")
Global Flags:
      --ca string          CA certificate path for TLS connection
      --cert string        Certificate path for TLS connection
  -i, --interact           Run cdc cli with readline
      --key string         Private key path for TLS connection
      --log-level string   log level (etc: debug|info|warn|error) (default "warn")
      --pd string          PD address, use ',' to separate multiple PDs (default "http://127.0.0.1:2379")
```

Syncpoint related options are `--sync-point` and  `--sync-interval`.

* `--sync-point` used to enable/disable syncpoint feature.
* `--sync-interval` is the time interval for maintaining the upstream and downstream TSO mapping, i.e. X in above flowchart。
After configuring the above two parameters, TiCDC will generate a set of TS-MAPs every X minutes and write them to a specific table in the downstream database. The table structure is as follows:

```{.bash .copyable}
MySQL [tidb_cdc]> SELECT * FROM syncpoint_v1 LIMIT 1;
+--------------+--------------------+--------------------+
| cf           | primary_ts         | secondary_ts       |
+--------------+--------------------+--------------------+
| ec-nfr-tsmap | 434101891612278786 | 434101892055564289 |
+--------------+--------------------+--------------------+
```

### Further reading

[TiCDC Syncpoint Feature Documentation​ - Part II](https://support.pingcap.com/hc/en-us/articles/7506405731095)