# TiDB Data Migration Overview

__*DM Principle*__
__*DM Features*__

+ Supports DML & DDL replication
+ Supports sharded databases and tables replication
+ Built-in filters
+ Centralized management
+ High availability


![right](./student-guide/diagram/DMPI_aa_001_20220623.png)
-------------------------------------------------------------
DM Overview

TiDB Data Migration is an integrated data migration task management platform, which supports the full data migration and the incremental data replication from MySQL-compatible databases (such as MySQL, MariaDB, and Aurora MySQL) into TiDB. It can help to reduce the operation overhead of data migration and simplify the troubleshooting process.

Basic features
- Compatibility with MySQL. DM is compatible with MySQL 5.7 and most of the features and syntax of MySQL 5.7.
- Replicating DML and DDL events. It supports parsing and replicating DML and DDL events in MySQL binlog.
- Migrating and merging MySQL shards. DM supports migrating and merging multiple MySQL database instances upstream to one TiDB database downstream. It supports customizing replication rules for different migration scenarios. It can automatically detect and handle DDL changes of upstream MySQL shards, which greatly reduces the operational cost.
- Various types of filters. You can predefine event types, regular expressions, and SQL expressions to filter out MySQL binlog events during the data migration process.
- Centralized management. DM supports thousands of nodes in a cluster. It can run and manage many data migration tasks concurrently.
- Optimization of the third-party Online Schema Change process. In the MySQL ecosystem, tools such as gh-ost and pt-osc are widely used. DM optimizes its change process to avoid unnecessary migration of intermediate data.- High availability. DM supports data migration tasks to be scheduled freely on different nodes. The running tasks are not affected when a small number of nodes crash.

DM-master manages and schedules the operations of data migration tasks:
- Storing the topology information of the DM cluster.
- Monitoring the running state of DM-worker processes.
- Monitoring the running state of data migration tasks.
- Providing a unified portal for the management of data migration tasks.
- Coordinating the DDL migration of sharded tables in each instance under the database sharding scenarios.


DM-worker executes specific data migration tasks:
- Persisting the binlog data to the local storage.
- Storing the configuration information of the data migration subtasks.
- Orchestrating the operation of the data migration subtasks.
- Monitoring the running state of the data migration subtasks.