# **练习-3 : Data Migration(DM)的管理** ## 概述 + 在本练习中，将对现有正在运行的 DM 集群进行缩容､扩容和状态查询管理，之后停止复制任务并删除集群｡ ## 练习环境要求 + 1. 注意，您的实验环境（包括IP，端口号，用户名，密码和目录）可能和手册不同。 + 2. 可以连接到实验用虚拟机。 + 3. TiDB 数据库提前部署完毕。 + 4. 需要完成练习-1或者练习-2。 ## 拓扑图 ## 步骤 ----------------------------------------------- #### 1. 对现有的 DM 集群进行缩容，将 free 状态的 worker 下线。 + **查询现有 DM 集群，是否有 worker 节点状态为 Free，如下:** ``` # tiup dm display dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test Cluster type: dm Cluster name: dm-test Cluster version: v6.1.0 Deploy user: tidb SSH type: builtin Grafana URL: http://172.31.0.232:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- 172.31.0.232:9093 alertmanager 172.31.0.232 9093/9094 linux/x86_64 Up /home/tidb/dm/data/alertmanager-9093 /home/tidb/dm/deploy/alertmanager-9093 172.31.0.175:8261 dm-master 172.31.0.175 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.232:8261 dm-master 172.31.0.232 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.233:8261 dm-master 172.31.0.233 8261/8291 linux/x86_64 Healthy|L /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.175:8262 dm-worker 172.31.0.175 8262 linux/x86_64 Free /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:8262 dm-worker 172.31.0.232 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.233:8262 dm-worker 172.31.0.233 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:3000 grafana 172.31.0.232 3000 linux/x86_64 Up - /home/tidb/dm/deploy/grafana-3000 172.31.0.232:9090 prometheus 172.31.0.232 9090 linux/x86_64 Up /home/tidb/dm/data/prometheus-9090 /home/tidb/dm/deploy/prometheus-9090 Total nodes: 9 ``` + **在现有 DM 集群，发现 172.31.0.175 的 dm-worker 状态为 Free** + **缩容现有 DM 集群,将 172.31.0.175 的 dm-worker 节点进行下线，如下：** ``` # tiup dm scale-in dm-test -N 172.31.0.175:8262 ``` + **查询现有 DM 集群,是否指定的 worker 节点已经下线，如下：** ``` # tiup dm display dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test Cluster type: dm Cluster name: dm-test Cluster version: v6.1.0 Deploy user: tidb SSH type: builtin Grafana URL: http://172.31.0.232:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- 172.31.0.232:9093 alertmanager 172.31.0.232 9093/9094 linux/x86_64 Up /home/tidb/dm/data/alertmanager-9093 /home/tidb/dm/deploy/alertmanager-9093 172.31.0.175:8261 dm-master 172.31.0.175 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.232:8261 dm-master 172.31.0.232 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.233:8261 dm-master 172.31.0.233 8261/8291 linux/x86_64 Healthy|L /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.232:8262 dm-worker 172.31.0.232 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.233:8262 dm-worker 172.31.0.233 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:3000 grafana 172.31.0.232 3000 linux/x86_64 Up - /home/tidb/dm/deploy/grafana-3000 172.31.0.232:9090 prometheus 172.31.0.232 9090 linux/x86_64 Up /home/tidb/dm/data/prometheus-9090 /home/tidb/dm/deploy/prometheus-9090 Total nodes: 8 ``` + **从结果中发现 172.31.0.175 的 dm-worker 已经下线** ----------------------------------------------- #### 2. 扩容 DM 集群。 + **编辑扩容配置文件 dm-scale.yaml ,加入一个 worker 节点，如下:** ``` # vi dm-scale.yaml ``` ``` worker_servers: - host: 172.31.0.175 ``` + **使用扩容配置文件,扩容现有 DM 集群，(需要输入一次 y)如下:** ``` # tiup dm scale-out dm-test dm-scale.yaml -uroot -p ``` + **查询现有 DM 集群,是否扩容成功，如下：** ``` # tiup dm display dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test Cluster type: dm Cluster name: dm-test Cluster version: v6.1.0 Deploy user: tidb SSH type: builtin Grafana URL: http://172.31.0.232:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- 172.31.0.232:9093 alertmanager 172.31.0.232 9093/9094 linux/x86_64 Up /home/tidb/dm/data/alertmanager-9093 /home/tidb/dm/deploy/alertmanager-9093 172.31.0.175:8261 dm-master 172.31.0.175 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.232:8261 dm-master 172.31.0.232 8261/8291 linux/x86_64 Healthy /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.233:8261 dm-master 172.31.0.233 8261/8291 linux/x86_64 Healthy|L /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.175:8262 dm-worker 172.31.0.175 8262 linux/x86_64 Free /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:8262 dm-worker 172.31.0.232 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.233:8262 dm-worker 172.31.0.233 8262 linux/x86_64 Bound /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:3000 grafana 172.31.0.232 3000 linux/x86_64 Up - /home/tidb/dm/deploy/grafana-3000 172.31.0.232:9090 prometheus 172.31.0.232 9090 linux/x86_64 Up /home/tidb/dm/data/prometheus-9090 /home/tidb/dm/deploy/prometheus-9090 Total nodes: 9 ``` + **从结果中发现 172.31.0.175 的 dm-worker 已经上线，并且状态为 Free** ----------------------------------------------- #### 3. 停止同步复制。 + **停止 DM 的复制任务，如下:** ``` # tiup dmctl --master-addr=172.31.0.232:8261 stop-task dm-task.yaml ``` ``` tiup is checking updates for component dmctl ... Starting component `dmctl`: /root/.tiup/components/dmctl/v6.1.0/dmctl/dmctl --master-addr=172.31.0.232:8261 stop-task dm-task.yaml { "op": "Delete", "result": true, "msg": "", "sources": [ { "result": true, "msg": "", "source": "mysql-replica-01", "worker": "dm-172.31.0.232-8262" }, { "result": true, "msg": "", "source": "mysql-replica-02", "worker": "dm-172.31.0.233-8262" } ] } ``` + **查询 DM 复制操作是否停止，如下:** ``` # tiup dmctl --master-addr=172.31.0.232:8261 query-status dm-task.yaml ``` ``` tiup is checking updates for component dmctl ... Starting component `dmctl`: /root/.tiup/components/dmctl/v6.1.0/dmctl/dmctl --master-addr=172.31.0.232:8261 query-status dm-task.yaml { "result": false, "msg": "task dm-task2 has no source or not exist", "sources": [ ] } ``` #### 4. 关闭 DM 集群，并销毁集群。 + **关闭 DM 集群（需要输入一次 y），如下:** ``` # tiup dm stop dm-test ``` + **确认 DM 集群状态，如下:** ``` # tiup dm display dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test Cluster type: dm Cluster name: dm-test Cluster version: v6.1.0 Deploy user: tidb SSH type: builtin Grafana URL: http://172.31.0.232:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- 172.31.0.232:9093 alertmanager 172.31.0.232 9093/9094 linux/x86_64 Down /home/tidb/dm/data/alertmanager-9093 /home/tidb/dm/deploy/alertmanager-9093 172.31.0.175:8261 dm-master 172.31.0.175 8261/8291 linux/x86_64 Down /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.232:8261 dm-master 172.31.0.232 8261/8291 linux/x86_64 Down /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.233:8261 dm-master 172.31.0.233 8261/8291 linux/x86_64 Down /home/tidb/dm/data/dm-master-8261 /home/tidb/dm/deploy/dm-master-8261 172.31.0.175:8262 dm-worker 172.31.0.175 8262 linux/x86_64 N/A /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:8262 dm-worker 172.31.0.232 8262 linux/x86_64 N/A /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.233:8262 dm-worker 172.31.0.233 8262 linux/x86_64 N/A /home/tidb/dm/data/dm-worker-8262 /home/tidb/dm/deploy/dm-worker-8262 172.31.0.232:3000 grafana 172.31.0.232 3000 linux/x86_64 Down - /home/tidb/dm/deploy/grafana-3000 172.31.0.232:9090 prometheus 172.31.0.232 9090 linux/x86_64 Down /home/tidb/dm/data/prometheus-9090 /home/tidb/dm/deploy/prometheus-9090 Total nodes: 9 ``` + **销毁 DM 集群，如下:** ``` # tiup dm destroy dm-test ``` + **确认 DM 不存在，如下:** ``` # tiup dm display dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test Error: dm cluster `dm-test` not exists Verbose debug logs has been written to /root/.tiup/logs/tiup-dm-debug-2022-06-26-20-26-41.log. ``` ----------------------------------------------- ## 输出样例 ----------------------------------------------- #### 步骤 1-缩容现有 DM 集群,将 172.31.0.175 的 dm-worker 节点进行下线输出参考： ``` # tiup dm scale-in dm-test -N 172.31.0.175:8262 ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm scale-in dm-test -N 172.31.0.175:8262 This operation will delete the 172.31.0.175:8262 nodes in `dm-test` and all their data. Do you want to continue? [y/N]:(default=N) y ... + Refresh instance configs - Generate config dm-master -> 172.31.0.232:8261 ... Done - Generate config dm-master -> 172.31.0.233:8261 ... Done - Generate config dm-master -> 172.31.0.175:8261 ... Done - Generate config dm-worker -> 172.31.0.232:8262 ... Done - Generate config dm-worker -> 172.31.0.233:8262 ... Done - Generate config prometheus -> 172.31.0.232:9090 ... Done - Generate config grafana -> 172.31.0.232:3000 ... Done - Generate config alertmanager -> 172.31.0.232:9093 ... Done + Reload prometheus and grafana - Reload prometheus -> 172.31.0.232:9090 ... Done - Reload grafana -> 172.31.0.232:3000 ... Done Scaled cluster `dm-test` in successfully ``` ----------------------------------------------- #### 步骤 2-使用扩容配置文件,扩容现有 DM 集群输出参考： ``` # tiup dm scale-out dm-test dm-scale.yaml -uroot -p ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm scale-out dm-test dm-scale.yaml -uroot -p Input SSH password: + Detect CPU Arch Name - Detecting node 172.31.0.175 Arch info ... Done + Detect CPU OS Name - Detecting node 172.31.0.175 OS info ... Done Please confirm your topology: Cluster type: dm Cluster name: dm-test Cluster version: v6.1.0 Role Host Ports OS/Arch Directories ---- ---- ----- ------- ----------- dm-worker 172.31.0.175 8262 linux/x86_64 /home/tidb/dm/deploy/dm-worker-8262,/home/tidb/dm/data/dm-worker-8262 Attention: 1. If the topology is not what you expected, check your yaml file. 2. Please confirm there is no port/directory conflicts in same host. Do you want to continue? [y/N]: (default=N) y ... + Refresh components conifgs - Generate config dm-master -> 172.31.0.232:8261 ... Done - Generate config dm-master -> 172.31.0.233:8261 ... Done - Generate config dm-master -> 172.31.0.175:8261 ... Done - Generate config dm-worker -> 172.31.0.232:8262 ... Done - Generate config dm-worker -> 172.31.0.233:8262 ... Done - Generate config dm-worker -> 172.31.0.175:8262 ... Done - Generate config prometheus -> 172.31.0.232:9090 ... Done - Generate config grafana -> 172.31.0.232:3000 ... Done - Generate config alertmanager -> 172.31.0.232:9093 ... Done + Reload prometheus and grafana - Reload prometheus -> 172.31.0.232:9090 ... Done - Reload grafana -> 172.31.0.232:3000 ... Done Scaled cluster `dm-test` out successfully ``` ----------------------------------------------- #### 步骤 4-关闭 DM 集群输出参考： ``` # tiup dm stop dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm stop dm-test Will stop the cluster dm-test with nodes: , roles: . Do you want to continue? [y/N]:(default=N) y ... Stopping component alertmanager Stopping instance 172.31.0.232 Stop alertmanager 172.31.0.232:9093 success Stopping component grafana Stopping instance 172.31.0.232 Stop grafana 172.31.0.232:3000 success Stopping component prometheus Stopping instance 172.31.0.232 Stop prometheus 172.31.0.232:9090 success Stopping component dm-worker Stopping instance 172.31.0.175 Stopping instance 172.31.0.232 Stopping instance 172.31.0.233 Stop dm-worker 172.31.0.233:8262 success Stop dm-worker 172.31.0.232:8262 success Stop dm-worker 172.31.0.175:8262 success Stopping component dm-master Stopping instance 172.31.0.175 Stopping instance 172.31.0.232 Stopping instance 172.31.0.233 Stop dm-master 172.31.0.233:8261 success Stop dm-master 172.31.0.232:8261 success Stop dm-master 172.31.0.175:8261 success Stopped cluster `dm-test` successfully ``` ----------------------------------------------- #### 步骤 4-销毁 DM 集群输出参考： ``` # tiup dm destroy dm-test ``` ``` tiup is checking updates for component dm ... Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm destroy dm-test ██ ██ █████ ██████ ███ ██ ██ ███ ██ ██████ ██ ██ ██ ██ ██ ██ ████ ██ ██ ████ ██ ██ ██ █ ██ ███████ ██████ ██ ██ ██ ██ ██ ██ ██ ██ ███ ██ ███ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ███ ███ ██ ██ ██ ██ ██ ████ ██ ██ ████ ██████ This operation will destroy dm v6.1.0 cluster dm-test and its data. Are you sure to continue? (Type "Yes, I know my cluster and data will be deleted." to continue) : Yes, I know my cluster and data will be deleted. ... Clean global directories 172.31.0.175 success Delete public key 172.31.0.232 Delete public key 172.31.0.232 success Delete public key 172.31.0.233 Delete public key 172.31.0.233 success Delete public key 172.31.0.175 Delete public key 172.31.0.175 success Destroyed cluster `dm-test` successfully ``` -----------------------------------------------