## Module 1: Deploy a TiDB Cluster Across Three Availability Zones ### Step 0. (Optional) A brief introduction to TiDB - The figure below shows a brief architecture of TiDB when deployed on-premise. - There are three major components of a TiDB Cluster: a Placement Driver (PD) cluster, a SQL layer cluster, and a storage cluster. - The Placement Driver (PD) cluster is the brain of TiDB. It manages metadata, provides Timestamp Oracle (TSO) for transactions, decides data location, etc. - The SQL layer is stateless. There is no business data stored in TiDB Servers. TiDB Servers accept and process SQLs from the clients, and also convert relational data into key-value pairs. - The TiKV in the storage layer is row storage. In TiKVs, data is split into Regions (256 MB by default from version 8.5), and each Region is replicated 3 times by default to form a Raft group. Then, the regions are scheduled into multiple TiKV nodes, in order to achieve high availability (HA). On each TiKV node, the data is persisted into disks by RocksDB. - The TiFlash in the storage layer is a columnar storage, which provides analytical capabilities. When TiFlash is enabled, TiDB cluster automatically decides whether to send your query (or part of a query) to TiFlash for better performance. TiFlash servers are optional in TiDB cluster - you do not have to deploy TiFlash servers, if you have no analytical queries in your workload. - For more details, you can refer to TiDB [documentation](https://docs.pingcap.com/tidb/stable/tidb-architecture/). ![TiDBClusterArchitectureNormal_aa_001_20220506.png 85%](https://lab-static.pingcap.com/images/2025/12/30/c1b377be79b802bc06a4b40ad818c9aa16855450.png) ### Step 1. SSH login to the lab virtual machine - Open three SSH connections to your lab virtual machine by clicking: - [Terminal 1](ssh://monitor1-ssh#1) - [Terminal 2](ssh://monitor1-ssh#2) - [Terminal 3](ssh://monitor1-ssh#3) - You will now have three active terminal sessions. ### Step 2. Start the TiDB cluster across multiple availability zones - In [Terminal 1](ssh://monitor1-ssh#1), review the provisioned cluster topology YAML: ```bash cat nine-nodes.yaml ``` ```output global: user: "ec2-user" ssh_port: 22 deploy_dir: "/tidb-deploy" data_dir: "/tidb-data" arch: "amd64" server_configs: pd: replication.max-replicas: 3 pd_servers: - host: ${LAB:HOST_PD1_PRIVATE_IP} client_port: 2379 peer_port: 2380 advertise_client_addr: "http://${LAB:HOST_PD1_PUBLIC_IP}:2379" - host: ${LAB:HOST_PD2_PRIVATE_IP} client_port: 2379 peer_port: 2380 advertise_client_addr: "http://${LAB:HOST_PD2_PUBLIC_IP}:2379" - host: ${LAB:HOST_PD3_PRIVATE_IP} client_port: 2379 peer_port: 2380 advertise_client_addr: "http://${LAB:HOST_PD3_PUBLIC_IP}:2379" tidb_servers: - host: ${LAB:HOST_DB1_PRIVATE_IP} port: 4000 status_port: 10080 deploy_dir: /tidb-deploy/tidb-4000 log_dir: /tidb-deploy/tidb-4000/log - host: ${LAB:HOST_DB2_PRIVATE_IP} port: 4000 status_port: 10080 deploy_dir: /tidb-deploy/tidb-4000 log_dir: /tidb-deploy/tidb-4000/log tikv_servers: - host: ${LAB:HOST_KV1_PRIVATE_IP} port: 20160 status_port: 20180 - host: ${LAB:HOST_KV2_PRIVATE_IP} port: 20160 status_port: 20180 - host: ${LAB:HOST_KV3_PRIVATE_IP} port: 20160 status_port: 20180 monitoring_servers: - host: ${LAB:HOST_MONITOR1_PRIVATE_IP} grafana_servers: - host: ${LAB:HOST_MONITOR1_PRIVATE_IP} alertmanager_servers: - host: ${LAB:HOST_MONITOR1_PRIVATE_IP} ``` - Fix system configurations in order to deploy a TiDB cluster: ```bash tiup mirror set tidb-community-server-${LAB:TIDB_VERSION}-linux-amd64 tiup cluster check nine-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --apply ``` ```output ... + Try to apply changes to fix failed checks - Applying changes on ${LAB:HOST_PD1_PRIVATE_IP} ... ⠧ Sysctl: host=${LAB:HOST_PD1_PRIVATE_IP} vm.swappiness = 0 + Try to apply changes to fix failed checks - Applying changes on ${LAB:HOST_PD1_PRIVATE_IP} ... ⠇ Sysctl: host=${LAB:HOST_PD1_PRIVATE_IP} vm.swappiness = 0 + Try to apply changes to fix failed checks + Try to apply changes to fix failed checks - Applying changes on ${LAB:HOST_PD1_PRIVATE_IP} ... Done + Try to apply changes to fix failed checks - Applying changes on ${LAB:HOST_PD1_PRIVATE_IP} ... Done + Try to apply changes to fix failed checks - Applying changes on ${LAB:HOST_PD1_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_PD3_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_KV2_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_DB1_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_PD2_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_KV1_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_KV3_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_DB2_PRIVATE_IP} ... Done - Applying changes on ${LAB:HOST_MONITOR1_PRIVATE_IP} ... Done ``` - Deploy a TiDB cluster: ```bash tiup cluster deploy tidb-demo ${LAB:TIDB_VERSION} ./nine-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --yes ``` ```output ... Cluster `tidb-demo` deployed successfully, you can start it with command: `tiup cluster start tidb-demo --init` ``` - Start the deployed TiDB cluster: ```bash tiup cluster start tidb-demo ``` ```output ... Started cluster `tidb-demo` successfully ``` ### Step 3. Verify the status of the TiDB cluster - The cluster's key components are distributed across three subnets: `10.90.1.0/24`, `10.90.2.0/24`, and `10.90.3.0/24`, with each subnet mapped to a separate Availability Zone for high availability. - The cluster includes 3 PD servers, 3 TiKV servers, and 2 TiDB servers. - Pay particular attention to subnet `10.90.1.0/24`, which hosts a PD server, a TiDB server, and a TiKV server. - While the IP addresses in your environment and sample output may vary, the subnet configuration remains consistent. ```bash tiup cluster display tidb-demo ``` ```output Cluster type: tidb Cluster name: tidb-demo Cluster version: v${LAB:TIDB_VERSION} Deploy user: ec2-user SSH type: builtin Dashboard URL: http://${LAB:HOST_PD1_PUBLIC_IP}:2379/dashboard Grafana URL: http://${LAB:HOST_MONITOR1_PRIVATE_IP}:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- ${LAB:HOST_MONITOR1_PRIVATE_IP}:9093 alertmanager ${LAB:HOST_MONITOR1_PRIVATE_IP} 9093/9094 linux/x86_64 Up /tidb-data/alertmanager-9093 /tidb-deploy/alertmanager-9093 ${LAB:HOST_MONITOR1_PRIVATE_IP}:3000 grafana ${LAB:HOST_MONITOR1_PRIVATE_IP} 3000 linux/x86_64 Up - /tidb-deploy/grafana-3000 ${LAB:HOST_PD1_PRIVATE_IP}:2379 pd ${LAB:HOST_PD1_PRIVATE_IP} 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379 ${LAB:HOST_PD2_PRIVATE_IP}:2379 pd ${LAB:HOST_PD2_PRIVATE_IP} 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379 ${LAB:HOST_PD3_PRIVATE_IP}:2379 pd ${LAB:HOST_PD3_PRIVATE_IP} 2379/2380 linux/x86_64 Up|L /tidb-data/pd-2379 /tidb-deploy/pd-2379 ${LAB:HOST_MONITOR1_PRIVATE_IP}:9090 prometheus ${LAB:HOST_MONITOR1_PRIVATE_IP} 9090/12020 linux/x86_64 Up /tidb-data/prometheus-9090 /tidb-deploy/prometheus-9090 ${LAB:HOST_DB1_PRIVATE_IP}:4000 tidb ${LAB:HOST_DB1_PRIVATE_IP} 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000 ${LAB:HOST_DB2_PRIVATE_IP}:4000 tidb ${LAB:HOST_DB2_PRIVATE_IP} 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000 ${LAB:HOST_KV1_PRIVATE_IP}:20160 tikv ${LAB:HOST_KV1_PRIVATE_IP} 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 ${LAB:HOST_KV2_PRIVATE_IP}:20160 tikv ${LAB:HOST_KV2_PRIVATE_IP} 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 ${LAB:HOST_KV3_PRIVATE_IP}:20160 tikv ${LAB:HOST_KV3_PRIVATE_IP} 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 Total nodes: 11 ```