## Module 2: Deploy A TiDB Cluster ### Step 1. View the topology file + View the provided topology file `solution-topology-ten-nodes.yaml`, which is used to deploy the TiDB cluster. Note that the IP addresses shown in the `sample_output` will differ from yours. ```bash cat solution-topology-ten-nodes.yaml ``` ```output global: user: "ec2-user" ssh_port: 22 deploy_dir: "/tidb-deploy" data_dir: "/tidb-data" arch: "amd64" server_configs: pd: replication.location-labels: ["region","zone"] replication.max-replicas: 3 pd_servers: - host: 10.90.3.184 client_port: 2379 peer_port: 2380 advertise_client_addr: "http://35.88.150.142:2379" config: server.labels: region: ${LAB:REGION_NAME} zone: az1 - host: 10.90.3.28 client_port: 2379 peer_port: 2380 advertise_client_addr: "http://18.236.135.17:2379" config: server.labels: region: ${LAB:REGION_NAME} zone: az1 - host: 10.90.3.247 client_port: 2379 peer_port: 2380 advertise_client_addr: "http://34.219.57.252:2379" config: server.labels: region: ${LAB:REGION_NAME} zone: az1 tidb_servers: - host: 10.90.1.164 port: 4000 status_port: 10080 deploy_dir: /tidb-deploy/tidb-4000 log_dir: /tidb-deploy/tidb-4000/log - host: 10.90.1.14 port: 4000 status_port: 10080 deploy_dir: /tidb-deploy/tidb-4000 log_dir: /tidb-deploy/tidb-4000/log tikv_servers: - host: 10.90.3.197 port: 20160 status_port: 20180 config: server.labels: region: ${LAB:REGION_NAME} zone: az1 - host: 10.90.3.39 port: 20160 status_port: 20180 config: server.labels: region: ${LAB:REGION_NAME} zone: az1 - host: 10.90.3.82 port: 20160 status_port: 20180 config: server.labels: region: ${LAB:REGION_NAME} zone: az1 tiflash_servers: - host: 10.90.4.194 tcp_port: 9000 flash_service_port: 3930 deploy_dir: /tidb-deploy/tiflash-9000 data_dir: /tidb-data/tiflash-9000 log_dir: /tidb-deploy/tiflash-9000/log monitoring_servers: - host: 10.90.4.156 grafana_servers: - host: 10.90.4.156 alertmanager_servers: - host: 10.90.4.156 ``` ### Step 2. Install required packages + Make sure to install `numactl` on all cluster nodes (the task should be already completed in the classroom environment, so there is no need to actually execute the installation on all nodes.) Enter the following command at the terminal prompt to install `numactl` on `TiKV server 1` for example (type `yes` if prompt for `SSH`): ```bash ssh -i ~/.ssh/pe-class-key-${LAB:REGION_NAME}.pem ec2-user@${HOST_KV1_PRIVATE_IP} ``` ```bash sudo yum -y install numactl ``` ```output Loaded plugins: extras_suggestions, langpacks, priorities, update-motd amzn2-core | 3.6 kB 00:00:00 60 packages excluded due to repository priority protections Package numactl-2.0.9-7.amzn2.x86_64 already installed and latest version Nothing to do ``` + Log off from the `TiKV server 1` and return to the TiUP control machine's prompt: ```bash exit ``` ### Step 3. Check system requirements by using `tiup cluster check` + Check system settings for cluster deployment prerequisites. Enter the following command at the terminal prompt and receive the results shown. You will find that some items fail to pass the system check. We will address this issue in the next step. ```bash tiup cluster check ./solution-topology-ten-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem ``` ```output + Detect CPU Arch Name - Detecting node 10.90.2.90 Arch info ... ⠙ Shell: host=10.90.2.90, sudo=false, command=`uname -m` - Detecting node 10.90.2.231 Arch info ... ⠙ Shell: host=10.90.2.231, sudo=false, command=`uname -m` - Detecting node 10.90.2.138 Arch info ... ⠙ Shell: host=10.90.2.138, sudo=false, command=`uname -m` - Detecting node 10.90.2.242 Arch info ... ⠙ Shell: host=10.90.2.242, sudo=false, command=`uname -m` ... 10.90.2.97 sysctl Fail fs.file-max = 376482, should be greater than 1000000 10.90.2.97 sysctl Fail net.core.somaxconn = 4096, should be greater than 32768 10.90.2.97 sysctl Fail net.ipv4.tcp_syncookies = 1, should be 0 10.90.2.97 sysctl Fail vm.swappiness = 60, should be 0 10.90.2.97 service Pass service firewalld not found, ignore 10.90.2.97 command Pass numactl: policy: default 10.90.2.97 timezone Pass time zone is the same as the first PD machine: UTC 10.90.2.97 os-version Pass OS is Amazon Linux 2 10.90.2.97 memory Pass memory size is 4096MB 10.90.2.97 limits Fail soft limit of 'nofile' for user 'ec2-user' is not set or too low 10.90.2.97 limits Fail hard limit of 'nofile' for user 'ec2-user' is not set or too low 10.90.2.97 limits Fail soft limit of 'stack' for user 'ec2-user' is not set or too low 10.90.2.97 selinux Pass SELinux is disabled 10.90.2.97 thp Fail THP is enabled, please disable it for best performance ``` ### Step 4. Run the auto fix for the failed checks + Enter the following command at the terminal prompt and receive the results shown. ```bash tiup cluster check ./solution-topology-ten-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --apply ``` ```output ... 10.90.3.79 cpu-governor Warn Unable to determine current CPU frequency governor policy, auto fixing not supported 10.90.3.79 sysctl Fail will try to set 'fs.file-max = 1000000' 10.90.3.79 sysctl Fail will try to set 'net.core.somaxconn = 32768' 10.90.3.79 sysctl Fail will try to set 'net.ipv4.tcp_syncookies = 0' 10.90.3.79 sysctl Fail will try to set 'vm.swappiness = 0' + Try to apply changes to fix failed checks - Applying changes on 10.90.2.231 ... Done - Applying changes on 10.90.2.138 ... Done - Applying changes on 10.90.2.242 ... Done - Applying changes on 10.90.2.158 ... Done - Applying changes on 10.90.1.56 ... Done - Applying changes on 10.90.4.138 ... Done - Applying changes on 10.90.2.90 ... Done - Applying changes on 10.90.1.169 ... Done - Applying changes on 10.90.4.126 ... Done - Applying changes on 10.90.2.95 ... Done ``` ### Step 5. Deploy a TiDB cluster named `tidb-demo` + Enter the following command at the terminal prompt and receive the results shown. We include `--yes` in the command. Without `--yes`, you will be prompted to check the cluster topology during the deployment. ```bash tiup cluster deploy tidb-demo ${LAB:TIDB_VERSION} ./solution-topology-ten-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --yes ``` ```output + Detect CPU Arch Name - Detecting node 10.90.2.90 Arch info ... ⠙ Shell: host=10.90.2.90, sudo=false, command=`uname -m` - Detecting node 10.90.2.231 Arch info ... ⠙ Shell: host=10.90.2.231, sudo=false, command=`uname -m` - Detecting node 10.90.2.138 Arch info ... ⠙ Shell: host=10.90.2.138, sudo=false, command=`uname -m` - Detecting node 10.90.2.242 Arch info ... ⠙ Shell: host=10.90.2.242, sudo=false, command=`uname -m` ... Cluster `tidb-demo` deployed successfully, you can start it with command: `tiup cluster start tidb-demo --init` ```