## Module 1: Initialization of TiDB Cluster In this module, participants will deploy a TiDB cluster and create three PLACEMENT POLICYs for use in subsequent modules. This module will utilize the Placement Rules in SQL feature. For more information about the Placement Rules in SQL feature, please refer to: ### Step 1. Connect to the TiUP control machine + Click here - [Terminal 1](ssh://monitor1-ssh#1) to open a new terminal connected to the virtual machine assigned for your hands-on lab. ### Step 2. Examine the topology YAML file of TiDB Cluster + Reviewing the provided topology file `nine-nodes.yaml`, which is used to deploy the TiDB cluster, the `pd` section under `server_configs` was located. It was found that `replication.location-labels` is set to ["region","host"], indicating this TiDB cluster defines a 2-layer topology structure, in the order of `region` and then `host`. ```bash cat nine-nodes.yaml ``` ```output ... arch: "amd64" server_configs: pd: replication.location-labels: ["region","host"] replication.max-replicas: 3 pd_servers: ... ``` + Reviewing the provided topology file `nine-nodes.yaml`, the `server.labels` under the `config` section for the three TiKV nodes within `tikv_servers` was located. It was found that the three TiKV nodes are assigned three distinct labels: `region1`, `region2`, and `region3`. ```output ... tikv_servers: - host: 10.90.1.136 port: 20160 status_port: 20180 config: server.labels: { region: "region1", host: "tikv1" } ... - host: 10.90.1.54 port: 20160 status_port: 20180 config: server.labels: { region: "region2", host: "tikv2" } ... - host: 10.90.1.84 port: 20160 status_port: 20180 config: server.labels: { region: "region3", host: "tikv3" } ... ``` + Reviewing the provided topology file `nine-nodes.yaml`, items prefixed with `in-memory-engine.` under the `config` section for the three TiKV nodes within `tikv_servers` were located, revealing: + For the TiKV node of `region1`, `in-memory-engine.enable` is `false`, while for the TiKV nodes of `region2` and `region3`, `in-memory-engine.enable` is `true`. This indicates that the MVCC In-Memory Engine feature is not enabled on the `region1` TiKV node, but is enabled on the `region2` and `region3` TiKV nodes. + Where the In-Memory Engine feature is enabled, the `in-memory-engine.capacity` for the TiKV nodes of `region1`, `region2`, and `region3` is uniformly set to `1GiB`, representing the usable memory size for the In-Memory Engine. + Where the In-Memory Engine feature is enabled, the `in-memory-engine.mvcc-amplification-threshold` parameter (which sets the MVCC read amplification threshold for selecting Regions to load) is set to `5` for the TiKV nodes of `region1` and `region2`, and to `100` for the TiKV node of `region3`. + Where the In-Memory Engine feature is enabled, the `in-memory-engine.load-evict-interval` for the TiKV nodes of `region1`, `region2`, and `region3` is set to `2m`. This indicates the time interval at which the In-Memory Engine automatically selects data to load based on request statistics is 2 minutes. ```output ... tikv_servers: - host: 10.90.1.136 port: 20160 status_port: 20180 config: server.labels: { region: "region1", host: "tikv1" } in-memory-engine.capacity: 1GiB in-memory-engine.enable: false in-memory-engine.mvcc-amplification-threshold: 5 in-memory-engine.load-evict-interval: 2m - host: 10.90.1.54 port: 20160 status_port: 20180 config: server.labels: { region: "region2", host: "tikv2" } in-memory-engine.capacity: 1GiB in-memory-engine.enable: true in-memory-engine.mvcc-amplification-threshold: 5 in-memory-engine.load-evict-interval: 2m - host: 10.90.1.84 port: 20160 status_port: 20180 config: server.labels: { region: "region3", host: "tikv3" } in-memory-engine.capacity: 1GiB in-memory-engine.enable: true in-memory-engine.mvcc-amplification-threshold: 100 in-memory-engine.load-evict-interval: 2m ... ``` + Based on the provided topology file `nine-nodes.yaml`, a topology diagram for the TiKV cluster can be derived, as shown below: ![IME_Topology.png 70%](https://lab-static.pingcap.com/images/2025/8/18/ab10f088aef5e642b32818f086bb4650822e3622.png) ### Step 3. Deploy and start a TiDB cluster + The commands in the following code block will execute a series of operations in sequence. Which are: 1. Change [TiUP mirror](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) setting, 2. Check system settings, 3. Fix the failed checks, 4. Deploy the TiDB cluster, and finally 5. Start the TiDB cluster. It takes approximately 3 minutes to complete. ```bash tiup mirror set tidb-community-server-${LAB:TIDB_VERSION}-linux-amd64 tiup cluster check nine-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --apply tiup cluster deploy tidb-demo ${LAB:TIDB_VERSION} ./nine-nodes.yaml \ --user ec2-user \ -i /home/ec2-user/.ssh/pe-class-key-${LAB:REGION_NAME}.pem \ --yes tiup cluster start tidb-demo sleep 5; tiup cluster display tidb-demo ``` ```output A new version of cluster is available: -> v1.16.1 To update this component: tiup update cluster To update all components: tiup update --all The component `cluster` version is not installed; downloading from repository. ... [ec2-user@ip-10-90-4-47 ~]$ tiup cluster display tidb-demo Cluster type: tidb Cluster name: tidb-demo Cluster version: v8.5.1 Deploy user: ec2-user SSH type: builtin Dashboard URL: http://54.250.180.235:2379/dashboard Grafana URL: http://10.90.4.47:3000 ID Role Host Ports OS/Arch Status Data Dir Deploy Dir -- ---- ---- ----- ------- ------ -------- ---------- 10.90.4.47:9093 alertmanager 10.90.4.47 9093/9094 linux/x86_64 Up /tidb-data/alertmanager-9093 /tidb-deploy/alertmanager-9093 10.90.4.47:3000 grafana 10.90.4.47 3000 linux/x86_64 Up - /tidb-deploy/grafana-3000 10.90.2.129:2379 pd 10.90.2.129 2379/2380 linux/x86_64 Up|L /tidb-data/pd-2379 /tidb-deploy/pd-2379 10.90.2.203:2379 pd 10.90.2.203 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379 10.90.2.254:2379 pd 10.90.2.254 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379 10.90.4.47:9090 prometheus 10.90.4.47 9090/12020 linux/x86_64 Up /tidb-data/prometheus-9090 /tidb-deploy/prometheus-9090 10.90.1.194:4000 tidb 10.90.1.194 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000 10.90.1.40:4000 tidb 10.90.1.40 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000 10.90.3.147:20160 tikv 10.90.3.147 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 10.90.3.49:20160 tikv 10.90.3.49 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 10.90.3.51:20160 tikv 10.90.3.51 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160 Total nodes: 11 ``` ### Step 4. Initialize TiDB Cluster + Connect to TiDB cluster. ```bash mysql -h ${HOST_DB1_PRIVATE_IP} --port 4000 -u root ``` ```output mysql> ``` + Set `tidb_gc_life_time` to 24 hours to ensure MVCC snapshots are preserved and not garbage collected within this timeframe. ```sql SET GLOBAL tidb_gc_life_time='1440m'; SHOW GLOBAL VARIABLES LIKE 'tidb_gc_life_time'; ``` ```output mysql> SET GLOBAL tidb_gc_life_time='1440m'; Query OK, 0 rows affected (0.01 sec) mysql> SHOW GLOBAL VARIABLES LIKE 'tidb_gc_life_time'; +-------------------+---------+ | Variable_name | Value | +-------------------+---------+ | tidb_gc_life_time | 24h0m0s | +-------------------+---------+ 1 row in set (0.00 sec) ``` + Check whether the In-Memory Engine feature is enabled on each individual TiKV node. ```sql SHOW CONFIG WHERE type='tikv' AND name='in-memory-engine.enable'; ``` ```output mysql> SHOW CONFIG WHERE type='tikv' AND name='in-memory-engine.enable'; +------+-------------------+-------------------------+-------+ | Type | Instance | Name | Value | +------+-------------------+-------------------------+-------+ | tikv | 10.90.3.147:20160 | in-memory-engine.enable | true | | tikv | 10.90.3.51:20160 | in-memory-engine.enable | true | | tikv | 10.90.3.49:20160 | in-memory-engine.enable | false | +------+-------------------+-------------------------+-------+ 3 rows in set (0.04 sec) ``` + Create 3 PLACEMENT POLICYs for use in subsequent modules. For more information about the Placement Rules in SQL feature, please refer to: + PLACEMENT POLICY `IME_R1` specifies that Regions with the `LEADER` role for a table are placed on the TiKV node in `region1`, while Regions with the `FOLLOWER` role are placed on the TiKV nodes in `region2` and `region3`. + PLACEMENT POLICY `IME_R2` specifies that Regions with the `LEADER` role for a table are placed on the TiKV node in `region2`, while Regions with the `FOLLOWER` role are placed on the TiKV nodes in `region1` and `region3`. + PLACEMENT POLICY `IME_R3` specifies that Regions with the `LEADER` role for a table are placed on the TiKV node in `region3`, while Regions with the `FOLLOWER` role are placed on the TiKV nodes in `region1` and `region2`. ```sql DROP PLACEMENT POLICY IF EXISTS IME_R1; DROP PLACEMENT POLICY IF EXISTS IME_R2; DROP PLACEMENT POLICY IF EXISTS IME_R3; CREATE PLACEMENT POLICY IME_R1 PRIMARY_REGION="region1" REGIONS="region1,region2,region3"; CREATE PLACEMENT POLICY IME_R2 PRIMARY_REGION="region2" REGIONS="region1,region2,region3"; CREATE PLACEMENT POLICY IME_R3 PRIMARY_REGION="region3" REGIONS="region1,region2,region3"; SHOW PLACEMENT; ``` ```output mysql> DROP PLACEMENT POLICY IF EXISTS IME_R1; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> DROP PLACEMENT POLICY IF EXISTS IME_R2; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> DROP PLACEMENT POLICY IF EXISTS IME_R3; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> CREATE PLACEMENT POLICY IME_R1 PRIMARY_REGION="region1" REGIONS="region1,region2,region3"; Query OK, 0 rows affected (0.52 sec) mysql> CREATE PLACEMENT POLICY IME_R2 PRIMARY_REGION="region2" REGIONS="region1,region2,region3"; Query OK, 0 rows affected (0.51 sec) mysql> CREATE PLACEMENT POLICY IME_R3 PRIMARY_REGION="region3" REGIONS="region1,region2,region3"; Query OK, 0 rows affected (0.52 sec) mysql> SHOW PLACEMENT; +---------------+------------------------------------------------------------+------------------+ | Target | Placement | Scheduling_State | +---------------+------------------------------------------------------------+------------------+ | POLICY IME_R1 | PRIMARY_REGION="region1" REGIONS="region1,region2,region3" | NULL | | POLICY IME_R2 | PRIMARY_REGION="region2" REGIONS="region1,region2,region3" | NULL | | POLICY IME_R3 | PRIMARY_REGION="region3" REGIONS="region1,region2,region3" | NULL | +---------------+------------------------------------------------------------+------------------+ 3 rows in set (0.01 sec) ``` + Disconnect the client session. ```sql EXIT ``` ```output mysql> EXIT Bye ``` ### Step 5. Prepare Monitoring for the TiDB Cluster + Log in to the [Grafana Dashboard](http://${LAB:HOST_MONITOR1_PUBLIC_IP}:3000) with username `admin` and password `admin`. Participants are encouraged to keep the monitoring pages open from this step and set the monitoring time range to the last 5 or 15 minutes, while changing the refresh rate to 5 seconds. ![70%](https://lab-static.pingcap.com/images/2025/7/24/9c787410ab425dd7eb7e7297cc46711ec63ccc22.png) + At this point, the TiDB cluster setup is complete and ready for learning about the In-Memory Engine feature.