1 00:00:08,640 --> 00:00:14,700 Welcome to the 101 TiDB Technical Essentials course, this is module 1 2 00:00:14,700 --> 00:00:20,100 TiDB database architecture overview. My name is Jeff and I am your instructor. For this module 3 00:00:20,820 --> 00:00:27,000 after completing this module you will understand the overall architecture of TiDB database and 4 00:00:27,000 --> 00:00:34,800 describe the main features of TiDB server, TiKV and the PD. Before we get started 5 00:00:34,800 --> 00:00:41,340 there's something you need to know when we say TiDB we can mean one of two things. The first is the 6 00:00:41,340 --> 00:00:47,580 database itself it's called TiDB, second there is a TiDB component within the TiDB database. 7 00:00:48,240 --> 00:00:54,360 Once you understand this it shouldn't cause a problem. Let's move on to TiDB architecture. 8 00:00:56,040 --> 00:01:01,740 TiDB is a distributed relational database that supports both online transaction processing and 9 00:01:01,740 --> 00:01:07,740 online analytical processing. Databases that support both OLTP and OLAP use cases 10 00:01:07,740 --> 00:01:13,020 are also known as Hybrid Transactional Analytical databases or HTAP databases. 11 00:01:13,740 --> 00:01:19,800 TiDB has horizontal scale out and scale in, financial grade high availability, 12 00:01:20,580 --> 00:01:27,840 real-time HTAP and Cloud native distributed support. In addition, TiDB is compatible with 13 00:01:27,840 --> 00:01:37,140 the MySQL 5.7 client protocol and it is compatible with the MySQL ecosystem. The three components of 14 00:01:37,140 --> 00:01:46,380 the TiDB database are PD cluster, TiDB server, and TiKV. The PD cluster is the control center of 15 00:01:46,380 --> 00:01:52,980 the entire database, think of it as the brains of TiDB database. It is responsible for generating 16 00:01:52,980 --> 00:02:00,300 Global Timestamp Oracle timestamps, scheduling the distribution of regions in TiKV nodes, 17 00:02:00,300 --> 00:02:07,260 collecting the health and distribution status of TiKV nodes and assigning transaction IDs to 18 00:02:07,260 --> 00:02:13,620 distributed transactions. PD itself is also composed of at least three nodes and has 19 00:02:13,620 --> 00:02:19,500 high availability capabilities. We recommend that you deploy an odd number of PD nodes. 20 00:02:20,940 --> 00:02:27,240 The TiDB server, this is the component I mentioned previously it is responsible for handling client 21 00:02:27,240 --> 00:02:33,660 Connections in other words when you connect to a TiDB database your client will connect to TiDB 22 00:02:33,660 --> 00:02:40,980 server component. The TiDB server is responsible for client connections, parsing SQL statements, 23 00:02:40,980 --> 00:02:46,860 compiling SQL statements, generating execution plans, and returning results to the client 24 00:02:46,860 --> 00:02:52,440 application. Execution plans are the steps the database will take in order to satisfy the query. 25 00:02:53,340 --> 00:03:01,080 You may be asking yourself where does TiDB store its data? That's a great question, the OLTP data 26 00:03:01,080 --> 00:03:07,560 is stored in the TiKV cluster in the form of key value pairs. The TiKV cluster is responsible for 27 00:03:07,560 --> 00:03:13,140 data persistence concurrency control, filtering data at the storage level so that queries run 28 00:03:13,140 --> 00:03:19,500 fast, High Availability and ensuring that the data is consistent. We will discuss how this is 29 00:03:19,500 --> 00:03:27,720 done in the TiKV module. The TiDB server layer is stateless, in practice we will run multiple TiDB 30 00:03:27,720 --> 00:03:33,060 server instances. Each of these instances will have its own IP address that users can use to 31 00:03:33,060 --> 00:03:39,660 connect to but this isn't how we do it in practice we want to provide a single IP address to the 32 00:03:39,660 --> 00:03:48,480 outside world through a load balancer such as LVS, HA proxy or F5. Using a load balancer ensures that 33 00:03:48,480 --> 00:03:55,740 the client connections can be evenly distributed across multiple TiDB server instances. TiFlash is 34 00:03:55,740 --> 00:04:03,060 a columnar storage engine that provides the OLAP capabilities to TiDB. TiKV is the primary target 35 00:04:03,060 --> 00:04:09,360 for the data. TiKV will synchronize its data with TiFlash using the Raft consensus algorithm, 36 00:04:09,360 --> 00:04:17,040 this ensures load balancing and strong consistent reads. TiFlash also supports real-time updates. 37 00:04:17,040 --> 00:04:26,220 The combination of TiKV and TiFlash is what makes TiDB a HTAP database. Next, we will discuss the 38 00:04:26,220 --> 00:04:33,000 main functions implementation principles and features of TIDB, TiKV, TiFlash and PD. 39 00:04:34,920 --> 00:04:40,560 Let's take a detailed look at the main functions and implementation principles of TiDB server. 40 00:04:40,560 --> 00:04:46,920 TiDB server itself is only responsible for the SQL Computing engine and does not store data, 41 00:04:46,920 --> 00:04:54,240 so TiDB server is stateless. When our system faces a large number of concurrent connections, we can 42 00:04:54,240 --> 00:05:00,300 load balance by adding TiDB server instances to improve high concurrency processing capabilities. 43 00:05:01,740 --> 00:05:08,100 The first thing we need to look at is one handling client connections. Users 44 00:05:08,100 --> 00:05:14,160 are authenticated by username and password and the connected users are empowered, that means 45 00:05:14,160 --> 00:05:20,460 that they can query the database these tasks are mainly done by the protocol layer module. 46 00:05:21,360 --> 00:05:27,420 Number two is parse and compile SQL statements. After listening to the SQL statement. 47 00:05:28,140 --> 00:05:34,200 TiDB server parses and compiles it during the compilation process it optimizes and 48 00:05:34,200 --> 00:05:39,720 generates an execution plan, these tasks are done by the parse and compile modules. 49 00:05:41,340 --> 00:05:49,020 Relational and KV data transformation. Since data is stored on TiKV in the form of key value pairs, 50 00:05:49,020 --> 00:05:53,820 there's a big difference between relational database tables which have tables, 51 00:05:53,820 --> 00:06:00,960 indexes and rows and a key value pair structure. If you want to execute SQL, you must be able to convert 52 00:06:00,960 --> 00:06:09,720 between relational data and KV storage. Next, SQL statement execution. According to the compiled 53 00:06:09,720 --> 00:06:18,000 execution plan, it is executed on TiKV in batches. The data is read by the executor module and the 54 00:06:18,000 --> 00:06:24,960 DistSQL module. And the transaction related operations are completed by the transaction 55 00:06:24,960 --> 00:06:33,240 module and the KV module. Execution of online DDL. The TiDB database is designed so that data 56 00:06:33,240 --> 00:06:39,780 definition language operations do not lock tables. This is done by the schema load module the worker 57 00:06:39,780 --> 00:06:47,100 module and the start job module. And finally, garbage collection. The garbage collection 58 00:06:47,100 --> 00:06:53,040 back-end module is responsible for recovering data from old versions that have expired and 59 00:06:53,040 --> 00:06:59,640 this part will be explained in the TiKV course. Next, let's take a look at the TiKV module. 60 00:07:02,040 --> 00:07:08,700 We've mentioned TiKV a number of times, it handles storing our relational data that is OLTP data. 61 00:07:09,600 --> 00:07:16,020 Persistent data. TiKV provides the TiDB database with persistent relational storage 62 00:07:16,800 --> 00:07:22,680 to make this happen PingCAP chose the RocksDB storage engine. RocksDB is based 63 00:07:22,680 --> 00:07:30,120 on the LSM tree data structure. RocksDB is the persistent storage engine for TiKV. 64 00:07:30,720 --> 00:07:36,360 The RocksDB engine uses mechanisms such as write-ahead logs to ensure that data is not lost. 65 00:07:36,360 --> 00:07:43,920 Next bullet, distributed transaction support, TiDB uses a two-phase commit approach that is based on 66 00:07:43,920 --> 00:07:49,020 the Percolator transaction model published by Google. The Percolator transaction model 67 00:07:49,020 --> 00:07:56,040 ensures that storage lock space is allocated separately on each KV node, this uses RocksDB 68 00:07:56,040 --> 00:08:03,120 column family locks, this is also called a column cluster. The Timestamp Oracle and the PD also 69 00:08:03,120 --> 00:08:09,960 play a part in this process. The TiDB database supports two locking modes, optimistic locking 70 00:08:09,960 --> 00:08:17,160 and pessimistic locking. Next bullet, strong consistency and high availability of replicas. 71 00:08:18,360 --> 00:08:24,420 In the Raft algorithm data is stored in regions, each region has three replicas, 72 00:08:24,420 --> 00:08:32,040 by default a replica cannot be larger than 96 megabytes, if a replica exceeds this size it is 73 00:08:32,040 --> 00:08:37,440 split into multiple regions, this helps ensure that TiDB can scale dynamically. 74 00:08:38,460 --> 00:08:47,640 Next is MVCC or multi-version concurrency control. MVCC is essential for highly concurrent reads and 75 00:08:47,640 --> 00:08:53,340 writes. It is the basis for functions such as concurrency control, snapshot isolation level, 76 00:08:53,340 --> 00:09:03,060 distributed transactions and fast data recovery. TiKV's MVCC implementation is done by adding a 77 00:09:03,060 --> 00:09:08,280 version number after the key and after the historical version data reaches a certain 78 00:09:08,280 --> 00:09:17,460 period of time, it is cleaned up by TiDB server garbage collection. Next up, coprocessor. This 79 00:09:17,460 --> 00:09:23,820 is really operator push down. The co-processor function enables TiKV to perform data filtering, 80 00:09:23,820 --> 00:09:31,380 partial aggregations, etc on the local node, so that the SQL calculation can be pushed down to the 81 00:09:31,380 --> 00:09:36,660 distributed storage nodes to complete it. This mechanism not only reduces Network interaction 82 00:09:36,660 --> 00:09:43,680 costs, but also makes full use of the Computing resources, CPU, cache and memory of the TiDB 83 00:09:43,680 --> 00:09:49,200 server and also makes full use of the distributed storage nodes parallel processing power. 84 00:09:50,460 --> 00:09:54,600 Next up we're going to talk about the placement driver, do you remember the placement driver? 85 00:09:55,380 --> 00:10:01,080 We mentioned it earlier it's most commonly referred to as the PD. It is the brains of 86 00:10:01,080 --> 00:10:08,040 the TiDB database. The PD is responsible for metadata storage for the entire TiKV cluster. 87 00:10:08,040 --> 00:10:12,660 The PD is responsible for maintaining where data lives on the TiDB cluster, 88 00:10:12,660 --> 00:10:16,440 it is responsible for maintaining information used by Raft as well. 89 00:10:17,160 --> 00:10:22,500 TiDB servers obtain routing information from the PD and then use that information 90 00:10:22,500 --> 00:10:30,540 to access data in the TiKV cluster. In addition, topological label information is stored in the PD, 91 00:10:30,540 --> 00:10:36,720 we will cover this in an upcoming module. I previously made a recommendation regarding the PD, 92 00:10:36,720 --> 00:10:41,520 do you remember what it was? I'm going to make an uncomfortable pause here. 93 00:10:46,080 --> 00:10:52,020 That was uncomfortable. The recommendation was there should be an odd number of PD nodes 94 00:10:52,020 --> 00:10:59,700 in the PD cluster. We'll cover why that is in an upcoming module. Next, the PD is responsible for 95 00:11:00,660 --> 00:11:06,960 assigning a global ID and a transaction ID. The global ID we use such as store ID, 96 00:11:06,960 --> 00:11:14,520 region ID, peer ID, table ID index ID, etc and the transaction ID of the distributed 97 00:11:14,520 --> 00:11:22,980 transactions are also assigned by the PD. The PD is responsible for generating a global timestamp 98 00:11:22,980 --> 00:11:29,160 using the Timestamp Oracle. Timestamp Oracle is a global timestamp used to guarantee the order 99 00:11:29,160 --> 00:11:36,840 of transactions in the database. It's very important for the database to know the exact 100 00:11:36,840 --> 00:11:42,000 order of transactions in case we have to roll things back we will introduce the details later. 101 00:11:43,380 --> 00:11:50,760 The PD is responsible for collecting cluster information for scheduling. The TiKV node 102 00:11:50,760 --> 00:11:56,880 periodically reports the status of the cluster to PD. The PD is responsible for scheduling the 103 00:11:56,880 --> 00:12:04,200 region among TiKV nodes according to relevant status. This has to do with what we talked about 104 00:12:04,200 --> 00:12:12,120 splitting replicas at 96 megabytes, etc, in the previous section. We will introduce the details 105 00:12:12,120 --> 00:12:20,400 of this later. And finally, the PD is responsible for providing TiDB dashboard service. The PD hosts 106 00:12:20,400 --> 00:12:26,400 a small web server that runs the TiDB dashboard. The dashboard provides a place where you can 107 00:12:26,400 --> 00:12:31,380 easily find information about your TiDB database. Once again we will discuss this later as well. 108 00:12:33,540 --> 00:12:39,900 The second component in our storage story is TiFlash, recall that TiFlash implements the 109 00:12:39,900 --> 00:12:47,460 analytics side of HTAP. This is a good time to mention that TiKV as the OLTP side of storage is 110 00:12:47,460 --> 00:12:54,180 required. PinpCAP has customers whose use cases don't require analytics, these customers only 111 00:12:54,180 --> 00:13:02,640 use TiKV for their storage. If really when they decide they need enhanced analytics capabilities 112 00:13:02,640 --> 00:13:08,940 they can easily install TiFlash thereby adding OLAP capabilities to their TiDB environment. 113 00:13:09,480 --> 00:13:12,900 Here are some capabilities which make TiFlash special. 114 00:13:14,760 --> 00:13:22,980 Asynchronous replication. TiKV is the primary storage component in fact TiFlash cannot work 115 00:13:22,980 --> 00:13:30,600 without it. Another way to say this I can have a TiDB database with only TiKV but I can't 116 00:13:30,600 --> 00:13:37,320 have one with only TiFlash. When you insert update or load data into TiDB it is written 117 00:13:37,320 --> 00:13:45,060 to TiKV first then replicated to TiFlash. This operation has performed asynchronous. There's 118 00:13:45,060 --> 00:13:50,220 even a special Raft role that performs this task, the Raft Learner handles this. 119 00:13:51,420 --> 00:13:58,080 If a TiFlash node goes down or if the network has high latency the data can be read from TiKV, 120 00:13:58,080 --> 00:14:05,160 This redundancy ensures normal operations. This replication mechanism also inherits the automatic 121 00:14:05,160 --> 00:14:10,680 load balancing and high availability of the TiKV system, it does not rely on additional 122 00:14:10,680 --> 00:14:16,560 replication pipelines, but directly receives TiKV data transmissions in a one-to-many 123 00:14:16,560 --> 00:14:23,220 manner as long as the data in TiKV is not lost a replica of TiFlash can be created at any time. 124 00:14:25,320 --> 00:14:32,880 Next, consistency TiFlash provides the same snapshot isolation level support as TiKV, 125 00:14:32,880 --> 00:14:39,000 this ensures that the read data is up to date and that is ensuring that previously written 126 00:14:39,000 --> 00:14:46,440 data can be read this consistency is achieved by verifying the replication progress of the data. 127 00:14:46,440 --> 00:14:52,500 Each time our read request is received, the region replica and TiFlash initiates a progress 128 00:14:52,500 --> 00:15:00,780 check a very light procedure call request to the leader node and only responds to reads if 129 00:15:00,780 --> 00:15:06,540 the process ensures that it contains at least the data covered by the read request timestamp. 130 00:15:08,100 --> 00:15:15,180 Next, number three or the third one. Column oriented store improves analytics query 131 00:15:15,180 --> 00:15:22,140 efficiency. Yeah we've said this before but it's worth repeating, TiFlash is a columnar data store. 132 00:15:22,680 --> 00:15:27,120 Columnar data stores were designed for analytics workflows, row based storage 133 00:15:27,120 --> 00:15:33,900 is best used for transactional processing, so row based storage is ideal for use cases 134 00:15:33,900 --> 00:15:39,420 involving indexes and small result sets. Columnar storage is ideal for use cases 135 00:15:39,420 --> 00:15:45,120 involving table scans and analytics techniques that require reading the data multiple times. 136 00:15:46,440 --> 00:15:53,040 The introduction of MPP are massively parallel processing functional architecture distributes 137 00:15:53,040 --> 00:15:59,340 computing tasks to multiple server nodes in parallel. After the calculation is completed 138 00:15:59,340 --> 00:16:04,500 on each node, the results of each part are summarized together to obtain the final answer. 139 00:16:04,500 --> 00:16:10,440 This processing model makes the analysis query service on TiFlash more efficient. 140 00:16:11,400 --> 00:16:17,400 Next, business isolation. This is a good one, as mentioned earlier the column store is very 141 00:16:17,400 --> 00:16:24,360 friendly to statistical analysis queries or OLAP and the row store is friendly to transaction OLTP, 142 00:16:24,360 --> 00:16:32,880 so TiKV and TiFlash can be used for two distinct business scenarios or use cases, OLTP and OLAP. 143 00:16:33,480 --> 00:16:40,980 Using one environment for both of these use cases saves admin and processing cost. Plus 144 00:16:40,980 --> 00:16:47,040 and this is huge it means that you don't need to create and manage OLTP to OLAP 145 00:16:47,040 --> 00:16:50,820 ETL processes, Thus saving more time and resources. 146 00:16:52,020 --> 00:16:59,280 And finally this one's my favorite, whether it's TiKV or TiFlash their SQL entry is the 147 00:16:59,280 --> 00:17:05,580 TiDB server. So what does that mean? This means that your queries for both sides go through TiDB 148 00:17:05,580 --> 00:17:15,300 server and the optimizer in TiDB server can choose between TiKV and TiFlash based on the query. So, 149 00:17:15,300 --> 00:17:23,280 this is based on the SQL statement. So, as a DBA or developer, you write the query and TiDB chooses 150 00:17:23,280 --> 00:17:28,860 the best place to go for the data. You don't have to worry about it. Totally transparent to you. 151 00:17:29,820 --> 00:17:35,400 This marks the end of the first module. We've taken a brief look at TiDB and its architecture. 152 00:17:35,400 --> 00:17:43,800 So you can now describe the overall architecture of the TiDB database and you understand ... well, 153 00:17:43,800 --> 00:17:50,160 understand isn't really a goal, let's say you can describe the main functions of TiDB server, TiKV 154 00:17:50,160 --> 00:17:56,820 and the placement driver. Can you describe the difference between TiDB database and TiDB server? 155 00:17:59,820 --> 00:18:06,060 Database refers to the entire database environment. TiDB server on the other hand 156 00:18:06,060 --> 00:18:10,920 refers to the SQL processing component of the TiDB database. 157 00:18:10,920 --> 00:18:15,720 So TiDB server is a component of TiDB database. 158 00:18:17,160 --> 00:18:21,300 Thank you for taking the time to view this module we hope you enjoyed it and learned a great deal.