[Music] We have briefly gone through TiDB server. We have introduced its server modules and its stateless nature. Now let's proceed to the architecture of TiKV cluster. Let's dig a little deeper into key value pair domain. We already learned TiKV is a key value model. TiKV provides an order traverse method. Key value pairs in the map are ordered according to the binary order of the key, meaning you can seek the position of the key and then continuously call the next() method to get the largest key in increasing order. In order to implement horizontal scaling, storage data will be spread across multiple nodes. TiKV divides the entire KV space into many segments according to the key range. Each segment is a series of consecutive keys. A region has several segments. The default size of one region is 96 megabytes. Each region can be described as the start key and end key with left close and right open interval. Region is also a replication unit. Each region has its own Raft replication group. Just like the region 1 shown in the slide. Each region has its own leader and follower nodes. Raft is a consistency protocol. Raft provides several important functions, such as leader election and member change management. Concretely, Raft is responsible for adding replicas, deleting replicas, transferring leaders, log replication, etc. When TiKV uses Raft data replication, each data changed will be placed in a Raft log. Following Raft log replication function, the data is securely and reliably synchronized to each node of the replication group. According to the Raft protocol, it is only necessary to synchronously replicate to majority nodes within the Raft group to safely affirm that data has been written successfully. TiKV does not write a data directly to the disk, but saves the data in RocksDB, which is responsible for the data landing. RocksDB is a single instance KV storage engine, open-sourced by Facebook. It was folked from Google's LevelDB originally. RocksDB can meet various requirements from TiKV as a single instance persistent engine. By distributing and replicating data by region, TiKV has become a distributed key value system with a certain disaster recovery capability as long as the majority TiKV nodes are alive. Any minority node failure does not break the data consistency. Now it's demo time again. I'm going to kill my TiKV server instance in the playground and see what happens. Let's check the status of the playground. This time my target is the TiKV instance. Please be noted my simple applications are still running. Just a headsup, there are two roles in my simple applcation. One keeps inserting randomly generated planet data into the planet table, while the other keeps reading the data and does some simple calculation. Detailed descriptions of my simple applcation can be found in the previous lecture. Let me kill one TiKV instance. Looks like nothing happened from the application side. Based on this result we can tell that killing one TiKV node does not affect my application. This behavior benefits from two facts. Firstly, the default count for reading replicas in my example is three, with one TiKV node gone, two are still alive. Thus, the majority is still alive. According to Raft protocol, the leader is there, no matter whether it was a newly elected leader or not, the leader is working so the system is open for read and write. Secondly, because the TiDB servers decoupled communication between the client application and the TiKV server. The SQL layer in the middle actually hides the TiKV node failure for the front-end application. However, from the operation point of view, if this is a production environment you already received an alarm about the unfortunate change in the back end, which is one TiKV node was down. You can config to react automatically or manually. The most important thing is that your application feels nothing. I am not going to kill one more TiKV instance again, similar to what I did on TiDB instance before. Because by doing this, it will break the majority rule in Raft protocol. You will need database operation skills to recover the cluster. Remember TiKV cluster is a stateful. We cover TiKV cluster recovery skills in TiDB administration course in database operation learning path. Please check the training catalog on education portal website for details. Let's go up one level from Raft protocol. Let's talk about MVCC which means multiversion concurrency control. The version number comes from timestamp oracle. When a row or a key value pair is updated, both old and the new entries are stored in TiKV. Essentially it is implemented by adding the version number after the key. The older version entries can be used to support various scenarios like flashback query, flashback table, and most importantly, the concurrent operations. Imagine there are two sessions updating the value of a key at almost the same time. Please note that for timestamp oracle or TSO, there is never a concept call the same time. The different two changes always have different version numbers. However in SQL layer TiDB still has the flexibility to implement the pessimistic and the optimistic transaction lock. We will cover these two locks in TiDB specific features course. Another key feature provided by TiKV is the operator push down and the distributed SQL. Because of the parallel execution nature, it can improve query performance. As we mentioned before, TiKV nodes are workhorses for distributed SQL. As shown on the slide the query needs to fetch the count from all planets that are in category 1. To avoid a large number of RPC calls between the TiDB and the TiKV servers, the predicate condition category ID equals 1 in the query will be pushed down to the storage node for processing. In addition to that, aggregate function count can also be pushed down to the storage nodes for pre-aggregation. Thus, each TiKV node only needs to return one count. Then, total counts are summed up by the SQL layer which is the TiDB server, so that only valid rows are returned and the meaningless network transfers are avoid. Let’s now compare TiDB's architecture with the ones of traditional databases. Both TiDB and many traditional monolithic, or non distributed, databases support ACID and strong consistency. However, TiDB has a distributed architecture. It decouples the SQL layer, storage layer and the management layer. Thus, TiDB is able to elastically scale in and scale out, with almost zero impact on your business. TiDB is also highly available. It supports automatic fail-over. A few failed replicas will be totally transparent to your business. Scaling up TiDB is also very convenient. There is no need to change your business logic or add sharding keys. TiDB is also designed to deal with highly concurrent situations. It can handle more concurrent reading and writing requests than traditional databases. Traditional databases also have some attempts to scale out. This slide shows one of those attempts: Multi Nodes Shared Storage Architecture. The high level idea of this architecture is to put existing hardwares together to act, as if it is a better hardware. It is actually not scaling out. It is still a means of scaling up. However, no matter how many nodes are configured for the cluster, the data is stored on the shared storage and the heartbeat in the middle is responsible for transmitting memory-level buffer cache as well, which is an obvious bottleneck. Moreover, this solution is very expensive, due to some special hardwares, such as network connections, are needed. Another more popular way is to use sharding. Data is put into multiple traditional databases according to the shard key. It is not uncommon in the real-world to have more than 50 shards of a database. By doing this, the database management system can no longer provide ACID properties for cross sharding transactions. The application has to implement ACID logic for cross sharding transactions by themselves. It is a nightmare for operations and management.