# Gp TiDB Course 101 Potential Non-Standard Narratives
+ 2022-05-14 22:19:12.924602

| Risk | Suggestion | Filename:Instance |
| :-- | :-- | :-- |
|  , | __potential-issue__ | chapter-03/11.md:Second, let's go to the second stage Commit of Percolator. At this time, we will get a commitTs from PD, assuming 110, and then it will be written in the Write column family (CF), key plus timestamp (commit_ts), and value is the startTs of this transaction. , The primary key was deleted in the Lock column family (CF) record.,chapter-05/10.md:1. The Transaction module begins to enter the second stage commit. The second stage commit will be introduced in Lesson 03: TiKV's “Distributed Transaction”. Here we briefly understand it as a two-step operation, the first is the prewrite operation, that is, modify the data and lock, and the second is the commit operation, that is, commit the modify , Release the lock. Well, in these 2 operations, modify data, adding locks, releasing locks, and committing modify are all write operations, so let's introduce write operations later. |
| ‘ | ' | chapter-06/09.md:and product.pct_date > ‘2021-09-30’,chapter-06/09.md:and sub_str(order.dic, 3) = ‘7c0’ |
| ’ | ' | chapter-06/09.md:and product.pct_date > ‘2021-09-30’,chapter-06/09.md:and sub_str(order.dic, 3) = ‘7c0’ |
| “ | " | chapter-02/15.md:The thread pool module related to SQL read/write is introduced below. As for the SQL reads and writes process, we will introduce it in detail in “Lesson 05 TiDB Database SQL Execution Flow”. Here we explain the role of the relevant module.,chapter-02/18.md:2. If the configuration item of oom-action is “cancel”, then when the memory usage of a SQL exceeds a certain threshold, TiDB will immediately interrupt the execution of the SQL and return an error to the client. The error message will specify in detail each physical execution that takes up more memory during the execution of the SQL The memory usage of the operator.,chapter-03/08.md:Support for the Column Families feature started with RocksDB 3.0. Each key value pair (key - value) of RocksDB corresponds to a unique column family (Column Family). If a Column Family is not specified, the key-value pair will correspond to the “default” Column Family (Column Family), which is also the default Column Family of the RocksDB instance.,chapter-03/33.md:First of all, we introduced a concept in “Raft - Leader elect”, which is the heartbeat time interval (the leader sends a heartbeat interval), which requires the heartbeat time interval to be less than min (election) timeout). That is, the leader will send a heartbeat every heartbeat time interval. We record this time point as start. If the majority of followers give a heartbeat response (at this time, you can determine if the leader is this node), Due to Raft's elect mechanism, followers will not initiate re-election after at least the elect timeout period, so even if an elect occurs, the new leader will be elect during the election timeout/clock drift bound. Well, from start to election timeout/clock drift bound, the leader is still the node during this time. We call this period a lease, or a lease period. Well, within a lease, the node does not need to issue a heartbeat to determine that it is still the leader node. In this way, we no longer need to send a heartbeat to determine if we are currently a leader like ReadIndex Read, which improves efficiency.,chapter-04/03.md:PD is the central controller of the TiDB database and is called the “brain” of the entire cluster. In the TiDB database cluster, it is responsible for storing metadata (mainly providing routes for TiDB Server, etc.), assigning global ID, transaction ID, global timestamp TSO generation, service discovery, collection of information such as the overall topology of the cluster, real-time data distribution and hot spot status, and scheduling of data in TiKV nodes. At the same time, TiDB Dashboard services are provided.,chapter-04/08.md:The current PD (leader) has a TSO of 700 (virtual value for ease of explanation), and PD has applied etcd for an “allocatable time window”. To know that the span of the time window can be specified by the parameter, the default configuration of the system is 3 seconds, and the example uses the default configuration, so the starting point of this window is PD current STO 700, and the end point of the time window is at STO 703. After writing etcd successfully, PD will get an “allocatable time window” from 700 to 703. In this time window, PD can use the physical time of the system as a high level, splice the logical time accumulated in memory, and assign a timestamp to the outside. The above design means that all PDs are assigned a timestamp high, that is, physical time, that is, always less than the maximum value stored in etcd. Then, if the PD leader goes down, the new master can read the maximum value stored in etcd and apply for a new “allocatable time window” based on this, so that the timestamp assigned by the new master will definitely be larger than the old master. In addition, in order to reduce communication overhead, each client can request multiple timestamps at a time, and the number of timestamps is passed by the client to PD as a parameter. It is important to note, however, that once cache on the client, the clock between multiple clients is no longer strictly monotonically increasing.,chapter-04/11.md:Different schedulers start from their own logic and requirements and consider various restrictions and constraints to generate Operators to be executed. Let's first explain what an Operator is. An Operator is a collection of operations applied to a Region that serves a certain scheduling purpose. For example, “Migrate the Leader of Region 2 to Store 5”, “Migrate a replica of Region 2 to Store 1, 4, 5,” and so on.,chapter-04/14.md:Among them, “dc”, “rack” and “host” respectively indicate a data center, cabinet or rack and host, which are all physical locations or devices, which are very understandable.,chapter-04/14.md:However, the concept of “zone” is actually a logical concept used to denote the smallest location where replica can be isolated, and the number of them is always the same as that of replica. For example, if we have a default of 3 replicas, and if we have 3 data centers (DCs), then we set up 3 zones, each zone corresponding to a data center (DC), and the replica are placed in different zones. If we have 1 datacenter, but we have 3 racks, then we also set up 3 zones, one rack for each zone, and the replica are placed in different zones.,chapter-05/08.md:Next, the thread pool will read data from RocksDB KV. You can refer to Lesson 03: “RocksDB - Query”, “Read Data - ReadIndex Read”, and “Read Data - Lease Read for RocksDB for TiKV Explained in “/Local Read” and “Data Reading - Follower Read”.,chapter-05/10.md:1. The Transaction module begins to enter the second stage commit. The second stage commit will be introduced in Lesson 03: TiKV's “Distributed Transaction”. Here we briefly understand it as a two-step operation, the first is the prewrite operation, that is, modify the data and lock, and the second is the commit operation, that is, commit the modify , Release the lock. Well, in these 2 operations, modify data, adding locks, releasing locks, and committing modify are all write operations, so let's introduce write operations later.,chapter-05/11.md:For details on writing RocksDB, refer to “RocksDB - Write” in Lesson 03: TiKV. |
| ” | " | chapter-02/15.md:The thread pool module related to SQL read/write is introduced below. As for the SQL reads and writes process, we will introduce it in detail in “Lesson 05 TiDB Database SQL Execution Flow”. Here we explain the role of the relevant module.,chapter-02/18.md:2. If the configuration item of oom-action is “cancel”, then when the memory usage of a SQL exceeds a certain threshold, TiDB will immediately interrupt the execution of the SQL and return an error to the client. The error message will specify in detail each physical execution that takes up more memory during the execution of the SQL The memory usage of the operator.,chapter-03/08.md:Support for the Column Families feature started with RocksDB 3.0. Each key value pair (key - value) of RocksDB corresponds to a unique column family (Column Family). If a Column Family is not specified, the key-value pair will correspond to the “default” Column Family (Column Family), which is also the default Column Family of the RocksDB instance.,chapter-03/33.md:First of all, we introduced a concept in “Raft - Leader elect”, which is the heartbeat time interval (the leader sends a heartbeat interval), which requires the heartbeat time interval to be less than min (election) timeout). That is, the leader will send a heartbeat every heartbeat time interval. We record this time point as start. If the majority of followers give a heartbeat response (at this time, you can determine if the leader is this node), Due to Raft's elect mechanism, followers will not initiate re-election after at least the elect timeout period, so even if an elect occurs, the new leader will be elect during the election timeout/clock drift bound. Well, from start to election timeout/clock drift bound, the leader is still the node during this time. We call this period a lease, or a lease period. Well, within a lease, the node does not need to issue a heartbeat to determine that it is still the leader node. In this way, we no longer need to send a heartbeat to determine if we are currently a leader like ReadIndex Read, which improves efficiency.,chapter-04/03.md:PD is the central controller of the TiDB database and is called the “brain” of the entire cluster. In the TiDB database cluster, it is responsible for storing metadata (mainly providing routes for TiDB Server, etc.), assigning global ID, transaction ID, global timestamp TSO generation, service discovery, collection of information such as the overall topology of the cluster, real-time data distribution and hot spot status, and scheduling of data in TiKV nodes. At the same time, TiDB Dashboard services are provided.,chapter-04/08.md:The current PD (leader) has a TSO of 700 (virtual value for ease of explanation), and PD has applied etcd for an “allocatable time window”. To know that the span of the time window can be specified by the parameter, the default configuration of the system is 3 seconds, and the example uses the default configuration, so the starting point of this window is PD current STO 700, and the end point of the time window is at STO 703. After writing etcd successfully, PD will get an “allocatable time window” from 700 to 703. In this time window, PD can use the physical time of the system as a high level, splice the logical time accumulated in memory, and assign a timestamp to the outside. The above design means that all PDs are assigned a timestamp high, that is, physical time, that is, always less than the maximum value stored in etcd. Then, if the PD leader goes down, the new master can read the maximum value stored in etcd and apply for a new “allocatable time window” based on this, so that the timestamp assigned by the new master will definitely be larger than the old master. In addition, in order to reduce communication overhead, each client can request multiple timestamps at a time, and the number of timestamps is passed by the client to PD as a parameter. It is important to note, however, that once cache on the client, the clock between multiple clients is no longer strictly monotonically increasing.,chapter-04/11.md:Different schedulers start from their own logic and requirements and consider various restrictions and constraints to generate Operators to be executed. Let's first explain what an Operator is. An Operator is a collection of operations applied to a Region that serves a certain scheduling purpose. For example, “Migrate the Leader of Region 2 to Store 5”, “Migrate a replica of Region 2 to Store 1, 4, 5,” and so on.,chapter-04/14.md:Among them, “dc”, “rack” and “host” respectively indicate a data center, cabinet or rack and host, which are all physical locations or devices, which are very understandable.,chapter-04/14.md:However, the concept of “zone” is actually a logical concept used to denote the smallest location where replica can be isolated, and the number of them is always the same as that of replica. For example, if we have a default of 3 replicas, and if we have 3 data centers (DCs), then we set up 3 zones, each zone corresponding to a data center (DC), and the replica are placed in different zones. If we have 1 datacenter, but we have 3 racks, then we also set up 3 zones, one rack for each zone, and the replica are placed in different zones.,chapter-05/08.md:Next, the thread pool will read data from RocksDB KV. You can refer to Lesson 03: “RocksDB - Query”, “Read Data - ReadIndex Read”, and “Read Data - Lease Read for RocksDB for TiKV Explained in “/Local Read” and “Data Reading - Follower Read”.,chapter-05/10.md:1. The Transaction module begins to enter the second stage commit. The second stage commit will be introduced in Lesson 03: TiKV's “Distributed Transaction”. Here we briefly understand it as a two-step operation, the first is the prewrite operation, that is, modify the data and lock, and the second is the commit operation, that is, commit the modify , Release the lock. Well, in these 2 operations, modify data, adding locks, releasing locks, and committing modify are all write operations, so let's introduce write operations later.,chapter-05/11.md:For details on writing RocksDB, refer to “RocksDB - Write” in Lesson 03: TiKV. |