metrics_diag_no_workload: 
  fact: There is no advice for clusters without workloads.
  suggestion: null
  link: null
metrics_diag_no_prod: 
  fact: There is no advice for non-production clusters because the number of TiKV instances is less than 3.
  suggestion: null
  link: null
metrics_diag_143: 
  fact: The operation counts of stmtPrepare, stmtExecute, and stmtClose are almost equal.
  suggestion: The client type should be checked. If JDBC used, it is recommended to add a configuration as 'useServerPrepStmts=true&cachePrepStmts=true&prepStmtCacheSize=1000&prepStmtCacheSqlLimit=20480&useConfigs=maxPerformance'. It is better to communicate with end-users without JDBC clients about how to implement the application, cache prepared statement objects on the client side, not reset calls to StmtPrepare and StmtClose, and reduce the number of command calls required to execute each query from 3 to 1.
  link: null
metrics_diag_144: 
  fact: The average connection idle duration is {{var_1}} ms and the average query handle duration in the TiDB cluster is {{var_2}} ms. The latency outside the TiDB cluster is much higher. 
  suggestion:  It is recommended to check if there is an issue on the client side. 
  link: null
metrics_diag_145:
  fact: Queries with plan cache ops account for less than 20% of queries with stmtExecute ops.
  suggestion: Hit rate of plan cache is low. The application on the client side might be optimized.
  link: null
metrics_diag_146: 
  fact: QPS drops along with Active Connection.
  suggestion: The decline in QPS is strongly related to active connection counts. If the decline in QPS is unexpected, it is recommended to confirm the reason for the decline in active connection counts in the application firstly.
  link: null
metrics_diag_147: 
  fact: QPS is imbalance among the TiDB nodes.
  suggestion: It is recommended to check the load balancer on the client side .
  link: null
metrics_diag_148: 
  fact:  the active connection counts are imbalance among the TiDB nodes.
  suggestion: It is recommended to check the load balancer on the client side.
  link: null
metrics_diag_149:
  fact: The minimum TiKV CPU usage is up to {{var_1}}% (suggestion is less than 60%).
  suggestion: It is recommended to scale in/out TiKV.
  link: null
metrics_diag_150:
  fact: The minimum TiKV disk usage is up to {{var_1}}% (suggestion is less than 70%).
  suggestion: It is recommended to scale in/out TiKV.
  link: null
metrics_diag_151:
  fact: The minimum TiDB CPU usage is up to {{var_1}}% (suggestion is less than 60%).
  suggestion: It is recommended to scale in/out TiDB.
  link: null
metrics_diag_152:
  fact: There is no queries ues plan cache ops.
  suggestion: It is recommended to configure the prepared-plan-cache parameter to enabled under TiDB Server to turn on the Plan cache feature.
  link: https://docs.pingcap.com/tidb/stable/sql-prepared-plan-cache
metrics_diag_156:
  fact: The latency between TiDB kv request duration and TiKV grpc duration is up to {{var_1}} ms (suggestion is less than 30ms).
  suggestion: It is recommended to check the network environment.
  link: null
metrics_diag_157:
  fact: The stmtPrepare accounts for less than 20% of the total CPS.
  suggestion: "The application seems to use the Statement Prepare interface insufficiently. It is recommended to use the Prepared Statement interface in the application (for JDBC parameter useServerPrepStmts is modified to true.) In addition, make sure to set the TiDB global variable set global tidb_ignore_prepared_cache_close_stmt=on; (introduced since TiDB v6.0.0, off by default) and set the TiDB configuration item prepared-plan-cache: {enabled: true} to enable the plan cache feature."
  link: https://docs.pingcap.com/tidb/stable/performance-tuning-practices#scenario-4-use-the-prepared-statement-interface-and-enable-execution-plan-caching
metrics_diag_159:
  fact: While the active connection counts increase, the QPS doesn't increase, and the latency increases.
  suggestion: Please check if there is a performance bottleneck inside the TiDB cluster.
  link: null
metrics_diag_158:
  fact: CPU usages are imbalance among the TiKV nodes.
  suggestion: It is recommended to check if there is a hotspot.
  link: null
metrics_diag_160:
  fact: The count of failed QPS is higher than 10 queries/second.
  suggestion:  It is recommended to check the root cause of increasing failed queries.
  link: null
metrics_diag_161:
  fact: Latency is imbalance among the TiDB nodes.
  suggestion: It is recommended to check the load balancer on the client side or query types on TiDB side 
  link: null
metrics_diag_162:
  fact: Memory usages are imbalance among the TiKV nodes.
  suggestion: It is recommended to check if there is a hotspot.
  link: null
metrics_diag_163:
  fact: CPU usages are imbalance among the TiDB nodes.
  suggestion: It may be caused by BR or some other tasks. It is recommended to check the load balancer on the client side or slow queries.
  link: null
metrics_diag_164:
  fact: Memory usages are imbalance among the TiDB nodes.
  suggestion: It may be caused by BR or some other tasks. It is recommended to check the load balancer on the client side or slow queries.
  link: null
metrics_diag_165:
  fact: The "TiKV Auto GC SafePoint" metric in the "TiKV Details" dashboard does not increase as expected.
  suggestion: There might be an anomaly with MVCC 
  link: null
metrics_diag_166:
  fact: The average P99 TiKV approximate region size is up to {{var_1}} MB (alert if larger than 1G). 
  suggestion:  Check the configuration or bugs about split. Please refer to the documentation link for recommended configuration parameters.
  link: https://docs.pingcap.com/tidb/dev/tune-region-performance#use-region-split-size-to-adjust-region-size
metrics_diag_153:
  fact: There is a scatter scheduler.
  suggestion: The scatter scheduler should not be enabled in a production environment. Please contact PingCAPers to disable it.
  link: null
metrics_diag_154:
  fact: There is no operation on the raft engine.
  suggestion: It is recommended to contact PingCAPers to enable Raft-engine, if v5.4.0 <= the TiDB cluster version < v6.1.0. On-promise users could enable it by themselves.
  link: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#raft-engine
metrics_diag_155:
  fact: The "store writer CPU" usage is nearly 0% and the TiKV CPU usage is lower than 75%.
  suggestion: You can consider to change configure value of raftstore.store-io-pool-size  to improve the performance.
  link: https://docs.pingcap.com/tidb/stable/tune-tikv-thread-performance#performance-tuning-for-tikv-thread-pools
metrics_diag_167:
  fact: Three schedulers, balance-hot-region-scheduler, balance-leader-scheduler, and balance-region-scheduler, don't always exist in this time range.
  suggestion: Please check if PD schedulers runs as expected.
  link: null
metrics_diag_168:
  fact: There are too many heartbeat messages.
  suggestion: Please check if the TiKV configuration raftstore.hibernate-regions is TRUE.
  link: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#hibernate-regions
tidb_get_token_duration_38:
  fact: TiDB gets tokens in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 1ms). 
  suggestion: get token phase usually only a few microseconds . The token is limited only when the number of connections to a single TiDB instance reaches the token-limit limit. If the get token duration is too long and impact the performance, you can consider to adapt token-limit configuration.
  link: https://docs.pingcap.com/tidb/stable/command-line-flags-for-tidb-configuration#--token-limit
tidb_diag_parser_duration_39:
  fact: SQL parsing duration in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 3ms).
  suggestion: "It is recommended that the application side refer to the documentation and use prepared statements to reduce the latency of SQL parsing."
  link: https://docs.pingcap.com/tidb/dev/dev-guide-prepared-statement

tidb_diag_compile_duration_40:
  fact: SQL compile duration in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 3ms).
  suggestion: "Please set the TiDB configuration item prepared-plan-cache: {enabled: true} to enable the plan cache feature if possible."
  link: https://docs.pingcap.com/tidb/stable/sql-prepared-plan-cache#sql-prepare-execution-plan-cache
tidb_wait_tso_duration_43:
  fact: TiDB wait for tso in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 8ms).
  suggestion: Please check PD CPU and network resource.
  link: null
tidb_wait_tso_duration_44:
  fact: PD receiving tso request in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 7ms).
  suggestion: Please check PD CPU and network resource.
  link: null
tikv_pessmistic_txn_conflict_51:
  fact: The mean time of lock wait in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 50ms).
  suggestion: Please adjust pessimistic-txn.wait-for-lock-timeout, pessimistic-txn.wake-up-delay-duration, or make transaction commit faster, or modify the service to reduce lock conflicts.
  link: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#pessimistic-txn
tikv_pessmistic_txn_conflict_5111:
  fact: The number of lock wait in all tikv instanaces is over {{var_1}}.
  suggestion: Please adjust pessimistic-txn.wait-for-lock-timeout, pessimistic-txn.wake-up-delay-duration, or make transaction commit faster, or modify the service to reduce lock conflicts.
  link: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#pessimistic-txn
tikv_raft_propose_wait_duration_78:
  fact: The raft propose wait duarion in tikv {{var_1}} takes more than {{var_2}} ms (suggestion is less than 15ms).
  suggestion: You can consider to change configure value of raftstore.store-io-pool-size  to improve the performance.
  link: https://docs.pingcap.com/tidb/stable/tune-tikv-thread-performance#performance-tuning-for-tikv-thread-pools
tikv_diag_mvcc_versions_too_many_85:
  fact: Old MVCC versions are too many.(Among all tags of sum(rate(tikv_coprocessor_scan_details{req!~"analyze.*|checksum.*"}[2m])), the tag of "next" is 5 times lager than "processed_keys")
  suggestion: Please check if there is transaction blocking gc process. Or reconfigurate the gc variables to make gc faster according to the documentation.
  link: null
tikv_diag_cop_task_duration_171:
  fact:  TiKV cop duration in {{var_1}} takes more than {{var_2}} ms (suggestion is less than 20ms).
  suggestion: Please contact PingCAPers to check old mvcc versions, tomstones and key range.
  link: null
metrics_diag_208:
  fact: The database time is unbalanced among TiDB instances.
  suggestion: Please check the load balance on the application side.
  link: null
metrics_diag_209:
  fact: The parse duration is unbalanced among TiDB instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_210:
  fact: The compile duration is unbalanced among TiDB instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_211:
  fact: The execute duration is unbalanced among TiDB instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_212:
  fact: The get token duration is unbalanced among TiDB instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_213:
  fact: The uptime of a TiDB instance falls down. 
  suggestion: Please check if the restart of some TiDB instances are expected.
  link: null
metrics_diag_215:
  fact: Some TiDB instances' CPU is over 80% for 5 minutes in this period. 
  suggestion: Please check the resource usage.
  link: null
metrics_diag_216:
  fact: The uptime of a TiKV instance falls down. 
  suggestion: Please check if the restart of some TiKV instances are expected.
  link: null
metrics_diag_217:
  fact: Some TiKV instances' CPU is over 80% for 5 minutes in this period.
  suggestion: Please check the resource usage.
  link: null
metrics_diag_218:
  fact: The uptime of a PD instance falls down. 
  suggestion: Please check if the restart of some PD instances are expected.
  link: null
metrics_diag_219:
  fact: The uptime of a TiFlash instance falls down. 
  suggestion: Please check if the restart of some TiFlash instances are expected.
  link: null
metrics_diag_220:
  fact: The disk usage is unbalanced among TiKV instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_221:
  fact: The disk usage of some TiKV instances might be over 80% in 8 hours or is already over 80%.
  suggestion: Please check the resource usage.
  link: null
metrics_diag_222:
  fact: The disk of some TiFlash instances might be over 80% in 8 hours or is already over 80%.
  suggestion: Please check the resource usage.
  link: null
metrics_diag_223:
  fact: The disk read IO is unbalanced among TiKV instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_224:
  fact: The disk write IO is unbalanced among TiKV instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_225:
  fact: The disk write IO is unbalanced among TiFlash instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_226:
  fact: The disk read IO is unbalanced among TiFlash instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_227:
  fact: The CPU usage is unbalanced among TiFlash instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_228:
  fact: The memory usage is unbalanced among TiFlash instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_229:
  fact: The disk usage is unbalanced among TiFlash instances.
  suggestion: Please check if it is reasonable.
  link: null
metrics_diag_230:
  fact: Some TiFlash instances' CPU is over 80% for 5 minutes in this period. 
  suggestion: Please check the resource usage.
  link: null
metrics_diag_231:
  fact: Get token in Database time takes too much time (suggestion is less than 2% database time). 
  suggestion: get token phase usually only a few microseconds . The token is limited only when the number of connections to a single TiDB instance reaches the token-limit limit. If the get token duration is too long and impact the performance, you can consider to adapt token-limit configuration.
  link: https://docs.pingcap.com/tidb/stable/performance-tuning-methods#parse-compile-and-execute-duration
metrics_diag_232:
  fact: Parse in Database time takes too much time (suggestion is less than 5% database time).
  suggestion: Parse in Database time takes too much time. Parse phase :The query statements are parsed into abstract syntax tree (AST). When the query statement is long, much CPU will be consumed to parse the SQL text. It is recommended to compare historical performance data, and if the Parse phase has increased sharply recently, you can check the CPU usage.
  link: https://docs.pingcap.com/tidb/stable/performance-tuning-methods#parse-compile-and-execute-duration
metrics_diag_233:
  fact: Compile in Database time takes too much time (suggestion is less than 15% database time).
  suggestion: 'Compile phase: execution plans are compiled based on the AST from the parse phase and statistics. You can follow the official tuning advice in the link.'
  link: https://docs.pingcap.com/tidb/stable/performance-tuning-methods#parse-compile-and-execute-duration
metrics_diag_234:
  fact: Idle connection duration not in txn is high (suggestion is less than half of average latency inside the database).
  suggestion: 'Connection Idle Duration not-in-txn: The interval between processing the previous SQL and receiving the next SQL statement when the connection is not within a transaction. If the workload is high, the performance bottleneck  of the system may not be in tidb. '
  link: https://docs.pingcap.com/tidb/stable/performance-tuning-methods#query-latency-breakdown-and-key-latency-metrics
internal_activity_is_high_259:
  fact: TiDB internal activity is over 10%.
  suggestion: Please contact PingCAPers to check if it is reasonable.
  link: null
short_connection_260:
  fact: Frequent database connections, disconnections per seconds is over 50.
  suggestion: It's recommended to set maxIdle euqual to maxActive for connection pools, so the connections between application and database are persistent.
  link: null