1. Metrics list
cpuResource:
  - name: "TiDB_CPU_P90"
    primaryQuery: 'quantile(0.9,sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component="tidb"}[$rate])) by (instance)) by (cluster_id)'
  - name: "TiDB_CPU_P80"
    primaryQuery: 'quantile(0.8,sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component="tidb"}[$rate])) by (instance)) by (cluster_id)'
  - name: "TiDB_CPU"
    primaryQuery: 'sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component="tidb"}[$rate])) by (cluster_id)'
  - name: "TiKV_CPU_P90"
    primaryQuery: 'quantile(0.9,  sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}[$rate])) by (instance)) by(cluster_id)'
  - name: "TiKV_CPU_P80"
    primaryQuery: 'quantile(0.8,  sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}[$rate])) by (instance)) by(cluster_id)'
  - name: "TiKV_CPU"
    primaryQuery: 'sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}[$rate])) by (cluster_id)'

memResource:
  - name: "TiDB_MEM_P90"
    primaryQuery: 'quantile(0.9,sum(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component="tidb"})by (instance,cluster))by (cluster)'
  - name: "TiDB_MEM_P80"
    primaryQuery: 'quantile(0.8,sum(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component="tidb"})by (instance,cluster))by (cluster)'
  - name: "TiDB_MEM"
    primaryQuery: 'sum(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component="tidb"})by (cluster)'
  - name: "TiKV_MEM_P90"
    primaryQuery: 'quantile(0.90,avg(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}) by (instance))by (cluster_id)'
  - name: "TiKV_MEM_P80"
    primaryQuery: 'quantile(0.80,avg(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}) by (instance))by (cluster_id)'
  - name: "TiKV_MEM"
    primaryQuery: 'sum(avg(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component=~".*tikv"})by(instance))by(cluster_id)'

latencyQueries:
  - name: "P99QueryLatency"
    primaryQuery: 'histogram_quantile(0.99, sum(rate(tidb_server_handle_query_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster"}[$rate])) by (le))'
  - name: "P999QueryLatency"
    primaryQuery: 'histogram_quantile(0.999, sum(rate(tidb_server_handle_query_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster"}[$rate])) by (le))'
  - name: "P99TiKVLatency"
    primaryQuery: 'histogram_quantile(0.99, sum(rate(tikv_grpc_msg_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster", type=~"coprocessor|kv_prewrite"}[$rate])) by (le,type))'
  - name: "CPS"
    primaryQuery: 'sum(rate(tidb_server_query_total{tidb_cluster_id="$tidb_cluster",result="OK"}[$rate]))'
  - name: "QPS"
    primaryQuery: 'sum(rate(tidb_executor_statement_total{tidb_cluster_id="$tidb_cluster"}[$rate]))'

diskQueries:
  - name: "ReadWaitLatency"
    primaryQuery: 'sum ( rate(node_disk_read_time_seconds_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) / rate(node_disk_reads_completed_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) )'
  - name: "WriteWaitLatency"
    primaryQuery: 'sum( rate(node_disk_write_time_seconds_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) / rate(node_disk_writes_completed_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) )'
  - name: "DiskUsage"
    primaryQuery: '(max((sum by (instance) (tikv_store_size_bytes{tidb_cluster_id="$tidb_cluster",type="used"}) / sum by (instance) (tikv_store_size_bytes{tidb_cluster_id="$tidb_cluster",type="capacity"}) * 100)))'
2. Metrics PromSQL Mapping  Grafana Panel 
2. Metrics PromSQL Mapping  Grafana Panel 
指标类型
CPU

MEM

QueryLatency

CPS
QPS
diskQueries



指标
process_cpu_seconds_total

process_resident_memory_bytes

tidb_server_handle_query_duration_seconds_bucket
tikv_grpc_msg_duration_seconds_bucket
tidb_server_query_total
tidb_executor_statement_total
node_disk_read_time_seconds_total

node_disk_reads_completed_total
node_disk_write_time_seconds_total
node_disk_writes_completed_total

tikv_store_size_bytes

name
（配置文件中name）
TiDB_CPU
TiDB_CPU_P90
TiDB_CPU_P80
TiKV_CPU
TiKV_CPU_P90
TiKV_CPU_P80
TiDB_MEM
TiDB_MEM_P90
TiDB_MEM_P80
TiKV_MEM
TiKV_MEM_P90
TiKV_MEM_P80
QueryLatency

P99TiKVLatency
CPS
QPS
ReadWaitLatency
WriteWaitLatency
DiskUsage


PromSQL

TiDB_CPU
sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component="tidb"}[$rate])) by (cluster_id)'
TiDB_CPU_Pxx
quantile(0.8,sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component="tidb"}[$rate])) by (instance)) by (cluster_id)
TiKV_CPU
sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component=".*tikv"}[$rate])) by (cluster_id)'
TiKV_CPU_Pxx
quantile(0.8,sum(rate(process_cpu_seconds_total{tidb_cluster_id="$tidb_cluster",component=".*tikv"}[$rate])) by (instance)) by (cluster_id)
TIDB_MEM
sum(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component="tidb"})by (cluster)
TIDB_MEM_Pxxx
quantile(0.9,sum(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component="tidb"})by (instance,cluster))by (cluster)

TiKV_MEM
sum(avg(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component=~".*tikv"})by(instance))by(cluster_id)
TiKV_MEM_Pxxx
quantile(0.90,avg(process_resident_memory_bytes{tidb_cluster_id="$tidb_cluster",component=~".*tikv"}) by (instance))by (cluster_id)

P99QueryLatency
histogram_quantile(0.99, sum(rate(tidb_server_handle_query_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster"}[$rate])) by (le))
P999QueryLatency
histogram_quantile(0.999, sum(rate(tidb_server_handle_query_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster"}[$rate])) by (le))

P99TiKVLatency
histogram_quantile(0.99, sum(rate(tikv_grpc_msg_duration_seconds_bucket{tidb_cluster_id="$tidb_cluster", type=~"coprocessor|kv_prewrite"}[$rate])) by (le,type))

CPS
sum(rate(tidb_server_query_total{tidb_cluster_id="$tidb_cluster",result="OK"}[$rate]))
QPS
sum(rate(tidb_executor_statement_total{tidb_cluster_id="$tidb_cluster"}[$rate]))'


ReadWaitLatency
sum ( rate(node_disk_read_time_seconds_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) / rate(node_disk_reads_completed_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) )
WriteWaitLatency
sum( rate(node_disk_write_time_seconds_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) / rate(node_disk_writes_completed_total{tidb_cluster_id="$tidb_cluster",device!~"dm-.*|nvme1.*|nvme0.*"}[$rate]) )
DiskUsage
(max((sum by (instance) (tikv_store_size_bytes{tidb_cluster_id="$tidb_cluster",type="used"}) / sum by (instance) (tikv_store_size_bytes{tidb_cluster_id="$tidb_cluster",type="capacity"}) * 100)))


Grafana Panel Link 

https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=168



https://clinic.pingcap.com/grafana/d/e53523ea9ee7fcf4tidbcloudv8_5_x/tikv-details-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=8

https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=3

https://clinic.pingcap.com/grafana/d/e53523ea9ee7fcf4tidbcloudv8_5_x/tikv-details-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=9
P999QueryLatency ：
https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=now-1h&to=now&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=81

P99QueryLatency ：
https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=now-1h&to=now&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=25

https://clinic.pingcap.com/grafana/d/459a3ee585229796tidbcloudv8_5_x/tikv-summary-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=98
https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=now-1h&to=now&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=2

https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1773244800000&to=1773331199000&var-tidb_version=v8.5.x&var-org=1372813089196930499&var-org_type=cloud&viewPanel=21
https://clinic.pingcap.com/grafana/d/clinic-dataplane-node/dedicated-dataplane-node-info?orgId=1&var-k8s_cluster=All&var-tidb_cluster=10989049060142230334&var-namespace=tidb10989049060142230334&from=1774397085000&to=1774404285000&viewPanel=22


https://clinic.pingcap.com/grafana/d/e53523ea9ee7fcf4tidbcloudv8_5_x/tikv-details-cluster-clinic?orgId=1&var-tidb_cluster=10989049060142230334&var-instance=All&var-store=All&from=1774397085000&to=1774404285000&var-tidb_version=v8.5.x&var-org=1372813089196930499&refresh=15m&var-org_type=cloud&viewPanel=5

Grafana Panel Link 
 变量
https://clinic.pingcap.com/grafana/d/8c16287e9fd72206tidbcloudv8_5_x/tidb-cluster-clinic?orgId={1}&var-tidb_cluster={10989049060142230334}&var-instance=All&var-store=All&from={1774397085000}&to={1774404285000}&var-tidb_version={v8.5.x}&var-org={1372813089196930499}&refresh=15m&var-org_type=cloud&viewPanel=168

变量与TiDB_CPU类似











Grafana Panel 截图
[图片]

[图片]
[图片]

[图片]
[图片]

[图片]
[图片]


[图片]

[图片]
[图片]

[图片]

[图片]

[图片]


[图片]

[图片]