# get logs in kubernetes Logs from individual components (tikv, tidb, pd, etc.) when running in Kubernetes are **not written to a file** inside the pod. Instead, they're written to stderr, which is consumed by Kubernetes and made available to the end-user via `kubectl`. ## Step-by-step guide 1. **Identify the pod(s) for which you want logs** ```bash $ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-qv2mn 1/1 Running 7 9d kube-system coredns-66bff467f8-vvmrc 1/1 Running 8 9d kube-system etcd-kind-control-plane 1/1 Running 6 9d kube-system kindnet-x8cp5 1/1 Running 7 9d kube-system kube-apiserver-kind-control-plane 1/1 Running 6 9d kube-system kube-controller-manager-kind-control-plane 1/1 Running 21 9d kube-system kube-proxy-7gfpx 1/1 Running 6 9d kube-system kube-scheduler-kind-control-plane 1/1 Running 20 9d local-path-storage local-path-provisioner-bd4bb6b75-m2524 1/1 Running 19 9d pingcap tidb-controller-manager-54cfcbff7b-bmq8q 1/1 Running 29 9d pingcap tidb-scheduler-6799885d99-dtgx2 2/2 Running 27 9d poc basic-discovery-59c77754b4-dzjpm 1/1 Running 7 9d poc basic-monitor-789d8f494c-r2nw5 3/3 Running 18 8d poc basic-pd-0 1/1 Running 1 78m poc basic-pd-1 1/1 Running 0 59m poc basic-tidb-0 2/2 Running 12 9d poc basic-tikv-0 1/1 Running 6 9d ``` 2. **Use kubectl to view the logs for the pod** ```bash $ kubectl logs -n poc basic-pd-0 | tail -n 10 \[2020/08/21 17:25:02.704 +00:00\] \[INFO\] \[grpc\_service.go:735\] \["updated gc safe point"\] \[safe-point=418913977006030848\] \[2020/08/21 17:33:13.453 +00:00\] \[INFO\] \[periodic.go:135\] \["starting auto periodic compaction"\] \[revision=40342\] \[compact-period=1h0m0s\] \[2020/08/21 17:33:13.456 +00:00\] \[INFO\] \[index.go:189\] \["compact tree index"\] \[revision=40342\] \[2020/08/21 17:33:13.459 +00:00\] \[INFO\] \[periodic.go:146\] \["completed auto periodic compaction"\] \[revision=40342\] \[compact-period=1h0m0s\] \[took=1h0m0.012216804s\] \[2020/08/21 17:33:13.516 +00:00\] \[INFO\] \[kvstore\_compaction.go:55\] \["finished scheduled compaction"\] \[compact-revision=40342\] \[took=59.177853ms\] \[2020/08/21 17:35:02.715 +00:00\] \[INFO\] \[grpc\_service.go:735\] \["updated gc safe point"\] \[safe-point=418914134297149440\] \[2020/08/21 17:43:22.702 +00:00\] \[INFO\] \[grpc\_service.go:783\] \["update service GC safe point"\] \[service-id=gc\_worker\] \[expire-at=-9223372035256744007\] \[safepoint=418914291588268032\] \[2020/08/21 17:45:02.724 +00:00\] \[INFO\] \[grpc\_service.go:735\] \["updated gc safe point"\] \[safe-point=418914291588268032\] \[2020/08/21 17:55:02.739 +00:00\] \[INFO\] \[grpc\_service.go:735\] \["updated gc safe point"\] \[safe-point=418914448873619456\] \[2020/08/21 18:03:22.706 +00:00\] \[INFO\] \[grpc\_service.go:783\] \["update service GC safe point"\] \[service-id=gc\_worker\] \[expire-at=-9223372035256742807\] \[safepoint=418914606164213760\] ``` 3. **You can also get logs for some items that are not pods, but not all Kubernetes resources support logs** 1. TidbCluster (tc) does not have logs ```bash $ kubectl logs tc/basic -n poc error: no kind "TidbCluster" is registered for version "[pingcap.com/v1alpha1](http://pingcap.com/v1alpha1)" in scheme "[k8s.io/kubectl/pkg/scheme/scheme.go:28](http://k8s.io/kubectl/pkg/scheme/scheme.go:28)" ``` 2. Deployment (deploy) does support logs... ```bash $ kubectl logs deploy/basic-monitor -n poc error: a container name must be specified for pod basic-monitor-789d8f494c-r2nw5, choose one of: \[prometheus reloader grafana\] or one of the init containers: \[monitor-initializer\] ``` 4. but sometimes you have to **specify a container name** if there are multiple containers in a pod (because each of them has a separate stderr stream) ```bash $ kubectl logs deploy/basic-monitor -n poc -c grafana | tail t=2020-08-21T18:16:12+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=3 name="TiKV channel full alert" changing state to=ok t=2020-08-21T18:16:13+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=16 name="TiKV channel full alert" changing state to=ok t=2020-08-21T18:16:13+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=17 name="approximate region size alert" changing state to=no\_data t=2020-08-21T18:16:13+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=15 name="server report failures alert" changing state to=ok t=2020-08-21T18:16:13+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=12 name="etcd disk fsync" changing state to=ok t=2020-08-21T18:16:13+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=33 name="TiKV gRPC poll CPU alert" changing state to=ok t=2020-08-21T18:16:15+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=23 name="TiKV Storage ReadPool CPU alert" changing state to=ok t=2020-08-21T18:16:16+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=24 name="TiKV Coprocessor CPU alert" changing state to=ok t=2020-08-21T18:16:16+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=10 name="TiKV Storage ReadPool CPU alert" changing state to=ok t=2020-08-21T18:16:18+0000 lvl=info msg="Alert Rule returned no data" logger=alerting.evalContext ruleId=18 name="TiKV raft store CPU alert" changing state to=ok ```