# Clinic Repository Module and Deployment Overview

## Purpose

This document maps the Clinic repository's source modules, executable entrypoints, container images, and app-delivery ownership. Use it before changing code to identify the correct Go module, test root, build target, deployment component, and environment overlay.

The repository evolves frequently. Update this document in the same PR when a change adds, removes, renames, or moves a module, executable, image, workflow, or app-delivery mapping.

## Repository at a Glance

Clinic is a multi-component repository with three Go modules plus Python and operational tooling:

| Source root | Go module | Go version | Primary responsibility |
| --- | --- | --- | --- |
| `/` | `github.com/pingcap-inc/clinic` | `1.24.2` | Clinic APIs, internal workers, NGM API, Kong plugins, shared storage/O11y/data-access packages, and operational binaries |
| `/clinic_data_service` | `github.com/pingcap-inc/clinic/clinic_data_service` | `1.25.0` | Data Proxy query APIs, metrics and log proxying, cluster metadata, diagnosis APIs, and health-check integrations |
| `/cloud_server` | `github.com/pingcap-inc/clinic_cloud_server` | `1.24.2` | TiDB Cloud event, alert, BYOC alert, and user-debug services |

Always run Go commands from the module that owns the changed package. Commands launched from the repository root do not cross into the nested `clinic_data_service` or `cloud_server` modules.

## Root Go Module

The root module contains multiple independently built executables that share packages under `pkg/`.

| Component | Entrypoint | Main source ownership | Image or artifact | app-delivery mapping |
| --- | --- | --- | --- | --- |
| Clinic API | `cmd/server/main.go` | `pkg/api`, `pkg/dashboard`, `pkg/datasource`, `pkg/proxy`, `pkg/rbac`, `pkg/storage`, `pkg/o11y` | `clinic` | `clinic-api`; app name `clinic` |
| Internal API | `cmd/internal_server/main.go` | `pkg/internal_api` plus shared datasource, DB, storage, and RBAC packages | `internal_api` | `clinic-api`; app name `internal-api` |
| Task Manager | `cmd/task_manager/main.go` | `pkg/task_manager`, health-check and performance-insight schedulers | `task_manager` | `clinic-api`; app name `task-manager` |
| Importer | `cmd/importer/main.go` | `pkg/importer`, event listeners, storage and metric import paths | `importer` | Version is stored in the `clinic-api` environment ConfigMap; app name `importer` |
| Uploader | `cmd/uploader/main.go` | `pkg/upload_api`, `pkg/upload_file`, upload event listeners | `uploader` | `clinic-api`; app name `uploader` |
| Cluster Recovery | `cmd/cluster_recovery/main.go` | `pkg/cluster_recovery` | `cluster_recovery` binary packaged in the `clinic` image | No independent app-delivery app |
| NGM API | `cmd/ngm_api/main.go` | `pkg/ngm` | `ngm_api` | `ngm`; app name `ngm-api` |
| Clinic Kong plugin | `cmd/api_gateway/clinic/main.go` | `pkg/api_gateway/auth`, `pkg/api_gateway/rbac` | `clinic_auth` binary inside the `api-gateway` Kong image | `clinic-kong` and `clinic-kong-cluster-plugins`; app name `api-gateway` |
| TiDB Cloud Kong plugin | `cmd/api_gateway/tidbcloud` | `pkg/api_gateway/auth/tidbcloud` and shared plugin validation | `dbaas_auth` binary inside the `api-gateway` Kong image | `clinic-kong` and `clinic-kong-cluster-plugins`; app name `api-gateway` |
| Cleanup User | `tools/cleanup_user/main.go` | User cleanup utility using root packages | `cleanup_user` | Built by a dedicated workflow; verify deployment ownership before release |

Other command directories, such as `cmd/association-server`, are not part of the main `Makefile` build or the Clinic version-bump mapping. Treat them as auxiliary until their current owner and release path are verified.

### Clinic API

`cmd/server/main.go` is the main Clinic control-plane service. It initializes the database, datasource clients, authentication and RBAC, storage, event handlers, scheduled work, gRPC RBAC service, and Gin routes. Its public API base is `/clinic/api/v1`; it also exposes proxy, system, health-check, metrics, and operational routes.

Important package groups include:

- `pkg/api`: Clinic HTTP APIs for clusters, organizations, reports, activities, uploads, diagnosis, users, health checks, and datasource forwarding.
- `pkg/dashboard`: dashboard and cluster-search behavior.
- `pkg/datasource` and `pkg/o11y`: datasource resolution and O11y routing.
- `pkg/db`: shared persistence and transaction helpers.
- `pkg/proxy`: Grafana, Loki, metrics, and related forwarding support.
- `pkg/rbac`, `pkg/auth`, and `pkg/middlewares`: access control and request middleware.
- `pkg/storage`, `pkg/upload_file`, and `pkg/stream`: diagnostic file storage and streaming.

The app-delivery `clinic-api` application is a deployment bundle rather than a one-to-one representation of `cmd/server`. It contains the Clinic Deployment, Internal API and Task Manager workloads, the Uploader StatefulSet, Grafana resources, Services, Ingresses, configuration, RBAC, and environment patches.

### NGM API

`cmd/ngm_api/main.go` initializes the NGM service and registers routes from `pkg/ngm`. Major capabilities include performance insight, slow query, statements, Top SQL/Top RU, continuous profiling, key visualization, plan replay, storage, and file download. Internal and external route groups are registered under `/ngm/api/v1` and `/api/v1/orgs/.../ngm` families.

The image is deployed by app-delivery as the `ngm` application, whose main workload is the `ng-monitoring` StatefulSet.

### Kong API Gateway Plugins

The `build-api-gateway` target must build the complete TiDB Cloud plugin package, not only a single `main.go`, because validation and helper definitions may live in sibling files. The resulting `clinic_auth` and `dbaas_auth` binaries are copied into the custom Kong `api-gateway` image.

Deployment ownership is split:

- `clinic-kong` owns the Kong installation, image tag, Helm values, secrets, and environment-specific gateway configuration.
- `clinic-kong-cluster-plugins` owns `KongPlugin` resources and the Clinic/TiDB Cloud plugin configuration consumed by the ingress controller.

## Clinic Data Service Module

`clinic_data_service` is an independent Go module and must be built and tested from `/clinic_data_service`.

Pull requests that change this module are validated by `.github/workflows/clinic_data_service_ci.yml`. The workflow uses the nested `go.mod`, runs every test-bearing package under `pkg`, checks module-file consistency, and builds the API entrypoint. It intentionally excludes the environment-dependent integration tests under `clinic_data_service/test`.

Its entrypoint is `clinic_data_service/cmd/api/main.go`; HTTP setup and route registration live under `clinic_data_service/pkg/server` and `clinic_data_service/pkg/handlers`.

Major responsibilities include:

- SQL Data Proxy single and batch queries across supported cloud/storage engines.
- Canonical `/data-proxy/api/v1` resource APIs and legacy compatibility routes.
- Metrics instant, range, and series proxying.
- Loki cluster logs, system logs, and serverless-parent logs.
- Cluster metadata, components, and activities backed by the Clinic internal API.
- Top RU, slow-query, Top SQL, statements, and continuous-profiling diagnosis APIs.
- Internal traffic-mirroring and health-check analysis routes.
- Object storage, query engine, datasource routing, tracing, and Prometheus instrumentation.

Key package groups include:

- `pkg/server`: Gin setup, middleware, dependency initialization, and route composition.
- `pkg/handlers`: HTTP contracts and legacy/canonical route registration.
- `pkg/service`: query orchestration and upstream clients.
- `pkg/engine`, `pkg/engine_router`, `pkg/query`, `pkg/duckdb`, and `pkg/dao`: query execution and data access.
- `pkg/clustercontext` and `pkg/clinic-internal-api`: cluster ownership and Clinic control-plane lookup.
- `pkg/thirdapi`: cloud, O11y, AI, slow-query, SQL statement, and external integrations.
- `pkg/config`: runtime and per-environment query/metrics configuration.

The container image is `clinic/data-service`. app-delivery owns it under `clinic-data-service`, including the Deployment, Service, Ingress, `KongIngress` timeout override, ExternalSecret, ServiceAccount, and environment ConfigMaps.

## Cloud Server Module

`cloud_server` is an independent Go module and must be built and tested from `/cloud_server`. Its entrypoint is `cloud_server/cmd/cloud_server/main.go`.

It provides:

- TiDB Cloud event and event-filter APIs.
- Alert and next-generation alert APIs.
- BYOC alert distribution and token management.
- User-debug APIs.
- EventBus and Activity Hub integration.
- Account, Billing, Core Service, Management Service, and O11y clients.

The container image is `cloud_server`. app-delivery owns its Deployment, Service, Ingress, ConfigMap, ExternalSecret, resources, and replicas under `cloud-server`.

## Health-Check and Operational Tools

The `health_check` tree contains independently operated tools rather than packages in a single Go module:

- `health_check/sql_health_check`: Python HTTP service for SQL plan and metrics-based health checks. It has a dedicated Dockerfile, CD workflow, and app-delivery `sql-health-check` application in `prod-cloud`.
- `health_check/slow_query_proxy`: Python slow-query proxy and storage experiments with its own Python project metadata and tests.
- `health_check/convert-image`: metrics comparison and image-generation utilities.
- `health_check/tablefullscan`: table-full-scan tooling.

Do not assume every tool under `health_check` is deployed through the standard Clinic app bundle. Check its workflow and app-delivery directory before changing or releasing it. At the time of this inventory, `sql-health-check` has a prod-cloud app-delivery overlay but no matching staging application directory.

## Other Repository Areas

| Path | Responsibility |
| --- | --- |
| `deploy/` | Dockerfiles for root-module executables and Clinic Data Service |
| `.github/workflows/` | CI, image build, CD, and app-delivery dispatch workflows |
| `docs/` | Swagger output, caller guides, migration notes, operational documents, and implementation plans |
| `docs/plans/` | Dated change plans using `YYYY-MM-DD-short-topic.md` |
| `rbac_model/` | Casbin/RBAC model and policy-related assets |
| `report-template/` | Diagnostic report templates |
| `tools/` | Operational utilities not exposed as primary Clinic APIs |
| `aidlc-docs/` | AI-assisted development lifecycle artifacts |

## app-delivery Relationship

Clinic owns source code, tests, Docker build definitions, and image-producing workflows. app-delivery owns the Kubernetes and Helm desired state used by GitOps.

The normal delivery chain is:

```text
Clinic branch or master commit
  -> Clinic GitHub Actions CD workflow
  -> build and push an environment-specific image tag
  -> dispatch app-delivery bump-clinic-apps-version.yaml
  -> update the target environment image tag or configuration reference
  -> create an app-delivery PR
  -> normal checks, review, and merge
  -> GitOps reconciliation and runtime verification
```

### Environment Layout

| Environment | app-delivery root | Overlay convention |
| --- | --- | --- |
| Staging | `staging/clinic` | `<app>/envs/staging` |
| Production cloud | `clinic` | `<app>/envs/prod-cloud` |
| Production OP | `clinic` | `<app>/envs/prod-op` where present; out of scope for the release workflow in root `AGENTS.md` |

The version-bump workflow also supports dev and maps it to `dev/clinic/<app>/envs/dev-cloud`.

### Source-to-Deployment Mapping

| Source/CD component input | Image name | app-delivery directory | Version update location |
| --- | --- | --- | --- |
| `clinic` | `clinic` | `clinic-api` | Environment `kustomization.yaml` image `newTag` |
| `internal-api` | `internal_api` | `clinic-api` | Environment `kustomization.yaml` image `newTag` |
| `task-manager` | `task_manager` | `clinic-api` | Environment `kustomization.yaml` image `newTag` |
| `uploader` | `uploader` | `clinic-api` | Environment `kustomization.yaml` image `newTag` |
| `importer` | `importer` | `clinic-api` | Environment `configmap/config.yaml` `k8s-config.image-name` |
| `clinic-data-service` | `clinic/data-service` | `clinic-data-service` | Environment `kustomization.yaml` image `newTag` |
| `ngm-api` | `ngm_api` | `ngm` | Environment `kustomization.yaml` image `newTag` |
| `cloud-server` | `cloud_server` | `cloud-server` | Environment `kustomization.yaml` image `newTag` |
| `api-gateway` | `api-gateway` | `clinic-kong` | Environment Helm values `image.tag`; plugin CRs remain under `clinic-kong-cluster-plugins` |
| `sql-health-check` | `sql-health-check` | `sql-health-check` | Environment `kustomization.yaml` image `newTag` |

### Workflow Ownership

- `.github/workflows/clinic_data_service_ci.yml` validates Clinic Data Service pull requests from the nested Go module.
- `.github/workflows/clinic_cd.yml` builds selected root components. Supported component inputs include `clinic`, `importer`, `internal-api`, `task-manager`, `uploader`, `clinic-data-service`, and `all`.
- `.github/workflows/clinic_data_service_cd.yml` is the dedicated Clinic Data Service workflow.
- `.github/workflows/build_ngm_api.yml` builds and dispatches `ngm-api`.
- `.github/workflows/cloud_server_cd.yml` builds and dispatches `cloud-server` from the nested module.
- `.github/workflows/build_api_gateway.yml` builds and dispatches the custom Kong image.
- `.github/workflows/sql_health_check_cd.yml` builds and dispatches the Python SQL health-check service.
- app-delivery `.github/workflows/bump-clinic-apps-version.yaml` invokes `.github/scripts/bump-clinic-app.sh`, updates the correct desired-state file, and creates the deployment PR.

For staging, the generated app-delivery PR normally enables bot approval and auto-merge. For `prod-cloud` and `prod-op`, the workflow does not enable those steps. Follow the authorization and release boundaries in root `AGENTS.md`; a green source-repository CD run does not prove that the app-delivery PR merged or that the runtime rollout completed.

## Change Routing Checklist

Before implementation, answer all of the following in the dated plan:

1. Which Go module or non-Go tool owns the change?
2. Which executable and API route, job, or plugin consumes it?
3. Which focused test command runs from the correct module root?
4. Which Docker build target and image are affected?
5. Which Clinic CD workflow and component input build that image?
6. Which app-delivery application and environment overlay own the desired state?
7. Does app-delivery require an image-only bump, or a separate configuration change and PR?
8. What staging behavior, Pod/image state, and logs demonstrate success?
9. What explicit authorization is still required before merge or production release?