# AGENTS.md

## Repository Structure

This is a Go monorepo with multiple modules, each with its own `go.mod`:

| Module | Purpose                                                                                     |
|---|---------------------------------------------------------------------------------------------|
| `api/` | Proto definitions (openapi + internal global), buf codegen, swagger                         |
| `tidb-mgmt-service/` | OpenAPI gateway — translates openapi proto types to internal global proto types             |
| `nextgen-global-service/` | Internal global service for nextgen (premium/byoc) clusters                                 |
| `dedicated-global-service/` | Internal global service for dedicated clusters                                              |
| `tidb-mgmt-portal/` | Portal service                                                                              |
| `serverless-global-service/` | Internal global service for serverless(starter/essential) clusters                          |
| `utilities/` | Shared utilities (bizerr, grpc helpers, etc.)                                               |
| `functional-test/` | End-to-end functional tests                                                                 |
| `deployments/` | Deprecated: Already migrated to ArgoCD in another repo. CDK deployment configs (TypeScript) |

## Build & Lint Commands

- **Root-level**: `make dev` runs dev target across all modules.
- **Per-module**: `cd <module> && make dev` (runs fmt, lint, test, etc.)
- **Generate**: `cd <module> && make generate` — runs `go generate ./...` (regenerates mocks, etc.). **In `api/` use `make gen` instead**: that module has several generation targets that must all run, in order — `make gen` is that set, so you never have to pick a subset.
- **Format**: `cd <module> && make fmt` — uses `golangci-lint v2.6 run --fix` with root `.golangci.yml`
- **Proto codegen**: `cd api && make gen` — the single entry point for regenerating everything committed in `api/`. Do not run individual `generate*` targets: `generate-buf` produces the `.pb.go` from the protos and `generate` then rewrites enum `String()` methods (`cmd/rewrite_enum_stringer`) to honour the `datatype.v1` name option. Running only the first leaves those enums silently reporting protobuf default names instead of failing loudly.
- **Why this matters beyond `api/`**: every service module reaches this one through `replace github.com/tidbcloud/tidb-management-service/api => ../api`, and tools that build services from a checkout (e.g. CP9 source build) compile `api/` straight from your working tree. Stale or half-generated output here becomes their build error or their wrong enum names.
- **AIP lint**: `cd api && make lint-gapi` — uses `api-linter@v1.69` with config at `api/spec/tidb_cloud_open_api/.api-linter.yaml`
- **Buf lint**: `cd api && make lint-buf`
- **Buf breaking change check**: `cd api && make lint-buf-breaking` — compares against `release-v0.84` branch
- **Go private**: All modules use `GOPRIVATE=github.com/tidbcloud/*`

**Before finishing any change, you must run `make generate` / `make fmt` / `make lint` for every affected module and fix all reported issues.**

## Proto API Conventions

### Two proto layers

- **OpenAPI protos** (`api/spec/tidb_cloud_open_api/`): Public-facing API definitions, subject to AIP lint rules.
- **Global/internal protos** (`api/spec/global/`): Internal gRPC between services. Not AIP-linted.

### Naming pattern

- OpenAPI uses user-facing names: `TidbId`, `CustomerManagedEncryptionKey`
- Global/internal uses internal names: `ClusterId`, `EncryptionKey`
- The `tidb-mgmt-service` handler layer converts between the two.

### AIP compliance rules (enforced by `lint-gapi`)

- **AIP-136 (custom methods)**: URI suffix must match method name (e.g., `SwitchFoo` → `:switchFoo`). No prepositions ("by", "for", "with") in RPC names. Response message name must match `<RpcName>Response` or the resource type.
- **AIP-131 (get)**: Don't name custom RPCs starting with "Get" unless they follow standard Get semantics. Avoid synonyms like "Lookup", "Fetch", "Retrieve" — the linter flags them.
- **AIP-134 (update)**: Don't name custom RPCs starting with "Update" unless they follow standard Update semantics (PATCH, field mask, resource in body).
- **AIP-141**: Use `string` or `int64` for IDs and large numbers — never `uint64`, `uint32`, `fixed64`, `fixed32`.
- **AIP-158 (pagination)**: All `List` RPCs must include `page_size` + `page_token` in request and `next_page_token` in response.
- **AIP-191 (file layout)**: Messages must appear before enums in proto files.
- **AIP-216**: State enum fields should be `OUTPUT_ONLY`.

## Code Patterns

### Handler / converter layer (`tidb-mgmt-service/app/openapi/nextgen/`)

- Each service has a package (e.g., `cmek/`, `tidb/`, `quota/`).
- Handlers implement the generated gRPC server interface from openapi protos.
- Converters in `common/` translate between openapi and global proto types.
- When openapi proto field type differs from global proto field type (e.g., openapi `string` vs global `uint64`), convert explicitly with `strconv.FormatUint` / `strconv.ParseUint`.

### Mock generation

- Interface + mock pattern: interface in `domain/adapter/<name>/<name>.go` with `//go:generate mockgen` directive.
- Mocks live in `domain/adapter/<name>/mock_<name>/`.
- After changing any embedded interface (e.g., adding an RPC to a proto service), **regenerate mocks**: `go generate ./domain/adapter/<name>/...`
- Forgetting to regenerate mocks causes `make fmt` failures (typecheck errors in test files).

### Proto changes checklist

1. Edit `.proto` files
2. `cd api && make gen` — regenerate all committed artifacts. Use `make dev` when you also want the lint/test gates; every sub-command must succeed except `lint-buf-breaking`.
3. Report `lint-buf-breaking` results to human review.

## Go Code Rules

### Error creation (`bizerr.New`)

- **Never** store `bizerr.New(...)` in a file-level `var`. `bizerr.New()` captures the call-site stack trace for debugging; a file-level var records the init-time stack, which is useless.
- Always call `bizerr.New(...)` inline at each `return` site so the stack trace points to the actual error location.

### Workflow DAG creation (`RunDagWithBizId`)

- `workflow.RunDagWithBizId` must be called **inside the same database transaction** as any related DB mutations (e.g., creating an operation record). This ensures atomicity — if the workflow fails to create, the DB record is rolled back, and vice versa.
- Follow the pattern in `cluster/create.go` and `encryptionkey/defaultkey.go`: wrap the DB write + `RunDagWithBizId` in `s.transaction.Transaction(ctx, func(ctx context.Context) error { ... })`.

### Workflow action DB writes

- If a workflow action modifies **multiple DB records/tables** in one run path, those writes must be wrapped in a **single transaction** in that action.
- Follow the pattern in `finalize_encryption_change.go`: inject `db.Transactioner` into the action and wrap all related repository updates in `transaction.Transaction(...)`.

### Workflow action error logging

- In workflow action `Run(...)`, before returning any non-nil error, call `tilog.Error(...)` with sufficient context (action name, key IDs, and the error).
- This applies to all early returns (missing vars, parse errors, repo/executor errors, retryable state errors), not only terminal failures.

### OpenAPI ↔ Global enum mapping

- Customer-facing OpenAPI enums should be simplified views of internal global enums. Hide backend implementation details (e.g., internal workflow stages) from external users.
- Use explicit `switch` mapping functions (not direct casts) when OpenAPI and global enums have different value sets. Place mapping functions in `tidb-mgmt-service/app/openapi/nextgen/common/`.

## Authorization: Two-Layer RBAC

All APIs exposed on `tidb-mgmt-service` must pass through **two** authorization layers. Both layers must be updated when adding a new API.

### Layer 1: Portal RBAC (`tidb-mgmt-portal` — IAM SDK)

The portal runs as the HTTP gateway in front of `tidb-mgmt-service`. It uses the IAM team's RBAC SDK (`github.com/tidbcloud/tiinf/sdk/auth`) to check permissions before forwarding requests.

**How it works:**

1. AuthN middleware authenticates the user (Auth0 token, API Key, or Access Token).
2. The HTTP `METHOD:PATH` is matched against `PathPattern2ResourceMap` in `tidb-mgmt-portal/domain/service/rbac/resource.go` to resolve a **resource URI** (e.g., `get:/dedicated/clusters` → `/dedicated/orgs/{org_id}/projects/{project_id}/clusters/list`).
3. The resource URI (with `{org_id}`, `{project_id}`, `{cluster_id}` placeholders filled) is passed to `authsdk.CheckIdentityPermission()`. For project-scoped resources the SDK is called per-project, filtering `user.Projects` to only those the user is authorized for.
4. On success, the portal sets gRPC metadata headers (`x-org-id`, `x-user-id`, `x-project-id-list`, `x-default-project-id`, `x-cluster-id`, etc.) and forwards to `tidb-mgmt-service` via gRPC-Gateway.
5. Unmatched paths return HTTP 404. Health/version/ping paths (`ignoreRBACPaths`) bypass RBAC.

**Adding a new API — portal side:**

- Add a regex → resource URI entry to `PathPattern2ResourceMap` in `tidb-mgmt-portal/domain/service/rbac/resource.go`.
- The resource URI **must exactly match** the corresponding entry in the account service's [`ALL_PRIVILEGE.json`](https://github.com/tidbcloud/account/blob/main/tools/rbac-management/ALL_PRIVILEGE.json) and [`ALL_ROLEBINDING.json`](https://github.com/tidbcloud/account/blob/main/tools/rbac-management/ALL_ROLEBINDING.json). Coordinate with the IAM team. See [Dedicated Mgmt Service RBAC SOP](https://pingcap.feishu.cn/wiki/P4G1wuFmuiNlMvk6P3Uc0tnZnmb).

### Layer 2: Permission Check Interceptor (`tidb-mgmt-service` — self-managed)

A gRPC `UnaryServerInterceptor` that validates the requesting user has access to the **specific resource** being acted on. It is registered as the last interceptor in the chain (`tidb-mgmt-service/cmd/tidb-mgmt-service/helper.go`).

**How it works:**

1. The interceptor looks up `info.FullMethod` in `methodToCheckerMap` (`tidb-mgmt-service/pkg/apimiddleware/authorization/permission/interceptor.go`).
2. If a checker is found, it calls `checker.Check(ctx, checkCtx, req, info)` which:
   - Extracts `RequestMeta` from gRPC metadata (populated by portal — contains `ValidProjectList`, `OrgID`, etc.).
   - Parses the resource ID from the request proto message (e.g., `cluster_id`, `tidb_id`).
   - Queries DB or calls a global service to resolve the resource's owning project.
   - Calls `reqMeta.CheckProjectIDsPermission(projectId)` to verify the project is in the user's `ValidProjectList`.
   - Sets `IsCheckPermission = true` in metadata on success.
3. If no checker is found for an OpenAPI/BFF method (`/tidb_cloud_open_api` or `/bff` prefix), the interceptor **returns an error** — every public API must have a checker.
4. If no checker is found for internal/proxy/core methods, the check is skipped and `IsCheckPermission` is set to `true`.

**Checker types** (in `tidb-mgmt-service/pkg/apimiddleware/authorization/permission/checker/`):

| Checker | Validates | Data Source |
|---|---|---|
| `ClusterPermissionChecker` | cluster's project ∈ ValidProjectList | `ClusterRepo` (DB) |
| `CreateClusterPermissionChecker` | project-id label from create request ∈ ValidProjectList | Request proto labels |
| `ProjectIdChecker` | project_id from request ∈ ValidProjectList | Request proto field |
| `TidbPermissionChecker` | TiDB instance's org matches user's org | `ClusterRepo` (DB) |
| `TidbGroupPermissionChecker` | cluster + TiDB group exist & project ∈ ValidProjectList | `ClusterRepo` + `DedicatedCli` |
| `CmekIdChecker` | CMEK's project ∈ ValidProjectList | `DedicatedCli` (gRPC) |
| `NetworkContainerIdChecker` | network container's project ∈ ValidProjectList | `DedicatedCli` (gRPC) |
| `VpcPeeringIdChecker` | VPC peering's project ∈ ValidProjectList | `DedicatedCli` (gRPC) |
| `MaintenanceTaskIdChecker` | task's project ∈ ValidProjectList | `MaintenanceTaskRepo` (DB) |
| `DefaultPermissionCheckerFunc` | No-op — just marks IsCheckPermission | — |
| `ApikeyChecker` | Simple secret API key match | gRPC metadata header |

**Adding a new API — interceptor side:**

- Add a `FullMethodName → checker` entry to `methodToCheckerMap` in `interceptor.go`.
- Pick the appropriate checker based on what resource is being accessed. If a new resource type is needed, add a new checker file in `checker/`.
- If the new API is an OpenAPI or BFF method and no entry is added, requests will fail at runtime with `"permission checker for method ... not found"`.

### Checklist when adding a new public API

1. **Portal RBAC**: Add regex → resource URI entry in `PathPattern2ResourceMap` (`tidb-mgmt-portal/domain/service/rbac/resource.go`). Coordinate with IAM team to add matching entries in `ALL_PRIVILEGE.json` / `ALL_ROLEBINDING.json`.
2. **Permission interceptor**: Add `FullMethodName → checker` entry in `methodToCheckerMap` (`tidb-mgmt-service/pkg/apimiddleware/authorization/permission/interceptor.go`).
3. **Verify**: Missing either registration causes runtime failures — portal returns 404 (layer 1) or interceptor returns PermissionDenied (layer 2).