# PRD: API-Level Service Topology Status: APPROVED | Date: 2026-03-31 | Owner: yifan --- ## 1. Problem cp9 classifies every service as REAL/STUB/SKIP when a developer runs `cp9 up`. It needs a pre-computed call graph. The call graph doesn't exist. ``` Developer: "I'm working on CMEK M2, these 4 APIs" | v +------------------+ | topology.json | <-- THIS IS WHAT WE'RE BUILDING +------------------+ | BFS walk: follow outgoing edges | v +-------------------------------+ | .cp9/service-graph.json | | REAL: 8 services (build) | | STUB: 3 services (stub) | | SKIP: 74 services (nothing) | +-------------------------------+ ``` Without it, cp9 can't function. Without cp9, the $56K/month shared dev EKS stays. ## 2. Current State | Metric | Value | Gap | |--------|-------|-----| | Repos with service-mapping | 69 of ~85 deployed | ~16 have zero data | | Legacy string-format files | 50 of 175 (29%) | Missing `target_repo` -- dead ends in graph walk | | Deep-scanned repos | 6 of 69 | Only 6 have method-level accuracy | | Service name variants | 20+ mismatches | `billing2` / `billing-svc` / `billing-service` | | tiinf blind spot | 5+ major services | `auditlog.Submit()`, `rbac.Handle()` hidden calls | ## 3. Goal Produce `topology.json`: - **Accurate** -- method-level edges, no missing downstream calls - **Complete** -- all deployed services (union of `app-registry/*.yaml` + `staging/app-registry/*.yaml`) - **Queryable** -- input: feature API list, output: REAL service set - **Maintainable** -- submodule update triggers incremental re-scan --- ## 4. Repository Structure ``` pingcap-kb-codebase/ │ ├── repos/ # git submodules (source code, on-demand) │ ├── tidb-management-service/ -> tidbcloud/tidb-management-service.git │ ├── account/ -> tidbcloud/account.git │ ├── billing/ -> tidbcloud/billing.git │ ├── app-delivery/ -> tidbcloud/app-delivery.git │ └── ... # ~85 deployed service repos │ ├── knowledge/ # scan outputs (committed) │ ├── repos/{name}/ │ │ ├── meta.json # metadata + scan_commit hash │ │ ├── service-mapping/*.json # per-binary dependencies │ │ └── interfaces/*.json # per-binary API definitions │ ├── topology.json # assembled call graph (THE OUTPUT) │ └── coverage-matrix.json # deployed services x scan status │ ├── construction/ # scripts (committed) │ ├── assemble-topology.py # scan data -> topology.json │ ├── query-topology.py # feature APIs -> REAL service set │ ├── detect-stale.sh # submodule HEAD vs scan_commit │ ├── rescan-stale.py # incremental re-scan + reassemble │ ├── build-coverage-matrix.py # app-delivery x KB scan status │ └── build-name-map.py # service name -> k8s canonical name │ └── docs/ ``` **Why submodules:** Reproducible (any machine, same source). Version-pinned (submodule commit = scanned version). On-demand (`git submodule update --init repos/billing`). **Exception:** tiinf is NOT a submodule. It's a Go library, versioned per-consumer via `go.mod`. Fetched with `go mod download`. --- ## 5. topology.json ### API Definition Types Services define APIs in two ways. The topology must handle both: ``` gRPC (majority) HTTP/Swagger (some services) ─────────────── ──────────────────────────── Defined by: .proto files Defined by: swag comments on Go handler functions Interface naming: Interface naming: ServiceName.MethodName HandlerGroup.HandlerName e.g. ClusterService.CreateCluster e.g. HTTP_API.CreateTenant In interfaces/*.json: In interfaces/*.json: {"name": "CreateCluster", {"name": "CreateTenant", "input": "CreateClusterReq", "method": "POST", "output": "CreateClusterResp"} "path": "/api/v1/tenants"} In service-mapping dep keys: In service-mapping dep keys: ClusterService.CreateCluster HTTP_API.CreateTenantHandler ``` Both use the same `ServiceName.MethodName` convention in dependency keys and edge `from_api`/`to_api` fields. The topology schema is API-type-agnostic -- it works for both gRPC and HTTP. ### Schema ```json { "version": "1.0", "generated_at": "2026-04-05T...", "coverage": { "total_deployed": 85, "scanned": 85, "missing": [] }, "name_map": { "account-service": "account-svc", "billing2": "billing-svc" }, "nodes": [ { "name": "tidb-mgmt-service", "repo": "tidbcloud/tidb-management-service", "binary": "tidb-mgmt-service", "type": "grpc_server" }, { "name": "billing-svc", "repo": "tidbcloud/billing", "binary": "server", "type": "http_server" } ], "edges": [ { "from": "tidb-mgmt-service", "to": "account-svc", "apis": [ { "from_api": "ClusterService.CreateCluster", "to_api": "AccountAPIService.GetProjectByID", "type": "grpc_unary", "source": "direct" } ] }, { "from": "tidb-mgmt-service", "to": "billing-svc", "apis": [ { "from_api": "ClusterService.CreateCluster", "to_api": "HTTP_API.CheckQuota", "type": "http", "source": "direct" } ] } ], "tiinf_templates": [ { "package": "github.com/tidbcloud/tiinf/sdk/auditlog", "implied_edge": { "to": "audit-service", "type": "async", "api": "AuditLog.Submit" } } ] } ``` **Field enums:** | Field | Values | |-------|--------| | `node.type` | `grpc_server`, `http_server`, `grpc_gateway`, `async_consumer`, `cron_job`, `cli` | | `apis[].type` | `grpc_unary`, `grpc_stream`, `http`, `async`, `mq_publish`, `client_init` | | `apis[].source` | `direct` (source scan), `tiinf_template` (inferred from import) | ### Name Canonicalization **Canonical name = Kubernetes service name** (what services use to address each other at runtime). **Service list source:** union of `app-delivery/app-registry/*.yaml` + `app-delivery/staging/app-registry/*.yaml`. Both scanned, deduplicated by k8s service name. Services not in either directory are excluded from topology. ``` In source code Canonical (k8s svc name) ───────────── ──────────────────────── account-service ──┐ account-provider ──┼──> account-svc account-central ──┘ billing2 ──┐ billing-service ──┼──> billing-svc billing-workers ──┘ ``` Built by `build-name-map.py`: parse ArgoCD manifests for k8s service names, match against all `services[].name` values in service-mappings. Unresolvable names flagged for human review and excluded. --- ## 6. Classification Algorithm ``` classify(topology, active_services, working_apis): # BFS from active services, downstream only reachable = set(active_services) queue = list(active_services) visited = set() while queue: svc = queue.pop(0) if svc in visited: continue visited.add(svc) for edge in outgoing_edges(svc): if working_apis and no edge.api matches working_apis: continue # API-level filtering reachable.add(edge.to) queue.append(edge.to) REAL = reachable STUB = {svc not in reachable, but called by some REAL service} SKIP = everything else # Manual overrides from cp9.yaml services.skip take precedence # Unknown services (not in topology) default to STUB + cp9 doctor warning ``` ``` Developer's active services | BFS downstream | +-----------+-----------+ | | | v v v [REAL] [REAL] [REAL] | | v v [REAL] [REAL] | +--not in BFS path--+ v v v [REAL] [STUB] [STUB] (called by REAL (referenced in but not on the config, not in code path) call topology) Everything else: [SKIP] ``` --- ## 7. Query Interface Input: entry binary + feature API list. Output: REAL service topology. Works with both gRPC and HTTP APIs -- same `ServiceName.MethodName` format. ```bash # gRPC APIs (majority) python3 construction/query-topology.py \ --entry tidb-mgmt-portal \ --apis "ClusterService.CreateCluster,CMEKService.EnableCMEK" \ --pretty # HTTP/Swagger APIs (some services like billing) python3 construction/query-topology.py \ --entry billing-svc \ --apis "HTTP_API.CreateTenantHandler,HTTP_API.CheckQuotaHandler" \ --pretty # Just service names (for cp9) python3 construction/query-topology.py ... --services-only # DOT graph for visualization python3 construction/query-topology.py ... --dot ``` ```json { "real": [ {"service": "tidb-mgmt-portal", "apis": ["ClusterService.CreateCluster"]}, {"service": "tidb-mgmt-service", "apis": ["ClusterService.CreateCluster"]}, {"service": "dedicated-global-service", "apis": ["ClusterService.CreateCluster"]}, {"service": "account-svc", "apis": ["AccountAPIService.GetProjectByID"]} ], "stub": [ {"service": "rbac-service", "reason": "middleware only, not on feature path"} ], "skip_count": 72 } ``` Without API-level filtering, cp9 classifies every downstream service as REAL (30+ services). With filtering, CMEK M2 touches 4 APIs, traces to ~8 services. 3-minute `cp9 up` vs 10-minute. --- ## 8. Recompute Workflow ``` repos/billing submodule knowledge/repos/billing/meta.json ┌─────────────────────┐ ┌──────────────────────────────┐ │ HEAD: abc1234 │ │ "scan_commit": "def5678" │ └─────────────────────┘ └──────────────────────────────┘ | | └───── abc1234 != def5678 ───────────┘ | STALE: billing | v ┌─────────────────────────┐ │ rescan-stale.py --repo │ │ billing │ │ │ │ 1. git diff def5678.. │ │ abc1234 (3 files) │ │ 2. LLM re-scans only │ │ changed files │ │ (~5-20K tokens) │ │ 3. Updates meta.json │ │ scan_commit=abc1234 │ │ 4. Reassembles │ │ topology.json │ └─────────────────────────┘ ``` ```bash python3 construction/rescan-stale.py --dry-run # show stale repos python3 construction/rescan-stale.py --repo billing # re-scan one python3 construction/rescan-stale.py --all # re-scan all stale ``` --- ## 9. tiinf Shared Library 5+ services import `github.com/tidbcloud/tiinf/**`. These packages wrap service calls invisible to source-only scan: | Package | Hidden call | Target | |---------|-------------|--------| | `sdk/auditlog` | `.Submit()` | audit-service | | `sdk/rbac` | `.Handle()` | rbac-service | | `transport/tieventbus` | `.Publish()` | eventbus | | `transport/tigrpc` | gRPC wrappers | (transport) | | `sdk/auth` | identity resolution | (auth infra) | **Approach:** Scan tiinf once, produce `tiinf_templates` in topology.json. Assembly auto-applies implied edges for every service that imports the package (detected from `go.mod`). **Process:** 1. `GOPRIVATE=github.com/tidbcloud/* go mod download` 2. LLM scans tiinf packages in `$GOPATH/pkg/mod/` 3. Output: `tiinf_templates` section in topology.json 4. `assemble-topology.py` applies templates to every importing service **Versions:** Diff tiinf packages across the 3 most common versions. If implied edges are identical, scan latest only. If they differ, create version-specific templates. --- ## 10. Build Process ### Coverage Matrix Maps deployed services to scan status. Drives the scan work queue. ``` build-coverage-matrix.py Input: app-delivery/app-registry/*.yaml app-delivery/staging/app-registry/*.yaml knowledge/repos/*/meta.json Output: coverage-matrix.json {"service": "account-svc", "repo": "tidbcloud/account", "scan": "scanned", "gap": null} {"service": "some-svc", "repo": "tidbcloud/some", "scan": null, "gap": "no_kb_entry"} ``` All entries with `gap != null` go into the scan queue. ### Legacy Format Migration 50 service-mapping files use string format (`"services": ["name (Method)"]`). Each is re-scanned by an LLM agent at scan quality using source code from submodule. Batched by repo (~20 repos). ### Scan Methodology Extension Current scan best-practise doc `construction/deep-scan-best-practise.md` extended with: **Dual API type support:** 1. For gRPC services: scan proto file registrations (`pb.Register*Server`) and gRPC client calls (`pb.New*Client`) 2. For HTTP/Swagger services: scan swag-annotated handler methods, gin/echo route registrations, and HTTP client calls to known service URLs 3. Both produce the same `ServiceName.MethodName` format in dependency keys 4. The `node.type` field distinguishes `grpc_server` vs `http_server` vs `grpc_gateway` (services that proxy HTTP to gRPC) **Shared library analysis:** 1. Check `go.mod` for tiinf imports 2. Ensure service-mapping includes tiinf-template edges 3. For non-tiinf shared libs: `go mod download` + analyze source in module cache 4. Record in meta.json: `"shared_lib_deps": ["tiinf/sdk/auditlog", ...]` ### Assembly ``` assemble-topology.py Input: knowledge/repos/*/service-mapping/*.json knowledge/repos/*/meta.json name-map.json tiinf-edge-templates.json coverage-matrix.json Steps: 1. Load all service-mappings 2. Normalize names via name_map 3. Create edges with API-level detail 4. Apply tiinf templates for importing services 5. Deduplicate, validate all edge targets exist 6. Write knowledge/topology.json ``` --- ## 11. Token Cost | Repo size | Example | Input | Output | Count | |-----------|---------|-------|--------|-------| | Large (7+ binaries) | tidb-management-service | ~200K | ~80K | ~5 | | Medium (2-4 binaries) | billing, account | ~50K | ~20K | ~30 | | Small (1 binary) | config-center | ~15K | ~8K | ~50 | Total estimate: ~4-8M tokens ($60-180). Validate with 3 pilot scans first. Incremental re-scan: ~5-20K tokens per repo. Assembly: zero LLM cost. --- ## 12. Execution | Phase | When | Deliverables | |-------|------|-------------| | **1. Infrastructure** | Week 1 | Submodules, coverage matrix, name map, tiinf templates, `detect-stale.sh`, `rescan-stale.py`, 3 pilot scans | | **2. Scan Campaign** | Week 1-2 | 50 legacy files re-scanned, all coverage gaps filled, `scan_commit` recorded | | **3. Assembly** | Week 2-3 | `assemble-topology.py`, `query-topology.py`, topology.json v1.0, validation (30-edge spot-check + 3 cp9 test scenarios) | | **4. Maintenance** | Ongoing | Incremental re-scan on submodule updates, monthly full validation | --- ## 13. Success Criteria | Criteria | Target | |----------|--------| | Node coverage | 100% of deployed services (both app-registry dirs, deduplicated) | | Edge completeness | 3 test scenarios: working local env, no runtime dependency errors. 30 random edges verified. | | Names | 0 unresolved aliases | | tiinf | All implied edges present | | Legacy | 0 string-format files | | Submodules | All ~85 repos linked | | Recompute | < 5 min incremental re-scan for typical PR | | Query | Matches manual tracing for 3 test features | --- ## 14. Open Questions 1. **tiinf edge targets:** Are audit-service, rbac-service, eventbus in `app-registry/*.yaml` or `staging/app-registry/*.yaml`? If not, they're infrastructure -- cp9 should SKIP by default. 2. **Regional routing:** Some services call regional variants. cp9 is single-region. Should topology capture this? 3. **Proto files:** cp9 STUB generation needs proto/swagger. Should KB track proto file locations per service? 4. **Multi-binary repos:** tidb-management-service has 7 binaries, each deployed separately. Topology nodes = binaries, not repos. Verify against app-registry granularity.