# Serverless control-plane info backend default ## Current Design Clinic has two related but different identities in Starter/Essential observability paths: - The TiDB Cloud cluster ID, for example `10978346059039804452`. - The control-plane project/logical Kubernetes name, for example `us-east-1-f01` or `us-east-1-f02`. Frontend code currently builds `control_plane_info` and Loki `cluster` label values from cluster detail fields: - `clinic-ui/packages/clinic-web/src/pages/cluster/cards/MetricsModal.tsx` builds metrics dashboard URLs with `var-control_plane_info`. - `clinic-ui/packages/clinic-web/src/utils/links.ts` builds serverless log Explore links and currently synthesizes the Loki `cluster` label from `envConfig.appReleaseEnv` plus `shootName`. Backend metrics and logs paths already resolve O11Y targets, but their use of control-plane information is uneven: - `/metric/query?metrics_scope=parent` reads `control_plane_info` from request/header/referer/cookie and requires it for parent metrics. - `pkg/o11y/urls.resolveParentMetricsTarget` parses `control_plane_info` as `projectID/vendor-region/applicationID` and calls O11Y `GET /api/v1/clusters`. - `/log/query` currently calls `ResolveLogsTarget`, which routes to child O11Y log targets for Starter/Essential. - `pkg/o11y/urls.resolveParentLogsTarget` exists, but the active `/log/query` entrypoint does not use it as the default serverless path. - `pkg/o11y/urls.selectServerlessO11yCluster` already calls mgmt realtime cluster API, reads `ServerlessCluster.Cluster.Kubernetes`, and uses it to disambiguate multiple serverless O11Y child clusters by matching the O11Y `projectID`. The backend can read the raw fields needed for parent resolution: - `provider` - `region` - `deploy_type` / `deploy_type_v2` - `tenant_id` - `shoot_name` - mgmt realtime `ServerlessCluster.Cluster.Kubernetes` - O11Y child cluster `projectID` However, there is no single backend helper that turns a TiDB Cloud cluster ID into the canonical serverless control-plane identity. Parent metrics still depend on frontend-provided `control_plane_info`, and serverless log links still depend on frontend-generated Loki `cluster` filters. ## Risks - Frontend build-time environment can be wrong for the deployed environment. If the UI synthesizes `dev-us-east-1-f01` in staging, logs query the wrong Loki label even if the backend routes to the correct O11Y URL. - Fallback values such as `${region}-f01` are not reliable once one region can contain multiple serverless control planes such as f01 and f02. - `control_plane_info` can be stale when it comes from URL, referer, or cookie. - Deleted clusters may no longer have complete mgmt realtime data. Clinic may need to fall back to DB snapshot fields or O11Y `showDeleted=true` child cluster metadata. - Parent and child O11Y targets are different semantics. A failed parent lookup must not silently fall back to child data. - Loki `logQueryUri` chooses the O11Y/Loki target, but it does not by itself prove that a query is scoped to one TiDB Cloud cluster. Label filtering may still be needed, but the label value should not be synthesized by frontend code. ## Target Contract The canonical input for Starter/Essential parent routing is the TiDB Cloud cluster ID. Frontend may continue to pass `var-control_plane_info` and existing Loki selectors for compatibility, but those values are not the default source of truth. Backend should provide one shared resolver: ```text ResolveServerlessControlPlaneInfo(ctx, user, cluster) -> controlPlaneInfo ``` The resolver should derive the actual f01/f02 mapping in this order: 1. Query mgmt realtime cluster and read `ServerlessCluster.Cluster.Kubernetes`. 2. Strip environment prefix `prod-`, `staging-`, or `dev-` from that Kubernetes name. 3. If mgmt realtime data is unavailable, use `cluster.ShootName` from Clinic's cluster model. 4. If the request is resolving a child O11Y target and O11Y returns one or more child clusters, use the selected child cluster `projectID` as a validation or fallback source. 5. Only as a last compatibility fallback, derive `${region}-a01` for Alicloud and `${region}-f01` for other providers. This fallback must be logged. The resolved `control_plane_info` format is: ```text /-/devtier-infra ``` Examples: ```text us-east-1-f01/aws-us-east-1/devtier-infra us-east-1-f02/aws-us-east-1/devtier-infra ap-southeast-1-a01/alicloud-ap-southeast-1/devtier-infra ``` Metrics and logs should use the same resolver: - Parent metrics: if request `control_plane_info` is absent, resolve it from backend cluster data. - Parent logs: if serverless logs need a parent/control-plane target or label scope, resolve it from backend cluster data. - Child logs/metrics: keep using child O11Y resolution, but share the same mgmt Kubernetes lookup helper instead of embedding it only in `selectServerlessO11yCluster`. ## Query Scope Contract For Loki logs, distinguish two separate responsibilities: - Target routing: choose the O11Y `logQueryUri`. - Data filtering: constrain LogQL to the correct cluster/control-plane/TiDB cluster labels. `logQueryUri` is the canonical target URL, but Clinic should not assume it is enough to filter data to a single TiDB Cloud cluster unless O11Y explicitly guarantees that contract. If LogQL still needs a `cluster` label: - The value must come from backend-resolved control-plane identity or O11Y metadata. - Frontend must not synthesize it from `REACT_APP_RELEASE_ENV` or fallback region conventions. - Backend may inject or replace the label in `/log/query` after permission and target resolution. If O11Y later guarantees that `logQueryUri` is already scoped tightly enough, frontend serverless log links can omit the `cluster` label entirely and keep only business selectors such as `namespace`, `container`, `component`, or `tidb_cluster_id`. ## Implementation Steps 1. Add a shared serverless control-plane resolver. - Place it near `pkg/o11y/urls` or a shared proxy/O11Y helper package. - Reuse the existing mgmt realtime Kubernetes lookup from `selectServerlessO11yCluster`. - Return both raw Kubernetes name and normalized project ID for logging/debugging. 2. Replace duplicated serverless Kubernetes lookup. - Update `selectServerlessO11yCluster` to call the shared resolver or its lower-level Kubernetes-name helper. - Preserve the current O11Y child cluster selection behavior. 3. Add backend defaulting for parent metrics. - In `/metric/query?metrics_scope=parent`, continue accepting request-provided `control_plane_info`. - If it is absent, compute it from the cluster via the shared resolver. - Keep request-provided values for backward compatibility, but log whether the value came from request or backend default. 4. Decide and implement serverless logs query scope. - First confirm whether O11Y `logQueryUri` is scoped to a single parent/control-plane or only to a regional/shared Loki target. - If label filtering is still required, inject or replace the required label in backend `/log/query`. - If label filtering is not required, remove frontend-generated `cluster` labels from serverless log links. 5. Update frontend only after backend defaulting exists. - Keep passing `var-cluster_id`. - Keep passing `var-control_plane_info` during compatibility period if needed. - Stop using `envConfig.appReleaseEnv` to synthesize serverless Loki `cluster` labels. 6. Add rollout logging. - Log source of control-plane info: request, mgmt Kubernetes, Clinic `shootName`, O11Y child `projectID`, or fallback. - Log the normalized project ID and deploy type, but avoid exposing sensitive tokens or full auth data. ## Verification Checklist - Unit test serverless control-plane resolver: - mgmt Kubernetes `staging-us-east-1-f02` resolves to `us-east-1-f02/aws-us-east-1/devtier-infra`. - mgmt Kubernetes `dev-ap-southeast-1-a01-cluster` or Alicloud equivalent preserves the expected project ID shape. - missing mgmt Kubernetes falls back to `cluster.ShootName`. - missing `ShootName` falls back to provider-specific region convention and emits a warning path. - Unit test parent metrics: - request-provided `control_plane_info` still works. - missing `control_plane_info` for Starter/Essential resolves through backend default. - non-Starter/Essential parent scope still returns a clear bad request. - failed parent lookup does not fall back to child metrics. - Unit test child O11Y selection: - multiple O11Y child clusters are disambiguated by mgmt Kubernetes suffix matching O11Y `projectID`. - when disambiguation fails, current fallback behavior is preserved and warning is logged. - Log query verification: - For a region with f01 and f02, the same serverless cluster opens logs against the expected O11Y target and query scope. - A UI bundle built for the wrong environment cannot make staging query `dev-*` labels. - Deleted Starter/Essential cluster behavior is tested with `showDeleted=true` and documented if mgmt no longer returns the Kubernetes mapping. - Manual staging checks: - Active Starter cluster in f01. - Active Starter cluster in f02. - Deleted Starter cluster. - Essential cluster if data exists. - Alicloud Starter/Essential cluster. ## Non-Goals - Do not change this behavior in the current incident slice. - Do not remove `var-control_plane_info` from frontend URLs until backend defaulting and compatibility logging have been deployed. - Do not use `clusters.parent_id` as the source of truth for Starter/Essential parent routing. - Do not infer f01/f02 only from region. - Do not make parent metrics or parent logs fall back to child O11Y targets when parent resolution fails. - Do not migrate `data-proxy/system-logs`; system logs remain datasource-scoped, not TiDB cluster scoped. ## Open Questions - Does O11Y `logQueryUri` guarantee parent/control-plane data isolation, or can one `logQueryUri` still contain multiple logical control planes? - Which Loki label should be canonical for serverless log scoping if `cluster` is not stable enough: `cluster`, `projectID`, `tidb_cluster_id`, namespace, or another O11Y-provided label? - For deleted clusters, will mgmt retain `ServerlessCluster.Cluster.Kubernetes`, or should O11Y child cluster `projectID` with `showDeleted=true` be treated as the primary fallback? - Should O11Y expose a direct child-cluster-to-parent metadata field so Clinic does not need to reconstruct `control_plane_info`?