# TiCDC Metadata for All Cluster Types and Clinic API Credential Discovery

## Background

Clinic Data Service exposes TiCDC changefeed metadata through internal and
Dedicated-only public Data Proxy routes. The original implementation maps
Dedicated to Dataflow and a limited Premium/BYOC allowlist to Dataflow-NG. As a
result, an internal Starter or Essential request returns HTTP 400 before any
upstream RPC, despite the DevOps TiCDC skill documenting support for all cluster
types. The initial remediation incorrectly assumed DevOps routed every
non-Dedicated cluster through Dataflow-NG; its current Essential implementation
uses Serverless instead.

The repository agent guide still documents a temporary `/private/tmp` API-key
file. Local Clinic API credentials now live in a user-owned configuration
directory and require separate staging and production files.

## Goal

Expand internal TiCDC metadata support without changing the public
Dedicated-only boundary. Dedicated uses Dataflow; Premium, BYOC, and shared
pool use Dataflow-NG; Essential uses Serverless. Update the root agent guide
with safe, environment-specific Clinic API-key setup and loading instructions
rooted at `$HOME/.config/clinic-api`.

## Current and Target Contracts

- Internal routes remain:
  - `GET /data-proxy/api/v1/orgs/{orgId}/clusters/{clusterId}/ticdc/changefeeds`
  - `GET /data-proxy/api/v1/orgs/{orgId}/clusters/{clusterId}/ticdc/changefeeds/{changefeedId}`
- Dedicated metadata selects Dataflow.
- Premium, BYOC, and shared-pool metadata select Dataflow-NG.
- Essential (`deployTypeV2=essential`, even when the legacy deploy type is
  `shared`) selects Serverless `ChangefeedOpsService`.
- Unrecognized non-Dedicated, non-Essential values retain the existing
  Dataflow-NG compatibility fallback.
- Public routes remain protected by existing Gateway and handler checks and
  support Dedicated only; this change does not expand public access.
- Existing ownership validation, generated Dataflow user metadata, request
  deadlines, response redaction, and Detail source-cluster equality checks are
  unchanged.
- Credentials are loaded only from an exact non-symlink file owned by the
  current user with mode `600`, inside a non-symlink directory owned by the
  current user with mode `700`.

## Implementation

1. Change `pkg/ticdc.ResolveArchitecture` to mirror the known DevOps routing:
   Dedicated selects Dataflow; Premium/BYOC/shared pool select Dataflow-NG;
   Essential selects Serverless.
2. Update focused architecture tests for known non-Dedicated values and retain
   Dedicated precedence.
3. Maintain the Essential Serverless implementation in the dedicated follow-up
   plan, including its endpoint configuration and opaque ID contract.
4. Update root `AGENTS.md` credential discovery, validation, setup, and loading
   examples for `$HOME/.config/clinic-api/clinic-api-{staging,prod}.env`.

## Compatibility and Non-goals

- No route, DTO, timeout, authorization-header, session, or error-envelope
  change is intended.
- No Gateway/Kong change and no public Starter/Essential access is added.
- The Essential follow-up adds a Serverless endpoint configuration; it does not
  change Dataflow or Dataflow-NG configuration.
- No key value, customer payload, or production response is written to source,
  plans, logs, or tests.

## Test Plan

- Run `go test ./pkg/ticdc ./pkg/handlers` from `clinic_data_service`.
- Build `cmd/api/main.go` with the module's Go toolchain.
- Verify the architecture unit tests dispatch Dedicated to Dataflow,
  Premium/BYOC/shared pool to Dataflow-NG, and Essential to Serverless.
- In staging, call internal List and Detail for one Dedicated and one
  Starter/Essential cluster; confirm the latter no longer returns the local
  unsupported-cluster-type 400. Check structured trace-aware logs for the
  selected upstream architecture.

## Deployment Boundary

- **Affected source module:** `clinic_data_service`; `pkg/ticdc`, entrypoint
  `cmd/api/main.go`, image `clinic/data-service`.
- **Required deployment module:** the `Clinic Data Service CD` workflow with
  `env=staging` (then `env=prod-cloud` after approval); app-delivery application
  `clinic-data-service`; workload `Deployment/clinic-data-service`. This
  dedicated workflow has no `components` input.
- **Documentation-only change:** root `AGENTS.md`; it has no runtime artifact
  or deployment effect.
- No deployment of `clinic`, `internal-api`, `ngm-api`, `task-manager`,
  `cloud-server`, or Gateway is required.