# Clinic API Caller Attribution ## Background Clinic authenticates requests at the Kong `clinic_auth` plugin using either a Clinic web session or an API key. The authenticated user is propagated to the Clinic API, but API-key verification returns only the user ID and drops the matched credential identity. The gateway request log is also emitted before authentication. As a result, production requests and their database cost cannot be attributed to a specific user or automation credential. Issue: https://github.com/pingcap-inc/clinic/issues/1681 ## Goal Emit one structured caller log for each request handled by the Clinic Kong plugin after authentication has run. Authenticated requests must identify either the Clinic UI user or the exact API credential. The log must also contain the request trace ID, method, and path without query parameters. ## Current Behavior - Session authentication returns a `CtxUser` without the authentication method. - API-key verification returns a user ID and discards the matched credential type, ID, and name. - The gateway logs the method and path before authentication and does not log the final response status or duration. - The first staging implementation attempted to read request, response, request-scoped context, and Nginx variables directly from the Go plugin's `Log` phase. Kong runs that phase in an `ngx.timer` context, where those PDK calls did not return the request values. The emitted event therefore had an anonymous caller, empty method/path, zero status, and no duration. - The in-process API-key fallback includes raw key material in several error logs. - Internal named API keys already store an ID and readable name. No separate audit storage exists or is needed for this change. ## Target Contract - `CtxUser` carries `auth_method`, `credential_type`, and `credential_id`. - UI sessions use `auth_method=ui_session`. - API credentials use `auth_method=api_key` and preserve the matched credential type and ID. The existing internal credential name is kept for logging. - Requests allowed by the current anonymous-route behavior are logged with `auth_method=anonymous`, making them visible before the separate default-deny change. - During `Access`, after authentication has identified any known caller, the plugin emits one `clinic_access` event with trace ID, method, path, and caller identity. Authentication and RBAC rejections retain a known caller when one was successfully verified. - Kong's native access log remains the source for final response status and duration. This change does not add cross-phase state just to duplicate those outcome fields in the caller log. - Raw credentials, authorization headers, cookies, and query strings are never included in attribution logs. - Existing authorization and anonymous-route behavior does not change. ## Affected Modules and Routes ### Affected source modules - Root Go module `github.com/pingcap-inc/clinic`. - `pkg/api_gateway/auth/api_key`: return matched credential identity. - `pkg/api_gateway/auth/session`: mark UI session authentication. - `pkg/auth/session`: carry the authentication identity in `CtxUser`. - `cmd/api_gateway/clinic`: retain request attribution and log it immediately after authentication in the Access phase. - `pkg/auth/api_key`: redact raw credential material from fallback error logs. - `AGENTS.md`: document the read-only caller-attribution lookup used during Clinic system-log diagnosis. - Executables/images: - `clinic_auth` in the `api-gateway` image for caller attribution. - `cmd/server` in the `clinic` image for the fallback-log redaction only. The behavior applies to all routes handled by the Clinic Kong plugin. It does not change the route allowlists in this issue. ## Implementation Steps 1. Introduce a small verified-credential result containing user ID, credential type, credential ID, and the existing readable name. 2. Update upload, legacy API, and internal API credential verification to return that result without changing validation behavior. 3. Add authentication identity fields to `CtxUser`; set API credential fields immediately after credential verification so a later user lookup failure retains the known caller, and mark UI-session authentication. 4. Replace the pre-authentication request log with a post-authentication `clinic_access` event in the same `Access` phase. Record trace ID, method, path without query parameters, and the verified caller identity. 5. Remove raw key values from the in-process fallback authentication error logs. 6. Add focused tests for each credential type, UI sessions, caller logging, anonymous attribution, verified credentials followed by user lookup failures, and redaction-sensitive paths. 7. Add a concise Clinic-repository runbook mapping caller fields to the existing user and credential tables. Keep this diagnostic guidance out of the Nutshell API-usage skill. ## Exclusions - No database schema migration or audit table. - No per-request database write. Existing internal-key `last_used_at` behavior remains unchanged. - No default-deny or anonymous-route change; that work remains separate in https://github.com/pingcap-inc/clinic/pull/1653. - No `clinic-internal-api` service-to-service credential migration. - No Clinic UI change. - No Nutshell skill change; that skill documents API usage rather than Clinic operator diagnosis. - No process-global request map, background cleanup, audit database, extra persistence layer, new upstream header, or other cross-phase store. - No status, duration, or response-source duplication in `clinic_access`. Kong's native access log already records request outcomes; this PR adds the missing caller identity at the authentication boundary. ## Compatibility The additional `CtxUser` JSON fields are backward compatible with the Clinic API's existing decoder. Credential validation, RBAC, response behavior, and API contracts remain unchanged. Legacy and upload credentials do not have readable names, so their logs use credential type and ID; internal named credentials also include their current name. ## Test Plan - Run focused tests for `pkg/api_gateway/auth/api_key`. - Run focused tests for `pkg/api_gateway/auth/session`. - Run focused tests for `cmd/api_gateway/clinic`. - Run focused tests for `pkg/auth/api_key`. - Add a focused caller-log test using only Access-phase request PDK methods. - Run the relevant root-module test set when focused tests pass. ## Staging Verification The first authorized staging check showed that direct request/context PDK calls from the Go plugin's timer-based `Log` phase returned empty values. The second and third checks showed that Kong 3.1 restored native request outcome fields in `kong.log.serialize()`, but neither custom serialize values nor modified upstream request headers retained caller data across that boundary. The final design therefore keeps caller logging in `Access`. Deploy only the API Gateway feature image, make one authenticated API-key call, then query the path and narrow timestamp in system logs. Confirm trace ID, method, path without query parameters, and the API-key caller. Keep staging evidence and secrets outside the repo. ## Release Boundary ### Required deployment modules - Caller attribution: Clinic API Gateway build workflow, `api-gateway` image, app-delivery `clinic-kong`, API Gateway workload, target environment selected by the later release request. - Fallback-log redaction: Clinic CD `components=clinic`, `clinic` image, app-delivery `clinic-api`, Clinic Deployment, target environment selected by the later release request. No deployment is part of this implementation task. Merge and release require their normal separate authorization.