# Clinic Local Agent Notes

## Querying Clinic System Logs

When checking Clinic control-plane logs, use the `clinic-api` skill system-log helpers with API-key auth. System logs are not scoped by a TiDB cluster ID, so do not use `/data-proxy/loki/{cluster_id}/...` for Clinic backend/control-plane logs.

### Load the helper and credentials safely

The helper is maintained in [tidbcloud/nutshell-skills](https://github.com/tidbcloud/nutshell-skills). By default, locate a `nutshell-skills` checkout next to the Clinic repository instead of assuming a developer-specific home or IDE workspace path:

```bash
CLINIC_REPO_ROOT="$(git rev-parse --show-toplevel)"
export NUTSHELL_SKILLS_ROOT="${NUTSHELL_SKILLS_ROOT:-$(dirname "$CLINIC_REPO_ROOT")/nutshell-skills}"
export CLINIC_API_SKILL_ROOT="${CLINIC_API_SKILL_ROOT:-$NUTSHELL_SKILLS_ROOT/skills/platform/clinic-api}"
test -f "$CLINIC_API_SKILL_ROOT/scripts/clinic-shell.sh"
source "$CLINIC_API_SKILL_ROOT/scripts/clinic-shell.sh"
```

This works with a portable sibling layout:

```text
workspace/
├── clinic/
└── nutshell-skills/
```

If `nutshell-skills` is checked out elsewhere, set `NUTSHELL_SKILLS_ROOT` or `CLINIC_API_SKILL_ROOT` explicitly. If it is missing, ask before cloning `https://github.com/tidbcloud/nutshell-skills`; do not automatically create directories or download repositories without authorization.

Do not paste a Clinic API key into chat, a command line, the repository, or a shell-traced session. Prefer an approved secret store.

Before reporting that Clinic API credentials are unavailable or falling back to an authenticated browser, check the explicit credential path and then the conventional environment-specific path. Production uses `$HOME/.config/clinic-api/clinic-api-prod.env`; staging uses `$HOME/.config/clinic-api/clinic-api-staging.env`. The `clinic-api` helper does not discover these files automatically because its default `ENV_FILE` is `.env`.

```bash
export CLINIC_ENV="${CLINIC_ENV:-prod}"
export CLINIC_API_SECRET_DIR="${CLINIC_API_SECRET_DIR:-$HOME/.config/clinic-api}"
export CLINIC_API_ENV_FILE="${CLINIC_API_ENV_FILE:-$CLINIC_API_SECRET_DIR/clinic-api-${CLINIC_ENV}.env}"

configure_clinic_api_env_file() {
  case "$CLINIC_ENV" in
    prod|staging) ;;
    *) return 1 ;;
  esac

  if [ ! -d "$CLINIC_API_SECRET_DIR" ] || [ -L "$CLINIC_API_SECRET_DIR" ] || \
     [ ! -f "$CLINIC_API_ENV_FILE" ] || [ -L "$CLINIC_API_ENV_FILE" ]; then
    return 1
  fi

  case "$(uname -s)" in
    Darwin)
      CLINIC_API_SECRET_DIR_MODE=$(stat -f '%Lp' "$CLINIC_API_SECRET_DIR")
      CLINIC_API_SECRET_DIR_OWNER=$(stat -f '%u' "$CLINIC_API_SECRET_DIR")
      CLINIC_API_ENV_MODE=$(stat -f '%Lp' "$CLINIC_API_ENV_FILE")
      CLINIC_API_ENV_OWNER=$(stat -f '%u' "$CLINIC_API_ENV_FILE")
      ;;
    *)
      CLINIC_API_SECRET_DIR_MODE=$(stat -c '%a' "$CLINIC_API_SECRET_DIR")
      CLINIC_API_SECRET_DIR_OWNER=$(stat -c '%u' "$CLINIC_API_SECRET_DIR")
      CLINIC_API_ENV_MODE=$(stat -c '%a' "$CLINIC_API_ENV_FILE")
      CLINIC_API_ENV_OWNER=$(stat -c '%u' "$CLINIC_API_ENV_FILE")
      ;;
  esac
  if [ "$CLINIC_API_SECRET_DIR_MODE" != "700" ] || [ "$CLINIC_API_SECRET_DIR_OWNER" != "$(id -u)" ] || \
     [ "$CLINIC_API_ENV_MODE" != "600" ] || [ "$CLINIC_API_ENV_OWNER" != "$(id -u)" ]; then
    return 1
  fi
  export ENV_FILE="$CLINIC_API_ENV_FILE"
}

configure_clinic_api_env_file
```

The final command must succeed before continuing. Check the exact path; do not rely on a glob such as `clinic-api.*`, which does not match `clinic-api-prod.env`. Never print, search, or inspect the file contents. If the directory or file does not exist, is a symbolic link, is not owned by the current user, or does not have mode `700`/`600` respectively, stop before sourcing it and ask the user to create or repair the credential file. Do not silently choose another environment's key.

Create the environment-specific file only when necessary. Use the same secret directory for both environments; do not copy a production key into the staging file.

Production:

```zsh
export CLINIC_ENV=prod
export CLINIC_API_SECRET_DIR="${CLINIC_API_SECRET_DIR:-$HOME/.config/clinic-api}"
mkdir -p "$CLINIC_API_SECRET_DIR"
chmod 700 "$CLINIC_API_SECRET_DIR"

umask 077
read -rs "CLINIC_API_KEY?Paste production Clinic API key: "
print
printf 'CLINIC_API_KEY=%q\n' "$CLINIC_API_KEY" \
  > "$CLINIC_API_SECRET_DIR/clinic-api-prod.env"
unset CLINIC_API_KEY
chmod 600 "$CLINIC_API_SECRET_DIR/clinic-api-prod.env"
```

Staging:

```zsh
export CLINIC_ENV=staging
export CLINIC_API_SECRET_DIR="${CLINIC_API_SECRET_DIR:-$HOME/.config/clinic-api}"
mkdir -p "$CLINIC_API_SECRET_DIR"
chmod 700 "$CLINIC_API_SECRET_DIR"

umask 077
read -rs "CLINIC_API_KEY?Paste staging Clinic API key: "
print
printf 'CLINIC_API_KEY=%q\n' "$CLINIC_API_KEY" \
  > "$CLINIC_API_SECRET_DIR/clinic-api-staging.env"
unset CLINIC_API_KEY
chmod 600 "$CLINIC_API_SECRET_DIR/clinic-api-staging.env"
```

Then select the matching environment before loading authentication:

```bash
export CLINIC_ENV=prod # or staging
export CLINIC_API_SECRET_DIR="${CLINIC_API_SECRET_DIR:-$HOME/.config/clinic-api}"
export CLINIC_API_ENV_FILE="$CLINIC_API_SECRET_DIR/clinic-api-${CLINIC_ENV}.env"
configure_clinic_api_env_file || exit 1
export ENV_FILE="$CLINIC_API_ENV_FILE"
source "$CLINIC_API_SKILL_ROOT/scripts/clinic-shell.sh"
get_clinic_auth

clinic_system_logs_datasources_table
clinic_system_logs_datasource_for_region aws us-west-2
```

Confirm that the directory exists, is owned by the current user, is mode `700`; that the matching environment file is owned by the current user and mode `600`; and that the datasource catalog request succeeds. Never print the file contents. Keep the environment-specific file only while it is intentionally reusable; remove it after the investigation when it is no longer needed.

### Resolve the datasource and time range

Use the canonical datasource UID returned by the catalog. If provider and region are known, resolve it explicitly with `clinic_system_logs_datasource_for_region <provider> <region>` and pass that UID to query helpers. In deployments that only configured the historical datasource name, the catalog exposes that stable name as the UID.

The helpers accept nanosecond Unix timestamps. Grafana Explore URLs normally encode `from` and `to` in milliseconds, so multiply those values by `1000000` before passing them to the helpers. Also convert the displayed local time to UTC explicitly and record both forms in incident notes to avoid timezone mistakes.

For a rolling 30-minute window, use the command appropriate for the local operating system:

```bash
# macOS
START=$(date -u -v-30M +%s)000000000
END=$(date -u +%s)000000000

# Linux/GNU date
# START=$(date -u -d '30 minutes ago' +%s)000000000
# END=$(date -u +%s)000000000

DATASOURCE_UID=$(clinic_system_logs_datasource_for_region aws us-west-2)
```

For an exact Grafana millisecond range:

```bash
GRAFANA_FROM_MS=1786402080000
GRAFANA_TO_MS=1786402799000
START="${GRAFANA_FROM_MS}000000"
END="${GRAFANA_TO_MS}000000"
```

### Discover streams before narrowing the query

Start with labels and series discovery. Do not assume the namespace, application, or container labels from a dashboard title:

```bash
clinic_system_logs_labels "$DATASOURCE_UID" "$START" "$END"
clinic_system_logs_series \
  '{cluster_env="prod",cluster="prod-clinic",app=~"clinic-gateway-prod-cloud-kong|clinic-data-service"}' \
  "$START" \
  "$END" \
  "$DATASOURCE_UID"
```

Useful production streams currently include:

- Kong proxy: `namespace="kong-prod-cloud"`, `app="clinic-gateway-prod-cloud-kong"`, `container="proxy"`.
- Kong ingress controller: the same `app`, with `container="ingress-controller"`.
- Clinic Data Service: `namespace="clinic-prod-cloud"`, `app="clinic-data-service"`, `container="data-service"`.

A component may have no logs in a narrow incident window. Expand the discovery range before concluding that the stream or component is absent.

### Query and correlate timeout failures

Use the same exact time window for gateway and application logs. Begin with boundary-specific queries, then add route, status, trace, or error filters:

```bash
clinic_system_logs_query_range \
  '{cluster_env="prod",cluster="prod-clinic",app="clinic-gateway-prod-cloud-kong",container="proxy"} |= `upstream timed out`' \
  "$START" "$END" 1000 backward "$DATASOURCE_UID"

clinic_system_logs_query_range \
  '{cluster_env="prod",cluster="prod-clinic",app="clinic-data-service",container="data-service"} |= `context canceled`' \
  "$START" "$END" 1000 backward "$DATASOURCE_UID"

clinic_system_logs_query_range \
  '{cluster_env="prod",cluster="prod-clinic",app="clinic-data-service",container="data-service"} |= `1m0.`' \
  "$START" "$END" 1000 backward "$DATASOURCE_UID"
```

Interpret the evidence by layer:

- Kong `upstream timed out ... while reading response header` plus application `context canceled` at the same boundary usually means the gateway canceled the inbound request. Because the application propagates the request context downstream, a client configured for 120 seconds can still stop at 60 seconds.
- Application-owned client timeouts more commonly surface as `context deadline exceeded` or `Client.Timeout exceeded while awaiting headers` without the gateway being the first boundary.
- Match timestamps, path, status, duration, and trace or request identifiers where available. Do not infer the failing layer from a single log line.
- A TCP error such as `EHOSTUNREACH` points to connectivity or a stale upstream target, not authentication, RBAC, or an application timeout.

For ingress-controller compatibility and synchronization checks:

```bash
clinic_system_logs_query_range \
  '{cluster_env="prod",cluster="prod-clinic",app="clinic-gateway-prod-cloud-kong",container="ingress-controller"} |~ `(?i)version|release|successfully synced|failed|invalid|config`' \
  "$START" "$END" 1000 backward "$DATASOURCE_UID"
```

Kong configuration support is version-dependent. In particular, the `konghq.com/connect-timeout`, `konghq.com/read-timeout`, and `konghq.com/write-timeout` Service annotations require Kong Ingress Controller 2.8 or later. Older 2.x controllers use a `KongIngress` `proxy` configuration referenced by the Service `konghq.com/override` annotation. A controller can log a successful sync while silently leaving unsupported annotations unapplied, so always verify the running controller version and generated Kong configuration.

### Verify desired state and live state separately

A merged source PR or correctly rendered Kustomize overlay does not prove that the running gateway uses the intended value. For timeout or routing incidents, verify the complete chain:

1. Confirm the application source default and the deployed image tag or commit.
2. Render the target app-delivery overlay and confirm the intended Service or custom-resource configuration.
3. Inspect the live Service, relevant custom resources, and EndpointSlices when approved Kubernetes access is available.
4. Inspect the running ingress-controller version and its sync logs.
5. Inspect Kong's generated/Admin API route, service, upstream, and timeout values when access is available.
6. Reproduce a request that crosses the old timeout boundary and confirm behavior in both Kong and application logs.

If the ingress controller reports `declarative config is invalid` or an Admin `/config` request returns `400`, Kong may retain its last valid configuration. After a Pod change, that can leave an obsolete upstream IP active even when the Kubernetes Service and EndpointSlice are correct. Compare the current Pod IP, Service endpoints, and Kong upstream target; fix the schema or configuration mismatch and confirm a successful sync before considering a restart.

### Attribute `clinic_access` callers

The Clinic Kong plugin emits `clinic_access` after authentication. Use its
`trace_id` to correlate the caller with Kong's native access log, which remains
the source for status and duration. The caller fields are:

- `auth_method=ui_session`: look up `user_id` in `users.id`.
- `auth_method=api_key`: use `credential_type` to select the credential table,
  then look up the row by both `credential_id` and `user_id`:
  - `internal` -> `internal_api_credentials`
  - `api` -> `api_credentials`
  - `upload` -> `credentials`
- `auth_method=anonymous`: no authenticated identity is available.

Credential IDs are only stable within their credential table, so never use a
bare `credential_id` without `credential_type`. For example, locate an internal
API caller with a read-only query:

```sql
SELECT
    u.id,
    u.user_name,
    u.email,
    c.id AS credential_id,
    c.name AS credential_name,
    c.status,
    c.last_used_at
FROM internal_api_credentials AS c
JOIN users AS u ON u.id = c.user_id
WHERE c.id = ? AND c.user_id = ?;
```

Treat the database result as sensitive. Report the caller name only when it is
needed and authorized; otherwise summarize that the IDs resolved successfully.
Never include credential hashes, plain keys, authorization values, session
cookies, or unrelated user data. An empty `credential_name` is valid; the
credential ID remains the stable attribution field.

### Keep query output reviewable

System-log queries can return up to 1000 entries and may contain sensitive request data. Save large results only to permission-restricted temporary files, summarize repeated lines by path/status/error with counts and first/last timestamps, and include only representative redacted samples in reports. Never include API keys, authorization headers, cookies, session IDs, CSRF values, customer identifiers, or unrelated production payloads.

Example error query:

```bash
clinic_system_logs_query_range \
  '{cluster_env="prod",cluster="prod-clinic",app=~"clinic|clinic-data-service"} |~ `(?i)\[error\]|panic|fatal`' \
  "$START" \
  "$END" \
  1000 \
  backward \
  "$DATASOURCE_UID"
```

Important boundaries:

- Use the correct environment and key: `CLINIC_ENV=dev|staging|prod`; prod keys do not work against staging.
- Discover UIDs with `GET /data-proxy/api/v1/system-log-datasources`; query through `/data-proxy/api/v1/system-log-datasources/{datasourceUID}/logs/...`.
- Put the datasource UID in the path. Do not add the legacy `datasource`, `provider`, or `region` routing query parameters to canonical requests.
- Do not add a `cluster_id`; control-plane logs do not have one.
- Keep LogQL regex filters in backticks, for example ``|~ `(?i)\[error\]` ``.
- Do not paste huge raw log dumps into the answer; group and summarize repeated patterns and include only representative redacted samples.

## Clinic Backend Development, Testing, and Release

Use this workflow for releases with `components=clinic`. Check the actual workflow before applying it to another component.

### Plan every change

Before modifying Clinic source code, configuration, deployment behavior, or documentation, create a written implementation plan under `docs/plans/`.

- Name the file `YYYY-MM-DD-short-kebab-case-topic.md`, for example `2026-05-07-dedicated-components-details.md`.
- Base the date on the day the plan is created. Use a concise topic that identifies the behavior or component being changed.
- Describe the background, goal, current behavior, target contract, affected modules and routes, implementation steps, exclusions, compatibility concerns, test plan, staging verification, and release boundary as applicable.
- Record relevant app-delivery changes and environment overlays when the change affects build or deployment behavior.
- Update the plan when implementation discoveries materially change the agreed design; do not leave the plan describing behavior that the code no longer implements.
- Never include credentials, tokens, customer data, or unredacted production evidence in a plan.

Read [the Clinic module and deployment overview](docs/clinic-module-overview.md) while planning. It maps source modules and executable entrypoints to images, GitHub Actions workflows, and app-delivery applications. Update that document in the same PR whenever module ownership, entrypoints, image names, workflows, or app-delivery mappings change.

### Keep the change small

- Start from the latest `master` unless the user explicitly asks for another base.
- Keep the PR limited to the requested behavior and its tests.
- Add a focused regression test for the behavior being fixed.
- Run the smallest relevant Go test first. Broaden the test scope only when it adds useful confidence.
- Use the Go version from `go.mod`. Local private-module access may require GitHub credentials and `GOPRIVATE`; do not commit temporary `go.mod` or `replace` changes made only to run local tests.

### API observability

- New or materially changed customer-facing API paths must emit structured lifecycle logs for request validation, authorization or feature gates, upstream outcomes, and successful completion where those events are meaningful.
- Every request-scoped log emitted by those paths must include a `trace_id` field derived from the request context. Use `pkg/log.WithContextL` and explicitly retain the `trace_id` field when a log helper adds other fields.
- Log only safe diagnostics such as operation name, status class, bounded sizes, counts, and necessary internal identifiers. Never log authorization headers, API keys, sessions, CSRF values, signed URLs, opaque page tokens, or full request/response payloads.

### Change impact review

Before every commit or pull-request handoff, perform and report an explicit
change-impact review. Do not rely only on the requested behavior: inspect the
final diff and the relevant callers, routes, authorization boundaries,
configuration, generated artifacts, and deployment consumers to identify
possible effects on adjacent logic.

- State the intended affected behavior, compatibility contract, and explicit non-goals.
- Identify any expected downstream impact and the regression checks needed when it is deployed.
- Preserve scope: do not broaden a change merely because a nearby component could be changed.
- If no unintended impact is expected, say so only with the evidence reviewed and validation actually run; otherwise state the uncertainty and required follow-up.

### Determine and report deployment impact

After implementation and relevant local validation, identify and report both the source impact and the deployment impact before handing the change back, creating or updating a PR, or starting a staging release.

Do not infer deployment scope from a changed directory, package name, API route, Kubernetes Service name, or conversational module name alone. Trace each runtime change through the complete ownership chain documented in [the Clinic module and deployment overview](docs/clinic-module-overview.md) and verify it against the current build target, GitHub Actions workflow, and app-delivery manifests:

```text
changed source
  -> owning Go module or non-Go tool
  -> executable entrypoint
  -> container image or artifact
  -> Clinic CD component input
  -> app-delivery application
  -> Kubernetes workload
```

For example, a request sent to a Service named `clinic-internal-api` is not enough evidence by itself. Confirm which executable and image back that Service before selecting a deployment component.

The implementation handoff and PR description must contain two explicit lists:

- **Affected source modules:** the changed Go modules or non-Go tools, important packages, executable entrypoints, and produced images.
- **Required deployment modules:** the exact Clinic CD `components` values, app-delivery applications, Kubernetes workloads, and target environments that must be deployed for the runtime change to take effect.

List every required deployable component when a change crosses image boundaries, even if all changes are in one Git repository or one user-visible request path. Also state which nearby components do not require deployment when the distinction is not obvious.

For documentation-only, test-only, or local development-tooling-only changes, explicitly report `Required deployment modules: none` after verifying that no runtime artifact, workflow input, or desired-state file changed.

Use the verified deployment list when selecting staging and production workflow inputs. Do not substitute `components=clinic` for another component merely because this section is part of the Clinic backend workflow; check the actual workflow before dispatch.

### Commit and pull request handoff

After implementation and relevant validation are complete, proactively ask the user whether to commit the changes, push the branch, and create a pull request if those actions have not already been explicitly authorized. A request to modify or test code does not by itself authorize publishing it to GitHub.

If the user already explicitly requested a commit, push, or PR as part of the active task, do not ask for redundant confirmation. Perform only the authorized actions and preserve the review and merge boundaries below.

Pull request titles and descriptions must be in English. The description should make the change understandable without relying on chat history and include:

- the problem and target outcome;
- the implementation approach and principal code or configuration changes;
- affected modules, APIs, compatibility behavior, and explicit non-goals;
- tests and validation that actually ran, with their results;
- staging or live-state checks that ran, and checks that were skipped with reasons;
- app-delivery or deployment impact and any follow-up authorization still required.

Do not claim that a PR was deployed, merged, or verified when only the local change, source PR, image build, or downstream workflow dispatch completed.

### Test in staging before merging

A Clinic backend PR does not need to be merged before it can be tested in staging.

1. Run the focused local test and record the command and result.
2. Trigger the GitHub Actions workflow **Clinic CD** from the feature branch with:
   - environment: `staging`
   - components: `clinic`
3. Clinic CD dispatches the app-delivery bump workflow. That workflow creates a staging PR and normally enables bot approval and auto-merge.
4. Wait for the app-delivery PR to merge and for the GitOps rollout. A green Clinic CD run means the image was pushed and the downstream workflow was triggered; it does not mean the staging rollout has finished.
5. Choose staging checks that match the change:
   - Verify the directly affected API, asynchronous job, or other observable behavior.
   - Check the UI only when the behavior is visible there. A screenshot is optional evidence.
   - If approved AWS access is available, check that the `clinic` Pod is Ready, has no unexpected restarts, and runs the expected image tag.
   - Check Clinic control-plane logs when log access is available and the logs provide a useful signal. Use the system-log rules already documented in this file; these logs are not scoped by a TiDB cluster ID.
6. If a check fails, investigate and stop the release. Do not bypass the GitOps path, required checks, or review to force it through.
7. Update the Clinic PR with the results that were actually observed. For any relevant check that could not be run, record the reason.

Do not require a screenshot for an API-only or backend-only change that has no UI surface. Use focused test results, the relevant API response, or logs when they are the better evidence. Keep screenshots outside the Clinic repository unless the user explicitly asks to commit them. They can be attached to a PR, a document, or another external record.

Before attaching screenshots or API/UI evidence, remove secrets, request headers, authentication-bearing URLs or query parameters, personal data, and unrelated production data. Redact tenant or customer identifiers unless they are necessary to identify the tested object and the reviewer is authorized to see them. Use only evidence that the reviewer is authorized to access.

### Staging environment and access

Clinic has one staging EKS cluster:

- AWS account: `552185537300`
- AWS region: `us-west-2`
- EKS cluster: `clinic-staging`
- Kubernetes namespace: `clinic-staging`

The AWS account ID is an infrastructure identifier, not a credential, and may be documented in this private repository. Treat the cluster name, region, and namespace as internal operational metadata; do not copy them to public issues or external documents without approval. Never write access keys, session tokens, Clinic API keys, or other secret values into the repository or test output.

Access to the staging AWS account must be requested separately. Do not assume the developer or agent already has it. Routine deployments must use Clinic CD and app-delivery GitOps. Request read-only EKS access for rollout inspection. Request write access only for a separately approved task that cannot use the normal GitOps path.

Local AWS access is useful but not always required:

- GitHub Actions builds and publishes the image with its own configured role.
- The generated app-delivery PR drives the GitOps deployment.
- AWS access is needed when directly checking the EKS Deployment, Pod image, readiness, or restart count.
- Functional verification can still use the staging Clinic API and UI when AWS access is unavailable.

### Review and merge boundary

- Do not merge the Clinic PR until repository-required checks and reviews pass and the user explicitly authorizes the merge. Do not bypass branch protection or required review.
- A successful feature-branch staging test is evidence for review; it is not approval to merge.

### Release after the Clinic PR is merged

Before production, prefer validating the final `master` image in staging. Require this when code changed after the feature-image test, merge conflict resolution changed the content, or the release policy requires validation of the exact release image.

For production:

1. Get explicit authorization for a production release. Confirm whether it covers only dispatching Clinic CD or also merging the generated app-delivery PR. If merge authorization is not included, stop before merging that PR.
2. Record the intended `master` SHA before dispatching CD. After dispatch, verify the workflow run head SHA and image tag; stop if they are not the intended release revision.
3. Trigger **Clinic CD** with:
   - environment: `prod-cloud`
   - components: `clinic`
4. Clinic CD dispatches the app-delivery bump workflow. It creates the `prod-cloud` PR and updates its FAD/check state, but it does not run the GitHub auto-approve step or enable GitHub auto-merge.
5. Inspect the generated app-delivery PR. It should only update the intended Clinic image tag.
6. Wait for all required checks, obtain the required review and explicit merge authorization, and merge using a repository-supported merge method.
7. Wait for GitOps sync and verify the real production behavior.

For work governed by this repository, production release scope is `prod-cloud` only. Do not select `prod-op`, even though the workflow currently exposes it as an option. Treat a `prod-op` request as out of scope and ask its owner for the correct process.

Production AWS access is not a hard requirement for every functional check. If a known production Clinic page or API provides direct evidence, use it to verify the behavior and record that the Pod and cluster runtime were not inspected. Do not use another person's authenticated session to bypass access controls, do not record authentication-bearing URLs, and do not claim a Pod or log check that was not performed.

### Credentials and permissions

| Task | Access needed | Notes |
| --- | --- | --- |
| Read and push Clinic code | GitHub repository access | Push only the task branch. |
| Download private Go modules | GitHub access to the private modules | Keep tokens out of commands, logs, and committed files. |
| Trigger Clinic CD | GitHub Actions permission on Clinic | Select the exact environment and component. |
| Inspect app-delivery PRs | app-delivery read permission | Confirm that only the intended image tag changes. |
| Merge an app-delivery PR | app-delivery write/merge permission | Use only with explicit merge authorization and after normal checks and review. |
| Call staging Clinic APIs | Staging Clinic API key | Keys are environment-specific. |
| Call production Clinic APIs | Production Clinic API key | Check the HTTP status before relying on the response. |
| Inspect staging EKS | Approved AWS access to account `552185537300` | Access must be requested; never assume it exists. |

Prefer an approved secret store or credential helper. If credentials must be loaded by the shell, read them from an untracked, permission-restricted file. Never put secret values in commands, shell history, logs, PRs, screenshots, or files loaded with shell tracing such as `set -x` or `zsh -x`. Document environment variable names only, never their values. A `401` means authentication failed for that request; verify the environment, key, and request method before continuing.

### What to report at the end

Report what actually ran. If a relevant check was skipped, state which check and why:

- affected source modules, executables, and images;
- required deployment modules, exact Clinic CD component inputs, app-delivery applications, workloads, and target environments, or explicitly `none`;
- focused local test command and result;
- Clinic PR and merge commit;
- Clinic CD run and image tag;
- app-delivery PR and merge state;
- staging and production behavior observed;
- Pod/image/restart status, only when EKS was inspected;
- system-log result, only when log access worked;
- in-scope environments intentionally not released.