# datasource-proxy

A tiny Go reverse proxy that holds the real API credentials so an **agent never
has to**. The agent talks only to the proxy; the proxy injects the right auth
header and forwards to an allowlisted upstream. The token never leaves the process.

Upstreams (matched by path prefix, prefix stripped before forwarding):

| prefix | upstream | auth |
|---|---|---|
| `/devops/{prod,staging}` | `*.devops.pingcap.com` | `X-API-TOKEN` (falls back to `DEVOPS_API_KEY`) |
| `/o11y/ds/{region}/{vendor}/…` | `www.ds.{region}.{vendor}.observability.tidbcloud.com` | Bearer (auto-rotated from `GITHUB_PAT`) |
| `/o11y/gs` | `www.gs.us-west-2.aws.observability.tidbcloud.com` | Bearer (auto-rotated from `GITHUB_PAT`) |
| `/clinic/{prod,staging}` | `clinic.pingcap.com` | Bearer |
| `/jira` | `tidb.atlassian.net` | Basic |
| `/github` | `github.com` | Basic (`x-access-token`, git HTTP) |
| `/dashscope` | `dashscope-intl.aliyuncs.com` | Bearer |
| `/bedrock` | `bedrock-runtime.{region}.amazonaws.com` | Bearer |

The proxy handles only the **read-only** o11y query surface (metrics, logs,
cluster lookup — all `GET`, all authenticated with the auth0 token). Two o11y
flows are deliberately *not* proxied because they are mutating and/or non-HTTP:
**data export** (`monitoringPipelines`, `POST`/`DELETE`, uses `O11Y_TOKEN`) and
**S3 access** (needs AWS credentials). To run those without holding the
`GITHUB_PAT`, opt in with `O11Y_EXPOSE_CREDENTIALS=true` (and `O11Y_ENABLED`):
the proxy then serves **`GET /o11y/credentials`**, returning the direct-flow
credentials from the same `/client/auth` exchange —
`{o11y_token, access_key_id, secret_access_key, session_token}`. The caller uses
these against the real hosts directly. This is the one route that *returns*
credentials rather than injecting them, so it's off by default (reads as absent /
404 when off). The auth0 token (used for the proxied queries) and the PAT never
leave the process.

## Threat model

The service exists so a compromised agent **can't read a token and upload it**.
That rests on one assumption: **the agent can't read the secrets store or the
proxy's memory.** If the agent shares the proxy's UID (or is root / has
`docker`/`sudo` reach) it can just `cat secrets.json` or `ptrace` the proxy, and
the proxy is worthless. So pick one of the two forms below — never run the agent
and proxy as the same unconfined user.

## Build & configure

```sh
go build -o datasource-proxy .
cp secrets.example.json secrets.json && $EDITOR secrets.json   # git-ignored; never commit
```

Fill only the keys you use (an unset key makes that route error). Set
`O11Y_ENABLED: true` for o11y token rotation. Edits are picked up without a restart.
Secrets path resolves from `--secrets`, then `DS_PROXY_SECRETS`, then next to the
binary. Listen address is `--listen` (default `127.0.0.1:8990`).

## Option A — Agent in a sandbox (recommended)

Wall off the untrusted side: run the agent in a sandbox/container that can't read
the host filesystem and can only egress to the proxy. The proxy then runs plainly
on the host as your normal user — no dedicated account. Identical on macOS and
Linux, and the simplest secure option on macOS.

```sh
./datasource-proxy --listen 0.0.0.0:8990 --secrets ~/.config/datasource-proxy/secrets.json
```

Bind to the interface the sandbox reaches (a Docker/OrbStack/Colima agent uses
`host.docker.internal:8990`; loopback is fine if shared) — keep it off any
externally routable interface. The sandbox must enforce two things: **no read
access to the secrets path** (free with containers; `sandbox-exec` needs
`(deny file-read* (subpath "…"))`) and **egress only to the proxy**.

## Option B — Linux systemd (proxy-side isolation)

For when the agent runs on the host and can't be sandboxed. Isolate the *proxy*:
`DynamicUser` gives it a throwaway UID, and `LoadCredential` feeds it the secrets
so the agent's UID can never read them. Unit: [`deploy/datasource-proxy.service`](deploy/datasource-proxy.service).

```sh
sudo install -Dm600 secrets.json    /etc/datasource-proxy/secrets.json   # root:root 0600
sudo install -Dm755 datasource-proxy /usr/local/bin/datasource-proxy
sudo cp deploy/datasource-proxy.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now datasource-proxy

# verify the wall holds — must be denied:
sudo -u <agent_user> cat /etc/datasource-proxy/secrets.json
```

Optionally block ptrace too: `echo 'kernel.yama.ptrace_scope=1' | sudo tee /etc/sysctl.d/10-ptrace.conf && sudo sysctl --system`.

## Container image

`infra/images/datasource-proxy/Dockerfile` builds the same binary on
`debian:bookworm-slim` (port `8990`) — what CI ships to staging via ArgoCD. It runs
as root inside the container: fine under Kubernetes, but on bare metal prefer A or B.