# Telemetry What this server records about its own behavior, what it will never record, where it goes, and how long it is kept. The short version: **a self-hosted install sends nothing to us, ever.** Recording is local JSONL on your own disk unless you explicitly point it at a database, and it can be turned off entirely. ## What is recorded One event per tool call, plus two counters. Every timestamp is UTC. | Field | Meaning | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ts` | UTC ISO timestamp. Conversion to local time happens at display, never at record time. | | `tenantFingerprint` | SHA-256 of the API key pair. The key itself is never stored, logged, or transmitted. | | `orgId`, `orgUnresolved` | The TiDB Cloud organization, resolved once per fingerprint. Analytics group by org so rotating a key does not split one customer into two tenants. | | `tool` | Tool name, e.g. `tidb_query`. | | `outcome` | `ok`, `error`, or `denied`. | | `errorCode` | A short machine-readable code such as `NotFoundError`. Never the error message, which can quote data. | | `serverMs`, `upstreamMs` | Total time, and the part of it spent waiting on TiDB Cloud (management API plus SQL). Kept separate because upstream slowness is not something we can fix. | | `clientName`, `clientVersion`, `clientInfoSource` | Which MCP client made the call, and which protocol path supplied it. `initialize` works on `stdio`, where one server instance serves the whole session. On the stateless HTTP endpoint the 2026-07-28 `_meta` envelope is the source, since it rides every request; a caller that supplies none is recorded as `unknown` rather than guessed at. It is never read from the `Mcp-Name` header, which names the tool being CALLED rather than the caller. | | `transport` | `http` or `stdio`. | | `protocolEra`, `protocolVersion` | Which MCP generation served the request (`legacy` or `modern`) and the revision the client declared, e.g. `2025-06-18`. Recorded because the endpoint answers both and used to keep neither, so "which revision does this client send" had to be inferred. It is also what decides when the 2025 era can be dropped, which is a question about traffic rather than about the calendar. Null when the caller declared none; never defaulted to a guess. | | `modeFlags` | The permission mode in effect: `readonly`, `write`, `admin`, `templates-only`. | | `argsSummary` | Allowlisted argument fields only. See below. | | `requestId` | Correlation id when the caller supplied one. | Two product metrics sit alongside the per-call record, and they are built differently on purpose: - `denied_call` is its own event kind: a client called a tool this mode did not expose. A gated tool is never registered, so the dispatch hook cannot see the attempt; the remote detects it from the request instead. This is demand data for the write tier. - **Unconfirmed preflights are DERIVED, not recorded.** Each call records whether it was a preview (`argsSummary.preflight = issued`) or an execution (`argsSummary.confirmed = true`), and `preventionCounters` pairs a preview against a later confirm on the same tenant, tool, and target within a window (15 minutes by default). A preview with no answer is an incident the confirm gate prevented. Why derived: recording "a preflight expired" would need server-side tokens and timers, and this server is deliberately stateless. On more than one machine, a preflight served by A and confirmed by B would also be miscounted as prevented. Pairing after the fact avoids both, and leaves the window tunable instead of baked into a timer. ## What is never recorded Argument capture is an **allowlist with a default of DROP**: a field that is not explicitly named in the policy never enters an event. The policy lives in one module (`packages/core/src/telemetry/redaction.ts`), not in per-call-site judgment. A test poisons every argument name the server accepts with a credential and with row data, across a representative tool per allowlist class plus an unregistered tool and a template tool, and asserts neither reaches an event. Never recorded, for any tool, under any configuration: - **SQL text and query parameters.** The `sql` and `params` fields are never allowlisted for any tool, and template tool arguments are never recorded at all, since they are user data by definition. - **Result rows.** Nothing a query returned is ever part of an event. - **Credentials.** API keys, SQL passwords, connection strings. The key appears only as a SHA-256 fingerprint. Allowlisted fields are additionally scrubbed of known secrets, so even an id field handed a key records nothing. - **Free-text the caller invents.** Cluster `displayName`, backup names, descriptions, labels, and the docs-search `query` string, because a user can paste anything into them. What IS allowlisted is identifiers, enums, and short scalars: cluster and branch and backup ids, project id, region, service plan, database and table names, SQL user name and role, the restore `pointInTime`, the docs URL (which the SSRF guard already restricts to `docs.pingcap.com`), and the `confirmTarget` of a destructive operation. Values are capped at 120 characters, scrubbed of known credentials, and reduced to `[object]` or `[array]` if not a primitive. The `confirmed` and `preflight` markers are derived server-side, never copied from a caller value. **One honest limit.** The allowlist filters field NAMES, not content. If a caller puts unexpected text in an allowlisted slot, for example passing a whole SQL statement as `database`, that text is recorded (capped and credential-scrubbed) because the field is on the list. Known credentials are still removed, so this is about arbitrary caller data rather than secrets, and it is the one route by which SQL-looking text can reach an event. Every field a caller is expected to fill with free text is off the list precisely to keep that route narrow. Over-recording would turn the telemetry store into a PII leak and make the claim above false, which is why the default is to drop. ## Where events go | Condition | Store | Notes | | -------------------------------------------------- | ----------- | ----------------------------------------------------------------------------------- | | `--no-telemetry` (or `TIDB_MCP_NO_TELEMETRY=true`) | none | Nothing is recorded, no file is written. | | `TELEMETRY_DSN` is set | TiDB | Batched inserts. This is how our hosted endpoint dogfoods TiDB. | | stdio, no DSN | local JSONL | Default for local use. | | http, no DSN | local JSONL | Same, plus a startup warning that a shared endpoint probably wants a durable store. | The local file defaults to `telemetry/events.jsonl` (override with `--telemetry-path` or `TIDB_MCP_TELEMETRY_PATH`) and rotates by size, keeping the rotated siblings alongside it. ### Schema changes on an existing table The TiDB store applies `TELEMETRY_SCHEMA_SQL` on every start, and that list is `CREATE TABLE IF NOT EXISTS` **plus** `ALTER TABLE ... ADD COLUMN IF NOT EXISTS`. Both halves are needed: a CREATE does nothing to a table that already exists, so a column added to it would reach new deployments only, and the next insert would name a column that is not there. MySQL rejects the whole statement and the recorder does not retry, so the batch is lost rather than degraded, and it is the long-running deployment (the one with history worth querying) that would silently record nothing. Every ALTER carries `IF NOT EXISTS` because it runs on every start. Do not add a bare `ADD COLUMN`: it succeeds once and fails on every later boot, which is worse than the gap it closes. **If you pre-provision the schema yourself** (an operator-created table, or `skipSchema`), apply the statements in `TELEMETRY_SCHEMA_SQL` in order after upgrading, not just the CREATEs. The column list in a CREATE is the current shape; the ALTERs are how a table reaches it from an older one. ### The self-hosted guarantee A self-hosted install **phones home to nobody**. This is mechanical, not a promise: the local store (`FileStore`) contains no network client at all, and a test asserts that its source imports no HTTP machinery. Events reach a database only if you set `TELEMETRY_DSN` yourself, pointing at your own cluster. ## Telemetry never blocks or fails a request Recording is fire-and-forget through a bounded in-process queue. Enqueue is synchronous and swallows every error, so a store that hangs forever or throws on every write cannot slow down or fail a tool call. Tests prove all three cases. It is O(1) in the normal case; once the queue is full it scans for the oldest droppable event, which is O(n) in the queue length and still far below any latency that matters. Every string that reaches a fixed-width column is clamped to that width, at both the recording facade and the store. Several of those fields are client-controlled, and a batch insert is a single statement with no retry, so an uncapped value could otherwise reject a whole batch of events. When the queue is full, the oldest **non-security** events are dropped and a visible drop counter is incremented (surfaced as a `telemetry_issue` log line). Security events are never dropped and never sampled: an executed confirm, any admin-group call, and every denial. Those are the evidence that a human approved a mutation, so where the two rules collide, a security event is admitted above the soft cap and an immediate drain is triggered rather than blocking the caller. That exemption is bounded. Past an absolute ceiling (ten times the soft cap) even a security event is dropped and reported as `security_dropped`, because unbounded memory growth is a worse failure than lost telemetry. A non-zero `securityDropped` means the store has been unavailable long enough to lose audit evidence, and is worth alerting on. Queued events are flushed on graceful shutdown, bounded by a timeout: losing queued telemetry is acceptable, refusing to exit is not. ## Retention Detail events are kept for 30 days by default (`--telemetry-path`'s sibling setting `telemetryRetentionDays`), then rolled up and pruned. The rollup (calls, errors, denials, and latency totals per day per tenant per tool) is preserved, so long-range adoption and retention questions stay answerable after the detail window closes. Raw logs kept forever are a liability, not an asset. Rolling up happens BEFORE deleting, and a store that cannot persist a rollup is not pruned at all, because destroying detail while keeping no summary is worse than keeping the detail. For the TiDB store both steps run in one transaction, writing `telemetry_daily` before deleting from `telemetry_events`; the local store writes `.daily.jsonl` beside the detail. The sweep runs on a daily interval in the long-lived hosted server. It deliberately does not run at startup, because a crash-looping process would otherwise prune on every boot, and pruning is the one telemetry operation that destroys data. A short-lived `stdio` session does not prune on its own, so a self-hosted operator keeping telemetry for a long time should run the sweep themselves or rotate the files. ## Reading at scale Aggregation streams. Every aggregate folds over a paged `scan()` rather than materializing the range, so memory is proportional to the number of groups, not the number of rows. This matters because the hosted endpoint runs on a small machine that is also serving requests, and a retention window's worth of events on the heap would arrive exactly when volume makes the numbers worth reading. A direct `read()` is bounded and REFUSES past its limit rather than returning a truncated page, since a silently partial read would make every aggregate quietly wrong. Every reader that can legitimately span a large range streams instead, including audit export: use `auditExportTo` to write JSONL to a sink line by line. `auditExport` returns the whole trail as one string and is the convenience form for small ranges. Timestamps and ids are read as text (`CAST(ts AS CHAR)`, `CAST(id AS CHAR)`), never as driver- converted values. Two reasons, both verified against a deliberately misconfigured connection: a naive `DATETIME` converted client-side can shift across the UTC day boundary and corrupt every bucket, and an `AUTO_RANDOM` id read as a JS number loses precision, which made the pagination cursor both repeat and skip rows. See [spikes/telemetry-live-sql.md](spikes/telemetry-live-sql.md). ## Audit export Given a tenant fingerprint or an org id and a time range, the server can emit that tenant's full event list as JSONL, oldest first, streamed so a heavy tenant over a full retention window does not have to fit in memory. Exporting by org id is not a plain `orgId = ?` filter. Org resolution is a cache fill, so an event recorded before the org was known carries a null org while still belonging to that tenant, and a naive filter would drop exactly the onboarding window an investigation cares about. An org export therefore widens through the fingerprints that ever resolved to that org and unions in their null-org events, using the stored events as its own source of truth. On the hosted path the org is also resolved once per key during key validation, so in practice those null-org events do not occur. This answers the only question an enterprise evaluator actually asks: _what did your server do with our key._ It is also our own defense, because the preflight-then-confirm pair in the timeline is the evidence that a human approved each mutation. An export with no subject is refused rather than dumping everything. ## Two stances worth stating plainly **We do not auto-retry mutating calls.** Automatic retry of a write is normally sold as guaranteed delivery. For this server it would manufacture the exact incident the confirm gate exists to prevent: a mutation running twice, or running after the human's intent has passed. A failed mutating call is reported, not retried. Read calls are safe to retry and the client may do so. **PII controls belong in the data layer, not in a proxy.** Rather than pattern-matching PII on its way through the server, access is constrained by SQL grant: every data operation runs as a short-lived least-privilege user, and a read tool physically cannot write. That is enforcement, not detection, and it does not depend on a regex recognizing a new field name. Telemetry inherits the same principle by not recording result data at all. ## Circular dependency, on purpose avoided Our hosted endpoint stores telemetry in TiDB, which means TiDB is both the thing being monitored and the place the monitoring lives. Health checks and alerting must therefore not read through the telemetry store: if TiDB Cloud has an outage, the service and its monitoring would go dark together. A store outage degrades to dropped telemetry and a logged counter, never to a failed request.