# PROJECT KNOWLEDGE BASE

**Generated:** 2026-03-09 10:09:18 CST
**Commit:** 443ed4c
**Branch:** mcp-server

## OVERVIEW
Python FastMCP server for TiDB observability. Main jobs: forward GitHub auth to backend services, resolve cluster-specific Prometheus endpoints, run PromQL tools, and read Delta Lake tables through DuckDB.

## STRUCTURE
```text
./
├── server.py                # stdio entrypoint; imports tools for registration side effects
├── pyproject.toml           # setuptools, pytest, ruff, optional mypy settings
├── docs/FEATURES_AND_FLOW.md# Canonical flow notes; known mismatches and follow-ups
├── src/tools/               # MCP-facing tool handlers
├── src/services/            # auth, resolve, Prometheus, DuckDB orchestration
├── src/utils/               # caches, stores, config, errors, response helpers
├── src/models/              # Pydantic request/response schemas
└── test/                    # executable spec; no conftest.py yet
```

## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| Start server / registration | `server.py`, `src/tools/query_range.py`, `src/tools/__init__.py` | `mcp` lives in `query_range.py`; import order matters |
| Add or change an MCP tool | `src/tools/` | All tools register on the shared `mcp` instance |
| Change auth flow | `src/services/auth.py` | Unified auth module: per-provider auth cache, server bootstrap, cloud creds/tokens, GS URL store |
| Change Prometheus resolution | `src/services/observability_resolve.py`, `src/services/prometheus.py` | Global-service resolve first, then backend query; `get_prometheus_client_for_cluster` lives in `prometheus.py` |
| Change Delta Lake reads | `src/tools/read_delta.py`, `src/services/duckdb_delta.py`, `src/services/gcp_parquet.py` | AWS/Alicloud use `delta_scan`; GCP uses GCS parquet download + local DuckDB query |
| Change in-memory state | `src/services/auth.py`, `src/utils/cloud_access_store.py` | Per-provider auth cache in `auth.py`; per-cluster metrics URL cache in `cloud_access_store.py` |
| Change validation schema | `src/models/` | `Vendor` is defined in `src/models/read_delta.py` and reused elsewhere |
| Check expected behavior first | `test/` | Tests are the most reliable contract for cache, auth, and DuckDB behavior |

## CODE MAP
| Symbol / Module | Type | Location | Role |
|-----------------|------|----------|------|
| `mcp` | FastMCP instance | `src/tools/query_range.py` | Shared server object for all tool decorators |
| `query_range` | tool | `src/tools/query_range.py` | Range PromQL query with redaction |
| `query_instant` | tool | `src/tools/query_instant.py` | Instant PromQL query |
| `list_metrics` | tool | `src/tools/list_metrics.py` | Metric discovery with optional filtering |
| `read_delta_tool` | tool | `src/tools/read_delta.py` | Delta reader via DuckDB + cloud creds; auto-resolves vendor/region from GS using env/deploy_type |
| `get_o11y_delta_path` | tool | `src/tools/get_o11y_delta_path.py` | Resolve Observability Delta Lake table paths using env/deploy_type (slowlog always; sql_statement only for premium) |
| `get_cluster_info` | service fn | `src/services/observability_resolve.py` | Resolve cluster metadata (vendor, region, o11y_id, tenantID) from GS API |
| `PrometheusClient` | service class | `src/services/prometheus.py` | HTTP client + truncation rules |
| `get_prometheus_client_for_cluster` | service fn | `src/services/prometheus.py` | Resolve cluster metrics URL and return configured client |
| `bootstrap_server_auth` / `get_valid_server_auth` | service fn | `src/services/auth.py` | Server-owned auth bootstrap and cached refresh by provider |
| `get_server_access_token` / `get_server_auth0_token` | service fn | `src/services/auth.py` | Async auto-refreshing token getters (always return valid token) |
| `get_delta_credentials` | service fn | `src/services/auth.py` | S3-compatible credential resolution (AWS/Alicloud) from provider-scoped cached AuthResponse |
| `read_gcp_delta_parquet` | service fn | `src/services/gcp_parquet.py` | Date-partition scoped GCS parquet download/cache with OAuth token, then local DuckDB query (deduplicated view) |
| `get_observability_global_service_url` | service fn | `src/services/auth.py` | Dynamic GS URL read from cached AuthResponse |
| `get_or_resolve_metrics_url` | service fn | `src/services/observability_resolve.py` | Cluster metrics URL cache + resolve |
| `read_delta` | service fn | `src/services/duckdb_delta.py` | Safe SQL + secret setup + Delta extension |
| `tool_text_response` / `tool_json_response` | helpers | `src/utils/tool_response.py` | Canonical MCP return shape |

## CONVENTIONS
- Python project, not TS/Node: `pyproject.toml` is source of truth for tooling.
- Shared tool registration pattern: import the tool module, do not create a second `FastMCP` instance.
- Auth flow is server-owned and cached: startup GitHub token -> JWT -> observability global service URL.
- Delta tools (`read_delta_tool`, `get_o11y_delta_path`) auto-resolve vendor/region from GS using `env`, `deploy_type`, and `cluster_id`.
- Prometheus tools require the cluster tuple: `vendor`, `region`, `tenant`, `cluster_id`.
- In-memory stores are the persistence layer here. No file/database backing.
- Tests patch where a dependency is imported, not where it originated.
- `pytest` uses `asyncio_mode = "auto"`; async tests still often use `@pytest.mark.asyncio` locally.

## ANTI-PATTERNS (THIS PROJECT)
- Do not reintroduce caller-supplied GitHub/JWT token flow for MCP tools; auth-backed Prometheus resolution now depends on server-owned startup auth.
- Do not create a new MCP instance in another tool file; import the shared `mcp` from `src/tools/query_range.py`.
- Do not call auth/S3 APIs on cache hit; the auth cache drops expired responses and only re-fetches on miss.
- Do not cache auth responses without a computable expiry; the auth cache explicitly drops them.
- Do not document or edit `.venv/` or `src/*.egg-info` as source; both are generated artifacts.
- Do not assume `test/test_client.py` matches `server.py` runtime directly; it targets `http://localhost:8000/mcp` while `server.py` runs stdio.
- Do not reintroduce separate auth utility files (`token_cache.py`, `s3_access_store.py`, `observability_service_store.py`, `auth.py`); all auth is unified in `src/services/auth.py`.

## UNIQUE STYLES
- Tool outputs are wrapped as MCP text payloads even when content is JSON.
- Prometheus results are redacted before returning to clients.
- Delta queries accept only `SELECT ... FROM __t__ ...`; SQL is intentionally constrained.
- For GCP `read_delta`, include exactly one `date='yyyymmdd'` predicate in SQL.
- In GCP flow, the date predicate is used for partition download scope and is stripped before local parquet query execution.
- GCP responses include `local_cache_path`; parquet is cached locally by `(bucket, prefix, date)` and reused across calls.
- Docs and tests, not code comments alone, record current known mismatches (`docs/FEATURES_AND_FLOW.md`).

## COMMANDS
```bash
pip install -e ".[dev]"
python server.py
ruff check src/
ruff format src/
pytest
```

## NOTES
- `fastmcp run server.py:mcp` is a standard FastMCP path externally, but this repo documents `python server.py` and relies on the `__main__` block.
- README examples lag the current required params in some places; trust models/tests over prose when they disagree.
- LSP for Python is currently unavailable in this environment because Ruff server needs `--preview`; use tests and file reads as the codemap source.