# Query Gateway

Date: 2026-03-16
Status: Frozen architecture note
Authors: Search Platform Architecture

Component: Query Plane (Read Path)

Scale target: 1B collections | 5 EB raw vector + metadata

## 1. Executive Summary

The Query Gateway is the stateless edge of the Search Fabric query plane. It owns the shared public `QueryGatewayService` surface for canonical query clients, including Elasticsearch-compatible and Turbopuffer-compatible read routes on the native-search compatibility edge, and it also owns `RelationalQueryService` as the trusted relational query surface for projected-query and primary-query execution that needs relational-engine-private query context. Admission to this surface is driven by trusted relational caller capability inventory; TiDB is the current deployment mapping. For external traffic it consumes authenticated caller/scope context from `Unified Edge/Auth Ingress`; for direct internal callers it consumes trusted service identity. It resolves route names into authoritative `CollectionRef` when name-based routing is used, lowers flexible source DSLs into the canonical protobuf query contract when source-wire compatibility is involved, performs fail-closed admission checks, and dispatches admitted work into the internal query-serving path.

This document is architecture-scoped only. Canonical ownership remains unchanged:
- machine-readable wire contract: `api/proto/searchfabric/v1/search_fabric.proto`
- normative API semantics: `docs/specs/core/search-fabric-api.md`
- canonical error semantics: `docs/specs/core/error-codes.md`
- global runtime topology: `docs/specs/core/search-fabric-architecture.md`

The Query Gateway does not redefine public RPC semantics, `SF_*` mappings, or token validation classes. It explains how the checked-out canonical contract should be realized at production scale in a Rust + Tokio implementation.

External trust-boundary model for this document:

```text
External Compatibility Clients
  -> Unified Edge/Auth Ingress
  -> Query Gateway
  -> Query Broker

Trusted Internal Callers
  -> Query Gateway
  -> Query Broker
```

- `Unified Edge/Auth Ingress` owns external bearer/API-key or edge-mTLS termination, coarse tenant/account scope derivation, routing, audit, and rate-governor policy.
- Query Gateway consumes that authenticated context, re-validates `RequestMeta` / caller-role / scope coherence at the query-plane boundary, and then performs query-serving admission.

Primary responsibilities:
- consuming authenticated caller/scope context from `Unified Edge/Auth Ingress` or trusted internal peers and re-validating query-plane identity/scope coherence
- serving the shared canonical `QueryGatewayService` caller surface, not only compatibility-edge traffic
- serving the trusted relational `RelationalQueryService` hot paths for projected-query and primary-query execution under capability-based admission; TiDB is the current caller mapping
- source-wire parsing and canonical AST lowering
- multi-tenant metadata resolution through CCS-backed caches
- anti-scan and payload guardrails before compute engagement
- cost-aware query admission backed by CCS-owned quota leases
- consistency orchestration around `safe_ts`, `effective_read_ts`, and stateless search-context tokens
- broker-affinity routing and source-profile response reconstruction

Non-responsibilities:
- terminating external bearer/API-key auth, edge mTLS, or rate-governor policy owned by `Unified Edge/Auth Ingress`
- owning PIT/search-context state outside self-contained tokens
- synthesizing authoritative typed metadata such as `index`, `completeness`, or `effective_max_result_window`
- replacing `GetReadiness` with catalog reads or replacing `GetCollectionStatistics` with descriptor-level `approx_stats`
- encoding TiDB-native fallback as a third outcome on the shared `QueryGatewayService` response contract
- bypassing Query Broker for query execution or redefining canonical token/error behavior

## 2. Query Processing Pipeline

All incoming requests follow one deterministic, fail-closed asynchronous pipeline:

```text
Authenticated Context Intake
  -> Catalog Resolution
  -> DSL Parse + AST Lowering
  -> Anti-Scan Guardrails
  -> Cost-Aware Admission
  -> Consistency Orchestration
  -> Helper-RPC Split or Broker Dispatch
  -> Egress Protection
```

Recommended Tokio task model:
- one lightweight request task owns parsing, authz context, and admission state
- cold metadata fetches are awaited through bounded async clients and singleflight suppression
- query execution is never started before admission, token, and consistency checks finish
- response serialization is moved onto non-blocking chunked writers for large bodies

Pipeline stages:

1. Authenticated Context Intake & Authz Coherence
- accept caller identity/scope already authenticated by `Unified Edge/Auth Ingress`, or trusted internal service identity on direct internal RPC paths
- validate `RequestMeta.client_id` / `client_kind` coherence against the caller's role binding
- validate that ingress-resolved scope and routed request shape remain coherent before any catalog or compute work
- reject authn/authz mismatches before any catalog or compute work

2. Catalog Resolution
- lower Elasticsearch `{index}` or Turbopuffer `{namespace}` into `CollectionNameBinding{namespace_id, collection_name}` using the authoritative `namespace_id` already established by `Unified Edge/Auth Ingress` for compatibility-edge callers
- resolve authoritative `CollectionRef`; already-canonical callers such as TiDB compute may arrive with `CollectionRef` directly
- fetch the public descriptor only when response projection or public metadata parity needs it
- fetch private admission facts from the CCS admission-state read model owned by `api/proto/searchfabric/v1/catalog_control_private.proto` when routing/admission needs lifecycle state, descriptor revision, serving-generation readiness, or `collection_name_binding_revision`

3. AST Normalization
- parse source JSON DSL into a defensive intermediate tree
- lower the source tree into canonical `QueryRequest` fields and typed protobuf nodes
- canonicalize route/profile defaults before digest computation or dispatch

4. Anti-Scan Guardrails
- enforce payload-size, predicate-node, aggregation-depth, composite-source, result-window, and vector-response budgets
- reject unsupported source features before Query Broker dispatch

5. Cost-Aware Admission
- estimate RCU, check namespace-scoped leases, and optionally shed under backend pressure
- only admitted requests consume reader or broker capacity

6. Consistency Orchestration
- derive or validate `read_ts`, `safe_ts`, and `effective_read_ts`
- perform pre-admission continuation/search-context coherence checks (request shape, presence rules, caller/scope sanity)
- delegate authoritative token interpretation, snapshot reconstruction, and request-lifetime snapshot pinning to Query Broker

7. Shared-Query vs TiDB Fast-Path Dispatch
- `ExecuteProjectedQueryOrFallback` performs gateway-owned admission/fallback selection first and reaches Query Broker only on the `success` branch
- `ExecuteQuery` and `StreamQuery` go to Query Broker
- internal broker `StreamQuery` dispatch consumes `QueryBrokerStreamEvent{public_frame|terminal_receipt}`; gateway forwards only public frames and uses the terminal receipt for reconciliation plus public pre-first-frame error projection
- `GetReadiness` is served through the broker-owned readiness lane because it must combine reader watermarks with the query-serving `latest READY` schema generation
- `GetCollectionStatistics` goes directly to the CCS statistics path because the statistics product is CCS-owned and must stay aligned to the current query-serving `latest READY` schema generation

8. Egress Protection
- apply profile-specific row-shape reconstruction
- emit canonical headers and source-profile status/body shape
- mask sensitive predicate constants in audit logs
- serialize large responses through async chunked JSON writers

## 3. Protocol Translation & AST Normalization

The gateway implements a defensive parser that converts flexible source JSON into the strictly typed protobuf contract.

**Clarification for the Rust production runtime in this repository:** the gRPC Query Gateway implementation
hosts the canonical `searchfabric.v1.QueryGatewayService` and `searchfabric.v1.RelationalQueryService`
surfaces and expects callers to send the canonical typed protobuf request payloads (for example
`QueryRequest.search_payload` as a typed AST). Elasticsearch/Turbopuffer JSON DSL parsing and HTTP
compatibility lowering are Layer-1 concerns (for example `Unified Edge/Auth Ingress` or dedicated
HTTP compatibility gateways) and are intentionally out of scope for this gRPC gateway runtime.

### 3.1 Normalization Strategy

Recommended implementation shape:
- each source AST node implements a `LowerToCanonical` trait
- lowering is recursive but budgeted
- normalization produces one canonical `QueryRequest` plus an immutable normalization record used for logging, debugging, and digest attribution

Normalization rules:
- source-specific syntax must be fully normalized before the request reaches the canonical query API
- canonical field-path resolution must happen during lowering, not inside Query Broker execution
- all digest-affecting defaults must be materialized before dispatch
- cursor or transport-only controls must not perturb `canonical_query_digest`

Recommended internal module split:
- `adapters_es::parser`
- `adapters_tp::parser`
- `query_gateway::lowering`
- `query_gateway::canonicalization`
- `query_gateway::validation`

### 3.2 Defensive Depth Guard

The parser must enforce structural budgets during parse and lower, not after full allocation:
- `max_aggregation_depth` default: `3`
- `max_predicate_ast_nodes` default: `4096`
- `max_in_list_elements`, `max_query_payload_bytes`, and composite-source limits come from the canonical API/service policy

Error mapping:
- unsupported source constructs such as unsupported Elasticsearch aggregations or scripts fail closed as `SF_UNSUPPORTED_QUERY_FEATURE`
- aggregation-depth rejection uses the canonical unsupported-feature detail keys plus `requested_aggregation_depth` and `max_aggregation_depth`
- AST-node overflow fails closed as `SF_TOO_MANY_OPERATIONS` with `limit_scope=max_predicate_ast_nodes`

### 3.3 Fail-Closed Principle

Lowering never silently drops semantics. The gateway must immediately reject:
- unsupported ES/Turbopuffer operators still outside declared profile scope
- source features that cannot be represented losslessly in canonical protobuf
- invalid page-window or token combinations
- enum-zero or malformed typed payloads that the canonical API classifies as invalid

This is especially important for:
- composite aggregation continuation
- highlight and projection controls
- nested query lowering
- scripts and source-side post-processing features
- request shapes that would otherwise expand into unbounded scans

## 4. Multi-Tenant L1 Cache & Catalog Resolution

To reduce CCS RTT on the steady path, the gateway keeps a short-lived L1 metadata cache. The cache is an optimization only; CCS remains authoritative.

### 4.1 Cache Scope

Recommended keys:
- `(namespace_id, collection_name)` for `CollectionNameBinding`
- `(namespace_id, collection_id)` for collection admission facts

Recommended cached values:
- resolved `CollectionRef`
- resolved `collection_truth_mode`
- lifecycle state needed for query admission
- serving-generation readiness facts
- `effective_max_result_window` (`> 0` only when `serving_readiness.state=READY`; explicit `0` sentinel otherwise)
- descriptor revision / etag
- namespace-scoped `collection_name_binding_revision`

The gateway must not cache or invent public typed metadata for independent truth. When a profile needs authoritative collection metadata for response reconstruction, it must still read the public descriptor surface from CCS.

### 4.2 Revision-Based Invalidation

`CollectionNameBinding` cache coherence is revision-bound, not TTL-bound:
- both positive and negative `CollectionNameBinding` cache entries must carry the namespace-scoped `collection_name_binding_revision`
- any revision change invalidates the cached result before reuse
- TTL is only a freshness hint and is not sufficient for ABA safety across tombstone GC and same-name recreation

Descriptor-carrying entries must also carry `descriptor_revision` or etag so stale admission-state snapshots are detected before reuse.

### 4.3 Singleflight and Admission-Path Collapse

Cold misses for the same collection should be collapsed with singleflight-style request suppression:
- one in-flight CCS lookup per hot key per gateway instance
- waiting requests share the same future
- failures are not memoized longer than a short negative-cache TTL

This protects CCS from thundering-herd amplification during:
- hot deploys
- cache flushes
- workload fan-out on one cold index
- same-name recreation after reservation release

### 4.4 Tenant Partitioning

For 1B-collection cardinality, one flat LRU is insufficient.

Recommended L1 policy:
- whale tenants receive reserved capacity slices so one large namespace cannot evict its own hot working set under burst
- long-tail tenants go through TinyLFU-style admission so one-off misses do not evict reusable entries
- negative name-binding entries use shorter TTL than positive entries, but both remain revision-bound

### 4.5 Controlled Stale Reuse for Non-Admission Metadata

During CCS hot periods, the gateway may reuse slightly stale cached public descriptor material only for non-admission paths such as:
- source-profile response rendering after admission has already succeeded
- audit/debug rendering that does not influence authorization, routing, or snapshot choice

Admission-critical facts are not eligible for SWR:
- resolved `collection_truth_mode`
- lifecycle state
- `effective_max_result_window` (including the explicit `0` bootstrap/convergence sentinel)
- descriptor revision / etag
- query-serving `latest READY` schema generation

Before reusing any cached admission-state entry, the gateway must validate both:
- namespace-scoped `collection_name_binding_revision` for name-binding ABA safety
- descriptor/admission `revision` or etag for stale-snapshot detection

If either freshness proof is unavailable, the gateway must fail closed with the canonical availability error instead of speculating.

## 5. Read Consistency & Visibility Coordination

The gateway is responsible for turning caller intent into one admitted snapshot boundary.

### 5.1 Effective Read Timestamp

The governing rule stays canonical:

```text
effective_read_ts = min(read_ts, safe_ts)
```

Mode handling:
- `STRICT`: require `read_ts > 0`; if `effective_read_ts < read_ts`, TiDB projected-query hot-path requests return explicit fallback from `RelationalQueryService.ExecuteProjectedQueryOrFallback`, while TiDB primary-query requests and shared-surface `COLLECTION_TRUTH_MODE_SEARCH_FABRIC` requests fail closed as `SF_VISIBILITY_TIMEOUT`
- `BOUNDED_STALENESS`: require `read_ts > 0` and `max_staleness_ms > 0`; admit only while lag stays within budget, with the same external behavior split as strict mode (`aux_index` explicit fallback on the TiDB fast path, `COLLECTION_TRUTH_MODE_SEARCH_FABRIC` fail-closed)
- independent of read-consistency mode, aux-index requests that cannot satisfy deterministic covering-read projection at the admitted schema snapshot also return explicit fallback on the TiDB fast path
- `BEST_EFFORT`: require `read_ts = 0`; first page executes at current `safe_ts`, and continuation reuses token-bound `effective_read_ts`

### 5.2 Visibility Waiting

For `STRICT` and `BOUNDED_STALENESS`, the gateway uses the remaining request deadline budget to wait for the reader side to catch up:
- call the broker readiness lane to obtain fresh `EngineWatermarks`
- strict admission condition: `safe_ts >= read_ts`
- bounded-staleness admission condition: `tso_physical_ms(read_ts) - tso_physical_ms(min(read_ts, safe_ts)) <= max_staleness_ms`
- if the active mode's admission condition is not yet satisfied, continue waiting with bounded polling / watch-based wakeup and fresh readiness refreshes
- if deadline budget expires first:
  - TiDB aux-index fast path returns explicit fallback for either strict staleness or bounded-staleness lag breach
  - TiDB federated-table execution and shared-surface `COLLECTION_TRUTH_MODE_SEARCH_FABRIC` execution return `SF_VISIBILITY_TIMEOUT`

Failure observability must stay canonical:
- This section defines only visibility-branch-specific fields; it does not narrow or replace the canonical top-level response envelope owned by the API/protobuf contract.
- `COLLECTION_TRUTH_MODE_SEARCH_FABRIC` error branch (including TiDB federated-table execution): strict -> `ErrorStatus.context.query.visibility_timeout.{required_read_ts,effective_read_ts,safe_ts}`
- `COLLECTION_TRUTH_MODE_SEARCH_FABRIC` bounded-staleness error branch (including TiDB federated-table execution): those same fields plus `max_staleness_ms` and `actual_lag_ms`
- TiDB aux-index fallback branch: fallback directive carries `reason`, `required_read_ts`, and bounded-staleness lag fields when applicable
- when known at the decision point, query-family fatal error context is preserved canonically through `ErrorStatus.context.query.execution`, while transport-`OK` `ExecuteQuerySuccessResponse` and `ProjectedQueryResponse` continue to preserve top-level observability such as `canonical_query_digest`, projected-route `relational_execution_digest` when applicable, `applied_schema_version`, and `applied_schema_generation_token`

### 5.3 `GetReadiness` Internal Serving Path

To remove ambiguity in the architecture stack, the recommended serving path is:

```text
Query Gateway
  -> Collection resolution
  -> Broker readiness lane
     -> reader watermarks
     -> CCS serving-generation readiness facts
  -> ReadinessSuccessResponse
```

Why this path:
- `GetReadiness` must expose current query-serving watermarks, not only catalog metadata
- it must also return the query-serving `latest READY` schema generation, not merely the active control-plane pointer
- keeping the join in the query-serving path prevents CCS-only implementations from drifting away from actual reader freshness
- the TiDB projected-query fast path reuses this same internal readiness lane inside `ExecuteProjectedQueryOrFallback`; it does not require a separate external `GetReadiness` call on the hot path

### 5.4 Stateless Search Context Coordination

Search-context handling remains token-driven:
- `search_context.token` is a self-contained signed snapshot carrier
- Query Gateway enforces only request-shape and caller/scope coherence before dispatch
- Query Broker is the authoritative token interpreter and reconstructs snapshot state from the validated token payload
- keep-alive may mint a fresh token in the same lineage with a later expiry
- the active public contract does not expose an early-close RPC; token invalidation is by natural expiry or by the token-bound serving lineage ceasing to be query-admissible

### 5.5 Best-Effort Metadata Staleness

For `BEST_EFFORT` reads, slightly stale public metadata is acceptable only after admission has already pinned the request against authoritative state. The gateway must never use stale metadata to:
- bypass ownership checks
- invent schema lineage
- substitute for the latest query-serving `READY` generation
- alter token-bound `effective_read_ts`

Best-effort availability optimizations therefore apply to post-admission rendering only; they must not weaken fail-closed admission.

## 6. Cost-Aware Admission (RCU)

The gateway computes and enforces tenant budgets before Query Broker or reader compute is engaged.

### 6.1 RCU Estimation Formula

Recommended estimation model:

```text
RCU_estimated
  = Base
  + ScanWork
  + ProjectionWork
  + AggregationWork
  + VectorWork
```

Where:
- `Base` covers per-request parsing, authz, route/cache lookup, and broker overhead.
- `ScanWork = W_scan * estimated_rows_scanned * TierMultiplier(max touched DataTier)`.
- `ProjectionWork = W_proj * projected_field_units * StructuralMultiplier(weighted projected FieldAccessIntent set)`.
- `AggregationWork = W_agg * lowered_aggregation_nodes`.
- `VectorWork = sum over vector clauses(W_vec * effective_query_dimensions * clause_topk * TierMultiplier(target tier set) * QualityMultiplier(effective VectorRetrievalIntent))`.

Additional modifiers:
- projected raw vectors add a response-bandwidth multiplier
- cross-AZ execution adds a network penalty multiplier
- multi-stage retrieval groups add a fusion overhead factor
- coefficient tables for `TierMultiplier`, `StructuralMultiplier`, and `QualityMultiplier` are service-policy state, but they MUST be derived from the admitted schema's temporal/structural/quality axes instead of backend-local guesswork

### 6.2 Token-Bucket Leasing

Quota authority remains CCS-owned. The gateway should maintain only short-lived local leases:
- namespace-scoped
- query-plane specific
- dimension-specific (`query_rcu`, `query_qps`, or similar internal budget names)

Recommended flow:
- request fast-path spends from local lease
- low-watermark background task refreshes lease asynchronously
- lease expiry or `quota_epoch` mismatch invalidates the local bucket immediately
- if a valid refreshed lease cannot be established because CCS quota authority is unavailable, reject with retryable `SF_BACKEND_UNAVAILABLE`
- if a valid lease is present but the tenant budget is exhausted, reject with `SF_RATE_LIMITED` and canonical detail keys

### 6.3 Adaptive Shedding

The gateway should shed before the backend saturates when any of the following rises above policy:
- broker queue depth
- reader p99 latency
- cross-AZ bandwidth debt

Recommended policy:
- lighter requests keep higher survival probability
- high-RCU scans are probabilistically dropped first
- already-admitted strict requests are favored over speculative best-effort traffic
- raw S3 throttling / slow-down feedback remains an internal retry/backoff signal and is not itself projected as API-level tenant throttling in this component design

Public projection is deterministic:
- admission-stage tenant lease exhaustion -> `429 SF_RATE_LIMITED`
- quota-authority/lease-refresh unavailability -> `503 SF_BACKEND_UNAVAILABLE`
- admission-stage adaptive shedding under backend pressure -> `429 SF_RATE_LIMITED`
- required detail keys remain canonical: `retry_after_ms`, `throttled_dimension`, `server_limit`
- recommended `throttled_dimension` values for query-plane shedding: `query_qps`, `query_rcu`, `query_backend_pressure`, `cross_az_bandwidth`

### 6.4 Asynchronous Reconciliation

RCU estimation is only the admission baseline. After execution:
- Query Broker returns execution stats such as CPU, IO, vector work, and object-store touches
- the gateway reconciles those against the tenant lease asynchronously
- persistent bias between estimated and actual cost feeds estimator recalibration

### 6.5 Mid-Execution Pre-emption

Long-running scans should not become zombie work:
- if actual cost exceeds the admitted estimate by a large multiple, the gateway or broker-side governor may cancel execution
- the cancellation threshold should be policy-driven; `5x` is a reasonable initial default
- pre-emption must surface as an explicit canonical failure, not as silent truncation
- public projection is `429 SF_RATE_LIMITED`, not timeout or backend-unavailable remapping
- required detail keys remain canonical: `retry_after_ms`, `throttled_dimension=query_rcu_runtime`, and `server_limit=<configured overrun threshold>`

## 7. High-Dimensional Payload Governance

Vector-heavy responses can saturate memory and network faster than ranking compute.

### 7.1 Dynamic Result-Window Clamping

When the request asks for high-dimensional exact vector payloads such as `VECTOR_PAYLOAD_FORMAT_EXACT_FLOAT`, the gateway should shrink the effective response budget before dispatch:
- compute estimated bytes = hits * projected vector bytes + scalar projection overhead + JSON framing overhead
- if the estimate exceeds the unary-response budget, reduce the admissible result window
- emit canonical limit errors rather than silently truncating beyond the admitted query contract

This interacts with the authoritative `effective_max_result_window` returned by CCS; the gateway may tighten it dynamically, but never loosen it, and it must treat CCS-provided `0` as a non-admissible bootstrap/convergence sentinel rather than as a usable window.

### 7.2 Unary vs Streaming Enforcement

Recommended unary-response threshold:
- if estimated serialized payload exceeds `2 MiB`, reject `ExecuteQuery` with `413 SF_PAYLOAD_TOO_LARGE`
- include canonical dynamic-payload detail keys
- instruct clients to retry via `StreamQuery`

`StreamQuery` is the intended path for:
- large hit exports
- vector-rich result sets
- unbounded export mode (`limit=0`)
- slow consumers that benefit from backpressure-aware chunking
- not grouped/aggregation queries; source requests that need aggregation trees or composite pagination stay on unary `ExecuteQuery`

### 7.3 Memory Safety Rules

The gateway should never buffer the full large response when it can stream:
- build source-profile rows incrementally from columnar hits
- bound concurrent serialization buffers per request
- cap per-request outstanding bytes on the socket writer

## 8. Shared Token Binary Specification

This section describes the recommended internal wire format used by the shared token library for query continuations. Canonical business bindings, admitted token families, and public validation/error mapping remain owned by `docs/specs/core/search-fabric-api.md` Section `3.6` plus the corresponding token comments in `api/proto/searchfabric/v1/search_fabric.proto`.

### 8.1 Envelope

Recommended binary layout:

```text
Version | Key_ID | Payload_Length | Canonical_Payload | Signature
```

Recommended primitive:
- signature/MAC scheme: implementation-defined (server-held key); must be deterministic and fast. The Rust production runtime currently uses a 32-byte keyed BLAKE3 MAC over `(domain || wrapper_bytes)` for high-throughput verification.
- signature input: the exact wrapper bytes up to but excluding the trailing `Signature` (plus any fixed domain-separation prefix)
- payload encoding: deterministic binary encoding of versioned typed payload structs
- no payload encryption requirement in the current revision; token validity relies on the shared signed binary wrapper plus server-only mint/verify authority rather than on ciphertext semantics

Canonical business bindings stay inside the versioned canonical payload emitted by the shared token library. This document intentionally does not restate the field-level binding list owned by the canonical API/proto contract.

The same wrapper format should be used for:
- `PageCursor.next_page.token`
- `PageCursor.search_context.token`
- `PageCursor.composite_page.token`

Different token purposes use different payload schemas.

### 8.2 Internal Payload Modeling

Implement the three current query-token purposes as versioned internal structs shared by Query Gateway and Query Broker:
- `QueryPageTokenV1`
- `QuerySearchContextTokenV1`
- `QueryCompositePageTokenV1`

Recommended struct decomposition:
- one common immutable snapshot-binding record shared by all three query-token families
- one family-specific continuation or keep-alive record
- one versioned wrapper carrying envelope metadata such as `key_id`, format version, payload length, and signature bytes

The shared token library should encode/decode typed structs, not ad hoc maps or hand-maintained JSON field lists, so implementation cannot drift from the canonical API/proto token contract.

### 8.3 Validation Path

All decode, expiry, caller/scope binding, and semantic checks must go through the shared query-token library.

Required implementation invariants (not optional hardening):
- shared query-token validation MUST return typed verdicts rather than collapsing every failure into one generic reject path
- reject malformed-wrapper, signature-invalid, caller-binding-mismatched, scope-binding-mismatched, semantic-mismatched, or key-window-invalid tokens before broker dispatch or local helper projection
- project token expiry by RPC contract rather than as a universal reject:
  - query execution / continuation rejects expired query-page, composite-page, or search-context tokens
- fail closed whenever token validity, caller binding, scope binding, or key-window legitimacy cannot be established for the active RPC projection
- support bounded key-rotation overlap so old tokens remain verify-only during rolling deploys

Operational guidance:
- record separate metrics for wrapper corruption, signature failure, expiry, caller mismatch, and semantic drift

Query Gateway uses the library for edge-level coherence checks; Query Broker remains authoritative for snapshot reconstruction, continuation interpretation, and keep-alive minting.

## 9. Affinity Routing & Load Balancing

The gateway chooses a preferred Query Broker shard before dispatch.

### 9.1 Broker Affinity

Recommended hash key:

```text
(namespace_id, collection_id)
```

Recommended algorithm:
- Jump Consistent Hash from the admitted collection scope to the broker ring

Why:
- stable routing under broker membership change
- good tenant-level cache locality in the broker tier
- no external routing table required on the hot path

### 9.2 Zone Awareness

The gateway should prefer same-zone brokers when possible:
- same-zone dispatch avoids avoidable cross-AZ fan-out
- cross-zone dispatch is allowed for failover or hot-shard relief
- cross-zone routing may add an internal RCU/network penalty to keep pricing and shedding aligned with real cost

### 9.3 Capability-Aware Routing

Before dispatch, the gateway should ensure the target broker shard can serve the request's required feature set:
- schema generation and serving-generation compatibility
- vector / nested / aggregation capability flags
- lifecycle tier pruning support for the requested `target_tiers[]`

This is a broker-membership concern, not a new public protocol. The gateway may source capability snapshots from internal discovery or control-plane pushed membership state.

### 9.4 Helper-RPC Routing Split

Recommended routing split:
- `ExecuteProjectedQueryOrFallback` -> gateway admission/fallback stage -> broker ring only on `success`
- `ExecuteQuery` / `StreamQuery` -> broker ring
- `GetReadiness` -> broker readiness lane
- `GetCollectionStatistics` -> CCS statistics path for the current query-serving `latest READY` lineage

This keeps query-serving freshness with the broker/reader path, keeps optimizer statistics with its owning CCS subsystem, and keeps TiDB-native fallback semantics out of the shared `QueryGatewayService` outcome contract.

## 10. Egress Protection & Architecture Hardening

The gateway is the last line of defense before source-profile JSON leaves the cluster.

### 10.1 Serialization Governance

Large responses must use async chunked serialization so Tokio worker threads are not blocked by:
- whole-response JSON materialization
- large vector base64 or numeric array emission
- row reconstruction for high-cardinality hit sets

Recommended implementation:
- one serialization task per admitted response
- bounded channels between broker result consumption and writer
- profile-specific writers for Elasticsearch and Turbopuffer JSON shapes

### 10.2 Columnar-to-Row Transformation

Canonical query responses are columnar in the projection payload. Compatibility routes need row-oriented JSON.

Gateway rules:
- use `projected_field_names[]` and aligned `projected_values[]` as the only source of truth
- never re-fetch row data for response shaping
- for `StreamQuery`, zero-hit success frames do not establish dictionaries
- for `StreamQuery`, reuse the first hit-bearing frame dictionaries on later hit-bearing frames within the same RPC attempt
- resumed `StreamQuery` RPC attempts must receive dictionaries again on their own first hit-bearing frame
- for `StreamQuery`, expect `warnings[]` and `total_hits` only on the terminal success frame
- StreamQuery dictionary/resume invariants for this component note are tracked by `docs/testing/api-contract-conformance-matrix.md` invariant `SF-CONF-QUERY-001`

### 10.3 Search Context Lifecycle Hygiene

Because search-context state is stateless and token-driven:
- the gateway tracks active keep-alive lineages only for metrics and client-connection cleanup, not for correctness
- abrupt client disconnect may trigger only internal cleanup metric updates; there is no public early-close call to issue
- orphan cleanup is advisory only; natural expiry remains authoritative

### 10.4 PII Masking and Audit Safety

Audit logs must not leak literal sensitive values from lowered predicate trees.

Recommended rule:
- mask or hash literal values in global hard filters, clause-local filters, and token-bound debug records
- preserve field path, operator, and type class for operational debugging
- never log raw search-context tokens, decrypted page-token payloads, or full vector values outside explicitly secured debug paths

### 10.5 Error Mapping

Section `10.5` is implementation-scoped only. Canonical compatibility HTTP status, header, and error-body mappings remain owned by `docs/specs/core/search-fabric-api.md` and `docs/specs/core/error-codes.md`.

Implementation guidance:
- apply the canonical compatibility projections at the edge rather than redefining status/header/body semantics locally in Query Gateway
- keep compatibility-route projection tables in shared code or generated constants so one implementation path serves Elasticsearch-compatible and Turbopuffer-compatible error/header projection
- treat any change to compatibility HTTP status, canonical header projection, or route-specific error-body shape as a contract change owned by the canonical API/error documents, not by this architecture document
- keep source-profile-specific body reconstruction aligned with the canonical mapping contract during edge serialization

### 10.6 Hardening Checklist

Production Query Gateway instances should additionally enforce:
- bounded request-body read before full JSON parse
- per-tenant concurrency caps
- per-broker circuit breaking
- structured telemetry for `effective_read_ts`, `safe_ts`, digest mismatches, cache invalidations, and shed decisions