# Metadata Batch Writer Status: Frozen architecture note Owner: Search Platform Architecture ## 1. Scope This document explains the execution design of `Metadata Batch Writer` (MBW), the metadata/text batch writer named by the canonical architecture for admitted `MutationBatch` work. This document is subordinate to the canonical owner stack: - machine-readable contract: `api/proto/searchfabric/v1/search_fabric.proto` - internal owner file for this boundary: `api/proto/searchfabric/v1/metadata_batch_writer.proto` - ingest/query semantics: `docs/specs/core/search-fabric-api.md` - system topology and state ownership: `docs/specs/core/search-fabric-architecture.md` - canonical error registry: `docs/specs/core/error-codes.md` It must not redefine public `MutationBatch`, `MutationAck`, snapshot-session RPC semantics, visibility boundaries, or durable replay-anchor ownership defined by those documents. ## 2. Canonical Position in the Write Path MBW is one write-plane executor in the canonical path: ```text Ingest Gateway -> Ingest Broker -> Write Planner -> Commit Coordinator -> Metadata Batch Writer -> Visibility Barrier -> Manifest Publisher ``` MBW receives already-admitted metadata/text work after: - request authz, schema, truth-mode, and replay checks are complete, - `Commit Coordinator` has already preserved or injected canonical `commit_ts`, - canonical batch identity, stream identity, and source order have already been fixed, - `Write Planner` has already chosen the metadata batch lane for the admitted work. MBW does not: - accept source-facing API requests directly, - own `StartSnapshotSync -> UploadSnapshotChunks -> CommitSnapshotSync`, - invent a second durable batch identity or ACK contract, - mutate manifests or publish visibility by itself, - own `wal_replay_start_anchor` / `wal_durable_anchor`, - own background compaction, GC, or long-running offline import orchestration. If a workload needs sessionized offline import with upload/cutover semantics, it belongs to the canonical snapshot-sync lifecycle and `docs/architecture/snapshot-import-worker.md`, not to this note. ## 3. Contract Binding MBW executes against the canonical admitted ingest carriers rather than writer-local envelopes: - batch scope: `MutationBatch` - row scope: `MutationEntry` and `RowMutation` - write policy: `IngestPolicy` - progress/reporting: `MutationAck.success.item_outcomes`, `MutationAck.success.rejection_context`, and `MutationAck.success.applied_watermarks` - stream identity: `MutationBatch.stream_identity` - replay identity: `batch_id` plus `batch_fingerprint` Coordinator-private slicing or batch-chunking may exist as internal execution units, but those units remain subordinate to one original admitted batch: - original `entry_index` and `source_seq` mapping must be preserved, - canonical winner order stays `(commit_ts, source_partition, source_epoch, source_seq)`, - downstream still owes exactly one truthful `MutationAck` per original admitted batch. `INGEST_MODE_BATCH` on `StreamMutations` is distinct from the snapshot-sync RPC lifecycle: - `MutationBatch` batch ingest is `MutationAck`-based and source-order preserving, - snapshot sync is session/chunk/cutover based and uses `SnapshotChunkAck` plus `CommitSnapshotSync`. ## 4. Responsibilities MBW is responsible for metadata/text batch execution only: 1. Consume already-admitted metadata/text row-bearing work selected for batch execution. 2. Build immutable metadata search artifacts from that admitted work. 3. Persist immutable artifacts to object storage. 4. Emit produced key-range coverage plus `max_commit_ts` metadata needed for downstream conflict checks and publish. 5. Emit completion signals needed by `Visibility Barrier` and downstream `MutationAck` reconstruction. MBW is not responsible for: - semantic/vector execution, - public request validation or schema-latest lookup with CCS, - public response shaping at the gateway, - snapshot upload/session leases or resume tokens, - manifest CAS publication, - reader-side visibility waiting, - cross-collection storage/layout coupling. ## 5. Execution Model ### 5.1 Input Ordering MBW executes admitted row-bearing work in the canonical order: - primary key: `commit_ts` - tie-breaks: `source_partition`, `source_epoch`, `source_seq` That ordering is already defined by the canonical ingest contract and must not be replaced by writer-local arbitration rules such as `doc_id` range ownership or external-sort winner selection over the public batch itself. ### 5.2 Internal Chunking and Artifact Build MBW may internally chunk one admitted batch for resource management or artifact sizing, for example to: - bound peak memory, - build multiple immutable segment objects, - pipeline upload and publish preparation. Those internal chunks are execution details only. They must not: - change canonical apply order, - become independent public ACK units, - reorder item outcomes relative to original batch entry order, - require gateways or callers to reason about writer-local chunk IDs. - weaken the original admitted batch atomicity contract. Atomicity remains owned by the original admitted `MutationBatch`: - under `ATOMICITY_POLICY_ALL_OR_NOTHING`, MBW may build/stage multiple artifacts internally, but no row from the batch may become visible before the entire original batch succeeds. - under `ATOMICITY_POLICY_FAIL_FAST_PARTIAL`, any incremental publish must remain a canonical accepted prefix only; MBW must not publish rows after an unresolved earlier position. MBW may use bounded RAM/NVMe spill while preparing immutable artifacts, but spill state is disposable: - it is not a durable authority, - it must not become a second replay source, - crash recovery must remain pointer-driven from canonical published state and idempotent object-store artifacts. ### 5.3 Durability, Publish Readiness, and ACK Boundary In the current public contract: - `INGEST_MODE_BATCH` is admitted only with `visibility=VISIBILITY_POLICY_WAIT_FOR_VISIBLE`, - `INGEST_MODE_BATCH` is admitted only with `durability=DURABILITY_POLICY_INGEST_ACK`, - the batch lane does not expose a separate pre-visibility durable ACK anchor. Therefore MBW may report internal completion only after the immutable artifacts and publish-coverage metadata required for the admitted batch scope are durably persisted. Public success still requires downstream completion of: 1. touched-modality internal persistence, 2. `Visibility Barrier` publish-readiness, 3. `Manifest Publisher` immutable manifest write and committed-pointer CAS publish. MBW does not by itself satisfy a public durable/read-visible ACK boundary. Any internal publish granularity must remain subordinate to the same atomicity rule described above; writer-local chunks are not public commit units. ### 5.4 Conflict and Coverage Reporting MBW must report produced coverage in the canonical batch-conflict vocabulary: - produced key ranges, - relevant `max_commit_ts`, - admitted batch lineage needed for downstream conflict checks. MBW must not: - move `wal_replay_start_anchor`, - invent collection-global `snapshot_ts` replacement rules, - rewrite existing immutable streaming objects during publish. Those concerns remain owned by the canonical manifest/WAL model or by snapshot-session cutover semantics outside this note. ## 6. Watermarks and ACK Reconstruction MBW must preserve the canonical one-batch / one-ACK model. For admitted row-bearing work: - `item_outcomes[]` remains complete for every admitted entry, - `rejection_context` remains tied to the first rejected canonical cursor when failures occur, - `acked_cursor` remains reconstructible against the original admitted source order, - `applied_watermarks` remains the downstream collection frontier after the admitted write scope completes. Because of that contract, MBW must not describe itself as an asynchronous background job that returns early while later reconstruction happens out of band. ## 7. Scheduling and Resource Isolation MBW is a write-plane executor, not a `Background Compute Coordinator` authority. Implementations may still use lane-specific placement or larger workers for batch execution, for example: - dedicated pools for whale tenants, - per-collection affinity, - separate queue budgets from streaming writers. Those are scheduling details only. They must not: - redefine plane ownership, - turn the user-facing batch ACK path into a background job contract, - require a second public scheduler or job handle. ## 8. Storage and Spill Behavior Durable identity remains collection-scoped and manifest-scoped. MBW must not introduce: - ad hoc public scratch prefixes, - cross-collection shared immutable objects, - manifest append-in-place semantics, - manifest-publisher-owned rewrites of existing immutable segment objects. If temporary staging is used internally, it must remain: - subordinate to the canonical collection/object-store layout, - disposable before publish, - safe to orphan-clean without becoming visibility truth. ## 9. Failure Handling MBW failure handling must preserve owner boundaries: - upstream schema fence validation stays owned by the canonical write-admission contract; MBW must not replace it with a writer-local "latest READY generation" rule, - stale or conflicting schema/version conditions remain canonical `SF_VERSION_CONFLICT` behavior, not `SF_OWNERSHIP_CONFLICT`, - write-side `wait_for_visible` expiry remains a stream-level retryable `SF_BACKEND_UNAVAILABLE` failure on `MutationAck.error`, - orphan unpublished artifacts are cleanup concerns for lifecycle/GC flows, not visibility truth. Crash recovery must be idempotent and pointer-driven: - read committed manifest lineage, - trust only published state plus idempotent persisted artifacts for the same admitted batch lineage, - ignore unpublished or unreferenced leftovers. Implementation tuning may still vary for artifact sizing, chunk sizing, spill thresholds, tenant scheduling budgets, and metadata segment-build parallelism, but any such choice must preserve the canonical ingest, ordering, ACK, durability, and visibility contracts described above.