# Write Planner

Status: Frozen architecture note
Owner: Search Platform Architecture

## 1. Scope

This document defines the internal control design of `Write Planner`, the write-plane owner that receives broker-private dispatch frames and turns them into planner-classified work for the canonical commit/write pipeline.

This document is subordinate to the canonical owner stack:
- machine-readable public contract: `api/proto/searchfabric/v1/search_fabric.proto`
- internal owner files for this boundary: `api/proto/searchfabric/v1/ingest_broker.proto` and `api/proto/searchfabric/v1/commit_coordinator.proto`
- public ingest/query/control semantics: `docs/specs/core/search-fabric-api.md`
- canonical error registry: `docs/specs/core/error-codes.md`
- system topology and state ownership: `docs/specs/core/search-fabric-architecture.md`

Relevant trusted internal boundaries already exist and remain subordinate to that stack:
- `api/proto/searchfabric/v1/ingest_broker.proto`
- `api/proto/searchfabric/v1/commit_coordinator.proto`

This document must not redefine:
- public `MutationBatch` / `MutationAck` semantics,
- replay identity or canonical batch ordering,
- `commit_ts` ownership,
- visibility or manifest publication rules.

## 2. Canonical Position in the System

`Write Planner` sits between the broker's memory-resident stabilization layer and the stateful commit/write path:

```text
Ingest Gateway
  -> Ingest Broker
  -> WritePlannerInternalService.DispatchFrame
  -> Write Planner
  -> Commit Coordinator
  -> Stream / Batch Writers
  -> Visibility Barrier
  -> Manifest Publisher
```

The planner is therefore:
- upstream of authoritative `commit_ts` finalization,
- downstream of all source-facing authz/schema/truth-mode/replay admission,
- the owner of modality/task expansion and downstream lane planning,
- not the owner of final public visibility or final manifest mutation.

## 3. Contract Binding

### 3.1 Trusted Input

The broker -> planner trusted boundary is the unary frame dispatch already defined in `ingest_broker.proto`:
- `WritePlannerDispatchRequest`
- `IngestBrokerDispatchFrame`

Key checked-out invariants on that boundary:
- one frame contains ordered original canonical `MutationBatch` values,
- broker preserves each enclosed `MutationBatch` unchanged,
- enclosed batches are homogeneous on collection scope, durability/visibility policy, ingest mode, and schema-generation fence,
- mark-bearing batches are singleton frames,
- planner response must still provide one indexed completion outcome per original batch.

### 3.2 Trusted Downstream Handoff

Downstream of planning, the planner hands row-bearing work to `Commit Coordinator`, which then:
- preserves or injects canonical `commit_ts`,
- constructs `CommitBatchSlice` / `WriteIntent`,
- registers with `Visibility Barrier`,
- fans durable execution into the selected modality writers.

`Write Planner` therefore owns planning, not final authoritative timestamping or publish gating.

## 4. Responsibilities and Non-Responsibilities

### 4.1 Responsibilities

`Write Planner` is responsible for:

1. Receiving broker-private dispatch frames while preserving original canonical batch order and identity.
2. Re-validating mechanically checkable frame invariants at the planner boundary so a malformed internal frame fails closed instead of drifting deeper into the write path.
3. Expanding each original batch into modality-aware work:
   - metadata/text,
   - semantic/vector,
   - both when the batch touches both modalities.
4. Turning admitted ingest policy into concrete downstream execution-lane planning for the current revision:
   - streaming lane work when the admitted policy requires the streaming path,
   - batch lane work when the admitted policy requires the batch path.
5. Preserving canonical batch lineage so later owners can still return exactly one truthful `MutationAck` per original batch.
6. Passing `ProgressMark` / `BarrierMark` through in canonical order without converting them into synthetic row-bearing work.

### 4.2 Non-Responsibilities

`Write Planner` is not responsible for:

- public source request parsing or authn/authz,
- catalog resolution or schema/truth-mode admission,
- `commit_ts` assignment or Flow-A preservation logic,
- writer durability anchoring,
- `Visibility Barrier` aggregation,
- manifest publication,
- public ACK wire-shape reconstruction.

## 5. Planning Model

### 5.1 From Dispatch Frame to Planner Work

The planner consumes one `IngestBrokerDispatchFrame` and processes the enclosed original batches in order:

```text
DispatchFrame
  -> original canonical MutationBatch[]
  -> entry-kind inspection
  -> modality expansion
  -> lane planning from admitted ingest policy
  -> Commit Coordinator handoff
```

Important invariant:
- the planner may split internal execution further for downstream efficiency,
- but it must not lose the mapping back to each original batch and its original ordered entry span.

### 5.2 Modality Expansion

For row-bearing work, modality expansion is driven by the admitted mutation semantics plus schema facts
(not by source ecosystem labels).

Planner rules in this revision:
- `patch`: touched modalities are derived from the explicit field set in `PatchPayload.fields`.
  - vector fields touch the semantic path even when the payload uses the atomic `null` explicit-clear marker.
  - non-vector fields touch the metadata/text path.
- `upsert`: treated as *full replace* work and therefore must be schema-aware.
  - omitted schema-declared fields are cleared, so when the schema declares any vector fields, `upsert` touches the semantic path even if the payload does not include those fields.
- `delete`: touches every enabled modality present in the schema (metadata/text and/or semantic/vector).
- when schema facts cannot be resolved for a row-bearing batch, the planner MUST fail closed by treating both
  metadata/text and semantic/vector as touched.
- mark-only batches (`ProgressMark` / `BarrierMark`) do not touch any modalities.

The planner must keep the touched-modality set explicit because later owners use it for:
- `WriteIntent.touched_modalities`,
- modality-selective durability waiting,
- truthful publish-readiness gates.

### 5.3 Lane Planning

Execution-class planning remains planner-owned only within the admitted policy envelope:
- the planner interprets admitted `policy.ingest_mode`,
- it must not invent a hidden lane that conflicts with the canonical batch policy,
- it may still choose concrete downstream task shapes, batching, and worker fanout inside that admitted lane.

This keeps public ingest semantics stable while still letting the planner realize the work efficiently.

### 5.4 Marks and Fences

`ProgressMark` and `BarrierMark` remain first-class canonical entry kinds.

Planner rules:
- preserve their original order,
- do not merge them into neighboring row-bearing slices,
- do not reinterpret them as synthetic row work,
- let downstream owners keep progress/barrier handling coordinator-local exactly as the checked-out architecture requires.

## 6. Completion and ACK Coverage

The broker -> planner RPC is unary, but completion truth is still batch-granular.

Therefore:
- one frame may complete only when the planner can return truthful indexed coverage for every original batch in that frame,
- `WritePlannerDispatchResponse.expected_batch_count` must echo the input batch count,
- `WritePlannerBatchAck.batch_index` coverage must be exact, unique, and complete,
- planner must not collapse many original batches into one synthesized ACK.

Public ACK timing remains the canonical contract and may depend on downstream owners:
- lightweight async durability routes may complete earlier,
- `wait_for_visible` routes complete only after the full downstream visibility path does.

The planner therefore preserves per-batch completion truth while relying on later owners for the final gate.

## 7. Failure Handling and Recovery

Failure rules:
- malformed or under-specified internal frames fail closed at the planner boundary,
- downstream dependency or transport failure remains a retryable internal availability error rather than a synthetic successful completion,
- planner-local overload may slow or reject new frame dispatch, but it must not invent a second public error vocabulary.

Recovery rules:
- `Write Planner` owns no durable visibility truth,
- broker retry is the recovery mechanism before truthful downstream completion exists,
- once downstream owners have accepted work, recovery remains governed by `Commit Coordinator`, writers, `Visibility Barrier`, and `Manifest Publisher`.

## 8. Deployment Guidance

The checked-out architecture allows the planner to be:
- an independent service/process in the cloud profile, or
- an internal module inside the ingest bundle.

In both profiles the invariants stay the same:
- canonical batches remain unchanged at the planner boundary,
- planning never overrides canonical policy/truth semantics,
- per-batch ACK coverage remains exact,
- durable and visible truth remain downstream.