# Runtime record mutation v1 Status: the semantic and protobuf grammar now includes the D3 external-ID upsert walking slice at protocol `1.2` and semantic mapping revision `7`. This source state is an unreleased local candidate; it is not proof of module publication, consumer pin adoption, AppService release, deployment, or business acceptance. ## Evidence baseline This contract was derived from these exact current-tree revisions: - NexusCRM: `bfa0524850286bbb67c41fef5370041be30767e6` - nexuscrm-app-packs: `1ad078b876614e6d1ff9472035b11ae3cc5c1ac1` - nexuscrm-api-host: `1b5bb523e8692ec614e638f62a31a47c0997a3d1` - nexuscrm-contract base: `ab963538c6f64cbbd94db3181c4aa06cb7a80e78` Observed current behavior, not issue text, establishes the first slice: - the L1 distribution registers single create/update/delete and external-ID upsert together at app-packs `partition.go:1134-1156`; - composite create/update call the in-process Records seam after L1 metadata guards at `composite_collections.go:24-85`; - composite delete resolves object types in L1 before calling the in-process Records seam at `composite_collections.go:88-127`; - composite collection requests reject more than 200 submitted records at `composite_collections.go:233-237`; - the generic buffered L1 transport ceiling is 25 MiB at api-host `dialecttransport/binder.go:501-512`; - the released Runtime transport keeps 128 KiB receive, 2 MiB send, and no compression; this contract does not widen those per-message limits; - independent bulk mode commits operations independently, while atomic mode uses one pessimistic transaction and returns a result for every input; - the current public routes expose no entity tag, expected version, or other optimistic-write precondition. The first published operation set was root-record create, update, and delete. Protocol `1.2` adds an explicit root-record `UPSERT`; it does not overload create or update. Tree creation, recycle-bin lifecycle, bulk ingest, metadata authoring, and other mutation families remain outside this contract. ### Reopened consumer evidence The first contract review proved atomic outcome and reconciliation shape, but a full walking-slice reread exposed two facts that operation/result modeling alone did not preserve: - the public route decodes caller JSON without owning Catalog field types; a JSON string can represent a string, date, datetime, reference, picklist, or JSON-compatible field, and a JSON number must not pass through `float64`; - Catalog `recordMutation` policy is keyed independently by single-record and collection surfaces, including conditional rules that may need the existing record or related-record evaluation. Those facts belong to the complete coarse Runtime use case. Requiring L1 to declare schema-semantic value kinds would force a metadata side channel, and inferring policy from operation count would select the wrong Catalog fact. The additive source-value and use-case fields below correct that ownership error: L1 owns product JSON grammar and response projection; Runtime owns revision- bound metadata interpretation and policy execution. ### Protocol 1.2 external identity The current consumer/owner closure is bound to these fetched `main` heads: - nexuscrm-contract: `e758cb1f137acd4301b54c0af56a112424e3725b` - NexusCRM: `06afeea83b3b9bacc5552364917508caa01588d7` - nexuscrm-api-host: `17ca03d5a62a44aced31f7755b6693f0212cf66b` - nexuscrm-app-packs: `d07efc9bbba382519f109b7e477de87d6f99b1c5` `UPSERT` carries `object`, one `external_identity { field, value }`, and the ordinary field payload. The external identity is not repeated in `fields`; the semantic validator rejects duplication case-insensitively. An older mapper is not allowed to project this command: mapping revision `7` and protocol minor `2` are exact fail-closed fences. Runtime resolves the identity field from the same request-bound Catalog view as the object and payload. The field must be Catalog-authorized for mutation and unique; L1 must not infer eligibility from Describe text or keep a second DTO or metadata registry. Runtime then performs the create-or-adopt/update decision in the Records-owned transaction, reusing the existing exact unique-index race adoption guard and the existing CRUD/FLS/RLS, required/reference/immutable, Flow/validation, audit/cache/lifecycle/rollup/outbox path. A duplicate on a different unique constraint, a non-exact match, or ambiguous/corrupt unique state fails closed. The result keeps the input position and includes the final record identifier and an explicit `created` boolean for both create and existing-update branches. Command identity, digest, status reconciliation, retry, rollback, independent partial results, and the 200-operation bound are unchanged. A caller retrying the same command is reconciled by Runtime; L1 never query-then-creates or query-then-updates. ## Call shape | L1 request | Runtime call | Result rule | |---|---|---| | single create/update/delete | one `MutateRecords` client stream with one operation | one terminal result | | composite create/update/delete | one `MutateRecords` client stream, up to 200 operations | exactly one position-correlated result per input | | composite external-ID upsert | one `MutateRecords` client stream, up to 200 explicit `UPSERT` operations | exactly one position-correlated result with record ID and `created` | | response lost or attempt outcome ambiguous | one or more `GetMutationStatus` calls using the same command identity | never infer commit or rollback from transport outcome | Client streaming solves one specific envelope mismatch. Each typed frame remains under the existing 128 KiB receive cap, while one logical HTTP request still crosses exactly one Runtime mutation RPC. It is not a raw JSON tunnel: the inner payload is `MutationCommand` and Runtime validates the decoded semantic model. ## Identity and idempotency `CommandID` is generated by the trusted L1 caller once per logical HTTP request. Runtime keys it with the admitted trusted `(Org, AppInstallation, principal)` tuple. Those authority facts are transport context and have no business protobuf fields. Runtime computes the request digest from `CanonicalCommandBytes` after semantic validation. The command ID is excluded from the digest. Field input ordering is normalized; operation ordering remains semantic. Reusing the scoped command ID with different content fails closed. The stream header SHA-256 protects encoded frame reassembly only and is not idempotency authority. For N-1 compatibility, `UNSPECIFIED` ends at the originally released v1 bytes. An explicit use case appends exactly this suffix after the last operation: ```text uvarint(17) || UTF-8("mutation-use-case") || use_case_tag ``` where `SINGLE_RECORD=1` and `COLLECTION=2`. This is an additive semantic extension, not a replacement prefix and not protobuf serialization. The checked-in conformance vectors include the complete canonical bytes and SHA-256 for an explicit-use-case request so non-Go implementations can reproduce the binding. For an `UPSERT`, the operation grammar appends this suffix immediately after the sorted field assignments and before any request-level use-case suffix: ```text uvarint(17) || UTF-8("external-identity") || uvarint(len(external_identity.field)) || UTF-8(external_identity.field) || uvarint(len(external_identity.value)) || UTF-8(external_identity.value) ``` No other operation kind appends an identity suffix. The checked-in `external_identity_digest_vector.json` freezes a complete collection-upsert request, canonical byte string, and SHA-256 so independent implementations can reproduce command idempotency and status reconciliation exactly. A terminal response carries the derived canonical semantic digest and `outcome_expires_at_utc`. Reconciliation sends the same derived digest with the command identity, and Runtime compares both with its durable binding before returning state. The digest proves immutable content; it is not a second command identity and must never be accepted as caller-authored authority. Runtime owns the retention policy and must keep the command identity, content binding, and complete terminal outcome through that instant. An unknown, expired, or content-mismatched identity is masked as `MISSING` only on `GetMutationStatus`; a missing or RLS-denied target during mutation is the per-operation `MISSING_OR_DENIED` outcome. `CommandShape` is the minimum validation proof: command ID, semantic digest, batch mode, and ordered operation kinds. It is derived, not authored. The L1 client derives it from the original request; Runtime derives the same shape from the durable command row and operation slots. The status protobuf carries only command ID plus digest because count, kinds, and batch mode already belong to those durable slots; the contract validator consumes the derived shape so a server-authored count cannot become a competing authority. ## Atomicity and partial results `INDEPENDENT` preserves current partial-result behavior: each operation owns an independent transaction and a failure does not roll back siblings. `ATOMIC` owns one transaction for all operations. A domain failure leaves no business mutation or required durable handoff committed. The failing operation keeps its specific error; every otherwise successful or unexecuted sibling gets `ATOMIC_OPERATION_ROLLED_BACK`. There is no missing result slot. Semantic response validation enforces the same state machine: an atomic terminal outcome is either all-success, or all-failure with exactly one causal operation and rollback-only sibling markers. Independent outcomes reject atomic rollback markers. Status response validation uses the command shape derived from the original request or durable slots, not a server-authored count, so truncation, extra slots, reordered kinds, and invalid atomic shapes fail closed. The future Runtime adapter must persist its command-operation result in the same transaction as the business mutation and required durable enqueue. A retry then locks the same operation slot and observes either the committed terminal result or a pending slot after rollback. The client is never allowed to choose the outcome. ## Typed values and disclosure The union has two input altitudes. Typed neutral callers may supply string, boolean, signed integer, exact decimal, date, UTC datetime, reference identifier, or explicit null. A product HTTP adapter that does not own Catalog field types supplies the lossless JSON scalar grammar instead: string, boolean, exact JSON number text, canonical JSON object/array, or explicit null. It must not guess that a JSON string is a date, datetime, or reference. Runtime resolves the field from the one request-bound Catalog revision and performs the schema-specific conversion before DML. JSON numbers never pass through `float64`. The source-JSON canonical grammar is part of this contract rather than a Go encoder convention. Decoded object keys sort by UTF-8 bytes, duplicate keys use the last value, arrays preserve order, exact number tokens remain lexical, and strings use minimal JSON escaping without HTML or U+2028/U+2029 escaping. Invalid UTF-8 and unpaired surrogate escapes fail closed. Canonicalization can only retain or reduce the accepted input byte count; semantic admission counts the canonical bytes used by the digest and wire mapping. Pointer arms retain false, zero, and empty string. A field absent from `Operation.fields` is omitted; it is not null. Exact typed decimals carry coefficient and scale and must never be reconstructed from `float64`. The outer neutral decimal envelope is 65 coefficient digits and scale 30, matching the maximum DECIMAL shape the current TiDB-backed Runtime can store. This is a substrate safety ceiling: the request-bound Catalog field definition owns and enforces any narrower precision or scale. Admission rejects larger shapes before conversion so a tiny request cannot trigger unbounded zero padding or allocation. Runtime owns metadata-bound field interpretation and canonicalization, required field checks, CRUD, FLS, RLS, Catalog-authored mutation-policy evaluation, validation, defaulting, formulas, lifecycle, and synchronous automation. The product adapter explicitly states whether the request is a single-record or collection mutation; this neutral use-case intent is independent of atomic commit mode. Runtime must not infer it from the number of operations. An N-1 caller that omits the use case is evaluated against the union of applicable single and collection policy so omission cannot bypass Catalog policy. Catalog-authored policy display text may return only as `POLICY_DENIED`. Product error codes and fixed product English remain L1 projection. The owning L1 route chooses its operation-specific fallback code; Runtime returns only the neutral, position-correlated error grammar. Missing and RLS-denied update/delete targets share `MISSING_OR_DENIED`; internal causes, SQL, authority, permission rules, and topology are unrepresentable. Object CRUD or field-write denial is a position-correlated `ACCESS_DENIED` result so an independent mixed-object batch does not lose its current partial-result behavior; it does not disclose whether a denied record exists. A create caller obligation that remains unsatisfied after considering explicit input, compound members, Catalog defaults, and server-defaulted fields returns `REQUIRED_FIELD_MISSING` plus sorted canonical field names. The neutral contract does not carry product error copy; L1 owns that projection. Mutation policy is evaluated for the complete command before any operation performs DML. On the first Catalog-authored denial, that position carries the single `POLICY_DENIED` result and every sibling carries only `PROCESSING_HALTED`; no result may succeed in either commit mode. This preserves the current collection route's request-wide preflight behavior while keeping a complete position-stable neutral response. L1 projects the authored denial into its one public HTTP error and does not expose the neutral sibling markers. Runtime must also evaluate the same policy inside the transaction that owns each mutation so mutable record or related-row state cannot create a time-of-check to time-of-use bypass. That transaction recheck is not a command-preflight result: - in independent mode, the denied position carries `POLICY_DENIED` and every sibling retains its own committed or failed position result; - in atomic mode, the denied position is the single causal failure and every sibling carries only `ATOMIC_OPERATION_ROLLED_BACK`. `PROCESSING_HALTED` therefore remains exclusive to the zero-effect command preflight grammar. It must never be mixed with a successful result or an atomic rollback marker. A resumed independent command with an already durable position does not re-open command preflight; pending positions still receive the mandatory transaction-local policy check. Object resolution is governed by the same durable command-preflight state machine. When one operation names an object absent from the bound Catalog revision, that position carries `OBJECT_NOT_FOUND`, every sibling carries only `PROCESSING_HALTED`, and no DML runs in either commit mode. This is not a top-level `MISSING` failure: command acceptance has already established durable outcome authority, so Runtime must persist a complete terminal result that can be reconciled after a lost response. `OBJECT_NOT_FOUND` is distinct from record-level `MISSING_OR_DENIED`; L1 owns its product-specific status, code, and fixed text projection. ## Preconditions, cancellation, and retry V1 intentionally has no caller-supplied write precondition because the current consumer has none. Runtime serializes the current writes and retains current last-writer behavior after its internal transaction retry policy. External-ID upsert supplies an identity, not a caller-authored version precondition. A future real consumer requiring an expected version needs an additive reviewed semantic change; V1 does not invent an unreachable conflict promise. Cancellation before commit must leave no partial mutation or required enqueue. Cancellation, deadline, unavailable, or internal failure can race a commit and therefore require reconciliation before any replay. Request/identifier limit failures are never retryable. Only fail-fast admission exhaustion is eligible for backoff, and stale state requires authority/catalog refresh inside the original L1 route budget. ## Envelopes | Fact | Classification | Value | |---|---|---:| | submitted operations | existing public composite behavior | 200 | | field assignments per operation | new structural transport guard, reusing reviewed Runtime root-field capacity evidence | 512 | | object / field / record identifiers | existing approved Runtime v1 facts | 64 / 64 / 255 bytes | | semantic request payload | derived compatibility boundary from current buffered mutation routes | 25 MiB | | complete semantic response | existing released Runtime v1 send envelope | 2 MiB | | decimal coefficient / scale | existing TiDB-backed Runtime physical ceiling; Catalog may narrow | 65 / 30 digits | | individual request frame | existing released Runtime v1 receive envelope | 128 KiB | | mutation chunk data | new derived framing value, leaving protobuf headroom | 120 KiB | | complete encoded command | new defensive wire reassembly cap; not a public HTTP limit | 32 MiB | | compression | existing Runtime v1 transport policy | disabled | The 512-field structural guard and 25 MiB count close the worst-case protobuf wrapper overhead below the 32 MiB reassembly cap. The 25 MiB count uses `SemanticRequestSize`, which counts authored identifiers and exact value content rather than verbose internal wrapper spelling. This prevents the neutral typed representation from rejecting an otherwise legal request merely because it added representation metadata. ## Publication sequence The source PR may stage the stdlib semantic root, reviewed proto, generated wire shape, and mapping together. During that one-owner review only, the nested wire module pins the exact semantic commit by pseudo-version so its standalone module graph is reproducible. That temporary pin is CI evidence, not a release that a downstream consumer may adopt. After merge, publish the exact semantic root tag first. A follow-up wire pin must replace the temporary pseudo-version with that released root tag, prove the exact mapping revision at every public mapper entry point, and only then publish the wire module tag. A missing or mismatched revision fails `INCOMPATIBLE`; it is never treated as the current shape. Next, nexuscrm-api-host must pin those published root/wire tags; NexusCRM Runtime must pin the same contract generation; and only then may nexuscrm-app-packs pin the released transport/contract set and publish the D3 AppService. L2 may adopt only a sealed AppRelease/BOM whose lineage contains all of those versions. No local `replace`, pseudo-version, or PR-head artifact may cross an owner boundary or appear in a downstream release.