# Issue 3725 Integration Durable Work And External Effect Ownership Date: 2026-07-16 Updated: 2026-07-26 — added the Runtime-handoff acceptance/rejection fence. Status: R0 design authority for [#3725](https://github.com/pingcap-inc/NexusCRM/issues/3725), implementing the whole-picture decision recorded on [#3723](https://github.com/pingcap-inc/NexusCRM/issues/3723#issuecomment-4990608454). This is a design/contract change. It adds no runtime code, schema, migration, connector pack, tenant overlay, credential, or provider secret. ## 1. Decision NexusCRM will use one provider-neutral durable-work protocol inside the L0 Integration boundary while preserving separate domain facts for admission, events, commands, effects, identities, credentials, subscriptions, and Runtime handoff. The implementation sequence is: 1. finish the current inbound cutover so `ConnectorWorkItem` is the only mutable executor state for active generations; 2. extract its claim, lease, retry, lane, drain, and recovery rules into the reusable Integration durable-work kernel; 3. migrate one existing HubSpot PATCH-existing slice onto the kernel and remove that slice's old claimant; 4. complete principal-generation, token-lifecycle, subscription, signature, and distributed-quota mechanisms; 5. prove the independent Integration Service walking slice; 6. add Outreach through L1 definitions and L2 bindings. Outreach does not add a new L0 queue, worker implementation, registry authority, or provider branch. Runtime work remains Runtime-owned. Integration work remains Integration-owned. ## 2. Authority And Scope The authority order is: 1. `docs/design/l0-l1-standard.md` owns L0/L1 placement. 2. `docs/design/nexuscrm-serviceification-target-architecture.md` owns the Runtime, Catalog, Integration, and L1 service boundaries. 3. This document owns the exact Integration work/effect, principal-generation, retry, migration, repair, and Outreach pilot decisions. 4. `docs/design/connector-root-architecture.md` owns the detailed inbound receipt/event/work-item cutover and cites this document for the target scheduler ownership. 5. `docs/design/issue-1782-connector-push-durable-protocol.md` remains the semantic oracle for remote-link identity, frozen requests, ambiguous remote effects, and writeback recovery during the HubSpot migration. The delegated connector-root authority and this document share the same current-tree binding: NexusCRM base `cc92b04b2f8fe1f108009ba38908699a26bf8530` with both trusted consumers pinned to connectors `v0.2.2`. A claimant, rollback, DLQ, or compatibility statement that diverges between them is a blocking SSoT error, not an implementation choice. This document does not merge every asynchronous operation into one table. Physical storage may remain partitioned by work family when transaction, retention, or workload evidence requires it. Every Integration work family still uses one logical state machine, one storage port contract, one metrics model, and one conformance suite. ## 3. Current-Tree Facts The R0 decisions are grounded in the current repositories. ### 3.1 Inbound Connector Execution The active path already separates: ```text verified HTTP request -> IngressReceipt -> ConnectorEvent -> ConnectorWorkItem -> Flow / Runtime mutation ``` The separation is incomplete: - `_System_ConnectorEvent` still carries mutable processing, retry, claim, and DLQ fields used by compatibility reads and bridges. - `_System_ConnectorWorkItem` owns the new claim token, lease, attempt, `RunAfter`, queue class, source lane, target mutation lane, and generation. - the connector module `main` supervisor constructs `NewWorkItemWorker`; its distributed WorkItem row-CAS path has no `SchedulerLease` hook or claim predicate. - NexusCRM's backend and distributions pin connectors `v0.2.2`. Its live supervisor calls `NewWorkItemWorker` with one complete service and activation context, admits only a current controlplane cutover receipt plus complete claim schema, historical Event-to-WorkItem coverage, and authoritative zero old Event claimers/leases/transactions, and uses WorkItem row CAS without `SchedulerLease`. The WorkItem-only supervisor does not support historical Event execution or legacy custom-service admission. - `v0.2.2` exposes WorkItem-first DLQ/replay authority with bounded compatibility projections for zero-attempt/Event-only history. It does not make Event executor state authoritative after WorkItem ownership. - the published `NewWorker` compatibility construction surface and DB lease methods remain for pinned/custom callers during drain. Unmarked custom services may still use the conditional `ConnectorSchedulerLease` path; they are not the live supervisor-managed path. `ConnectorSchedulerLease` is a retained custom/pinned compatibility surface, not supervisor authority. The WorkItem-only consumer pin is already source-complete at this base through `v0.2.2`; no additional compatibility release or consumer re-pin is an R0 dependency. Current-receipt, schema, and mirror readiness govern activation. R1 deletes the compatibility surface only after `test1` mixed-version and drain evidence show that custom, pinned, and older-binary callers no longer need it. Multi-replica WorkItem CAS safety and liveness remain required evidence for that deletion decision. ### 3.2 Outbound Record Synchronization `ConnectorPushAttempt` currently combines: - a frozen external request and authority snapshot; - remote-effect and ambiguity evidence; - claim token, lease, retry count, and worker scheduling; - local compatibility-mirror recovery. `ConnectorRemoteLink` already has the correct separate identity responsibility. The request-freezing, remote-link, ambiguous-effect, and writeback semantics remain required. The independent `ConnectorPushWorker` claim loop is the part that moves to the Integration durable-work kernel. The first migration proof is the existing HubSpot sparse update: ```text Lead.Title -> PushToConnectorDeltaTitle -> PATCH existing HubSpot contact.jobtitle ``` This slice is suitable because it is production-shaped, has an established remote link, sends a sparse PATCH, and already configures one HTTP attempt. ### 3.3 Connected Per-User Delivery The connected-delivery implementation and generation schema are still open. Their blocking review findings identify two invariants that apply to every Integration work family: - credential refresh cannot happen before the worker owns the dispatch fence; - reconnect cannot change the remote identity behind queued work. The permanent fix is the `BindingGeneration` and `TokenVersion` model in Section 8. `ConnectedDeliveryJob` remains a per-user connected-channel consumer. It does not become the universal outbound record-sync model. ### 3.4 Operator Delivery Evidence `_System_ExternalDeliveryAttempt` is an append-only operator-facing evidence store for email/webhook sends and manual resends. It is not a scheduler, claim authority, external-command inbox, or remote-effect owner. Its read and resend surfaces remain separate from the Integration work queue. ## 4. Target Fact Ownership | Fact | Sole mutable owner | Authoring or creation path | Runtime read path | |---|---|---|---| | Runtime outbox/inbox | Runtime | Runtime transaction and idempotent command acceptance | Runtime worker/inbox only | | `IngressReceipt` | Integration ingress | verified webhook admission | receipt router and operator receipt view | | `NormalizedEvent` | Integration materializer | immutable derivation from one receipt | event/provenance lookup only | | `IntegrationWorkItem` | Integration durable-work kernel | materialization or accepted external command | one WorkItem claim/DLQ path | | `ExternalCommand` | Integration command acceptance | idempotent acceptance from Runtime or Integration reconciliation | effect preparation by command ID | | `ExternalEffectAttempt` | Integration effect journal | one fenced preparation and network attempt | outcome/reconciliation view | | `RemoteLink` | Integration remote-identity service | explicit create/adopt/finalize/rebind transition | outbound target lookup | | `PrincipalBinding` | shared-principal or per-user-principal adapter | admin/app installation or user authorization | pinned binding generation | | `AuthorizationGrant` | Integration token lifecycle | encrypted token mint/refresh/revoke transition | token version within one binding generation | | `RemoteSubscription` | Integration subscription reconciler | desired definition plus observed provider state | subscription health/cleanup view | | `IntegrationBindingView` | Catalog publication | immutable projection from one Catalog revision | one revision-bound Integration view | | `ExternalDeliveryAttempt` | Runtime delivery evidence store | append-only post-send evidence | operator evidence/resend only | Immutable facts may have projections and caches. A projection carries its source revision or identity and cannot become another authoring or mutable claim path. ## 5. IntegrationWorkItem Contract `IntegrationWorkItem` owns when work runs and which claimant may execute it. Required identity and binding fields: - `WorkID`; - `WorkFamily`; - `SubjectKind` and `SubjectID`; - org and App installation or installed-pack generation; - immutable definition/binding reference; - Catalog revision and digest lineage; - payload schema version; - required engine capability version; - queue class and partition/lane keys. Required mutable execution fields: - `State`; - `RunAfter`; - `Attempt`; - `LeaseOwner`; - `LeaseToken`; - `LeaseUntil`; - `OwnershipEpoch`; - `RollbackExecutionFenceID`, nullable outside an inbound compatibility rollback; - terminal legacy-outcome receipt version and digest, nullable until reconciliation; - terminal outcome, failure class, and sanitized operator summary. The state machine is: ```text pending -> processing -> succeeded -> skipped -> retry -> dead -> cancelled retry -> processing pending or retry -> awaiting_legacy_outcome -> reconciled_legacy_terminal -> legacy_outcome_unknown ``` The rollback-only transitions are permitted only after the WorkItem claimant has drained to zero and no attempt has dispatched or has an ambiguous effect outcome. `reconciled_legacy_terminal` is terminal; `legacy_outcome_unknown` is non-executable and blocks another ownership cutover. `succeeded`, `skipped`, `dead`, and `cancelled` are terminal for that WorkItem. Operator replay creates a new WorkItem with explicit replay lineage. It does not reopen or mutate the historical terminal row. ### 5.1 Claim And Fencing The distributed ownership authority is a compare-and-swap transition on the WorkItem row. A successful claim installs a new lease token and ownership epoch. Renewal, retry, completion, cancellation acknowledgement, credential refresh, effect preparation, effect dispatch, and finalization all require the live token/epoch appropriate to that transition. A stale claimant cannot: - refresh or replace a token; - reserve distributed quota; - prepare or dispatch an external effect; - create a later effect attempt; - finalize, retry, or dead-letter the WorkItem; - deliver a Runtime completion command. Process-local schedulers own fairness, polling cadence, admission, and graceful drain. They may run in every Integration replica. They do not own distributed exclusivity. The connector module `main` supervisor has no pack-wide scheduler lease in WorkItem claim eligibility: `NewWorkItemWorker` uses WorkItem row CAS. NexusCRM's pinned `v0.2.2` live supervisor directly constructs that WorkItem-only worker after current-receipt, claim-schema, and historical coverage verification; it does not select a legacy Event executor. Event owner and mirror state are guarded transition/compatibility facts, not a second supervisor claim path. Published custom and pinned `NewWorker` callers that do not opt into distributed row CAS may retain SchedulerLease. The current `v0.2.2` pin fact is source-complete at this base; readiness and `test1` evidence still govern activation. R1 deletes the retained custom/pinned compatibility surface only after mixed-version and drain evidence, with multi-process CAS proof. ### 5.2 Ordering And Pressure Source ordering and target mutation serialization remain separate lane facts. The durable-work kernel enforces the declared policy over both. Queue depth and pressure derive from WorkItems or transactionally maintained, rebuildable read models. They cannot become a second queue authority. ## 6. ExternalCommand And ExternalEffectAttempt Contract `ExternalCommand` owns immutable outbound intent: - command ID and idempotency key; - operation and logical target; - remote-link identity or create-slot identity; - principal binding ID and pinned `BindingGeneration`; - exact operation definition and `IntegrationBindingView` revision; - frozen request or content-addressed payload reference; - originating Runtime command/outbox identity; - local-completion command contract. Idempotent command acceptance creates or finds one command identity and creates at most one active WorkItem for the command/work type. ### 6.1 Runtime handoff acceptance/rejection fence The Integration command inbox is also the sole authority for whether one Runtime handoff key may still create an `ExternalCommand`. `EffectHandoffKey` alone is the stable unique admission key. Its canonical grammar includes the exact `(OrgID, AppInstallationID, RuntimeCapabilityID, RuntimeWorkBindingDigest)` derived from the verified Runtime binding; this is the `TrustedRuntimeWorkScope`, and callers cannot supply it through ordinary payload fields. The row stores that same immutable complete scope, originating Runtime outbox identity, and payload digest as validation attributes, not as additional uniqueness dimensions. It stores one terminal admission fact per `EffectHandoffKey`: - `COMMAND_ACCEPTED`, with the immutable command identity; or - `HANDOFF_REJECTED`, with a versioned safe rejection receipt and no command. Command acceptance and `RejectIfUnaccepted(TrustedRuntimeWorkScope, EffectHandoffKey, payload digest, Runtime outbox identity)` first validate that trusted context against the key-encoded and stored complete work scope, then lock or insert that same digest-independent unique inbox key. If any org/install/capability/binding component differs, or if the stored payload digest or originating outbox identity differs, the operation rejects/quarantines the conflict without disclosing cross-scope state and cannot create another admission row, command, or WorkItem. Acceptance atomically writes `COMMAND_ACCEPTED`, the `ExternalCommand`, and at most one active WorkItem. Rejection returns an existing accepted receipt or, when no command was accepted, atomically writes `HANDOFF_REJECTED`. A later acceptance for a rejected key fails deterministically and cannot create a command or WorkItem. `RejectIfUnaccepted` is a trusted service command authorized only for the originating Runtime handoff. It is idempotent after a lost response. A payload digest, Runtime outbox identity, org, App installation, Runtime capability, or binding-digest mismatch is quarantined. None of the four command arguments may be omitted or reconstructed from an unscoped lookup. A lookup that merely observes no command is not evidence of rejection; a caller may treat non-acceptance as proven only after receiving the durable `HANDOFF_REJECTED` receipt. The owner retains both admission outcomes beyond the Runtime outbox, redelivery/repair, and originating workflow retention windows. This closes the late-acceptance race: an in-flight acceptance either wins the unique-key transaction and is returned to the rejector, or loses to the rejection tombstone and cannot commit afterward. `ExternalEffectAttempt` owns one externally visible attempt: - effect-attempt ID and command ID; - frozen method, relative target, safe headers, body hash/reference, and idempotency material; - the WorkItem lease token/ownership epoch that authorized preparation; - pre-dispatch validation and quota evidence; - dispatch-start checkpoint; - response status and safe provider receipt/correlation evidence; - one terminal outcome classification; - reconciliation or manual-repair disposition. The outcome classes are: | Outcome | Meaning | Automatic next action | |---|---|---| | `ACCEPTED` | provider accepted and the required durable result evidence exists | complete or enqueue local completion | | `REJECTED` | provider returned a terminal, known rejection | dead or complete with rejected result | | `RETRYABLE_PRE_DISPATCH` | failure is proven to have occurred before external dispatch | schedule a new WorkItem attempt | | `REPLAY_SAFE_RETRY` | the frozen provider contract proves the same effect can be retried with the same idempotency identity or queried outcome | schedule according to the frozen policy | | `OUTCOME_UNKNOWN` | dispatch may have happened and the outcome cannot be proven | reconciliation or explicit operator repair | Every externally visible call uses `DoOnce`. An in-process HTTP retry loop is allowed only when the frozen operation contract proves replay safety. The durable scheduler and HTTP client cannot independently retry the same effect. `ExternalEffectAttempt` never owns `RunAfter`, worker leases, or queue claim state. `IntegrationWorkItem` never becomes the source of truth for the remote effect outcome. ## 7. Remote Identity, Conflict, Delete, And Reconciliation `RemoteLink` remains the single local/remote identity owner. A source record's materialized remote-ID field is a compatibility mirror after cutover. The first Outreach pilot uses these policies: - external object: Outreach `Prospect`; - local L2 pilot object: CRM `Lead`; - inbound: `prospect.created`, `prospect.updated`, and `prospect.destroyed` webhook events; - outbound: PATCH an already-linked Prospect from a Runtime post-commit outbox; - remote create and remote delete: deferred from the first walking slice; - principal: one organization-scoped Outreach S2S app installation; - webhook payload version: version 2; - one field has one direction owner in a binding. L2 classifies each mapped field as inbound-owned or outbound-owned; the first slice has no last-writer-wins field; - partial webhook attributes update only declared inbound-owned fields. A route may hydrate the current Prospect when the Flow requires a complete snapshot; - `prospect.destroyed` retires or suspends the RemoteLink and records observed remote deletion. It does not delete the local Lead; - local deletion retires local synchronization state. It does not delete the remote Prospect; - ambiguous identity, multiple local matches, conflicting remote locators, and binding drift fail closed for reconciliation. Outreach webhooks use HMAC-SHA256 over the raw body and do not supply a timestamp token for the standard webhook signature. L1 supplies the header, payload, resource/action, response-status, and signature-template definitions. L0 supplies the parameterized verifier. Outreach sends partial create/update webhook payloads, has a five-second response window, and does not redeliver based on HTTP failure status. The pilot therefore requires: - a provider-compatible `200` response only after durable receipt; - no synchronous Flow or provider API work before the receipt boundary; - alerting for every pre-receipt rejection or timeout; - incremental reconciliation using the provider's `updatedAt` cursor and stable Prospect ID; - explicit full or bounded repair for cursor gaps, deleted resources, and operator-selected periods. Generic L0 never assumes that a `429` or `5xx` response causes provider redelivery. The L1 route contract declares the provider response/redelivery semantics, and L2 declares reconciliation frequency and operational policy. The initial idempotency identity uses a provider delivery identifier when the binding exposes one. Otherwise it uses a deterministic key over subscription generation, event name, resource type, resource ID, delivered-at value, and raw body hash. ## 8. PrincipalBinding, BindingGeneration, And TokenVersion Two principal authoring models remain distinct: - shared principal: authored by an admin, app installation, or named-credential binding for organization-level work; - per-user principal: authored by one user's authorization grant. They share the L0 token-lifecycle kernel and fault model. ### 8.1 BindingGeneration `BindingGeneration` changes when queued work could otherwise select a different remote identity or authorization authority, including: - disconnect followed by reconnect; - issuer or remote subject change; - Outreach app uninstall/reinstall or installation-ID change; - named credential rebind to another remote tenant/account; - explicit principal replacement or authorization-scope rebind that changes effective authority. Every `ExternalCommand` pins the binding ID and `BindingGeneration`. Dispatch fails closed if that generation is no longer current. Reconnect cannot retarget queued work. ### 8.2 TokenVersion `TokenVersion` changes when secret material rotates within the same binding generation: - access-token refresh; - rotating refresh-token replacement; - Outreach S2S access-token mint for the same app installation; - encryption-key rewrap that preserves the same effective authority. A worker may use a newer `TokenVersion` only within the pinned `BindingGeneration`. Refresh or S2S minting requires: 1. a live WorkItem claim; 2. a refresh ownership CAS/lease scoped to the binding generation; 3. single-winner provider refresh/mint; 4. atomic activation of the new encrypted token bundle and `TokenVersion`; 5. secret-safe failure evidence and deterministic crash recovery. The winning refresh owner may return the activated token to the live dispatch claimant. A stale worker cannot call the provider refresh/mint endpoint. ## 9. Subscription And Distributed Quota Ownership L1/Catalog owns the desired remote subscription definition. Integration owns observed subscription state: - provider subscription ID; - binding and subscription generation; - active/disabled health; - secret generation; - encrypted cleanup token; - last successful reconciliation; - create/update/delete/cleanup outcome. Subscription reconciliation uses `IntegrationWorkItem` and `ExternalEffectAttempt`. It does not create another command queue. Orphan cleanup is fenced by subscription generation. Provider quota enforcement is distributed before multi-replica Integration production cutover. L1 declares quota dimensions and policy references. Integration atomically reserves capacity by the declared scope, which may include org, principal binding, endpoint class, and provider quota key. Process-local buckets may remain admission hints; they cannot be the quota authority. ## 10. DLQ, Replay, Reconciliation, And Operator Repair The WorkItem row owns retry/dead state. Operator DLQ reads: ```text terminal IntegrationWorkItem + immutable event or command + external effect attempts + receipt, remote-link, and binding summaries ``` Replay rules: - replay authorization is checked at request time; - replay creates a new WorkItem and records lineage to the terminal WorkItem; - event/command/effect history remains immutable; - current definition, route, binding generation, and Catalog revision must be resolved explicitly; - replay fails closed when identity, authority, or idempotency safety is ambiguous; - `OUTCOME_UNKNOWN` cannot be replayed as a new external call until reconciliation proves no effect, proves a replay-safe identity, or an operator records an explicit repair disposition. Manual repair may: - attach provider evidence and mark the known outcome; - bind an existing remote record after uniqueness checks; - retire/supersede a blocked command; - create a new explicit command with repair lineage. Manual repair cannot edit an old attempt into a different historical request or silently reopen terminal work. ## 11. Migration And Rollback Every work-family cutover uses a durable `OwnershipEpoch` or generation with one active claimant. ### 11.1 Inbound Cutover The source baseline already pins the WorkItem-only `v0.2.2` supervisor. Deployment activation and compatibility retirement remain separate gates: 1. Materialize WorkItems idempotently while WorkItem execution remains disabled, quiesce Event claim admission for the selected pack/generation, and prevent every older/custom claim-capable binary from restarting. 2. Drain every already-owned Event lease and active Event claim transaction. Prove zero Event-owned processing rows, claim tokens, claim transactions, and claim-capable old processes; no WorkItem claim may begin during drain. 3. Only then write/verify the durable controlplane cutover receipt binding that zero-old-claimer proof, complete claim schema, Event `ExecutionOwner`, and historical Event-to-WorkItem coverage. Start the `v0.2.2` supervisor only after every process observes that receipt. A gap fails closed; this binary never falls back to historical Event execution. 4. Prove multi-process WorkItem CAS, stale reclaim, lane ordering, DLQ/replay authority, and `test1` mixed-version drain. 5. After older binaries and published custom/pinned callers are proved drained, remove Event mutable executor compatibility reads/writes and `ConnectorSchedulerLease` predicates, lifecycle APIs, and schema. Rollback before destructive removal first stops all WorkItem claim/reclaim admission and prevents WorkItem-capable processes from starting/restarting. It then proves zero active/expired WorkItem leases, WorkItem-owned processing rows, `Started` effect/execution attempts, claim/reclaim transactions, and running claim-capable processes. The rollback-fence transaction then moves every remaining eligible `pending` or pre-dispatch `retry` WorkItem to `awaiting_legacy_outcome` with a `RollbackExecutionFenceID`; prior dispatch, ambiguous effect outcome, or unverifiable attempt history blocks rollback. Only a new durable owner-generation receipt binding that complete zero proof and the complete fenced-WorkItem set may admit a previously approved rollback-bridge-compatible legacy binary. That minimum compatible binary must write a versioned `LegacyOutcomeReceipt` atomically with every Event terminal transition. A durable projector/reconciler uses the receipt to move the matching WorkItem to `reconciled_legacy_terminal` and store its receipt version/digest, including after a crash between receipt commit and WorkItem projection. Unknown/quarantined outcomes remain non-executable and block re-cutover. Re-cutover requires zero `awaiting_legacy_outcome` and `legacy_outcome_unknown` rows and a confirmed terminal receipt for every rollback fence. A reconciled terminal WorkItem is immutable and is never reopened, reset, or reused; the existing unique Event/work-type identity proves legacy execution already owned that logical work. The materializer creates an executable WorkItem only for an eligible non-terminal Event with no WorkItem row. Missing, conflicting, or ambiguous receipts fail closed. This protocol cannot enable a legacy Event fallback inside `v0.2.2`, activate both claimants, or require manual SQL rescue. ### 11.2 HubSpot Outbound Cutover 1. Keep `ConnectorPushAttempt` as the parity oracle. 2. Accept one immutable external command for the existing `Lead.Title -> contact.jobtitle` PATCH slice. 3. Create one Integration WorkItem and one external effect attempt. 4. Fence the old ConnectorPush claimant for the slice by ownership epoch. 5. Prove the same remote link, sparse payload, one-attempt behavior, local completion, and operator evidence. 6. Remove the old claim loop for that slice. Rollback reactivates only the old slice owner while the new WorkItems remain non-executable audit/migration rows. ### 11.3 Principal Generation Migration Generation columns land through an upgrade-safe nullable/default/backfill/tighten sequence. Jobs and commands pin the generation before reconnect can mutate or replace the binding. Schema changes cannot create a period where a queued job has only a principal ID while the principal ID can select multiple generations. ## 12. Implementation Spine | Phase | Deliverable | Exit gate | |---|---|---| | R1 | WorkItem-only inbound execution, Event DLQ/read cutover, SchedulerLease compatibility retirement | source pin to `v0.2.2` confirmed; current receipt and schema/mirror coverage; authoritative zero old claimers, leases, rows, transactions, and processes before activation; rollback-fenced pending WorkItems reconcile to confirmed terminal legacy receipts before re-cutover; `test1` drain; no mutable Event claim state; and multi-process CAS proof | | R2 | provider-neutral Integration durable-work/effect kernel, command admission/rejection fence, and conformance suite | all Integration work families can adapt without copying claim/retry or handoff-fencing code | | R3 | HubSpot `Lead.Title` sparse PATCH migration | one active claimant, semantic parity, old slice claimant deleted | | R4 | token lifecycle, principal generation, optional-timestamp HMAC, subscription reconciliation, distributed quota | stale claimant and reconnect fault matrix passes | | R5 | revision-bound Integration Service walking slice | no Runtime-table write or shared repository fallback | | R6 | Outreach L1 definitions and L2 Lead pilot | Prospect webhook/PATCH/reconciliation proof without L0 provider code | ## 13. Required Conformance Proof The durable-work/effect kernel must prove: - WorkItem claim admission remains disabled until authoritative zero old Event claimers, leases, processing rows, claim transactions, and claim-capable processes are bound into the cutover receipt; - forward cutover and rollback each drain the old owner to zero before admitting the next owner, with no Event/WorkItem overlap window; - rollback cannot admit Event ownership while any WorkItem processing row, `Started` attempt, lease, claim/reclaim transaction, or claim-capable process can still execute or later reclaim; - rollback atomically fences every remaining unstarted WorkItem as `awaiting_legacy_outcome`; the compatibility owner writes a durable terminal legacy-outcome receipt; and a receipt projector converges a crash before the WorkItem projection; - re-cutover has zero awaiting or unknown rollback-fenced WorkItems, never reopens a `reconciled_legacy_terminal` row, skips its existing unique Event/work-type identity, and materializes only eligible Events with no WorkItem row; - duplicate legacy terminal delivery is idempotent, while a missing, conflicting, or ambiguous legacy outcome blocks re-cutover; - concurrent claim has one winner; - stale claimant cannot refresh, dispatch, complete, retry, or finalize; - process kill before and after claim; - lease expiry and reclaim; - `RunAfter`; - source and target lane ordering; - cancellation and graceful drain; - dead/replay authorization; - no secret or raw sensitive payload leakage in logs, metrics, issue evidence, or operator summaries; - multi-process TiDB execution; - no mixed Catalog/Integration binding revision; - one external attempt for ambiguous writes; - command acceptance racing `RejectIfUnaccepted` produces exactly one accepted command or one durable rejection tombstone; - a repeated `EffectHandoffKey` with a different payload digest or Runtime outbox identity quarantines on the existing row and cannot create a second admission fact; - a cross-org acceptance or rejection request for an existing handoff key fails closed without creating or exposing another org's admission fact; - a same-org request with a different App installation, Runtime capability, or RuntimeWorkBinding digest fails closed on the existing handoff fence and cannot accept or expose another work scope; - late delivery after `HANDOFF_REJECTED` cannot create a command or WorkItem; - absent command without a rejection receipt remains outcome-unknown; - reconnect cannot retarget queued work; - replica scaling does not multiply provider quota. The Outreach pilot additionally proves: - HMAC verification over byte-identical raw request body; - durable receipt before `200`; - request completion within the provider deadline under the approved workload; - duplicate network delivery deduplication; - partial update hydration; - webhook-loss reconciliation; - destroyed Prospect behavior without local delete; - binding reinstall/reconnect fencing; - outbound PATCH ambiguity and operator repair. ## 14. External Provider References The Outreach pilot decisions use the official contracts: - [Outreach webhooks](https://developers.outreach.io/api/webhooks) - [Outreach S2S API access](https://developers.outreach.io/api/s2s-access) - [Outreach API request semantics](https://developers.outreach.io/api/making-requests) - [Outreach Prospect reference](https://developers.outreach.io/api/reference/prospect/prospect-relationships) Provider documentation defines the L1 adapter and L2 operational policy. It does not become generic L0 runtime vocabulary. ## 15. Authority This document is the single design authority for tracker #3725 R0. Later PRs may refine implementation details while preserving the fact boundaries, generation semantics, one-active-claimant rule, retry/ambiguity policy, and Outreach pilot decisions. A change to those decisions updates this document and the tracker snapshot in the same round.