# Refactor learned ## Stopped source-set replacement — installation attempts are not activation ### Decision - `_System_PackGeneration` owns each replacement attempt (`preparing`, `failed_clean`, `failed_dirty`, `active`, or `retired`). - `_System_PackActivation` is the only selector for the serving generation. - `_System_InstalledPack` remains the compatibility projection of that selected generation during preparation. The final activation transaction promotes the candidate, moves the activation pointer, retires the old generation, and updates the compatibility projection together. - A historical deploy that overwrote the compatibility projection is repaired from the verified activation plus immutable generation before a new attempt. ### Mistake The stopped source-set path reused the ordinary one-row installation state machine. It changed `_System_InstalledPack` to a fresh `installing` incarnation before candidate apply, then changed the same row to `failed` on error, while `_System_PackActivation` still selected the healthy old generation. Admission later compared those intentionally different lifecycles and blocked every retry. ### Lesson An attempt, a selected generation, and a compatibility projection are different facts. Do not encode the newest attempt in the active projection or reconstruct activation by comparing two independently written status rows. Candidate-owned components receive their fresh incarnation through operation-local authority; runtime consumers continue to follow only the activation pointer. ## M1 Catalog publication — source-backed generation authority ### Decision - Phase-7 reconciliation has one classifier for every enumerated source-backed pack row. Only exact `active`, `installing`, and `failed` states are admitted. - Every admitted row needs a canonical source identity. Retries and active `official_repo` rows also need a stored revision; active `local_path` retains its existing current-source development semantics. - Incomplete active ConnectorPack authority fails closed before retry, repair, ownership backfill, or schema mutation. Connector adoption, removal recovery, rolling upgrade, and Catalog serving remain outside this PR. ### Mistake The first implementation independently filtered active and retry rows. Active rows without source identity were silently skipped, while the retry filter also trimmed status values. Together those filters left states that neither preflight owned, so another pack could be mutated before strict verification exposed the missing authority. ### Lesson “Preflight exists” is not enough. For a global mutation barrier, tests must prove that the union of all input states is classified exactly once before the first write. Focused happy-path tests did not prove that claim; a state/identity matrix plus an outer mutation-barrier test and a real-TiDB walking slice did. ### Next step Keep phase-7 fail-closed. If a launch environment contains a legacy active ConnectorPack without source authority, its owner must provide an explicit, reviewed migration contract in a separate delivery rather than adding inference or repository fallback here. ## M1 Catalog publication — immutable authority needs a reconstructible source ### Decision - `PackGeneration` / `PackActivation` publication is limited to revision-addressed `official_repo` sources. The generation freezes the exact source revision, installation incarnation, payload hash, host requirements, and immutable Runtime declarations. - `local_path` remains a mutable operator-owned development input. It keeps the existing `_System_InstalledPack` convergence path and cannot be used as Catalog publication authority. Apply, phase-7 reconcile, activation, and ambiguous-commit verification all reject any existing local `PackGeneration` / `PackActivation` state before it can be preserved. - A Git-backed `local_path` may use its recorded revision to prove an exact stopped-attempt retry, but that provenance does not turn the path into an immutable publication coordinate. ### Mistake The generation builder promoted `SourceRevision` from an optional source fact to a universal generation precondition. That contradicted the existing `local_path` contract, the nullable core schema, and the phase-7 classifier. Changing local generations to store a NULL revision fixed only the first activation. It still froze the payload hash and declarations, so the next legitimate same-path content update failed against the first generation while generation replacement was explicitly outside this PR. ### Lesson Content verification is not content addressing: `local_path` rereads its current directory and `PackGeneration` does not store enough payload to reconstruct old bytes. Derive identity from the resolver and lifecycle, not from a nullable revision column. A focused builder test with an unchanged payload could not expose that error; a real TiDB test that rewrote content at the same path did. ### Next step Keep one publication-eligibility predicate at every generation boundary: activation, commit reconciliation, ordinary repair admission, and phase-7 adoption. Prove both sides before review: `official_repo@revision` publishes and restarts with one immutable generation; `local_path` can update in place and never becomes a Catalog read fallback. The Runtime serving delivery must apply the same persisted-source rule: official sources require exact Catalog authority and may not fall back to `_System_InstalledPack`; mutable local development sources may use only their compatibility projection. ## M1 Catalog publication — preserving an incarnation requires payload identity ### Decision - Explicit stopped-upgrade adoption may preserve the legacy `PackInstallId` only when the stored `AppliedPayloadHash` proves that the resolved revision-addressed official pack is the same payload. - Both the current v1 hash and the already-supported legacy-v0 hash are valid proofs. Missing, malformed, or different hashes fail closed before pack component mutation or generation publication. - A changed payload requires a future replacement lifecycle with a new incarnation. Adoption cannot silently substitute for that protocol. ### Mistake The adoption preflight proved status, source-identity absence, and the exact legacy `PackInstallId`, but omitted the stored applied-payload fact. The implementation could therefore preserve an old incarnation while applying different current official content, reusing per-incarnation state under a new meaning. The integration fixture masked this by overwriting the stored hash with an arbitrary value and then expecting adoption to converge. ### Lesson Identity preservation is a semantic claim, not just a row-key claim. Whenever a migration preserves an existing incarnation, enumerate every fact whose change would require a new incarnation and prove them before mutation. Reuse the canonical payload validator so legacy compatibility and current hashes have one rule instead of inventing an adoption-only comparison. ### Next step Keep the stopped-upgrade adoption path narrow: exact payload plus exact incarnation creates first immutable authority; any content change remains fenced until a separately reviewed replacement protocol owns new-incarnation state transfer. ## M1 Catalog consumer views — prove the publication path, not just the reader ### Decision - App and integration consumer grants are committed by the Catalog publication owner and read only through the active immutable Catalog lineage. - `AppMetadataView` is scoped to the exact AppInstallation. An `IntegrationBindingView` additionally requires the exact authorized workload; bindings for sibling workloads are never disclosed. - One repeatable-read transaction selects the active Catalog revision, pack activation/generation, grants, and payload. Missing authority, mixed generations, or a stale requested revision fails closed; there is no `_System_InstalledPack`, repository, or global fallback. - Semantic failures must satisfy the released wire contract. Catalog staleness carries its axis, required/served revisions, and receipts; invalid cursors carry an invalid-component detail. Non-disclosing absence remains `MISSING`. ### Mistake The first integration test manually inserted the reader's final tables and covered only `AppMetadataView`. It could not prove that the real publication lifecycle committed a coherent selection, that integration bindings were workload-scoped, or that advancing the active Catalog lineage preserved an unchanged app release at the new revision. The reader also returned bare `STALE_STATE` and `INVALID_REQUEST` failures. The semantic unit tests accepted them, but the released gRPC mapper would correctly rewrite those incomplete failures to `INTERNAL`. ### Lesson A read model is not proven by testing SQL against rows shaped like its desired output. The test must enter through the owning producer lifecycle and exit through the released serving boundary. Error classifications are part of that boundary: validate their required details before claiming stable failure semantics. ### Next step Keep the Controlplane delivery limited to immutable publication and the app-blind reader. The Core M1 composition must adapt this reader onto the existing single runtime-api server and add an actual wire round-trip test for success, stale, invalid, missing, and cancellation outcomes. Flow/Command/Report interpretation remains with their Runtime owners and must consume the opaque view rather than becoming a second Catalog metadata authority. ## M1 Catalog consumer views — opaque identity must stay byte-exact in TiDB ### Decision - Consumer workload, capability, pack, installation, payload, and business-key identities introduced by this delivery use binary SQL storage semantics. - Every SQL read that reconstructs a canonical grant or integration-binding set orders the identity tuple explicitly with `BINARY`, matching Go's bytewise sort, duplicate detection, and digest input. - The production database's default collation is not an identity-normalization authority. ### Mistake The initial schema used ordinary `VARCHAR` columns in a `utf8mb4_unicode_ci` database while Go treated the same values as case-sensitive opaque bytes. TiDB could therefore collapse valid `A`/`a` identities under a unique key or return a different order from the one used to compute the committed digest. ASCII-only fixtures hid the mismatch. ### Lesson When a durable set is committed by a canonical digest, storage equality, unique indexes, SQL ordering, in-memory ordering, and digest serialization are one invariant. Test adversarial valid identities—including case variants and non-ASCII values—against the real database collation before review. ### Next step Keep this rule local to the new consumer authority. Future opaque identity columns must declare their comparison semantics at schema design time; broader legacy collation cleanup is not part of M1. ## PackGeneration replacement — finish the lifecycle before enforcing immutability ### Decision - A source-revision advance creates a fresh `PackInstallId` and a fresh immutable `PackGeneration`; it never rewrites G1 in place. - The terminal ApplyAll transaction inserts the new generation and moves `PackActivation` from the prior generation to the new one while incrementing `ActivationEpoch` as the active-pointer CAS revision. - `PreviousPackInstallId` remains NULL in the stopped-upgrade model. Existing generations are retained immutable publication receipts, not a rollback or draining lineage. Exact retries are read-only, and missing, malformed, mixed, or stale current authority fails closed. - Catalog producers and consumers bind the exact current active generation. They do not reinterpret Controlplane's epoch or previous-generation lineage; they reject an operation-bound generation as soon as it is no longer active. ### Mistake The first delivery published G1 and then treated every established activation as permanently single-generation. The pre-existing source-revision workflow still minted a fresh installation incarnation, so finalization rejected every real content upgrade after G1 existed. Focused first-publication tests passed because they never ran the ordinary G1-to-G2 upgrade. The same assumption was duplicated in Catalog as `PreviousPackInstallId IS NULL` and `ActivationEpoch = 1`. The first replacement amendment then overcorrected by making Catalog validate Controlplane lineage and by scanning all historical generations on every active-authority check. Neither behavior was required to prove the exact current generation. Removing those readers while continuing to write `PreviousPackInstallId` was also incomplete: it retained a lineage claim without an owner capable of proving the whole chain. ### Lesson An immutable snapshot mechanism is incomplete until its replacement transition and every consumer of its active pointer are proven together. Do not enforce "immutable" by forbidding upgrades, and do not patch the symptom by mutating the immutable row. One fact still needs one owner: Controlplane owns activation publication and the single active-pointer transition, while Catalog verifies only that its bound installation and digest are the exact current active generation. Full-history retention is not a serving precondition. If rolling upgrade, draining, or generation rollback is approved later, design its complete history authority then; do not smuggle it into a single optional previous pointer. Test the normal upgrade, failed-attempt retry, restart no-op, later generation, stale Catalog binding, and mixed active authority against a real database before review. ### Next step Keep the stopped-upgrade rollout model. Build the deployment input from the new active generation, republish Catalog for that exact generation, and verify the frozen release tuple. Rolling-upgrade compatibility and concurrent Catalog reconcile are separate work and are not prerequisites for this repair. ## PackGeneration advance — missing authority is not a first generation ### Decision - The explicit stopped-upgrade adoption path is the only operation allowed to publish generation 1 for an already-active legacy pack. - A source-revision advance must prove an active immutable generation at its exact `FromRevision` in both preflight and the terminal transaction. - A failed or installing target incarnation without that authority fails closed; it cannot turn the target revision into generation 1. ### Mistake The source-revision retry preflight allowed a failed/installing compatibility row when both `PackActivation` and `PackGeneration` were absent. The terminal writer then inserted the target as generation 1. That confused two different operations: adopting an already-active legacy revision and advancing from a proven revision. Existing finalization tests repeated the same invalid fixture, so focused tests did not expose the bypass. ### Lesson Do not infer missing immutable authority from mutable lifecycle state. A target row can prove that an attempt exists, but it cannot prove which prior revision was active. Keep adoption and advance as separate capabilities, and enforce the distinction again in the transaction writer so a skipped preflight cannot publish unauthorized authority. This is independent of rolling upgrade: stop-the-world deployment still requires exact provenance. ### Next step Keep the production upgrade sequence explicit: adopt the exact active legacy revision, then advance to the authorized target revision. Preserve the real-TiDB `G1 -> G2 -> restart -> G3` test and the missing-authority rejection tests as the release gate; do not add lineage, rollback, or draining behavior. ## Catalog later-generation support — relax the value, preserve the shape ### Decision - Catalog accepts any positive `ActivationEpoch`; it does not require epoch 1 and does not infer generation history from the value. - Catalog still requires `PreviousPackInstallId` to remain NULL and the epoch to be a well-formed positive integer before producing or serving an IntegrationBinding view. - The existing `ApplyAllSourceRevisionAdvances` path remains the only replacement-generation owner. Controlplane does not add a second `reapply-pack` replacement protocol; deployment composition must invoke the existing ApplyAll plan. - The terminal generation writer performs only an exact `FromRevision` to `ToRevision` transition. Already-committed retry and ambiguous-commit classification remain owned by the outer durable ApplyAll execution. - Activation and generation installation identities must be canonical opaque values, not merely non-empty after trimming. - Every deterministic terminal transition rejection that is knowable from the frozen proof must also run before the apply atom. Category changes and an exhausted activation epoch are rejected at preflight; the terminal writer repeats the checks as a transaction-bound defense. ### Mistake When the first-generation-only Catalog check was removed, the patch deleted both the obsolete condition (`ActivationEpoch == 1`) and the still-required authority-shape checks (`PreviousPackInstallId IS NULL` and positive, parseable epoch). Focused later-generation tests used only valid rows, so they proved G2 acceptance without proving malformed G2 rejection. A follow-on audit also initially treated the old deployment's `reapply-pack` call as a missing Controlplane writer instead of distinguishing the already-implemented ApplyAll owner from the separate M1-C wiring task. The terminal writer also included an "already active target" shortcut. That mixed transition authority with retry reconciliation and could accept a mixed state that no successful atomic transition can produce. The durable outer ApplyAll owner already handles committed retry and ambiguous commit. Finally, authority validation checked only that installation IDs were non-empty after trimming; matching corrupted rows containing whitespace could therefore pass as a valid authority tuple. The initial preflight/terminal comparison also stopped too early. Exact FromRevision and malformed authority were checked before execution, but pack category compatibility and activation-epoch capacity were left only in the terminal writer. Those failures are predictable from the frozen current/target proof; discovering them after the apply atom risks component mutations that the terminal transaction cannot undo. ### Lesson When broadening an accepted state, identify which predicate encodes the old limit and which predicates encode structural validity. Remove only the former. Test the new valid value and every preserved invalid shape side by side. Also derive the intended entry point from the current owner and release contract before adding another path: a missing caller composition is not evidence that the owning module needs a duplicate protocol. Transaction bodies should own one state transition; durable orchestration should own retry classification. For opaque database identities, validate canonical representation at the shared authority reader so every downstream transition fails closed together. Treat preflight and terminal checks as one matrix: preflight must contain every deterministic terminal rejection knowable before mutation, while the terminal writer repeats them against locked current truth. ### Next step Keep the producer and consumer malformed-authority tests, including real-TiDB coverage. The M1-C owner must pass the exact source-revision plan into ApplyAll for stopped production upgrade; this PR remains limited to Controlplane and Catalog correctness. ## M2 command execution authority — derive, do not re-author ### Decision - Controlplane projects command execution bindings only from the same operation's canonical `IntegrationBindingSelection` values and the verified AppRelease command registration. - `AuthorizedWorkloadID`, exact definition ref/version, and `OwnerPack` remain existing Catalog facts. The producer introduces no command route config, latest/current lookup, or repository fallback. - A Runtime command selected by an Integration binding must have one verified AppRelease registration before effective metadata can be published. - Internal deduplication uses comparable structured identities rather than delimiter-concatenated opaque strings. - The producer test crosses the real artifact boundary: effective metadata is compiled into a Catalog snapshot, projected for the exact AppInstallation, reopened, source-verified, and queried for the exact workload/definition authority. ### Mistake The first extraction inherited delimiter-built composite keys from the larger experimental implementation and initially proved only the serialized JSON fields. That could repeat the same identity-alias class already fixed in the Core compiler, and it did not prove that the new producer output was actually accepted and scoped by the released Catalog artifact pipeline. ### Lesson A producer is complete only when its output survives the owning consumer's real compile/open/project boundary. Rechecking JSON shape is not an integration proof. Keep authoring validation at its existing owner, use typed internal identities for joins, and let the Catalog compiler remain the final fail-closed authority instead of duplicating its complete validator in configuration code. ### Next step Publish this producer against the exact Core source-release closure and the minimal app-packs descriptor revision that matches that Runtime registration. Command admission and Worker execution remain separate Runtime deliveries. ## M2 producer release closure — do not validate against an accidental tuple ### Decision - Controlplane pins every NexusCRM Core module to the same source revision; it does not compose new metadata/shared with older engine modules. - Source-backed pack verification uses the explicit app-packs revision under test. For this producer, `2425c00c77e9` is the minimal revision containing the already-reviewed connector descriptor alignment; later M2 pack content is outside this PR. - The scheduled reservation bootstrap assertion follows the canonical `OccurrenceIdentityDigest` plus lease/ownership schema from that Core release. It must not preserve the superseded four-column index as a release requirement. ### Mistake The first release pin updated only metadata/shared because those were the modules imported by the new producer code. Controlplane's closure gate correctly rejected that mixed Core tuple. The next full test accidentally consumed a stale sibling app-packs checkout, while a retry against latest main consumed unrelated later M2 pack changes. Both were environment-selection mistakes, not evidence that producer behavior or the whole latest app pack belonged here. ### Lesson Dependency scope is defined by the repository's release invariant, not only by the imports touched in the diff. At the same time, source fixtures must be selected by exact reviewed revision; "nearest sibling" and "latest main" are both invalid evidence when they silently change the tested product tuple. Before changing code after a full-suite failure, identify the exact module and fixture revisions that produced it. ### Next step Run the full Controlplane test, race, vet, and Catalog artifact chain against Core `fe2d3d992732` plus app-packs `2425c00c77e9`, then open only this producer PR. The following Runtime admission PR must consume the published Catalog authority rather than introducing another command configuration or read path. ## M2 consumer-only commands — a Runtime requirement is not an L1 handler ### Decision - An Integration binding with a `RuntimeCommand` causes the command registration and command projection to be included in the same immutable Catalog snapshot even when no L1 handler selects command acceptance. - The registration still comes only from the verified AppRelease and the execution binding still comes only from the canonical Integration binding. Consumer-only projection does not create a second command authoring path. - The regression proof deliberately selects only an unrelated handler Runtime capability, then opens the derived command view and resolves the exact workload and definition authority. ### Mistake The first producer implementation assumed that command metadata would already be loaded because command acceptance appeared in the L1 handler selection list. That contradicted the v4 ownership model: the independent Integration workload consumes a required L0 capability, while sfcrm owns no command handler for that handoff. The result rejected a valid consumer-only AppRelease before Catalog could publish its binding. ### Lesson Do not infer engine-consumer ownership from L1 handler composition. When a verified operation contains a consumer-only fact, derive the necessary immutable projection from that fact at the Catalog compiler boundary. Tests must omit the tempting fake handler so they prove the actual deployment shape. ### Next step Keep Runtime command admission consumer-owned. It must bind the authenticated Integration workload to the current revision-bound command view and reject an unbound exact payload before durable acceptance; it must not manufacture an sfcrm handler or fall back to repository metadata. ## M2 connector execution producer — reuse installed-generation authority ### Decision - Connector execution mappings and inbound authorship are published only when the immutable operation contains the corresponding exact Integration binding. - Effective and pack-authored mappings remain separate typed Catalog facts; owner matching is byte-exact, including after a TiDB lookup. - `catalogapp` remains the sole reader for active pack-generation authority. Pack installation materializes the typed connector contract into that immutable generation; Catalog reconcile reads only that persisted projection during the same publication transaction. - Authorship `Generation` is the exact `PackInstallationID`; the payload digest remains a separate equality fence and must not replace incarnation identity. ### Mistake The first implementation copied the active-generation SQL and used case-folded owner matching. Although focused tests passed, that would have created a second authority decision which could drift, and a case-insensitive database collation could have mixed two owner identities in one revision. The next draft reused the authority query but reopened the source checkout from Catalog reconcile. That contradicted the reconcile boundary and would have turned source availability into a serving-publication dependency. It also initially labeled the payload digest as `Generation`; both values are immutable-looking, but only `PackInstallationID` is the installed-generation identity consumed by the Runtime binding set. The first publication guard also assumed that a connector execution binding would only occur in an operation already selecting a mutation-shaped Runtime view. The contract permits a consumer-only command requirement, so that assumption could silently omit connector execution facts instead of rejecting an unusable command composition. ### Lesson An immutable artifact is not safe merely because every input was read inside one transaction. Each authority decision must also have one implementation, and opaque owner identities must not be normalized more broadly than their contract. Source-backed facts must be materialized by install/activation and then selected from the exact immutable generation; reconcile must never reopen source or repository fallback. Falsify the producer at the selector boundary, not only at JSON serialization. Typed projection boundaries must also fail before publication when a selected binding has no compatible consumer view. Do not solve that mismatch by leaking mutation facts into a command-only view; reject the incomplete operation and keep each projection meaning closed. ### Next step Release this focused producer against Core `2e065d5f5dde`, then let the bounded Platform-command Worker consume only the resulting Runtime metadata view. Do not reopen source repositories, mapping tables, or pack generation state in the Worker. ## M2 connector mapping authority — a pack owner is not a component identity ### Decision - Every `connector_field_mapping` component now materializes its normalized projection into the immutable pack generation under its exact business key. - Catalog reconcile verifies the binding's active installation and payload digest, reads that exact generation, and selects only the bound component. - The selected immutable rows fence the live pack-authored and effective reads: an unselected same-owner component is ignored, while missing or changed selected rows fail publication closed. - Historical generations without this projection are not reconstructed from source or live tables. The stopped-upgrade path reapplies the pack before the new producer is enabled. ### Mistake The first correction reused exact pack authority for push contracts but reduced field-mapping bindings to `OwnerPack`. That made a generation-bound binding look safe while the actual row set still came from the current owner-wide mapping table. Two mapping components in the same pack could therefore be merged, and a later live change could be published under an older immutable binding. ### Lesson Authority must close over the exact fact consumed, not merely its nearest container. `OwnerPack`, `PackInstallationID`, payload digest, and component business key are all required to identify this mapping projection. A review that exposes a missing identity dimension is in scope when the PR claims an immutable revision-bound view; the narrow fix is to preserve that dimension at install time and remove the broader read, not to add upgrade fallback. ### Next step Merge and release this exact-generation producer after fresh review. The Platform-command Worker must consume the published Catalog mapping rules only; it must not reopen pack generations or live mapping repositories. ## M2 connector mapping startup verification — selector spelling is not owner authority ### Decision - The installed pack name remains the byte-exact owner used to verify every persisted mapping `Source` during startup. - The component BusinessKey prefix retains its existing case-insensitive pack match; it identifies the component but does not replace the installed owner. - Runtime and Catalog selected-owner checks remain byte-exact. ### Mistake The first persisted-projection verifier derived the mapping owner from the BusinessKey prefix. Pack validation intentionally accepts that prefix case-insensitively, so a valid case variant could install rows under the exact pack name and then fail startup verification after the projection was read. ### Lesson Do not promote a validated selector spelling into identity authority. When two existing facts have different comparison contracts, preserve both contracts at their owning boundaries rather than globally normalizing or weakening the stronger identity check. ### Next step Keep the case-variant lifecycle and forged-Source tests in the release gate, then complete fresh review without changing global pack identity semantics or adding stopped-upgrade compatibility paths. ## M2 Catalog V12 producer — extend the immutable owner, do not add a reader ### Decision - Catalog reconciliation extends the existing V11 connector-execution producer to publish the V12 finalization closure in the same transaction and Catalog revision. - Callout-profile finalization facts and both mapping directions are read only from the exact active pack generation named by the authorized binding. - Publication is staged locally and appended only after every V12 dependency is complete, so a rejected finalization closure cannot leak partial V11 facts. - Historical projectionless profile generations fail closed. The stopped upgrade path reapplies the pack before enabling this producer; there is no live/source reconstruction or rolling-upgrade compatibility path. ### Mistakes - The first implementation started a second resolver before checking current `main`; the V11 repository owner already existed. The duplicate was deleted and the existing owner was extended. - An early broad test used an unrelated newer app-pack checkout and mixed its fixture drift with the producer result. Verification now pins the compatible exact app-pack revision and separates environment prerequisites from product failures. - Combining inbound and outbound mapping reads initially let an outbound-only mapping pass a mapping-only V11 binding. The producer now preserves the existing inbound requirement unless a selected push contract actually needs the outbound closure. - After Core made executable finalization coordinates mandatory, two positive producer fixtures still omitted them. Focused compilation caught the stale fixtures before publication evidence was claimed. - The first PR creation command assigned the body and expanded that variable in the same simple command, so shell expansion occurred before the assignment and GitHub received an empty body. Pass the literal body or populate the variable in an earlier command, then read the remote PR body back immediately. ### Lessons Reuse is an ownership decision, not merely a code-reduction preference. A new Catalog fact belongs in the existing transaction/revision producer when it is a closure of the same selected bindings. Staging the whole closure makes partial publication structurally impossible, and exact-generation tests must contain every coordinate required to execute the published contract. ### Next step Pin the merged Core release that validates the complete V12 authority, remove all local development replacements, run the exact release gates, and deliver this producer as one focused Controlplane PR. The Worker must consume only this published view and must not reopen pack generations or live repositories. ## M2 Catalog V12 producer — validate before deduplicating authority ### Decision Every workload-scoped callout-profile binding is resolved and credential- authorized before identical immutable profile projections are deduplicated. Catalog may publish one shared profile fact, but it may not infer one workload's authorization from another workload's binding. ### Mistake The first producer deduplicated profile bindings by pack, generation, digest, and business key before resolving them. That key intentionally omitted the workload and credential set, so an authorized binding could mask a later binding whose credential set did not authorize the selected profile. ### Lesson Projection identity and consumer authorization are different facts. Deduplicate the immutable projection only after every consumer-scoped authorization has passed. This is a real revision-bound disclosure blocker, but its correct fix is one ordering change and a multi-workload regression test—not another registry, RPC, or compatibility path. ### Next step Complete fresh review of the exact corrected head, then merge and release the producer before enabling Runtime consumption of Catalog V12. ## M2 Catalog V12 producer — preserve workload scope through the whole closure ### Decision The producer now assembles each push contract with its callout profile and outbound mappings only from bindings carrying the same `AuthorizedWorkloadID`. It validates that complete workload-scoped closure before deduplicating byte-identical immutable authorship and finalization facts for Catalog publication. ### Mistake The first review fix validated every profile binding but still discarded the workload dimension before joining profile, mapping, and push-contract kinds. That meant workload A's push contract could borrow workload B's otherwise valid profile or outbound mapping. The individual facts were authorized, but their cross-kind combination was not. ### Lesson Authorization scope must survive until the last operation that depends on it. Validating each row is insufficient when the published authority is a join of multiple row kinds. Assemble and validate the complete scoped closure first; only the resulting immutable publication may be deduplicated. The narrow fix remains inside the existing Catalog producer and requires no new schema, RPC, registry, fallback, or upgrade mode. ### Next step Keep the cross-workload profile and mapping rejection tests plus the identical complete-closure deduplication test in the release gate. Re-run the exact fixture, race, TiDB, vet, and Linux build evidence before requesting review on the corrected head. ## M2 Catalog V12 producer — evidence harnesses need the same safety discipline ### Decision - Controlplane pins every NexusCRM Core module to the same source-release SHA; selecting only the changed `metadata` module is not a valid release closure. - Exact app-pack evidence uses the repository's sibling discovery path for the full suite, because a process-wide source-root override changes tests that intentionally own their own temporary dependency roots. - Temporary cleanup names only prevalidated, explicit absolute targets. An unset variable is never passed to a cleanup command. ### Mistakes - The first dependency update mixed `metadata` at Core `b55ce0dd` with the remaining Core modules at `eaec0a90`; the existing source-release closure gate correctly rejected that unproved combination. - The first full-suite invocation exported the exact app-pack root globally, overriding test-owned fixture roots and creating false source-not-found failures. - A failed temporary-file initialization left an empty cleanup argument. The cleanup utility interpreted it as the current directory and moved the local worktree to Trash. The worktree was rebuilt from the pushed head and the uncommitted change was reconstructed and reverified. ### Lesson Local-first proof includes the harness itself. Dependency pins, fixture discovery, log capture, and cleanup must preserve the repository's release and test contracts. A green focused test does not excuse a mixed release closure, and a broken harness is neither product PASS nor product FAIL. ### Next step Keep the full-suite Core closure gate in the PR evidence. Use explicit task-named containers and fixed validated cleanup paths for the remaining M2 integration tests, then verify cleanup before leaving each delivery loop. ## M2 Approval publication — publish the neutral sidecar, not the product source ### Decision Controlplane accepts the verified AppRelease component `metadata.approval-definition.v1`, validates its canonical neutral `approvalir` envelope, and publishes the definition into the existing effective metadata transaction and Catalog revision. Approval execution and read remain separate projection profiles; execution requires mutation facts, while read cannot acquire mutation authority. The Runtime invocation table is published through the existing managed-schema bootstrap. ### Mistake avoided The tempting shortcut was to reopen product `approval-process` metadata or live `ProcessDefinition` rows while reconciling Catalog. That would make Controlplane a second product compiler and create a repository fallback beside the immutable AppRelease. Another unsafe shortcut was to let an execution projection with no selected definition publish a generic mutation closure. During pre-merge validation I also repeated an earlier harness mistake: the first full-suite command exported `NEXUSCRM_OFFICIAL_APP_PACKS_ROOT` globally. That redirected tests with self-owned synthetic dependency roots into the real app-packs checkout and produced many false `source.not_found` failures. The focused Approval producer tests were green; after clearing the override, the complete non-cached suite was green as well. ### Lesson A serving metadata producer should validate and bind an already-lowered neutral sidecar; it must not rediscover product meaning. Capability authority also has to be structural: an execution profile without an approval definition is not an empty result, but an invalid authority closure. A read profile may legitimately project an empty set and must remain write-blind. A recorded harness lesson must become a pre-run condition, not remain prose: full-suite source-backed tests use the exact sibling checkout with the process-wide override explicitly unset. ### Next step Pin the exact merged Core release containing the Approval Catalog schemas and fail-closed projection invariant, rerun the complete source-backed suite, and deliver this as one Controlplane publication PR. Runtime serving must then open only the resulting revision-bound Catalog view and use the bootstrap-owned invocation table for same-transaction replay. ## D3 clean-install publication — keep prior selection inside its authority ### Decision The stopped-deployment Catalog adoption path derives the active publication under the publication authority's database fences. Its caller supplies only the command identity, Runtime audience, and immutable candidate. Assignment-view persistence is exposed through that same publication owner; reconcile no longer imports the internal immutable store. All Core module replacements are pinned to one reviewed source commit. ### Mistakes The first wrapper read the active pointer before calling the authority. After a successful selection, an exact retry therefore rebuilt a different request and conflicted with its own durable command. It also wrote the internal assignment store directly and updated only the metadata module pin. Focused tests missed both repository-level invariants; the complete suite found them immediately. ### Lesson Idempotency state belongs to the owner that persists the command. A caller cannot safely reconstruct a prior from mutable current state, even during a stopped upgrade. Likewise, a new producer should deliberately extend the existing artifact owner instead of bypassing an internal-store boundary. Run the full release-closure and dependency-boundary suite before claiming a local composition is release-ready. ### Next step Rebuild the exact Controlplane image, repeat clean installation against an empty TiDB, retry the identical publication command after success and process restart, and only then use the resolved D3 tuple for the launch-journey matrix. ## Stopped replacement generations — prepare locally, retire atomically ### Decision A stopped replacement records its candidate in `_System_PackGeneration`, and the caller installs returned-error cleanup as soon as preparation returns. Candidate-owned connector manifest cleanup is scoped to the candidate `PackInstallId`. The old connector incarnation is demoted and retired only inside the same transaction that switches `_System_PackActivation` and the `_System_InstalledPack` compatibility projection. A `preparing` or failed candidate has a NULL `AppliedPayloadHash`; only the activation transaction copies `DesiredPayloadHash` into that column. If a process dies after preparation, the next exclusive stopped-apply lock holder terminalizes every abandoned `preparing` row as `failed_dirty` before creating a fresh candidate. ### Mistakes The first implementation installed the candidate-failure defer after two more fallible dependency reads, so either read could strand `preparing` authority. Moving the defer still did not cover process death or an ambiguous preparation commit. Three consumer integration fixtures had also drifted from Core's nullable `_System_PackGeneration.AppliedPayloadHash` schema and incorrectly made a valid prepared candidate impossible to insert. The retry path originally treated those durable attempts as live authority forever instead of owning their recovery. The first recovery amendment then scanned a hard-coded set of supposedly nonterminal generation statuses and required that set to equal the selected installation. Test2 retained an unselected historical row, so this second authority rejected a valid redeployment even though `_System_PackActivation` still named a healthy current generation. It also taught incarnation sweepers to preserve the candidate but left owner-wide route, handler-config, and callout-profile manifest sweepers free to mutate the still-active incarnation. On a successful switch, preserving both old and candidate rows also left the old route current after activation. ### Lesson "Preserve the candidate" is not a sufficient lifecycle rule. Preparation must be generation-local, including destructive reconciliation, while retirement belongs to the activation transaction. Cleanup guards must begin at the commit that creates the fact they are responsible for closing—not at the first expected mutation. Generation-local applies only to projections whose schema carries an immutable generation coordinate. Owner-scoped metadata remains an in-place SSoT by design; duplicating it would create a second read path. If an in-place mutation has started and the stopped apply fails, `failed_dirty` plus the invalidated ApplyAll marker means Runtime must remain stopped until a full retry converges. The prior activation pointer is historical publication authority, not a claim that every mutable row was rolled back. A defer is returned-error cleanup, not restart recovery. Once an exclusive stopped-apply lock has been reacquired, an older `preparing` row cannot still have a live writer. Recovery must conservatively classify it as dirty because the durable generation row alone cannot prove whether owner-scoped writes had started. Status, rather than SQL NULLability, owns the distinction between a prepared payload and an applied payload. Historical generation statuses do not select the current Runtime and must not be compared with the activation pointer as a second current-version read path. Only `_System_PackActivation` owns that selection; generation rows retain attempt and publication history. ### Proof Focused component tests assert candidate-only sweeps, active registry preservation, and exact previous-incarnation retirement. Real TiDB tests prove the NULL unapplied representation, reproduce the durable post-interruption state with both first-generation and active-replacement candidates `preparing`, recover them on the next exclusive retry without changing the selected generation, and activate a fresh generation. The entrypoint test also forces a post-admission missing-marker error, proves `failed_clean` with no remaining `preparing` row, then retries while retaining the failed attempt as history. The retained-history regression adds a non-selected historical generation next to a valid activation and proves that a fresh candidate is admitted while the activation remains unchanged. Catalog integration fixtures now initialize the canonical L0 schema instead of copying partial PackGeneration/PackActivation DDL.