# FlowDefinition Ownership Realignment

## Summary

`#1852` correctly moves the Lead Status timestamp rule out of generic platform code and into tenant metadata plus Flow. The remaining design question is narrower: how the system should represent **admin takeover of a pack-seeded `FlowDefinition`**.

This document chooses the following split:

| Layer | Mechanism | Decision |
| --- | --- | --- |
| `FlowDefinition` | `IsCustomized` | Adopt |
| `Flow` version | `Source ∈ {standard, custom}` | Keep existing |

This replaces both rejected alternatives:

- Rejected A: `FlowDefinition.Source`
- Rejected B: `_System_MetadataPackage` + `_System_FlowDefinitionPackageOwnership`

The result is a mixed Path C:

- definition-level admin takeover uses the same invariant already used by other pack-seeded metadata
- version-level provenance stays where it already exists today

## Current Code Anchors

The decision in this document is grounded in the current codebase:

- `backend/internal/controlplane/operations.go`: `ApplyTenant` wires in exactly one tenant overlay and keeps tenant identity out of runtime input.
- `backend/internal/bootstrap/standard_flows.go`: current flow seeding contains an aspirational comment about “IsCustomized reconciliation at the FlowDefinition level” and already uses `Flow.Source` for version auto-advance decisions. This design makes that definition-level comment real.
- `backend/internal/interfaces/rest/flow_tooling.go`: all admin flow writes already pass through the Tooling API surface.
- `backend/internal/features/metadata/metadata_objects.go`: flow create / update / activate paths already share a tooling transaction boundary.
- `backend/internal/infrastructure/persistence/repo_flows.go`: flow writes already persist `Flow.Source`, but current read paths do not expose `Source`, which matters for source-sensitive activation semantics.
- `backend/internal/controlplane/tenants/pingcap/flows.go`: tenant flow seeding is currently implemented on top of package ownership and must be rewritten if this design is adopted.
- `backend/internal/domain/ports/repositories.go`: package-ownership methods are currently part of the metadata repository port and must be removed or replaced.
- `backend/internal/bootstrap/standard_actions.go`: established `IF(IsCustomized=0, ...)` and tombstone-restore seeding pattern.
- `docs/metadata-reconciliation.md`: documents the `IsCustomized` invariant for field labels and page layouts; this design extends the same invariant to `FlowDefinition`.

## Context

The branch currently contains three different directions across time:

1. PR `#1852` originally introduced `FlowDefinition.Source` to move definition ownership out of generic record persistence.
2. The local branch then evolved into a package-ownership experiment with `_System_MetadataPackage` and `_System_FlowDefinitionPackageOwnership`.
3. The business-rule extraction itself stayed correct throughout: `PicklistOption.Tags`, `EntersPickTag`, tenant flow JSON, and tenant-only seeding are the right metadata-driven fix for `Lead.Status -> Date_Status_Changed__c`.

This design keeps item 3 and realigns item 1/2 to the project’s existing ownership model.

## Goals

- Keep the Lead Status timestamp rule metadata-driven and tenant-quarantined.
- Protect admin edits to seeded flow definitions from later pack apply.
- Reuse existing repository invariants instead of inventing a flow-only ownership subsystem.
- Preserve the current standard-flow upgrade behavior driven by `Flow.Source`.

## Non-Goals

- Building a general installed-package framework.
- Replacing `Flow.Source`.
- Introducing tenant identity conventions in `DeveloperName`.
- Solving arbitrary multi-package coexistence beyond the product shape we actually ship today.
- Providing a general “hand back to pack” API that flips `IsCustomized=1` back to `0`.
- Exposing `FlowDefinition.IsCustomized` as a Tooling API field. It is internal state, not admin-editable metadata.

## Why This Direction

### 1. It matches the real platform boundary

`ApplyTenant` currently wires in exactly one tenant overlay in code and explicitly avoids resolving tenant identity from user input or environment variables. The system is not yet a generic “N packages can be installed side by side” control plane.

That makes a flow-only package registry premature. It solves a more general problem than the product currently has, while widening the platform surface with new tables, repositories, and lifecycle rules.

### 2. `IsCustomized` is already the repository’s standard ownership invariant

Pack-seeded metadata in `_System_*` tables already uses `IsCustomized` to mean:

- `0`: code / pack still owns reconciliation
- `1`: admin has taken over this row; future apply must preserve it

This is documented in `docs/metadata-reconciliation.md` and already used by standard actions, field metadata, layouts, and object metadata. `standard_flows.go` already describes the desired flow behavior as “IsCustomized reconciliation at the FlowDefinition level.”

The key point is that `IsCustomized` is not limited to a single writer. `_System_Field` already receives writes from both the platform baseline and the tenant overlay, while still using `IsCustomized` to protect admin changes. So “multi-writer” is not, by itself, a reason to invent a second ownership model.

### 3. `Flow.Source` already solves the version-level problem

`Flow.Source` is the existing version-level provenance signal:

- pack-seeded versions are `Source='standard'`
- tooling-created or tooling-edited versions become `Source='custom'`

Standard-flow upgrade behavior already depends on this. If the active version is still `standard`, bootstrap auto-advances to the new shipped version. If the active version became `custom`, bootstrap publishes the new shipped version as a draft and leaves activation alone.

That behavior is correct and should stay in the version layer. Adding a second version-ownership mechanism would duplicate existing logic without improving the model.

### 4. Path B is not actually more unified

The local package-ownership experiment adds definition-layer package tables, but it still leaves version-layer upgrade decisions on `Flow.Source`. That means it does not replace the old mechanism; it adds a new flow-only ownership system alongside it.

This increases complexity without removing a corresponding amount of old complexity.

## Rejected Alternatives

### Rejected A: `FlowDefinition.Source`

Reasons:

- It creates a new definition-level provenance field even though the repository-wide definition-level convention is `IsCustomized`.
- It duplicates the “source” concept across two layers with different semantics.
- It is not needed once definition-level ownership is expressed as “pack-owned vs admin-taken-over.”

### Rejected B: `_System_MetadataPackage` + `_System_FlowDefinitionPackageOwnership`

Reasons:

- It over-generalizes past the current product boundary.
- It introduces a flow-specific package subsystem instead of reusing the existing metadata reconciliation invariant.
- It still keeps `Flow.Source`, so it is not actually one unified ownership model.
- The added control-plane state is not admin-authored business metadata; it is internal provenance machinery. Adding more of it needs a stronger product requirement than this PR provides.

## Target Design

### Definition layer

Add `IsCustomized TINYINT(1) NOT NULL DEFAULT 0` to `FlowDefinition`.

Meaning:

- `IsCustomized=0`: the definition is still owned by shipped metadata and may be reconciled by standard-flow bootstrap or tenant overlay apply
- `IsCustomized=1`: an admin has taken over this definition through Tooling API behavior; future apply must preserve the row

No `FlowDefinition.Source`.
No `_System_MetadataPackage`.
No `_System_FlowDefinitionPackageOwnership`.

### Version layer

Keep the existing `Flow.Source` behavior unchanged.

Meaning:

- `standard`: shipped version row
- `custom`: admin-authored or admin-edited version row

The existing standard-flow publish logic stays intact:

- auto-advance only when the active version is still `standard`
- otherwise publish the new shipped version as a draft

## Tooling Write Contract

All admin-authored flow writes already pass through the tooling transaction boundary. Under this design, definition takeover is expressed by marking the parent definition customized inside that same boundary.

Introduce a single repository primitive:

- `MarkFlowDefinitionCustomized(ctx, definitionID)`

This matches the existing naming style such as `MarkLayoutCustomized`.

### Required coverage

The following tooling paths must be covered explicitly; this is not a generic “every write marks customized” rule.

### Paths that always establish admin ownership

- `POST /tooling/sobjects/FlowDefinition`
- `PATCH /tooling/sobjects/FlowDefinition/:id` for normal field updates
- `POST /tooling/sobjects/Flow` when `DefinitionId` is omitted and the handler auto-creates a definition
- `POST /tooling/sobjects/Flow` when adding a new version to an existing definition
- `PATCH /tooling/sobjects/Flow/:id`
- `DELETE /tooling/sobjects/Flow/:id` when the deleted version is not the last remaining version on the definition

Rules:

- Tooling `CREATE` must set or imply `IsCustomized=1`, not only tooling `UPDATE`.
- The auto-create path in `POST /tooling/sobjects/Flow` must behave the same as explicit `POST /tooling/sobjects/FlowDefinition`.
- Deleting any shipped non-last version is also an admin takeover signal. Even when the parent definition survives, apply should not treat that definition as pristine pack-owned state afterward.

### Paths that are source-sensitive

- Flow activation through `PATCH /tooling/sobjects/FlowDefinition/:id { Metadata.activeVersionNumber }`
- Flow activation through `PATCH /tooling/sobjects/Flow/:id { Status: "Active" }`

Rules:

- Activating a `custom` version must set the parent definition `IsCustomized=1`.
- Re-activating an existing `standard` version must **not** set `IsCustomized=1`. Otherwise a pure operational toggle would permanently opt the definition out of future pack auto-advance.
- Activating a `standard` version also never resets `IsCustomized` from `1` back to `0`. It only avoids setting the bit on definitions that are still pristine.
- Because current `FlowVersion` reads do not expose `Source`, implementation must add either a repo helper or read-model support for `Flow.Source` before this behavior can be implemented correctly.

### Paths that must preserve delete intent

- `DELETE /tooling/sobjects/FlowDefinition/:id`
- `DELETE /tooling/sobjects/Flow/:id` when deleting the last version auto-removes the parent definition
- Flow deactivation through either supported tooling path when the intended outcome is “this definition is intentionally inactive”

Rules:

- For shipped definitions, deletion must preserve a tombstone (`IsDeleted=1`) together with admin ownership (`IsCustomized=1`) or an equivalent durable signal. A hard delete loses the override and lets the next apply recreate the row.
- The last-version auto-delete path must use the same rule as direct definition delete.
- Deactivation is also not a blanket mark operation. If product semantics say “this standard flow was intentionally disabled,” the implementation must persist that outcome in a way bootstrap cannot silently reactivate on the next apply. The simplest shape is `ActiveVersionId = NULL` plus `IsCustomized=1` on the definition.

The reason for the tooling-only write gate remains transactional correctness and cache invalidation, not ownership spoofing.

## Seeder Semantics

Both standard-flow bootstrap and tenant flow apply should use the same definition-level reconciliation model.

### Upsert rules

When seeding a shipped `FlowDefinition` by `DeveloperName`:

- insert new rows with `IsCustomized=0`
- on duplicate, update pack-controlled fields only when `IsCustomized=0`
- preserve admin-owned rows when `IsCustomized=1`

The SQL pattern should follow the existing standard-action precedent:

```sql
ON DUPLICATE KEY UPDATE
  MasterLabel       = IF(IsCustomized = 0, VALUES(MasterLabel), MasterLabel),
  Description       = IF(IsCustomized = 0, VALUES(Description), Description),
  ProcessType       = IF(IsCustomized = 0, VALUES(ProcessType), ProcessType),
  TriggerType       = IF(IsCustomized = 0, VALUES(TriggerType), TriggerType),
  TriggerObjectOrEvent = IF(IsCustomized = 0, VALUES(TriggerObjectOrEvent), TriggerObjectOrEvent),
  IsDeleted         = IF(IsCustomized = 0 AND IsDeleted = 1, 0, IsDeleted),
  LastModifiedDate  = IF(IsCustomized = 0, VALUES(LastModifiedDate), LastModifiedDate)
```

### Restore behavior

Delete-then-restore must resurrect non-customized rows.

If a shipped `.flow.json` disappears and later comes back, apply must restore the tombstoned `FlowDefinition` row when `IsCustomized=0`. This is the same restore rule already used by standard actions and prevents “once deleted, never comes back” drift.

### Collision behavior

Collisions must not be silent.

Before or during upsert, if a row with the same `DeveloperName` already exists and `IsCustomized=1`, apply should emit an explicit warning and preserve the existing row unchanged.

This warning serves two purposes:

- expected case: the row was originally seeded and later taken over by an admin
- unexpected case: an unrelated admin-created definition reused the same `DeveloperName`

The current system shape does not justify a heavier package registry just to disambiguate those cases. A visible warning is sufficient and keeps reconciliation non-destructive.

### Consequence of customization

This design intentionally follows the same rule as other pack-seeded metadata: once a definition becomes admin-owned, it opts out of future pack reconciliation.

That means:

- admin-edited rows do not receive later pack field updates
- admin-tombstoned rows do not get silently recreated by apply

Re-attaching a row to the pack is out of scope for this proposal. If that capability is ever needed, it should come through an explicit repair workflow, not an implicit side effect.

## What Stays From `#1852`

This realignment does **not** change the business-rule fix itself. Keep:

- `PicklistOption.Tags`
- `EntersPickTag`
- tenant-quarantined `picklist_tags.json`
- tenant-quarantined flow JSON for `Lead_StatusChangeTimestamp`
- tenant flow seeding under `backend/internal/controlplane/tenants/<tenant>/`
- CI and specdrift coverage for the new metadata path
- tooling transaction/cache hardening added for flow writes

These pieces are the actual metadata-driven solution to issue `#1848`.

## Migration / Rollback Plan

This pivot is still the right direction, but it is not a trivial comment-only swap. Path B exists as live code and may already have been applied to dev or CI databases. The implementation therefore needs both a surgical file plan and a small data cleanup spec.

### Surgical file list

1. `backend/internal/bootstrap/system_tables.json`
   Add `FlowDefinition.IsCustomized`; remove `_System_MetadataPackage` and `_System_FlowDefinitionPackageOwnership`; keep `Flow.Source`.
2. `shared/pkg/models/salesforce_flow.go` and generated model artifacts
   Add `FlowDefinition.IsCustomized` as internal state with `json:"-"`; add whatever read-model support is needed to inspect `Flow.Source` during activation decisions.
3. `backend/internal/infrastructure/persistence/repo_flows.go`
   Persist and read `FlowDefinition.IsCustomized`; implement `MarkFlowDefinitionCustomized`; change `DeleteFlowDefinition` from hard `DELETE` to soft delete (`IsDeleted=1` plus `IsCustomized=1`) because this is a behavior change for the existing `DELETE /tooling/sobjects/FlowDefinition/:id` endpoint; make the last-version auto-delete path use the same tombstone semantics; expose `Flow.Source` where activation logic needs it.
4. `backend/internal/infrastructure/persistence/repo_flow_packages.go`
   Remove the file.
5. `backend/internal/features/metadata/metadata_objects.go`
   Replace `releaseFlowDefinitionOwnership` calls with `MarkFlowDefinitionCustomized` or source-sensitive activation/deactivation logic; update direct delete and last-version auto-delete behavior.
6. `backend/internal/interfaces/rest/flow_tooling.go`
   Ensure definition create, version create, activate, deactivate, and delete routes all flow through the new semantics.
7. `backend/internal/controlplane/tenants/pingcap/flows.go`
   Rewrite collision detection, resurrect, and orphan-delete logic from package-ownership queries to `IsCustomized` / `IsDeleted` guards.
8. `backend/internal/domain/ports/repositories.go`
   Remove package-ownership methods from the metadata repository port and add any replacement helper needed by the mixed Path C design.
9. `backend/internal/features/records/persistence_service.go`
   Remove `_System_MetadataPackage` and `_System_FlowDefinitionPackageOwnership` from the tooling-only table whitelist.
10. Tests and mocks
   Update repository tests, metadata service unit tests, tenant flow tests, and every mock metadata repository that currently carries package-ownership methods.

### Data cleanup

If any environment has already booted with the Path B schema, add a follow-up `data-specs/` cleanup that:

- drops `_System_MetadataPackage`
- drops `_System_FlowDefinitionPackageOwnership`
- preserves or discards any branch-local ownership rows explicitly, rather than leaving orphaned state behind

Per the platform boundary rules, this cleanup belongs in a data spec, not in long-lived runtime code.

## Consequences

### Benefits

- Aligns flow definition ownership with the repository’s existing `_System_*` reconciliation model.
- Keeps version provenance where it already works.
- Avoids shipping a flow-only package subsystem before there is a product requirement for one.
- Preserves the metadata-driven fix for `Lead.Status` without reopening a platform-boundary violation.

### Tradeoff

The model intentionally does not encode “which package owns this definition.” That is acceptable for the current product shape. If the platform later grows into true multi-package installation with independent package lifecycle, this decision should be revisited and generalized across metadata types, not only flows.