# Content Store Architecture

> Canonical path: `specs/architecture/content-store.md`
> Status: living architecture spec
> Scope: native tree-version ownership, payload persistence, integrity, and rollout safety.

## Purpose

AgentTeam uses the **Content Store** as the single cloud version boundary for
product content. It owns opaque `VersionedContent` identity, immutable
`ContentVersion` records, remote-head compare-and-swap, bounded history,
events, and retention. Legacy repository mappings belong to the dedicated
product migration seam, not to the Content Store contract.

Every active aggregate is a complete immutable tree persisted through the
`TreePayloadBackend` contract. Product callers do not select a model or backend
per request. A document model, document payload backend, or local-path contract
is not part of the active API, DTO, service, or domain contract.

Knowledge Base, Skill, Role, Automation, and cloud Memory checkpoints use the
same tree version boundary. Memory remains local-first: its local working set
and journal own live edits, while a coalesced cloud checkpoint is a complete
tree version used for backup, history, and synchronization.

The Content Store does not recreate GitHub-compatible APIs, Git Smart HTTP,
branches, pull requests, issues, releases, collaborators, or repository tokens.
Consumer migration, protocol retirement, data backfill, one-way cutover,
post-cutover recovery, and final subsystem deletion are defined by the
[AGS Removal Architecture](./ags-removal.md).

## Ownership and contracts

The Content Store owns:

- `versioned_content`, including workspace scope, domain binding, head version,
  authority generation, quotas, and monotonic revision;
- immutable `content_versions` and their bounded path/digest projections;
- expected-head CAS and transactional `content_version_committed` outbox rows;
- tree payload staging plus its reconciliation and GC ledger.

The dedicated content-migration store owns exact legacy repository/commit
mappings and exact-target job cursors. Repository identity and full commit SHA
may appear only in that store and in the Role/Skill source exporters. The
Content Store core receives only a parsed bounded tree payload, an exact
binding/head, and a stable storage-neutral operation identity; its namespace,
contracts, and DTOs never expose AGS repository, branch, ref, or OID semantics.

Product domains store only `contentId`, `versionId`, an optional logical path,
and the authority generation needed by their contract. They never receive a
repository, branch, Git OID, filesystem path, object-store key, or mutable
payload reference. Domain authorization remains with the product owner and is
enforced at a narrow policy port.

The common operations are:

- create or resolve one exact VersionedContent binding;
- publish a complete immutable tree with expected head and generation;
- get/list exact versions with bounded pagination;
- query bounded path history and compare two exact versions;
- stream one exact file or bounded archive;
- consume idempotent version events.

Lineage creation uses either the model's complete default quota profile or a
complete internal-owner profile for retained versions, stored payload bytes,
and pins. Partial profiles are rejected, and the three values are persisted by
the initial lineage insert rather than repaired after creation. Generic HTTP
callers cannot select a quota profile.

## Tree payload backend

`TreePayloadBackend` manages immutable manifests and content-addressed blobs.
The built-in implementation uses an internal bare Git repository strictly as a
payload engine. Git repository identity, refs, commit OIDs, branches, paths,
and commands never escape the backend interface. An optional S3 implementation
must satisfy the same contract and may not add object-prefix scans to request
paths.

- A manifest is a complete sorted path index. Reads never enumerate a storage
  directory or object-store prefix.
- Paths are relative regular-file paths. Duplicate paths, traversal, absolute
  paths, symlinks, submodules, devices, sockets, and normalization collisions
  are rejected.
- File count, per-file bytes, total bytes, manifest bytes, and archive output
  are explicitly bounded by the operation.
- Blobs use lowercase SHA-256. Every open verifies digest and size before
  returning data; corruption fails closed.
- Manifest open verifies scope/version identity, sorted unique entries,
  aggregate limits, and root digest.
- Publication uses durable staging and an immutable version ref. An ambiguous
  database acknowledgement never triggers synchronous payload deletion.

The backend pins its trusted root for its lifetime and closes it explicitly
after in-flight operations drain. Callers never concatenate storage paths.

## Publication transaction

Version publication follows this order:

1. resolve the exact binding and verify registered native capability,
   authorization, authority state, and generation;
2. validate tree paths and declared file/byte/manifest limits;
3. allocate opaque version, outbox, and GC identifiers;
4. register pending GC and stage/verify the immutable tree payload;
5. lock the requested VersionedContent row and quota accounting;
6. verify `expectedHeadVersionId` and expected generation;
7. transactionally insert the immutable version/path projection, update the
   lineage revision and quota counters, enqueue the outbox event, retain GC,
   and, for ordinary publication, advance the head.

A stale head or generation returns conflict. Native storage, policy, or
authority failure fails closed; it never dynamically falls back to AGS.
Missing authority rows mean `legacy_active` and ordinary reads do not create a
default row.

The migration owner also has one separate internal-only detached publication
operation. It uses the exact live head only as a CAS fence and accepts an
independent parent version plus explicit generation; a non-empty parent must
belong to the same workspace, content, generation, and model. The same
transaction and quota accounting above insert the immutable version and path
projection, outbox, operation receipt, and retained GC record and increment the
lineage revision, but do not advance the live head. Receipt replay still
requires the head fence and returns the receipt-time revision/quota snapshot.
This operation is not a mode on ordinary publication, has its own closed-set
authorization action, and is absent from generic HTTP routes and DTOs. Content
Store does not receive repository IDs, commits, refs, tags, or DAG semantics;
product owners attach logical labels later through the existing pin operation.

## Rollout and capability registration

`MESHX_CONTENT_STORE_ENABLED` is the single global rollout switch and defaults
to false. Enabling it opens the tree payload backend but does not register a
generic HTTP capability, maintenance owner, workspace, or product binding.
Exact binding fixtures are constructed only in tests. A production binding is
native only when all conditions hold:

1. the global switch is enabled;
2. live AgentTeam Core membership verifies the authenticated subject against the
   claimed workspace;
3. the exact binding's owning-domain policy permits the requested closed-set
   action and scope;
4. the owning runtime registered that exact native capability;
5. persisted authority for the exact workspace/domain/binding is
   `native_active` at the expected generation.

Unregistered or missing-authority bindings remain under implicit
`legacy_active` ownership; ordinary legacy reads and writes do not materialize
an authority row. Persisted authority state and its generation are internal
consistency facts, not a second external feature switch. Product bindings are
added only by their owning domain migration.

### Role and Skill domain ownership

The resource runtime composes one shared tree core and authority store, then
gives Role and Skill separate authorized services. Each service rechecks live
AgentTeam Core workspace membership and its own current grant before Content Store
access. The shared runtime does not add a generic route or a cross-domain ACL.
When the global switch is disabled, the runtime constructs neither the shared
authority seam nor its product adapters: Role and Skill use their existing AGS
stores without authority lookups or authority-row side effects.

Role uses `domain=role` and the Role ID as the exact binding key. Every native
Role version stores the exact `contentId + contentVersionId`; each persisted
portable manifest item also stores its own complete
`contentId + contentVersionId + contentPath` projection and a matching opaque
content reference. Portable items are the Role definition, portable knowledge,
and inline Skills without a source resource ID. Knowledge Base, Memory, and
referenced-Skill items remain external resource references: they may omit the
local projection but must not carry a partial or mixed local projection. A
native read validates every item according to that classification before
opening any payload or writing realization state. Role lists read only Role and
manifest projections and never hydrate payloads or authority rows item by item.

Skill uses `domain=skill` and the Skill ID as the exact binding key. The Skill
row stores the active `contentId + contentVersionId`, while immutable
`SkillVersion` rows store the exact pair plus digest, file, byte, and Skill-count
summary. Native import, read, history, backup, restore, and rollback operate on
those immutable versions. History is limited to 100 rows per page, has a
10,000-row offset ceiling, uses `(created_at, id)` as a stable ordering, and
uses one lookahead row rather than counting the full history. Backup/pin reads
are capped at 256 entries. History and backup pagination validate multiplication
and addition bounds before any product, authority, or Content Store access, so
an oversized page cannot wrap into a negative offset or slice bound. Tree
operations retain the Skill archive's explicit file, per-file-byte, total-byte,
manifest-byte, and two-minute limits.

New Role and Skill IDs remain under implicit `legacy_active` ownership; normal
legacy traffic does not create a persisted authority binding. Native create is
unavailable until an owning domain can atomically consume a one-shot exact
reservation for the server-generated ID; a wildcard authority row never
captures future resources. Existing exact bindings may be activated only after
their selected versions are migrated and verified. Migration neither waits for
already accepted AGS work to drain nor adds a cross-system writer lease or dual
write. One bounded best-effort delta observation is followed by an exact-binding
legacy mutation cutoff that rejects every new AGS write. Only after the cutoff
is proven may the atomic one-way authority switch materialize exact
`native_active` ownership; work accepted before the cutoff may finish and be
absent from native history. After activation, new AGS mutations fail closed,
expected generation and Content Store CAS protect native writes, and every read
continues from the native lineage; AGS is not a fallback or a write rollback
owner.

Skill import, backup restore, and history rollback use a stable server-owned
Content Store operation key. If Content Store publishes successfully but the
Skill projection/activity transaction fails, a retry resolves the exact durable
publication receipt, publishes no second version, and idempotently finalizes
the active Skill and immutable SkillVersion projection with a compare-and-swap.
The receipt lookup is exact to workspace, content, and operation key; it never
enumerates content history or weakens owning-domain authorization.

Role public and hidden package mutations follow the same recovery rule. Their
stable operation identity is derived from the prior Role projection and
canonical package digest, or from the owning hidden operation key and request
digest. Before staging payloads, the Role owner looks up the exact authorized
publication receipt. A retry reuses that ContentVersion, reconstructs the
deterministic manifest references, and idempotently finalizes RoleVersion,
manifest, and product receipt state in one transaction. A failed product
transaction must not create a second staged or published ContentVersion. On a
receipt miss, the Content head must equal the exact ContentVersion referenced by
the request's prior RoleVersion before Content publication performs its own
compare-and-swap. On a receipt hit, the receipt version must still be the live
Content head. Product finalization independently compares and swaps the prior
active RoleVersion. A stale request therefore changes neither Content operation
state nor the Role projection.

Role history is available only while the exact Role binding reads natively. It
returns narrow immutable version summaries from the active RoleVersion's exact
Content lineage without per-row manifest or payload hydration, orders by
`(created_at DESC, id DESC)`, limits a page to 100 rows, caps offsets at 10,000
rows, and uses one SQL lookahead rather than a full history count. Pagination
arithmetic is validated before product or authority I/O. History rollback
requires both the expected active RoleVersion and one exact target RoleVersion
from the same Content lineage. It requires the exact Role binding to be native
at admission, derives a stable publication operation key from that baseline and
target, republishes the target package as a new immutable version, and applies
both the Content-head and active-RoleVersion compare-and-swap rules above.
Legacy-only RoleVersion rows remain migration input until their exact mappings
have been verified; they are not exposed as native history.

Any native Skill mutation verifies exact native authority and its generation
once during routing and admission, before choosing its persisted content or
activity identity. Authority cannot revert after that one-way admission;
Content Store generation/head CAS and the product transaction's own CAS provide
final consistency without a second authority check. The legacy path remains
unchanged until the one-way switch. Native activity uses
`contentstore://<content>/<version>`; only legacy Skills reach the repository
activity projector. Catalog refresh happens before the final pagination query
so newly projected commit time affects ordering and `updatedAt` in the same
request. With Content Store enabled, visible candidates are limited to 2,000
rows plus one SQL lookahead before one authority batch; over-limit requests
fail before repository projection. Skill activity uses the same visible-set
bound. With Content Store disabled, the existing unbounded legacy-visible
refresh followed by the final product query remains unchanged.

Native Role or Skill errors never retry through AGS. Agent-facing Skill
projections expose the exact content/version pair and omit repository, branch,
commit, and AGS sync-token data. Mixed agent access may issue an AGS token only
for the remaining legacy repository set. Native agent-binding first paint caps
both assignable Skill and workspace-binding SQL at 2,000 rows plus one
lookahead before performing one authority batch; it never chunks a workspace
scan into repeated authority calls. Ordinary Role and Skill catalog lists do
not resolve authority or payload state per item.

Role and Skill content projections belong to the resource-domain MeshX
database schema. GORM `AutoMigrate` adds the additive projection columns,
`skill_versions` table, and exact-scope indexes; `VerifySchema` checks their
columns and index order for both fresh databases and upgrades. They are not
owned by the AgentTeam Core TiDB bootstrap or its numbered migrations.

## Workers, diagnostics, and cost bounds

Outbox, GC, backfill, mapping verification, and diagnostics use indexed,
bounded claims. Every worker operation declares row/batch, file, byte, time,
cursor, lease, and concurrency limits appropriate to its input. A worker may
process only explicitly selected native rows or an explicit diagnostic target
set; it may not scan all workspaces or run backfill from startup/request paths.

Role and Skill migration is an explicit exact-target operation, not a
background agentteam. Each owning exporter names one workspace/binding, one product
source cursor, one legacy repository and full commit SHA, one expected Content
head, and the product tree limits. The shared migration worker is constructed
with an immutable Role or Skill domain; callers cannot select a domain or
claim the other owner's jobs. It has no pending-job list or workspace scan
operation and is not registered at process startup or through a generic HTTP
route. Each process admits one run per domain worker; the durable exact-target
job row adds a unique per-attempt lease token, a lease of at most five minutes,
a maximum of ten attempts, and the `pending → published → mapped → completed`
cursor. A run is capped at two minutes, 2,000 files, 16 MiB per file, 64 MiB
total, and an 8 MiB manifest; Role and Skill owners apply their smaller product
limits before source export.

The owning exporter computes the deterministic source root before Content
Store I/O, and the migration worker checks the stable Content publication
receipt before staging. Publication,
verified mapping, and product projection are separate durable cursor steps. A
retry after mapping skips AGS export and Content publication and repeats only
the owning product's idempotent, compare-and-swap projection. The Role or Skill
migration job does not fence AGS writes. Verified mapping never changes
authority; after one best-effort delta observation, cutover remains a separate
one-way exact-binding owner operation.

Owners that must import retained history without changing the active version
use the distinct detached exact-target worker entry point. Its request fixes
the live-head fence, independent parent, generation, complete create-time quota
profile, tree bounds, and stable operation key. Completion and every completed
replay re-read the one exact binding, imported version, and legacy mapping and
require the head to remain equal to the fence and all immutable facts to match.
A checkpoint retry resolves the durable receipt and does not create a second
version, outbox event, receipt, or retained-payload transition. These checks
are fixed-count exact-key reads plus work bounded by the requested tree; they
never enumerate workspace history or pending migrations.

Normal request cost scales with one requested binding/version and its declared
tree bounds, never with unrelated workspace content or full history. Ordinary
reads do not write defaults, repair mappings, or start migration work.

## AGS migration

Migration selects every product-visible revision referenced by the inventory at
full-backfill time or by the single best-effort delta observation, including
citations, backups, labels, and automation pins. The durable mapping key is
`ags_repository_id + ags_commit_sha`; the result records opaque content and
version IDs, tree digest, file/byte counts, verification state, and time.

Migration is an explicit admin operation with bounded batches, resumable
cursors, idempotent mapping, historical import, one best-effort delta
observation, and content/reference verification. Product references switch
only after selected mappings verify. A later AGS write in the documented
deployment loss window may be omitted. AGS data remains temporarily read-only
for audit or manual recovery, not as a write rollback target.

Migration durability is owned by AgentTeam's product migration store. AGS should be
read through existing interfaces; a missing capability may add only a narrow,
internal, bounded read/export seam for one immutable revision. Content Store
migration must not add an AGS-side generic receipt, replay queue, projection
worker, GC pin system, deterministic commit protocol, or reverse synchronizer.

Skill migration verifies the live repository ID, performs a depth-one clone of
only the canonical repository branch, verifies its exact HEAD commit, and
rejects symlink, gitlink, and other non-regular HEAD tree entries before reading
blobs. It projects the result through the existing SkillVersion
compare-and-swap owner without a user-visible activity event. Role migration
accepts only the active RoleVersion, resolves every portable manifest item
from one caller-selected final repository commit through a response reader
bounded for JSON/base64 overhead, verifies the repository ID and stored content
hashes, and atomically attaches the existing RoleVersion and all portable
manifest items to the exact ContentVersion. It creates no new RoleVersion,
realization, activity, or authority transition. External Knowledge Base,
Memory, and referenced-Skill items remain unprojected resource references.

## Rollout gate

The platform is dormant until a domain-specific migration proves:

- exact registered binding and workspace authorization;
- create/version/history/compare/file/archive and restart persistence;
- expected-head CAS, stale generation, cross-workspace denial, and corrupt
  payload failure;
- bounded outbox/GC/backfill/mapping/diagnostic behavior;
- additive rolling-upgrade-safe schema and no read-path default writes;
- a one-way cutover with a documented legacy-write loss window, an
  exact-binding legacy mutation cutoff that rejects stale AGS writes after
  activation, and post-cutover Content Store-compatible code rollback or
  forward-fix.

No compatibility facade may keep AGS and native storage as concurrent writers.