# SCIM 2.0 identity sync — L1 AppService target design

Status: **target design + prerequisite tracker; interim exception active.**
The neutral Runtime serving an L1 SCIM AppService is not yet carrying
production traffic for the capabilities SCIM needs (read, mutation), so the
AppService target remains blocked (§5, §6). The reviewed, time-boxed CRM
distribution exception in §6.1 is the only functional serving form meanwhile.
Supersedes the reverted in-process L0 attempt (#3707, reverted by #3882).

> Version note: audit derived from the current trusted base —
> `backend/go.mod:16-18,118` pins `nexuscrm-contract v0.2.1`, `wire-go v0.1.12`,
> `nexuscrm-api-host v0.14.0`. "Production-served" is judged by the production
> composition allowlist in `backend/internal/runtimeproduction/assemble.go`
> (used by `backend/cmd/runtime-api/main.go`), **not** by gRPC service
> registration in `process.go` — the latter is not sufficient for production
> serving. Re-confirm at build time.

## Why this exists

The first SCIM implementation (#3707) was built **in-process in L0**:
`nexuscrm-scim` called the records engine directly (`sm.Persistence.Insert`),
mounted routes via `RuntimeRouteSurfaces`, held a `ServiceManager.Scim` field,
ran an in-process reconcile goroutine, and added five `_System_Scim*` scope-floor
tables. That violates two standards and was reverted:

- **L0/L1 standard** (`docs/design/l0-l1-standard.md`): no second L0 consumer
  (fails T0); `/scim/v2` / URN / `filter` / `PATCH` / error grammar is pinned by
  RFC 7643/7644 (fails the T2 rename test) — L1 dialect, not L0 mechanism.
- **Serviceification boundary**
  (`docs/design/nexuscrm-serviceification-target-architecture.md` §7.4): L1
  executable code must be an **independent AppService** talking to L0 only via
  **neutral Runtime APIs**; it may not touch the platform DB, hold DB
  credentials, register in-transaction callbacks, or keep a second source of
  truth for platform records (§4.4, §5, §7.4).

## Target design (the correct shape)

SCIM becomes an **independent L1 AppService**, modeled on `sfcrm`
(`nexuscrm-app-packs/appservices/sfcrm/`):

- **Independent process/image**, not compiled into the L0 binary (arch §11.1);
  own `cmd/`, distroless image, `processhttp`-owned lifecycle/health/TLS.
- **Depends only on neutral contracts** (`nexuscrm-contract/pkg/runtime/*`) +
  the app-blind host client (`nexuscrm-api-host/runtimegrpc`, `processhttp`) —
  never `github.com/nexuscrm/*` core, never `backend`, never `database/sql`.
- **Public surface (L1-owned):** `/scim/v2` + Identity Sync admin UI/API — wire
  grammar, DTOs, `filter`, `PATCH`, error envelopes.
- **Record effects via neutral Runtime RPCs, not the DB** — when those
  capabilities are production-served (see §5): read `User` via SelectedRoot,
  find/list via Query, create/update/deactivate via Mutation.
- **State** (provisioning-link, journal, breaker window, quarantine,
  pending-manager) is the AppService's **own local store**, never platform DB or
  `_System_*` floor tables (arch §5). Removes the five floor entries — no
  amendment.
- **Auth:** user-initiated calls pass the end-user `Bearer` to
  `Runtime.IdentityContext.Exchange` (`NewIdentityExchangeClient`); transport to
  Runtime uses a workload token + `AppInstallationID`. IdP-push/reconcile have no
  end-user bearer → need a service/RunAs actor (§5, gap C).

## What this design deliberately does NOT do (resolving the earlier review)

- No new L0 seam (an L0 seam needs a real second consumer per T0).
- In the **AppService target form**, no in-process route mount and no direct DB —
  the surface is the AppService's and authz is via `IdentityContext.Exchange`.
  (The interim serving path recorded in §6.1 under arch §7.7 is the reviewed,
  time-boxed exception, and even it uses only the existing `RuntimeRouteSurfaces`
  seam + neutral `contract.Records` ports — never direct DB or a new L0 seam.)
- No pack-declared ConnectedApp `client_credentials`/`RunAsUserId` — forbidden to
  source-backed components today (`source-backed-connected-app-contract.md:135-142`).

## §5 — Prerequisites (audited 2026-07-24 against main)

The serving *contracts and compositions* have advanced since this ADR's first
audit: the production Runtime composition now allowlists and composes both the
read family and record mutation. What still blocks a production SCIM AppService
is no longer missing contracts — it is deployment, the no-end-user actor, and
second-app admission:

| # | Prerequisite SCIM needs | Status | Evidence |
|---|---|---|---|
| A | **Record mutation serving** (create/update/deactivate `User`) | **landed at code level**; not production-deployed (see A0) | `backend/internal/runtimeproduction/assemble.go:45-52` allowlists `recordmutationv1` (with outcome retention); `backend/internal/runtimeapi/process.go:134-158,333` composes and registers `RecordMutationsServiceServer`; the boundary guard now *requires* the runtime commands to compose it (`serviceification_record_mutation_substrate_test.go`, `wantUse: true`). The released sfcrm AppRelease still stages its mutation projection as non-serving (`appservices/sfcrm/README.md`) |
| B | **Read serving** for `User` fields | **landed at code level**; not production-deployed (see A0) | `assemble.go:45-51` allowlists `recordaccessv1` / `queryv1` / `relatedcountsv1` / `recyclebinv1` / `recordviewv1` / `recordattachmentsv1`; the sfcrm AppService consumes them (`appservices/sfcrm/cmd/sfcrm-appservice/main.go:37-70`) |
| A0 | **Production deployment of the Runtime serving path** | **blocked until D4** | the target-architecture header keeps phase-one production replacement BLOCKED until D3/D4 (legacy monolith is the only production deployment); `infra/runtime/README.md` marks the runtime topology as D1/test, "not an org deployment"; the L2 deployment repo carries no runtime-api / AppService manifests |
| C | **Service/RunAs actor** for no-end-user flows (IdP push carries no end-user bearer) | **still deferred** | `runtimeidentity/issuer.go:135-137` — `IdentityContext.Exchange` hard-requires an end-user bearer; `runtimeapi/security_admission.go:387-435` — every data RPC requires a delegation bound to a durable end-user session; no workload/service-principal branch exists. The only landed no-interactive-user token is the legacy `client_credentials` RunAs (`oauth_handler.go:1133-1214`), and source-backed self-declaration of it is forbidden (`source-backed-connected-app-contract.md:135-142,251-253`) |
| D | **AppService-usable async-work / scheduler** (reconcile loop) | not landed; **not needed for the push-only form** | no neutral schedule/subscribe client in `runtimegrpc`; a push-only receiver has no reconcile loop |
| E | **AppInstallation / HandlerBinding admission** for a new (scim) AppService | machinery landed for sfcrm; second-app onboarding **unbuilt and gated** | serving is bound to Catalog rows (`runtimeapi/catalog_serving.go:16-40`); no scim AppInstallation / HandlerBinding / AppRelease / workload binding / serving selection / source lock exists (`backend/distributions/appsource/source.go`); arch §13.2 gates 2,4 |

Net: a SCIM AppService can be *built* today to the same point sfcrm is at, but it
cannot *serve production* until the Runtime path is production-deployed (A0/D4),
a no-end-user actor contract lands (C), and scim admission artifacts exist (E) —
none of which has a committed date.

## §6 — What can proceed now vs. later

- **Now (non-functional only):** the SCIM AppService **skeleton** (`cmd/`,
  `processhttp` lifecycle, health) and the SCIM **wire library** (URN / DTO /
  `filter` / `PATCH` / error grammar) as pure, unit-tested code that makes **no**
  Runtime calls. This compiles against the landed contracts and is reusable, but
  it cannot read or provision anything.
- **Blocked for the AppService form:** production serving waits on the Runtime
  path being production-deployed (§5 A0 / D4), a no-end-user service actor
  (§5 C), and scim admission artifacts (§5 E). The AppService remains the
  target; the only sanctioned functional path meanwhile is the recorded interim
  exception in §6.1.

## §6.1 — Interim in-process serving (recorded exception under arch §7.7)

Although the mutation and read serving contracts have landed at the code level
(§5 A/B), a SCIM AppService still cannot serve production: the Runtime serving
path is not production-deployed until D4 (§5 A0), there is no no-end-user
service actor (§5 C), and second-app admission artifacts do not exist (§5 E) —
none with a committed date. The tenant needs SCIM provisioning in production
now. Under the interim capability exception mechanism of the serviceification
target architecture (§7.7), this ADR therefore records a **time-boxed
exception**: a functional SCIM receiver mounted **in-process in the CRM
distribution** (`backend/distributions/crm`) until a SCIM AppService successor
takes production ownership of SCIM provisioning (see the removal gate below).
It is NOT a reversal of the target (§4); it is a bridge to it, and it is
deliberately narrow:

- **Push-only scope.** The exception covers only the **push-only receiving
  form**: the IdP pushes creates/updates/deactivations to `/scim/v2` and the
  admin surface serves state for what was pushed. It includes **no reconcile
  worker or scheduled loop** (which the push-only form does not need, §5 D);
  adding reconcile behavior would require the async-work substrate (§5 D) and a
  new reviewed amendment — it is **not** authorized by this exception.

- **Existing seam only.** It registers via `RuntimeRouteSurfaces` — the same
  sanctioned product route-surface seam the shipping CRM distribution already
  uses for its other surfaces (CometD, instance-proxy, root-identity,
  related-list-candidates, gmail-mailbox). It adds **no new L0 seam**, no
  `ServiceManager.Scim` field, and **no `_System_Scim*` scope-floor tables**
  (the specific L0 welding that got #3707 reverted).
- **Neutral ports only.** All reads/writes go through `contract.Records` /
  `contract.Query` (engine-enforced CRUD / FLS / validation, and
  deactivate-revokes-sessions). **No raw DB, no `sm.Persistence.Insert`.**
- **Bounded principal.** Authn is a bearer validated by the existing API-token
  service, bound to a dedicated config-named service principal (RunAs) whose
  live profile is re-verified per request; a non-service token is refused.
  Profile facts come from config, never hardcoded.
- **One tenant-policy SSoT (L1 shape, L2 value).** The CRM L1 pack owns the
  `IdentitySyncSetting` object shape and its `Name="default"` singleton
  constraint. The PingCAP L2 pack is the only writer of that one record; the
  CRM runtime reads it with a system-context neutral query. There is no
  configuration UI, no configuration write route, no one-time environment
  migration, and no independently editable `Enabled` field. A record is
  serveable only when it names the default profile, least-privilege service
  profile, service username, and at least one exempt principal. Missing,
  unreadable, or incomplete policy fails closed before an SCIM mutation;
  mutations force a fresh read and re-bind the service principal. L2 must be
  applied only after the L1 schema is installed. Locale defaults and bounded
  breaker/token tuning also live in this record.
- **Release-paired L1 guard.** The app-packs backend checkout and the CRM
  per-product source lock advance together to the L1 revision that contains
  the object shape and locked `IdentitySyncSetting` public-write policy. The
  current source contract requires that policy at startup; accepted historical
  per-product revisions explicitly do not, so the bounded transition window
  remains bootable while the new SCIM route still fails closed until the new L1
  pack and its L2 record are applied.
- **Off by default (enforceable gate).** The whole surface is behind an explicit
  opt-in flag `NEXUSCRM_SCIM_INPROCESS_ENABLED` (default **false**): with it unset
  the `/scim/v2` and admin routes **do not mount at all**, so a default build is
  not a second compatibility path — an operator turns it on deliberately, per
  environment, for the transition window. This flag is deployment topology
  only (the single serving replica), not tenant configuration; it additionally
  fails closed when the L2 policy is incomplete.
- **Bounded blast radius.** The minted bearer carries a purpose scope (`scim`)
  required on the SCIM route and rejected by the host ScopeEnforcer on generic
  routes, expires (clamped TTL), and is rotated on re-mint (prior tokens revoked
  first; rotation failures abort the mint). Deactivations honor the exemption
  list and the layered volume bounds below, any of which halts further
  deactivations.
- **Layered deactivation safety — with each layer's guarantee stated
  exactly.**
  1. **Exemption list (absolute).** The never-deactivate exemption list is
     enforced from configuration on every request. It is per-request and
     stateless, so it holds exactly regardless of replica count or concurrency.
     Protected/break-glass accounts can never be deactivated by this path.
  2. **Per-replica cap (exact, atomic, synchronous).** Within a replica, every
     deactivation MUST acquire an atomic in-process reservation (a
     mutex-serialized counter with a rolling window) before the write, and the
     reservation MUST refuse at the cap. Concurrent valid requests therefore
     CANNOT race a count-then-write past the cap on a replica: the reservation,
     not a read-back count, is the per-replica enforcement, it is synchronous
     with the request, and no record edit can reset it. The consecutive-failure
     circuit breaker and half-open probe ride the same state and halt a
     misbehaving replica early.
  3. **Transactionally visible immutable bound (audit head rows,
     fail-closed).** Before EACH deactivation write the receiver MUST also
     count the SCIM service principal's `User.IsActive` transitions in the
     current window from the **audit head rows** of `_System_AuditLog` and
     MUST refuse the write at the cap. This source has exactly the properties
     the backstop needs, none of them assumed: (a) **transactionally
     visible** — the head row (with its `ChangesJSON` set of changed field
     names) commits inside the deactivation's own business transaction
     (`nexuscrm-audit/emitter_db.go` EmitTx), so there is NO asynchronous
     fan-out dependency, no pending/poisoned-outbox blind spot, and a
     crash-restart loop cannot launder the count — every committed
     deactivation is immediately visible to every replica's next check; (b)
     **append-only** — later edits add rows and never rewrite prior ones, so
     the cap can never be reset by subsequent activity; (c) **not subject to
     caller-visibility silent-drops** — it is read through the neutral
     privileged audit-log port (`contract.AuditLog`, which the platform
     already defines as the RLS/FLS-independent audit read surface), used by
     the serving layer solely for its own safety accounting, so a
     least-privilege service profile cannot produce a successful
     silently-zero count; (d) **conservative** — head rows identify changed
     field names but not direction, so reactivations of `IsActive` count
     toward the cap as well, biasing the bound toward refusal, never past
     it. The implementing change extends the existing audit-log port's
     request with additive, dialect-neutral actor/time filters (the port and
     its storage exist today; only the filter shape grows). Fail-closed
     behavior: an unavailable port, a query error, or an unreadable head row
     refuses the deactivation, and after each successful deactivation the
     receiver verifies the count advanced — if the audit trail is observably
     not recording, further deactivations latch off until an operator
     intervenes.
- **Single-replica activation (binding).** The interim path MUST be activated
  on at most one serving replica per environment; enabling
  `NEXUSCRM_SCIM_INPROCESS_ENABLED` on more than one replica is out of
  contract. The in-process form cannot itself detect peer replicas (that
  admission control is Catalog/serving-selection machinery, §5 E — another
  reason the AppService is the end state). Under the binding single-replica
  requirement the cap is enforced exactly by layer 2. Because layer 3 is
  transactionally visible, even an out-of-contract multi-replica activation
  is bounded at cap plus the writes concurrently in flight across replicas at
  the moment the cap is crossed — not per-replica-cap × replicas, and never
  open-ended; layers 1 and 3 are the guarantees that survive the
  misconfiguration.
- **Expiry, owner, self-disable (per §7.7).** This exception **expires
  2026-12-31** unless renewed before then by a new reviewed docs amendment.
  Owner: the CRM identity-sync maintainer (shuai.feng@pingcap.com). The
  implementation enforces the expiry itself: past that date the surface refuses
  to mount even with the activation flag set, so an unrenewed exception disables
  itself.

**Removal is tracked, not open-ended — and gated on the successor serving.**
The flag exists specifically so this path can be deleted in one commit, but
deleting it is authorized only when a SCIM AppService successor has been built,
admitted (§5 E), and accepted as the production owner of SCIM provisioning
under the migration's acceptance gates (arch §11.2 D3/D4). The substrate
prerequisites landing (§5 A0, C, E) make that successor *buildable*; the
ownership handover — not prerequisite landing — is what retires the interim
path, so retirement can never itself cause a provisioning outage. Track it
with the serviceification migration (arch §11.2 / §13.2). The shared dialect
wire library (`dialects/dialect-sf-crm/scim`) is used by both forms, so
migration to the AppService is a re-wire, not a rewrite.

## Recommendation

Track this against the serviceification migration (arch §11.2 / §13.2). The
AppService form becomes buildable-to-serve once the Runtime path is
production-deployed (§5 A0 / D4) and the run-as + admission substrate (§5 C, E)
lands; the interim path is retired when that AppService is admitted and takes
production ownership of SCIM provisioning (never merely when the prerequisites
land). Until then the recorded §6.1 exception is the only sanctioned functional
path, and it self-disables at its expiry unless renewed. Enabling SCIM in a live
environment (secret provisioning + first sync, which can deactivate real
accounts) remains a separate operator action, out of scope for the code work.

---

*This design file is the authoritative plan of record; the PR description
mirrors it.*