# AGS repository sync operations Repository sync is an AGS subsystem with a separate long-running worker. Run `gh-server` for the control API and at least one `ags-sync-worker` process for scheduled/manual Runs. Both processes use the same tenant configuration, database, and Git repository storage. Multiple workers are safe: a generation-checked Binding lease permits only one active Run per Binding. ## Data and execution model A Binding connects one immutable AGS repository ID to one immutable GitHub repository ID. Exactly one side is the authority; normal Runs only copy from authority to standby. Handoff performs a final forward sync, reverse dry-run, stable-digest check, and atomic authority epoch switch. Each ObjectLink stores both endpoint identities plus the last read-back verified revision/hash and mapper version. A normal Run inventories the authority, then selects only new, changed, missing, unreliable, or mapping-version-dirty objects as candidates. Unchanged managed objects are counted as `checkpoint_skipped`; their target carriers are not read and they do not create RunItems. A `full` audit selects every managed ObjectLink to detect standby drift, but still does not enumerate unrelated target-only data. The provider contract separates `InventoryIndex` from candidate `ReadBatch`. The current AGS/GitHub adapters derive the index from their complete paginated list collectors and strip payloads; resource-specific list projections can be added later without changing the planner or durable model. Each RunItem is the durable per-object intent, audit, idempotency, and crash recovery unit. Physical execution groups compatible items by dependency phase, resource type, action, item count, and serialized byte count. Defaults are 500 items and 8 MiB. The sync engine and legacy importer share the adaptive shard and bounded-apply primitives; the importer remains available as a separate one-time migration command. - Empty standby: complete source inventory becomes `create` RunItems directly, without a per-object target join, then bulk/bounded apply and verification. GitHub's provider-created default labels do not make a new repository non-empty; an actual label-key collision is still preserved and reported. - First non-empty baseline: complete current-state diff preserves existing adoption/conflict reporting and sends differences through the same bulk executor. - Later Runs: checkpoint candidate selection reads only candidate source detail and linked target carriers. A requested full audit reads all managed carriers, never the whole unmanaged standby. Git bootstrap does not move commits through JSON or one ref at a time. The first physical ref batch performs one native fetch of writable heads/tags and pushes missing refs in bounded atomic groups (500 refs by default); later logical ref batches reuse that completed bootstrap and still verify every RunItem. GitHub `refs/pull/*/head` are materialized as writable `refs/heads/ags-sync/pr//head` shadow branches. Same-repository PRs keep their original writable branch; fork and deleted-head PRs use the shadow while ObjectLink metadata retains the original GitHub head repo/ref for round trips. LFS transfers are grouped by OID and happen before ref updates. For a non-empty target or later Run, Git transport initializes a small bare transfer repository and fetches only the refs in the changed physical batch; it does not clone every branch of a TiDB-sized repository for a one-ref update. The fetched OID must still equal the planned source revision before the conditional atomic push is attempted. Supported durable data includes Git refs and objects, LFS, Wiki Git refs, labels, milestones, Issues, Issue/PR comments, PRs, reviews, review requests, release metadata/assets, provenance, and archive-only reaction metadata. Provider/runtime configuration such as permissions, branch protection, secrets, workflow history, Checks, deployments, stars, and notifications is not copied. Provider identities are not treated as user identities. Without an explicit account map, assignees/reviewers are not assigned by matching login text. Historical approvals and review requests are represented as attributed or projected conversation records; GitHub review-request projections include a read-back-comparable visible body so a Run cannot claim convergence from DB metadata alone. Historical merged PRs that GitHub cannot recreate naturally use a closed Issue carrier rather than manufacturing a merge commit. ## Deployment The container image contains both binaries plus `git-lfs`. Run two workloads from the same image: ```text control plane command: gh-server worker command: ags-sync-worker ``` Set `ENABLE_REPOSITORY_SYNC=true` for both. The worker needs outbound HTTPS to the configured GitHub API/Git endpoints and read/write access to AGS Git storage. Use the same `AGS_CONFIG_FILE` for structured multi-tenant deployment, or the same legacy tenant environment variables in single-tenant deployment. Required GitHub App repository permissions are: - Metadata: read - Contents: read and write - Issues: read and write - Pull requests: read and write - Workflows: read and write (required when Git pushes workflow files) Install the App only on repositories that may be bound. Store the PEM private key in the secret-manager-materialized structured tenant file as `github_private_key`, or inject `REPOSITORY_SYNC_GITHUB_APP_PRIVATE_KEY` for the legacy single-tenant configuration. Database rows contain only installation connection references, never installation tokens or the private key. For self-service `Connect GitHub`, configure the GitHub App's setup/callback URL as the tenant API callback `/api/ext/v1/sync/github/install/callback` and enable user authorization during installation. AGS first sends the user through GitHub user authorization and persists the verified GitHub identity against that individual AGS user. The user may then grant repository access through GitHub's App installation screen. Before redirecting, AGS records the process-visible installation-ID baseline in the one-time request. GitHub's OAuth callback can omit the newly created installation ID and `/user/installations` can lag immediately after an organization install, so the callback uses the same short-lived user token to wait a bounded interval for an installation not present in that baseline. Only then is the grant successful. New installation grants are merged with the user's existing grants; an explicit user reauthorization refresh replaces the grant set with GitHub's current authoritative view. The user token is then discarded, and AGS persists only non-secret identity, installation metadata, and user-to-installation grants. One AGS user can never inherit another user's picker access merely because they share a tenant. Existing Bindings continue to use short-lived App installation tokens and do not depend on a browser user's token. Important settings: ```text repository_sync.enabled repository_sync.github_api_url (default https://api.github.com) repository_sync.github_web_url (default https://github.com) repository_sync.github_app_slug repository_sync.github_app_id repository_sync.github_private_key repository_sync.github_client_id repository_sync.github_client_secret repository_sync.poll_interval (default 5s) repository_sync.lease_ttl (default 2m) repository_sync.full_audit_interval (default 168h / weekly) repository_sync.max_concurrency (default 8) repository_sync.max_asset_bytes (default 2 GiB) ``` The public control surface is `/api/ext/v1/sync/*`; it is intentionally not a GitHub-compatible `/api/v3` API. `Sync now` creates a Run and returns quickly; the worker executes it asynchronously. Run status and the per-resource report are read through the Run endpoints. The front end uses: - `GET /sync/github/installations` for the authenticated AGS user's connected GitHub identity and only the App installations that identity explicitly authorized. Installation responses include the GitHub management URL so authorization failures can link users to the correct access settings without exposing a token; - `GET /sync/github/repositories` for the de-duplicated repositories available through that user's authorized installations. Each repository carries the connection ID required when creating a Binding; - `DELETE /sync/github/account` to remove only the authenticated user's setup authorization. It does not uninstall the App or break existing Bindings; - `POST /sync/github/install/start` to create a short-lived one-time state and return the first-party GitHub App installation URL; - `POST /sync/github/authorize/start` to connect or refresh the authenticated user's GitHub identity and recover App installations that already exist on GitHub but are not yet linked to that AGS user; - `GET /sync/github/install/callback` as the public, state-validated GitHub App authorization callback; - `GET /sync/overview` for the global management page. It returns authorized Bindings, connection health, each latest/active Run, and batched active RunItem progress in one response instead of one request per Binding; - `GET /sync/bindings?ags_repository_id=...` for the repository settings page; - `POST /sync/bindings/{id}/runs` with optional `audit_mode=checkpoint|full`; - `GET /sync/runs` for administrator-filtered, newest-first activity across Bindings, with status filters and a descending `after_id` cursor; - `GET /sync/runs/{id}` for summary, durable execution stage, and final report; - `GET /sync/runs/{id}/progress` for lightweight total and per-object-type state counts without loading every RunItem; - `GET /sync/runs/{id}/items` for cursor-paginated object changes/errors; - `POST /sync/runs/{id}/recover` for read-back-only recovery of an unknown external result. Run stages are `queued`, `checking_access`, `inventory_source`, `inventory_target`, `planning`, `applying`, `verifying`, `finalizing`, and `completed`. They are persisted independently from terminal Run status so the web UI never has to guess whether a long repository inventory is still active. Final problem reports include stable `error_code`, `problem_type`, `suggested_action`, and `retryable` fields. The UI maps these to actions such as granting repository access, reviewing a collision, waiting for a rate limit, or invoking read-back recovery. New Bindings and Bindings after authority handoff have `sync_enabled=false`. Manual dry-run/apply remains available. The scheduler can be enabled only after a successful `full` apply in the current authority epoch and when no Run or unresolved external outcome exists. If the standby is more public than the authority, create/handoff/apply requires `confirm_visibility_exposure`. The stored confirmation is hashed over both immutable repository identities, direction, and current visibility values. A later visibility change invalidates it, blocks the Run before any write, and turns scheduling off; PATCH the Binding with a fresh confirmation before the next manual apply. ## Failure and recovery The worker persists intent before external writes and verifies every object by read-back. On restart it reconstructs physical batches from unfinished RunItems. Create retries first recover by ObjectLink marker/natural key; ambiguous or result-unknown writes become blocked instead of creating a second object. An `apply_result_unknown` item is a Binding-wide fence: it blocks new Runs, Handoff, schedule enablement, and Binding close. Recovery requeues only the ambiguous items and performs target read-back; if the outcome still cannot be proven, no external write is retried. Git ref updates/deletes use force-with-lease. Source changes during a normal Run produce `completed_with_pending_changes` and enqueue another Run; during Handoff they prevent the epoch switch. Delete intents re-read the linked authority identity immediately before apply; an object that reappeared after planning supersedes the delete. Plans, per-object completion, exceptional termination, final Run completion, and the Handoff epoch switch are generation-checked against the active Binding lease. A stale worker therefore cannot finish or fail a Run after another worker has claimed it. Endpoint locators are refreshed after an immutable-ID-validated rename, while ObjectLinks retain both sides' immutable ID, number, URL, representation, provenance metadata, and verified checkpoints. A queued Run with an old mapper version is superseded before execution. If an older worker may already have an external intent in flight, the Run is blocked instead; lease checks also fence every subsequent write on mapper version. AGS standby writes are locked while GitHub is authority. GitHub cannot be forcibly frozen by AGS, so a GitHub-to-AGS Handoff still requires an explicit maintenance window and operator confirmation that authority writes stopped. Scheduled ticks are coalesced and require `sync_enabled=true`: while a Binding has an active Run its due time remains pending; the first scheduler pass after completion creates one Run and advances to the next future cron occurrence. Cancellation is checked before inventory/apply and again before final convergence or epoch switch, including zero-difference Runs. Scheduled Runs normally use checkpoint mode. Once `repository_sync.full_audit_interval` has elapsed since the last successful full apply, the next coalesced scheduled Run uses full audit. The default is weekly, not hourly; operators can also request a full audit explicitly through `Sync now`. The existing `ags-migrate` importer is intentionally retained. Repository sync does not depend on `ImportJob`/`ImportBatch`, but both paths share `internal/sync/bulk` limits/sharding and `internal/sync/gitmirror` native Git bootstrap primitives. This is the project decision that overrides the proposal's importer-removal paragraph.