# Drive9 Git Fast Workspace Design ## Background Drive9 has two high-priority requirements for agent workspaces: - `git clone` into a mounted directory must be fast, without checking out the full working tree into Drive9's generic file tables. - After a sandbox or container is replaced, the agent's partially edited working tree and `.git` state must be recoverable from Drive9 so `git status`, `git add`, `git commit`, and `git push` can continue. This is therefore not a pure generic filesystem optimization. It is a Git-aware fast workspace layered on top of the generic Drive9 FUSE semantics. The design intentionally reuses Git's own object model: the clean tree is represented by a Git manifest, dirty working tree changes are represented by a Drive9 overlay, and `.git` is kept in the local overlay with backend checkpoints. ## Goals - `drive9 git clone --fast ` registers only the HEAD tree manifest and does not check out clean file content into `file_nodes`. - The default local `.git` object database is a full clone, while the working tree checkout is virtual. Build-time reads are served from local Git objects instead of Drive9 file rows. - `drive9 git clone --fast --blobless` is an opt-in mode that keeps clone latency low with a partial local clone, then hydrates clean blob content out of the FUSE hot path. - Blobless mode avoids build-time and Git-porcelain lazy-fetch amplification by hydrating clean blobs into both a hidden local source tree cache and the local `.git/objects` database. - Working tree changes such as edit/create/delete/chmod/symlink are persisted to the Drive9 backend overlay. - `.git` keeps local disk read/write performance while lightweight state and small local-only Git objects are checkpointed to Drive9 for cross-sandbox recovery. - `git add`, `git commit`, and `git push` should remain as close as possible to the native Git workflow, without introducing a separate push API. - `drive9 git worktree add --fast` should create linked worktrees without native checkout writes into FUSE. - `drive9 git worktree remove --fast` should remove a linked fast workspace without recursively unlinking every virtual clean file. ## Non-Goals - Do not add a full artifact-style command suite or turn this into a general content-addressed checkout tool. - Do not write clean tree file content into `file_nodes` or `contents`. - Do not persist the full Git object database in Drive9. Cross-sandbox recovery may depend on the original Git remote, such as GitHub, to repopulate clean objects. - Do not store remote clean blobs in Drive9. The local `.git/objects` hydrate is a rebuildable cache; the only Git objects Drive9 may store are local-only objects needed to restore staged or local Git state. - Do not treat the hidden hydrated clean-tree cache as authoritative state. It is local, rebuildable, and may be discarded with the sandbox or local root. - Do not introduce local SQLite. Local state continues to use the existing local overlay, shadow files, and `journal.wal`; authoritative state lives in Drive9 backend Git workspace tables. - Do not implement complete semantics for complex Git features in the first version, such as optimized submodule/LFS handling, merge conflict assistance, or automatic remote branch synchronization. ## Data Model `git_workspaces` - One row represents one fast workspace. - Key fields: `workspace_id`, `root_path`, `repo_url`, `remote_name`, `branch_name`, `base_commit`, `head_commit`, `mode`, and `status=active`. - `mode=fast` is the default full-object local clone; `mode=fast-blobless` is the opt-in blobless local clone. - `root_path` is unique and maps to the repo root directory inside the mounted tree. - `workspace_kind=main` is a primary fast clone. `workspace_kind=linked` is a Git linked worktree with its own clean tree manifest and dirty overlay. - `common_workspace_id` points from a linked worktree to the main/common workspace that owns the shared Git common dir. - `worktree_name` is the stable name under the common Git dir's `worktrees/` directory. - `gitdir_rel` stores the linked gitdir path relative to the common `.git` directory, normally `worktrees/`. `git_workspace_tree_nodes` - Stores the base/head commit tree manifest. - Paths are relative to the workspace root and do not have a leading `/`. - `path_hash` and `parent_path_hash` are SHA-256 hex hashes used for TiDB/MySQL indexes so long Git paths do not exceed key-length limits. The original `path` and `parent_path` remain authoritative for exact semantics and API responses. - Includes `kind`, `mode`, `object_sha`, and `size_bytes` for directories, files, symlinks, and submodules. - Clean tree file content is not stored in Drive9's generic file content tables. - `size_bytes` is still populated without relying on checkout file content. Manifest generation uses `git ls-tree -r -t -z HEAD` and must not use `git ls-tree -l`; `-l` asks Git to print blob sizes and is unsafe if a future mode uses a partial clone again. - For GitHub repositories, the CLI fills blob sizes through the GitHub Trees API. If the recursive response is truncated, it falls back to walking trees level by level. This preserves accurate `stat` and `git status` sizes while still avoiding blob downloads during clone. - For non-GitHub repositories, unavailable GitHub API responses, or temporarily missing sizes, `size_bytes=-1` means unknown size. FUSE reports size 0 to the kernel attr path as a fallback, but real read/write/truncate/rename-copy paths load the base blob on demand and update the local inode size from the actual content length. This avoids treating unknown-size files as empty files. `git_workspace_overlay` - Stores working tree changes relative to the clean tree. - `op=upsert` means a file or directory was created or modified; `op=whiteout` means a clean tree entry was deleted. - `path_hash` is the SHA-256 hex hash used as the durable key; the original `path` is still stored and checked on lookup. - File payloads are currently stored inline in `content_blob`; large-file support can later extend this to S3/storage refs. - Directory creation is stored as an overlay entry with `kind=dir`. - Dirty overlay entries include file content and metadata changes such as chmod, directory creation, symlink creation, rename copy/whiteout, and delete whiteouts. FUSE applies them to the in-memory workspace view immediately, while the remote backend write follows the same `--durability` / write policy used by ordinary files. `git_workspace_git_state` - Stores `.git` directory checkpoints. - The checkpoint intentionally excludes Git object databases such as `.git/objects` and `.git/modules/*/objects`. - The current format is `storage_type=tar.gz-no-objects`, with the lightweight payload in `content_blob`. - When sandbox B mounts the same drive, FUSE recreates local `.git` from `repo_url`, applies any Drive9 local-only object packs, then overlays the Drive9 checkpoint to restore refs, index, config, logs, and other lightweight state. `git_workspace_object_packs` - Stores inline Git packfiles for local-only objects that are not available from the remote. - `pack_id` is the SHA-256 checksum of the pack bytes. Uploads are idempotent by `(workspace_id, pack_id)`. - V1 stores packs inline in `content_blob`; no S3/storage-ref path is used. - Per local-only blob cap: 5 MiB. Total inline pack cap: 256 MiB. - Oversized staged objects are not packed. Their working-tree content still survives through `git_workspace_overlay`, but staged state is downgraded to unstaged on restore. - Local refs, stash, or reflog state that would require omitted oversized objects is dropped from the checkpoint rather than restoring broken refs. `file_nodes` - Still represents ordinary Drive9 filesystem directories and files. - Clean or dirty Git file content under a fast workspace should not leak into `file_nodes` file rows. - The repo parent directory and repo root directory may exist as ordinary directory rows. Hidden local clean cache - Blobless workspaces use a hidden, rebuildable cache under `/git-workspaces///`. - `tree/` stores a materialized clean source tree for fast build-time reads. - `blobs//` stores read-through clean blob cache entries when a file is read before hydrate finishes. - `hydrate.json` records the last hydrate provider, tree status, object database status, file count, byte count, object count, skipped/mismatch/fallback counts, duration, and error string. - The cache key includes workspace ID and head commit so old clean content is not reused after a manifest change. Coding-agent local overlay policy - The coding-agent mount profile routes heavyweight local state and generated output to `/overlay` instead of Drive9 backend storage. - Default local-only paths include VCS state (`.git`, `.hg`, `.svn`), dependency directories (`node_modules`, `.venv`, `.pnpm-store`), build outputs (`dist`, `build`, `target`, `coverage`), temporary/cache directories (`tmp`, `.tmp`, `.cache`, `.turbo`, `.next/cache`, `.vitepress/cache`), and tool-specific generated output such as `.tmp-api-extractor`. - These local-only paths are still merged into FUSE directory listings with tracked Git workspace entries, so generated directories under a tracked source directory remain visible to local build tools without being uploaded to Drive9. - Local-only dependency and generated-output files are a rebuildable performance layer. Their ordinary FUSE `Flush` path does not force `fsync`; it refreshes local inode metadata only. Explicit `Fsync` still syncs the local file. - Lightweight `.git` state is checkpointed asynchronously and coalesced per workspace. Foreground `Flush`, `Fsync`, `Release`, `Rename`, and `Unlink` on local `.git` files only perform the necessary local filesystem operation and schedule a checkpoint; `FlushAll`/unmount drains pending checkpoints. - Linked worktree `.git` is a file, not a directory. It is local-only too. On restore, Drive9 rewrites it with a relative `gitdir:` target that points through the mounted path back to the common workspace's `.git/worktrees/` directory, so later Git metadata writes still pass through FUSE and can be checkpointed. - Transient Git lock files such as `.git/index.lock`, `.git/packed-refs.lock`, `.git/HEAD.lock`, and `refs/**/*.lock` are never checkpointed. The durable state is captured after Git atomically renames the lock file to its final destination. - `.git/objects` remains a local object database/cache and is not checkpointed directly. ## Clone Flow 1. The user first mounts Drive9: ```bash drive9 mount --mode=fuse --profile=coding-agent --local-root --cache-dir --durability=write-sync :/ ``` 2. The user runs one of: ```bash drive9 git clone --fast https://github.com/org/repo.git //repo drive9 git clone --fast --blobless https://github.com/org/repo.git //repo drive9 git clone --fast --blobless --hydrate=sync https://github.com/org/repo.git //repo ``` Blobless hydrate modes are: - `--hydrate=background` (default for `--blobless`): return after workspace registration and hydrate the clean tree plus local Git object database in the background. - `--hydrate=sync`: materialize the clean tree and local Git object database before returning, useful for deterministic diagnostics and benchmarking. - `--hydrate=off`: preserve the pure lazy-fetch behavior. 3. The CLI runs a no-checkout clone under the mounted path: ```bash git clone --no-checkout git clone --filter=blob:none --no-checkout # with --blobless ``` `.git` is routed to local disk by the coding-agent local overlay and does not enter Drive9's generic file tables. In default mode the local object database is complete; in blobless mode the clone starts with a partial object database and hydrate later fills clean blobs from the hidden clean tree cache into `.git/objects`. 4. The CLI reads `HEAD`, branch, and `git ls-tree -r -t -z HEAD`, generates the tree manifest, and fills file sizes according to the `git_workspace_tree_nodes.size_bytes` rules. 5. The CLI initializes the Git index: ```bash git read-tree --reset HEAD ``` This tells Git the clean tree object IDs without checking out file content. The CLI also sets repo-local performance knobs such as `gc.auto=0`, `maintenance.auto=false`, `core.untrackedCache=true` when supported, and `core.splitIndex=true` when supported. Automatic foreground GC is disabled because local `.git/objects` is a rebuildable cache and Git porcelain should not unexpectedly auto-pack while FUSE is servicing `.git/index.lock` operations. 6. The CLI calls Drive9 APIs: - upsert `git_workspaces` - replace `git_workspace_tree_nodes` - archive the local overlay `.git` directory without object databases and upsert `git_workspace_git_state` 7. In blobless mode, the CLI starts `drive9 git hydrate ` in the background unless `--hydrate=sync` or `--hydrate=off` was selected. Hydrate runs in two stages: - Materialize `tree/`. GitHub repositories use codeload tarballs for batch hydrate; non-GitHub repositories fall back to a separate Git index plus `git checkout-index` into the hidden local cache. - Populate local `.git/objects` from the hydrated tree using `git hash-object -w`, verifying each generated blob SHA against the manifest. Mismatches or unsupported entries are counted and may fall back to Git's own object lookup. 8. After FUSE rediscovers the workspace, directory listings come from the synthetic view of `git_workspace_tree_nodes` plus `git_workspace_overlay`. FUSE also best-effort starts hydrate for `mode=fast-blobless` workspaces so replacement sandboxes still warm their local cache. 9. In the coding-agent mount profile, FUSE treats repository-ignored generated paths as local-only. The policy first applies explicit local/remote patterns; for otherwise remote-default paths inside a Git workspace, it runs cached `git check-ignore` against the hidden hydrated clean tree and the local `.git` state. Paths that Git would ignore are routed to the local overlay, while tracked clean files and durable Git overlay entries keep their normal Git workspace semantics. ## Fast Worktree Flow V1 uses explicit Drive9 commands and does not transparently intercept native `git worktree add/remove`: ```bash drive9 git worktree add --fast [-b ] [--detach] [--blobless] [--hydrate=auto|background|sync|off] [] drive9 git worktree remove --fast [--force] ``` `drive9 git worktree add --fast`: 1. Resolves both paths through Drive9 mount metadata and requires them to be in the same mount. 2. Looks up `` as an active `workspace_kind=main` fast workspace. 3. Runs native Git only for linked worktree metadata: ```bash git -C worktree add --no-checkout [-b |--detach] git -C read-tree --reset ``` The command intentionally avoids a checkout. It creates only the worktree root, the linked `.git` file, the common `.git/worktrees/` metadata, and the Git index. 4. Generates a clean tree manifest with `git ls-tree -r -t -z `, enriches sizes through the GitHub Trees API when possible, and registers a new `git_workspaces` row with `workspace_kind=linked`. 5. Stores linked `.git` state as a separate `git_workspace_git_state` row and also checkpoints the common workspace `.git`, because Git updated common metadata such as `worktrees/` and refs. 6. Uses the same hydrate behavior as fast clone. A linked workspace inherits blobless behavior from a blobless common workspace; `--blobless` is a validation flag rather than a way to convert a full common object database into a partial one. `drive9 git worktree remove --fast`: 1. Resolves `` to a linked fast workspace. 2. Refuses removal when `git status --porcelain` reports staged, unstaged, or untracked changes, unless the caller passes `--force`. 3. Removes the common `.git/worktrees/` metadata from the local overlay and runs `git worktree prune` as best-effort local cleanup. 4. Checkpoints the common workspace `.git`. 5. Marks the linked `git_workspaces` row deleted and removes the linked local overlay root. 6. Best-effort removes the empty mounted worktree root. It does not generate clean-tree whiteouts and does not recursively unlink virtual manifest files. Linked worktree restore: 1. FUSE matches workspaces by longest `root_path`, so the main repo and each linked worktree get independent runtimes. 2. Access to a linked worktree `.git` file, or to the common path `.git/worktrees//...`, triggers linked restore. 3. FUSE restores the common workspace `.git` first from `repo_url`, common object packs, and the common Git state checkpoint. 4. FUSE recreates the linked `.git` file, restores the linked gitdir under the common `.git/worktrees/` directory, unpacks linked local-only object packs into the common object database, and extracts the linked lightweight Git state. 5. Dirty working tree files restore from the linked workspace's own `git_workspace_overlay`, isolated from the common workspace and other linked worktrees. Performance coverage for issue #476: - `git worktree add --fast` avoids checkout writes for every clean source file. - `git worktree remove --fast` avoids recursive unlink/rmdir over virtual clean tree entries. - `git status` and `git add` read tracked metadata from the manifest and overlay instead of round-tripping to generic remote `file_nodes`. - `.git` probes, lock files, linked gitdir files, and common worktree metadata stay on local overlay hot paths with coalesced checkpoints. - The expected perf-counter movement is lower aggregate `remote stat`, `remote mutation`, `remote write`, and FUSE unlink time for worktree smoke workflows. ## Read/Edit/Add/Commit/Push Flow Read a clean file: - FUSE looks up the workspace manifest and constructs a virtual inode. - On read, FUSE first checks the hidden materialized clean tree, then the local blob cache, and only then falls back to `git cat-file blob `. - If fallback reads a clean blob, FUSE writes it to the local blob cache with singleflight protection so concurrent reads of the same object do not fan out into repeated Git lazy fetches. - In blobless mode, hydrate is intended to move remote blob download off the build and Git command hot paths. If hydrate has not finished, fallback Git lazy fetch still works for correctness. - File content is not written into `file_nodes`. Edit a file: - When a tracked clean file is written, FUSE stores the new content in a `git_workspace_overlay` `upsert` entry. - New files and directories also enter the overlay. - In the coding-agent profile, untracked paths matched by the repository's `.gitignore` are local-only. This keeps repo-specific build outputs such as generated web assets, package build directories, and temporary tool output off Drive9 without hand-maintaining per-repo mount patterns. - Deleting a clean file writes a `whiteout`. - Writable opens for tracked files preload from the Git workspace manifest or dirty overlay entry into a hidden local dirty mirror under `LocalRoot`. FUSE serves the write handle from that local file and converts it into a Drive9 overlay snapshot on flush/release. This is especially important for append-style writes: local file semantics preserve the complete current file content before appending the new bytes, while Drive9 still owns the durable cross-sandbox overlay. - Clean tracked files may use the kernel page cache because their content is immutable for a given manifest entry. Dirty overlay files are different: they are the authoritative edited working tree view, so read-only opens for content-changing overlay entries bypass the kernel page cache. This keeps `git add` and index refresh from reading stale clean pages or seeing size/content mismatches after append-heavy edits, while preserving cache hits for clean build reads. - Dirty overlay file content and metadata use one durability policy: - `writeback` with interactive durability updates the local runtime overlay before returning and commits the backend `git_workspace_overlay` entry on an ordered background path. `FlushAll`/unmount drains pending overlay writes. - `close-sync`, `write-sync`, and strict durability wait for the backend overlay write before reporting success on the relevant FUSE operation. - Background overlay writes are ordered per mount, so rename copy/whiteout sequences reach the backend in the same order they became visible locally. `git add`: - Git reads the clean+overlay synthetic working tree through FUSE. - `.git/index`, objects, logs, and related files are written to the local overlay. - Writable lightweight `.git` handles schedule a coalesced checkpoint instead of blocking the close/rename/unlink hot path. The checkpoint packs local-only Git objects first, then writes `git_workspace_git_state`. - Git lock files are local-only and transient; closing or unlinking them must not run checkpoint work. This prevents `git status`, `git diff`, and `git add` from paying `rev-list`/object-pack cost on every `.git/index.lock` lifecycle. - `.git/objects/**` is not checkpointed directly and does not enter Drive9 backend storage. Clean objects are rebuilt from remote or hydrate cache; local-only objects needed by staged state or local refs are packed during the next lightweight `.git` checkpoint. - Read-only `.git` handles do not checkpoint, which prevents commands such as `git status` from repeatedly uploading the full `.git` archive. - If a staged blob is larger than 5 MiB, restore downgrades that staged state to unstaged. The file content remains durable through the Drive9 dirty overlay. `git commit`: - Git updates local `.git` through its native workflow. - When the working tree overlay matches the Git index after commit, `git status` reports clean. - The first version does not automatically write the new commit tree back as the new clean manifest or clear the overlay. The overlay remains relative to the original base tree. `git push`: - Git pushes natively from the local `.git`. - Drive9 does not participate in the push protocol. ## Sandbox Replacement Flow 1. Sandbox A mounts Drive9 and fast-clones a repo. 2. The agent edits files, runs `git add`, `git commit`, and `git push`, then leaves another uncommitted change. 3. A unmounts or stops. 4. Sandbox B mounts the same Drive9 path with a new local root and cache. 5. When B accesses the repo path, FUSE loads from the backend: - `git_workspaces` - `git_workspace_tree_nodes` - `git_workspace_overlay` - `git_workspace_git_state` 6. FUSE runs a no-checkout clone from `repo_url` into B's local overlay: - `mode=fast`: full clone. - `mode=fast-blobless`: blobless partial clone. 7. FUSE downloads and unpacks local-only object packs, then extracts the lightweight `.git` checkpoint over the local clone. 8. B sees working tree = clean tree manifest + durable overlay; `git status`, `git log`, and file content are restored to A's last persisted state, subject to the explicit oversized staged/local-ref downgrade rules. For linked worktrees, B restores the common workspace first, then each linked gitdir on demand. Edited files come from the linked workspace overlay, staged state comes from the linked `.git` checkpoint plus small object packs, and local commits/refs are recovered through the common and linked checkpoints together. ## Local State and SQLite The current implementation does not introduce SQLite. Local fast workspace state includes: - `/overlay/.../.git`: local `.git`. - `/overlay/.../`: coding-agent local-only files for paths matched by static local-only patterns or repository `.gitignore` rules. - `/git-workspaces///tree`: hidden hydrated clean source tree cache. - `/git-workspaces///blobs`: hidden read-through clean blob cache. - `/overlay/.../.git/objects`: local Git object database. In blobless mode hydrate fills clean blobs here from the hidden tree cache so `git status`, `git diff`, `git add`, and `git commit` do not fan out into promisor lazy fetches. - `//journal.wal`: the existing FUSE write/cache journal. - shadow/cache files used by the existing write-back and strict durability mechanisms. Authoritative cross-sandbox working tree state lives in backend DB Git workspace tables. Local files are the current sandbox's performance layer and restore target. Clean Git object content is recovered from the Git remote or rebuilt into `.git/objects` from a local hidden hydrate cache; local-only Git object content may be recovered from inline Drive9 object packs under the v1 size limits. ## Dev E2E Validation Validated in the dev environment on 2026-05-27: - endpoint: `http://example-k8s-endpoint.local` - repo: `https://github.com/example/repo.git` - run_id: `fastclone-e2e-YYYYMMDDHHMMSS` - tenant_id: `TENANT_ID_PLACEHOLDER` - workspace_id: `WORKSPACE_ID_PLACEHOLDER` Covered scenario: - A mounts Drive9 and runs `drive9 git clone --fast`. - Modify `README.txt`, create `notes/drive9-note.txt`, and delete `fix.txt`. - Run `git add`, `git commit`, and push to a pre-mirrored bare remote. - After the commit, modify `build.gradle` and leave it uncommitted. - A unmounts; B mounts the same Drive9 path with a new local root/cache. - B restores `.git`; `git log -1` shows the new commit and `git status --porcelain=v1` shows ` M build.gradle`. Backend table checks: - `git_workspaces`: active fast workspace row exists. - `git_workspace_tree_nodes`: 26 rows, equal to the base commit tree count. - `git_workspace_git_state`: `storage_type=tar.gz-no-objects`; payload contains lightweight `.git` state and excludes object databases. - `git_workspace_object_packs`: empty for the historical full-clone pushed-commit validation unless staged/local-only objects need recovery. - `git_workspace_overlay` contains: - `README.txt` upsert file - `notes` upsert dir - `notes/drive9-note.txt` upsert file - `fix.txt` whiteout file - `build.gradle` upsert file - `file_nodes`: file row count under the workspace is 0; only the run directory and repo root directory rows exist. - Local SQLite file count is 0. ## Known Limits and Follow-Ups - Object packs are inline only in v1. Local-only Git objects above the 5 MiB per-blob cap or 256 MiB pack cap are downgraded instead of being preserved as staged/local refs. - Restoring large local-only Git objects without downgrades will require S3/storage refs or an object/pack dedup layer. - After `git commit`, the overlay remains relative to the base tree. Later versions can add an explicit `drive9 git checkpoint` or an automatic manifest-advance mechanism. - Large overlay files need to move from inline `content_blob` to S3/storage refs. - Directory renames inside a Git workspace are capped at 100 affected tree/overlay entries in v1 and return `EXDEV` above that limit. This avoids thousands of sequential overlay RPCs; a later batch endpoint can lift the limit. - Branch switching, merge, rebase, conflicts, submodules, and LFS need deeper semantic validation. - V1 hydrate does not optimize LFS or submodule object fetching. Non-GitHub repositories use a Git checkout-index fallback, which can be slower than GitHub codeload but still keeps the work out of FUSE read calls. - Object database hydrate is a performance layer, not authoritative storage. If a sandbox loses its local root, replacement sandboxes rebuild clean objects from the remote or hidden tree hydrate and restore local-only objects through the existing object-pack checkpoint path. - The repeatable live smoke path is `e2e/git-workspace-smoke-test.sh`. It covers fast clone, agent edit/add/commit, sandbox replacement, and fast linked worktree add/edit/stage/commit/remount/remove. Release gates still need to decide when to enable this heavier FUSE workflow by default.