Projected Checkpoints Cut Checkout 96%

Halfway through the investigation, the honest status was “not done yet.” The first cold checkpoint seed had hit the 300-second deadline, the Git server was still spending most of its time constructing a roughly 204 MB pack, and the hoped-for fast child run could not start until one parent seed survived.

<figure class="forge-blog-figure forge-blog-storyshot">
<img src="/blog/layered-source-checkpoints/coding-agent-diagnosis.webp" alt="A coding-agent conversation in dark mode. The user asks how it is going; the response says the work is not done, identifies a 300-second cold-clone deadline and a slow 204 MB Git pack, and says production speedup proof is still pending." width="1170" height="1182" loading="lazy" decoding="async">
<figcaption><span>Thread checkpoint</span><q>Latest visible state: not done yet. […] The root issue is confirmed: Forge’s Git server spends too long constructing OverGrid’s ~204 MB pack with no output.</q><small>Authentic Codex side conversation captured during the production canary; excerpt reproduced semantically for legibility.</small></figcaption>
</figure>

The finished path cut OverGrid's connected Deploy source checkout from **366.931 seconds to 14.701 seconds**, a **96.0% reduction**, without weakening the exact-commit guarantee. Forge reduced the declared source boundary, restored the matching projected parent, and applied only the changed Git objects.

The decisive measurement was not the final duration. A projected child initially spent **26.438 seconds** walking the complete selected tree but only **1.464 seconds** materializing it. Parent reuse cut that traversal to **452 milliseconds**, or **98.3%**, while the source archive shrank from roughly **180.25 MB to 22.69 MB**. A concurrent generic Action still needed the complete repository and spent **211.627 seconds** in checkout; these are production canaries for two different input contracts, not a general latency distribution.

Exact-SHA caching was correct but incomplete

Forge's [first exact-source cache](/blog/from-two-minute-checkouts-to-fourteen-seconds) already keyed artifacts by repository, exact 40-character commit SHA, schema, project root, and archive format. A hit could not drift with a branch. A miss fell back to exact Git transport.

That contract made repeated builds predictable, but every normal push creates a new SHA. OverGrid's repository was about 204 MB. Its historical cold seed took **196.829 seconds**, including **178.500 seconds** for the clone and **15.564 seconds** to publish the checkpoint.

The first attempt restored the direct parent's checkpoint, then asked Git Smart HTTP for the child commit. For a one-file change, the server still walked and packed **1,561 objects**. That “delta” fetch took **118.229 seconds**, and checkout still took **168.700 seconds**.

The cache identity was sound. The work performed after a miss was not proportional to the change.

Put source selection in the build contract

OverGrid contains generated `docs/ai/**` research that the application build does not need. Eleven JSON receipts inside that directory are imported by application code. The project now declares that boundary beside its build command in `forgeBuild.ts`:

```ts
build: {
command: 'pnpm run build',
context: {
exclude: ['docs/ai/**'],
include: [
'docs/ai/artifacts/ai-program-summary.json',
// Ten more application inputs.
],
},
}
```

The policy starts with the complete project, applies exclusions, then restores explicit exceptions. Forge also forces the build configuration, referenced Wrangler file, package manifests, lockfiles, workspace dependencies, and project root into the selected closure.

Forge reads this policy from the requested commit before checkout. Its normalized digest becomes part of the immutable v4 cache key. Changing the policy therefore cannot reuse a checkpoint created for a different source set. Omitting the policy still means complete project source.

This is a declared-input contract, not import inference. [Docker removes ignored files from a selected build context](https://docs.docker.com/build/concepts/context/), while [Bazel keys remote-cache entries from declared action inputs](https://bazel.build/remote/caching). Forge uses the same safety principle without asking projects to adopt another build language.

Reuse one projected parent

Projection alone reduced the payload to **1,539 Git objects and 22,687,273 compressed bytes**, but the first projected children still took **40.227** and **48.457 seconds**. Phase-level profiling explained why: Forge rebuilt the projected closure for every new commit.

The corrected path reuses a checkpoint only when the target has exactly one parent and that parent's checkpoint has the same projection digest. Forge then walks changed paths between the two trees and creates a bounded layer containing:

- the target commit;
- changed ancestor trees;
- changed or newly reachable blobs; and
- Git's native compressed object bytes.

The checkout path stays explicit:

```text
requested exact SHA
├─ target checkpoint exists → restore target → verify
└─ target checkpoint missing
├─ matching direct-parent checkpoint exists
│ └─ restore parent → apply changed objects → verify
└─ no eligible parent → bounded cold exact checkout → verify
```

The sandbox validates archive members and declared bounds, imports the objects, rebuilds only the projected index, and verifies the target SHA and selected tree before repository code runs. Tests cover file additions and deletions, executable-mode changes, symlinks, and file-to-directory transitions. A changed projection, merge commit, invalid checkpoint identity, integrity error, or exceeded bound cannot enter the fast path. Structural ambiguity in an excluded path falls back to a fresh projected traversal rather than guessing.

Each successful target becomes another complete immutable checkpoint. Forge never follows a recursive chain of deltas, so deleting an older parent cannot strand a newer target. The checkpoint remains derived acceleration state; repository objects remain the authority.

Profiling rejected the tempting fix

The first fresh projected-child receipt took **38.763 seconds**. Tree traversal consumed **26.438 seconds**, archive transfer **9.244 seconds**, and materialization **1.464 seconds**. That split ruled out a tempting shortcut.

Cloudflare Sandbox backups restore through a SquashFS/FUSE overlay. A snapshot could help when extraction dominates, but it would have attacked roughly one or two seconds here while leaving the 26-second walk intact. The measured bottleneck was repeated object discovery, so Forge reused the projected parent's object closure instead.

On the final one-line canary, changed-path traversal fell to **452 milliseconds**. Restoring the parent took **6.717 seconds**, the four-object child archive transferred in **67 milliseconds**, and parent-plus-delta materialization took **2.880 seconds**. The runner's inner source phase reported **14.357 seconds**; the enclosing exact-SHA checkout receipt reported **14.701 seconds**.

<figure class="forge-blog-figure forge-blog-performance-figure">
<div class="forge-blog-waterfall" role="img" aria-label="Common-scale comparison of OverGrid connected Deploy checkout. The old full-source path took 366.931 seconds. The projected-parent path took 14.701 seconds, including 0.452 seconds of changed-path traversal, 6.717 seconds of parent transfer, 2.880 seconds of materialization, and 4.652 seconds of validation, freezing, lookup, and other measured checkout work.">
<div class="forge-blog-waterfall-axis" aria-hidden="true"><span>0s</span><span>100s</span><span>200s</span><span>300s</span><span>366.9s</span></div>
<div class="forge-blog-waterfall-row">
<div class="forge-blog-waterfall-label"><strong>Before</strong><span>full source</span></div>
<div class="forge-blog-waterfall-shell">
<div class="forge-blog-waterfall-track" style="width: 100%">
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--base" style="width: 46.47%"></span>
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--hot" style="width: 53.36%"></span>
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--reference" style="width: 0.17%"></span>
</div>
</div>
<strong class="forge-blog-waterfall-total">366.931s</strong>
</div>
<div class="forge-blog-waterfall-row">
<div class="forge-blog-waterfall-label"><strong>After</strong><span>projected parent + delta</span></div>
<div class="forge-blog-waterfall-shell">
<div class="forge-blog-waterfall-track" style="width: 4.01%">
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--source" style="width: 3.07%"></span>
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--publish" style="width: 45.69%"></span>
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--reuse" style="width: 19.59%"></span>
<span class="forge-blog-waterfall-segment forge-blog-waterfall-segment--reference" style="width: 31.65%"></span>
</div>
</div>
<strong class="forge-blog-waterfall-total">14.701s</strong>
</div>
<div class="forge-blog-waterfall-legend" aria-hidden="true">
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--base"></i>old exact-source attempt: 170.525s</span>
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--hot"></i>old cold clone: 195.769s</span>
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--source"></i>changed-path traversal: 452ms</span>
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--publish"></i>parent transfer: 6.717s</span>
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--reuse"></i>parent + delta materialization: 2.880s</span>
<span><i class="forge-blog-waterfall-key forge-blog-waterfall-key--reference"></i>validation, freeze, lookup, and other checkout work</span>
</div>
</div>
<figcaption><strong>One common scale:</strong> checkout fell 96.0%, source bytes fell 87.4%, and direct-child traversal fell 98.3%. Immutable parent transfer is now larger than discovery or delta application.</figcaption>
</figure>

The figure is a comparison between one original full-source receipt and one direct-child production canary. It does not establish a percentile or promise the same duration for repositories with different trees, contexts, or cache state.

Freeze once, publish while the build runs

The first fast child exposed another trap. Checkout succeeded, but the optional publisher performed a second full traversal of the target and stalled. A fast consumer had recreated the expensive producer work.

Forge now freezes the already verified sandbox object database before repository code runs. It removes duplicate downloaded tar files, deterministically archives the unique parent-plus-delta objects, and hashes fixed **16 MiB** parts. Publication uses digest-fenced multipart upload through the Worker's R2 binding and overlaps install, build, and tests. A changed part cannot be published under the prepared checkpoint identity.

[Run #307](/swyx/strata-field-guide/actions/runs/961) published before the application build ended (`source_seed_join_ms=0`). [Run #308](/swyx/strata-field-guide/actions/runs/962) then restored that exact target with zero traversal. The repeat took **18.188 seconds** in the runner source phase; transferring the immutable target, not rediscovering its graph, dominated the result.

Push fan-out also needs restraint. A Sites consumer waits for an active sibling Action only when the target is a default-branch push, a complete compatible parent exists, and that sibling is producing the same repository and SHA. The wait is bounded at 90 seconds. A truly cold repository skips it, so coordination cannot add a wait before the same cold checkout.

The fast path has a deliberate edge

`forgeBuild.ts` declares build inputs. It does not describe arbitrary shell commands. OverGrid's generic Action runs 321 test files; 35 refer to generated `docs/ai` evidence beyond the 11 receipts required by the application build. Reusing the Deploy projection would make that workflow faster by making the test run incomplete.

The safe extension is an explicit run-level source contract, defaulting to full source. A lean deploy-validation Action could opt into the build context, while scheduled research-integrity checks keep the complete repository.

Full-source work also has a different remaining bottleneck. The same changed-path algorithm cut one Action's tree walk to **481 milliseconds**, but moving its 180.25 MB parent still took **127.311 seconds**. Total checkout was **145.612 seconds**, only **18.6%** better than its prior same-SHA baseline. Faster transport or a native snapshot would address measured work there; another traversal optimization would not.

This change also does not accelerate external `git fetch`, and it does not make OverGrid's 1.03 GB pnpm store worth restoring. Forge keeps that dependency cache above its 512 MiB ceiling and runs the smaller frozen install instead.

Cache the contract, not the branch

The durable rule is compact:

> A build receives the exact requested commit and its declared source context. The normalized context is part of immutable cache identity; no explicit context means the complete project.

That invariant let Forge remove 96% of checkout time without weakening source correctness. The next test is no longer whether smaller Git deltas work. It is whether full-source archive transport and explicit Action contexts can improve the remaining slow paths without turning an optimization into hidden authority.

The public release boundary remains the [Forge Deploy specification](/spec/deploy/v1), with agent-readable guidance in [`llms.txt`](/llms.txt).