Engineering
Exact-SHA Archives Cut Checkout 89%
The fastest dependency cache was the one Forge skipped.
Immutable exact-SHA archives cut warm source checkout from 130.682s to 14.649s, while a 512 MiB circuit breaker rejected a one-gigabyte pnpm cache that made installs slower.
The most revealing cache hit made the build slower. Forge mounted a valid one-gigabyte pnpm store, reused 548 packages with zero downloads, and then spent 70.495 seconds installing them. The clean install had taken 31.055 seconds; saving the cache had already cost another 57.329 seconds.
A second cache looked healthy for a different reason: the restored workspace exposed roughly 484 MB of files. Yet the exact-commit probe failed immediately, so Forge threw the restore away and prepared source again. Both artifacts were valid. Both had negative end-to-end value.
The fix was not “cache more.” Forge gave exact source its own immutable artifact and put a 512 MiB circuit breaker in front of automatic dependency caches. Repeat source checkout for the large TypeScript repository fell from 130.682 seconds to 13.261 and 14.649 seconds, while the oversized dependency store was deliberately bypassed.
One label hid two failures
The OverGrid canary is a large TypeScript application with a long test suite and a pnpm store just over one gigabyte. Its repeat builds exposed two unrelated cache failures.
First, two runs restored a prepared workspace whose visible tree was roughly 484 MB. The exact-commit probe, local reset, and remote refresh still failed in less than 250 milliseconds, so Forge discarded the restore and spent about 69 seconds preparing source again. The backup metadata described the mounted directory before capture; it did not prove that a new backup of that restored filesystem contained the same useful bytes.
The storage mechanism explains the mismatch. Cloudflare Sandbox backups are SquashFS images, and production restore exposes the image through a copy-on-write FUSE mount. A mounted directory can be readable without being a good input to another backup. Forge had measured the visible view, not the artifact it later tried to reuse.
The dependency cache failed in the opposite direction. It contained 1,017,708,447 bytes of real pnpm store data. One run saved it in 57.329 seconds. The next mounted it in 4.140 seconds, reported 548 reused packages and zero downloads, then spent 70.495 seconds on the frozen install. A clean install had taken 31.055 seconds.
Neither cache was corrupt. Both were valid artifacts with negative end-to-end value.
Exact source became the reusable unit
Forge already treated the requested 40-character commit SHA as authoritative. The change was to make that same identity the cache boundary.
On a trusted default-branch push, Forge performs the exact Git checkout, verifies HEAD, prepares a deterministic packed-object archive, and publishes it to private object storage. A later consumer asks for the internal repository ID and exact SHA before it walks loose objects. On a hit, it transfers and materializes the archive, then asks Git to verify the requested commit again.
trusted push: exact checkout -> verify HEAD -> pack -> publish repo + SHA
repeat build: lookup -> transfer -> materialize -> verify HEAD -> run
cache miss: bounded exact Git transport -----------------------> run
The archive remains acceleration state, not source authority. Its key and metadata bind repository identity, commit, schema, and archive format. Extraction validates the archive members. Git validates the objects and final checkout. A damaged or missing archive falls back to the authoritative Git path.
Write authority is narrower than read authority. Trusted pushes may publish a seed after verification. Pull requests and manual consumers can read a matching artifact, but they cannot replace shared source state. This prevents a faster checkout from weakening the rule that every run executes one immutable revision.
The first consumer still pays for transport. OverGrid's verified packed closure was about 177 MiB, and two live attempts exhausted the original 180-second bound. Forge raised the cold exact-source limit to five minutes based on that observation. The warm path stayed separate and measurable.
The dependency win was a bypass
Forge did not turn OverGrid's pnpm store into a fast dependency cache. It added a circuit breaker before restore and save.
Automatic manager caches now require one authoritative lockfile. Their fingerprints include the lockfile contents, package manager and version, Node runtime, platform, architecture, libc, install policy, project root, runner image, and cache schema. A declared manager without its matching lockfile disables automatic reuse instead of guessing.
Manager stores also have a default 512 MiB logical-size ceiling. Restore checks the recorded size before mounting the backup; save measures the store before creating a new one. The cache tests assert both behaviors, including that an oversized pnpm entry returns bypassed_by_size without calling the restore operation.
For the successful warm canary, Forge found no admissible pnpm cache in 37 milliseconds, ran a clean frozen install in 23.461 seconds, and rejected the 1,017,655,199-byte save in 247 milliseconds. That is not a dependency-cache hit. It is a policy decision that avoided a measured 70.495-second mounted install and 57.329-second save.
Forge also began a separate same-SHA exact dependency-snapshot experiment for generated Sites. That work has a different trust boundary and, at this point in the evidence, no production warm-build result. It does not belong in the checkout speedup claim.
Phase timings made the result auditable
The source canary recorded lookup, object traversal, transfer, materialization, and exact-HEAD verification separately. It also kept install, build, tests, lint, deploy, and live verification outside the checkout number.
| Run | Source checkout | What the measurement includes | Outcome |
|---|---|---|---|
| Cold publication | 130.682 s | 108.702 s exact Git transport and 17.242 s publication, plus setup overhead | Source published; an application test later stopped on infrastructure |
| Warm A | 13.261 s | 48 ms lookup, 5.771 s transfer, 5.567 s materialization, zero object traversal | Full workflow and live verification passed |
| Warm B | 14.649 s | 60 ms lookup, 4.908 s transfer, 7.591 s materialization, zero object traversal | Full workflow and live verification passed |
Warm B removed 116.033 seconds, or 88.8%, from source checkout relative to the cold publication path. Exact-HEAD verification took 46 milliseconds. The deployed application's independent public health endpoint returned HTTP 200.
This table does not claim that every build became nine times faster. Application test time remained noisy and much larger than the checkout delta. The evidence proves a bounded source-preparation improvement for repeat consumers of the same SHA.
Interrupted runs tested the boundary
Two canary sandboxes stopped after repository test commands had begun. Forge did not replay them automatically because arbitrary build steps may have side effects. Neither interrupted run deployed.
The source artifact survived those interruptions because publication belonged to the verified checkout phase, not to application code. A later same-SHA run could reuse it and complete normally. Another interrupted warm run had already finished source checkout in 11.151 seconds, but it remains interruption evidence, not a successful deployment measurement.
This failure behavior is part of the cache design. An acceleration layer may disappear without changing the requested revision. It must not publish partial state, advance a mutable pointer, or convert an infrastructure retry into silent command replay.
What remains expensive
The first build of a new large SHA still pays for exact Git transport and archive publication. Layered source checkpoints became the follow-up: restore the projected parent and apply only the changed Git objects. Each retained archive consumes object storage, so lifecycle and retention policy matter. This first release also reused only the same SHA; it did not claim that a shared lockfile makes dependency output safe across commits, because install scripts can inspect arbitrary source and mutate the installed tree.
A future manager-store design may hydrate onto local disk before install, or separate a script-free dependency payload from lifecycle scripts rerun for every revision. Either option needs end-to-end timing and equivalence evidence. Download counts and package reuse totals are not enough.
The durable lesson is narrower: cache identity establishes correctness, but phase timing establishes value. Forge made exact source immutable, measured it where the work occurred, and treated an oversized dependency cache as a regression to avoid. The next test is not whether another cache reports a hit. It is whether the complete build gets faster without weakening exact-SHA execution.
The full measurement boundary and canary matrix are recorded in the runner source and dependency caching decision. The public execution contract remains the Forge Deploy specification.
The sequel, A 14-Second Checkout Still Produced a 16-Minute Build, follows the next bottleneck: queue data ruled out more runners, release-gate timing exposed repeated compilation, and one trusted producer stopped sibling workflows from compressing the same exact dependency snapshot.
