Repository browse metadata index#

Forge serves directory "last commit" metadata through three layers with different durability guarantees:

1. Cache API stores only complete results for an exact repository, commit SHA, path, and format version. It is an edge acceleration layer and may be evicted at any time.
2. D1 stores a lazy mutable-branch index for directories that have actually been browsed. Readers use it only when both its active and desired heads equal the branch head resolved for the request.
3. Exact-head history scans remain the source-of-truth fallback for first visits, tags, and recovery. The request scans adaptively for at most 750 ms and 500 commits; durable Workflows resume incomplete scans.

Push-time invalidation#

Every application ref writer ultimately changes the D1 refs table. Migration 0036_repository_path_indexes.sql installs unconditional ref triggers there, so a branch SHA update and the transition of every tracked directory to a new pending generation are one database statement. This covers Git receive-pack, web commits, merges, branch operations, and fork ref copies without depending on each caller to publish an event.

The path-index row is itself the durable outbox. Git pushes try to create its Workflow immediately, and the minute cron reconciles pending rows if the request disconnects or Workflow creation fails.

Ref deletion and rename remove the old mutable index. Repository deletion cascades through both exact scan and path-index state, including private paths, authors, and commit messages.

Incremental update and recovery#

For a normal push, a directory Workflow walks first-parent commits from the new head back to the exact indexed head. The newest change to each current child wins; unchanged children inherit their prior metadata, and deleted children are removed.

If the old head is not reachable, the update is a force-push recovery. The Workflow continues to the new history root and does not inherit any metadata from the old branch history. A first visit uses the same full-history rule.

All checkpoints live in generation-fenced D1 columns. Workflow parameters and step results contain only opaque IDs and small progress summaries. Each Workflow instance stops below a conservative subrequest budget, schedules a successor epoch, and resumes the same D1 checkpoint. Terminal failures use bounded retries with backoff before the index becomes stale; stale reads still fall back to an exact-head scan rather than serving mismatched metadata.

Invariants#

• Never serve a mutable index unless active head = desired head = resolved branch head.
• Never write Cache API from a partial or blocked scan.
• Never publish a Workflow result without matching index ID, generation, desired head, active base head, and current D1 ref.
• Treat Cache API as disposable and D1/R2 as authoritative.
• Keep the foreground deadline cooperative inside both object reads and large directory comparisons.
• On an algorithm-format change, invalidate the durable index version and perform a fresh exact-head backfill.