SmolForge by swyx#

A fully-realized GitHub clone built entirely on Cloudflare infrastructure. Push code, browse files, manage issues, create pull requests — all backed by Workers, D1, R2, and Durable Objects.

Live: forge.smol.ai · Engineering blog: forge.smol.ai/blog · RSS: feed.xml

What This Is#

SmolForge implements the core GitHub experience from scratch: user auth, repository CRUD, independent repository forks, Gist-style multi-file snippets, the Git Smart HTTP protocol (push/clone/fetch with a real git CLI), file browsing, branch management, issues, pull requests, collaborators, and an organization/team permission system.

Forge is now a routed Cloudflare system rather than a single Worker. Twelve release components separate the public edge, identity, content control, repository data plane, deployment control plane, notifications, build runner, hosted-site edge, deployed-application runtime, wiki, AI routing, and Slack agents. They share explicit contracts and are built, tested, and released according to component impact.

Forge began in a single session using Claude in Cowork mode, with early scaffolding help from ralph-claude-code for autonomous development loops. The system has since grown into the component architecture below.

Architecture#

                       forge.smol.ai
                             |
                     +-------v-------+
                     |  public edge  |
                     | web/blog/docs |
                     +-------+-------+
                             |
       +------------+------------+------------------+
       |            |            |                  |
+------v------+ +---v--------+ +--v----------+ +-----v--------+
|  identity   | |  content   | | repository  | | deploy-control|
| auth/account| | gists/cards| | Git + repos | | projects/releases |
+-------------+ +------------+ +------+------+ +------+-------+
                             |                      |
                    +--------v--------+    +--------v--------+
                    | runner / wiki / |    | deploy-runtime / |
                    | notifications   |    | sites-edge / AI |
                    +-----------------+    +-----------------+

   control D1 + domain D1s · R2 Git/artifacts · Durable Objects

The component graph is declared in config/forge-components.json. Identity, repository, Deploy control, and notifications have independent workspace entrypoints and Wrangler contracts; packages/api is their internal implementation kernel while feature code is moved to its final owner. Changes are routed to affected validation suites and runtime components, with a fail-closed full gate when ownership cannot be established.

Cloudflare D1 stores relational state behind explicit domain ownership. The current product/control-plane source is cloudforge-next; an executable inventory in config/forge-database-domains.json maps every canonical table to its future database and reports cross-domain foreign-key blockers. Domain databases are additive shadow targets until schema, row-count, and content-digest parity pass and the owning runtime has no cross-domain transaction. The AI router already owns its reservation and immutable invocation ledger in cloudforge-ai-usage.

Cloudflare R2 stores Git objects, packfiles, release artifacts, and other immutable blobs. Git objects are keyed by repository and object SHA.

Durable Objects provide repository coordination and the production release lease. The release lease issues a monotonic fencing token; stale controllers cannot mutate providers.

Hono remains the Workers-native HTTP framework for the API components.

Tech Stack#

Git Protocol Implementation#

The most interesting part. SmolForge implements the Git Smart HTTP Protocol from scratch:

Endpoints:

• GET /:owner/:repo.git/info/refs?service=git-upload-pack — Reference discovery (clone/fetch)
• GET /:owner/:repo.git/info/refs?service=git-receive-pack — Reference discovery (push)
• POST /:owner/:repo.git/git-receive-pack — Receive pushed objects
• POST /:owner/:repo.git/git-upload-pack — Send objects for clone/fetch

What's implemented:

• Full packfile parsing with zlib decompression (RFC 1950)
• OFS_DELTA and REF_DELTA resolution for delta-compressed objects
• Custom Huffman decoder for scanning deflate streams without full decompression
• Packfile generation with proper SHA1 checksums
• Object graph walking for git clone (collects all reachable objects)
• Stateless fetch negotiation with multi_ack_detailed, no-done, and immediate ACK <sha> ready pack streaming for incremental pulls
• pkt-line protocol encoding/decoding
• HTTP Basic Auth for git operations

The packfile parser (packages/api/src/git/packfile.ts) includes a hand-written deflate stream scanner that traverses the block structure (stored, fixed Huffman, dynamic Huffman) to find object boundaries without decompressing — needed because packfile entries are concatenated zlib streams with no length prefixes.

Product Implementation Details#

The landing page intentionally keeps implementation details light. This section is the deeper technical reference for how the product surface is built.

Frontend and repo browsing#

• React 18, React Router v6, Vite, and Tailwind CSS power the web UI.
• The file browser includes a VS Code-style tree, breadcrumbs, syntax highlighting, filetype icons, hover prefetching, and inline file editing with a commit panel.
• Markdown rendering includes heading anchors, generated table of contents, copy buttons on code blocks, scrollable code blocks, table rendering, and dark mode support.
• Accessibility options include light/dark/system themes, high-contrast mode, configurable font sizes, and reduced motion.

Git and repository operations#

• Git Smart HTTP support covers clone, fetch, push, refs, and packfile exchange.
• Push operations use repo-level coordination to avoid concurrent write races.
• Repositories support public, unlisted, and private visibility. Unlisted repositories are readable by direct URL but excluded from public profiles and discovery.
• Forking runs as a durable, resumable repository job with byte/object progress, bounded copy concurrency, three attempts, cancellation, and cleanup. The destination remains private until every Git object and snapshotted ref is present. It keeps source attribution and licensing while excluding issues, collaborators, secrets, workflows, webhooks, and transcripts.
• Repository headers display SPDX license identifiers. The curated one-click choices are MIT, Apache-2.0, and AGPL-3.0-or-later; adopting one commits canonical SPDX license text as LICENSE, including for a repository's initial commit.
• Pull requests include branch comparisons, commit lists, changed-file summaries, merge status, and permission-controlled fast-forward merges with stale-head/base protection.
• Repository collaboration includes collaborators, organizations, teams, role hierarchy, and team-repo permissions.

CI/CD Actions#

• Workflow configs, runs, jobs, steps, logs, artifacts, and secrets are modeled in the database.
• Workflows can run on push, pull request, or manual triggers.
• Container jobs support manager-aware cache: auto, cache: npm for download/tool caches, and cache: npm-exact for dependency snapshots. Generated Deploy builds choose auto; every cache identity includes the normalized project root so monorepo projects warm independently, and cache failures fall back to a clean install.
• Jobs support dependencies through needs, with real-time logs, cache timing telemetry, and cancel/rerun controls.

Forge Deploy preview#

Forge Deploy hosts authorized public or private repository applications at *.sites.smol.ai with immutable assets, an HTTP Worker, Forge-managed durable state, bounded realtime, exact-SHA previews, automatic or manual publication, and pointer-based rollback. Applications describe their provider-neutral shape in a typed, statically evaluated forgeBuild.ts.

Monorepos may also add a static root .forge/config.json to map project roots, shared defaults, and reviewed Cloudflare binding proposals. It never replaces the per-project forgeBuild.ts or grants builds DNS authority; inspect/import and provider Apply remain separate administrator actions. See the repository configuration contract.

Cloudflare remains the runtime provider; Forge adds the Git-native release control plane. It binds application code and assets to one exact commit, creates an immutable preview before production, blocks automatic activation when the branch has advanced, records the complete release receipt, and rolls back by moving a production pointer instead of rebuilding an old checkout.

Start with the public Forge Deploy preview guide, the live forgeBuild.ts v1 specification, or the three conformance fixtures. Large public binaries can remain standard Git LFS pointers backed by Forge Assets: OpenNext plans their exact provider identity from verified digests and streams only provider-requested bytes instead of buffering a release in memory.

AI transcripts and safety#

• Transcript upload accepts raw JSONL from Codex, Claude Code, Cowork, Cursor, Factory Droid, and other agent formats that can be parsed server-side. Devin and OpenCode can be captured through CLI best-effort SQLite extraction when their local stores are available.
• Commit linkage works through explicit commit_sha uploads and AI-Session git trailers.
• Secret masking supports built-in credential masks plus global and repo-specific patterns before transcript display.
• Content safety includes blocklists and reports for repository content.

Gists#

• Signed-in users can create public or unlisted snippets containing up to ten root-level text files.
• Public Gists appear in discovery. Unlisted Gists are link-readable, not private, and do not appear in public discovery.
• Owners can edit or delete their Gists; dedicated personal access token scopes support API clients.
• Per-file, per-Gist, per-account, and write-rate limits bound D1 storage use.

See the Gists guide for limits and API behavior.

Schema coverage#

The database spans users, repositories (including visibility, fork ancestry, and SPDX license metadata), refs, issues, comments, labels, pull requests, collaborators, organizations, teams, team members, stars, Gists and Gist files, workflow configs, workflow runs, jobs, steps, logs, artifacts, secrets, transcripts, secret masks, webhooks, content blocklists, and content reports.

Project Structure#

forge/
+-- config/
|   +-- forge-components.json   # Component ownership and dependency graph
|   +-- forge-releases/         # Per-component runtime contracts
+-- packages/
|   +-- edge/                   # Public routing, web assets, blog, RSS, social cards
|   +-- content/                # Public content and blog metadata
|   +-- web/                    # React product UI
|   +-- identity/               # Identity Worker entrypoint and provider contract
|   +-- repository/             # Git/repository Worker entrypoint and provider contract
|   +-- content-control/        # Gists, skins, and product social metadata
|   +-- deploy-control/         # Deploy control Worker entrypoint and provider contract
|   +-- notifications/          # Notification Worker entrypoint and provider contract
|   +-- api/                    # Internal implementation kernel during domain source extraction
|   |   +-- src/components/     # Component composition modules
|   |   +-- src/git/            # Git protocol and storage internals
|   |   +-- src/features/       # Domain implementations
|   +-- runner/                 # Trusted CI/build orchestration and provider adapter
|   +-- deploy-runtime/         # Hosted application runtime
|   +-- sites-worker/           # Hosted-site edge routing
|   +-- wiki-worker/            # Wiki execution boundary
|   +-- ai-router/              # Model routing + owned AI usage D1 ledger
|   +-- slack-agent/            # Slack agent boundary
|   +-- cli/                    # `sf` / `smolforge` CLI
|   +-- shared/                 # Versioned cross-component contracts
+-- migrations/                 # Canonical control schema plus generated domain schemas
+-- scripts/                    # Validation, impact, contracts, and recovery tooling
+-- docs/                       # Public guides plus excluded operational material
+-- examples/forge-deploy/      # Deploy conformance fixtures

Development Timeline#

Forge began as a single-Worker project built in one extended session. This is the origin story; the current component architecture is described above.

1. Scaffolding — Used ralph-claude-code to bootstrap the monorepo structure, set up Hono, and generate initial route stubs.

2. Git Protocol (the hard part) — Implemented the Smart HTTP protocol from scratch: packfile parsing with a custom deflate stream scanner, delta resolution (OFS_DELTA + REF_DELTA), pkt-line encoding, and object graph walking. This required reading RFC 1951 (DEFLATE), RFC 1950 (zlib), and the git pack format spec.

3. API Routes — Built out all CRUD endpoints: repos, issues (with auto-increment numbering and comments), pull requests, branches, commits, file browsing, and collaborators.

4. Frontend — 18-page React SPA with TailwindCSS: login/register, dashboard, repo view with file browser, issue tracker, PR management, branch list, commit history, and settings.

5. Deployment — Deployed the Worker to Cloudflare. Hit several issues:

   • Wrangler CLI's /memberships endpoint failed with API tokens → switched to Cloudflare REST API for D1/R2 creation
   • git clone returned "bad line length character: PACK" → fixed sideband-64k encoding, then simplified to raw pack data without sideband
   • git clone returned "bad object" → the want line parser wasn't stripping capabilities from the SHA
   • JWT_SECRET was a [vars] binding conflicting with secret → removed from vars, set via wrangler secret put
   • Frontend served on same domain via [assets] in wrangler.toml, eliminating CORS issues

6. Testing — 119 unit tests (pkt-line, objects, delta, packfile) + 20 API E2E tests + 32 privacy E2E tests, all passing.

7. Privacy Fixes — Found and fixed private repo info leaking via issues endpoint (returned data instead of 404 for unauthorized access).

8. Organization/Team System — Designed and implemented org-based auth: organizations, teams, team-repo mappings, role hierarchy (owner > admin > member), and permission levels (read < write < admin). See the organizations and teams design for the full design.

AI Agent Integration (llms.txt)#

SmolForge ships a comprehensive llms.txt file designed for AI coding agents (Claude Code, Codex, etc.) to consume programmatically. It includes:

• Complete API reference — every endpoint with auth requirements, request/response formats, and example payloads
• Transcript hook automation — CLI-managed hooks for Claude Code, Codex, Cursor, Factory Droid, Devin, OpenCode, Aider, Continue, generic JSONL agents, and Git hook backstops
• Backfill & incremental workflows — bulk-upload all existing transcripts or hook into your commit flow for continuous uploads

The recommended setup is CLI-managed:

sf hooks install alice/demo --agent all --git post-commit,pre-push
sf hooks status alice/demo

This creates a repo-local upload script, installs native lifecycle hooks where available, and installs Git hook backstops for Codex and other agents. Codex is supported through the Git hook path and ~/.codex/sessions JSONL discovery. Git has no native post-push hook, so SmolForge uses pre-push for the final missed-upload check.

See llms.txt for the full guide.

CLI#

SmolForge also ships an installable CLI package, @smolai/forge, with two equivalent binaries: smolforge and sf.

For new JavaScript and TypeScript repositories, Forge strongly recommends pnpm as the default package manager. Bun is the high-speed alternative when project compatibility allows. npm and Yarn repositories remain fully supported; Forge's UI and CLI provide advisory guidance and never silently migrate a project.

# npm
npm install -g @smolai/forge

# Homebrew, once the tap is published
brew install smol-ai/tap/smolforge

# Local formula from this repo
brew install --HEAD ./Formula/smolforge.rb

Basic usage:

smolforge login --username alice
sf repo create demo --description "Edge-hosted git repo"
git remote add origin $(sf clone-url alice/demo)
git push origin main

# Install transcript capture before agents start writing code
sf hooks install alice/demo --agent all --git post-commit,pre-push
sf hooks status alice/demo

# Upload Codex, Claude Code, Cowork, Cursor, Factory Droid, or other JSONL agent transcripts
sf transcript upload alice/demo --file ~/.codex/sessions/2026/03/29/session.jsonl

# Work with Actions and the live machine-readable API guide
sf actions runs alice/demo --limit 5
sf llms --save llms.txt

The CLI defaults to https://forge.smol.ai and can be pointed at another deployment with --base-url or SMOLFORGE_BASE_URL.

Releasing Forge#

Normal production releases use forge-release/v2. Feature work submits an immutable pull-request SHA; affected CI produces signed evidence; the protected merge queue is the only authority allowed to advance main; and one resumable controller promotes the affected components under a Durable Object fencing token.

SHA="$(git rev-parse HEAD)"
sf release enqueue "$SHA" \
  --ref codex/my-feature \
  --repo swyx/forge \
  --pr 123 \
  --yes \
  --json

sf release status fmq_request_id --repo swyx/forge --json
sf release watch fmq_request_id --repo swyx/forge --json

Enqueue records durable canonical state; it does not activate a provider inline. The controller owns managed Workers and connected edge activation, persists phase history and receipts in D1, and rejects stale provider mutations. Component input digests and successful activation receipts determine impact, so unchanged Workers need not share the edge source SHA. The canonical observation commands are sf release status/watch. Direct local deployment is audited break-glass recovery, not a normal release path. See the release state machine and production runbook.

Queue validation reuses signed exact-candidate evidence and joins an in-flight run for the same production base instead of launching duplicate CI. Managed Workers activate in dependency-safe parallel waves. Each Worker owns its health binding list beside its runtime entrypoint, so adding a component does not edit one global binding registry or select every existing Worker.

Related Documentation#

• Forge engineering blog — Architecture, reliability, Git, CI, and deployment field notes
• Blog RSS feed — Subscribe to new Forge engineering posts
• llms.txt — Machine-readable API reference and AI agent integration guide
• Documentation — User guides, product contracts, engineering reference, design, operations, and project history
• Organizations and teams — Quick curl examples for the org/team API
• Production deployment — Release, recovery, rollback, and smoke checks
• Character skins — Animation families, variants, sprite strips, manifests, and validation

Running Locally#

This repository predates Forge's pnpm-first recommendation and intentionally retains its committed npm lockfile until a measured migration is performed.

# Install dependencies
npm install

# Start the API (with D1/R2 local emulation)
cd packages/api
npx wrangler dev

# In another terminal, start the frontend
cd packages/web
npm run dev

Running Tests#

npm run typecheck
npm test                  # Local-only unit and integration tests
npm run test:migrations   # Clean schema plus data-preserving 0015 -> current-head upgrade
npm run test:smoke        # Real Worker: push, clone, pull, issue, PR, and merge

Remote E2E tests are opt-in, require E2E_BASE_URL to name a disposable staging deployment, and refuse known production hosts:

E2E_BASE_URL=https://staging.example.test npm run test:e2e:remote:api
E2E_BASE_URL=https://staging.example.test npm run test:e2e:remote:web

Deploying#

Production changes enter the protected merge queue with sf release enqueue. After signed affected CI passes, the queue advances main and hands the exact merge SHA to the fenced, resumable controller. A release is complete only when its canonical status is terminal, required components and bindings are healthy, and the public verification receipt is recorded. A Git push or successful build alone is not deployment evidence.

See production deployment for break-glass recovery, rollback, and smoke checks.

What's Next#

• Pull request conversations, reviews, approvals, and requested changes
• Real merge commits plus squash and rebase strategies
• Additional isolated runner images and package-manager cache strategies
• Fork-network browsing and optional upstream synchronization

Archive#

The archive/ralph/ directory contains files from the ralph-claude-code autonomous development loop that was used during initial scaffolding. These are preserved for historical interest — they show the prompts, fix plans, and session logs from the automated build process.