# SmolForge by swyx A fully-realized GitHub clone built entirely on Cloudflare infrastructure. Push code, browse files, manage issues, create pull requests — all backed by Workers, D1, R2, and Durable Objects. **Live:** [forge.smol.ai](https://forge.smol.ai) · **Engineering blog:** [forge.smol.ai/blog](https://forge.smol.ai/blog) · **RSS:** [feed.xml](https://forge.smol.ai/blog/feed.xml) ## What This Is SmolForge implements the core GitHub experience from scratch: user auth, repository CRUD, independent repository forks, the Git Smart HTTP protocol (push/clone/fetch with a real `git` CLI), file browsing, branch management, issues, pull requests, collaborators, and an organization/team permission system. Forge also provides a versioned SKIT registry for distributable skill bundles. Forge is now a routed Cloudflare system rather than a single Worker. Thirteen independently released Workers separate the public edge, identity, content control, repository data plane, deployment control plane, notifications, build runner, hosted-site edge, deployed-application runtime, wiki, content delivery, AI routing, and Slack agents. They share explicit contracts and are built, tested, and released according to project impact. Forge began in a single session using Claude in Cowork mode, with early scaffolding help from [ralph-claude-code](https://github.com/frankbria/ralph-claude-code) for autonomous development loops. The system has since grown into the component architecture below. ## Architecture ```text forge.smol.ai | +-------v-------+ | public edge | | web/blog/docs | +-------+-------+ | +------------+------------+------------------+ | | | | +------v------+ +---v--------+ +--v----------+ +-----v--------+ | identity | | content | | repository | | deploy-control| | auth/account| | gists/cards| | Git + repos | | projects/releases | +-------------+ +------------+ +------+------+ +------+-------+ | | +--------v--------+ +--------v--------+ | runner / wiki / | | deploy-runtime / | | notifications | | sites-edge / AI | +-----------------+ +-----------------+ control D1 + domain D1s · R2 Git/artifacts · Durable Objects ``` The component graph is declared in [`config/forge-components.json`](config/forge-components.json). Identity, repository, Deploy control, and notifications have independent workspace entrypoints and Wrangler contracts; `packages/api` is their internal implementation kernel while feature code is moved to its final owner. Changes are routed to affected validation suites and runtime components, with a fail-closed full gate when ownership cannot be established. **Cloudflare D1** stores relational state behind explicit domain ownership. The current product/control-plane source is `cloudforge-next`; an executable inventory in [`config/forge-database-domains.json`](config/forge-database-domains.json) maps every canonical table to its future database and reports cross-domain foreign-key blockers. Domain databases are additive shadow targets until schema, row-count, and content-digest parity pass and the owning runtime has no cross-domain transaction. The AI router already owns its reservation and immutable invocation ledger in `cloudforge-ai-usage`. **Cloudflare R2** stores Git objects, packfiles, release artifacts, and other immutable blobs. Git objects are keyed by repository and object SHA. **Durable Objects** provide repository coordination and one small publication lane per production Worker. Each lane stores only serialization, a monotonic fence, and the current in-flight deployment; Cloudflare's active deployment remains runtime truth. **Hono** remains the Workers-native HTTP framework for the API components. ## Tech Stack | Layer | Technology | Why | |-------|-----------|-----| | Runtime | Cloudflare Workers | Edge compute, zero cold starts, global deployment | | Database | Cloudflare D1 (SQLite) | Serverless relational DB at the edge | | Object Storage | Cloudflare R2 | S3-compatible, no egress fees, stores git objects | | Concurrency | Durable Objects | Repo locks plus fenced production release authority | | API Framework | Hono | Lightweight, Workers-native, great DX | | Auth | JWT (HMAC-SHA256) + PBKDF2 | Stateless auth via Web Crypto API | | Frontend | React 19 + React Router SSR + Vite + TailwindCSS | Edge-rendered public routes and hydrated product UI | | Build/Deploy | Forge main generations + Wrangler | Complete PR previews, exact-main immutable versions, per-Worker fenced publication | | Testing | Vitest | Fast, TypeScript-native test runner | | Monorepo | npm workspaces | Simple, no extra tooling needed | ## Git Protocol Implementation The most interesting part. SmolForge implements the [Git Smart HTTP Protocol](https://www.git-scm.com/docs/http-protocol) from scratch: **Endpoints:** - `GET /:owner/:repo.git/info/refs?service=git-upload-pack` — Reference discovery (clone/fetch) - `GET /:owner/:repo.git/info/refs?service=git-receive-pack` — Reference discovery (push) - `POST /:owner/:repo.git/git-receive-pack` — Receive pushed objects - `POST /:owner/:repo.git/git-upload-pack` — Send objects for clone/fetch **What's implemented:** - Full packfile parsing with zlib decompression (RFC 1950) - OFS_DELTA and REF_DELTA resolution for delta-compressed objects - Custom Huffman decoder for scanning deflate streams without full decompression - Packfile generation with proper SHA1 checksums - Object graph walking for `git clone` (collects all reachable objects) - Stateless fetch negotiation with `multi_ack_detailed`, `no-done`, and immediate `ACK ready` pack streaming for incremental pulls - pkt-line protocol encoding/decoding - HTTP Basic Auth for git operations The packfile parser (`packages/api/src/git/packfile.ts`) includes a hand-written deflate stream scanner that traverses the block structure (stored, fixed Huffman, dynamic Huffman) to find object boundaries without decompressing — needed because packfile entries are concatenated zlib streams with no length prefixes. ## Product Implementation Details The landing page intentionally keeps implementation details light. This section is the deeper technical reference for how the product surface is built. ### Frontend and repo browsing - React 18, React Router v6, Vite, and Tailwind CSS power the web UI. - The file browser includes a VS Code-style tree, breadcrumbs, syntax highlighting, filetype icons, hover prefetching, and inline file editing with a commit panel. - Markdown rendering includes heading anchors, generated table of contents, copy buttons on code blocks, scrollable code blocks, table rendering, and dark mode support. - Accessibility options include light/dark/system themes, high-contrast mode, configurable font sizes, and reduced motion. ### Git and repository operations - Git Smart HTTP support covers clone, fetch, push, refs, and packfile exchange. - Push operations use repo-level coordination to avoid concurrent write races. - Repositories support `public`, `unlisted`, and `private` visibility. Unlisted repositories are readable by direct URL but excluded from public profiles and discovery. - Forking runs as a durable, resumable repository job with byte/object progress, bounded copy concurrency, three attempts, cancellation, and cleanup. The destination remains private until every Git object and snapshotted ref is present. It keeps source attribution and licensing while excluding issues, collaborators, secrets, workflows, webhooks, and transcripts. - Repository headers display SPDX license identifiers. The curated one-click choices are MIT, Apache-2.0, and AGPL-3.0-or-later; adopting one commits canonical SPDX license text as `LICENSE`, including for a repository's initial commit. - Pull requests include branch comparisons, commit lists, changed-file summaries, merge status, and permission-controlled fast-forward merges with stale-head/base protection. - Repository collaboration includes collaborators, organizations, teams, role hierarchy, and team-repo permissions. ### CI/CD Actions - Workflow configs, runs, jobs, steps, logs, artifacts, and secrets are modeled in the database. - Workflows can run on push, pull request, or manual triggers. - Container jobs support manager-aware `cache: auto`, `cache: npm` for download/tool caches, and `cache: npm-exact` for dependency snapshots. Generated Deploy builds choose `auto`; every cache identity includes the normalized project root so monorepo projects warm independently, and cache failures fall back to a clean install. - Jobs support dependencies through `needs`, with real-time logs, cache timing telemetry, and cancel/rerun controls. ### Forge Deploy preview Forge Deploy hosts authorized public or private repository applications at `*.sites.smol.ai` with immutable assets, an HTTP Worker, Forge-managed durable state, bounded realtime, exact-SHA previews, automatic or manual publication, and pointer-based rollback. Applications describe their provider-neutral shape in a typed, statically evaluated `forgeBuild.ts`. Monorepos may also add a static root `.forge/config.json` to map project roots, shared defaults, and reviewed Cloudflare binding proposals. It never replaces the per-project `forgeBuild.ts` or grants builds DNS authority; inspect/import and provider **Apply** remain separate administrator actions. See the [repository configuration contract](docs/deploy/repository-config.md). Cloudflare remains the runtime provider; Forge adds the Git-native release control plane. It binds application code and assets to one exact commit, creates an immutable preview before production, blocks automatic activation when the branch has advanced, records the complete release receipt, and rolls back by moving a production pointer instead of rebuilding an old checkout. Start with the public [Forge Deploy preview guide](docs/deploy/forge-deploy-preview.md), the live [`forgeBuild.ts` v1 specification](https://forge.smol.ai/spec/deploy/v1), or the three [conformance fixtures](examples/forge-deploy/README.md). Large public binaries can remain standard Git LFS pointers backed by Forge Assets: OpenNext plans their exact provider identity from verified digests and streams only provider-requested bytes instead of buffering a release in memory. ### AI transcripts and safety - Transcript upload accepts raw JSONL from Codex, Claude Code, Cowork, Cursor, Factory Droid, and other agent formats that can be parsed server-side. Devin and OpenCode can be captured through CLI best-effort SQLite extraction when their local stores are available. - Commit linkage works through explicit `commit_sha` uploads and `AI-Session` git trailers. - Secret masking supports built-in credential masks plus global and repo-specific patterns before transcript display. - Content safety includes blocklists and reports for repository content. ### SKIT registry - A SKIT is an atomic versioned bundle containing one or more independently addressable skills. - Forge stores editable draft revisions, immutable release archives, child hashes, favorites, cloud library intent, and device-confirmed installation state. - `sf add skit`, `sf skit install`, `skit add`, and `skit install` use one preservation-first implementation. Claude Code is the Stage 1 writable target; other agent roots remain inventory-only until their compatibility gates pass. - The former Gist product is replaced by SKIT and a minimal read/export-only `content.snippet` preservation sink during the verified semantic cutover. See [SKIT Registry Stage 1](packages/skit/STAGE1.md) for the current contract. ### Schema coverage The database spans users, repositories (including visibility, fork ancestry, and SPDX license metadata), refs, issues, comments, labels, pull requests, collaborators, organizations, teams, team members, stars, SKIT drafts/releases/skills, the minimal snippet migration sink, workflow configs, workflow runs, jobs, steps, logs, artifacts, secrets, transcripts, secret masks, webhooks, content blocklists, and content reports. Large SKIT and snippet bodies remain in content-addressed R2 objects. ## Project Structure ```text forge/ +-- config/ | +-- forge-components.json # Component ownership and dependency graph | +-- forge-releases/ # Per-component runtime contracts +-- packages/ | +-- edge/ # Public routing, web assets, blog, RSS, social cards | +-- content/ # Public content and blog metadata | +-- web/ # React product UI | +-- identity/ # Identity Worker entrypoint and provider contract | +-- repository/ # Git/repository Worker entrypoint and provider contract | +-- content-control/ # Gists, skins, and product social metadata | +-- deploy-control/ # Deploy control Worker entrypoint and provider contract | +-- notifications/ # Notification Worker entrypoint and provider contract | +-- api/ # Internal implementation kernel during domain source extraction | | +-- src/components/ # Component composition modules | | +-- src/git/ # Git protocol and storage internals | | +-- src/features/ # Domain implementations | +-- runner/ # Trusted CI/build orchestration and provider adapter | +-- deploy-runtime/ # Hosted application runtime | +-- sites-worker/ # Hosted-site edge routing | +-- wiki-worker/ # Wiki execution boundary | +-- ai-router/ # Model routing + owned AI usage D1 ledger | +-- slack-agent/ # Slack agent boundary | +-- cli/ # `sf` / `smolforge` CLI | +-- shared/ # Versioned cross-component contracts +-- migrations/ # Canonical control schema plus generated domain schemas +-- scripts/ # Validation, impact, contracts, and recovery tooling +-- docs/ # Public guides plus excluded operational material +-- examples/forge-deploy/ # Deploy conformance fixtures ``` ## Development Timeline Forge began as a single-Worker project built in one extended session. This is the origin story; the current component architecture is described above. 1. **Scaffolding** — Used ralph-claude-code to bootstrap the monorepo structure, set up Hono, and generate initial route stubs. 2. **Git Protocol (the hard part)** — Implemented the Smart HTTP protocol from scratch: packfile parsing with a custom deflate stream scanner, delta resolution (OFS_DELTA + REF_DELTA), pkt-line encoding, and object graph walking. This required reading RFC 1951 (DEFLATE), RFC 1950 (zlib), and the git pack format spec. 3. **API Routes** — Built out all CRUD endpoints: repos, issues (with auto-increment numbering and comments), pull requests, branches, commits, file browsing, and collaborators. 4. **Frontend** — 18-page React SPA with TailwindCSS: login/register, dashboard, repo view with file browser, issue tracker, PR management, branch list, commit history, and settings. 5. **Deployment** — Deployed the Worker to Cloudflare. Hit several issues: - Wrangler CLI's `/memberships` endpoint failed with API tokens → switched to Cloudflare REST API for D1/R2 creation - `git clone` returned "bad line length character: PACK" → fixed sideband-64k encoding, then simplified to raw pack data without sideband - `git clone` returned "bad object" → the `want` line parser wasn't stripping capabilities from the SHA - JWT_SECRET was a `[vars]` binding conflicting with `secret` → removed from vars, set via `wrangler secret put` - Frontend served on same domain via `[assets]` in wrangler.toml, eliminating CORS issues 6. **Testing** — 119 unit tests (pkt-line, objects, delta, packfile) + 20 API E2E tests + 32 privacy E2E tests, all passing. 7. **Privacy Fixes** — Found and fixed private repo info leaking via issues endpoint (returned data instead of 404 for unauthorized access). 8. **Organization/Team System** — Designed and implemented org-based auth: organizations, teams, team-repo mappings, role hierarchy (owner > admin > member), and permission levels (read < write < admin). See the [organizations and teams design](docs/engineering/organizations-design.md) for the full design. ## AI Agent Integration (`llms.txt`) SmolForge ships a comprehensive `llms.txt` file designed for AI coding agents (Claude Code, Codex, etc.) to consume programmatically. It includes: - **Complete API reference** — every endpoint with auth requirements, request/response formats, and example payloads - **Transcript hook automation** — CLI-managed hooks for Claude Code, Codex, Cursor, Factory Droid, Devin, OpenCode, Aider, Continue, generic JSONL agents, and Git hook backstops - **Backfill & incremental workflows** — bulk-upload all existing transcripts or hook into your commit flow for continuous uploads The recommended setup is CLI-managed: ```bash sf hooks install alice/demo --agent all --git post-commit,pre-push sf hooks status alice/demo ``` This creates a repo-local upload script, installs native lifecycle hooks where available, and installs Git hook backstops for Codex and other agents. Codex is supported through the Git hook path and `~/.codex/sessions` JSONL discovery. Git has no native `post-push` hook, so SmolForge uses `pre-push` for the final missed-upload check. See [`llms.txt`](llms.txt) for the full guide. ## CLI SmolForge also ships an installable CLI package, `@smolai/forge`, with two equivalent binaries: `smolforge` and `sf`. For new JavaScript and TypeScript repositories, Forge strongly recommends pnpm as the default package manager. Bun is the high-speed alternative when project compatibility allows. npm and Yarn repositories remain fully supported; Forge's UI and CLI provide advisory guidance and never silently migrate a project. ```bash # npm npm install -g @smolai/forge # Homebrew, once the tap is published brew install smol-ai/tap/smolforge # Local formula from this repo brew install --HEAD ./Formula/smolforge.rb ``` Basic usage: ```bash sf auth login --username alice sf repo create demo --description "Edge-hosted git repo" git remote add origin $(sf clone-url alice/demo) sf auth git-credential alice/demo sf auth status alice/demo git push origin main # Install transcript capture before agents start writing code sf hooks install alice/demo --agent all --git post-commit,pre-push sf hooks status alice/demo # Upload Codex, Claude Code, Cowork, Cursor, Factory Droid, or other JSONL agent transcripts sf transcript upload alice/demo --file ~/.codex/sessions/2026/03/29/session.jsonl # Work with Actions and the live machine-readable API guide sf actions list alice/demo --branch main --limit 5 sf llms --save llms.txt ``` The CLI defaults to `https://forge.smol.ai` and can be pointed at another deployment with `--base-url` or `SMOLFORGE_BASE_URL`. The API login and repository Git credential are intentionally separate; `sf auth status owner/repo` reports both states and gives the exact remediation without printing a bearer. Repository credentials last seven days. On a trusted durable machine, the CLI also guides users to the explicit broader mode: ```bash sf auth login --git-scope global --expires never --yes ``` That mode creates a non-expiring `repo:read,repo:write` credential for every accessible Forge repository, stores it only in protected OS storage, never prints it, and makes `sf auth logout` revoke it. It is deliberately never the silent login default. ## Releasing Forge Normal production releases validate an immutable pull-request SHA against the workflow registered from base `main`, serialize the merge, then let one trusted post-merge authority build and deploy the exact merged SHA under one production lease and fence. ```bash SHA="$(git rev-parse HEAD)" sf release enqueue "$SHA" \ --ref codex/my-feature \ --repo swyx/forge \ --pr 123 \ --yes \ --json sf release status merge_request_uuid --repo swyx/forge --json sf release watch merge_request_uuid --repo swyx/forge --json ``` Enqueue records durable merge state; it does not activate a provider inline. After merge, the ProductionDeployer reloads workflow/config from merged `main`, uploads affected immutable provider versions, performs observed-prior CAS, verifies public health, and writes one compact append-only receipt. Retry uses the same SHA with a new attempt and workflow run. The canonical observation commands are `sf release status/watch`. See the [trusted release contract](docs/deploy/trusted-release.md) and [production runbook](docs/operations/production-deployment.md). Queue validation reuses a successful trusted exact-SHA run for the same production base. Provider versions activate in dependency-safe waves. Each Worker owns its health binding list beside its runtime entrypoint, so adding a component does not edit one global binding registry or select every existing Worker. ## Related Documentation - [Forge engineering blog](https://forge.smol.ai/blog) — Architecture, reliability, Git, CI, and deployment field notes - [Blog RSS feed](https://forge.smol.ai/blog/feed.xml) — Subscribe to new Forge engineering posts - [llms.txt](llms.txt) — Machine-readable API reference and AI agent integration guide - [Documentation](docs/) — User guides, product contracts, engineering reference, design, operations, and project history - [Publishing the Forge CLI to npm](docs/operations/cli-npm-publishing.md) — Maintainer preflight, dry run, publication, and verification - [Organizations and teams](docs/guides/organizations.md) — Quick curl examples for the org/team API - [Production deployment](docs/operations/production-deployment.md) — Release, recovery, rollback, and smoke checks - [Character skins](docs/character-skins.md) — Animation families, variants, sprite strips, manifests, and validation ## Running Locally This repository predates Forge's pnpm-first recommendation and intentionally retains its committed npm lockfile until a measured migration is performed. ```bash # Install dependencies npm install # Start the API (with D1/R2 local emulation) cd packages/api npx wrangler dev # In another terminal, start the frontend cd packages/web npm run dev ``` ## Running Tests ```bash npm run typecheck npm test # Local-only unit and integration tests npm run test:migrations # Clean schema plus data-preserving 0015 -> current-head upgrade npm run test:smoke # Real Worker: push, clone, pull, issue, PR, and merge ``` Remote E2E tests are opt-in, require `E2E_BASE_URL` to name a disposable staging deployment, and refuse known production hosts: ```bash E2E_BASE_URL=https://staging.example.test npm run test:e2e:remote:api E2E_BASE_URL=https://staging.example.test npm run test:e2e:remote:web ``` ## Deploying Production changes enter the protected merge queue with `sf release enqueue`. After trusted affected CI passes, the queue advances `main` and hands the exact merge SHA to ProductionDeployer. A release is complete only when its canonical status is terminal, required components and bindings are healthy, and the public verification receipt is recorded. A Git push or successful build alone is not deployment evidence. See [production deployment](docs/operations/production-deployment.md) for break-glass recovery, rollback, and smoke checks. ## What's Next - Pull request conversations, reviews, approvals, and requested changes - Real merge commits plus squash and rebase strategies - Additional isolated runner images and package-manager cache strategies - Fork-network browsing and optional upstream synchronization ## Archive The `archive/ralph/` directory contains files from the [ralph-claude-code](https://github.com/frankbria/ralph-claude-code) autonomous development loop that was used during initial scaffolding. These are preserved for historical interest — they show the prompts, fix plans, and session logs from the automated build process.