SmolForgeField Notes

Notes on systems,
code, and craft.

All notes

Engineering

We Compared Flue, Mastra, and AI SDK Harness

One shared suite hid differences that restart, build, and sandbox probes exposed.

Forge compared three agent runtimes with a shared conformance suite and runtime-specific probes; only Flue survived a Worker restart, and none became production authority.

We compared Flue 2.0.1, Mastra AgentController 1.55.0, and AI SDK 7 with Harness 1.0.54 in two stages. First, every candidate implemented the same seven-behavior Forge contract. Then each had to cross the runtime boundary it claimed to fit: restart a Worker conversation, build the controller as a Worker, or acquire an acceptable coding sandbox.

All three passed the shared conformance suite. The boundary probes immediately split them apart. Mastra's exact composition did not build as the target Worker. Harness constructed under workerd but never acquired the coding sandbox. Only Flue completed the local Worker restart probe.

That contrast was the result. A green adapter suite measured what the Forge shim could supply as well as what each runtime owned. The runtime-specific probes exposed the difference. No candidate became Forge's production authority or shipped runtime. Forge Agent V1 later shipped its own Instant runner and OpenCode Workspace driver behind a boundary that kept repository grants, exact source identity, approvals, public events, and artifact acceptance portable.

We compared runtimes behind one Forge contract

Forge gives each repository a durable, multi-turn agent. A thread can begin with a read-only question inside Instant compute, then move to a writable Workspace for a code change. Changing compute must not change the thread identity, widen the repository grant, or discard the event history.

That requirement ruled out making any framework session the public authority. Forge still had to own:

  • the actor and repository grant;
  • the exact accepted source SHA and output branch;
  • public thread, message, run, event, and approval IDs;
  • duplicate-message admission, budgets, cancellation intent, and audit records; and
  • independent diff, test, commit, and publication acceptance.

Framework conversation IDs and resume pointers could remain private. A runtime could run the loop, but it could not decide what the loop was authorized to do.

The bakeoff therefore asked a narrower question: which private runtime removes real execution work without absorbing Forge's authority?

One shared contract, then three boundary probes

We defined a small BakeoffRuntime interface with seven required behaviors: restart memory, cursor replay, duplicate-message admission, approval, scoped coding delegation, cancellation, and a portable checkpoint. The same deterministic Forge host supplied the repository grant and was the only component allowed to return a successful diff-and-tests receipt.

The mechanism looked like this:

Forge authority
  grant · exact SHA · approval · artifact acceptance
                    │
                    ▼
private BakeoffRuntime
  send · replay · cancel · checkpoint
        ┌───────────┼───────────┐
        ▼           ▼           ▼
      Flue       Mastra      Harness
        │           │           │
  Worker restart  Worker build  sandbox bridge

The common suite tested public behavior. A second evidence receipt classified each behavior as framework-native, Forge-owned, or unsupported. Finally, each candidate had to cross the boundary it claimed to fit. This last step mattered most: a Forge shim can make a weak runtime look conformant.

The checked-in suites still pass: 24 test cases across the shared contract and three candidates. Those tests use deterministic fixtures. They do not prove hosted model quality, production latency, tenant isolation, or a deployed Worker.

The results were role-specific

Candidate What the prototype proved What it did not prove
Flue 2.0.1 Native conversation persistence, replay, admission, cancellation, and subagent execution; generated Worker state survived a local workerd restart Remote deployment, multi-region behavior, run-scoped cancellation, or exactly-once external effects
Mastra AgentController 1.55.0 Strong live controller state, tool approval and resume, persisted messages, and cancellation while parked The exact AgentController composition did not build as the target Worker
AI SDK 7 + Harness 1.0.54 A useful coding-driver seam; Codex ran locally and Harness constructed inside workerd Harness and Codex did not run together because no acceptable bridge sandbox was acquired

This is not a framework quality ranking. It records three prototypes from August 2, 2026, on Node 22.22.2 and local workerd. The environments and integration depth differ, so a single numeric score would imply precision the experiment did not have.

Flue won the local restart probe

Flue used its real Node runtime, SQLite persistence, SDK admission, stream history, durable tools, and submission idempotency. Its Cloudflare fixture used the documented generated Worker and SQLite Durable Object path.

The probe sent a first turn that introduced the codename CINDER, waited for the canonical stream to settle, killed the Vite/workerd process, and started a fresh process against the same Durable Object storage. The new process recovered the conversation and answered CINDER on the next turn.

That is direct evidence for one property Forge wanted from an Instant kernel: a single structural owner can resume a durable conversation after process replacement. It was not evidence of a production deployment. It also exposed two unresolved boundaries.

First, Flue's public abort stopped the conversation instance, while Forge cancellation named one public run. That mismatch is safe only while one run can be active. Second, a durable tool step can be recorded once while its external side effect executes again around a crash. Forge would still need deterministic task IDs and idempotent effect endpoints.

Even the compatibility path carried a warning. The first local Flue build used @cloudflare/vite-plugin 1.39.2 and failed at runtime because its workerd lacked a tracing method. Version 1.50.0, still inside Flue's declared range, passed. “Supports Cloudflare” was not enough; the generated artifact and exact runtime graph had to execute.

Mastra won the controller round, not the Worker round

Mastra's AgentController provided the clearest interaction model. The prototype exercised native sessions, streamed tool approval, approval resume, abort while waiting, reducer-owned display state, LibSQL message persistence, and a constrained subagent configuration.

The Worker build changed the recommendation. The exact AgentController composition, including Mastra's Cloudflare Durable Object storage adapter, reached a Node-only dependency path through execa, npm-run-path, and unicorn-magic. Wrangler stopped before Miniflare could run. Removing the optional Workspace object did not remove that path.

The precise conclusion was limited to the tested versions: AgentController 1.55.0 was not a direct Forge Instant Worker engine under Wrangler 4.112.0. Mastra's approval and display-state design remained useful reference material, but its live event stream did not replace Forge's durable public cursor or fenced ownership.

Harness proved a seam, not a coding runtime

AI SDK Harness addressed a different layer: normalizing coding-agent sessions. The candidate compiled the real HarnessAgent with the official Codex adapter. A local smoke launched Codex CLI 0.141.0 in a temporary read-only directory. A separate workerd probe imported and constructed the Harness modules.

The two probes never joined. The Worker response reported that no sandbox was acquired. A bridge-backed coding CLI needs isolated compute that can expose a port, carry narrow provider authentication, enforce egress, and tear down deterministically. The available adapters we inspected were version-skewed, intended for owner-local use, or lacked the containment Forge required.

Harness therefore looked like a good private Workspace driver abstraction, not a durable conversation store. Its green conformance results depended on Forge-owned persistence, replay, deduplication, approval, and artifact logic.

Production kept the boundary and changed the engines

The prototypes were committed as executable evidence in b1f09d5. They were not merged into the production repository-agent dispatch path.

Forge Agent V1 later shipped two different implementations behind the same public thread model:

  • Instant uses a Forge-native Worker runner. It reads a bounded evidence set from the exact accepted SHA, calls Workers AI, validates citations, and records the terminal result in Forge's D1 event and receipt ledger.
  • Workspace Alpha uses pinned OpenCode inside a run-scoped Cloudflare Sandbox. Forge supplies a short-lived model token, independently inspects the stopped filesystem, reruns declared tests, reconstructs the change from the accepted SHA, and publishes a fenced draft pull request.

Build and Machine remain unavailable. The Repository Agents V1 contract describes the shipped boundary in detail.

This does not make the bakeoff obsolete. It corrects its scope. The experiment did not select production code; it established what production code must not own. The shipped driver interface preserves Forge IDs and records provider identifiers only as private mappings. REST and MCP both call the same protocol-neutral repository-agent service.

What the comparison actually changed

Three lessons survived the candidate churn.

Conformance needs provenance. A passing behavior must say whether the framework supplied it or the host shim did. Otherwise an adapter can recreate the entire platform and award the framework a green check.

Import, build, and execution are separate gates. Mastra had a Cloudflare storage adapter but failed the complete Worker build. Harness constructed under workerd but never acquired a coding sandbox. Flue built under one compatible plugin version and failed at runtime under another.

Public authority must outlive private engines. A Forge thread can change drivers only because the repository grant, exact SHA, approval ledger, event cursor, and artifact acceptance never belonged to Flue, Mastra, Harness, Workers AI, or OpenCode.

The next useful test is no longer another feature matrix. It is a tier-handoff failure test: accept one public thread in Instant, move an authorized turn into Workspace, kill the execution owner around a consequential effect, and prove that recovery neither duplicates the effect nor changes the public identity. If a candidate cannot pass that test behind Forge's authority boundary, its ergonomic advantages are secondary.

Read the prototype contract, Flue's Cloudflare guide, Mastra AgentController, and the AI SDK Harness source.