# Forge release state machine v2

This is the normative state and authority contract for platform releases. The
machine-readable policy is
[`../../config/forge-release-system-v2.json`](../../config/forge-release-system-v2.json).
The incident basis is
[`../operations/release-modernization-incident-synthesis.md`](../operations/release-modernization-incident-synthesis.md).

## Authority and durable truth

An immutable release record and required-component manifest in D1 are the
durable read model. Provider versions, artifact digests, traffic, bindings,
triggers, and hostname observations are external facts referenced by that
record. An environment-scoped Durable Object serializes managed-Worker
promotion and owns its monotonic fencing token independently of D1. Connected
edge promotion is serialized separately by generalized Deploy's provider-target
lease, exact provider-version compare-and-swap, and production-pointer
generation.

The Durable Object is not a second edge deployer. Losing, expiring, cancelling,
or superseding its managed-Worker lease makes all earlier managed-Worker tokens
permanently stale. The controller enqueues an exact edge artifact and observes
its generalized Deploy receipt; it never performs the edge traffic mutation.

Only generalized Deploy may activate connected edge artifacts. Direct local
provider access is audited break-glass recovery only and may never compete for
traffic, triggers, or active pointers.

## Snapshot isolation

Each release is identified by its D1 release ID, exact source SHA, exact base
SHA, and sealed plan digest. The controller must seal that plan before entering
`promoting`, and every provider authorization revalidates the digest. The
current branch head is never an input to resume, retry, activation, or rollback.

A later `main` update only enqueues another release. Descendants may coalesce
while `queued` or `validating`; after provider work begins, the active release
finishes or rolls back under its existing fence before a later release may
promote. These rules are executable policy in `releaseIsolation` inside the
machine-readable configuration.

## Canonical phases

```text
resolve_source
→ authorize_environment
→ acquire_lease
→ read_provider_provenance
→ calculate_impact
→ resolve_validation_evidence
→ run_validation
→ resolve_artifacts
→ verify_bootstrap
→ promote_components
→ reconcile_edge
→ run_production_canary
→ verify_public_manifest
→ mark_deployed
→ enqueue_large_snapshot
→ publish_receipts
→ release_lease
```

Phases are monotonic for one fenced lease. Resumption reads the durable release
record and provider facts, rechecks the fence, and continues at the first phase
whose completion receipt is missing or no longer true. It does not infer
completion from a lost command response.

## Release states

| State | Meaning |
| --- | --- |
| `planned` | Exact source, target, impact, gates, and draft manifest resolved; no promotion authority implied |
| `validating` | Missing trusted gates are running |
| `building` | Immutable affected-component artifacts are being produced or resolved |
| `promoting` | A current fenced coordinator is mutating providers in dependency order |
| `verifying` | Promotion is complete; edge, public manifest, and synchronous canaries are being proven |
| `deployed` | Every synchronous completion criterion is exact and healthy |
| `degraded` | An independent subset may be healthy, but a required component failed and final edge activation was withheld |
| `failed` | A terminal pre-activation or invariant failure prevents progress |
| `cancelled` | The current authority deliberately stopped the release |
| `superseded` | A newer explicitly authorized release replaced this one |
| `rolled_back` | Promotion mutated a provider, verification failed, and the controller restored the recorded rollback target |

Terminal states are `deployed`, `degraded`, `failed`, `cancelled`, and
`superseded`, plus `rolled_back`. A release never transitions directly from
build or promotion to `deployed`.

## Required-component manifest

The final manifest is `forge-required-component-manifest/v2`. It contains every
required component, including unaffected components legitimately running an
older source SHA. Each component entry identifies:

- source SHA;
- immutable artifact digest;
- provider version ID;
- binding-policy/contract digest;
- expected traffic;
- rollback provider version;
- dependency-health proof;
- public or service-binding observation as applicable.

The planner initially emits a draft. A selected component is
`immutable-artifact-required`; an unaffected component may
`reuse-live-provider-version`; missing live provenance remains unresolved and
fails closed. The manifest becomes immutable only after every selected artifact
and expected provider version has been resolved, before promotion begins.

`deployed` requires all of the following:

- current managed-Worker fencing token, or the connected edge provider-target
  lease and exact production-pointer generation;
- exact required-component manifest;
- exact provider versions and unambiguous traffic;
- required bindings and dependency graph healthy;
- edge version, triggers, and traffic reconciled;
- correct version on every public hostname;
- successful small push, clone, and strict-fsck canary.

Optimized large-snapshot readiness is deliberately absent from this list. It is
an asynchronous readiness sub-state with its own receipts and SLOs.

## Large-snapshot handoff and attempt history

`enqueue_large_snapshot` means only that the coordinator durably submitted one
job and persisted `repository_id`, `refs_generation`, and `job_id` in release
evidence. Snapshot completion is not a lease phase and cannot delay
`deployed`. After durable enqueue and synchronous receipt publication, the
coordinator releases the production lease. A separate reconciler observes
`queued`, `running`, `ready`, `failed`, `cancelled`, or `superseded` readiness.

Every execution attempt is append-only. A terminal attempt is never reset to
`queued` or `running`. A retryable failure creates the next numbered attempt on
the same job. Operator requeue creates a new job ID with operator identity,
reason, and prior job ID. Generation supersession also creates a new job ID,
names the prior terminal job, and requires a strictly newer refs generation.
These rules preserve exhausted-attempt evidence while allowing safe recovery.

The 30-second snapshot SLO applies to a warm incremental generation after a
small ref change. It is not a cold-snapshot deadline. Cold Forge-sized work has
a measured speedup target and remains asynchronous even when a repository has
thousands of objects or hundreds of megabytes of pack data.

## Planning and evidence

Affected CI and the controller read the exact Git tree, hash every declared
component input, expand workspace dependency verification, and emit
`forge-release-plan/v2`. The controller compares those digests with successful
per-component activation receipts; provider source SHAs remain provenance and
are not impact baselines.

Ordinary changes validate affected components. Schema, authority, binding,
shared-contract, release-system, explicit-full, or uncertain changes force the
documented broader policy. Release-system-only changes rerun contract proof
without rebuilding unaffected Workers.

Reusable feature evidence is `forge-feature-ci-evidence/v2`, authenticated with
HMAC-SHA256 by the trusted runner. Its signature covers the exact candidate,
expected target-base SHA, immutable changed files, component input digests,
runner identity, result, and expiry. Broader offline validation evidence may
use `forge-validation-evidence/v2` with Ed25519. Evidence is reusable only when
the declared inputs and policy identity match exactly.

The coordinator always reruns source provenance, lease checks, impact planning,
recovery preflight, provider authority, binding verification, live health,
activation fencing, public manifest proof, and the production canary.

## Promotion and failure behavior

Builds may run concurrently. Promotion order is dependency constrained:

1. canonical D1 schema (verification in normal releases; recreation is never implicit);
2. minimum operator identity bootstrap;
3. deploy-control bootstrap readiness;
4. runner;
5. repository services;
6. supporting domain Workers;
7. edge bindings and triggers;
8. edge traffic;
9. public hostname verification.

A failed required component stops its dependents. Independently successful and
healthy promotions remain recorded, the release becomes `degraded`, final edge
activation is withheld, rollback targets remain intact, and the receipt names
the exact mismatch plus `next_safe_action`. The system does not claim a
distributed atomic rollback.

Retry is allowed only for named bounded classifications in the v2 policy.
Structured application failures, foreign or unexpected bindings, invalid auth,
ambiguous traffic, permanently wrong versions, and absent rollback evidence are
terminal. Arbitrary sleeps are not part of the contract.

Snapshot R2 transport, Sandbox transport, tar hydration transport, and upload
transport failures are retryable within their bounded budgets. Missing Git
objects, digest mismatches, and invalid or corrupt packs are integrity failures
and terminal. In particular, `SnapshotIntegrityError: Git object is missing`
must never be converted into a generic retryable infrastructure failure.

## D1 safety boundary

Normal release execution is non-destructive. It may verify and apply an approved
canonical migration path, but it may not recreate production D1. A D1 cutover is
a separate explicit command and approval boundary with export, Time Travel,
old/new database IDs, binding verification, bootstrap, authentication, Git
reachability, abort, and rollback evidence.

Recovery preflight spans both storage systems. It records the old and new D1
repository identities and their corresponding R2 object namespaces, verifies
every restored ref tip and reachable object in the new namespace, and reports
missing-object counts before and after repair. Any repair copies only immutable
content-addressed objects, verifies every copied object against its content
address, and exact-compares restored ref-tip objects. Restoring D1 rows without
reconciling R2 namespace provenance cannot pass preflight.

## Active release paths

The protected merge queue, evidence verifier, Durable Object lease, D1 ledger,
managed-component executor, and generalized Deploy edge promoter are active.
There is one promoter for each component and no mutable runtime switch between
competing deployers. Release tasks submit immutable candidates and observe
`sf release status/watch`; they do not negotiate deployment ownership with one
another.