Two skills recently failed us in different ways. One encouraged unsafe autonomy. Another applied maintainability advice too broadly. Deleting them stopped the immediate problems, but it did not answer the useful questions: when did each skill load, what exact instructions did the agent receive, what changed afterward, and did the same version help anyone else?

The missing system was not another download counter. Agent skills need package identity, runtime evidence, and outcome correlation. Forge's proposal is to build those as three products that work together but remain independently useful: a SKIT package manager, a vendor-neutral OpenTelemetry layer, and a hosted correlation and effectiveness service.

This is the full product and engineering plan, lightly edited for public reading. It is a proposal, not a shipped-capability announcement. The first milestone is intentionally narrower than an effectiveness score: produce a privacy-safe event trail that can identify the exact package, contained skill, agent session, repository, activation, and subsequent work.

Outcome

Forge should become the system of record for reusable agent skills and the evidence of how those skills perform:

1. Authors package and publish one or more related skills as a versioned SKIT (skill kit).
2. sf add skit discovers, installs, updates, and synchronizes complete SKIT releases while allowing contained skills to be enabled independently; sf skit install is an exact alias.
3. sf can install a local OpenTelemetry collector and agent-specific instrumentation that emit vendor-neutral OTLP to Forge, user-selected observability backends, or both.
4. Forge ingests normalized skill activation and agent-run telemetry with repository, session, commit, and SKIT-version identity.
5. Forge integrates with a pinned Git AI CLI to connect skill activations and agent/tool operations to explicit checkpoints, attributed line ranges, commits, review, rework, reverts, and code durability.
6. The web application presents a skill library, installation state, a public leaderboard based on observed use rather than installs, and—after enough evidence exists—effectiveness comparisons.

Forge must provide value independently of any one upstream project or hosted directory. It should interoperate with the open Agent Skills ecosystem and Vercel's skills CLI while retaining its own installer, registry, release identity, OTel pipeline, organization controls, and product UI.

The first useful milestone is not an effectiveness score. It is a trustworthy, privacy-safe event trail answering:

• Which exact SKIT release and contained skill were available?
• Was that individual skill selected, injected, or successfully loaded?
• Which agent, repository, session, and turn used it?
• What happened afterward?

Product vocabulary

• Skill: an independently addressable activation, telemetry, permission, and effectiveness unit. Within a SKIT its stable ID is namespaced as <skit-id>:<skill-name> and its exact version is identified by content hash.
• SKIT: an atomically published, versioned, installed, and distributed skill kit containing one or more skills plus optional shared scripts, references, assets, hooks, and MCP dependency declarations. A single-skill SKIT is a bundle of one.
• Skill source: a local directory, Git repository, Forge SKIT, plugin, or other location from which a skill is obtained.
• Collection: a lightweight discovery and curation grouping of independently versioned SKITs. Collections are not installed atomically.
• Installation: a complete materialized SKIT release in one or more agent-specific skill roots, plus local enablement state for its contained skills.
• Activation: an agent or user selects one contained skill and the host attempts to load that skill's full instructions. Loading one skill does not activate its siblings.
• Observation: a normalized event derived from native telemetry, a host plugin/hook, transcript evidence, or a fallback sensor.
• Outcome: evidence after activation, such as tool/test results, commit production, user correction, revert, duration, token use, or explicit feedback.
• Git AI checkpoint: Git AI's per-edit/checkpoint identifier (t_<14hex>) within a session attestation key. Forge normalizes it as git.ai.checkpoint.id; it is not an OpenTelemetry trace ID.
• Authorship outcome: explicit Git AI evidence that generated lines were accepted, committed, modified, overridden, abandoned, reverted, or survived later repository changes.

Bundle contract

SKIT: owner/code-quality
Release: 2.3.0

Skills:
  owner/code-quality:maintainability
  owner/code-quality:dead-code
  owner/code-quality:architecture-review

The contract is:

1. A SKIT release has one version and is the atomic publishing, installation, update, testing, and attestation unit.
2. Every contained skill has a stable namespaced identity and content hash, and remains the activation, permission, telemetry, and effectiveness unit.
3. Loading one skill does not activate the rest of its SKIT.
4. Users may disable individual contained skills after installing the complete SKIT release.
5. Shared executable capabilities and permissions are disclosed at the SKIT level.
6. Contained skills do not have independent semantic versions initially; their content hashes identify their exact contents within a SKIT release.
7. A skill that needs an independent release cadence should become its own SKIT.
8. Collections group SKITs for discovery and curation without creating another atomic installation boundary.

Distribution and implementation lineage

• Use the dormant, ISC-licensed balgaly/skit project as a vendored starting point for local package-management behavior, not as a live runtime dependency.
• Preserve its license and attribution, record the exact upstream commit, review the imported code for filesystem and archive safety, and put Forge-owned tests around every retained behavior.
• Retain useful concepts such as a canonical local library, README-frontmatter package metadata, cross-platform links, source tracking, profiles, synchronization, and doctor diagnostics.
• Evolve the implementation behind Forge's sf command surface and data contracts; compatibility with the upstream skit binary is not a product constraint.
• Interoperate with Vercel's ecosystem through Agent Skills-compatible repositories, source URLs, archives, and agent-directory conventions. Do not require the Vercel CLI, API, telemetry endpoint, or skills.sh to install or operate a Forge SKIT.
• Serve Forge releases through direct, immutable download URLs that generic clients such as npx skills add <url> can consume.
• Allow sf to install and observe compatible Git, URL, archive, and local-path skills without requiring authors to publish them to Forge first. Synthesize a provisional SKIT identity from a privacy-safe canonical source fingerprint and a provisional contained-skill identity from that SKIT identity plus relative skill path; use the computed content hash for the exact observed contents. Never silently replace provisional identities when content is later published—record an explicit identity alias so historical observations remain reconstructable.

Existing Forge assets to reuse

Transcript capture and correlation

• packages/cli/src/hooks.ts already installs and inspects agent-specific hooks, writes a repo-local managed script, finds local transcripts, and handles Claude Code, Codex, Cursor, OpenCode, Devin, Factory, Aider, Continue, and generic fallbacks.
• packages/cli/src/commands.ts already uploads raw JSONL with repository, commit, agent, and session hints.
• packages/api/src/features/transcripts/ already parses heterogeneous agent formats, masks secrets, persists normalized messages, and correlates sessions to commits.
• packages/web/src/pages/TranscriptListPage.tsx and transcript components already provide a repository-scoped evidence UI.

This should become a general agent evidence capture subsystem. Transcript upload remains one input; OTLP observations become another.

Git AI as a complementary attribution engine

Git AI owns a different and complementary layer: explicit agent checkpoints, line-range authorship, Git Notes under refs/notes/ai, and attribution preservation through ordinary Git history rewriting. Forge should integrate with that implementation rather than recreate or vendor its Git mechanics.

The division is:

• SKIT/Forge: which skill loaded, why it activated, exact release/content hash, permissions, instructions, load failures, feedback, incidents, and vendor-neutral OTel traces
• Git AI: which agent checkpoint produced which line ranges and whether those lines were accepted, committed, modified, overridden, abandoned, reverted, or retained

skill activation
  → agent/tool operation
  → Git AI checkpoint
  → attributed line ranges
  → commit / pull request
  → review, rework, revert, durability

Pin the first compatibility baseline to upstream commit c57c6c24be03ba89ce7d47783841d2f49ba8ee45, Git AI Standard authorship/3.0.0, and the Apache-2.0 license at that revision. Forge does not vendor the engine source, follow moving main, depend on Git AI Cloud, or require maintainer coordination. The sf distribution owns a checksum-pinned Git AI installation payload or artifact manifest, and the default sf observe install flow installs and configures that tested Git AI build alongside the collector and adapters. Forge-managed installations disable automatic Git AI upgrades; updates are deliberate compatibility changes with fixtures, schema checks, release notes, and rollback. If the upstream standard cannot carry trustworthy checkpoint-level correlation, Forge publishes and implements its own additive correlation extension without forking the attribution engine unless later forced to do so.

Gist domain and UI

Forge Gists already provide useful product patterns:

• owner and visibility
• multiple files
• immutable revisions
• stars, forks, comments, share links, collections, and tags
• bounded validation and quotas
• public discovery and owner views
• a file-oriented editor and workbench UI

Reuse validation patterns, collaboration primitives, social components, and selected UI components during the backend migration. Do not store SKITs in the Gist tables or add an is_skill discriminator. SKITs need nested paths, multiple independently addressable child skills, shared bundle resources, README-frontmatter descriptors, releases, installation/provenance state, compatibility, dependency declarations, and two-level telemetry identity.

Platform foundations

• Existing repository identity, PAT scopes, organizations, teams, and visibility rules can own SKIT authorization.
• D1 is suitable for canonical SKIT metadata, relationships, installation state, correlation indexes, and bounded rollups.
• R2 is suitable for immutable SKIT archives and raw or quarantined OTLP batches.
• Existing release/migration verification should gate all new schema and component boundaries.

Target architecture

Component inventory

1. Shared SKIT contracts

Create versioned shared types and schemas before routes or UI:

• SKIT identity: owner, slug, stable ID, source, visibility
• editable draft and immutable release records
• root README.md YAML frontmatter describing the SKIT and its contained skills
• contained skill identity: stable namespaced ID, relative root, primary SKILL.md, content hash, and default enablement
• recursive release file inventory with normalized relative paths
• per-skill SKILL.md metadata and validation diagnostics
• bundle and per-skill content SHA-256 values plus optional source Git commit
• shared resource, hook, executable capability, and MCP dependency declarations
• declared trigger modes, mutation scopes, external-side-effect capabilities, approval expectations, and expected outputs for every contained skill
• compatibility and declared dependencies
• complete-release installation state, per-skill enablement overrides, target, and agent adapter status
• observation and outcome envelopes

Define three separate telemetry contract families with separate consent, retention, and access rules:

1. Package-management analytics: installs, updates, removals, source resolution, and CLI health. These describe distribution, not runtime use.
2. Runtime observations: selection, injection, loading, activation, tool execution, and adapter/collector health. These power usage views and the public leaderboard.
3. Outcome evidence: tests, builds, commits, corrections, reverts, time, tokens, cost, and explicit feedback. These support effectiveness analysis and require stronger correlation and privacy controls.

Never infer runtime usage from an install event, and never present correlated outcome evidence as causal effectiveness without an appropriate comparison design.

The Agent Skills metadata map should carry portable author metadata. Forge should enrich it with independently computed hashes and provenance rather than trusting author-provided version strings alone.

Suggested portable metadata:

metadata:
  observability/skit-id: "owner/code-quality"
  observability/skill-id: "owner/code-quality:maintainability"
  observability/owner: "owner"

The SKIT release and computed content hashes belong to the published release and Forge-managed local installation state, not to independently authored child-skill semantic versions.

On-disk SKIT contract

A SKIT uses its root README.md as both human-facing package documentation and the bundle descriptor. There is no skit.yaml, generated lockfile, or repo-committable installation state.

code-quality/
├── README.md
├── LICENSE
├── skills/
│   ├── maintainability/
│   │   ├── SKILL.md
│   │   ├── scripts/
│   │   ├── references/
│   │   └── assets/
│   └── dead-code/
│       ├── SKILL.md
│       └── references/
├── shared/
│   ├── scripts/
│   ├── references/
│   └── assets/
├── hooks/
└── mcp/

The README.md begins with Forge YAML frontmatter:

---
skit: 1
id: owner/code-quality
skills:
  - name: maintainability
    path: skills/maintainability
    default_enabled: true
    shared:
      - from: shared/references/code-quality
        to: references/code-quality
  - name: dead-code
    path: skills/dead-code
    default_enabled: true
capabilities:
  executables:
    - shared/scripts/analyze.ts
  hooks:
    - hooks/observe.json
  mcp:
    - mcp/source-control.yaml
---

# Code Quality
Human-facing description, usage, compatibility, and release notes.

Each child owns its directory and may not traverse into a sibling. Shared files are available only through explicit mappings in the root frontmatter; Forge materializes or links those mappings into the installed child projection. A child may declare its own version in SKILL.md, but Forge does not require one. Forge always computes and records each child skill's content SHA-256, which is the exact identity when no child version exists. The published SKIT release remains the atomic version and installation unit.

2. SKIT registry backend

Provide first-class API and persistence for:

• create and update drafts in Forge or through synchronized local/Git sources
• validate a SKIT directory, every contained skill, and shared resources
• publish immutable releases
• list files and download a release archive
• search and discovery
• forks, stars, comments, collections, and tags
• source repository linkage
• deprecation, quarantine, and replacement guidance
• release signatures or attestations in a later phase

Recommended storage split:

• D1: SKIT metadata, contained skill identities, release metadata, paths and hashes, social relationships, install and enablement records, visibility, and aggregate counters
• R2: immutable release archives, larger file bodies, and optional source snapshots

Do not copy Gist's ten-root-file constraint. Validate safe relative paths, reject traversal and unsafe symlink materialization, and set separate per-file, per-skill, per-SKIT, release-count, and owner quotas. Include public, unlisted, private, user-owned, and organization-owned authorization shapes in the initial schema so private organization SKITs do not require a later data-model rewrite.

Forge draft and local/Git synchronization contract

Forge-editable drafts and bound local or Git sources are peers. Synchronization is revisioned three-way reconciliation, never last-write-wins:

1. Every source binding records the Forge draft revision, source revision or Git commit, last common bundle hash, and per-file hashes.
2. sf skit sync compares Forge, the source, and that last common snapshot.
3. Non-overlapping file additions, deletions, and edits merge automatically. Overlapping text edits use a normal three-way merge.
4. Conflicting edits, rename/delete conflicts, binary changes, and incompatible README-frontmatter changes require explicit resolution.
5. Forge never overwrites dirty local files, never silently force-pushes Git, and never publishes a draft with unresolved conflicts.
6. A resolved merge creates a new Forge draft revision and updates the common snapshot only after both sides acknowledge it.
7. The MVP may produce a patch or working-tree merge for the user to commit. A later integration may create a branch or pull request, but direct source pushes are not part of the default sync path.

Synchronization metadata belongs to Forge's draft/source-binding records and local managed state. It is not stored in a SKIT lockfile.

3. Gist-to-SKIT conversion

Perform a one-time backend migration as part of introducing the SKIT domain:

• convert every existing Gist record and its files into an editable SKIT draft owned by the same user or organization
• identify contained SKILL.md files; when a record cannot yet validate as a publishable SKIT, retain it as an unpublished draft with migration diagnostics
• carry forward content, author, visibility, description, and tags into the SKIT model
• take a private, access-restricted migration snapshot before cutover and retain it for 30 days solely for operational rollback
• remove Gist routes, APIs, commands, navigation, tables, and product terminology in the same cutover
• reject old Gist API calls after deployment rather than maintaining a compatibility or read-only surface

There is no conversion button, owner-review queue, deprecation window, old-Gist redirect, or preserved Gist product. Migration verification must prove record and file counts before the legacy storage is dropped, but SKIT becomes the only surviving product model immediately. The private rollback snapshot is not a product surface and is automatically destroyed after 30 days.

4. sf skit CLI family

Proposed initial surface:

sf add skit <source> [--version x.y.z] [--agent codex,claude,opencode|all]
sf skit init [path]
sf skit validate [path] [--json]
sf skit publish [path] [--version x.y.z]
sf skit pull <owner/name> [--version x.y.z]
sf skit install <source> [--version x.y.z] [--agent codex,claude,opencode|all]
sf skit enable <owner/name:skill> [--agent ...]
sf skit disable <owner/name:skill> [--agent ...]
sf skit update [owner/name|--all]
sf skit remove <owner/name> [--agent ...]
sf skit list [--local|--remote]
sf skit sync [path]
sf skit doctor

sf add skit is the canonical cross-resource command; sf skit install must invoke exactly the same implementation and produce identical output, telemetry, and installed state. <source> may be a Forge owner/name, immutable Forge release URL, a standard cloneable Git URL with an optional ref and subpath, direct SKILL.md, supported archive, or local path. Forge does not implement GitHub shorthand, GitHub tree-URL parsing, or any other GitHub-specific source contract.

The installer should maintain Forge-managed local state containing SKIT ID or provisional SKIT identity, release ID or source ref, every contained skill ID or provisional skill identity and content hash, per-skill enablement overrides, installed paths, target agents, source, and installation time. It does not generate a project lockfile. Installation and update always materialize the complete release; enable/disable operates on individual contained skills. All mutations must use atomic writes and preserve user-owned files and unrelated configuration.

Source compatibility contract

Guaranteed, fixture-tested sources are:

• Forge SKIT IDs and immutable Forge release URLs
• local SKIT directories whose root README.md has valid SKIT frontmatter
• local single-skill directories containing SKILL.md
• standard cloneable Git URLs, including self-hosted Git, with explicit ref/subpath syntax
• direct HTTP(S) SKILL.md URLs
• .zip, .tar, .tar.gz, and .tgz archives containing a valid SKIT or Agent Skills layout
• Agent Skills found at the repository root, skills/<name>, skills/<category>/<name>, .agents/skills/<name>, .claude/skills/<name>, or .opencode/skills/<name>

Best-effort imports include recursive discovery outside those roots, agent-specific marketplace/plugin manifests, host-specific browser URLs, additional agent-directory conventions, mutable HTML discovery pages, and other archive formats. Best-effort import behavior may improve without becoming a compatibility promise. Forge release URLs and archives are the canonical hosted distribution mechanism; generic Agent Skills clients may consume them directly without GitHub or another code host.

5. Agent adapters

Each adapter owns four concerns separately:

1. Discover supported agent versions and configuration locations.
2. Install or link a complete SKIT release into the correct skill roots and apply contained-skill enablement overrides.
3. Enable the best available observability signal without overwriting user configuration.
4. Diagnose drift, conflicts, unsupported versions, and partial installs.

Initial adapters:

• Claude Code: configure native OTel logs/metrics; consume claude_code.skill_activated; optionally install hooks only when synchronous enforcement is requested.
• OpenCode: install a small Forge plugin that observes tool.execute.before/after for the native skill tool and emits normalized OTLP events.
• Codex: configure the OTel metrics exporter for codex.skill.injected, collect related run/tool logs, and clearly mark the current lack of per-invocation skill hooks and any plugin-skill blind spots.

Do not pretend all adapters have equivalent fidelity. Every normalized observation must include an evidence source and confidence/fidelity classification.

Contained-skill enablement uses native host controls where possible:

Agent	Native mechanism	Forge projection
Codex	Path-based [[skills.config]] entries with enabled = false	Materialize the complete release and maintain Forge-managed enablement entries.
Claude Code	skillOverrides or Skill(name) permission rules for directly installed skills	Install children directly and set disabled children to off; do not rely on plugin packaging where child overrides are required.
OpenCode	permission.skill.<pattern> = "deny"	Maintain explicit per-child allow/deny entries.
Other agents	No verified individual control	Keep the complete release in Forge's canonical library but project only enabled children into that agent's discovery path.

The canonical library, not an agent-specific directory, is the source of truth. Name collisions or hosts without reliable isolation fail safely with a diagnostic rather than exposing a disabled child.

5A. Git AI bridge and checkpoint correlation

Treat Git AI as a pinned external CLI and standard implementation. The Forge bridge owns compatibility, correlation, local event production, and ingestion without requiring Git AI Cloud or a future upstream SDK.

Adopted behavior

• reuse Git AI's existing cross-agent adapters and explicit before/after-edit checkpoint pattern instead of independently inferring authored lines
• consume its refs/notes/ai authorship records and preserve its generated, accepted, human-overridden, abandoned, and rewrite-aware outcomes
• rely on Git AI for line-attribution survival across supported rebases, squashes, stashes, cherry-picks, resets, merges, amendments, and linked worktrees
• retain local-first asynchronous operation; unavailable Forge export must never interrupt checkpointing or Git work
• use git ai stats --json and supported blame/usage JSON as the first import surfaces, validated against schemas snapshotted at the pinned compatibility revision

Identifier and extension contract

Git AI Standard v3 uses s_<14hex>::t_<14hex> as the attestation key. Forge maps the components as:

s_<14hex>  → git.ai.session.id
t_<14hex>  → git.ai.checkpoint.id

Although Git AI calls the t_ component a trace ID, Forge must never place it in an OTel trace_id field. A real OTel trace/span links to the checkpoint through attributes and span links.

Forge publishes a versioned checkpoint-correlation extension keyed by the full Git AI attestation key:

{
  "schema_version": "forge/git-ai-correlation/0.1",
  "git_ai_attestation_key": "s_c9883b05a2487d::t_9f8e7d6c5b4a32",
  "git_ai_checkpoint_id": "t_9f8e7d6c5b4a32",
  "traceparent": "00-...-...-01",
  "agent_session_id": "...",
  "agent_tool_call_id": "...",
  "skill_activations": [
    {
      "activation_id": "act_...",
      "skill_id": "owner/code-quality:maintainability",
      "skit_release": "1.4.2",
      "content_sha256": "...",
      "attribution_fidelity": "shared"
    }
  ]
}

The bridge writes this first to a local append-only sidecar store and emits it as OTLP. Git AI's checkpoint ID already survives in the line attestation, so Forge can join sidecar metadata to rewritten commits without modifying refs/notes/ai. An optional refs/notes/forge-skit projection may provide portable repository-local correlation later, but the MVP must not mutate or replace the Git AI authorship note format.

Forge-owned event stream and schemas

Until Git AI exposes a stable local event API or OTLP exporter, the bridge produces one itself by combining:

1. composed agent hooks around checkpoint creation
2. changes to refs/notes/ai
3. pinned-schema parsing of git ai stats --json, blame JSON, usage JSON, and checkpoint output
4. the local checkpoint-correlation sidecar

Emit git.ai.checkpoint.created, git.ai.attribution.updated, git.ai.lines.accepted, git.ai.lines.overridden, git.ai.session.abandoned, git.ai.commit.attributed, and git.ai.attribution.survived events with the pinned CLI/schema version and evidence fidelity. Forge owns stable JSON Schemas for every imported shape and quarantines unknown or incompatible output rather than silently misparsing it.

Hook coexistence

sf must merge with Git AI's agent hook adapters rather than overwrite them. The installer snapshots existing configuration, preserves Git AI commands and ordering requirements, adds Forge observers as separate handlers, and verifies both tools with end-to-end canaries. sf observe doctor detects either installer replacing the other, repairs only Forge-managed blocks, and reports when the pinned Git AI adapter layout changed. Forge-managed Git AI installs disable automatic updates so hook changes occur only through an explicit sf compatibility update.

MVP degradation

For the first slice, sf observe install installs the pinned Git AI build by default, or adopts an already installed byte-compatible build after verification. It passes session-granular SKIT context through GIT_AI_CUSTOM_ATTRIBUTES, ingests git ai stats --json, and labels resulting skill attribution session_shared. Trustworthy activation-level attribution begins only when the Forge checkpoint sidecar records the active activation set for each t_ checkpoint. Session attributes must never be presented as checkpoint-exact.

6. Local collector installer and supervisor

Add a global collector family rather than overload transcript hooks:

sf observe install
sf observe status
sf observe doctor
sf observe update
sf observe uninstall
sf observe link [owner/repo]
sf observe unlink [owner/repo]
sf observe destination add <name>
sf observe destination list
sf observe destination remove <name>
sf observe git-ai install
sf observe git-ai status
sf observe git-ai doctor
sf observe git-ai update

sf observe install should:

• provide one-command setup for the collector, Claude Code adapter, pinned Git AI CLI, composed agent hooks, and Forge/local/third-party destinations; --without-git-ai is an explicit opt-out rather than the default
• explain the metadata-only default during onboarding, then separately explain the benefits and risks of optional prompt/tool-content collection and what product capabilities it unlocks
• create a rotatable user-device Forge ingest credential scoped only to telemetry writes when Forge export is enabled; a Forge account is not required for local or third-party OTLP destinations
• install a pinned, checksum-verified OTel Collector distribution or reuse a compatible existing collector
• configure local OTLP gRPC/HTTP receivers
• configure batching, bounded disk-backed retry, redaction, and one or more independent destinations: optional Forge export, arbitrary OTLP endpoints, and local debug/file output
• keep generated collector configuration inspectable and standards-compatible so users can consume, extend, or replace it without reverse-engineering a proprietary transport
• install the correct user service: launchd, systemd user service, or Windows service/task
• merge agent-specific telemetry configuration with backups and rollback metadata
• verify an existing Git AI installation or install the checksum-pinned Forge-tested build, configure composed hooks, disable automatic Git AI updates, and run a checkpoint-to-stats canary
• install once per user and dynamically resolve repositories from working directory, Git remotes, and explicit sf observe link mappings
• add stable resource attributes for Forge user/org, device, repository hint, environment, and adapter version; the server must authorize and resolve repository hints rather than trusting them
• run an end-to-end canary per configured destination and report delivery, redaction, retry, and drop state

Non-interactive installation needs explicit flags for consent, destinations, and mutation. Destination credentials must come from protected configuration, environment references, or the OS credential store rather than appearing in process arguments or generated diagnostics. Uninstall must remove only Forge-managed blocks and files.

Local session and repository resolution

Adapters register each active host session over a Forge-owned Unix socket or Windows named pipe using host session ID, process ID, CWD, agent identity, and any host turn identifiers. The local service independently resolves canonical paths, Git worktree root, Git common directory, worktree-specific Git directory, normalized remotes, current commit, and explicit sf observe link mappings. It assigns separate pseudonymous repository, worktree, and workspace-instance fingerprints so concurrent worktrees never collapse into one session.

These fingerprints are attribution evidence, not authorization. Forge resolves them server-side only within repositories available to the authenticated device/account. Unregistered or ambiguous events remain repo_unresolved; the collector never guesses from its own current directory. A client can forge telemetry, but no client-supplied repository attribute may grant access to, reveal, or publish data from another repository.

Collector extension contract

sf observe destination provides the curated path, while advanced users may add arbitrary Collector components through user-owned configuration fragments loaded alongside Forge's inspectable managed configuration. Forge validates the merged Collector configuration before reload and never writes destination credentials into generated diagnostics.

The supported advanced extension point runs after Forge's default metadata redaction. Users may fully replace the pipeline, but doing so visibly disables Forge's redaction and managed-upgrade guarantees. User fragments survive sf observe update; the managed file remains replaceable and must not be edited by hand.

sf and its adapters should emit standard OTel logs, metrics, and traces with documented semantic attributes. Forge-specific enrichment may add identities and hashes, but the base signal must remain useful to another OTLP consumer without a Forge backend.

Start with a pinned upstream otelcol-contrib distribution because it is the simplest path to a working collector. Forge owns the generated configuration, checksum verification, service lifecycle, and upgrade cadence. Build a smaller distribution only if the prototype demonstrates a material size, security, or operational problem.

7. OTLP ingest gateway

The public ingest boundary should accept OTLP/HTTP first. A local collector can translate gRPC agent exports to HTTP upstream, avoiding a public gRPC requirement in the first release.

Responsibilities:

• authenticate a rotatable user-device credential with telemetry-write scope
• accept protobuf and optionally OTLP JSON
• enforce compressed and uncompressed byte limits
• apply event-count, attribute-count, and cardinality limits
• reject or quarantine prompt/tool bodies unless explicitly enabled
• attach server-authoritative account and repository identity
• deduplicate retries using stable event IDs or batch fingerprints
• persist the raw batch to R2 under the account's raw-retention policy: 90 days on the free tier and at least one year on qualifying paid tiers
• enqueue normalization without blocking agent execution
• return standard OTLP partial-success responses

The collector must fail open with respect to the coding agent: unavailable Forge ingestion should buffer within limits and eventually drop telemetry, never prevent skill loading or code work.

8. Portable OTel semantic convention and Forge normalization

Treat the wire contract as a public product. Publish a versioned, machine-readable Agent Skill OpenTelemetry semantic convention that another collector or backend can use without Forge. Pin the OpenTelemetry semantic-convention and GenAI semantic-convention versions used by each release and set the corresponding OTLP schema_url; do not silently follow a moving main branch.

Use existing OpenTelemetry attributes when their meaning is exact:

• OTel LogRecord top-level fields: Timestamp, ObservedTimestamp, TraceId, SpanId, severity, body, resource, instrumentation scope, attributes, and EventName
• resource identity: service.name, service.version, service.instance.id, telemetry.sdk.*, deployment.environment.name, os.type, and host.arch
• session and GenAI context: session.id, gen_ai.conversation.id, gen_ai.agent.*, gen_ai.request.model, gen_ai.usage.*, and gen_ai.operation.name
• errors and immutable packages: error.type, exception.*, artifact.version, artifact.hash, and artifact.purl where a valid package URL exists
• source control attributes from the OTel VCS registry when the underlying host exposes them and the destination's privacy policy allows them

Do not overload an existing OTel attribute with Forge-specific meaning. New portable attributes use an experimental agent.* namespace, initially agent.skill.*, agent.action.*, agent.host.*, agent.adapter.*, agent.runtime.*, and agent.turn.*. Server-authoritative Forge enrichment uses forge.*. This remains a Forge-published schema; an OpenTelemetry standards proposal is outside the current plan.

Signal model

• Resources identify the emitting host process and adapter. service.name is the coding-agent or sf process, not a skill name.
• Traces represent agent turns and downstream work. Reuse a host-provided invoke_agent span when present. Otherwise create an internal turn span without inventing a provider operation.
• Activation spans are internal spans named activate_skill <skill-name>, beginning at selection and ending after full instructions have loaded or failed. They measure selection-to-load latency, not the skill's entire downstream influence.
• Tool/action spans retain their native OTel or GenAI/MCP conventions and add linked activation IDs. If several skills are active, preserve the whole set rather than assigning exclusive credit to one.
• Events are OTel LogRecords with a stable non-empty EventName, flat typed attributes, trace context when available, and an empty or short non-sensitive body. Events carry high-cardinality identities needed for diagnosis and reporting.
• Metrics are derived from events and spans. Emit only bounded-cardinality counters and histograms from clients; do not put skill IDs, release hashes, repository IDs, session IDs, or activation IDs on metric series.

Initial derived client/collector metrics:

Metric	Instrument	Unit
agent.skill.activation.count	Counter	{activation}
agent.skill.activation.duration	Histogram	s
agent.skill.load.failure.count	Counter	{failure}
agent.skill.user_intervention.count	Counter	{intervention}
agent.skill.telemetry.dropped.count	Counter	{event}

Allowed metric dimensions are initially limited to host agent, adapter, source type, trigger type, result, evidence source, and fidelity. Forge produces per-skill leaderboard and reporting rollups from events, not from high-cardinality client metrics.

Required event envelope

Every normalized event includes, when available:

Group	Attributes	Notes
Schema	agent.skill.schema.version, agent.skill.event.id, agent.skill.event.sequence	Event ID is stable across retries; sequence is monotonic within a known turn or activation when the host provides ordering.
OTel context	EventName, timestamps, TraceId, SpanId, severity, resource, instrumentation scope	Preserve source and observed timestamps separately.
Host	service.name, service.version, agent.host.name, agent.host.version, agent.adapter.name, agent.adapter.version	Adapter fields identify normalization code, not the model provider.
Agent run	session.id, gen_ai.conversation.id, agent.turn.id, gen_ai.agent.id, gen_ai.agent.name, gen_ai.request.model, agent.runtime.autonomy.mode, agent.runtime.approval.mode, agent.runtime.sandbox.mode, agent.runtime.network.mode	Never fabricate gen_ai.conversation.id by hashing content. Runtime modes are normalized bounded enums, not raw host configuration.
Skill package	agent.skill.package.id, artifact.version, artifact.hash, artifact.purl	Package ID may be a Forge SKIT or provisional source identity.
Contained skill	agent.skill.id, agent.skill.name, agent.skill.content.hash, agent.skill.source.type, agent.skill.source.ref	Raw local paths and credential-bearing URLs are forbidden.
Activation	agent.skill.activation.id, agent.skill.trigger.type, agent.skill.trigger.source, agent.skill.phase, agent.skill.result	One activation ID follows selection, load, actions, feedback, and incident evidence.
Git AI correlation	git.ai.session.id, git.ai.checkpoint.id, git.ai.attestation.key, git.ai.version, git.ai.schema.version	git.ai.checkpoint.id is the t_ checkpoint component and must never be used as an OTel trace ID.
Evidence	agent.skill.evidence.source, agent.skill.evidence.fidelity, agent.skill.evidence.observer	Fidelity is exact, derived, inferred, or unknown.
Repository	forge.repository.id, vcs.* where allowed	Forge repository identity is attached or verified server-side. Raw remotes stay local by default.
Failure	error.type, agent.skill.error.stage, agent.skill.reason.code	Use bounded documented enums; exception messages and stacks follow separate privacy controls.

Core enums:

• agent.skill.trigger.type: explicit, implicit, host_policy, skill_dependency, restored, unknown
• agent.skill.phase: eligible, selected, load_started, loaded, load_failed, deactivated
• agent.skill.result: ok, error, blocked, cancelled, unknown
• agent.skill.evidence.source: native_otel, native_hook, plugin, transcript, filesystem, user_report, static_analysis
• agent.skill.evidence.fidelity: exact, derived, inferred, unknown
• agent.runtime.autonomy.mode: interactive, autonomous, full_access, unknown
• agent.runtime.approval.mode: always, on_request, never, unknown
• agent.runtime.sandbox.mode: read_only, workspace_write, unrestricted, unknown
• agent.runtime.network.mode: disabled, restricted, enabled, unknown

Example OTel event after normalization:

{
  "event_name": "agent.skill.loaded",
  "timestamp": "2026-08-09T18:42:10.123Z",
  "observed_timestamp": "2026-08-09T18:42:10.141Z",
  "trace_id": "4fd0...",
  "span_id": "18b2...",
  "severity_text": "INFO",
  "resource": {
    "service.name": "claude-code",
    "service.version": "1.2.3",
    "agent.adapter.name": "sf-claude-adapter",
    "agent.adapter.version": "0.1.0"
  },
  "attributes": {
    "agent.skill.schema.version": "1.0.0",
    "agent.skill.event.id": "01K2...",
    "session.id": "sess_...",
    "agent.turn.id": "turn_...",
    "agent.skill.package.id": "owner/code-quality",
    "artifact.version": "2.3.0",
    "artifact.hash": "sha256:...",
    "agent.skill.id": "owner/code-quality:maintainability",
    "agent.skill.content.hash": "sha256:...",
    "agent.skill.activation.id": "act_...",
    "agent.skill.trigger.type": "implicit",
    "agent.skill.trigger.source": "frontmatter_description",
    "agent.skill.phase": "loaded",
    "agent.skill.result": "ok",
    "agent.skill.instructions.file_count": 3,
    "agent.skill.instructions.byte_count": 18420,
    "agent.skill.instructions.truncated": false,
    "agent.skill.evidence.source": "native_otel",
    "agent.skill.evidence.fidelity": "exact"
  }
}

Event catalog

EventName	Purpose	Important event-specific attributes
agent.skill.inventory.changed	A skill became available, changed hash, was enabled/disabled, or disappeared.	agent.skill.inventory.action, install scope, previous/current hash, actor type.
agent.skill.audit.completed	Static or declared capability/trigger review at publish, install, or update.	ruleset ID/version, declared/inferred capabilities, risk flags, trigger breadth, finding counts.
agent.skill.eligible	Host considered a skill a candidate. Optional and sampled because it may be high volume.	candidate count, trigger source, selection rank when available; never prompt text.
agent.skill.selected	Host or user selected one skill.	activation ID, explicit/implicit trigger, selector identity, bounded reason code.
agent.skill.load_started	Full instruction loading began.	expected files/bytes when known.
agent.skill.loaded	Full instructions were successfully injected.	files/bytes/tokens loaded, load depth, truncation, cache status, duration.
agent.skill.load_failed	Instructions could not be fully loaded.	error.type, failure stage, missing/invalid file counts, truncation reason.
agent.skill.conflict.detected	Two instructions, skills, policies, or versions conflict.	other skill activation IDs, conflict class, winner/precedence source, resolution.
agent.skill.policy.decision	A skill-related capability or action was allowed, blocked, or required approval.	capability, decision, policy source/version, approval state, reason code.
agent.action.completed	A downstream mutation or material action occurred while skills were active. Prefer enriching an existing tool span; use this event as fallback.	action type, mutation/risk/scope, result, active activation IDs, exact/inferred attribution.
agent.change.summary	Bounded repository-change summary at turn/checkpoint/end.	file/line counts, outside-initial-scope count, commits, pushes, deployments, reverts; no paths by default.
agent.user.intervention	User corrected, stopped, undid, rolled back, disabled, or uninstalled after an activation.	intervention type, reason code, target activation/skill, elapsed time.
agent.skill.feedback	Explicit structured positive/negative feedback.	sentiment, reason codes, optional opt-in redacted body reference.
agent.skill.incident.reported	A user or operator associates harmful behavior with a skill/version.	incident class/severity, related activations, resolution, reporter type.
agent.skill.quarantine.changed	A release or skill was quarantined, restored, or superseded.	action, reason code, replacement ID, actor, policy version.
git.ai.checkpoint.created	Git AI recorded an agent edit checkpoint.	Git AI session/checkpoint/attestation IDs, real trace/span link, active activation IDs, correlation fidelity.
git.ai.attribution.updated	Attributed line ranges changed after edits, commit, or history rewriting.	generated/accepted/overridden/abandoned counts, commit SHA, rewrite operation, source schema/version.
git.ai.attribution.survived	Previously attributed code remains after a later commit or review window.	checkpoint, original/current commit, surviving lines, elapsed window, fidelity.

Inventory, disable, uninstall, quarantine, and incident events must be retained even when there is no runtime trace. They are essential negative signals.

Capability, authority, and risk fields

At publish/install time, record both author declarations and independently inferred findings:

• agent.skill.capability.declared: string array such as filesystem.write, process.execute, git.commit, git.push, network.request, cloud.deploy, message.send, secret.use
• agent.skill.capability.inferred: same vocabulary, derived by a versioned static-analysis ruleset
• agent.skill.capability.undeclared: inferred minus declared
• agent.skill.trigger.breadth: explicit_only, narrow, broad, unbounded, unknown
• agent.skill.mutation.scope: allowed scopes such as none, working_tree, repository, remote_repository, external_service, production
• agent.skill.authority.effect: none, narrows, expands, unknown
• agent.skill.confirmation.effect: none, requires, suppresses, unknown
• agent.skill.risk.flags: bounded array including broad_implicit_trigger, authority_expansion, approval_suppression, production_mutation, credential_reuse, destructive_action, scope_expansion, architecture_override, private_content_access, external_communication
• agent.skill.audit.ruleset.id and agent.skill.audit.ruleset.version

Static analysis is evidence, not truth. Preserve its ruleset version, findings, and confidence; never silently convert a heuristic into an exact runtime fact.

Capability and risk review has three explicitly separated layers:

1. Author-declared: network domains, environment variables, subprocesses, hooks, MCP servers, install-time actions, writable scopes, secrets, telemetry, and binary payloads.
2. Deterministically observed: archive traversal, unsafe symlinks, executable bits, shebangs, binary types, literal URLs, subprocess and filesystem APIs, network clients, dynamic evaluation, credential-access patterns, and declared-versus-observed mismatches. Language-aware AST rules may report statically_possible when runtime behavior cannot be proven.
3. Opt-in model-assisted: description/body mismatch, deceptive instructions, hidden scope expansion, exfiltration intent, prompt injection, suspicious obfuscation, excessive authority, and risky workflow design.

Publishing blocks on unsafe archives, invalid structure, undeclared executables or hooks, deterministic critical findings, and capability-declaration mismatches. Model findings are advisory and evidence-linked until a deterministic rule or human reviewer confirms them. Every finding is labeled declared, deterministically_observed, statically_possible, model_suspected, or not_analyzed.

For downstream tool/action spans or fallback events, add:

• agent.action.type: filesystem.read, filesystem.write, filesystem.delete, process.execute, git.commit, git.push, network.request, cloud.deploy, database.migrate, message.send, secret.use, or other
• agent.action.mutating: boolean
• agent.action.scope: workspace, repository, remote_repository, external_service, production, or unknown
• agent.action.risk: none, low, medium, high, critical, or unknown
• agent.action.approval.state: not_required, requested, granted, denied, suppressed, or unknown
• agent.action.result: ok, error, blocked, cancelled, or unknown
• agent.skill.activation.ids: every active activation linked to the action
• agent.skill.attribution.fidelity: exact, shared, inferred, or unknown

When multiple skills are active, attribution defaults to shared; Forge must not claim that one skill caused an action merely because it was loaded first. Absence of an approval request does not prove suppressed; record that state only when the host exposes an exact policy decision, otherwise keep the runtime value unknown and preserve any static-analysis finding separately.

User intervention and incident taxonomy

agent.user.intervention.type values:

• correction, scope_reduction, stop, undo, rollback, reject_diff, disable_skill, uninstall_skill, negative_feedback, report_incident

agent.skill.reason.code values initially include:

• trigger: false_positive, missed_trigger, overbroad_trigger
• authority and safety: authority_expansion, approval_suppression, destructive_action, unexpected_external_side_effect, production_risk
• behavior and quality: scope_expansion, architecture_override, instruction_conflict, quality_regression, test_regression, retry_loop, cost_regression
• loading and compatibility: load_failure, invalid_skill, truncated_instructions, incompatible_agent, stale_version
• privacy and supply chain: secret_exposure, private_content_export, untrusted_source, undeclared_capability
• lifecycle: obsolete, duplicate, superseded, unknown

Free-form feedback is never required for diagnosis. It is separately opted in, redacted, bounded, and stored by reference rather than as a metric or indexed attribute.

Change summaries and outcome linkage

To diagnose a skill that causes excessive refactoring or scope drift without collecting source code, capture bounded before/after summaries:

• working-tree dirty state and file counts at turn start
• files added/modified/deleted, lines added/deleted, and top-level area count
• count of changed files outside an explicit or inferred initial scope, with the inference fidelity recorded
• tests, typechecks, builds, and linters attempted/passed/failed
• commits, pushes, migrations, deployments, external mutations, reverts, and rollbacks
• tool count, retries, elapsed time, input/output tokens, and reported cost

Exact paths, diffs, commands, prompts, tool arguments, outputs, and system/skill instruction bodies remain local or content-opt-in. Default remote telemetry includes counts, enums, hashes, IDs, and risk classifications only.

For the metadata-only MVP, initial scope is computed locally from the strongest structured source available: explicit path globs, task-system path metadata, a host-provided structured plan, or files observed before the first mutating action. The baseline freezes at the first write or other side effect. The adapter emits only scope_source, scope_confidence, baseline cardinality, outside_initial_scope_count, and new_path_count; filenames and prompt text remain local. An empty or unavailable baseline produces outside_initial_scope_count = null, never zero. Inferred UI copy must say “outside the pre-change observed file set,” reserving “outside declared scope” for an explicit baseline.

How the deleted-skill incidents become diagnosable

A JFDI-style unsafe-autonomy incident should be reconstructable as:

1. agent.skill.audit.completed reports broad trigger breadth plus authority_expansion, approval_suppression, production mutation, or credential-use flags.
2. agent.skill.selected records whether invocation was explicit or implicitly matched a broad phrase.
3. Downstream actions record mutation scope, risk, approval state, and all active skill activation IDs.
4. A stop, rollback, disable, uninstall, or incident report records a structured reason such as authority_expansion or approval_suppression.
5. The incident view shows the exact release/hash and whether the risky capability was declared, inferred, and observed.

A maintainability-guardrail-style overreach incident should be reconstructable as:

1. Static audit reports broad_implicit_trigger, scope_expansion, or architecture_override risk.
2. Selection records an implicit trigger on general feature work rather than an explicit maintainability request.
3. Change summaries show unusually large file/line/top-level-area counts or changes outside the initial task scope.
4. User corrections, rejected diffs, disables, or uninstalls carry overbroad_trigger, scope_expansion, or architecture_override reason codes.
5. Version and agent cohorts reveal whether the behavior is isolated to one release, host, model, or skill combination.

These reconstructions must work from metadata-only telemetry. Content can improve qualitative review but cannot be a prerequisite for the safety signal.

Forge canonical storage model

Keep raw OTLP immutable in R2. Normalize into append-only facts and derived projections:

• observation_events: one row per normalized event with event/schema identity, timestamps, trace/span IDs, protected actor/repository keys, skill/package/activation keys, evidence source/fidelity, bounded reason/result fields, raw-batch pointer, and normalizer version
• skill_activations: one derived row per activation ID joining selection, load result, duration, trigger, instruction-load summary, and active-skill set
• activation_actions: action/tool facts linked to one or more activation IDs with attribution fidelity, approval, risk, scope, and result
• change_outcomes: bounded change/test/build/commit/deploy/cost summaries by turn or checkpoint
• git_ai_checkpoints: Git AI session/checkpoint/attestation identity joined to real OTel trace/span, agent session/tool call, active skill activations, source version, and fidelity
• git_ai_attribution_outcomes: generated, accepted, committed, overridden, abandoned, reverted, and surviving line counts by checkpoint and commit, with raw Git Note/JSON evidence pointers
• skill_feedback_incidents: interventions, structured feedback, uninstall reasons, incident state, quarantine action, and resolution
• usage_rollups: privacy-thresholded daily/weekly aggregates for product views and the public leaderboard

Every derived row retains normalizer_version plus raw-event IDs so it can be replayed and explained. Corrections append new facts and projection versions; they do not rewrite raw evidence.

Correlation keys

Host attribution uses the strongest honest relationship available:

Host	Activation identity	Downstream action attribution
Claude Code	Exact Skill tool/session/prompt/tool-use identity when exposed	Exact interaction membership; exact active-skill linkage only when host telemetry or an opted-in skill-scoped hook provides it, otherwise shared within the interaction.
OpenCode	Exact Forge-plugin identity from the native skill tool session and call	Plugin-maintained active set for the current response, labeled shared rather than causal.
Codex	Exact skill-injection result when its native metric is available	Join to tool turns only when a common host identifier exists; otherwise transcript- or time-derived attribution remains inferred.
Filesystem/transcript fallback	Read or transcript evidence	Session-window or transcript attribution only; never exact.

Forge assigns agent.skill.activation.id at the earliest exact adapter boundary and propagates it onto emitted Forge spans/events where it controls the context. It does not manufacture an exact tool relationship when a host exposes only turn-level or aggregate skill evidence.

Use correlation keys in this priority order:

1. OTel trace/span relationships and exact host-provided session, conversation, turn, tool-call, and activation IDs
2. Forge-issued repository and pseudonymous device identity
3. SKIT release ID plus contained skill ID and content hash
4. normalized working tree and Git remote mapping
5. transcript session ID
6. commit and branch evidence
7. timestamp-window inference, marked as inferred rather than exact

Never collapse inferred and exact observations into the same unexplained count. Never synthesize a standard OTel identifier from prompt or instruction content.

Cardinality, sampling, and governance

• Events and spans carry high-cardinality skill, release, activation, session, and repository identities; metrics do not.
• Always retain selection, load failure, user intervention, disable/uninstall, incident, and quarantine events. Sample successful eligibility and repetitive read events first.
• Deduplicate on stable agent.skill.event.id; when a source cannot supply one, derive a batch-local fingerprint and record that deduplication is inferred.
• Publish JSON Schema or Weaver definitions, TypeScript types, enum registries, compatibility fixtures, and example OTLP payloads from one source of truth.
• Schema changes are additive within a major version. Deprecated fields remain readable through the raw-retention window. Enum additions are forward-compatible; meaning changes require a major version.
• Pin and expose adapter version, normalizer version, OTel schema URL, static-audit ruleset version, and privacy-policy version on every relevant record.
• Maintain golden fixtures for normal activation, load failure, multiple active skills, JFDI-style unsafe autonomy, maintainability-style over-triggering, user correction, uninstall, offline retry, duplicate delivery, and redaction.

Initial operator and product views must answer:

• Which skills are selected implicitly, and which are followed by immediate correction, stop, disable, or uninstall?
• Which skills or versions are associated with unusually broad or risky mutation scopes?
• Did an action require, receive, bypass, or suppress approval?
• Was the skill fully loaded, partially loaded, truncated, stale, or in conflict with another skill?
• Which declared capabilities differ from static findings or observed actions?
• Which incidents are isolated to one release, agent, model, repository cohort, or skill combination?
• Can every public usage count and effectiveness claim be traced back to evidence with a disclosed fidelity and privacy policy?

9. Usage, outcome, and effectiveness engine

Start with simple usage statistics after activation identity and correlation quality are visible:

• activations by SKIT, contained skill, release, agent, repository, and time
• unique users, sessions, and repositories; keep installation counts as a separate distribution metric
• explicit versus implicit trigger mix
• attempted, loaded, failed, and unknown activation phases
• enabled versus disabled contained skills
• collector/adapter coverage and fidelity gaps

Treat these as adoption and operational statistics, not effectiveness claims. Add outcome evidence later:

Public usage leaderboard

The public leaderboard ranks observed runtime use, never downloads or installs. It should expose separate, understandable measures rather than one opaque popularity score:

• weekly and monthly active users, repositories, and sessions
• exact activations and repeat-use/retention cohorts
• fastest-growing observed use
• usage by agent, SKIT release, and contained skill
• task-associated activations with observed success, shown as a separately qualified outcome count once Stage 4 outcome evidence exists—not folded into the popularity ordering
• evidence-source and fidelity coverage for every published count

Deduplicate retries, repeated reads, and duplicate adapter events within a stable turn or tool-call boundary. Rank unique users, repositories, and sessions separately from event volume; do not let raw activation volume dominate through an undisclosed composite weighting. Identify and exclude tests, CI, synthetic traffic, and suspected manipulation from default rankings while making methodology visible. Public SKITs may participate only from users who enabled public aggregate contribution. Consented activity from private repositories may contribute only to thresholded aggregate counts; private SKITs, repository identities, sessions, and organization identities must never appear or be inferable from the leaderboard.

Usage and effectiveness are distinct. High usage must not imply high quality, and effectiveness measures must not silently change leaderboard rank.

Outcome evidence can include:

• tool failures and retries
• test, lint, typecheck, and build results
• commits or pull requests produced
• Git AI checkpoints and attributed line ranges
• generated lines, accepted lines, committed lines, and human overrides
• abandoned checkpoint/session output and reverted attributed code
• attributed-code survival after review and later 7-, 30-, and 90-day repository changes
• user corrections, undo, or explicit negative feedback
• reverted or abandoned changes
• wall time, tokens, cost, and tool count
• task completion or evaluator result
• explicit user rating

Later derived effectiveness views:

• trigger precision and missed-trigger evaluation sets
• success/failure and correction rates
• generated-to-accepted, accepted-to-committed, and committed-to-surviving funnels by exact skill hash and SKIT release
• review rework, human-override, revert, and code-durability rates for attributed checkpoints
• token/time/tool deltas versus a baseline
• version-over-version comparison
• agent compatibility comparison
• cohort and controlled holdout experiments

Do not ship a single opaque "effectiveness score" initially. Show the underlying measures and evidence quality. Causal claims require controlled comparisons, not merely correlation.

Git AI provides unusually strong outcome evidence but still does not prove causality. A checkpoint linked to an active skill establishes that the skill was active when the attributed operation occurred; exclusive credit requires an exact host relationship or controlled comparison. Shared activation sets remain shared throughout the attribution and durability funnel.

10. Web product surfaces

Global SKIT library

• search and browse public SKITs
• public usage leaderboard based on deduplicated, consented runtime observations rather than installs
• owner/org collections and tags without implying atomic collection installation
• compatibility and install command
• release and provenance indicators
• stars, forks, comments, and deprecation state

SKIT detail

• Overview
• Files and rendered SKILL.md
• Releases and diffs
• Installation and compatibility
• Usage observations
• Outcomes and comparisons
• Issues/feedback

For multi-skill SKITs, every usage and outcome view must support both bundle-level aggregation and contained-skill drill-down. Never present a bundle activation count that implies every contained skill ran.

Repository Skills tab

• observed and installed SKITs
• exact versions/hashes used
• activations by agent and session
• linked transcripts and commits
• Git AI checkpoint and attributed-line evidence, including generated, accepted, overridden, abandoned, reverted, and surviving outcomes
• a visible distinction between checkpoint-exact, session-shared, and inferred skill attribution
• gaps where identity or correlation is uncertain
• collector and adapter health

Setup and privacy

• what is collected by default
• prompt/tool-content opt-in state
• device credentials and revocation
• current raw-event retention tier, normalized-data retention, upgrade options, and deletion controls
• last collector heartbeat and dropped-event counts

11. Privacy, safety, and supply-chain controls

• Default to metadata-only collection; prompt and tool bodies remain off. Onboarding should explain what optional content collection unlocks before requesting a separate explicit opt-in.
• Reuse transcript secret masking where content ingestion is explicitly enabled, but redact at the earliest possible boundary.
• Ingest only Git AI identifiers, attribution ranges/counts, versions, and outcome statistics by default. Do not ingest Git AI prompt records, message URLs, raw prompts, human-author email, or hosted-session bodies without the same separate content opt-in and redaction policy.
• Restrict GIT_AI_CUSTOM_ATTRIBUTES and the Forge checkpoint extension to correlation-safe IDs, hashes, versions, and fidelity. Never place secrets, prompts, paths, or personal identity in Git Notes or correlation attributes.
• Separate public SKIT content from private usage observations.
• Treat contribution to public aggregate usage rankings as a separate opt-in from metadata collection. Apply minimum cohort thresholds and suppress dimensions that could identify a user, organization, private repository, or session.
• Apply repository visibility authorization to all correlated observations.
• Store source, release hash, installer version, and install paths for provenance.
• Validate archives against path traversal, absolute paths, device files, and unsafe symlinks.
• Clearly identify executable scripts and declared external dependencies before installation.
• Support quarantine and emergency disable without deleting historical evidence.
• Add signed releases/attestations only after the release model is stable.

12. Operations and dogfooding

• instrument the SKIT ingest and normalization pipeline itself
• track rejected, quarantined, duplicated, delayed, and dropped events
• track schema validation failures, unknown enum values, orphaned spans/events, incomplete activation lifecycles, and exact-versus-inferred correlation rates by adapter version
• track exporter-specific delivery, retry, redaction, queue depth, and drop state without logging destination credentials
• track pinned Git AI compatibility, hook coexistence, unknown JSON fields, note-schema drift, sidecar gaps, orphaned checkpoints, and checkpoint-to-activation join fidelity
• keep raw-batch retention tier-aware and independently configurable from normalized data: 90 days free, at least one year on qualifying paid tiers
• provide replay from R2 into a new normalizer version
• make normalization idempotent
• version the canonical observation schema
• run golden OTLP fixtures through every collector, adapter, normalizer, projection, and dashboard release
• test collector upgrade, rollback, offline buffering, and credential rotation

Delivery stages

Stage 0 — Decisions and thin prototypes

Deliverables:

• agree on the SKIT bundle and contained-skill identity contract
• vendor the selected balgaly/skit baseline with license, upstream commit, security review, and characterization tests
• define Agent Skills and Vercel CLI interoperability fixtures without introducing a hosted-service or GitHub dependency
• publish the first machine-readable agent.skill.* semantic convention, event catalog, enum registry, JSON Schema/Weaver definitions, TypeScript types, and example OTLP payloads from one source of truth
• pin the exact core OTel and dedicated GenAI semantic-convention releases and schema URLs used by the prototype
• versioned package-management, runtime-observation, and outcome event envelopes with privacy defaults
• prototype declared-versus-inferred capability, trigger-breadth, authority-effect, and approval-effect static audit
• prototype Claude native event ingest
• document OpenCode and Codex adapter contracts and current fidelity gaps without placing their implementation on the first-slice critical path
• pin and characterize Git AI commit c57c6c24be03ba89ce7d47783841d2f49ba8ee45, Standard v3, its hook adapters, refs/notes/ai, and JSON command outputs without contacting or depending on maintainers
• publish Forge's forge/git-ai-correlation/0.1 schema and golden fixtures proving that git.ai.checkpoint.id remains distinct from a real OTel trace ID
• prototype one global collector installation on macOS only
• prove collector fan-out to Forge, one generic OTLP endpoint, and a local debug destination
• add golden fixtures for normal activation, load failure, multiple active skills, JFDI-style unsafe autonomy, maintainability-style over-triggering, intervention/uninstall, duplicate delivery, and redaction
• decide initial high-volume storage after measuring real event size and rate

Exit criterion: one Claude Code session produces inspectable raw events with known fidelity and no prompt/tool content, the same normalized canary signal is visible in Forge, a generic OTLP consumer, and a local debug destination, the two deleted-skill failure patterns can be reconstructed from metadata-only golden fixtures, and one Git AI checkpoint can be joined to a real OTel trace without treating its t_ identifier as that trace ID.

Stage 1 — SKIT registry MVP

Deliverables:

• begin the stage with a verified private rollback snapshot and the one-time backend migration of every Gist into the SKIT domain, then remove the Gist product, routes, APIs, commands, and storage in the same cutover; destroy the snapshot automatically after 30 days
• shared multi-skill SKIT contracts and validation
• root README-frontmatter declarations for contained skills, shared mappings, trigger modes, mutation scopes, capabilities, approval expectations, and expected outputs plus versioned static-audit results
• D1/R2 schema and API for drafts and immutable releases
• Forge-editable draft UI and APIs
• revisioned three-way synchronization using a recorded common snapshot, per-file hashes, conflict blocking, and no automatic force-push
• sf add skit plus its exact sf skit install alias, alongside sf skit init, validate, publish, and pull
• Forge-managed local installation state containing exact child hashes and durable per-agent targets, with no generated SKIT lockfile
• installation from Forge, standard Git clone URLs, direct URL/archive, and local-path sources without requiring Vercel or GitHub infrastructure
• immutable Forge release downloads consumable by generic Agent Skills clients
• contained-skill enable/disable support
• one production install target for Claude Code; keep OpenCode and Codex path contracts as fixtures until Stage 3
• basic public/owner list and SKIT detail with files/releases
• public/private-ready authorization schema, even if organization management UI follows later

Exit criterion: an author can edit and publish a multi-skill SKIT, another user can atomically install an exact release, disable one contained skill, and Forge can verify the bundle and per-skill content hashes; a generic Agent Skills client can install the same immutable release URL.

Stage 2 — Observability vertical slice

Deliverables:

• one-command sf observe install for the collector, Claude adapter, pinned Git AI build, composed hooks, and configured destinations, plus status/doctor/uninstall
• pinned local collector and managed service on macOS
• configurable Forge, generic OTLP, and local debug/file destinations with per-destination canaries
• global user-device ingest credentials and repository link/resolution
• OTLP/HTTP gateway, raw R2 storage, queue, and idempotent normalizer
• Claude Code production adapter only
• pinned Git AI installation/doctor support, hook-coexistence checks, session-level GIT_AI_CUSTOM_ATTRIBUTES, validated git ai stats --json ingestion, and a local checkpoint-correlation sidecar
• exact activation lifecycle, instruction-load summaries, structured intervention/disable/uninstall events, and downstream action linkage where hosts expose it
• diagnostic repository UI showing per-skill activation timelines, trigger mode, load completeness, active-skill conflicts, policy decisions, interventions, simple usage statistics, and collector health

Exit criterion: a fresh dogfood user can run one setup command, install the global collector, Claude adapter, and pinned Git AI build, opt into metadata-only telemetry, activate one contained skill, and see the correct SKIT release, skill identity/hash, trigger, load result, agent, repository, session, and activation evidence in Forge while independently exporting the same standards-compatible signal to another OTLP destination; the same timeline includes one attributed checkpoint and its stats with explicitly session-shared or checkpoint-exact fidelity; a subsequent correction or uninstall remains linked to that activation. Stage 2 remains dogfood-only until this path is reliable.

Stage 3 — OpenCode, Codex, transcript convergence, and public usage

Deliverables:

• OpenCode production plugin adapter with exact native skill tool observation
• Codex OTel adapter with explicit fidelity labels and gap detection
• production package-install projections for OpenCode and Codex
• unify current transcript hooks under the agent-evidence installer
• link activation events to existing transcripts and commits
• emit Forge-owned checkpoint and attribution OTLP events by composing agent hooks, watching refs/notes/ai, validating pinned Git AI JSON, and joining the checkpoint sidecar
• normalize Git AI session, checkpoint, attestation, commit, generated, accepted, overridden, abandoned, and rewrite-survival identities without requiring Git AI Cloud
• normalize tool/action risk, mutation scope, approval state, multi-skill attribution, and bounded repository change summaries
• skill incident timeline and operator queries for broad implicit triggers, risky mutations, immediate interventions, load failures, conflicts, and release regressions
• public usage leaderboard with deduplication, consent, cohort suppression, fidelity labels, and synthetic-traffic filtering
• resumable transcript backfill and event reprocessing
• Linux collector service support
• adapter contract tests against pinned agent versions

Exit criterion: repository timelines can move from activation to agent/tool operation to Git AI checkpoint to attributed line ranges and commit without relying on timestamp inference when the host exposes stable IDs, and every public leaderboard count can be traced to deduplicated, consented runtime observations with a disclosed fidelity class.

Stage 4 — Outcome views

Deliverables:

• normalized tool/test/build/commit/correction outcomes
• Git AI generated, accepted, committed, human-overridden, abandoned, reverted, and surviving-line outcomes by checkpoint, activation set, skill hash, and release
• review-rework and 7-/30-/90-day attributed-code durability views
• activation funnels and failure/correction rates
• token, time, cost, and tool-count attribution where hosts expose it
• version comparison views
• explicit user feedback capture
• retention and privacy UI

Exit criterion: users can inspect evidence for why one SKIT version appears better or worse, including its generated-to-surviving code funnel, without an opaque composite score or a false claim that line attribution alone proves causality.

Stage 5 — Managed skill library

Deliverables:

• organization management, collections, private sharing, forks, comments, and tags on the private-ready Stage 1 schema
• source-repository sync and multi-SKIT repository publishing
• update policies, installed-hash verification, drift detection, and fleet inventory
• Windows collector support
• deprecation, quarantine, and replacement workflows

Exit criterion: a team can curate approved SKITs, manage exact versions across agents, and audit adoption and outcomes.

Stage 6 — Evaluation and experimentation

Deliverables:

• should-trigger/should-not-trigger evaluation sets
• version and no-skill holdouts
• offline replay where permitted
• rubric/evaluator jobs with evidence links
• causal experiment reporting and confidence intervals
• recommendations based on transparent measures

Exit criterion: Forge can support defensible statements about effectiveness rather than only usage correlation.

Recommended first vertical slice

Use Claude Code on macOS for the first end-to-end slice:

• Claude Code provides a native skill_activated event and skill-attributed usage metrics.
• Claude Code's trace and hook model provides the strongest single-agent path for joining activation, tool operation, and the bundled Git AI checkpoint.
• macOS matches the current primary development environment.
• OpenCode follows after the Claude dogfood path is reliable; Codex follows with its per-invocation and plugin-skill gaps represented honestly.

The slice should include exactly one SKIT containing at least two independently addressable skills, one repository, one release, one global collector, and one activation timeline showing that activating one child did not activate the other. With the pinned Git AI baseline installed, one activation should continue through an agent/tool operation, checkpoint, attributed line range, and commit while preserving explicit attribution fidelity. Avoid building discovery, social features, or aggregate effectiveness scoring before this works.

Recorded product decisions

1. A SKIT is an atomic bundle containing one or more independently addressable skills; a single-skill SKIT is a bundle of one.
2. SKIT drafts are editable in Forge. Local and Git synchronization uses a recorded common revision, per-file hashes, and explicit three-way merging. Non-overlapping edits may merge automatically; conflicts block publishing and are never resolved by last-write-wins, dirty-file overwrite, or an automatic force-push.
3. sf observe install installs one global per-user collector. Repository association is dynamic and server-authorized.
4. Metadata-only telemetry is the mandatory default. Onboarding explains optional content collection, its risks, and the benefits it unlocks before offering a separate opt-in.
5. The first distribution pins upstream otelcol-contrib for implementation speed and simplicity; a custom distribution is contingent on measured need.
6. The vertical slice is macOS-first, followed by Linux and then Windows.
7. Gists are removed in a hard backend cutover. Every existing Gist is migrated into an editable SKIT draft, migration integrity is verified, and the Gist product, routes, APIs, commands, and legacy storage are removed without a conversion button, compatibility surface, or preservation period. A private rollback snapshot is retained for 30 days and is never exposed as a product surface.
8. The initial schema and authorization contracts are private-organization-ready. Full organization management workflows may arrive after the public/owner MVP.
9. The first product metrics are simple usage and coverage statistics, explicitly not an effectiveness score.
10. Free accounts retain raw OTLP for 90 days. Qualifying paid tiers retain it for at least one year, with longer policies available by tier. Normalized records have a separate retention policy.
11. Forge vendors the dormant balgaly/skit implementation as attributed seed code and owns its future evolution behind sf.
12. sf add skit <source> is canonical, and sf skit install <source> is an exact alias with identical behavior and installed state.
13. Forge interoperates with Vercel's Agent Skills ecosystem but has no runtime dependency on Vercel's CLI, APIs, telemetry, directory, GitHub, or another hosted code forge. Forge IDs and immutable Forge release URLs are the canonical hosted source contract; standard Git clone URLs remain a generic transport without host-specific shorthand.
14. The public leaderboard is based on deduplicated, consented runtime usage—not installs—and remains separate from effectiveness views.
15. sf observe is a vendor-neutral OTel emitter and collector manager. Forge is one optional destination alongside arbitrary OTLP and local destinations.
16. The normalized wire contract begins solely as a public, versioned, Forge-owned experimental agent.* schema centered on agent.skill.* and agent.action.*. Forge will not pursue an OpenTelemetry standards proposal until production use demonstrates that the contract is stable and worth standardizing. Forge enrichment remains isolated under forge.*.
17. Events and spans carry diagnostic identities; client metrics stay bounded-cardinality. Per-skill reporting and leaderboards are derived from events rather than high-cardinality metric labels.
18. Selection, load failure, intervention, disable/uninstall, incident, and quarantine are durable first-class facts. Metadata-only telemetry must be sufficient to diagnose unsafe autonomy and overbroad-trigger incidents.
19. Skill attribution is never exclusive when several skills are active unless the host provides exact causal linkage; otherwise it is explicitly shared or inferred.
20. A SKIT is described by YAML frontmatter in its root README.md. Forge does not use skit.yaml and does not generate a SKIT lockfile. Exact child identity is tracked with Forge-computed content hashes whenever a child skill has no declared version.
21. The global observer uses adapter session registration plus independently resolved repository/worktree evidence. Client repository claims aid attribution but never authorize access.
22. Per-child enablement uses native Codex, Claude Code, and OpenCode controls where verified; other agents receive a projection containing only enabled children while the canonical local library retains the complete release.
23. Advanced users may extend the generated Collector with arbitrary components. Forge guarantees its curated, post-redaction extension path; full pipeline replacement is user-owned and forfeits managed redaction guarantees.
24. Capability review separates author declarations, deterministic findings, statically possible behavior, and opt-in model findings. Only deterministic critical failures and confirmed declaration mismatches block publishing initially.
25. Metadata-only scope expansion uses a locally computed structured or pre-mutation baseline and emits counts plus fidelity, never prompt text or filenames.
26. Activation IDs propagate exactly only across host relationships that expose a common identifier. Turn-shared, transcript-derived, and time-window attribution remain labeled as such.
27. Git AI is a pinned complementary dependency distributed and installed through sf, not vendored engine source. The initial compatibility baseline is git-ai-project/git-ai commit c57c6c24be03ba89ce7d47783841d2f49ba8ee45, Git AI Standard authorship/3.0.0, under Apache-2.0. Forge upgrades it deliberately and does not depend on Git AI Cloud or maintainer coordination.
28. Forge adopts Git AI's checkpoint, refs/notes/ai, agent-adapter, rewrite-preservation, and JSON-statistics behavior. Forge does not recreate its line-attribution or Git-history engine unless external integration becomes impossible.
29. Git AI's t_<14hex> value is normalized as git.ai.checkpoint.id, never as an OTel trace ID. Forge publishes its own checkpoint-correlation schema linking that ID to real OTel trace/span, agent session/tool call, and one or more skill activation IDs.
30. Forge produces its own stable local checkpoint event stream and JSON schemas by composing hooks, observing refs/notes/ai, validating pinned Git AI command output, and maintaining an append-only checkpoint sidecar. No upstream API, exporter, hosted endpoint, or maintainer change is required.
31. sf and Git AI hooks must coexist. Forge preserves Git AI handlers, modifies only Forge-managed blocks, detects drift, disables automatic Git AI updates for Forge-managed installations, and requires explicit compatibility upgrades.
32. Session-level GIT_AI_CUSTOM_ATTRIBUTES and git ai stats --json are acceptable MVP inputs but remain labeled session_shared; they cannot support activation-level effectiveness claims without checkpoint-sidecar correlation.
33. The first end-to-end slice supports Claude Code on macOS only. OpenCode follows after dogfood reliability; Codex follows with its fidelity gaps preserved explicitly.
34. sf observe install is the default one-command setup for the collector, Claude adapter, pinned Git AI build, composed hooks, and destinations. Git AI is installed by default with an explicit --without-git-ai opt-out.
35. Stage 2 is dogfood-only. Public rollout follows reliable dogfood operation, but the privacy-thresholded public usage leaderboard remains in the Stage 3 product scope.

Remaining design questions

1. What public-leaderboard cohort thresholds, deduplication window, and abuse controls provide useful rankings while including consented, anonymized private-repository adoption? Keep the thresholds configurable until real traffic exists; the storage model must preserve private adoption as eligible aggregate input without exposing private repository identity.

External schema references

• Git AI repository

• Pinned Git AI baseline

• Git AI Standard v3 at the pinned baseline

• Git AI Apache-2.0 license

• Git AI data-privacy contract at the pinned baseline

• OpenTelemetry Logs Data Model

• OpenTelemetry Semantic Conventions

• OpenTelemetry attribute registry

• OpenTelemetry session attributes

• OpenTelemetry artifact attributes

• OpenTelemetry VCS attributes

• OpenTelemetry metric semantic conventions

• OpenTelemetry GenAI Semantic Conventions

• OpenTelemetry GenAI agent spans

Explicitly deferred from the first slice

• syscall-level file-read monitoring
• a universal effectiveness score
• public package signing infrastructure
• automatic SKIT execution permissions
• model-generated skill rewrites based on telemetry
• a custom OTel collector distribution unless the upstream prototype fails

---

This public proposal was published from Forge's dated engineering plan on August 9, 2026. External contracts are linked above at exact revisions where the plan depends on them.