SKIT examples and isolated test cases#

SKIT examples use one deterministic directory per contained skill and case:

examples/<skill-name>/<case-id>/
├── case.yaml
├── prompt.md
├── fixtures/
│   └── input.json
└── screenshots/
    ├── 01-input-description.png
    └── 02-output-description.png

Both <skill-name> and <case-id> use lowercase kebab-case. Screenshot names begin with a two-digit display order followed by their role and a concise description. Keep screenshots inside their case so the prompt, evidence, alt text, and evaluation contract cannot silently drift apart.

Examples live at the SKIT root rather than inside skills/<name>. They are part of the atomic SKIT release, but they do not change the contained skill's runtime content hash. This keeps activation and effectiveness telemetry stable when an author only improves a screenshot or test fixture.

case.yaml#

Every case uses schema_version: forge.skill-example/v1 and declares:

• the contained skill and stable case id;
• a prompt file and whether it is exact, reconstructed, or synthetic;
• a recommended isolation level, network policy, and optional fixture files;
• optional rubric checks;
• ordered screenshots with input, process, output, or comparison roles, alt text, and optional captions;
• whether the case is reviewed and public-safe or private.

Forge discovers manifests only at examples/<skill-name>/<case-id>/case.yaml. The manifest identity must match the directory identity, all referenced files must exist, and screenshot files must use an image media type.

The manifest's execution.isolation is the author-recommended starting point:

• skill_only loads only the addressed skill (the default and most diagnostic mode);
• full_skit loads every skill in the same SKIT release or draft revision;
• active_context loads the addressed skill plus the signed-in tester's currently active skills.

Forge's run UI may let the tester change this setting for one run. The selected level is recorded with the run, so comparisons remain reconstructable.

Isolated execution#

An isolated Forge run receives the selected skill context plus the ordinary agent baseline. skill_only is the diagnostic default; broader context levels are explicit, visible run settings.

The initial Skill Lab profile uses one fixed text model with no tools, shell, workspace, network, or production access. It may read declared text fixtures. The runner records:

• exact draft revision, matching release when one exists, and contained skill content hash;
• case and prompt content hashes;
• the exact context level, constituent skill IDs, revisions, and content hashes;
• model identity, token usage, status, and content-addressed output;
• each rubric result and evaluator provenance (rubrics begin as not_evaluated).

Later execution profiles may add a fresh ephemeral workspace, richer sandbox policies, tool transcripts, produced files, or generated screenshots. Those capabilities must be explicit versioned profiles rather than silent changes to the initial text-only runner.

Run results are immutable Forge evidence linked to the release. Do not commit generated runs/ directories into a SKIT. A screenshot is evidence of one run, not proof that every agent or future release behaves the same way.

Example#

schema_version: forge.skill-example/v1
id: activation-review
skill: ceo-creator
title: Activation review
prompt:
  path: prompt.md
  provenance: reconstructed
execution:
  isolation: skill_only
  network: disabled
  fixtures: [fixtures/activation.json]
checks:
  - id: preserve-privacy
    kind: rubric
    criterion: Keep private activity aggregate-only.
screenshots:
  - path: screenshots/01-input-cohort.png
    role: input
    alt: Activation cohort dashboard supplied to the agent.
  - path: screenshots/02-output-analysis.png
    role: output
    alt: Decision-oriented cohort analysis produced by the agent.
privacy:
  classification: public_safe
  reviewed: true