---
name: ir:onboarding-factory
description: >
  Maintain the canonical scenario × agent fixture matrix for irrlicht. A slim
  dispatcher that routes intent to four focused subagents — `create-scenario`
  (add a matrix row), `create-agent` (add a matrix column), `assess` (judge one
  cell across the three pillars and write its spec), and `record` (drive the
  live agent and verify every websocket observation). Every read and every
  write goes through the `of` factory CLI (tools/onboarding-factory) — the skill
  itself never touches `replaydata/`. Each subagent returns a short summary
  (≤6 lines, ≤7 for `record`) so the parent keeps its context for strategic
  decisions instead of drowning in per-cell tool output. Use when the user
  says "/ir:onboarding-factory",
  "onboard agent", "add a scenario", "assess fixtures", "record fixtures", or
  "regenerate recordings". For unattended / overnight runs ("push through",
  "onboard everything", "implement everything that can be"), see the Overnight
  push-through mode section — implement every feasible unlock and deliver a
  report-as-PR, autonomously.
---

# Irrlicht onboarding factory — dispatcher

This skill maintains the **scenario × agent fixture matrix**: for each
behavior (a scenario row) and each agent CLI (a column), it judges whether the
agent supports it, whether the daemon can observe it, and whether the recording
harness can drive it — then captures a live recording and verifies it.

**One iron rule: the factory owns `replaydata/`.** The skill is pure
orchestration. It NEVER reads or writes catalog files, cell metadata, specs, or
recordings by hand — every operation is an [`of`](#the-of-cli) call, and `of`
validates schema + referential integrity before it touches disk. If you find
yourself about to `jq`, `Edit`, or `Write` something under `replaydata/`, stop:
there is an `of` verb for it.

**Why a dispatcher of subagents.** A full onboarding sweep runs hundreds of
shell invocations and dumps every `events.jsonl` / `transcript.jsonl` into the
transcript — enough to exhaust the parent's context long before the work is
done. So each cell's work is ONE `Agent` call that burns its OWN context and
returns a short summary. The parent decides *which* cells to touch; the
subagents do the work and report back. See
[`return-contract.md`](return-contract.md) for the shared envelope (≤6 lines, ≤7 for `record`).

## Overnight push-through mode (the maintainer's default for unattended runs)

Triggered by "push through", "everything", "onboard fully", "overnight", or any
unattended / scheduled run. The goal is **a valid report and a reviewable PR by
morning, with every cell that *can* be observed actually observed** — not just
the easy ones. Run **fully autonomously**: never call `AskUserQuestion` /
`ExitPlanMode` / block on a prompt mid-run — make the sensible call, document it,
and keep going. Emit a one-line status at each stage boundary; the final PR
body *is* the morning report.

This is more than the standard sweep. Work three passes until the matrix stops
moving:

1. **Sweep** — assess every cell, then record every `pending-record` cell (the
   normal dispatcher flow described below).

2. **Unlock — implement what's missing.** Do **not** accept a frozen verdict at
   face value. For each non-`observed` cell (`blocked-daemon` / `blocked-driver`
   / `unobservable` / `unknown`), ask: *is the blocker a daemon / parser / driver
   feature that is feasible to build?* If yes, **build it** — this is the one
   kind of work the parent does directly (Go code under `core/`, never
   `replaydata/`, so the iron rule still holds): an adapter parser fix, a new
   daemon capability, a `Source` / discovery extension, a driver seam. And
   **empirically probe every `unobservable` verdict** — drive the agent into that
   edge state and read what actually lands on disk; assessments are sample-based
   and routinely under-call what's observable. Every feature gets **unit tests
   AND a live end-to-end check**. Then re-assess + re-record the now-unblocked
   cells (back through the subagents). Repeat pass 2 until nothing is feasibly
   unlockable. (Precedent: the antigravity run shipped PID-binding, subagent
   linking, and three parser fixes this way — moving ~7 cells from frozen to
   observed — while probing *confirmed* that permission/interrupt states are
   genuinely unobservable.)

3. **Freeze honestly — but never fake.** A cell stays frozen ONLY when the signal
   genuinely doesn't exist: the agent lacks the feature, the data isn't persisted
   anywhere readable (live-API-only / TUI-only), or it needs an upstream change.
   **Never ship a fragile guess to force a green cell** — e.g. decoding unlabeled
   protobuf with no ground truth to validate against. The test: *would a reviewer
   accept this as correct, or is it a plausible guess?* If a guess, freeze it,
   document the exact blocker, and file a follow-up issue.

Then **deliver**: get the full suite green (`go test ./core/... -race`, `go test
./tools/onboarding-factory/... -race`, `tools/replay-fixtures.sh`, `of
validate`), commit in clean logical commits, push, and open / update **one PR**.
The PR body is the report: cells observed (count + delta), cells frozen (grouped
by honest reason), features implemented (with their tests), follow-ups filed.
Update the relevant memory.

**Scope guard.** "Implement everything that can be implemented" means everything
**feasible and verifiable** — proven by tests + a live run. It does NOT mean
force-greening the impossible. A cell that is frozen for a real reason, with a
filed follow-up, is a valid outcome and belongs in the report — that is success,
not a gap.

## The four verbs

| The user wants to… | Verb | Subagent |
|---|---|---|
| track a NEW behavior the matrix lacks | `create-scenario <slug>` | [`create-scenario`](create-scenario/SKILL.md) |
| onboard a brand-NEW agent CLI (a column) | `create-agent <slug>` | [`create-agent`](create-agent/SKILL.md) |
| judge whether `<agent>` does `<scenario>` + write its spec | `assess <agent> <scenario>` | [`assess`](assess/SKILL.md) |
| capture + verify a live recording for a cell | `record <agent> <scenario>` | [`record`](record/SKILL.md) |
| see matrix status (no cost) | `/ir:onboarding-factory` (no args) | nothing — `of status` inline |
| run an orchestrator scenario (gastown) | `<orch> [<scenario>]` | inline (hermetic — see below) |

There is no separate `extend-driver` verb: a driver gap is surfaced by `assess`
(the **driver** pillar = `gap:<primitive>`) and closed inside `record`, which
ports the missing step from the reference driver before it drives.

### Not every cell is worth a sweep (#1369)

A sweep over all 46 scenarios is no longer the price of admission. Two things
changed, and both are visible in `of status --summary`:

- **12 of the 46 scenarios are core**; the other 34 are optional and block no
  maturity promotion. Prioritise the core set when onboarding a new agent —
  `of status --summary` prints each adapter's `core` column (settled / 12) and
  its claimed vs earned maturity. The set lives in
  `tools/onboarding-factory/internal/matrix/vocabulary.go`.
- **A structurally dead cell needs no directory.** Instead of writing an
  assessment for a scenario the agent simply cannot do, declare the missing
  capability — through the CLI, like every other write:

  ```
  of agent update --id <agent> --capability <trait>=absent     # agent lacks the feature      → n/a
  of agent update --id <agent> --capability <trait>=untraced   # has it, leaves no trace      → unobservable
  of agent update --id <agent> --capability <trait>=traced     # removes the declaration again
  ```

  The matrix synthesizes the cell from that one line. Trait ids are a closed
  set in `internal/matrix/capability.go`; the verb refuses an invented trait or
  state, and `of validate` additionally rejects a declaration that contradicts
  a cell already on disk. `of agent add` writes a `planned` entry for a new
  column automatically, so a freshly registered agent validates clean.

Do **not** use a capability declaration for a cell that is merely *not
recorded yet* — that is `record_blocked` on the assessment, and the two mean
different things: one says the adapter cannot, the other says we have not.

## Dispatching a subagent

For `create-scenario`, `create-agent`, `assess`, and `record`, spawn ONE
`general-purpose` Agent and let it run the corresponding self-contained
`SKILL.md`. Brief it minimally — the SKILL.md carries the full contract — but
ALWAYS bind its working directory explicitly. A spawned agent does **not**
inherit the dispatcher's cwd: it starts in the main checkout even when you are
in a worktree. Pass the absolute repo root and make the subagent `cd` there and
assert the branch BEFORE any `git` or `of` write. Skipping this once landed 43
cells on `main` instead of the worktree. Compute the root once
(`git rev-parse --show-toplevel`) and substitute it for `<repo-root>`:

```
Agent(
  subagent_type: "general-purpose",
  description: "<verb> <agent>/<scenario>",
  prompt: "Read and execute .claude/skills/ir:onboarding-factory/<verb>/SKILL.md.
           Repo root: <repo-root>. FIRST cd there, then assert
           `git rev-parse --show-toplevel` == <repo-root> and
           `git branch --show-current` == <branch>; abort if either differs.
           Run every of/git command from that root.
           Inputs: agent=<agent> scenario=<scenario>.
           Follow it exactly and return ONLY the summary it specifies."
)
```

Collect each subagent's ≤6-line summary into a table for the user. Do **not**
run the per-cell mechanics yourself — that is what blows the context budget.

## Deriving the work-list — from `of status`, never from prose

When the user asks for a sweep ("assess every scenario for codex", "record all
pending claudecode cells"), compute the cell list **from the factory, never
from a prior subagent's summary.** The per-cell summaries are deliberately lossy
(≤6 lines, to protect parent context); enumerating the next stage's work from
that prose is how a cell silently drops — a real regression once left a scenario
sitting un-assessed and un-recorded because one summary omitted it. Ask the
authoritative source instead:

```bash
of status --agent <agent> --json     # every cell + its display_state, route, disposition, 3 pillars
of status --json                      # the whole matrix
```

The cells whose `display_state` is **`pending-record`** are the record
work-list; cells with no assessment yet are the assess work-list. Dispatch
**one Agent per assess cell**; **`record` cells may be batched a few per
subagent** (see the parallelism rules). Then re-run `of status` to confirm none
remain before reporting "done". `display_state` ∈ `observed` (terminal),
`pending-record` (→ assess or record), `blocked-daemon` / `blocked-driver` /
`unobservable` / `n/a` (terminal — documented, no recording), `unknown`
(→ assess). A sweep is finished only when every cell is terminal.

For "how far along is agent X" — a progress read, not a work-list — use the
per-agent counts instead of scrolling the dump:

```bash
of status --summary                   # recorded / pending / blocked / unobservable / n/a / unknown / total, plus maturity / earned / core, per agent
of status --summary --agent <agent>   # one row; composes with --scenario too
```

Its buckets always sum to the agent's cell total, so nothing is hidden by the
aggregation. **It is still not a work-list**: `--summary` gives you a count,
never the cell ids, so deriving the next stage's work from it is the same
mistake as deriving it from subagent prose. Enumerate from the full dump.

To ask a different question of the same catalog — how much of the recorded
corpus actually exercises the hook channel — use:

```bash
of coverage --hooks                   # per adapter: recordings carrying a hook_received event
```

It joins the catalog to the daemon's adapter registry, so it separates the two
cases a raw count conflates: an adapter that **declares** a hooks permission and
has zero hook-bearing recordings (a `GAP` — the regression net does not cover a
channel the daemon depends on) from one that declares none (fine). Scope is
catalog cells; the non-catalog `regressions/` tree is excluded and the output
says so.

## Parallelism + ordering rules for sweeps

- **`assess` fans out; the PARENT commits.** assess is read-only of the live
  system (web + file research, no daemon), so dispatch assess cells in parallel
  waves to save wall-clock. But an assess subagent only WRITES its cell (via
  `of cell write` / `of cell spec`) — it does **not** commit. N parallel
  `git commit`s on one worktree race, scramble attribution, and once stranded a
  wave mid-`reset`. After each wave returns, the parent stages and commits the
  cells serially (one commit per cell). Committing is version control, not
  authoring — the iron rule forbids the parent *authoring* `replaydata/`, not
  committing what the subagents already wrote through `of`.
- **`record` is serialized and self-commits.** It drives a live CLI under a
  single recording daemon (attached or coexisting); concurrent recordings on one
  daemon interleave. Run record cells one at a time — serialization also assumes
  exclusive use of the worktree, since a concurrent commit smears the recording's
  provenance SHAs. Because record is serial there is no commit race, so the
  record subagent commits its own recording (part of its contract).
- **Batch a few serial cells per `record` subagent.** A record subagent records
  one cell at a time internally, so handing it ~4 cells per dispatch is far
  cheaper on dispatcher context/turns than strict one-subagent-per-cell, with no
  downside (the recordings still serialize). Keep batches small enough that the
  ≤7-line returns stay legible — one return block per cell in the batch.
- **Commit every assessment before the first record.** Cell `metadata.json` +
  `expected.jsonl` dirty `replaydata/`; the recording precheck refuses a dirty
  tree. So land all the parent-side assessment commits first, then record.
- After a `record` subagent returns, the recording is already committed (part
  of its contract). Don't re-stage, re-diff, or re-commit — just relay the
  summary.

## Human-blocking prerequisites

Some recordings need a human action the subagent cannot perform (e.g. switch a
subscription to an API key, install a CLI, set an env var). A subagent NEVER
asks the dispatcher a question — it runs to a terminal outcome and returns
`status: prereq_blocked` with the concrete blocker in `notes`. The dispatcher
surfaces that line to the human and moves on to the next cell. `of record
prereq-check --agent <agent>` lists an agent's known prerequisites up front.

## Filing daemon-bug issues (parent step, with consent)

A `record` subagent CANNOT file GitHub issues — outward-facing writes are denied
in its permission context, so for a `daemon=bug` cell it returns an `issue:`
payload (a temp-file body + a one-line title) instead of filing it. When a
record return carries a non-`none` `issue:`, the dispatcher — not the subagent —
files it: show the user the title + body, and ONLY on their confirmation run

```bash
gh issue create --repo ingo-eichhorst/Irrlicht --label bug \
  --title "<title from the payload>" --body-file "<path from the payload>"
```

Report the new issue number to the user. Wiring the number back into the cell,
if wanted, is a follow-up `assess`/`record` touch (a subagent) — the dispatcher
never authors `replaydata/`.

## Orchestrators (inline — no subagent)

Orchestrator scenarios (gastown) are hermetic: no live CLI, no auth, no token
cost. The driver replays canned inputs through the Go test harness and diffs
goldens, so the parent runs it directly:

```bash
replaydata/orchestrators/gastown/driver.sh <scenario>
```

Read the emitted `run-manifest.json` (`verdict: OK | CHANGED | ERROR`). For a
`CHANGED` cell, diff staged vs committed goldens; the maintainer reviews and
commits orchestrator goldens by hand (unlike agent cells, which `record`
commits).

## The `of` CLI

`of` is the factory binary (`go run ./cmd/of` from `tools/onboarding-factory`,
or the built `of` binary). The verbs the skill drives:

```
of status   [--agent a] [--scenario s] [--runs] [--summary] [--json]  coverage / run-log
of validate [--json]                                                  schema + referential integrity
of coverage [--hooks] [--json]                                        derived rollup, or hook coverage
of scenario add|update --name n [--id i] [--description d]
                       [--process-file f] [--acceptance-file f]
of agent add --id i --name n --provider p [--min-version v] [--prereq p]...
of cell write --agent a --scenario s --file metadata.json [--folder f]
of cell spec  --agent a --scenario s --file expected.jsonl [--folder f]
of verify --agent a --scenario s [--json]
of record prereq-check --agent a
of record run    --agent a --scenario s [--attach] [--dry-run]
of record verify --agent a --scenario s
```

Exit codes: `0` ok, `1` validation/operation failed, `2` usage error. Every
write verb validates-then-writes atomically and forces the foreign keys
(`scenario_id`), so a subagent cannot create a dangling cell.

## Anti-patterns

- **Don't touch `replaydata/` directly.** No `jq -i`, no `Edit`, no `Write`
  under `replaydata/`. Use the `of` write verbs — they are the only sanctioned
  writer and they validate first.
- **Don't run per-cell mechanics in the parent.** Research, driving CLIs,
  authoring specs, verifying recordings — all of that belongs inside a
  subagent. The parent routes and summarizes.
- **Don't enumerate a sweep's next stage from subagent prose.** Derive it from
  `of status`. Lossy summaries drop cells.
- **Don't record on a dirty `replaydata/` tree, and don't re-commit after
  `record`.** Commit assessments first; a clean tree is the record handoff
  signal and `record` commits its own recording.
- **Don't run an isolated recording daemon on production's port while
  production `irrlichd` is up.** Either `of record run --attach`, or coexist on
  a separate `IRRLICHT_ONBOARD_HOME` + non-7837 `IRRLICHT_ONBOARD_BIND_ADDR`;
  the precheck enforces both shapes. Onboarding a **new** adapter is always the
  coexist case — the running release has no such adapter compiled in and would
  observe nothing. See `record/SKILL.md` Precondition 4.
- **Don't accept a frozen verdict without probing it (push-through mode).** An
  `unobservable` / `blocked-*` cell may just need a feature built or an
  empirical probe — see Overnight push-through mode. But don't swing the other
  way and **fake green**: never ship a guess (e.g. reverse-engineered unlabeled
  data) to force a cell observed. Freeze honestly + file a follow-up.