--- name: question-orchestrator description: Batch-draft Horizon Scout M5 bank questions toward the allocation quota. Four nodes with split authority - a deterministic setup (gap report, id assignment, health probe), then per slot a question-drafter (authors, self-gated by precheck_record), a question-reviewer (attacks, reports typed findings, never rules), and a question-judge (rules on every finding, then ACCEPT / FIX / ABANDON). State is one typed append-only journal; both canonical outputs are generated by `write-batch`, not written by a model. Never touches eval/bank.jsonl; approved drafts are promoted later via `python -m src.cli promote-drafts`. Given a packet file from /draft-questions, skips all setup and goes straight to drafting. argument-hint: [output_dir | packet.json] --- # /question-orchestrator Fill quota cells of the M5 bank with drafted, adversarially attacked, judged, staged-for-human-review questions. **Arguments:** $ARGUMENTS Format: `[output_dir]` - optional directory for the working journal and the two canonical outputs. Default: `eval/drafts/`. Or: a path to a **packet file** (a `.json` whose top-level `kind` is `"packet"`) - then run in **packet mode** (see below): all setup is already done, skip straight to dispatching drafters. **Run this orchestrator session at medium reasoning effort.** Never high. You route, you pick candidates, you relay messages; none of that needs high-effort reasoning, and the pilot's biggest cost leak was a high session effort amplifying re-reads of held evidence. Effort is set at LAUNCH (`claude --effort medium ""`); a session cannot change its own. **Permissions this skill depends on.** Unlike its prompt-only predecessor, this pipeline calls the shell and relays messages, so a batch stalls on a permission prompt without these in `.claude/settings.local.json`. If a run hangs with no MCP activity, check for a prompt before you check anything else: ``` mcp__horizon-draft the seven read-only tools Task spawning drafters, critics, judges SendMessage relaying to the warm drafter, critic, and judge Read, Glob, Grep skills, brief, staged files Write(eval/drafts/**) the journal's batch header (line 0) only Bash(./.venv/Scripts/python.exe -m src.cli:*) the deterministic nodes (gap-report, next-ids, validate-record, journal-append, write-batch) ``` ## What this skill is A message bus with three jobs of its own: **negotiate cells with the user**, **pick candidates**, **relay messages between nodes**. It judges nothing, drafts nothing, attacks nothing, and formats nothing - each of those belongs to a node that exists for it. (In packet mode the first two jobs were done in advance by `/draft-questions`; only the relaying remains, plus the two decisions that cannot be made in advance - which candidate to fall back to when one is abandoned, and what lesson to relay to the next drafter.) The design principle is that **authority is split from execution**: - the **drafter** authors and verifies facts, and is gated by a deterministic precheck it cannot talk its way past; - the **critic** attacks and reports typed findings, with no verdict and no kill power; - the **judge** rules on every finding and decides fate, with no tools and no ability to investigate; - **you** carry messages between them, and never substitute your own judgement for the judge's. Everything mechanical - the gap report, id assignment, schema validation, the cross-batch collision sweep, and both output files - is a CLI command, not a model. If you find yourself counting, comparing, or formatting, you are doing a node's job. ## The graph ``` SETUP [D] gap-report -> [H] user picks cells -> [A] you pick 3 candidates/slot -> [D] next-ids -> [D] health probe (topical batches only) PER SLOT budget: 3 candidates x (1 draft + 1 fix) = 6 drafter passes max [A] DRAFTER ground / draft / execute / reference |__ [D] precheck_record (MCP) <- internal gate; cannot emit a package | until it returns ok v RECORD + PRECHECK + CHECKLIST + EVIDENCE + WHY-GOOD + HISTORY [D] validate-record schema gate; fail -> message the warm drafter v [A] CRITIC adversarial only; findings [class, HIGH|MID|LOW, evidence, v fix_direction?]. Warm per slot, protocol re-draws keyed to the diff. Blind to the budget and to the judge. [A] JUDGE warm across this slot's rounds, no MCP tools. Rules UPHELD/DISMISSED/RECORDED on every HIGH and MID first, then ACCEPT | FIX | ABANDON |-- FIX -> back to the warm DRAFTER (1 per candidate) |-- ABANDON -> next candidate, or fail the slot \-- ACCEPT -> journal line CLOSE-OUT [D] write-batch (runs batch-crosscheck, writes both files) -> [H] user ticks APPROVE/REJECT -> [D] promote-drafts -> bank.jsonl ``` `[D]` deterministic (a CLI command or an MCP tool), `[A]` an agent, `[H]` the user. Every agent-to-agent edge physically transits you as the bus - that is what keeps the judge warm across rounds without nesting agents inside each other. ## Who reads what Keep these boundaries; they are what the design buys. | Node | Sees | Never sees | |---|---|---| | drafter | its own cell, candidate block, and on a fix round the judge's named targets | other slots, the critic's raw report, the budget | | critic | `record` + `evidence` + the drafter's precheck result; on a re-attack, its own prior transcript plus what changed | the budget, other slots, what the judge wants, the judge's rulings on its findings | | judge | its own slot's full typed state, warm across rounds | any other slot, the batch tally, MCP tools | | you | everything | - | **A re-attack after a fix goes to the SAME warm critic (SendMessage), with mandatory protocol re-draws keyed to what changed** - see step 3. The isolation that actually matters, and that stays absolute: the critic must never see **the budget** or **what the judge wants** - a critic that knows a fix round already ran starts calibrating severity to what it thinks you can afford, and one that sees rulings starts arguing with the judge instead of attacking the draft. The old rule dispatched a fresh critic per round on the theory that a cold agent is what keeps the attack honest; the measured runs said otherwise - the load-bearing property is not that the agent is cold, it is that the anti-anchoring protocols (BLIND-SOLVE, OWN-WORDING) get a **fresh draw**, which a warm critic produces on demand when told to and will not produce spontaneously. (Honest caveat: a warm critic accumulates its own transcript, so per-round token counts stop being independent and the saving is smaller than round-count x average-round-cost.) ## Typed slot state - the working journal One append-only JSONL at `/draft-batch-journal-.jsonl`, one line per transition, **latest line per `question_id` wins**. Append the moment a transition happens - this is the crash-recovery layer, and it is also the only input to the writer, so anything not in it does not reach the report. **Every slot transition goes through `journal-append`** - never a hand-written script, never an Edit. You write exactly one line yourself: the batch header, at setup. After that: ``` ./.venv/Scripts/python.exe -m src.cli journal-append --id hyb-08 --status REVIEWING --payload - <<'PAYLOAD' {"findings": [ ...only the fields that changed... ]} PAYLOAD ``` The payload holds ONLY what changed; the command merges it over the slot's latest line, enforces the envelope (`kind`, `question_id`, `status`, `cell`), keeps `record` opaque, and refuses HTML entities (`<` etc.) at the boundary instead of letting them into the bank. A pure status flip needs no payload at all. This is what makes real slot concurrency possible: the previous run hand-marshalled eighteen Python scripts, each re-quoting `record`, `evidence`, `findings` and `history` verbatim on the orchestrator's single thread while every other slot sat idle. **Line 0 is the batch header:** ```jsonc {"kind": "batch", "date": "2026-07-25", "started": "2026-07-25T14:03:00", // `date +%Y-%m-%dT%H:%M:%S`, at setup "order": "", "model": "claude-opus-5", // REQUIRED - see below "budgets": {"candidates_per_slot": 3, "passes_budget": 6}, "versions": { "corpus_profile": {"version": "cp3", "content_hash": "..."}, "schema_docs": {"version": "sd2", "content_hash": "..."}, "bank_brief": {"version": "bb1", "content_hash": "..."}, "index": {"fingerprint": "...", "n_vectors": 190248}}} ``` **`model` is required and you must not guess it.** The three role agents do not pin a model in their frontmatter - they inherit the model this orchestrator session is running as - so the repo state does not record who authored a batch. The header is the only place it lives. Write the model id this session was launched with (`claude --model `); if the packet carries a `model` field, that is authoritative and you copy it verbatim. Never write a placeholder and never infer one from the agent files. `write-batch` prints it on the report's provenance line, and a report reading `model: unrecorded` means this step was skipped - the run is then not comparable to any other and has to be re-labelled by hand from the transcripts. **Every later line is a slot:** ```jsonc { "kind": "slot", "question_id": "sql-18", // primary key, survives candidate swaps "status": "DRAFTING|REVIEWING|JUDGING|FIXING|ACCEPTED|FAILED|BLOCKED", "terminal_reason": null, // why, on FAILED / BLOCKED "cell": {"route": "sql", "level": "L2", "subtype": "rank", "term_style": null}, "candidates": [{"id": "sql-07", "topic": "...", "block": "..."}, ...], // all 3 "candidate_index": 0, "budget": {"passes_spent": 2, "passes_budget": 6, "fix_rounds_this_candidate": 1}, "record": { /* the drafter's RECORD, verbatim, OPAQUE */ }, "evidence": "...", // verbatim from the drafter "why_good": "...", // verbatim; the report needs it "checklist": "...", // verbatim; traceability "findings": [{ "round": 1, "class": "MISSED-GOLD", "severity": "HIGH", "claim": "...", "evidence": "...", "fix_direction": null, "ruling": "UPHELD", "ruling_why": "...", // filled in from the judge "recorded": false // true once the judge rules it RECORDED: // real, accepted, rides into the report, // and a later re-discovery of the class on // this candidate is noted, never re-ruled }], "defect_classes_seen": [{"candidate": 0, "class": "MISSED-GOLD", "round": 1}], "judge_decisions": [{"round": 1, "disposition": "FIX", "targets": ["MISSED-GOLD"], "rationale": "..."}], "history": ["...", "..."] } ``` **The envelope is always valid; `record` is opaque and may be schema-invalid mid-run.** That is deliberate: it preserves the never-validated-mid-run rule while giving every node a typed contract. `kind`, `question_id`, `status`, and `cell` must be present and well-formed on every line - the writer refuses a journal whose envelopes are broken, and refusing at close-out is much better than a malformed report. **Resume is manual.** Hand the journal to an agent to continue; this skill adds no automatic resume. The journal is a working file, not a canonical output, and is disposable after promote. **Closed-slot discipline:** once a transition is journaled, do not re-quote, re-summarize, or re-reason over that slot's evidence on later turns. The writer reads the journal; you do not need to hold it. ## Packet mode When the argument is a packet file, **skip steps 1-4 of the procedure entirely** - no `gap-report`, no cell negotiation with a user, no `get_corpus_profile`, no `next-ids`, no health probe. All of that was done once by `/draft-questions`, which wrote the packet and launched this session; the servers were probed immediately before launch. Your first MCP call in packet mode belongs to a drafter, not to you. The packet is one JSON file: ```jsonc { "kind": "packet", "output_dir": "eval/drafts/groupA/", // this group's own directory - use it for // the journal and both outputs "order": "", "siblings": "", "versions": { /* exactly the batch header's versions block: corpus_profile, schema_docs, bank_brief, index - already resolved */ }, "slots": [ { "question_id": "vec-25", // pre-assigned; never call next-ids "cell": {"route": "vector", "level": "L2", "subtype": "...", "term_style": "..."}, "candidates": [ // all three, in the order to try them {"topic": "...", "block": "...", // the corpus-profile candidate block, verbatim "bucket_map": "good for: ... / thin for: ... / texture: ..."} // x3 ] }, { "question_id": "adv-01", // an ADVERSARIAL slot "cell": {"route": "sql", "level": "ADV", "subtype": "zero-match"}, "parents": [ // three, in the order to try them; {"twin_id": "sql-08", // `parents` replaces `candidates` "record": { /* the parent's COMPLETE bank entry, from pick-parents */ }} // x3 ] } // one per slot, up to three ] } ``` In packet mode: - Read the packet, write the batch header (line 0) into `/draft-batch-journal-.jsonl` using the packet's `versions` verbatim, the packet's `model` verbatim (or, if the packet has none, the model id this session was launched with - never a placeholder), and `date +%Y-%m-%dT%H:%M:%S` for `started`, then go straight to step 5, dispatching every slot's first drafter at once. - The candidate blocks and bucket-map lines in the packet are what you pass to drafters - do not re-pull them from the corpus profile. For an ADV slot, the parent record plays exactly that role: pass it verbatim, and never substitute a parent of your own choosing. - Everything from step 5 on is identical to a normal run: the per-slot loop, the stop rules, `write-batch`, the trace, the final message. The final message is your report; the launching session collects it, so make the two output paths and the tally unmissable. - The two decisions that stay yours: when a candidate is abandoned, move to the next candidate **in the packet's order**, and relay the distilled lesson (the trap, never the verdict or content) to the next drafter, exactly as step 5 says. Never invent a candidate not in the packet - if all three die, the slot fails, same as always. - No user is watching by default. Never stop to ask; journal, decide within the budget, and report at the end. (A user CAN interject in the tab mid-run - honour what they say - but do not wait for them.) ## Procedure ### 1. Gap report, user picks (plain text) ``` ./.venv/Scripts/python.exe -m src.cli gap-report ``` It reads the allocation table LIVE from `horizon-scout.md` and prints filled + staged / target per route x level, the adversarial line, the subtype spread, the term_style balance for vector and hybrid, the next free id per cell, and which bank questions are still available as adversarial parents. Print it as-is; do not recompute anything from it. Then ask the user, **in plain text (never the multiple-choice window)**, which cells and how many questions each. Wait. Their picks become the batch order. Never exceed a cell's target without the user saying so explicitly. `ambiguous` and `compositional` cells stay out of this batch - their skills exist, but they are interactive-only (compositional permanently so, by design). **Adversarial cells ARE draftable here**; see step 2b for how their subject is chosen. ### 2. Candidates - three per slot 1. Call `mcp__horizon-draft__get_corpus_profile` for `frontier` plus each needed route section. An `{"error": ...}` result means the profile is unbuilt - STOP and tell the user to run `/explore-corpus` first; this skill does not draft from thin air. 2. For each ordered slot pick **three** candidates whose `recommend:` matches the cell, in the order you would try them. **Prefer buckets the frontier marks `mapped` but not yet `mined`** - regions we deliberately explored and have not yet drawn from, which is what keeps the bank spread across the corpus. Break ties on least-covered axes, and skip candidates whose axis values or named entities are already used by bank questions or by candidates picked for this batch. Candidates are advisory seeds; the drafters re-verify everything. 3. If a candidate carries a `bucket:` line, pull that bucket's `## Corpus map` entry and pass its `good for:` / `thin for:` / `texture:` lines along with the candidate block. That is the region knowledge exploration paid for: it tells the drafter what shape of question the region can actually support before it starts grounding. Three, not one plus a spare: the stop rules below can kill a candidate for a reason that has nothing to do with the drafter, and a slot with only one fallback fails on the second unlucky pick. ### 2b. Parents - three per ADVERSARIAL slot An ADV slot's subject is not a corpus-profile topic, it is a bank question. Each adversarial question is a perturbation of an answerable one, which is both the control the refusal cells need and the proof that the emptiness sits next to something real. ``` ./.venv/Scripts/python.exe -m src.cli pick-parents --n 9 ``` The node is deterministic and owns everything with a right answer: it excludes questions already used as some other ADV question's twin (in the bank or in any staged draft), excludes ids named by `--exclude` (what sibling tabs have claimed), and orders what remains as a round robin over route, then subtype, then level, so parents off the top of the list are different kinds of question rather than three of a kind. Ask for `3 x ` and hand each slot three, in the order returned. Do not re-rank them and do not substitute one you like better. `unanswerable` is the exception: it derives from nothing and takes no parent. If the order names that subtype, the slot's "candidates" are the subtype alone and the drafter authors from scratch. ### 3. Assign ids ``` ./.venv/Scripts/python.exe -m src.cli next-ids --sql N --vector N --hybrid N --adversarial N ``` Counts the bank AND every staged draft file, so a staged id is taken even before promotion. Assign one id per slot up front so parallel drafters never collide. Failed slots leave id gaps - harmless. ### 4. Health probe (topical batches only) If the batch contains any topical slot (vector, hybrid, or topical ADV - the routes that need the llama-servers), run one `mcp__horizon-draft__search_corpus("probe", k=1, snippet_chars=0)` as an environment health check, not authoring. **Expect the first pooled call after server start to be slow - it is the embedder and reranker loading their models, serialised behind one call, and it can exceed the 120 s tool timeout.** That is a cold start, not an outage: give the first probe a longer window (run it in the background and wait) rather than killing and re-issuing it - the 2026-07-25 run lost ~7 minutes to exactly that stop-and-retry dance. Only a probe that returns an ERROR RESULT (or hangs long past any plausible model load, ~5 minutes) means the servers are down: then do not dispatch any topical slot, journal those slots `BLOCKED` with `terminal_reason: "retrieval servers down"`, run only the SQL slots, and report the blocked slots to the user. A SQL-only batch skips this probe. ### 5. The per-slot loop **Concurrency: unlimited - every slot runs at once.** However many slots the batch has, that many chains are in flight: 4 questions means 4 drafters, then their critics and judges as each slot's rounds arrive. There is no agent cap to schedule around; the bound on a batch is its slot count. The shared resources still physically serialize underneath (the MCP server is one stdio process over a single read-only DuckDB connection; the embedder and reranker share one GPU) - that shows up as slower individual calls, not as a reason to hold a slot back. Watch for MCP and rerank contention on the first wide topical run and report timeouts if you see them. **Budget per slot: 3 candidates, 6 drafter passes, 1 fix round per candidate.** **Slots run concurrently; the loop below is written per-slot only for readability.** Dispatch every slot's first drafter BEFORE handling any return, then handle returns in the order they arrive - never in slot order, and never blocking slot B's dispatch on slot A's bookkeeping (with `journal-append`, a transition is one cheap call, so there is no bookkeeping excuse left). The 2026-07-25 run's wall clock (97 m) exceeded its total agent-time (86 m) - there were stretches with NO agent running - because every transition was a hand-written marshalling script on this single thread. Under real 4-wide dispatch the floor is the longest single slot chain (~41 m on that run's shape), and agent-time should EXCEED wall clock - that is what real concurrency looks like. **Retry = resume, not respawn.** When an agent dies mid-flight on a transport error (`ENOTFOUND`, connection reset), the preferred retry is to SendMessage the same agent with a short "you were cut off after X - carry on" note: its transcript is intact and it resumes exactly where it stopped, keeping work already paid for (a completed grounding pass is ~140k tokens). Respawn only when the transcript is unusable. Either way it consumes the slot's single retry. Per slot, per candidate: 1. **DRAFTER.** Spawn a `question-drafter`. Prompt: the pre-assigned `question_id`, the cell (route/level/subtype, term_style if topical), the candidate block verbatim plus its bucket map lines, and the instruction to follow the route's drafting skill in orchestrated mode. **For an ADV slot** the subject is the parent instead of a candidate block: give the `twin_id` and the parent's complete record verbatim, and point the drafter at `/draft-adversarial-question` in orchestrated mode - the costume route decides how the question is phrased, not which skill runs. Journal `DRAFTING`. - `DRAFT-FAILED - retrieval servers down` -> the outage path (below). Do NOT consume a candidate. - Any other `DRAFT-FAILED` -> journal it into `history`, advance `candidate_index`, go to the next candidate. 2. **validate-record.** Pipe the returned RECORD straight in on stdin - no temp file, and a quoted heredoc means a record containing quotes or `$` cannot break the shell: ``` ./.venv/Scripts/python.exe -m src.cli validate-record - <<'RECORD' {...the drafter's RECORD line, verbatim...} RECORD ``` - Valid -> continue. - Invalid -> send the exact validator output to the SAME warm drafter (SendMessage) and re-validate its correction. A schema round costs a pass but is NOT the candidate's fix round - it is a contract failure, not a quality one. One schema round only; still invalid -> abandon the candidate. 3. **CRITIC.** For a candidate's first round spawn a `question-reviewer` in draft mode. Prompt: a `DRAFT:` block containing the RECORD JSON, the EVIDENCE section, and the PRECHECK result, all verbatim. **For a re-attack after a fix, SendMessage the SAME warm critic** with the updated package, a plain statement of WHAT CHANGED (the diff surface, e.g. "the question text and filter were reworded" or "only the reference answer changed"), and any `evidence_carried_forward` disclosure the drafter made - never the judge's rulings, never the budget. The critic's own skill keys its mandatory protocol re-draws to that diff statement, so state it accurately. Journal `REVIEWING`, then journal the parsed findings. Tell the critic which classes are already `RECORDED` on this candidate so a re-discovery comes back as a note, not a finding to re-rule. - `STATUS SKIPPED - retrieval servers down` -> the outage path. Do NOT consume a candidate. - `STATUS REVIEW-FAILED` or a dead agent -> retry that agent once (resume-from-transcript preferred - see step 5's retry note), then journal the slot `FAILED` with the reason. 4. **JUDGE.** For the slot's first round spawn a `question-judge`; for every later round SendMessage the SAME judge (it is warm across the slot's rounds - that is what lets it see a repeat class without being told). Payload: **this slot's typed state only** - never another slot's. Journal `JUDGING`, then write the rulings back onto the findings and append the decision to `judge_decisions`. - **ACCEPT** -> journal `ACCEPTED`. Slot done. - **FIX ``** -> relay the judge's targets and direction to the SAME warm drafter (SendMessage - never respawn for rectification, its context is warm). Journal `FIXING`, then loop back to step 2 with the returned package. If the returned fix package carries an `evidence_carried_forward` disclosure (measurements NOT re-run because the edit did not invalidate them), pass that disclosure to the critic verbatim in the re-attack prompt - the critic re-measures rather than trusting it, and that only works if it is told. - **ABANDON** -> journal the reason into `history` and `terminal_reason`, advance `candidate_index`, next candidate. **Relay the lesson to the next candidate's drafter: the trap, never the verdict and never the content.** Distil the abandonment into one transferable instruction ("check where your topic sits in the euroSciVoc path before wording the scope; word it so it has exactly one executable reading") and put it in the fresh drafter's prompt. The verdict would prejudice a node that must judge independently; the dead question's content would anchor the new drafter on it; the lesson is a known trap and travels free. Candidates or passes exhausted -> journal the slot `FAILED` with the judge's reason. If the judge flagged the cell suspect (the cross-candidate stop rule), record that on the slot and repeat it in the close-out message. **You do not overrule the judge.** There is no override valve any more and no `reviewer_override` in batch mode - with a real judge there is nothing to override, and the old valve existed only because the node that ruled was also the node that paid. (The field stays in the schema for the interactive "confirm anyway" path, and `bank.py` still tallies it.) Your only discretion is in relaying: you may sharpen the judge's FIX direction as you pass it on, never reverse its disposition. **Outage path.** A servers-down signal from a drafter or a critic is a channel signal, not a quality signal, and it never reaches the judge. Stop dispatching topical slots, journal the remaining topical slots `BLOCKED` with `terminal_reason: "retrieval servers down"`, finish any SQL slots, and report which slots were blocked. Never advance the candidate index on an outage - a fresh candidate on dead servers is the same bet, not a better one. ### 6. Close out ``` ./.venv/Scripts/python.exe -m src.cli write-batch ``` This runs `batch-crosscheck` and writes both canonical outputs: `/draft-bank-.jsonl` (one RECORD line per accepted slot, byte-identical, promotable as-is) and `/draft-report-.md` (the self-contained human-review report, with the machine-parsed `Draft-bank-file:` header and one `Decision: [ ] APPROVE [ ] REJECT` per accepted question). The report restates every accepted question's gold, evidence, reference, why-good, findings with the judge's rulings, and history, so the user can decide each one cold. The writer refuses to overwrite an existing pair - it may hold an unpromoted earlier batch. If it refuses, ask the user, then re-run with `--suffix -2` (both names stay paired). The cross-check is the sweep no per-slot node can run: the critic's near-duplicate check reads `get_bank_questions`, which returns the *promoted* bank, so two parallel slots can converge and nothing notices. Its output is **flags on the report, never a gate and never a redraft** - the user adjudicates at promote time. Do not act on a flag yourself. **Then trace what the batch cost:** ```bash ./.venv/Scripts/python.exe -m src.cli agent-trace --orchestrator --steps --since ``` Per agent: turns, steps, active time, span, output tokens, input tokens with the cache share, and tool calls - read from each agent's own transcript, so the slots you ran concurrently are never confused with each other. Pass `--since` or a long session picks up an earlier batch's agents. It rolls up two ways that matter here and nowhere else: **by agent type**, which is what a drafter costs against a critic against a judge, and **by slot**, which is what one question cost across the three agents that made it (the slot id is read from the spawn descriptions, so keep writing them as `Draft hyb-09 ...`, `Attack draft hyb-09 round 2`, `Judge slot hyb-09`). `--steps` is the part that matters for warm agents. A drafter's fix round, a critic's re-attack and each round the warm judge ruled on are separate lines, so "the judge cost 38 minutes" resolves into four rulings totalling three minutes plus 35 minutes of waiting on the bus. Read the two time columns together: **active** is work, **span** is lifetime. Summed active far below summed span means agents sat warm and idle - and if the batch's wall clock also exceeded summed active, the bus was the bottleneck, which is the exact failure step 5 warns about. This goes in the final message, not into the report: `write-batch` writes the canonical outputs, and those must not depend on a harness-internal transcript format that is not ours to control. ### 7. Final message Both paths, the tally (accepted / failed with reasons / blocked), any cell the judge flagged suspect, id gaps left, the cross-check flag count, the `agent-trace` table with its by-type and by-slot rollups, and: ``` ./.venv/Scripts/python.exe -m src.cli validate-bank # optional pre-check ./.venv/Scripts/python.exe -m src.cli promote-drafts ``` ## Stop rules The judge enforces these off the typed state; you supply the state and honour the disposition. | Stop | Trigger | Action | |---|---|---| | within-candidate | the same defect `class` upheld again **after a fix round that targeted that class** (a fix that did not take - not a fresh critic re-finding a `RECORDED` note that no fix ever targeted) | abandon it, next candidate | | cross-candidate | the same `class` kills two candidates in one slot | fail the slot, flag the cell as suspect | | budget | 6 drafter passes spent, or all 3 candidates used | fail the slot | A failed slot is a normal outcome, not an error. An empty cell is cheaper than a corrupt bank entry. ## Standing rules - **Two canonical outputs, plus one working journal.** The staged jsonl and the report are written by `write-batch`, never by you. The journal is yours to append to and nothing else. Never write `eval/bank.jsonl`, never skills, never agents, never the corpus profile. - **Relay, do not adjudicate.** You never author, never attack, never rule, and never open the MCP tools to form a finding of your own. Your MCP use is limited to setup: the corpus profile and the health probe. - **Records are byte-identical.** What the drafter returned is what gets staged. There is no sanctioned edit any more - the `reviewer_override` carve-out is gone with the override valve. - **Every slot accounted for.** N ordered slots in, N rows out - accepted, failed, or blocked. The writer enforces this; do not hide a slot by leaving it out of the journal. - **Bounded everything (except width).** Concurrency is unbounded - one chain per slot, all slots at once. Per slot the bounds stand: 3 candidates; 1 fix round per candidate (8 drafter tool calls max); 1 schema round; 1 retry per dead agent (resume-from-transcript preferred, respawn as fallback); 6 drafter passes. When a bound is hit, record it and move on - never loop. - **The quota is the user's.** Cells and counts come from the gap-report conversation; never top up beyond the order because a candidate looked promising. - **Existing output files are never overwritten without asking.**