---
name: qa-precon
description: Run the full QA pipeline for the next batch of 5 precon decks. Runs games, checks automated gates, prepares logs, AI-reviews gameplay, logs issues, and updates the QA index.
---

# QA Precon Skill

## Overview

Runs the complete QA pipeline for **5 precon decks at a time**. Every deck gets AI reviewed — no skipping. Runs 5 games each, checks automated quality gates (informational only), reviews all 5 game logs turn-by-turn, and logs any issues found.

A deck is **PASSED** only when AI review finds zero CRITICAL or MAJOR issues.

## Modes

| Mode | Flag | What it does |
|------|------|-------------|
| **New decks** | `--new` (default) | Pick next 5 decks that have NEVER been AI reviewed (`aiReviewed: false`) |
| **Recheck** | `--recheck` | Pick next 5 decks with status `HAS_ISSUES` for another review round |
| **Single deck** | `--deck <id>` | Target one specific deck |

The goal is to run `--new` until all decks have been reviewed once, then run `--recheck` in rounds until issues stop appearing (or are accepted).

## Status Flow

```
                    ┌──────────┐
                    │ PENDING  │ (never run)
                    └────┬─────┘
                         │ run games + gate check
                         v
              ┌──────────────────────┐
              │   NEEDS_AI_REVIEW    │ (gates checked, awaiting AI)
              └──────────┬───────────┘
                         │ AI reviews game logs
                         v
            ┌────────────┴────────────┐
            │                         │
    ┌───────v───────┐        ┌────────v───────┐
    │    PASSED     │        │  HAS_ISSUES    │
    │ (zero issues) │        │ (issues logged)│
    └───────────────┘        └────────┬───────┘
                                      │ --recheck (after fixes applied)
                                      │ re-run games, re-review
                                      v
                            ┌─────────┴─────────┐
                            │                     │
                    ┌───────v───────┐     ┌───────v───────┐
                    │    PASSED     │     │  HAS_ISSUES   │
                    └───────────────┘     └───────────────┘
                                          (repeat until clean)
```

- **PASSED**: AI review found 0 CRITICAL/MAJOR issues. This is the "done" flag.
- **HAS_ISSUES**: AI review found issues. Eligible for `--recheck` after fixes.
- **NEEDS_AI_REVIEW**: Games run, gates checked, waiting for AI review.
- **GATE_FAIL**: Gates failed but deck STILL gets AI reviewed.
- **PENDING**: Never been run.

**Key rule**: Gate failures are informational. They do NOT block AI review. Every deck gets reviewed.

## Process

### Step 1: Pick the Next 5 Decks

**For `--new` mode** (default):
1. Scan `results/qa_index.json` for decks where `aiReviewed` is `false` (or missing)
2. Pick the first 5 in any order

**For `--recheck` mode**:
1. Scan `results/qa_index.json` for decks with status `HAS_ISSUES`
2. Pick the first 5

**For `--deck <id>` mode**:
1. Use the specified single deck

### Step 2: Run Games for All 5 Decks

Run **5 games per deck**. For each deck:

```bash
node scripts/run_precon_tests.js --qa --games 5 --deck decks/precons/<deck_file>.txt
```

Find deck files by checking `decks/precons/manifest.json` or scanning `decks/precons/*.txt`.

**Run all 5 decks sequentially** in this step before moving to gate checks.

### Step 3: Check Automated Gates (All 5)

For each deck:
```bash
node scripts/qa_gate_check.cjs --deck <deck_id>
```

**Record the gate results but do NOT skip AI review for failures.** Gate info is useful context for the AI review (e.g., knowing there are TRIGGER_OVERFLOW violations helps focus the log review).

Note which gates failed — this context helps during AI review.

### Step 4: Prepare Logs for AI Review (ALL decks)

For **every** deck (regardless of gate results):
```bash
node scripts/prepare_qa_logs.cjs --deck <deck_id>
```

This creates `results/qa_review/<deck_id>/game_1.md` through `game_N.md`.

### Step 4b: Engine Health Check (ALL decks)

For **every** deck, generate an engine-health report. This is a pure-JS
analyzer (no AI) that cross-references each sample game's tagged
win-path against the mechanics actually observed in the log. It
catches cases where the engine claims a win via "Aristocrat Drain
Loop" but the game log shows zero sacrifices — pointing at engine
attribution bugs rather than deck issues.

```bash
node scripts/coach_regression.js \
  --deck decks/precons/<deck_file>.txt \
  --games 5 \
  --bracket <bracket_from_manifest> \
  --output results/qa_review/<deck_id>/
```

This produces two extra files alongside the per-game markdown:
- `<deck_id>_<timestamp>_briefing.md` — full Deck Report (same content the UI produces)
- `<deck_id>_<timestamp>_engine_health.md` — health classification + flags

**Scan the engine health report FIRST in Step 5.** The rollup line
tells you if the engine is executing or misrepresenting wins:

```
Engine health: 5/5 games executed their tagged win path.   ← clean
Engine health: 1/5 executed (20%), 4 flagged.              ← engine bug territory
```

**Health gate heuristic** — if `LABEL_TYPE_MISMATCH ≥ 2 of 5`, treat it
as an engine bug rather than a deck issue. The deck is winning through
a path the engine isn't correctly identifying, which masks engine
attribution problems. Log it as a separate engine issue in
`docs/PRECON_QA_ISSUES.md` under `Category: ENGINE_ATTRIBUTION`.

Skip the `--ai` flag here (we don't need the narrative coach during QA).

### Step 5: AI Review (READ the game logs)

For each deck, read in this order:
1. `results/qa_review/<deck_id>/_summary.md` (context)
2. `results/qa_review/<deck_id>/<deck_id>_*_engine_health.md` (pre-digested mismatches)
3. Each `game_N.md` (all 5 games)

The engine health report accelerates review — if mismatches are flagged,
you know where to look. If the rollup is clean, you can focus review on
intent (is the deck doing what it should?) rather than attribution.

**Review efficiently**: Scan for red flags rather than analyzing every line. Focus on:

1. **Legal Magic** — Mana math errors, double land drops, wrong zones
2. **Deck Strategy** — Commander cast and used? Win path advancing?
3. **Battlefield Activation** — Permanents being used? Rocks tapped?
4. **Intelligent Play** — Ramp early? Not holding bombs? Good discards?

**Skip detailed analysis of repeated ability activations** (e.g., 20x Helix Pinnacle PUMP lines). Just note the pattern if it looks wrong.

### Step 6: Log Issues (if any)

If issues are found, append to `docs/PRECON_QA_ISSUES.md`:

```markdown
## QA-NNN
- **Status**: OPEN
- **Deck**: <deck_id>
- **Commander**: <commander_name>
- **Game/Turn**: Game N, Turn M
- **Category**: <LEGAL_MAGIC|DECK_STRATEGY|BATTLEFIELD_ACTIVATION|INTELLIGENT_PLAY>
- **Severity**: <CRITICAL|MAJOR|MINOR>
- **Summary**: <one-line description>
- **Evidence**:
  - Turn X: <what happened>
  - Turn Y: <what should have happened>
- **Root Cause**: TBD
- **Fix**: TBD
- **Verified**: TBD
```

**Severity**:
- CRITICAL: Impossible/illegal state (casting without mana, wrong zones)
- MAJOR: Significant failure (key permanents unused 5+ turns, never executing strategy)
- MINOR: Suboptimal but acceptable (slightly wrong sequence, missing one opportunity)

**Deduplicate**: If an issue is the same root cause as an existing QA issue (e.g., same engine bug in a different deck), add the deck to the existing issue's evidence rather than creating a new one.

Update the Summary table counts.

### Step 7: Update QA Index

Update `results/qa_index.json` for each deck:

- If AI review found **zero** CRITICAL/MAJOR issues → status: `PASSED`
- If AI review found issues → status: `HAS_ISSUES`, record issue IDs
- Set `aiReviewed: true` and `lastChecked: <today>`
- Record gate results in the `gates` field (informational)

To update programmatically:
```bash
node scripts/qa_gate_check.cjs --deck <deck_id>
```
Then manually edit the status and `aiReviewed` in `results/qa_index.json`.

### Step 8: Update Process Log

Append to Changelog in `docs/PRECON_QA_PROCESS.md` — one entry per deck:

```markdown
### <deck_id> (YYYY-MM-DD)
- Status: PASSED / HAS_ISSUES
- Gates: [which passed/failed]
- AI Review: [clean / N issues found]
- Notes: [key observations]
```

### Step 9: Report

Summary for all 5 decks:

```
QA Batch: 5 decks reviewed (mode: new/recheck)
  <deck_1>: PASSED / HAS_ISSUES
  <deck_2>: PASSED / HAS_ISSUES
  <deck_3>: PASSED / HAS_ISSUES
  <deck_4>: PASSED / HAS_ISSUES
  <deck_5>: PASSED / HAS_ISSUES
New issues: N (X critical, Y major, Z minor)
Progress: N/202 PASSED (X.X%)
```

## Key Files

| File | Purpose |
|------|---------|
| `results/qa_index.json` | Master QA progress tracker |
| `results/precons/<deck_id>.json` | Test results with statistics |
| `results/qa_review/<deck_id>/` | Prepared game logs for AI review |
| `docs/PRECON_QA_ISSUES.md` | Issue log |
| `docs/PRECON_QA_PROCESS.md` | Process docs + changelog |
| `scripts/qa_gate_check.cjs` | Automated gate evaluation |
| `scripts/prepare_qa_logs.cjs` | Log preparation for AI review |
| `scripts/run_precon_tests.js` | Test runner (use `--qa` flag) |

## What NOT to Flag

- Goldfisher limitations (no blocking, no opponent interaction)
- Opponent rotation being simple/predictable
- Conformance grade being low due to scorer calibration (if the deck IS playing well)
- MANA_REUSE integrity warnings (known engine limitation, excluded from gates)
- Win rate or speed alone (unless something is clearly broken)

## Windows Bash Rules

**NEVER use inline `node -e` or complex bash one-liners.** Write `.temp/lookup_qa.cjs` scripts instead.