--- name: red-team-pr description: Red-team a GitHub PR by writing focused failing tests, pushing them to the PR branch as CI proof, then fixing. Use to adversarially attack or validate a change before merge. --- # Red-Team PR ## First-run setup (one-time) If the literal token `` appears anywhere in this file, this is the first invocation. Run this setup BEFORE anything else; do not proceed to the rest of the skill until it is done. 1. Use `AskUserQuestion` to ask: "What is your GitHub username? Used to detect AUTO mode (PRs you own get auto-push; everyone else's get gated push)." Single text input. If the user wants to disable AUTO mode entirely, tell them to enter a sentinel that won't match any real author (e.g. `__no_auto__`). 2. Locate this SKILL.md on disk. The common paths are `~/.claude/skills/red-team-pr/SKILL.md` (which may be a symlink; `realpath` it before editing) or a project-local `.claude/skills/red-team-pr/SKILL.md`. If neither resolves, ask the user where the skill is installed. 3. Use `Edit` with `replace_all: true` on the resolved path to replace every occurrence of `` with the captured username. 4. Use `Edit` on the same path to remove this entire `## First-run setup (one-time)` section, from the H2 line through the blank line just before the next H2 (`## Branch Ownership Determines Mode`). After the edit, the file should open directly with the next section. 5. Continue with the skill below as if first-run never happened. If `` does NOT appear in this file, skip this section (setup already ran). ## Branch Ownership Determines Mode Resolve the PR's branch owner via `gh pr view --repo --json author,headRefName` (or the equivalent). Whose code this is dictates two things: push behavior and fix responsibility. **AUTO mode** (PR author is ``, or the user's own work): - Push red tests and fixes without asking. Speed matters; the user owns the branch and can revert anything. - Authoring the red test means you broke your own CI, so you MUST drive it green: fix the bug (or mark the test xfail / revert it if the invariant turns out to be wrong) before stopping. - The loop is: red -> push -> CI red -> fix -> push -> CI green -> next pass. Don't stop on a red CI you caused. **REVIEW mode** (any other author): - Always pause and request explicit user approval before `git push`, both for the red test and for any fix. Surface the SHA, local failure, one-sentence user-visible failure, and target branch. - You have no responsibility to fix the bug; that's on the implementor. Default to: push red test (after approval) -> confirm CI red -> stop and hand off. Only write a fix if the user explicitly asks for one. - Never push a fix to someone else's PR branch without an explicit ask. ## Core Loop 1. Isolate the PR in a safe worktree. 2. Resolve the PR author and pick AUTO vs REVIEW mode (see above). State the mode in your first user-facing message of the session. 3. Understand the intended behavior and changed code. 4. Hunt for an edge case that should fail because of the PR. 5. Add the smallest focused test that fails locally for the right reason. 6. Push gating: - **AUTO**: push the red test directly. - **REVIEW**: stop and request approval (show SHA, local failure, user-visible failure sentence, target branch), then push only after explicit approval. 7. Wait for CI to show the red test failing on the pushed commit. 8. Fix responsibility: - **AUTO**: write a fix, mention the red-test commit SHA in the fix message, push (no further approval needed), and watch CI go green. - **REVIEW**: stop here. Report red commit SHA + CI evidence and hand off. Do not write or push a fix unless the user explicitly asks. Stop only when additional red tests are speculative, duplicate existing coverage, require unreasonable infrastructure, or are outside the PR's behavioral scope. In REVIEW mode, also stop after CI confirms red unless the user wants more passes. ## Working Safely - Read repo instructions first, but never edit instruction files unless explicitly asked. - Check `git status --short --branch` before touching files. - Prefer a separate `git worktree` for PR branches so dirty user work remains untouched. - Fetch the PR branch explicitly and track the remote head. - Do not rewrite or revert unrelated local changes. - Keep each red-test commit narrow: test files only unless a fixture/helper is truly needed. - Prefer real DBs/services over mocks; a red test against a mock proves nothing. ## Find A Red Test Start from the PR summary, diff, and review comments. Identify the invariant the PR claims to preserve, then look for mismatches between: - new typed behavior and existing optimization/prefilter behavior; - backend implementations that should be equivalent; - direct query-builder SQL tests and higher-level integration behavior; - explicit API paths and inferred/default paths; - equality, ordering, null, boolean, numeric, mixed-type, empty, and reversed-operand cases; - fast paths, cache paths, and fallback paths. Prefer one broader test with multiple assertions when the repo's test style asks for it, but keep a red-test commit focused on one failure mode. A good red test fails locally before the fix, has a precise name, and would pass if the suspected invariant were implemented correctly. ### Severity filter Score each candidate on L x I x N (likelihood, impact, non-obviousness; 1-3 each). Skip anything <6. If you can't write the user-visible failure in one sentence, the test is contrived - drop it. Put that sentence in the commit body so a reviewer can sanity check severity. Suppress: contrived races needing artificial sleeps/interleavings; inputs the type system or upstream layer already rejects; bit-equivalence between paths the product doesn't promise to keep equivalent; failures only reachable through mocks the PR didn't touch; resource-exhaustion when standard handling exists. Spend on: auth/attribution boundaries, wrong values returned to users, migration/schema safety, backend-equivalence diffs the PR claims, real-input edges (null/empty/dup/tz/large/negative), wire-format/persisted-state compat. 3 real bugs > 15 nitpicks. If nothing survives the filter, the correct output is a short attack log, not a forced test. Before pushing a red test, verify: - the failure is deterministic locally; - the assertion failure points at the intended behavior; - lint/format checks on the changed test file pass when practical; - the test does not depend on external secrets unless the PR already requires them. In **REVIEW mode**, stop here and request user permission to push (red commit SHA, local failure, one-sentence user-visible failure, target branch). Wait for explicit approval. In **AUTO mode**, push directly, but still surface the SHA + failure in the next user-facing message so the user can interrupt if the test is misaimed. ## CI Proof After pushing the red-test commit: - Record the commit SHA. - Wait for the relevant CI check to fail on that SHA. Use `gh pr checks --repo --watch` or the repo's normal CI tooling. - If CI fails for an unrelated reason, do not treat that as proof. Inspect the failing job enough to distinguish unrelated infra from the red test. - If CI unexpectedly passes, re-check that the pushed commit contains the failing test and that CI ran the relevant shard. ## Fix Skip this whole section in **REVIEW mode**: the PR author owns the fix. Only proceed if the user explicitly asked for a fix, or you're in AUTO mode. In **AUTO mode**, fix only after CI has demonstrated the red test failing, unless the user explicitly says local proof is enough. The fix is mandatory: you broke your own CI, you drive it green. The fix commit message must mention the red-test commit SHA, for example: ```text fix(scope): handle bool prefilter numeric forms Fixes red test from 7c08f4f229. ``` After the fix: - run the formerly failing test and nearby tests; - run lint/format checks for changed files; - inspect the diff for accidental broadening; - **AUTO**: push directly to the PR branch, then monitor CI; - **REVIEW** (only if the user asked for the fix): stop and request approval (fix SHA, test passing locally, target branch) before pushing. ## Repeat In **AUTO mode**, once the fix is pushed, start another pass from the updated branch. Cap at 3 rounds or when correctness, auth/attribution (if applicable), and backwards-compat have each been exercised - whichever comes first. In **REVIEW mode**, default to one pass and hand off; only continue if the user asks for more. In the final response, report: - mode (AUTO or REVIEW) and PR author; - red-test commit SHA(s); - fix commit SHA(s) (AUTO only, or REVIEW-with-explicit-ask); - CI evidence observed; - local validation commands and results; - any residual risks or edge cases not tested.