--- name: fact-check disable-model-invocation: true description: > Verify factual accuracy of a document against the codebase and git history. Use when asked to verify, fact-check, or audit claims in a report or document. Accepts a file path or auto-detects the most recent HTML report. argument-hint: "[file-path] [--format html|md] [--lang ]" allowed-tools: Read, Glob, Grep, Edit, AskUserQuestion, Bash(git diff *), Bash(git log *), Bash(git show *), Bash(git rev-parse *), Bash(git branch *), Bash(git shortlog *), Bash(wc -l *), Bash(ls -t *) --- # Fact Check Verify the factual accuracy of a document against the actual codebase and git history. Extracts verifiable claims, checks each against source, corrects inaccuracies in place, and adds a verification summary. This is not a re-review. It does not second-guess analysis, opinions, or design judgments. It does not change the document's structure or organization. It is a fact-checker — it verifies that the data presented matches reality, corrects what doesn't, and leaves everything else alone. ## Instructions ### Format Detection `--format` controls which verification-summary block to inject: | Flag | Values | Default | Meaning | |------|--------|---------|---------| | `--format` | `html` \| `md` | auto | Auto-detected from the input file extension (`.html` → html block, `.md` → markdown block). Override when the input is a generic text file or when you want a markdown summary inside an HTML document | **Principle:** Fact-check edits the source document in place. The format flag only governs the **verification summary** appended at the end — HTML summary uses KPI cards + `
` blocks, markdown summary uses a table + bullet lists. Auto-detection is almost always correct; override only for edge cases. ### Target File Detection Determine what to verify from `$1`: 1. **Explicit path**: Verify that specific file (`.html`, `.md`, or any text document) - Resolve relative paths against cwd 2. **No argument**: Auto-detect the most recent report of either format: ``` ls -t ${CLAUDE_PLUGIN_DATA}/reports/*.html ${CLAUDE_PLUGIN_DATA}/reports/*.md 2>/dev/null | head -1 ``` If no reports found, inform the user and stop. **Document type detection** — auto-detect from page content to adjust verification strategy: | Document Type | Detection | Verification Focus | |--------------|-----------|-------------------| | diff-visual report | Contains "Diff Visual" in title/heading | Verify against the git ref the review was based on | | doc-visual report | Contains "Doc Visual" in title/heading | Verify file references, names, architecture claims | | plugin-visual report | Contains plugin analysis markers | Verify plugin structure, file paths, feature descriptions | | Markdown document | `.md` extension | Verify file references, function/type names, behavior descriptions | | Other | Fallback | Extract and verify whatever factual claims about code it contains | ### Language Detection Determine the output language for the verification summary: 1. **Explicit argument**: `--lang ` (e.g., `--lang ko`, `--lang fr`, `--lang zh`) → use that language. Any language code is valid 2. **User message text**: Detect the language of the message (excluding path) and match it - Examples: Korean text → Korean, Japanese text → Japanese, "en español" → Spanish, "auf Deutsch" → German 3. **Document language**: Match the language of the document being verified 4. **Default**: English ### Feedback File Detection After determining the target file, check for a companion `feedback.json`: 1. **Explicit argument**: `--feedback path/to/feedback.json` 2. **Auto-detect**: Check `~/Downloads/feedback.json` (macOS default download location) — verify `report_path` matches the target file. If multiple `feedback*.json` exist (e.g., `feedback (1).json`), use the most recent one. 3. **No feedback**: Proceed with standard full verification When feedback.json is present, adjust verification strategy: - **Sections with status "issue" + feedback text**: These are the user's primary concerns. Verify these sections FIRST and with extra scrutiny. The feedback text describes the specific problem — use it to guide what to check. - **Sections with status "ok"**: User reviewed and approved. Still verify, but at lower priority — only check quantitative claims and names. - **Sections with status "not-reviewed"**: Standard verification. In the Phase 5 Report, include feedback-driven summary: ``` Feedback-guided verification: {N} sections flagged by user {N} issues confirmed and corrected {N} issues not reproduced (user concern was unfounded) ``` ### Phase 1: Extract Claims *Why: Systematic extraction prevents cherry-picking. Every verifiable claim must be identified before verification begins.* Read the target file. Extract every verifiable factual claim into 5 categories: 1. **Quantitative**: Line counts, file counts, function counts, module counts, test counts, any numeric metrics 2. **Naming**: Function names, type names, module names, file paths referenced in the document 3. **Behavioral**: Descriptions of what code does, how things work, before/after comparisons 4. **Structural**: Architecture claims, dependency relationships, import chains, module boundaries 5. **Temporal**: Git history claims, commit attributions, timeline entries **Skip** subjective analysis: opinions, design judgments, readability assessments, severity ratings, recommendations. These aren't verifiable facts. ### Phase 2: Verify Against Source *Why: Each claim category requires a different verification method. Using the wrong method (e.g., Grep for quantitative claims) produces false confirmations.* For each extracted claim, go to the actual source: **Naming claims** — Glob + Read: - Verify every file path exists - Verify every function name, type name, and module name exists at the claimed location - Check for typos, renames, or stale references **Quantitative claims** — Bash git commands: - Re-run `git diff --stat`, `git log`, `git diff --name-status` and compare output against the document's numbers - Verify line counts with `wc -l` - Verify file counts with Glob **Behavioral claims** — Read source files: - Read every file referenced and check function signatures, type definitions - For diff-reviews: read both the ref version (`git show :file`) and working tree version to verify before/after claims - Check that described behaviors match actual code logic **Structural claims** — Grep + Read: - Verify import/dependency relationships - Check that architecture descriptions match actual module boundaries - Verify that claimed connections between modules exist **Temporal claims** — Git commands: - Re-run `git log` commands to verify activity narrative - Verify commit hashes, authors, dates, and messages - Check that timeline entries match actual git history Classify each claim: - **Confirmed**: Claim matches the code/output exactly - **Corrected**: Claim was inaccurate — note what was wrong and what the correct value is - **Unverifiable**: Claim can't be checked (e.g., references a file that doesn't exist, or requires runtime testing) ### Phase 3: Correct In Place *Why: Surgical corrections preserve the document's structure and style. Over-editing risks breaking HTML layout or changing the author's voice.* Use the `Edit` tool for surgical corrections: **Do correct**: - Incorrect numbers (line counts, file counts, commit counts) - Wrong function names, type names, file paths - Inaccurate behavior descriptions - Swapped before/after comparisons - Wrong git hashes, dates, or attributions - Factual errors in Mermaid diagram node labels or edge descriptions **Do NOT change**: - HTML layout, CSS, or animations - Document structure or section organization - Mermaid diagram styling or layout (only fix factual labels/edges) - Subjective analysis, opinions, or design judgments - Writing style or tone If a section contains a factual error, fix only the factual part. If a section is fundamentally wrong (not just a detail error), rewrite that section's content while preserving the surrounding HTML/markdown structure. ### Phase 4: Add Verification Summary *Why: Transparency — readers can see what was checked, what changed, and what couldn't be verified.* Insert a verification summary into the document. Choose block type based on resolved `--format` (explicit flag wins, otherwise auto-detect from file extension: `.html` → HTML block, `.md` → Markdown block, other text → Markdown block). **For HTML files** — insert a verification section matching the page's existing design: ```html

Verification Summary

{total} Claims Checked
{confirmed} Confirmed
{corrected} Corrected
{unverifiable} Unverifiable
Corrections Made
  • {description of each correction with file:line reference}
Unverifiable Claims
  • {claim that could not be verified and why}
``` Place the verification section as the last content section, before `` or the closing layout wrapper. **For Markdown files** — append at the end: ```markdown ## Verification Summary | Metric | Count | |--------|-------| | Claims Checked | {total} | | Confirmed | {confirmed} | | Corrected | {corrected} | | Unverifiable | {unverifiable} | ### Corrections Made - {description of each correction} ### Unverifiable Claims - {claim and reason} ``` ### Gotchas - **Over-correcting opinions as facts**: "This architecture is well-designed" is a subjective judgment, not a factual claim. Only correct things that can be verified against source — names, numbers, behaviors, paths. When in doubt, skip it. - **Modifying HTML structure**: The Edit tool is for surgical text corrections only. Do not reorganize sections, move content between sections, or change HTML wrapper elements. If a section is fundamentally wrong, rewrite the text content inside the existing `
` tags. - **Stale git refs in diff-visual reports**: A diff-visual report captures a snapshot. If new commits landed since the report was generated, the fact-checker sees different data than the report author did. Verify against the same ref the report was based on (look for commit hashes in the report), not HEAD. - **Feedback.json from wrong report**: The auto-detect checks `~/Downloads/feedback.json` which may be from a completely different report. Always verify the `report_path` field matches the target file before using feedback data. - **Counting claims too aggressively**: Not every number in a report is a "claim" worth verifying. Focus on claims that matter — metrics in KPI cards, file counts in summaries, function names in architecture descriptions. Ignore incidental numbers in prose. - **Mermaid diagram labels**: Mermaid node labels that contain function or file names are factual claims. If a diagram says `validateAuth()` but the actual function is `verifyAuth()`, that's a correction. But don't change diagram layout or styling. ### Phase 5: Report Output a summary to the user: ``` Fact-check complete: {file path} {total} claims checked {confirmed} confirmed {corrected} corrected {unverifiable} unverifiable {If corrections were made, list the top 3-5 most significant corrections} {If nothing needed correction, note that verification confirms accuracy} ```