--- name: nocap-robust-review description: Use when the user requests critical review, verification, or analysis. Mode A handles iterative document editing (regression checking, content loss detection, register alignment, cross-reference propagation across file dependencies). Mode B handles analytical/factual review (evidence evaluation, position stability, multi-theory comparison, depth beyond surface). Read this skill when the user asks for review, verification, critical analysis, or editorial checking. Integrates with nocap. --- # Robust Review Protocol Author: HyperWorX (https://github.com/HyperWorX) License: MIT Read this document when the user requests any form of review, verification, critical analysis, or editorial checking. This defines how to conduct reviews that go beyond surface-level confirmation. --- ## 1. Mode Selection ### 1.1 Mode A: Evolutionary Edit Review Context: reviewing iterative edits to a document being workshopped. Any situation where prior versions exist (including prior versions produced by Claude in the same conversation) and the task is to assess whether edits improved, preserved, or degraded the document. Triggers: "review this edit", "check I haven't lost anything", "compare versions", "is this still aligned", "workshop review", "regression check", or any iterative editing context. ### 1.2 Mode B: Analytical/Factual Review Context: evaluating claims, approaches, recommendations, or reasoning for robustness. Any situation where the question is "is this actually right" or "does this hold up." Triggers: "verify this", "is this actually right", "review critically", "challenge this", "fact check", "does this hold up", or any request for analytical scrutiny. ### 1.3 Both Modes Some reviews require both. Example: reviewing an edited document that contains factual claims. Apply Mode A to structure and content preservation; Mode B to claims and reasoning. Report findings separately and clearly labelled. **Execution order.** When both modes apply: 1. Run Mode A first (edit review: regression, cross-reference propagation, intra-document impact, standing-directive compliance). Mode A surfaces structural changes the edit introduced. 2. Run Mode B on the content that changed. Mode B does NOT re-review content that Mode A confirmed as unchanged; analytical review applies to new or altered claims only, except where the user has explicitly scoped Mode B to the whole document. 3. Report findings in two clearly labelled sections: "Mode A findings" and "Mode B findings". If a finding appears in both (e.g., a factual claim was both moved and altered), note it in Mode A with a Mode B cross-reference rather than duplicating. **Conflict handling.** If Mode A and Mode B produce contradictory findings (Mode A says preserve, Mode B says the preserved content contains an unsupportable claim), surface the contradiction explicitly to the user: "Mode A preservation check passed; Mode B identifies [claim X] as unsupportable. Resolution choice: preserve original (accept weaker claim), revise (alter original intent), or flag for user decision." Do not silently resolve. ### 1.4 Ambiguous Trigger If unclear which mode applies: ask one question. "Structural/ content review of the edits, or analytical review of the claims and reasoning?" ### 1.5 When This Protocol Does Not Apply Simple requests like "proofread this" or "fix the grammar" do not require this protocol. This is for substantive review, not mechanical correction. --- ## 2. Mode A: Evolutionary Edit Review ### 2.1 Regression Checking Systematic checks across iterations: 1. Content inventory: enumerate key elements from the prior version. Verify each is present in the current version. If absent, flag the specific content that was lost. 2. Intent alignment: restate the document's stated purpose. Does the current version still serve that purpose, or has it drifted toward serving the edits instead? 3. Nuance preservation: identify specific nuanced phrasings, qualifications, or subtleties from prior versions. Are they preserved, diluted, or removed? 4. Register consistency: is the voice, tone, and formality level consistent throughout, including newly added sections? 5. Structural coherence: does the document read as a unified whole, or do the edits create visible seams? This applies equally when Claude produced the prior versions. Claude's own edits are subject to the same regression checks. ### 2.2 Feedback Integration Quality When edits respond to feedback: - Has feedback been adapted to fit the document's register, or inserted verbatim? - Does the integration disrupt flow, rhythm, or voice? - Has the document been re-evaluated as a whole after edits, or were edits applied locally without global review? - Are feedback items treated as a checklist to clear, or as input to improve the whole? ### 2.3 The Contraction Problem The risk in iterative editing is treating feedback as a checklist: fold it in, tick it off, move on. The actual requirement: integrate feedback in a way that serves the document's overall intent, register, and coherence. "It's in there" is not the same as "it's in there well." After integration, the question is not "did I add it?" but "does the document read as though this was always part of it?" If a feedback item would disrupt the document when added, that needs to be flagged, not silently forced in. ### 2.4 No Prior Version Available If no prior version exists: - State what can be checked: register consistency, internal coherence, intent alignment within the current version. - State what cannot be checked: content loss, nuance preservation relative to prior versions. ### 2.5 Cross-Reference Propagation When the reviewed file references or is referenced by other files, changes may break or invalidate those references. Files that reference each other form a dependency graph. A change to one node can break connected nodes. This check is mandatory for skill file edits and applies to any file set where cross-references exist. **Step 0: Define scope.** Before checking references, state the search scope: which files are subject to cross-reference checking? Defaults by context: - Skill file edits: all files in the skill directory plus custom instructions. - Project work: the project's file set as defined by the user or the working directory. - Codebase: the relevant source tree and configuration files. The user may define or adjust scope. State the scope in the process trace. Versioning: if a file within scope was reviewed in a previous round and has not changed since, it can be skipped as a starting node. It cannot be skipped as an affected node. If the current change's Grep tool hits an unchanged file, that reference still needs the narrow check in Step 2. What gets skipped is re-reviewing unchanged files from scratch, not checking whether they are affected by the current change. Log any skips and the reason in the trace. **Step 1: Discover the affected set.** Mechanical, not judgment. Use the Grep tool or equivalent: - Search all files within scope for references to the changed file by name or section number. - Search the changed file for outward references to other files. - The union is the affected set. When using Claude Code tools, adapt the search path to the defined scope: Grep tool: search for "file-name" or "section-number" in [scope path] Limitation: Step 1 discovers explicit references (file names, section numbers, direct mentions). Conceptual dependencies, where a file uses an idea from another file without naming it, are not discoverable by the Grep tool. If the user is aware of conceptual dependencies, they should add them to the affected set. The user's knowledge is authoritative here. Out-of-scope references: the Grep tool may reveal references to targets outside the defined scope. Not every external reference needs flagging. Apply this filter: - Skip silently: standard library imports, widely-maintained packages the user does not control (e.g. numpy, React, matplotlib, standard framework APIs). These do not change as a result of the user's edit. - Flag: user-authored code in other repositories, internal shared libraries, community packages with low maintenance or known instability, configuration or deployment files outside the project tree. The test: could the user or their team have written, modified, or configured this target? If yes, flag. Flagged out-of-scope references are informational. They do not block the procedure. Report in the trace: "Reference to [target] in [file] is outside current scope. Cannot verify." The user decides whether to expand scope or accept the risk. Before flagging anything as out of scope, verify it is genuinely not in the defined scope. If the scope is a codebase and the reference targets a file in a subdirectory that was not searched, that is a discovery failure, not an out-of-scope finding. Expand the search before flagging. **Step 2: Narrow check on each affected file.** For each file in the affected set, check only the referencing lines. This is not a full Mode A review. The question is narrow: does this specific reference still point at something that exists, and does it still mean what it meant before the change? Four outcomes per reference: (a) Still valid. The referenced target exists and its meaning is unchanged. No action. (b) Broken. The referenced target was removed, renumbered, or renamed. Fix the reference. (c) Out of scope. The reference target is outside the defined search scope and meets the flagging criteria from Step 1. Report per Step 1 procedure. Continue checking remaining references. (d) Meaning shifted. The referenced target still exists but its content has changed. Apply the following sub-procedure to decompose the assessment: (d.1) State what the referencing line uses from the target. Be specific: which function, definition, rule, or behaviour does it rely on? (d.2) State what changed in the target. Diff-level, not summary. (d.3) Classify the overlap: (i) No overlap. The referencing line uses an unchanged portion of the target. Treat as (a). (ii) Partial overlap. The referencing line's use is affected but not invalidated. The reference remains correct but may need updating for precision. Flag and state what shifted. (iii) Direct overlap. The thing the reference relies on is what changed. The reference is functionally broken even though the target still exists. Treat as (b) and fix. This sub-procedure decomposes one vague judgment ("meaning shifted") into three steps, two of which are mechanical (what does the reference use, what changed) and one of which is a constrained comparison on explicitly stated inputs. **Step 3: Propagate if changes were made.** If Step 2 resulted in changes to a referencing file, that file becomes a new starting node. Return to Step 1 for that file, but only for the changed lines, not the whole file. Track in thinking. Include search terms used for auditability: ``` Cross-reference propagation: Scope: [defined scope] Searched: [Grep tool queries or equivalent] Round 1: Changed [file] [section] -> [other file] [line]: (a) valid -> [other file] [line]: (b) broken, fixed [description] -> [other file] [line]: (d.ii) partial overlap, flagged Out-of-scope: [target] in [file], flagged Skipped: [file], unchanged since [review/round] Round 1 result: [changes made / no changes]. [Continue/Stop.] ``` A file that has been checked is not checked again unless a new change is made to it during propagation. **Step 4: Termination.** Stop when a round produces no new changes to any file. Hard ceiling: 3 propagation rounds. If changes are still cascading after 3 rounds, stop and report: "Changes still propagating after 3 rounds. Remaining affected files: [list]. This suggests structural coupling, not a local fix." This follows the same pattern as nocap-systematic-analysis Phase 4.4: repeated failures indicate a structural problem. ### 2.6 Intra-Document Impact Checking When an edit changes content within a document, check whether that change breaks, invalidates, or contradicts something elsewhere in the same document. This is 2.5's logic applied within a single file rather than across files. Recommended Mode A check order: 2.1 (did the edit lose anything?), then 2.6 (did the edit break anything within the document?), then 2.5 (did the edit break anything across files?). **Step 1: Classify the change.** Apply FCP. What did the edit change? Categories: (a) Wording only. Meaning unchanged. Skip impact checking. (b) Claim, definition, position, or rule changed in meaning. Proceed to Step 2. (c) Structural change (section added, removed, renumbered, reordered). Proceed to Step 2. (d) Constraint compliance. The edit responds to a standing directive (nocap §11.6) or stated requirement. Proceed to Step 2, specifically checking whether the response is consistent with all active constraints, not just the one it addresses. (e) Unclear. Treat as (b) and proceed. **Step 2: Discover intra-document dependencies.** Two categories of dependency, checked separately: Explicit dependencies (mechanical discovery): - Direct references within the document ("as stated in Section X", "see above", "the definition in paragraph 2") - Citations and citation chains (claim A cites source X, conclusion B depends on claim A) - Numbered cross-references, internal links, defined terms used elsewhere in the document - Structural dependencies (table of contents, section numbering, headers that reflect content) Search method: scan the document for references to the changed section by name, number, or direct mention. With Claude Code tools, use the Grep tool on the document file. Without Claude Code tools, scan in thinking. Implicit dependencies (judgment required, apply FCP): - Logical chains (conclusion A is a premise for argument B) - Consistency constraints (character traits, timeline, world rules, stated positions, register) - Narrative flow (does section order still make sense?) - Tonal/register consistency (did the edit shift voice in a way that clashes with surrounding content?) These cannot be discovered mechanically. For each, state in thinking: (imp.1) What does the changed content establish? (A claim, a rule, a character trait, a tonal register, etc.) (imp.2) Where else in the document does something depend on or assume that established content? (imp.3) Classify: (i) no downstream impact, (ii) downstream content needs updating, (iii) unclear, needs closer examination. The bias on implicit dependency checking is toward "no impact" because finding impact means more work. This is the same pattern as 2.5 Step 2(d). FCP classification counters this by requiring the reasoning to be stated before the conclusion. **Step 3: Check and fix.** For each dependency found: - Explicit: does the reference still point at the right thing? Apply 2.5 Step 2 outcomes (a), (b), and (d). Outcome (c) (out of scope) does not apply within a single document. - Implicit: does the downstream content still hold given the change? If a premise changed, does the conclusion still follow? If a character trait changed, are later scenes still consistent? If a definition changed, are all uses still correct? Fix broken dependencies. Flag ambiguous ones for the user. **Step 4: Standing directive check.** Verify the edit complies with all active standing directives (nocap §11.6). Standing directives relevant to the document include register requirements, structural constraints, style rules, and any user-stated rules about content. If no standing directives exist, state "no standing directives active" and skip. If a standing directive is being violated by the edit, flag it. Do not silently fix it if fixing would change the edit's intent. Report the conflict. **Scope control.** Not every edit triggers a full dependency trace. Step 1 classification (a) skips entirely. For (b), (c), and (d), the scope scales with the change: - Single-sentence change to a non-load-bearing paragraph: check explicit references only. Implicit check is skipped (the change does not alter downstream conclusions by structural test: "load-bearing" defined below). - Change to a definition, premise, or structural element: full check, both explicit and implicit. - The judgment of "load-bearing" means: if this content is wrong, does it change a conclusion elsewhere in the document? If yes, it is load-bearing and warrants the full check. **Document type notes.** Technical/argumentative: primary risks are broken citation chains, invalidated premises, and conclusions that no longer follow from their evidence. Creative: primary risks are character inconsistency, timeline contradictions, broken world rules, register shifts, and narrative arc disruption. Code: primary risks are broken method contracts, stale comments, state assumption violations, and API signature mismatches. Mixed documents (like these skill files): primary risks are cross-references between sections, defined terms used elsewhere, and procedures that build on earlier procedures. --- ## 3. Mode B: Analytical/Factual Review ### 3.1 Depth Requirement Do not stop at the first two layers. This user has explicitly requested depth. Layer 1: The claim and its immediate justification. Layer 2: Is the justification supported by evidence, or merely asserted? Layer 3: What competing justifications exist? What does each one's evidence look like? Layer 4: Meta-factors: recency, sample size, methodological rigour, journal/source quality, credibility, consensus status. Layer 5: Does the conclusion hold if approached from a different framework or discipline entirely? Minimum for any Mode B review: Layer 3. Layers 4 and 5 when the question warrants it or evidence is contested. ### 3.2 Evidence Quality Dimensions When evaluating competing positions, assess evidence on: 1. Source type: primary research, systematic review, meta- analysis, expert opinion, anecdotal, training data pattern. 2. Methodology: sample size, controls, blinding, replication. 3. Recency: when established? Updated or superseded since? 4. Publication quality: peer-reviewed journal, preprint, blog, forum, unknown provenance. 5. Consensus level: widely accepted, contested, emerging, fringe. 6. Applicability: does this evidence apply to the specific case, or is it generalised beyond its scope? ### 3.3 Do Not Validate the First Approach The first plausible answer is the most statistically likely completion, not necessarily the best answer. Before settling on any assessment, generate in thinking: "One alternative approach is..." and actually complete the sentence with a named alternative. This forces token generation that prevents "I looked for alternatives and there are none" without actually looking. If unable to name a concrete alternative after generating the prompt tokens, state that explicitly in the review. "No alternative approach identified" after genuinely attempting to generate one is informative. "No alternative approach identified" without attempting is first-path bias. This applies the Forced Classification Protocol principle (nocap Section 12.0): forced token generation creates procedural checkpoints harder to skip than abstract guidelines. ### 3.4 Multiple-Theory Presentation When multiple valid positions exist, present them with parallel structure and equivalent detail: "Position A: [description]. Supported by [evidence with quality indicators]. Strengths: [specific]. Limitations: [specific]. Position B: [description]. Supported by [evidence with quality indicators]. Strengths: [specific]. Limitations: [specific]. Comparison: [differentiating factors]. A may be considered stronger because [specific reasons with evidence basis]. B should not be dismissed because [specific reasons]. Factors that could shift this: [specific]." This structure prevents premature commitment and forces explicit evidence comparison. ### 3.5 Good Scientist Framing Present findings the way a careful researcher would: "From the available evidence, X suggests Y for these reasons. Z also has merit for these reasons. X may be considered stronger because [specific factors including evidence quality]. Z should not be dismissed because [specific factors]." Never state "X is correct" or "Z is wrong" without explicit qualification and cited evidence basis. ### 3.6 The Flip Test Before presenting any weighted assessment, apply the Forced Classification Protocol principle (nocap Section 12.0) with this concrete procedure: 1. State the assessment and its evidence basis in thinking. 2. Generate 3-5 sentences arguing the opposite position as strongly as possible in thinking. Actually produce these tokens. Do not summarise what they would say. 3. Classify: (a) Generating the opposite was difficult because specific evidence contradicts it. The assessment is grounded. State what evidence would need to be refuted for the position to change. (b) Generating the opposite was easy and equally plausible. The assessment is not grounded. Present both positions as roughly equivalent with respective merits stated. The mechanism: Step 2 forces generation of counter-argument tokens. If the counter-arguments are strong, the assessment cannot be presented as weighted without a visible coherence tension. "Simulate" was too vague to produce this effect. Generating actual counter-argument tokens does. ### 3.7 Challenge-Resistance Protocol If the user challenges an assessment after presentation, apply the Forced Classification Protocol (nocap Section 12.0) with the same categories as nocap Section 12.2: (a) New evidence: user provided information not previously available. (b) New reasoning: user provided an argument not previously considered. (c) Disagreement: user disagreed without providing new information or reasoning. (d) Restatement: user restated the opposite position. (e) Exposed flaw: challenge reveals a flaw in the original reasoning that was always present. (f) Exposed gap: question or reframing illuminates something the reasoning never addressed. If classified (a) or (b): state the specific new information. If nothing specific can be pointed at, reclassify to (c) or (d). If evidence test passes, update with specifics: "That changes the assessment because [specific reason]." If classified (c) or (d): maintain. Generate in thinking: "This is disagreement without new information. The cited evidence still supports [original position] because [reasons]." Then state in visible output that the challenge was considered and the evidence basis holds. Known LLM behaviour: models are trained on feedback data that disproportionately penalises continued disagreement over capitulation, producing an observed tendency to update position downward under challenge regardless of whether the challenge supplies new evidence. The magnitude of this asymmetry varies by model and prompt and has not been quantified here with a specific multiplier; treat it as a directional bias, not a fixed ratio. The FCP classification procedure is the mitigation: forcing explicit categorisation of what the challenge actually contains, anchored to specific evidence rather than to confidence level. ### 3.8 Chain-of-Verification After forming an initial assessment: 1. Generate 2-3 verification questions that would test it. 2. Answer those questions independently of the initial assessment. 3. Check whether the answers support, weaken, or contradict the assessment. 4. Revise if warranted. Report what changed and why. ### 3.9 Evidence Search When reviewing factual claims, evidence gathering is not reactive searching. It is planned investigation. The steps below replace the default pattern of "search for the claim and see what comes back." #### 3.9.1 Identify Load-Bearing Claims Before searching, identify which claims carry structural weight. Not every claim needs verification. A load-bearing claim is one where being wrong has consequences beyond the claim itself: it changes the conclusion, invalidates the approach, or propagates error downstream. Apply FCP (nocap Section 12.0) to this identification. For each candidate claim, classify: (a) Load-bearing: if wrong, the conclusion or approach changes. State the cost of being wrong. (b) Supporting: strengthens the case but the conclusion survives without it. (c) Definitional or established: the modal completion is effectively the only completion. Training data is sufficient (per nocap Section 4). No search needed. Classify (c) conservatively. If uncertain whether something is established, classify as (a) or (b) and search. Output in thinking: a numbered list of load-bearing claims with stated cost of error for each. #### 3.9.2 Search Planning Plan in thinking before executing any search. For each load-bearing claim from 3.9.1: 1. What evidence would strengthen this claim? Be specific about what form that evidence would take. 2. What evidence would weaken or refute it? Be equally specific. 3. Where does this evidence likely live? Academic sources, government data, industry reports, technical documentation, news reporting, domain-specific databases. 4. What search queries would reach that evidence? Generate 2-3 queries per claim, targeting different source types or angles. Query selection is subject to first-path bias (nocap Section 10). Multiple queries from different angles mitigate this. 5. What would "no results" mean? Distinguish between: the claim is too niche for web sources, the claim is wrong and no one supports it, or the queries were poorly constructed. These are different and lead to different next steps. Report the search plan in the process trace. The plan is auditable: the user can see what was targeted and what was not. #### 3.9.3 Search Execution Execute the plan from 3.9.2. Rules: - Search for the claim AND for counterarguments. One-sided searching is confirmation bias. For each load-bearing claim, at least one query should target evidence against it. - Apply evidence quality dimensions (Section 3.2) to every result. Source type, methodology, recency, publication quality, consensus level, applicability. - Evaluate between searches. After each search, assess in thinking: did this change the picture? Does the plan need adjusting? New information from early searches should inform later queries, not be ignored. - If a search returns nothing: apply the "no results" assessment from 3.9.2 step 5. Do not treat absence of evidence as evidence of absence without stating why. - If results conflict with each other: present the conflict explicitly using the multiple-theory structure from 3.4. Do not resolve the conflict by selecting the result that matches the initial assessment. Mode A typically does not require evidence search unless the document contains factual claims (in which case, apply Mode B to those claims). ### 3.10 Deliberative Review Entry Point When Mode B review involves contested claims with multiple valid analytical frameworks, or when Mode A review involves structurally complex edits with systemic effects, apply the assessment gate (nocap Section 14.1) via the deliberative review procedure at Section 4.1. Section 4.1 is under "Self-Calibration (Both Modes)" because it applies to both Mode A and Mode B. This cross-reference exists because Mode B's position in the document is distant from Section 4.1 and the trained tendency is to process only the immediately relevant mode section. --- ## 4. Self-Calibration (Both Modes) After completing any review, state: - Confidence in the review: low, medium, or high, with the basis for that assessment. - What would change the assessment: specific new information or evidence that would alter the findings. - What was not checked: areas that were outside scope or that could not be verified with available information. ### 4.1 Deliberative Review When Mode B review involves contested claims with multiple valid analytical frameworks, or Mode A review involves structurally complex edits with systemic effects across a codebase or document set, deliberative agents strengthen the review by providing independent perspectives. The assessment gate (nocap Section 14.1) determines whether deliberation applies. This is not a judgment call; it is an FCP classification. Run the gate. Follow the result. **Review-specific gate triggers.** The gate in §14.1 was defined for work containing decision points that produce state change. Review work produces findings, not state change, so the generic triggers do not map directly. For review tasks, use these review-specific trigger patterns in place of §14.1's code-oriented triggers: - **Mode A (edit review):** deliberation applies when the edit crosses 3+ files OR alters a shared interface used by 2+ downstream consumers OR modifies content inside a concealment architecture or load-bearing section where downstream impact is not mechanically traceable. Fewer than 3 files and no interface changes and no architectural implications -> self- execute, no deliberation. - **Mode B (analytical review):** deliberation applies when the claim under review has 2+ competing analytical frameworks with equivalent explanatory power OR 3+ independent load-bearing sub-claims whose evidence weight is contested OR the review must evaluate whether a framework itself is apt for the domain. A single-framework analytical review with clear evidence -> self-execute. - **If uncertain whether any trigger above applies,** apply the generic §14.1 FCP categories (a)-(d) and let the evidence- gathering step determine the classification. Procedure: 1. Apply the assessment gate with the review-specific triggers above. If classified (a) deliberative or (d) hybrid, proceed. Otherwise, conduct the review directly. 2. Run FCoP (nocap Section 14.2) to determine agent count and assign review focuses. For review work, the FCoP perspective inventory enumerates evaluation angles (frameworks, quality dimensions, assumption sets), not generation angles. 3. Spawn generation panel agents, each reviewing the same material from a different perspective: - Different analytical frameworks (quantitative, qualitative, comparative, first-principles). - Different quality dimensions (correctness, completeness, consistency, systemic impact, evidence quality). - Different assumption sets about the reviewed material. 4. Collect all review findings. Spawn arbitration panel (nocap Section 14.6) to synthesise findings into a unified review that identifies agreements, disagreements, and areas requiring further investigation. 5. Present the synthesised review. Where arbiters disagree, present both positions with evidence. All agents must follow the Protocol Inheritance Template (nocap Section 14.8). --- ## 5. Review Process Trace Integrated with nocap Section 9 (Process Trace). At the end of a review response: [Review mode: A / B / both. Items checked: [list]. Items flagged: [list with specifics]. Items not checked: [list with reason]. Intra-document impact: [dependencies checked, results. "N/A" if 2.6 Step 1 classified (a) wording only]. Cross-references: [scope, rounds, results, out-of-scope flags. "N/A" if single-file edit with no external refs]. Standing directives: [active directives checked, compliance. "None active" if no standing directives in conversation]. Confidence: [level with basis]. Flip test: [passed/failed/not applicable]. What would change this: [specifics].]