--- license: Apache-2.0 name: causal-reasoning-klein description: Expert causal reasoning and mental model construction for diagnosis and prediction in complex domains category: Cognitive Science & Decision Making tags: - causal-reasoning - mental-models - expertise - diagnosis - ndm --- # SKILL.md — Causal Reasoning in the Wild (Klein & Hoffman) license: Apache-2.0 --- ## Metadata ```yaml name: causal-reasoning-klein description: > Skill for naturalistic causal reasoning — diagnosing why things happened, attributing causes under uncertainty, building explanatory frames, and deciding when to stop analyzing and start acting. Based on Klein & Hoffman's empirical taxonomy of how experts actually reason about causality in messy, real-world domains. activation_triggers: - "why did this happen" - "what caused this failure" - "root cause analysis" - "debugging / post-mortem / incident review" - "causal attribution" - "explain this outcome" - "what went wrong" - "how do we prevent this" - "diagnosis / diagnosing" - "what's the real reason" - "multiple possible causes" - "I can't find the cause" version: 1.0 source: > Klein, G. & Hoffman, R. — "Causal Reasoning: Initial Report of a Naturalistic Study of Causal Inferences" ``` --- ## When to Use This Skill Load this skill when: - **Diagnosing failures** — a system, process, decision, or outcome went wrong and you need to explain why - **Causal attribution is contested** — multiple competing explanations exist and you need a framework for evaluating them - **The "obvious" cause feels too simple** — someone is reaching for a single root cause in a situation that is clearly complex - **You're designing diagnostic agents** — building an agent or orchestration system that must investigate and explain failures - **Closure is hard** — analysis keeps expanding and you need principled criteria for when to stop - **Explanation type matters** — you need to choose *how* to explain something, not just *what* to explain - **Post-mortems or retrospectives** — organizational, technical, or strategic failure reviews - **Experts disagree about cause** — different framings produce different cause-sets and you need to understand why Do **not** primarily rely on this skill for: - Pure statistical causal inference (use Judea Pearl / do-calculus frameworks instead) - Formal fault tree analysis with known component failure rates - Situations where full information is available and the cause is unambiguous --- ## Core Mental Models ### 1. Five Explanation Types — Form Shapes What Gets Found Real-world causal reasoning uses five structurally different explanation forms. The form chosen is not neutral — it determines what counts as a "relevant cause" to search for. | Type | What it looks like | Domain bias | |---|---|---| | **Event** | "X happened, which caused Y" — counterfactual, reversible | Sports, military decisions | | **Abstraction** | "This is an instance of pattern P" — category-based | Law, medicine, science | | **Condition** | "The environment made this inevitable" — structural | Policy, safety analysis | | **List** | "Factors A, B, C contributed" — enumeration without mechanism | Journalism, early diagnosis | | **Story** | "A complex interaction of X, Y, Z unfolded over time" — mechanistic narrative | Economics, complex failures | **Why it matters**: An agent using only event-type reasoning in a systemic failure will miss structural conditions. An agent defaulting to list-type reasoning will enumerate causes without modeling how they interact. --- ### 2. The Reciprocal Model — Frames and Causes Co-Construct Each Other The search for causes and the construction of an explanatory frame happen **simultaneously**, not sequentially. The frame determines what counts as a relevant cause; the causes found reshape the frame. ``` [Candidate Causes] ←→ [Explanatory Frame] ↑ ↑ └─── each reshapes the other in real time ───┘ ``` **Critical implication**: If you start with the wrong frame, you will systematically miss causes that don't fit it — not because you're careless, but because the frame defines the search space. Changing the frame isn't "trying harder"; it's accessing a different set of candidate causes entirely. --- ### 3. Three Attribution Criteria — Propensity, Reversibility, Covariation These are the workhorses of causal judgment. They operate independently and can produce conflicting verdicts: - **Propensity**: Could this plausibly lead to that? (mechanism, plausibility) - **Reversibility (Mutability)**: Would removing this cause eliminate the effect? (counterfactual test) - **Covariation**: Do cause and effect travel together statistically? (correlation, pattern) **The watch-factory problem**: Context can flip all three verdicts simultaneously. A cause that passes all three criteria in isolation may fail all three in a different systemic context. Always apply all three; always ask whether context reverses the verdict. --- ### 4. The Reductive Tendency — Necessary Trap Humans systematically compress dynamic, nonlinear, simultaneous causality into linear, sequential chains. This is both: - **Cognitively necessary** — you cannot act on a fully simultaneous multi-causal model - **Epistemically dangerous** — the compression creates fiction that becomes invisible The correct response is not to eliminate reduction but to **know which register you're in**: - **Diagnostic mode**: resist reduction, hold complexity, use story-type explanation - **Action mode**: embrace reduction, identify the actionable cause, achieve closure Agents that operate only in one register will fail in the other. --- ### 5. Indeterminate Causation — The Norm, Not the Exception In organizational, political, military, and economic domains, **there is no single true cause**. Multiple interacting partial causes are the rule. The quest for *the* root cause: - Is an epistemological error - Produces oversimplification - Generates interventions that fix the named cause while leaving structural vulnerabilities intact The correct output of causal reasoning in complex domains is a **partial-cause set** with explicit representation of uncertainty, not a single root cause with false precision. --- ## Decision Frameworks ### Which Explanation Type Should I Use? ``` IF the domain is competitive/event-driven AND reversibility is high → USE Event explanation (counterfactual framing) IF you recognize a known failure pattern from prior cases → USE Abstraction explanation (pattern matching) IF the failure was structurally overdetermined (would have happened anyway) → USE Condition explanation (enabling conditions framing) IF causes are numerous, interactive, and mechanism is unclear → USE Story explanation (rich causal narrative) IF you are at an early diagnostic stage with incomplete information → USE List explanation (enumeration) AS A TEMPORARY SCAFFOLD → Plan to upgrade to Story or Condition as information arrives ``` --- ### Am I Using the Right Frame? ``` IF you've been searching for causes and keep finding nothing → The frame is probably wrong, not the search → Load: references/reciprocal-framing-and-causal-search.md → Deliberately adopt an alternative frame and restart IF experts with different backgrounds disagree about cause → They are likely using different frames, not different evidence → Map which explanation type each expert is using → Look for the cause-set each frame is structurally incapable of seeing IF the causal story feels too clean or too linear → You have over-reduced → Load: references/reductive-tendency-and-when-to-resist-it.md → Ask: what simultaneity and interaction have I compressed away? ``` --- ### Evaluating a Candidate Cause ``` FOR each candidate cause, apply all three criteria: Propensity test: "Is there a plausible mechanism by which this could produce the effect?" IF no → weak candidate, but don't eliminate (unknown mechanisms exist) IF yes → proceed to reversibility Reversibility test: "If we remove this cause, does the effect go away?" IF no → this is an enabling condition, not a trigger cause IF yes → strong candidate IF unclear → the system may be overdetermined (effect has multiple sufficient causes) Covariation test: "Does this cause co-occur with this effect across cases?" IF no → likely spurious or context-specific IF yes → corroborating evidence (not proof) Context reversal check: "Does the systemic context flip any of these verdicts?" Load: references/three-causal-criteria-for-agent-diagnosis.md ``` --- ### When to Stop Analyzing (Closure Decision) ``` IF stakes are low AND time pressure is high → Close early, use action-cause (the single most actionable cause) → Document that full causal story is unresolved IF stakes are high AND time is available → Continue until Story-type explanation is coherent → Require that enabling conditions are identified, not just trigger causes IF analysis keeps expanding without convergence → Check: are you holding multiple frames simultaneously? → Apply proportionality: closure depth must match decision stakes → Load: references/closure-decisions-and-the-action-understanding-tradeoff.md IF you must act before understanding is complete → Design the action as a probe: what will this action reveal about cause? → Action-as-investigation is how experts close the analysis/action gap ``` --- ### Designing a Diagnostic Agent or Orchestration ``` IF building parallel investigation agents → Deliberately assign DIFFERENT initial frames to each agent → Frame diversity is not inefficiency — it is the mechanism for finding causes that a single frame would miss IF deciding when to commit an orchestrator to an output → Apply the closure criteria above → Require dual output: action-cause (for intervention) + causal story (for structure) IF decomposing a complex failure into sub-investigations → Map to explanation type first → Story-type failures → decompose into parallel causal streams modeled jointly → Condition-type failures → decompose around structural features → NEVER decompose simultaneous processes as sequential (reductive trap) → Load: references/causal-stories-as-system-models.md ``` --- ## Reference Table Load these files **on demand** when the specific sub-problem arises. Do not load all at once. | File | Load When… | |---|---| | `references/five-forms-of-causal-explanation.md` | You need to identify which explanation type is being used, or choose the right type for a diagnosis task. Core taxonomy reference. | | `references/reciprocal-framing-and-causal-search.md` | The search for causes is stalling or producing thin results. Reframing is needed. Frame/cause reciprocity must be explained or operationalized. | | `references/three-causal-criteria-for-agent-diagnosis.md` | Evaluating specific candidate causes. Propensity, reversibility, or covariation criteria need detailed application. Context-reversal cases. | | `references/reductive-tendency-and-when-to-resist-it.md` | The causal account feels too linear or simple. A system failure involved simultaneity or interaction that may have been compressed away. | | `references/indeterminate-causation-and-partial-explanations.md` | Multiple causes are present and cannot be cleanly ranked. "Root cause" pressure is distorting the analysis. Need to represent partial causation. | | `references/domain-expertise-and-causal-frame-selection.md` | An expert's causal judgment seems opaque or unjustified. Modeling why experts pick different frames than novices in the same situation. | | `references/causal-stories-as-system-models.md` | The failure is complex, systemic, or organizational. A narrative causal model is needed. Designing an agent to reason about multi-stream interactions. | | `references/closure-decisions-and-the-action-understanding-tradeoff.md` | Analysis must end and action must begin. Deciding proportional closure depth. Designing probe-actions. Orchestration commit decisions. | --- ## Anti-Patterns These are the mistakes Klein & Hoffman's research most directly warns against: **1. Root Cause Fundamentalism** Treating "find the single root cause" as the goal of causal analysis. In complex domains, this produces a named cause that absorbs blame while leaving structural vulnerabilities intact. The right output is a partial-cause set with interaction structure. **2. Frame Blindness** Searching harder within a frame that isn't working, rather than changing the frame. When a search stalls, the failure is usually in the frame, not in the effort. Frame-switching is a first-class diagnostic move, not an admission of failure. **3. Sequential Compression of Simultaneous Causation** Describing processes that happen in parallel and interact as if they happened in sequence. This is the reductive tendency at its most dangerous — the narrative becomes plausible and the simultaneity that produced the failure becomes invisible. **4. Explanation Type Mismatch** Using event-type explanation for condition-caused failures (e.g., "if only the pilot had done X differently" when the system was structurally unsafe). Or using list-type explanation when a story-type account is needed to model interaction effects. **5. Premature Closure Under Complexity** Achieving closure at the action-cause level and treating it as complete understanding. Action-causes (the trigger to intervene on) are not the same as causal stories (the structural understanding needed to prevent recurrence). Both are needed; treating one as sufficient produces intervention that fixes nothing durable. **6. Commentator Reasoning Applied to Participant Situations** Analyzing expert decisions as if the expert had full information, no time pressure, and was not causally entangled in the situation. Commentator reasoning (post-hoc, reflective, unconstrained) is structurally different from participant reasoning (real-time, action-pressured, self-entangled). Applying the former standard to the latter situation produces unfair and useless diagnosis. **7. Covariation Treated as Sufficient for Causation** Statistical co-occurrence confirms a causal hypothesis but does not establish it. Propensity (mechanistic plausibility) and reversibility (counterfactual test) must also be applied. Covariation-only reasoning produces interventions that address correlates rather than causes. --- ## Shibboleths *How to tell if someone has actually internalized this book vs. skimmed a summary:* **They will spontaneously ask "which explanation type is this?"** Someone who has read only the summary will say "there are five types of causal explanation." Someone who has internalized it will, when encountering a causal claim, immediately ask which type is being used — and notice when the type is wrong for the situation. **They treat frame-switching as a first move, not a last resort.** Naive readers conclude "try harder if the search fails." Internalized readers ask "is the frame the problem?" early and treat deliberate reframing as a normal diagnostic tool, not a sign of failure. **They distinguish action-causes from causal stories without prompting.** They will say something like: "We know what to fix right now, but we don't yet understand why the system was vulnerable — those are two different questions and we need both answers." **They are suspicious of single-cause explanations in complex domains.** Not because they oppose simplicity, but because they have a specific concept — indeterminate causation — that makes single-cause accounts in organizational or systemic contexts feel epistemologically naive, not just incomplete. **They understand why experts and novices reach for different frames.** They won't say "the expert is biased" or "the novice is naive." They will explain, mechanistically, which causal features each frame makes visible and which it occludes — and why an expert's frame reflects accumulated knowledge about which features are typically diagnostic in that domain. **They use "propensity," "reversibility," and "covariation" as working vocabulary.** And more importantly, they apply them independently — they know that a cause can satisfy one criterion and fail another, and they know that context can reverse verdicts they thought were settled. **They think about closure as a design parameter, not a discovery.** They don't ask "when will we know enough?" They ask "given the stakes, how deep should closure be, and what is the minimum causal story sufficient for proportionate action?" --- *This skill indexes eight reference documents. Load them on demand as specific sub-problems arise. The skill itself is sufficient for most high-level causal reasoning tasks; the reference files provide working depth for complex cases.*