---
name: sensitivity-analysis
description: Use this skill after multiverse-analysis and before self-review. Identifies which parameters and assumptions, if changed within plausible ranges, would move the answer enough to change the decision. Differs from multiverse — multiverse varies analytical *paths*; sensitivity varies *parameters within a fixed path*. Trigger when results are about to be summarized for decision-making, especially when assumptions are non-obvious (discount rates, lookback windows, attribution windows, threshold cutoffs, smoothing parameters).
---

# Sensitivity Analysis

A finding's robustness has two flavors. Multiverse asks: does the finding hold under different *paths*? Sensitivity asks: does the finding hold under different *parameter values within the chosen path*?

## What this skill catches

- **Headline numbers that ride on one parameter setting.** A 12% lift that depends entirely on a 90-day lookback window — and would have been 4% with a 60-day window or 18% with a 180-day window — is decision-sensitive, but gets reported as if the parameter were given.
- **Defaults that snuck in.** Discount rates, smoothing windows, attribution windows often arrive as defaults from a tool or template. They get used without anyone defending them, and small shifts in the default would change the conclusion.
- **Threshold-cutoff knife-edges.** A "% above 10%" threshold and a "% above 5%" threshold may produce dramatically different counts. If the cutoff is conventional rather than principled, a sensitivity check exposes the dependence.
- **The illusion of precision.** Reporting a finding to three decimal places when reasonable parameter shifts move it by a full point.

## What this skill enforces

For each parameter or assumption in the analysis:

1. **Identify the parameter** — discount rate, lookback window length, attribution window, smoothing parameter, threshold cutoff, etc.
2. **Define a plausible range** — what values would a reasonable analyst defend?
3. **Re-run the analysis at the endpoints** of that range (and ideally one mid-point).
4. **Report how much the answer moves** — and whether enough to change the decision.

A finding is **decision-robust** if the conclusion stands across the plausible parameter range. **Decision-sensitive** if reasonable parameter shifts flip the conclusion. Decision-sensitive findings need either tighter parameter justification or a wider hedge in the writeup.

## Multiverse vs. sensitivity

The two are easy to conflate. The shorthand:

- **Multiverse** — "if I had picked a different path here (different segmentation, different outlier rule), would the answer change?"
- **Sensitivity** — "if I had used a different value for this parameter (lookback length, discount rate, threshold), would the answer change?"

Multiverse varies analytical *choices*; sensitivity varies *values* inside choices already made. You need both.

## Common sensitivity targets

- Lookback window length (30 / 60 / 90 / 180 day)
- Attribution window
- Discount rate (for any LTV-style or NPV-style calculation)
- Outlier threshold (top/bottom 1%, 5%)
- Smoothing parameter (rolling-window length)
- Statistical thresholds (significance level, effect size)
- Imputation method for missing data
- Cohort-membership cutoff dates

## Logging

Log this skill's activation:

```bash
python3 .analytics/db.py log-skill \
    --skill sensitivity-analysis \
    --question-slug <slug> \
    --completed
```

Record each parameter swept as a path decision, marked as a multiverse run since it's an alternative-path comparison:

```bash
python3 .analytics/db.py log-path \
    --question-slug <slug> \
    --decision-type "sensitivity" \
    --chosen "<parameter = value tested>" \
    --rationale "sensitivity sweep; result moved from X to Y vs. headline" \
    --multiverse-run \
    --fragility-note "<decision-robust | decision-sensitive>"
```

## Boundary cases

**"There are too many parameters to sweep all of them."** Sweep the ones most likely to swing the result. Defaults that arrived from a template, parameters chosen by convention, and any value the analyst couldn't defend on principle in thirty seconds are the high-priority sweeps.

**"The answer doesn't move much across the range."** Excellent — that's a robust finding. Note the range and the small movement in the writeup. Robustness across reasonable parameter ranges is the whole point of having done the sweep.

**"The answer moves enough to change the decision."** That's the most important sensitivity finding to surface. The headline is no longer the single number; the headline becomes "the conclusion depends on assumption X — under defensible alternatives the answer ranges from A to B." Don't bury this.

## Handoff

After sensitivity is documented, hand off to `analytical-self-review/SKILL.md`.