--- name: scorpio-lessons description: Curate high-signal lessons from postmortems, QA reports, and retrospectives into concise, non-redundant entries with root causes and mitigations. Use to distill actionable knowledge from completed work and feed it back into the pipeline. --- # Lessons Curator A methodology for extracting, deduplicating, and curating actionable lessons from pipeline runs. This skill transforms raw failure data and QA findings into concise entries that improve future runs. ## When to Use - After a scorpio-postmortem identifies recurring failure patterns - After a scorpio-retro surfaces process improvements - When `lessons-learned.md` has grown noisy and needs pruning - When starting a new project and want to seed it with lessons from past work - Periodically (after every 3-5 runs) to keep the lessons file current ## Prerequisites One or more of these sources: | Source | Path | Producer | |--------|------|----------| | scorpio-postmortem reports | `.scorpio/runs//postmortem.json` | scorpio-postmortem skill or `postmortem-analyzer.sh` | | Lessons file | `.scorpio/lessons-learned.md` | `extract_lessons()` in scorpio.sh | | QA reports | `{docs.qaReports}/*.md` | scorpio-qa-final, scorpio-qa | | scorpio-retro reports | `docs/retros/RETRO-*.md` | scorpio-retro skill | | Triage reports | `{docs.qaReports}/TRIAGE-*.md` | scorpio-qa-rework | ## Curation Phases ``` Phase 1: Collect → Gather raw lessons from all sources Phase 2: Normalize → Standardize each entry into a common format Phase 3: Deduplicate → Merge entries that describe the same underlying issue Phase 4: Classify → Tag each lesson by category and applicability Phase 5: Prioritize → Rank by recurrence frequency and impact Phase 6: Write → Produce the curated lessons artifact ``` ## Phase 1: Collect Scan all available sources for lesson-worthy material. ### From Postmortems Extract from `postmortem.json`: - Each finding's `title`, `root_cause_class`, and `remediation.action` - Recommendations with `type` and `description` ### From Existing Lessons File Read `.scorpio/lessons-learned.md`: - Each existing entry (may be noisy or duplicate) - Failure patterns and decision points already recorded ### From QA Reports Scan `{docs.qaReports}/`: - Recurring finding categories (e.g., "missing error handling" across 3 tasks) - Patterns that caused rework ### From Retrospectives Read `docs/retros/RETRO-*.md`: - Insights with evidence counts - Recommended convention updates ### Collection Template ```markdown ## Raw Lessons Collected ### From scorpio-postmortem (run: ) 1. [Finding title] — [root cause class] — [remediation] 2. ... ### From QA reports 1. [Pattern] — [N occurrences] — [affected tasks] 2. ... ### From scorpio-retro () 1. [Insight title] — [category] — [recommendation] 2. ... ### From existing lessons-learned.md 1. [Entry text] 2. ... ``` ## Phase 2: Normalize Convert each raw entry into a standard lesson format: ```markdown ### [Lesson Title] **Category**: config | code | process | tooling | testing | documentation **Cause**: [One sentence: what went wrong or what was missing] **Mitigation**: [One sentence: what to do differently] **Evidence**: [Source reference — e.g., "F-003 from run 20260226-063112", "QA finding in CORE-471"] **Recurrence**: [N times across M runs/tasks] ``` ### Normalization Rules - **Title**: Short, imperative form. "Validate timeout config before run" not "The timeout was misconfigured" - **Cause**: Root cause, not symptom. "Missing null check" not "Session crashed" - **Mitigation**: Specific and actionable. "Add idle_timeout_sec to config schema validation" not "Be more careful with config" - **Evidence**: Always link to the source. No lesson without evidence. - **Recurrence**: Count how many times this pattern appeared. Higher = more important. ## Phase 3: Deduplicate Merge entries that describe the same underlying issue. ### Deduplication Criteria Two lessons are duplicates if they share: 1. The same root cause (even if symptoms differ), OR 2. The same mitigation (even if causes differ slightly) ### Merge Strategy When merging duplicates: - Keep the clearest title - Combine evidence from all instances - Sum recurrence counts - Take the most specific mitigation ```markdown ### Before (duplicates) - "Session timed out due to idle detection" (run A) - "False timeout killed active session" (run B) - "Timeout fired during long AI response" (run C) ### After (merged) **Title**: Tune idle timeout to account for long AI response latency **Cause**: Default idle timeout (120s) is shorter than some AI model response times **Mitigation**: Set idle_timeout_sec >= 180s for sessions using slow models; use composite liveness when available **Evidence**: Runs A, B, C — 3 occurrences across 3 runs **Recurrence**: 3 ``` ## Phase 4: Classify Tag each curated lesson by category and applicability scope. ### Categories | Category | Description | Applies to | |----------|-------------|-----------| | **config** | Pipeline or project configuration | `pipeline.config.json`, env vars | | **code** | Implementation patterns or bugs | Task code, scorpio.sh | | **process** | Workflow or methodology | Skill usage, session management | | **tooling** | Tool configuration or limitations | CLI tools, AI models, MCP | | **testing** | Test coverage or strategy | Test files, QA process | | **documentation** | Missing or misleading docs | CLAUDE.md, skill docs, contracts | ### Applicability Scope | Scope | Meaning | |-------|---------| | **universal** | Applies to all projects using scorpio | | **project-specific** | Only relevant to the current project | | **skill-specific** | Only relevant when using a particular skill | ## Phase 5: Prioritize Rank lessons by impact and recurrence. ### Priority Formula ``` Priority = Recurrence × Impact Weight Impact weights: critical (blocked run) = 4 high (failed session) = 3 medium (degraded quality) = 2 low (cosmetic/informational) = 1 ``` ### Priority Tiers | Tier | Criteria | Action | |------|----------|--------| | **Must-know** | Priority >= 6 or any critical | Add to CLAUDE.md or config defaults | | **Should-know** | Priority 3-5 | Add to lessons-learned.md | | **Nice-to-know** | Priority 1-2 | Keep in lessons-learned.md, review periodically | ## Phase 6: Write Produce the curated output artifact. ### Output: Curated Lessons File Write to `.scorpio/lessons-learned.md` (replace, don't append): ```markdown # Pipeline Lessons Learned > Curated from postmortems, QA reports, and retrospectives. > Last curated: . Sources: postmortems, QA reports, retrospectives. ## Must-Know ### [Lesson Title] **Category**: config | **Cause**: [cause] | **Mitigation**: [mitigation] Evidence: [references] | Recurrence: [N] ### [Lesson Title] ... ## Should-Know ### [Lesson Title] ... ## Nice-to-Know ### [Lesson Title] ... --- *Curated by scorpio-lessons skill. Edit freely — next curation merges your changes.* ``` ### Output: Convention Recommendations For must-know lessons, also output recommended additions to CLAUDE.md or project config: ```markdown ## Recommended Convention Updates | # | Lesson | Target | Recommended Change | |---|--------|--------|--------------------| | 1 | [Title] | CLAUDE.md | Add: "[convention text]" | | 2 | [Title] | pipeline.config.json | Set `idle_timeout_sec` default to 180 | | 3 | [Title] | skill/scorpio-execute | Add "[check]" to self-QA prompt | ``` These recommendations are suggestions — the operator decides what to adopt. ## Integration with extract_lessons() scorpio.sh's `extract_lessons()` function (line ~5120) appends raw entries to `.scorpio/lessons-learned.md` after every run. This skill complements it: | Aspect | `extract_lessons()` | scorpio-lessons skill | |--------|---------------------|----------------------| | Trigger | Automatic, every run | Manual or periodic | | Scope | Single run's failures | All available sources | | Quality | Raw, may be noisy | Deduplicated, prioritized | | Format | Append-only markdown | Structured, replaces file | | Evidence | Failure reason only | Full evidence chain | **Recommended workflow**: Let `extract_lessons()` accumulate raw entries, then periodically invoke scorpio-lessons to clean up and prioritize. ## What NOT To Do - Do not invent lessons without evidence from actual failures or findings - Do not keep stale lessons that no longer apply (e.g., fixed bugs, deprecated features) - Do not make lessons vague — "be careful with timeouts" is not actionable - Do not duplicate content already in CLAUDE.md — reference it instead - Do not curate mid-run — wait until the run completes - Do not conflate with **scorpio-retro** — scorpio-retro analyzes a development cycle's process; scorpio-lessons distills cross-run knowledge into reusable entries ## Skill Chain ``` [multiple runs complete] │ ▼ extract_lessons() ──→ raw lessons-learned.md (automatic, per-run) │ ▼ scorpio-postmortem skill ──→ postmortem.json (deep single-run analysis) │ ▼ scorpio-retro ──→ RETRO-*.md (cycle-level patterns) │ ▼ scorpio-lessons ──→ curated lessons-learned.md + convention recommendations (this skill) │ ├──→ CLAUDE.md updates (operator applies) ├──→ pipeline.config.json defaults (operator applies) └──→ skill file updates (operator applies) ``` ## Differentiation from Other Skills | Skill | Focus | Scope | Output | |-------|-------|-------|--------| | **scorpio-lessons** | What should we remember? | Cross-run aggregation | Curated lessons file | | **scorpio-postmortem** | Why did this run fail? | Single run | Diagnosis + fix tasks | | **scorpio-retro** | What patterns emerged? | Development cycle | Process improvements | | **extract_lessons()** | What failed this run? | Single run (automatic) | Raw lesson entries |