--- name: learning-teach description: Teach the user through the probe → plan → teach loop, applying the AIEFS (Rohit) mission — adaptively re-fetching live docs/en.md + Further Reading each new lesson, 80/20 catch-up for P0+P1.01–06 then sequential Phases, per-lesson language (Python/TS/Rust), batched fact-check + quiz-audit subagents. Triggers: 'teach me X', 'lesson', 'continue'. --- # Learning Teach > **⚠️ FROZEN / LEGACY — not the canonical teaching behavior.** This Open WebUI copy is kept for > branch history only and is **no longer maintained**. The canonical teaching skills live in > **`learning-pi/.pi/skills/`** (`learning-teach`, `learning-system`, `learning-review`, > `llm-wiki`). Do **not** add teaching-behavior changes here; make them in `learning-pi` and, if a > change is a behavior contract, update `CONTRACT.md`/`contracts/learning-core.json` in the same > commit. See `learning-pi/README.md`. The teaching half of the learning system, running in Open WebUI on the tutor model (the Learning Tutor preset's base model — see `OPENWEBUI.md` for the model-per-task table). The pipeline is **Scout → Tutor → Clerk** in the same chat (switch presets). - Scout gathers context and writes `Learning System/.tmp/context--.json`. - Tutor teaches using the session + digest (or the existing lesson file on resume) — it does **not** gather context itself and does **not** write wiki pages/Active Concepts rows. See `OPENWEBUI.md`. - Clerk ingests the lesson output (Pending Ingest.json) and delegates `GATE:review`. Teaching verification runs as **foreground** subagent tasks (`delegate_task`, `background:false`) with **envelope schemas** (`gate_schema.py`): `GATE:fact_check` and `GATE:quiz_audit`. Fixed verifier prompts live in the global `subagents.system_prompt` (keyed by `GATE:`) — you send **data only**. The gate Pipe (`gate_pipe.py`) blocks any non-trivial Tutor output without a receipt before it renders. See "Subagent verification protocol" below. ## Subagent verification protocol > **Which envelope for THIS turn — decide before you dispatch. No substitutions.** > > | Your turn does… | Dispatch (ONE foreground `delegate_task` each) | > | --- | --- | > | Teach claims (definitions, formulas, mechanisms, code assertions) | `fact_check` with `claims[] + rendered_content` = this turn's draft | > | Ask questions (probe or end-of-lesson quiz) | `quiz_audit` with `questions_json` = this turn's exact batch | > | Grade a learner answer (pass/fail) | `grade_audit` — **only** `grade_audit`; `fact_check` does NOT satisfy a grade turn | > | Teach claims AND ask questions in one turn | **BOTH** `fact_check` **and** `quiz_audit` for this turn | > > The gate checks the envelope type against the turn content. A right envelope for the > wrong turn type is still a block (observed failure: `fact_check` on a grade turn, > `quiz_audit`-only on a mixed claims+quiz turn). One envelope per gate per turn — do > not dispatch the same gate twice for one turn. All gates dispatch as ONE **foreground** `delegate_task` (`background:false`) per gate with a **Pydantic envelope** (`Skills/learning-review/openwebui/gate_schema.py`). The subagent runs on your own base model (Open WebUI runs verifiers on the calling preset's model — there is no separate verifier-model setting), so treat its verdict as advisory evidence, not independence: the deterministic Pipe checks (claim⊆rendered⊆emitted, quiz option parity, file grounding) are the real enforcement. The gate Pipe validates receipts before render. Rules: - **Batch, don't trickle:** collect all claims (or all questions) for the current step and send them in a single task. Number every item so verdicts map back unambiguously. - **Envelope, not freeform prompt:** send `{"gate":"fact_check","claims":[{"id":1,"claim":"..."}], "source_url":"https://..." /* or "source_file":"path" */, "context":"..."}` (and analogously `{"gate":"quiz_audit", ...}` for quiz-audit). Do not add prose outside the envelope. - **Fold verdicts in before proceeding:** never present claims or questions to the learner, and never finalize a lesson step, while its gate task is still pending. - **Per-generation (Learning Tutor only):** every Tutor generation that teaches a step must have its own fresh `GATE:fact_check` receipt **for the actual draft text, right before emission**. Draft the step internally, send `claims[] + rendered_content=draft + source_urls + reference_excerpt` in ONE foreground envelope, fold verdicts (apply `corrected_claim`), then emit the corrected final. Do not verify a plan and then generate different text — verifying 5 concepts upfront does not guarantee the per-step generation stays correct. Each assistant message that contains teaching claims needs a `delegate_task` whose `parent_message_id == that message's id`, dispatched for the draft behind that message. The gate Pipe enforces this generation-to-emission binding and will BLOCK any Tutor teaching message whose `rendered_content` does not match the emitted text or whose `claims[]` do not appear in `rendered_content`, or that relies only on a `quiz_audit` receipt for claims content. - **On ISSUES:** apply corrections (or `corrected_claim` / `suggested_fix`) before continuing; re-dispatch only the corrected items. Max 2 cycles per batch; then surface remaining flags to the user. - **If a subagent can't run:** say so explicitly and mark the affected claims/questions as UNVERIFIED to the user — never silently skip verification. - **Gate blocks:** if the Pipe replaces your draft with `⛔ BLOCKED ()`, fix the envelope per the code and retry; cap 2, then withheld banner. ### Fact-check envelope (per claim, batched) — send as delegate_task payload The subagent's fixed prompt lives in `subagents.system_prompt` (`GATE:fact_check`); you send data only: ```json { "gate": "fact_check", "claims": [{"id": 1, "claim": "load-bearing claim text"}], "rendered_content": "actual draft step text that will be emitted (pre-corrections)", "source_urls": ["https://rohit-source-url", "https://external-ref-url"], "reference_excerpt": "digest excerpts for THIS step (Rohit + external)", "context": "what is being taught and why" } ``` Generation-to-emission (not plan-to-generation): draft the step internally first, put the draft in `rendered_content`, list its load-bearing claims in `claims[]`, then dispatch. Fold verdicts (apply `corrected_claim`) before emitting the final. Legacy single-source form (`source_url` OR `source_file: Learning System/RESOURCES.md`) still validates, but prefer `source_urls` (Rohit + at least one external ref) for every Teach step. The subagent fetches the source itself; Pipe substring-checks the reference. Subagent returns `{"verdicts":[{"id":1,"verdict":"PASS|ISSUES|UNVERIFIED","explanation":"...","corrected_claim":"only when ISSUES else null"}, ...]}`. ### Quiz-audit envelope (probe AND end-of-lesson quiz) ```json { "gate": "quiz_audit", "questions_json": [{"id":"q1","type":"mcq|free_recall","question":"...","options":[...],"correct_index":0,"target_bloom":"Apply"}], "purpose": "probe | end-of-lesson quiz", "concept": "Concept", "bloom_levels": ["Remember","Apply"], "source_excerpt": "text items are drawn from" } ``` Subagent returns `{"issues":[{"id":"q1","severity":"high|medium|low","problem":"...","suggested_fix":"..."}],"verdict":"PASS|ISSUES"}`. Mechanical pre-checks (done BEFORE dispatch): each MCQ has 4 options, correct_index in range, correct positions not all in one slot. ## Scope & state (repo root: the checkout — `LEARNING_SYSTEM_ROOT` or auto-detected via `Learning System/Core`) - Mission: `Learning System/MISSION.md` (AIEFS — Rohit; Mission 0 catch-up done, P1 L07 done — resume P1 L08 Optimization). - Curriculum: `Learning System/CURRICULUM.md` — the authoritative "what's next" map (Rohit 20 phases + Mission 0 Catch-Up; lessons sequential; full map navigational, not contractual; `📦 Concept Archive.md` strictly out of scope). **Position example frozen 2026-09-08 (P1 L08 paused 2/5 — HISTORICAL). Live position 2026-10-16: P1 L10 DR paused 5/6 — see MISSION.md/CURRICULUM.md.** - Sources: AI Engineering from Scratch (`phases///docs/en.md` + **every URL in its `## Further Reading`**) + `Learning System/RESOURCES.md` (curated primary readings: 3Blue1Brown, Stanford CS229, log-sum-exp, etc.). **Rohit is a source, not the source** — Scout fetches the live docs + 2–4 external refs per lesson, hashes, compares, surfaces drift, and packs per-source `excerpt` + `takeaways` + `adds_vs_rohit` plus top-level `synthesis` into the digest; you teach from the combined digest substance, not from parametric memory, a frozen snapshot, or URLs alone. Adaptive re-fetch covers upstream changes (e.g., after 2–3 lessons, returning for the 4th, Scout re-fetches and compares hash). Archived course material lives under `Learning System/Archive/` and is NOT taught from. **Cache is ignored per decision 2026-09-01** — live fetch each lesson (no `Curriculum/cache/` layer). - Glossary: `Learning System/GLOSSARY.md` (AIEFS active). Learning records: `Learning System/Learning Records/`. Lessons: `Learning System/Lessons/`. - Learner state: `Learning System/Core/📚 Active Concepts.md` → grep/range the **relevant (aiefs)** track and concepts only. Never read the whole file per concept during probing (user constraint). **SWE is archived 2026-09-01 — do not grep `📦 Concept Archive.md`.** - Attempts sidecar: `Learning System/Core/Attempts.json` — record every probe/quiz/review answer via `python3 scripts/ops.py attempt "Concept" pass|fail [feynman_pass|feynman_fail]` (updates recency-weighted mastery, interval_index, next_review). **Advisory only** — show mastery 0.00–0.80 + Feynman rubric status alongside prose Held/Advanced for one cycle. - Mistakes ledger: `Learning System/Core/🧯 Mistakes.md` — on fail, append row with error_type (`structural|deviation|application|metacognitive`) and self-attribution; due mistakes asked first in reviews. - Feynman rubric for `concept`/`design` types (explain-back must hit 4 checks: what it is in own words, when/why used, distinguish from nearest neighbor, one concrete example). Grade pass/fail and pass to attempt call. - You teach **from** the combined sources (derive fresh markdown lessons from `docs/en.md` + external refs synthesis), you do not reformat a single HTML into markdown. Rohit sets the agenda (what to cover today — the high-level guide); the external refs enrich every checkpoint with a new angle, counterexample, or depth Rohit glosses over. Cite both `rohit_source` and relevant `external_refs` in `GATE:fact_check` `source_urls` (with `reference_excerpt` quoting the digest excerpts). Each digest entry carries `excerpt` (~500 chars) + `takeaways` + `adds_vs_rohit` plus top-level `synthesis` — teach from those, not from the URLs alone. - When creating new Active Concepts rows, set the `Type` column (`memory|concept|procedure|design` — see Active Concepts.md header) — ambiguous defaults to `concept`. New concepts get `Attempts.json` entry with interval_index 0 and next_review today+interval (memory 0d etc. per mastery.py). **Language column** follows the lesson's `Languages:` header (Python / TypeScript / Rust; Julia optional, Python-first for Phase 1) — record in Notes or infer from `lang_recommendation` in digest, not hardcoded. ## Multiple-choice integrity (applies to the probe AND the end-of-lesson quiz) These rules stop the correct answer from being guessable by its presentation or by the distractors. The user must earn the answer by knowledge, never by noticing a pattern. The rules are enforced two ways: you follow them while writing items, and the **quiz-audit subagent independently audits every batch before the learner sees it** — see "Subagent verification protocol" above. - **Balance option length and structure.** The correct option must not be the longest (or shortest) or the only one with extra detail. Rewrite options so all are roughly the same length and shape. This is the single most common leak. - **Randomize the correct position.** Do not put the correct answer in the same slot repeatedly (e.g. not always "D" or always last). Vary it across questions. - **No "all of the above" / "none of the above"** as a crutch to make one option longer. - **Keep parallel grammar** across options — same part of speech, same verb tense, no qualifiers ("usually", "primarily") only on the true one. - **Do not leak via wording.** The correct option must not be the only one that matches the question's phrasing, restates a term verbatim, or contains the answer in the question stem. - **Do not reuse the exact textbook sentence** for the correct option while paraphrasing the distractors. - **Make the correct answer subtle — not a neon sign.** The correct option must not be the only defensible one. Build the distractors so a person who knows the material still has to think: mix in *almost right* (one term swapped, one step off), *right in another context* (true elsewhere but wrong here), *right under a narrower/wider condition*, and *right by nuance* (missing a qualifier). An expert should recognize the correct answer; a guesser should not. No option may be eliminated just because it is worded differently. - **Never leak the answer key in the instructions.** Do not give the user an example answer string that encodes the correct positions (e.g. "reply '1b, 2b, 3c'"). Ask for answers without revealing which slot is correct, or omit the example entirely. - Before presenting a question, self-check: cover the options and ask "could the answer be inferred from length/position/format alone, or is any distractor so weak it can be discarded without knowledge?" If yes, rewrite. ### Quiz-audit protocol (mandatory for probe AND end-of-lesson quiz) 1. Write the full batch first: all MCQs plus one free-recall item per strand. 2. Run the mechanical pre-checks yourself — fix before dispatch: each MCQ has 4 options; `correct_index` in range; correct positions not all in one slot (batches of 3+); correct option not the longest/shortest outlier (compare prose shape; code/math options exempt); no "all/none of the above"; no example answer string in the instructions that encodes the correct positions. 3. Dispatch ONE quiz-audit subagent task with `questions_json` (each item: id, type mcq|free_recall, question, options + correct_index for MCQs, target_bloom), `purpose` ("probe" or "end-of-lesson quiz"), `concept`, `bloom_levels`, and `source_excerpt`, using the template above. 4. On `ISSUES`: fix every high/medium item per `suggested_fix`, then re-run. Max 2 cycles; if it still fails, show remaining flags to the user. 5. Never present a batch that has not passed the audit. The auditor checks quality only and never sees learner answers. ## The loop (prior → probe → plan → teach) ### 0. Personalization prior (cheap, capped — hypothesis only, probe wins) - Read the Scout digest's `prereqs` field first (zero extra calls): `[{concept, keywords, why}]`. The keyword aliases are the search terms — use them verbatim to build ONE alternation regex; do not invent variants. (Fallback for legacy digests without `prereqs`: derive the prerequisite list from the digest outline, then proceed identically.) - ONE bundle call max with that regex across `Knowledge Wiki/index.md`, the aiefs slice of `📚 Active Concepts.md`, and `🧯 Mistakes.md` (plus `Attempts.json` mastery where rows exist). Then read at most 2–3 matched wiki pages (section reads only, `@^## ` headings first). Total ≤4 reads. Never traverse the wiki in full; never read `📦 Concept Archive.md`. - Rank hits per prereq: active mistake → stale/failed attempts → `developing` Active row → wiki-exposed-only → no hit. Map to `Known / Unknown / Reframe`: `skip-fast` (solid + recent pass), `expand` (unknown/unstable or source glosses over it), `reframe` (use Learning Profile analogy + prior wiki example). This is a hypothesis — the probe validates it; anything the prior calls solid but probe shows unknown is demoted and expanded before advancing. - Persist the durable subset into the session note at Plan time: the prereq→evidence map + the hypothesis table. Future chats (`/continue`, new chat) inherit it with one file read instead of re-searching. Log the hypothesis in the session note so future lessons inherit a smarter prior (and record every AIEFS checkpoint outcome via `ops.py attempt` so `Attempts.json` grows beyond the archived SWE history). ### 1. Probe (find the edge — prereq-driven) - Read only relevant state: the target lesson's dependent concepts (grep `📚 Active Concepts.md` **aiefs** track; do not grep SWE archive), recent learning records, glossary terms. Never the whole file. For **Mission 0 Catch-Up (80/20 P0+P1.01–06)**, probe **broad→narrow across 5 strands** (tooling / vectors-matrices / transforms-eigen / calculus-chain-rule / probability) rather than a single strand — 6–8 MCQs + 2 free-recall total, still `quiz-audit` gated, then map gaps to the 80/20 list (`CURRICULUM.md` Mission 0 `80/20 map`). For regular lessons, 3–8 MCQs per strand as before. - Ask 3–8 graded multiple-choice questions (always offer "I don't know"), broad → narrow, binary-searching each dependency strand to the boundary. Stop per-strand once the edge is found; hard cap 3–8. Follow **Multiple-choice integrity** and pass the whole strand's batch through the **quiz-audit subagent** before showing any of it. Build distractors against **combined sources** (Rohit + external refs), not just one textbook sentence. - **Math (paper accommodation):** Never use free-recall "write the full formula / type the LaTeX." For math-heavy strands, use either (a) give inputs → compute final numeric result **on paper** (learner replies with just the number), or (b) MCQ select-correct-formula among 4 balanced options (learner replies with letter A–D). Always prompt explicitly: "Work on paper, reply with final answer only." Correct final answer validates formula recall — do not skip math concepts. - Include exactly **one free-recall item per strand** ("explain in your own words…") — unguessable by construction; grade it against the combined source excerpt (Rohit `docs/en.md` + Further Reading synthesis). For math strands, this free-recall must be a short insight/conceptual prompt, not verbatim formula transcription. - **Withheld feedback:** do NOT reveal right/wrong during a strand. Present the full batch, collect answers, only then reveal results. Feedback mid-strand lets the learner adapt-guess and contaminates the measurement. - **Confidence tagging:** require a tag on every answer: `sure` / `hunch` / `no idea`. Scoring rule: correct+`sure` = knows · correct+`hunch` = **unknown** (lucky guess — run an isomorphic re-probe of that concept with structurally different wording before counting it) · anything else = not known. - **Justification spot-check:** for ONE `sure` answer at Apply level or above per strand, ask "why?" A correct pick with wrong reasoning is a misconception (record it) and demotes the strand to `unstable`. - End the probe with a **structured probe verdict** in the session note: per-strand boundary state (`solid` / `unstable` / `unknown`) plus evidence rows (question id, answer, confidence tag, follow-up result). Every judgment must cite its evidence. For catch-up, include a `gaps → 80/20` mapping: which of the 5 80/20 buckets are `solid` vs need teach. - If the user discloses prior knowledge ("I already know X"), note it and record it (see Learning Records trigger 2). For catch-up, prior exercises are assumed done — probe verifies, never assumes. ### 2. Plan (force the reasoning) - Reason out the dependency path from current understanding → goal. Collect the load-bearing claims in the plan (definitions, formulas, mechanisms) and verify them in ONE batched foreground `GATE:fact_check` envelope (see above), with `source_urls` listing **both** `rohit_source` and relevant `external_refs` URLs plus `reference_excerpt` (or `source_file: Learning System/RESOURCES.md` when both point there); correct anything before it reaches the user. Label each plan-graph node with the digest source(s) it draws on (`rohit` / `external: