Accuracy Oracle

Use after cpu-model-wiring to prove the CPU-enabled model is numerically correct. ORDERING (critical): on UNOPTIMIZED code do NOT gate correctness on full-model generation or a task harness (too slow — watchdog); the feasible correctness gate is the per-layer fingerprint parity vs the GPU/HF baseline on ONE short prefill. CONFIG LADDER (do not mistake the proxy for the gate): a TINY arch-faithful config is the BRING-UP proxy (op-TYPE coverage) ONLY — the AUTHORITATIVE correctness gate is FULL-MODEL DUMMY parity (ALL layers at REAL shapes, deterministic-dummy weights so NO real-weight load, MULTI-GPU reference when the full model exceeds one GPU), established BEFORE optimizing and reused to regression-guard EACH optimization; REAL weights only for the dtype-bridge + task finale. Optimize AFTER correctness; regression-guard with the SAME fast parity; full generation + task accuracy come LAST once fast. LAYERED checks so a failure localizes: (0) a cheap real-prompt COHERENCE smoke test first ('it ran' on random/dummy inputs is NOT 'it's correct'; catches gross zeroed/no-op/routing/layout bugs and silently-stubbed forward paths), (0.5) a per-kernel LOW-BIT PARITY gate for every dtype bridge (fp4/mxfp4/fp8/int4 weights on a bf16/fp8 kernel) — kernel output vs an independent torch dequant oracle from the same packed bytes, because 'a bf16/fp8 kernel exists' is coverage, not correctness, (1) per-layer / logits parity vs a trusted reference (HF or the GPU SGLang path) under tolerance, and (2) end-to-end task accuracy via the in-repo harnesses (gsm8k, mmlu, hellaswag) within N points. Includes a correct-by-construction ISOLATION technique to bisect MoE vs shared-path bugs. Blocks enablement on any regression; a per-layer diff pinpoints the offending op for the wiring step to fix.

mkdir -p ~/.claude/skills/accuracy-oracle
curl -L https://claudskills.com/skills/accuracy-oracle/SKILL.md \
  -o ~/.claude/skills/accuracy-oracle/SKILL.md

@misc{accuracy-oracle-2026,
  author    = {bharat-kaul},
  title     = {Accuracy Oracle [Claude Code skill]},
  year      = {2026},
  publisher = {ClaudSkills},
  url       = {https://claudskills.com/skills/accuracy-oracle/}
}