Use when evaluating the quality of a GitHub README, judging whether it reads as human-written or AI-generated, and producing concrete fixes.
Eval rubric refinement engine · Solo Founder edition harness-optimizer. Scans all historical eval results (idea-eval / skill eval / agent eval), identifies dimensions where…
Use when user says regression test, eval check, eval regression, test before commit, compare versions, check regressions, run evals, benchmark skill, or /eval-regression.
Generate an aggregate agent quality report from evaluation results, showing scores, regressions, and recommendations
Analyzes AI agent evaluation results - primarily from Copilot Studio (the worked example here, via its CSV export) but also from custom harnesses or any evaluator that produces…
Analyze CodeFlowMu risks and gaps from EVAL findings. Use when EVAL needs to classify expected vs actual behavior, risk type, severity recommendation, evidence, and owner.
Design a scoring rubric and LLM-as-judge prompt to evaluate the quality of an AI feature's output. Use when asked to create an eval rubric, define quality dimensions, build an LLM…
Use when scoring AI legal output on formatting and structural quality — whether the output is organized appropriately for its type, uses correct heading hierarchy, presents tables…
Evaluate any output file against a structured evals.yaml assertions file and produce a score report with per-assertion pass/fail results.
Run eval scenarios to benchmark Mycelium effectiveness. Execute tasks using reflexion loop, validate against success criteria, record metrics.
Judge a single AI News Briefing card against the 5-axis quality rubric (factuality, novelty, source diversity, signal density, coherence) and persist the score.
对一次已完成的 sid-code 任务会话做三段式评估(结果 + LLM 过程 + harness),通过轨迹找出 sid-code 自身的 bug 和可优化地方,先产出带 file:line 证据的评估结果,经人类二次确认后再派生 todo-fix 清单,目的是驱动 sid-code 后续优化。执行者可以是任意 agent(Claude Code /…
Measures whether an agent skill fires when it should and whether its output beats no skill at all, then fixes what the measurement exposes.
Audit all skills in the current project for frontmatter completeness, effort level appropriateness, allowed-tools scoping, and content quality.
Evaluate the speckit-extension end-to-end against a real spec — verify lifecycle hooks fired, .spec-context.json captured each step into canonical history[], timing is real (not…
Use when the user wants to batch evaluate multiple content pieces — run the eval pipeline across an entire content library, campaign assets, or set of deliverables to get a…
Scaffold an out-of-tree eval/ package that imports the model but the model never imports it. The cross-cutting eval-curator shared agent enforces the separation.
Plan DeepEval, Promptfoo, and RAGAS evaluation suites for an LLM feature. Use when the user asks to plan LLM evaluations or invokes /eval-suite-planning.
readings.ts의 요약 필드가 원본 kr 마크다운과 비교하여 잘 작성되었는지 평가합니다. 이슈별로 개선사항을 제안하고 사용자 컨펌 후 readings.ts를 수정합니다. 사용: /eval-summary week1 또는 /eval-summary week1/slug
Runnable evaluation template scripts for ML tasks. Match task_type to template, adapt CONFIG, run.
UX 전문가 관점에서 서비스를 평가하는 Agent. Jakob Nielsen의 10가지 휴리스틱 기준으로 체계적으로 사용성을 평가하고 심각도를 분류한다.
Run evaluation tests against a multi-agent workflow to assess orchestration quality and failure archetype resistance
Analyze evaluation results and close the loop. Specification failures create local CDRs to fix agent rules; generalization failures go to evaluator backlog.
Refine, cluster, and accept draft criteria into the published goldset. Isolates 20% holdout split and publishes goldset.md + goldset.json.
Provides context about the Roo Code evals system structure in this monorepo. Use when tasks mention "evals", "evaluation", "eval runs", "eval exercises", or working with the evals…
Analyze, design, or triage LLM evaluation workflows. Use when the user asks for evaluator design, error analysis, judge prompts, RAG evals, synthetic data, or review tooling.
Extract eval criteria from product specs and production failure traces (bottom-up error analysis). Writes proposed criteria to .adlc/drafts/evals/.
Evaluación de impacto de IA — genera una evaluación de impacto completa para un sistema de IA, cubriendo clasificación de riesgo, derechos fundamentales y medidas de mitigación…
Evaluate an existing visual artifact against a tradition, intent, and L1-L5 cultural/visual rubric.
Evaluate a deployed web-accessible agent against scenarios declared in an agent.yaml manifest. Drives the agent via the Playwright MCP server in a real, visible browser, captures…
Run repeatable EvalScope benchmark suites for LLM, VLM, RAG, and agent workflows, then inspect traces and reports before changing models or prompts.
Evaluate AI Compliance Engine — measure check accuracy, evidence quality, framework coverage, risk scoring calibration, false positive/negative rate.
Use Claude Code to run a structured vendor diligence workflow that questions vendor agents, cross-checks claims, and returns evidence-backed scorecards.
Use when user wants to evaluate, review, or assess code changes from current branch, a named branch, or a PR (number or URL)
Evaluate channels on contribution and marginal ROI under a stated attribution model — not average ROI or lead count. Reach for this on a channel or budget question.
Use ParseBench to run repeatable document-parser evaluations before an agent relies on PDF, table, chart, or enterprise-document output for downstream decisions.
Evaluate Enterprise AI Governance Hub — risk classification accuracy, policy enforcement, review compliance, registry completeness, dashboard accuracy.
Critically assess external feedback (code reviews, AI reviewers, PR comments) and decide which suggestions to apply using adversarial verification.
Body-read gate for a single job posting — fetch, save, check limits/comp/fit, emit GATE/FIT/COMP/ACTION, and write calibrated fit back to the tracker.
Design repeatable language-model capability, quality, behavior, regression, or safety evaluations. Use when creating cases and graders, comparing a treatment with a baseline, or…
Evaluate and compare levitation mechanisms for a given application through a structured trade study. Covers magnetic (passive diamagnetic, active feedback, superconducting),…
Black-box evaluation of a shipped LLM feature — adversarial probes for hallucination, prompt-injection, and cost-runaway vs stated expectations. Not RAG/embedding.
Use WildClawBench to benchmark agents on hard end-to-end OpenClaw tasks covering tool orchestration, multimodal work, coding, safety, and long-horizon planning.
Analiza el riesgo asociado a la profundidad de la cobertura del ducto y emite recomendaciones de integridad basadas en los estándares de seguridad frente a excavaciones y…
MMI評価エージェント - Modularity Maturity Indexによるモジュール成熟度評価（Cohesion/Coupling/Independence/Reusability）。/evaluate-mmi [対象パス] で呼び出し。 — from wfukatsu/nexus-architect
Use SandboxFusion to run and judge LLM-generated code in controlled sandboxes across many languages and benchmark-style evaluation tasks.
Measure model performance on test datasets. Use when assessing accuracy, precision, recall, and other metrics.
Independently evaluate an implementation plan posted on a GitHub issue. Posts findings and adds plan-reviewed label. Usage: /evaluate-plan
Evaluate plugin quality. Use when user says "evaluate plugin", "review plugin quality", "score my plugin", "check plugin", "rate plugin".
Measure whether a candidate skill improves routing and behavior over its baseline before adoption. Use only when explicitly requested and never install the candidate automatically.
Assess Story complexity, identify required domains, and determine delivery tier (MicroDelivery / Core Team / Core Team + Specialists).
Use when handed a real deck, one-pager, or pitch to assess end to end — the orchestration that turns the seven reads into the ordered deliverable.
Score a set of customer support tickets with the ambient Claude model against user-written rules and a typed schema, writing a Qval eval file (*.qval.json).
Evaluate tool selection, arguments, schemas, execution, observation use, recovery, and side effects. Use when comparing tool-capable checkpoints, prompts, parsers, agent loops,…
Evaluate one or more traces against an existing Truesight live evaluation. Use when a deployed live evaluation already exists and the user wants run outputs with optional handoff…
Structures abdominal pain workups with differential by quadrant and surgical consultation criteria. Use when assessing acute abdomen, determining imaging needs, or identifying…
Evaluates software quality from an end user's perspective. Use when emulating a real user to assess a running web app, CLI tool, or API.
Make an evidence-based hiring decision and produce a Candidate Evaluation Decision Pack (criteria + scorecard, signal log, work sample/trial plan + rubric, reference check script…
Help users make better hiring decisions. Use when someone is evaluating job candidates, making hiring decisions, conducting reference checks, reviewing work samples or take-homes,…
Analyzes concession terms with revenue sharing, performance requirements, hand-back conditions, and termination provisions.