Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
LLM testing and evaluation tools including promptfoo, trulens, and evals frameworks
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
Build automated LLM evaluation pipelines with benchmarks, regression tests, RAGAS, and human eval workflows.
Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing. — from majiayu000/claude-skill-registry
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking.
LLM Fine-Tuning expert. Covers LoRA, QLoRA, PEFT, dataset preparation, Hugging Face Trainer/TRL, RLHF, DPO, quantization (GPTQ/AWQ/GGUF), model merging, distributed training with…
Comprehensive guide to using LLMs throughout the game development lifecycle - from design to implementation to testingUse when "ai game development, llm game dev, claude game, gpt…
Diagnostic playbook for triaging a self-hosted LLM gateway (LiteLLM or any Postgres-backed OpenAI-compatible proxy) running via Docker Compose on a homelab host.
LLM gateway and routing configuration using OpenRouter and LiteLLM. Invoke when: - Setting up multi-model access (OpenRouter, LiteLLM) - Configuring model fallbacks and r — from…
LLM gateway and routing configuration using OpenRouter and LiteLLM. Invoke when: - Setting up multi-model access (OpenRouter, LiteLLM) - Configuring model fallbacks and r — from…
Design input and output guardrails for an LLM app — decide what to check (injection patterns, PII, secrets, policy, schema, leakage, toxicity), place them as input vs.
Specify the safety and reliability guardrails for an LLM feature before it ships. Use when asked to define LLM guardrails, add safety controls to an AI feature, prevent prompt…
Optimize LLM response latency with caching and architecture. TRIGGERS - Use when user needs help with llm-latency-optimizer related tasks.
LLM inference load testing for throughput and concurrency limits. Token/s benchmarks, concurrent request sweeps, latency-vs-throughput curves, and breaking-point identification.
Guide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools.
Instrument LLM applications with request tracing, token accounting, quality signals, and feedback loops.
面向团队与企业用户的 Whisper 语音转文字工具(专业版)。核心能力: - 涵盖免费版全部能力(本地转录、翻译、多格式输出) - 批量处理:目录级递归转录,支持任务队列 - GPU 加速:CUDA / Metal / MPS 全面支持 - 说话人分离(diarization):多人对话识别 - 自定义词典:专业术语与品牌名词优化 - — from…
基于 Whisper v1 稳定版本的企业级语音转文字工具(专业版)。核心能力: - 涵盖免费版全部能力(v1 稳定 CLI、多格式输出、翻译) - 批量处理:目录递归与任务队列 - 模型管理:多版本预加载与热切换 - 性能调优:GPU 加速、半精度推理、批处理 - 自定义词典:initial_prompt 注入领域术语 - 服务化部署: — from…
Implement semantic and exact-match caching for LLM responses to reduce cost 40-60% and latency. Activate on: LLM caching, semantic cache, reduce API costs, cache AI responses.
Use when designing how one canonical agent core runs across Codex, Claude Code, Gemini CLI, Cursor, and AGENTS.md-compatible tools.
LLM inference infrastructure, serving frameworks (vLLM, TGI, TensorRT-LLM), quantization techniques, batching strategies, and streaming response patterns.
Build production LLM streaming UIs with Server-Sent Events, real-time token display, cancellation, error recovery. Handles OpenAI/Anthropic/Claude streaming APIs.
Install and configure LLMem for an agent harness. Handles CLI install, plugin deployment, skill registration, and provider setup.
Antes de fetchear, scrapear o llamar cualquier endpoint de un dominio web, buscá su archivo `/llms.txt` y, si existe, leelo COMPLETO (no solo la sección `## Skills`).
Use the CLI info command to summarize what llms-txt-php-cli detects/configures for the current repository.
Guide the user to generate an initial llms.txt for a repository using llms-txt-php-cli, choosing sensible defaults and verifying output.
Validate an existing llms.txt with llms-txt-php-cli and guide the user through fixing validation errors.
LMCache multiprocess (MP) mode — standalone LMCache server in its own pod/process that vLLM connects to over ZMQ.
Builds and queries code knowledge graph for dependency analysis, references, implementations, and architecture overview.
Builds and queries code knowledge graph for dependency analysis, references, implementations, and architecture overview.
Builds and queries code knowledge graph for dependency analysis, references, implementations, and architecture overview.
Builds and queries code knowledge graph for dependency analysis, references, implementations, and architecture overview.
Builds and queries code knowledge graph for dependency analysis, references, implementations, and architecture overview.
Creates core project docs (requirements, architecture, tech stack, patterns catalog). Use for any project regardless of type.
Creates infrastructure.md and runbook.md (Docker-conditional). Use for DevOps documentation in any project.
Creates reference docs (ADRs, guides, manuals) for nontrivial tech stack choices. Use when project needs justified architecture decision records.
Creates test documentation (testing-strategy.md, tests/README.md) with Risk-Based Testing philosophy. Use when setting up test strategy for a project.
Executes test tasks (label 'tests') through Todo to To Review with risk-based limits. Use for test task execution. Not for implementation tasks.
Worker that checks DRY/KISS/YAGNI/architecture compliance with quantitative Code Quality Score. Validates architectural decisions via MCP Ref: (1) Optimality - is chosen approach…
Orchestrates test planning pipeline (research → manual → auto tests). Coordinates ln-511, ln-512, ln-513. Invoked by ln-500-story-quality-gate.
Checks DRY/KISS/YAGNI/architecture compliance with quantitative Code Quality Score. Use when implementation tasks are Done and need quality scoring.
Performs manual testing of Story AC via executable bash scripts saved to tests/manual/. Creates reusable test suites per Story. Worker for ln-510.
Auto-fixes low-risk tech debt (unused imports, dead code, commented-out code) with >=90% confidence. Use when audit findings need safe automated cleanup.
Plans automated tests (E2E/Integration/Unit) using Risk-Based Testing after manual testing. Calculates priorities, delegates to ln-301-task-creator. Worker for ln-510.
Analyzes application logs: classifies errors, checks log quality, maps stack traces to source. Use when logs need review after test runs or during development.
Orchestrates test planning pipeline: research, manual testing, automated test planning. Use when Story needs comprehensive test coverage planning.
Performs manual testing of Story AC via executable bash scripts in tests/manual/. Use when Story implementation needs hands-on AC verification.
Plans automated tests (E2E/Integration/Unit) using Risk-Based Testing after manual testing. Use when Story needs a test task with prioritized scenarios. — from engineering/testing
Audit code comments and docstrings quality across 6 categories (WHY-not-WHAT, Density, Forbidden Content, Docstrings, Actuality, Legacy).
Architecture audit worker (L3). Checks DRY (7 types), KISS/YAGNI, layer breaks, error handling, DI patterns. Returns findings with severity, location, effort, recommendations.
Use when auditing the test surface through the evaluation platform with mandatory research, coordinated test audit workers, and structured summaries.
Detects tests validating framework/library behavior instead of project code. Use when auditing test business logic focus.
Validates E2E coverage for critical paths (money, security, data integrity). Risk-based prioritization. Use when auditing E2E test coverage.
Scores each test by Impact x Probability, returns KEEP/REVIEW/REMOVE decisions. Use when auditing test value and pruning low-value tests.
Identifies missing tests for critical paths (money, security, data integrity, core flows). Use when auditing test coverage gaps.