Inference Decode Step Budget

Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable lever is. Uses vLLM's native /start_profile + /stop_profile HTTP endpoints (enable once, then restart-free shell-in captures in seconds). Encodes the correctness gates that prevent contaminated or under-counted budgets (clean single-stream driver, GPU-busy must include CUDA-graph execution, reconcile against driver-measured TPOT, reject load-time/lull/truncated captures). Triggers on "decode step budget", "c=1 hot path", "where does decode time go", "TPOT breakdown", "is decode kernel-bound or host-bound", "latency-tier profile", "GPU idle during decode", "start_profile", "torch profiler vllm", or any combination of "decode / c=1 / latency / TPOT / ITL" with "profile / budget / breakdown / hot-path / where" on a vllm deploy.

mkdir -p ~/.claude/skills/inference-decode-step-budget
curl -L https://claudskills.com/skills/inference-decode-step-budget/SKILL.md \
  -o ~/.claude/skills/inference-decode-step-budget/SKILL.md

@misc{inference-decode-step-budget-2026,
  author    = {cfregly},
  title     = {Inference Decode Step Budget [Claude Code skill]},
  year      = {2026},
  publisher = {ClaudSkills},
  url       = {https://claudskills.com/skills/inference-decode-step-budget/}
}