--- name: agent-cost-audit description: Audits why an AI coding setup burns tokens, model quota, CI minutes, and build minutes far faster than the work seems to justify. Measures the always-loaded per-turn context floor from Claude Code transcripts and attributes it to MCP servers, context files, and the skill/subagent/plugin roster; measures real billed CI minutes with GitHub's per-job minute rounding applied, plus step-level duplication and deploy volume by branch; and verifies every claim against consumption before reporting it. Use this whenever someone says they blew through their weekly limit, that usage is burning faster than expected, that context fills up or "races" despite compaction and handoffs, that turns feel slow, that GitHub Actions minutes or Vercel build minutes are unexpectedly high, or when auditing CLAUDE.md / AGENTS.md size and MCP server sprawl. Also use it proactively before adding more MCP servers, subagents, or parallel worktree sessions to an existing setup. --- # Agent Cost Audit ## Why this exists People optimize the layer they can see. Interfaces show a context percentage and a usage bar, so people shorten conversations, write handoff docs, and compact often — and stay expensive anyway. The visible layer is usually not the expensive one. Every turn re-sends the entire request: system prompt, tool definitions, always-loaded context files, the description of every enabled skill and subagent, then the conversation. Everything before the conversation is a fixed floor paid on every turn of every session. A real setup measured at **66,770 tokens before the user typed a word**, and no amount of handoff discipline moves it. Two things to get right before quoting that number at anyone: - **Express it against the actual window.** The same floor is 33% of a 200k window and 6.7% of a 1M one. Quoting a bare percentage without naming the window has been wrong often enough to be worth the extra clause. - **Check whether tool schemas are actually eager.** "Every MCP server adds its full tool surface to every turn" was true and is now setup-dependent: Claude Code can defer MCP tool schemas behind a search tool, in which case only tool *names* and server instructions are always-on, and the MCP lever is far smaller than it looks. Verify against `/context` for the setup in front of you; do not assert it from memory. And the part people skip: **a large share of any floor is the harness's own system prompt and built-in tool definitions, which the user cannot change at all.** In the measured setup that was ~45k of the 66.8k. Establish the uncontrollable base *before* agreeing to a target, or you will promise a number that cannot be reached and the whole audit reads as wrong. The same pattern hides money elsewhere. A job that builds the same bundle six times costs a third of the run, and looks completely normal in config. ## The one rule **Measure consumption. Then use config to explain the measurement. Never the reverse.** Config inspection is a hypothesis generator, not a measurement. A cron string tells you what a workflow is permitted to do, not what it cost, not whether it ran, not whether it was disabled last quarter. Reading config and reporting it as a finding produces confident, specific, wrong answers — and it systematically misses the largest real costs, because those leave no trace in config at all. This applies to the token side too: a 97% cache hit rate is a discount, not a saving. Volume is the metric. ## Workflow ### 1. Locate the transcripts Claude Code writes per-session JSONL to `~/.claude/projects//*.jsonl`. Each assistant record carries `message.usage` with `input_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens`, `output_tokens`. If the environment cannot read `~/.claude` (Cowork's device bridge refuses it, and the folder picker rejects it as the desktop app's own config directory), ask for a copy: ```bash mkdir -p ~//_cc-usage && rsync -a --include='*/' --include='*.jsonl' --exclude='*' ~/.claude/projects/ ~//_cc-usage/ ``` Without transcripts this audit degrades to guesswork. Say so rather than proceeding. ### 2. Measure the floor ```bash python3 scripts/measure_floor.py --days 3 ``` Read for: **median floor** against the context window; **minimum floor**, which approximates the irreducible base, so floor-minus-minimum is what configuration adds; **output share** — under ~1% means spend is re-reading context, not producing work; **session and turn counts**, since floor × turns is the baseline cost; **subagent and worktree counts**, because each spawn pays the floor again; and **model mix**, since Opus-class models multiply everything. ### 3. Attribute the floor — look globally, not just at the repo ```bash python3 scripts/mcp_attribution.py --repos --days 3 ``` The headline is the ratio of servers loaded to servers actually called. Report it concretely: "N MCP calls out of M total tool calls." Be holistic about where servers come from. A repo's `.mcp.json` is usually the smaller half. Check `~/.claude/settings.json`, installed plugins, and any globally configured servers — a plugin bundling a browser driver or an issue tracker adds its full tool surface to every turn of every session in every repo. A repo with no `.mcp.json` at all still pays a floor, and `enableAllProjectMcpServers` is a no-op there; if you find yourself blaming a repo-level flag for a floor that exists everywhere, the cause is global. ### 4. Audit the always-loaded surface If the user is at the terminal, the cheapest and most authoritative move is to have them run **`/context`**, which shows exactly what loaded at startup and what each part costs. Reconstructing it from disk is a fallback, not the primary method. Context files: ```bash find -maxdepth 2 \( -name CLAUDE.md -o -name AGENTS.md \) -exec wc -c {} + ``` Watch for a root `CLAUDE.md` symlinked to `AGENTS.md` — one file then loads in every harness. `references/context-files.md` has the refactoring rules and scoring rubric. Then the roster, which is usually the bigger surprise and is almost always missed: ```bash python3 scripts/measure_roster.py --project ``` Every enabled skill and subagent puts its name and full `description:` into the system prompt of every turn. A plugin is all-or-nothing: install it and you pay for all 30 of its skills whether or not you invoke one. In one real setup this was 8.2k tokens, of which a single plugin was 2.5k for 33 skills. **Cut the roster before you cut guidance.** Anthropic's own guidance is that a bloated CLAUDE.md makes Claude *ignore* instructions — "important rules get lost in the noise" — so trimming it is a quality fix, not just a cost one. But the roster costs quality nothing, so spend it first. The official rule for what belongs where: CLAUDE.md is for what applies *broadly*; "for domain knowledge or workflows that are only relevant sometimes, use skills instead — Claude loads them on demand without bloating every conversation." A reference table or a CLI manual in an always-loaded rules file is the textbook case for moving to an on-demand skill. Size the claim honestly. Cutting a 24k-char file saves ~6k tokens from a 72k floor — real, but 8%. **And a large part of any floor is the harness's own system prompt and built-in tool schemas, which the user cannot change at all.** Establish that uncontrollable base before agreeing to a target, or you will promise a number that is not reachable and the user will conclude the audit was wrong. ### 5. Measure CI, then explain it Measurement first: ```bash python3 scripts/measure_ci.py --repo --days 30 --sample 20 ``` **Actions bills each JOB rounded up to a whole minute** — "GitHub rounds the minutes and partial minutes each job uses up to the nearest whole minute" ([GitHub Docs][gh]). An eight-second job costs a full minute. Two consequences that invert the usual instinct: - Splitting a workflow into more parallel jobs can cost *more* than it saves. In one measured fleet, 1,255 minutes of runtime billed as 1,574 — **20% was rounding**, paid entirely on jobs that finish in seconds. - Replacing one long-running job with N short event-triggered ones is usually a trap: each of the N bills a full minute. Merging two short jobs that share setup halves their cost. Keep `if: always()` on every step and no signal is lost — that is a step-level property, so merging jobs does not reintroduce the masking that made someone split them in the first place. Do **not** use `/actions/runs/{id}/timing`: it returns `billable: {}` on personal Free/Pro plans, which yields "0 billable minutes" for every workflow — a wrong answer wearing the costume of a measurement. `measure_ci.py` uses per-job durations instead. Its other valuable output is **duplicated work** — the same step running several times per run. Repeated builds are routinely a third of a run's cost and are invisible in config; they come from several jobs each building independently, or a Playwright `webServer.command` that runs a full build per project or shard. Fix by building once and passing the artifact downstream. Allowances for private repos: Free 2,000 min/month, Pro and Team 3,000. Linux 2-core is $0.006/min after the January 2026 price cut. Only after you have measured minutes, generate hypotheses about why: ```bash python3 scripts/audit_ci.py ``` Two of its checks earn their place above the rest, because both are pure waste with no gate attached: - **`continue-on-error: true` on the PR path.** A job or step that cannot fail a PR buys nothing by running per-PR. Measured once at 121 min/week in a single repo — a Lighthouse job re-running a build another job had already done, plus a visual-regression step whose baselines had never landed. Move both to push-to-main. - **A path filter applied to `push` but not `pull_request`.** `paths`/`paths-ignore` sit under each *event*. Filtering `push` while `pull_request` runs bare filters the minority of runs, and reads as "already handled" to anyone skimming. Everything else that script prints is labelled a hypothesis on purpose. Use it to explain the measured numbers, and discard anything the measurement does not support. [gh]: https://docs.github.com/en/billing/reference/actions-minute-multipliers ### 5b. Measure build minutes, not just CI minutes CI minutes and *build* minutes are separate bills, and on an agent-heavy setup the build bill is usually the larger one. Measure deploys directly rather than inferring them from config: ```bash python3 scripts/measure_builds.py --project --team ``` **Agent branches are now the dominant source of build spend.** An agent that pushes per edit generates preview builds at a rate no human workflow does. One measured project ran ~26 full Next.js builds/day: over 7 days, 146 deployments — 33 production, 113 preview, and 64 of those previews (57%) on agent branches, with 15 on a single `worktree-*` branch and 9 on one `codex/*` branch. **Use a full week.** The first pass at that project sampled the most recent 20 deployments, landed inside a 9.5-hour stretch where one agent was pushing hard, and reported ~51/day — roughly double. The finding survived; the headline number did not. A sample short enough to fit inside one agent's working session is not a rate, and this is the single easiest way for this audit to produce a number that gets corrected in public. **Derive the agent-branch prefix list from a measured branch-frequency count, not from memory.** The same first pass wrote a prefix list from recall and missed `claude/*` entirely; the count over 146 deployments surfaced it. The fix is a branch-prefix guard in the ignore step, not just a docs filter. In a team repo, skipping only agent-branch prefixes unless the commit opts in (`[vercel-preview]`) removes most of the spend while leaving every human's preview untouched — which is what makes it land without an argument. ### 6. Verify every claim before it goes in the report This step is not optional, and skipping it is how this audit produces confident nonsense. For each candidate finding, all four must pass: **Liveness — is it actually running?** A cron inside a comment is not a schedule. A workflow can be disabled in the UI with its file untouched. A `vercel.json` in a sample, reference, or archive directory is not a deployed project. `measure_ci.py` reports real workflow state; use it. **Recency — was it already fixed?** Run `git log --since='6 months ago' -- ` and read the surrounding comments. Teams often document a decision in the file itself. If you find yourself re-deriving a conclusion that is written in a comment three lines above, you are reporting their own past work back to them. **Magnitude — does a measured number support the size of the claim?** "~1,440 runs a month" is arithmetic on a string. "22 billable minutes per run × 114 runs" is a measurement. If you cannot produce the second kind, the item belongs in a clearly labelled *Hypotheses* section, never in *Findings*. **Mechanism — does the cause explain the number?** If measured minutes are dominated by one workflow and your explanation concerns a different one, the explanation is wrong even if the config observation is true. A finding without a number is a hypothesis. Label it as one. ### 7. Report ```markdown # Agent cost audit — ## Measured ## What explains it ## Fixes, ranked by measured impact ## Hypotheses — not yet confirmed ## Already handled ``` Rank by measured impact, not by ease. Read `references/remediation.md` for fix patterns. ## Failure modes this skill has actually produced These are real, from a live audit. They are here because each one felt convincing. **Reported a disabled cron as the main drain.** The `*/30` string matched inside a comment block that explained the workflow had been disabled months earlier. The audit then "discovered" the reasoning already written in that comment and presented it back as a new finding, with a projected saving that could not exist. **Reported a missing `ignoreCommand` on a directory that was not a deployed project.** A `vercel.json` under a references folder is a file, not a build. **Blamed a repo-level MCP flag in a repo with no `.mcp.json`**, where the flag is inert and the floor came entirely from global config. **Missed a build running six times per CI run** — in verify, in Lighthouse, in the Playwright `webServer`, and three times across browser gates — roughly a third of every run's billable time. No config check could have found it. Only step timings could. **Reported a build rate that was roughly double.** Sampled the most recent 20 deployments, which happened to sit inside one agent's busy 9.5-hour stretch, and projected it as a daily rate. The finding held; the number did not. Use a full week. **Trusted its own estimator over the tool that knows.** Reconstructed the roster cost by summing frontmatter descriptions from disk, globbed the `agent.md.tmpl` template files sitting beside the real ones, and inflated one plugin from 3 subagents to 6. `/context` was available the whole time and is authoritative. Character-based estimates are for when you genuinely cannot run it. **Proposed a fix that was more expensive than the problem.** A polling gate that held a runner idle for ~4.7 min looked like the worst job in the fleet. Rewriting it to re-trigger on `check_run: completed` bills a full minute *per re-trigger* under per-job rounding — 4-6 per PR — so the "fix" cost more. It was also unsafe: a job skipped by a job-level `if:` reports conclusion `skipped`, which GitHub counts as **passing** for a required check, so an unrelated check completing could turn a red merge gate green. Cost fixes touching a required check need a safety argument, not just an arithmetic one. The common root of the first four: measuring configuration and calling it consumption. Everything in this skill's CI path exists to stop that. The root of the last three: trusting a derived number, or an appealing fix, without the check that would have falsified it. ## Record what must NOT be "fixed" An audit that only lists savings gets partially applied and then re-derived from scratch six months later, including the parts that were considered and correctly rejected. Give the report a section for things that look wasteful and are not, with the reason: - a dispatcher job that exists to path-filter a fan-out is structurally required, even though it bills a whole minute for eleven seconds of work - a gate whose idle wait is inherent to what it gates - anything already disabled, already fixed, or never running Name them, or the next pass will spend its budget rediscovering them. ## Interpretation notes **Handoffs and compaction cannot fix a floor problem.** Both operate on conversation history. If the floor is 35% of the window, every fresh session starts there. When someone reports diligent handoff discipline and no improvement, the floor is the explanation — and saying so is a relief, not a criticism. **Subagents and worktrees multiply the floor.** Each spawn is a new session paying the full baseline. Fan-out that looks free in wall-clock time is not free in tokens. **Blocking `Stop` hooks are silent double-turns.** A `Stop` hook returning `{"decision": "block"}` re-invokes the model with the entire conversation. The person never sees a message they sent but pays a full turn. The gate usually belongs at commit time instead. **Tone matters.** People arrive at this audit frustrated and often blaming themselves. None of this is visible in any interface — they optimized what they could see. Lead with the measurement, not with what they should have done. And when a finding turns out to be something they already fixed, say that plainly; it is information, not an embarrassment to smooth over. --- ## Layout note for the gist copy Gists are flat, so the bundled files are named `scripts--.py` and `references--.md`. To install, recreate the directories: ```bash mkdir -p ~/.claude/skills-library/agent-cost-audit/{scripts,references} # SKILL.md -> agent-cost-audit/SKILL.md # scripts--*.py -> agent-cost-audit/scripts/*.py (strip the "scripts--" prefix) # references--*.md -> agent-cost-audit/references/*.md ``` Put it in `skills-library/` (loaded on demand via an index skill) rather than `~/.claude/skills/` — this skill's own advice is that a rarely-used skill in the always-loaded roster costs its full description on every turn, forever.