---
name: sindri-scaffold-benchmark
description: Scaffold .claude/scripts/sindri/benchmark.py by scanning the repo, proposing a measurement in natural language, translating the user's refinements into Python, and running the script once to verify it emits a valid METRIC line. Never ask the user for Python or shell syntax.
tools: Bash, Read, Write, Edit, Glob, Grep
---

# sindri-scaffold-benchmark — write the benchmark script

Goal: produce `.claude/scripts/sindri/benchmark.py` that, when run, prints one line matching `METRIC <metric_name>=<number>` as its last stdout line. You translate user intent (natural language) into Python; the user never writes code.

## Inputs

- `$METRIC_NAME` — snake_case identifier (e.g., `bundle_bytes`)
- `$GOAL_TEXT` — the user's original goal statement, for context
- Resolve `FORGE` to this plugin's `scripts/forge.sh` (this skill lives at `<plugin-root>/skills/sindri-scaffold-benchmark/SKILL.md`, so the wrapper is `<plugin-root>/scripts/forge.sh`). The validation step below uses `"$FORGE"`.

## 1. Scan the repo

Detect:

- **Package manager:** `pnpm-lock.yaml` → pnpm; `yarn.lock` → yarn; `bun.lock` → bun; `package-lock.json` → npm; `uv.lock` → uv; `poetry.lock` → poetry; `Cargo.lock` → cargo.
- **Build tool:** look for `vite.config.*`, `webpack.config.*`, `next.config.*`, `rollup.config.*`, `esbuild.config.*`. For Python, look for build scripts in `pyproject.toml`.
- **Output directory:** inspect `package.json`'s `"build"` script or the build-tool config. Common: `dist/`, `build/`, `out/`, `.next/`, `public/`.
- **CI script references:** `.github/workflows/*.yml` (helpful when the metric is CI-time).

## 2. Propose a measurement

Based on `$METRIC_NAME` + what you found, propose a concrete measurement. Examples:

| Metric | Stack detected | Proposal |
|---|---|---|
| `bundle_bytes` | Vite + pnpm, output `dist/` | Run `pnpm build`. Sum byte size of `dist/`. |
| `bundle_bytes` | Next.js + npm | Run `npm run build`. Sum byte size of `.next/static/` + `.next/server/`. |
| `ci_seconds` | GHA workflow `ci.yml` exists | Trigger `gh workflow run ci.yml`, poll until done, read the `conclusion` + `duration` from `gh run view --json`. |
| `test_runtime_seconds` | `pytest` in pyproject | Run `pytest --durations=0`, capture total wall-clock. |
| `lighthouse_score` | Next.js + deploy target | **Halt** — ask the user which deployed URL to audit; too stack-specific for inference. |

Print your proposal in plain English first. Example:

> *"I see you're on Vite + pnpm with output in `dist/`. Proposed benchmark: run `pnpm build`, then sum bytes in `dist/` and print `METRIC bundle_bytes=<sum>`. Accept? Refine? (reply with changes in plain English)"*

## 3. Iterate in natural language

Accept refinements like:

- "use npm not pnpm" → swap `pnpm build` for `npm run build`
- "measure gzipped" → pipe through `gzip -c | wc -c` per file, sum
- "skip the build, assume dist/ is already there" → drop the build step
- "also warm the cache first" → prepend a throwaway build

Never ask the user to write or edit Python. Keep the conversation in prose; your job is translation.

## 4. Generate `benchmark.py`

Write the final script to `.claude/scripts/sindri/benchmark.py`. Required shape:

```python
#!/usr/bin/env python3
"""Sindri benchmark — auto-generated. <one-line purpose>.

Emits exactly one `METRIC <name>=<number>` line as the final stdout line.
All other output goes to stderr so the orchestrator can ignore it.
"""
from __future__ import annotations

import subprocess
import sys
from pathlib import Path


def main() -> int:
    # <step 1: build/prep>
    # subprocess.run([...], check=True, stderr=sys.stderr, stdout=sys.stderr)

    # <step 2: measure>
    # value = <computed number>

    # <step 3: emit METRIC line as the LAST stdout line>
    # print(f"METRIC <metric_name>={value}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Substitute `<metric_name>` with `$METRIC_NAME` exactly. The `METRIC` line must be on stdout; build/prep noise must go to stderr so a downstream parser sees only the metric.

`chmod +x .claude/scripts/sindri/benchmark.py` after writing.

## 5. Validate with one real run

```bash
"$FORGE" validate-benchmark --expected $METRIC_NAME
```

If it fails:

- **Non-zero exit:** show the stderr to the user, ask what's wrong, iterate.
- **No `METRIC` line found:** show the user the last 20 lines of stdout, ask which value is the metric, regenerate.
- **Wrong metric name:** regenerate with the right name.

Iterate until `validate-benchmark` exits 0 and prints the parsed value.

## 6. (Optional) Scaffold `checks.py`

Ask: *"Run tests or lint as a gate before each experiment commits? (recommended: yes if you have a fast test suite)"*.

If yes, generate `.claude/scripts/sindri/checks.py`:

```python
#!/usr/bin/env python3
"""Sindri pre-commit check — exits 0 if all checks pass, non-zero if they fail."""
from __future__ import annotations
import subprocess
import sys


def main() -> int:
    # subprocess.run(["pnpm", "test", "--run"], check=True)
    # subprocess.run(["pnpm", "lint"], check=True)
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Make it executable and run once to confirm it passes.

## Fallback — exotic stacks

If the scan yields nothing useful (proprietary observability, multi-cluster k8s, exotic language), ask the user in plain English: *"Describe how you would measure this manually today — what command, what file, what API? I'll translate that into Python."* Then do so. Never demand code from the user.

## Anti-patterns

- Never emit the `METRIC` line anywhere but the last stdout line.
- Never put build logs on stdout — only on stderr.
- Never leave the script non-executable.
- Never declare "done" without running `validate-benchmark` successfully.