---
name: structured-output-llm
description: Gets machine-parseable JSON out of an LLM reliably — prefer provider-enforced grammar (OpenAI `response_format:{type:"json_schema",strict:true}`, Anthropic tool-calling / `tool_choice:{type:"tool"}`, Gemini `responseSchema`) over a "respond in JSON" prompt, define the contract once as a Pydantic/Zod model and validate every response (never trust the string), use constrained decoding / grammars for open weights (Outlines, llguidance, llama.cpp GBNF, vLLM `guided_json`), keep schemas SHALLOW with required fields + enums + `additionalProperties:false`, and build a repair-and-retry loop that handles refusals, `length`/`max_tokens` truncation, and streaming partial JSON (`partial-json-parser`). Schema-guided generation eliminates parse-error retries; validate-and-repair is the backstop, not the primary mechanism.
when_to_use: You need a model to emit JSON/objects a program consumes — extraction, classification into fixed labels, function arguments, form-filling, or feeding output into another service — and string-parsing or regex on free text keeps breaking. Distinct from prompt-engineering (crafts the instruction/few-shot prose; this skill enforces the output SHAPE via grammar+validation) and agent-tool-mcp-builder (defines the tools/MCP an agent decides to call; this skill is the one-shot "make THIS call return a typed object" half, even when implemented via a single forced tool).
---

## When to Use

Reach for this skill when a program — not a human — consumes the model's output and the shape must be guaranteed:

- "Extract these 8 fields from this invoice/email as JSON I can `JSON.parse`"
- "Classify into one of {refund, dispute, question} — give me just the label"
- "The model returns ```json fences / prose / trailing commas and my parse breaks ~3% of the time"
- "Function arguments come back malformed or with hallucinated keys"
- "Fill this form schema" / "map this text to my DB row"
- "Open-weights model won't reliably produce valid JSON"
- "Long extraction gets cut off mid-object" or "streaming JSON is unparseable until complete"

NOT this skill:
- Writing the *instruction*, few-shot examples, role/system prompt, or chain-of-thought prose → prompt-engineering (it shapes WHAT to say; this skill enforces the machine SHAPE of the reply)
- Defining tools/an MCP server an agent autonomously selects among across many turns → agent-tool-mcp-builder (this skill is one forced, schema-constrained call returning a typed object)
- Reliability of the whole LLM app — timeouts, fallbacks, circuit-breakers, idempotency around the call → harden-llm-app-reliability (this skill owns only the output-shape contract)
- Token/latency/cost tuning, caching, model selection → optimize-llm-cost-latency (note: strict schemas cost a cache miss on first use — that skill weighs it)
- Stopping a malicious payload in retrieved/user text from hijacking the model → defend-llm-prompt-injection (orthogonal; a valid-schema output can still be adversarial content)
- Retrieval/chunking/grounding that feeds the prompt → rag-pipeline (this skill structures rag's *output* step)
- Scoring whether the extracted values are *correct* across a dataset → llm-eval-harness (this skill guarantees parseable; that one measures accuracy)
- Validating ordinary user-submitted form data (no LLM) → build-form-validation
- The wire/serialization contract between your own services → rest-graphql-contract
- Scraping the HTML/DOM the text came from → scrape-structured-web-data

## Steps

1. **Prefer provider-enforced schema over a "respond in JSON" prompt — it's a different mechanism, not a stronger hint.** A prompt is a request the model may ignore (fences, preamble, extra keys); enforced schema masks the decoder's token logits so only schema-valid tokens are emittable — invalid JSON becomes *impossible*, not just unlikely. Use the strongest enforcement the provider offers:

   | Provider | Enforcement | How |
   |---|---|---|
   | **OpenAI** (gpt-4o+, gpt-4.1, o-series) | Structured Outputs, 100% guaranteed | `response_format:{type:"json_schema",json_schema:{name,schema,strict:true}}` — or `strict:true` on a function tool |
   | **Anthropic Claude** | Tool-calling (no native json_schema) | define one tool with `input_schema`, set `tool_choice:{type:"tool",name:"..."}` to force it; read `tool_use.input` |
   | **Google Gemini** | Controlled generation | `generationConfig:{responseMimeType:"application/json",responseSchema:{...}}` (also supports `responseSchema` enums) |
   | **Azure OpenAI** | same as OpenAI | `response_format` json_schema on `2024-08-01-preview`+ |
   | **Open weights** (vLLM/TGI/llama.cpp/Ollama) | Constrained decoding (step 5) | `guided_json` / GBNF grammar / Outlines |

   `json_object` mode (the older `{type:"json_object"}`) only guarantees *syntactically* valid JSON, NOT your schema — treat it as a weak fallback, never the goal.

2. **Define the contract ONCE as a typed model; let the SDK emit the schema and parse the result.** Don't hand-write JSON Schema and a parser separately — they drift. Use the model class as the single source of truth:
   - **Python** → Pydantic v2. OpenAI: `client.beta.chat.completions.parse(..., response_format=MyModel)` → `.choices[0].message.parsed` is a typed instance (or `.refusal`). Or `MyModel.model_validate_json(raw)`.
   - **TS/JS** → Zod + the OpenAI helper `zodResponseFormat(MySchema, "name")`, then `completion.choices[0].message.parsed`; or `MySchema.parse(JSON.parse(raw))`. (Instructor / `instructor-js` wrap this with retries.)
   - **Anthropic** → derive `input_schema` from Pydantic via `Model.model_json_schema()` and pass as the tool's input_schema; validate `tool_use.input` back through the model.
   The rule: **validate every response through the typed model even when the provider guarantees the schema** — guarantees cover JSON-schema-expressible constraints, not your business invariants (date ranges, cross-field rules, enum-of-enums), and self-hosted/fallback paths have no guarantee at all.

3. **Keep the schema SHALLOW and tight — depth and looseness are where models fail and where strict mode rejects you.** Constraints that materially improve reliability:
   - **`required` everything + `additionalProperties:false`.** OpenAI strict mode *requires* every property be in `required` and forbids additional props — model "optional" as `required` + nullable union (`{"type":["string","null"]}`). This also blocks hallucinated keys.
   - **Enums over free strings for categories** (`"type":"string","enum":["refund","dispute","question"]`) — turns classification into a closed set the decoder can't escape.
   - **Flatten.** Prefer a flat object or a top-level `{"items":[...]}` array wrapper over 4-level nesting. Deep/recursive schemas raise latency, hit provider depth/property caps (OpenAI: ≤5000 props, ≤5 nesting for strict), and degrade accuracy.
   - **Avoid where unsupported:** OpenAI strict mode disallows `minLength`/`maximum`/`pattern`/`format` and many keywords — enforce those in step 2's validator, not the wire schema. (Anthropic/Gemini tolerate more but don't *guarantee* them.)
   - **Order fields so reasoning precedes the answer:** put a `reasoning`/`evidence` string field *before* the `answer`/`label` field — the model generates them in order, so it "thinks" before committing (cheap, schema-native CoT).

4. **Don't ask for JSON and reasoning in the same free-text turn.** If you need chain-of-thought, either (a) put it inside the schema as a leading field (step 3), or (b) do a two-call split: one reasoning call (free text), one extraction call (strict schema) over the reasoning. Mixing "think step by step, then output JSON" in one un-enforced response is the classic source of prose-before-the-brace parse failures.

5. **For open-weights / self-hosted, use constrained decoding (grammar-level), not prompting.** These enforce validity at the token sampler:

   | Tool | Use | Invoke |
   |---|---|---|
   | **vLLM** | production serving | `guided_json=<schema>` / `guided_choice=[...]` / `guided_regex` in `extra_body` (backed by xgrammar/outlines) |
   | **Outlines** | library, any HF model | `outlines.generate.json(model, PydanticModel)` → returns typed object |
   | **llguidance / xgrammar** | fast grammar engines | embedded in vLLM/TGI; sub-ms per-token masking |
   | **llama.cpp / Ollama** | local GGUF | GBNF grammar file, or `format:"json"`+schema (Ollama) |
   | **TGI** | HF Inference | `grammar:{type:"json",value:<schema>}` |

   Constrained decoding makes invalid JSON *structurally impossible*. Caveat: an over-tight grammar can force the model down an unnatural path and *lower content* quality (it'll fill a required field with junk rather than refuse) — keep the grammar permissive on values, strict on structure, and still validate semantics in step 2.

6. **Build repair-and-retry as the BACKSTOP for paths with no hard guarantee.** Even guaranteed providers can return refusals or truncation; self-hosted/`json_object` paths can emit broken JSON. Ladder:
   1. **Salvage first (no LLM call):** strip ```` ```json ```` fences and any pre/post prose; extract the outermost balanced `{...}`/`[...]`; libraries: `json-repair` (py), `jsonrepair` (js) fix trailing commas/single quotes/unescaped newlines. Cheap, deterministic — try before re-prompting.
   2. **Re-prompt with the error:** on `ValidationError`, send the broken output + the exact validator message back ("Your previous reply failed validation: `<pydantic error>`. Return ONLY valid JSON matching the schema.") — Instructor's `max_retries` does exactly this. Cap at **2–3 attempts**; a 4th rarely converges.
   3. **Feed the schema again** in the repair turn; models drift from it across long contexts.

7. **Handle refusals, truncation, and length explicitly — they are NOT parse errors and must not be "repaired" into garbage.**
   - **Refusal:** OpenAI returns `message.refusal` (non-null) instead of `parsed`; Anthropic may emit a text block, not the forced tool. Branch on it — surface to the user/safety path, don't retry-loop into the rate limit.
   - **Truncation:** check `finish_reason == "length"` (OpenAI) / `stop_reason == "max_tokens"` (Anthropic) → the JSON is cut mid-object and is *un*-repairable by salvage. Fix the cause: raise `max_tokens`, shrink the schema/array, or paginate the extraction — don't json-repair a truncated object (it'll silently drop fields).
   - **Empty/whitespace:** treat as failure, retry once, then fail loud.

8. **Streaming structured output — buffer or parse incrementally, never `JSON.parse` mid-stream.** A partial stream `{"name":"Al` is invalid JSON. Two correct patterns:
   - **Simplest:** accumulate all deltas, parse once on `finish`. Use when you don't need live UI.
   - **Incremental:** `partial-json-parser` (js) / `pydantic` partial / OpenAI's streamed-parse helpers / `jsonriver` to coerce the buffer into a valid partial object on each delta for live rendering. For tool-calls, concatenate `tool_call.function.arguments` deltas across chunks (they arrive fragmented) and parse only when the tool-call completes.

9. **Verify.** (a) Round-trip: 100+ live calls on representative inputs → 100% deserialize into the typed model with zero `JSON.parse`/`ValidationError`; assert in CI against a recorded/replayed set. (b) Schema-escape probe: feed adversarial inputs that tempt the model to add commentary or a key — confirm `additionalProperties:false` + strict mode still yields clean objects. (c) Edge branches: force a refusal (policy-tripping input) and a truncation (tiny `max_tokens`) and assert each hits its dedicated branch, not the JSON repairer. (d) Enum closure: classification only ever returns a member of the enum. (e) Open-weights: same round-trip on the self-hosted path with constrained decoding ON, then OFF — confirm the constraint is what's carrying validity. Done = the program never sees a string it can't parse into the typed model, categories are closed enums, refusals/truncation route to their own branches, repair is rare (a metric you watch, not the load-bearing path), and the typed model — not the wire schema — is the single source of the contract.