---
name: extract-structured-data
description: Get structured data (JSON) out of any website, end to end. Use when the user wants data from a site — products, listings, prices, profiles, search results — and you need the most reliable extraction path, not just a raw page fetch.
---

# Extract structured data with Better Fetch

The reliable order of attack is API-first: most sites render from an internal JSON API, and calling that beats parsing HTML every time. Work down this ladder, stopping at the first rung that delivers.

## The ladder

1. **Find the data source.** Call `discover_apis` on the page with `include_bodies: true` and a `session` string. Scan the `body_preview` of each call for the data you're after.
2. **Call it directly.** Call the winning endpoint with `scrape_json`, passing the **same `session`** (the page load's cookies often authorize the API call) and any `extra_headers` the endpoint needs — a `Referer`, an `x-api-key`, a token visible in the captured URL.
3. **Fall back to the rendered page.** No clean endpoint? Use `fetch_url` — default `format: "text"` for prose, `format: "html"` when you need to parse structure — with `wait_selector` set to a selector that only exists once the data has rendered.
4. **Blocked anywhere?** Add `country` (matching the target's region) + `session` and retry — that enables automatic block-retry with headed escalation. See `bypass-bot-walls`.

## Scaling and hygiene

- Many pages of results? Once step 2 works for page one, pagination is usually a single query param — iterate the API call, not the pages. For page-by-page crawling, see `crawl-pages`.
- Keep one `session` per site per task; never share it across unrelated tasks.
- Every call is metered: prefer one `scrape_json` over re-fetching HTML, and keep `max_chars` modest on triage fetches.