---
name: synthdata-tutorial
description: >
  Guided interactive walkthrough of the synthdata plugin's capabilities. Use this skill whenever the
  user asks for a "synthdata tutorial", "walkthrough", "demo", "guided tour", "show me what synthdata
  can do", "how does synthdata work", "getting started with synthdata", or wants to explore the
  synthetic-data skills step by step. Also trigger on "onboarding", "product tour", "feature overview",
  or "teach me how to generate fake data" in the context of synthdata / synthetic data generation.
version: 0.1.0
allowed-tools: Read Bash Glob Write
---

# Synthdata Interactive Tutorial

A hands-on walkthrough of the synthdata skills. The tutorial covers five core skills hands-on
(generate, extract, extend, anonymize, serve) and introduces the compute and prompt-builder skills
at the end. Rather than explaining features abstractly, the user works with real datasets as they
go — each step producing a file they can inspect. The arc: **tour templates → generate from
template → write a custom schema → extract → extend → anonymize → serve as MCP server**.

## Prerequisites

Check that the Python dependencies are installed:

```bash
python3 -c "import openpyxl, faker, numpy, pandas, yaml, mcp" 2>&1 && echo "DEPS_READY" || echo "DEPS_MISSING"
```

If `DEPS_MISSING`, tell the user to run:

```bash
pip install openpyxl faker numpy pandas pyyaml mcp --break-system-packages
```

Pick a scratch directory for tutorial outputs (default: `/tmp/synthdata-tutorial/`). Create it with
`mkdir -p` before Step 1.

## Starting the Tutorial

When triggered, introduce the walkthrough conversationally:

> I'll walk you through synthdata's key capabilities with a working example. We'll tour the built-in
> templates, generate a dataset, write a custom YAML schema for something outside the catalog,
> extract it to JSON, extend it with more rows, anonymize a "real-looking" file, and serve a
> dataset as an MCP server that Claude can query. About 15-20 minutes. Ready?

Wait for confirmation before proceeding. Also ask: **"Any particular domain you'd rather use?"**
— if they say e-commerce, healthcare, IoT, etc., swap the template in Step 1 accordingly. Otherwise
default to `hr-directory` (small, fast, easy to reason about).

---

## The Tutorial Arc

### Step 1: Tour the templates — "What can I generate out of the box?"

**Skill used:** `synthdata-generate`
**What it demonstrates:** The 12 built-in templates — each is a production-ready schema you can
use as-is or fork.

List the templates:

```bash
python3 <synthdata-generate>/scripts/generate.py --list-templates
```

Walk the user through the catalog. Group them so it's easier to scan:

**People & orgs**
- `hr-directory` — employees, departments (2 tables, classic starter)
- `crm-pipeline` — contacts, companies, deals, activities (sales pipeline)

**Commerce & finance**
- `ecommerce-orders` — customers, products, orders, order_items (4 tables, FK chains)
- `financial-transactions` — accounts, customers, transactions
- `saas-metrics` — accounts, users, events, subscriptions (usage + billing)

**Healthcare**
- `healthcare-patients` — patients, providers, encounters, claims
- `healthcare-hrm-security` — users, threat events, phishing sims, training, DLP, monthly risk
  (7 tables, richest template)

**Operations & telemetry**
- `security-events` — users, devices, alerts, incidents
- `log-events` — services, requests, errors
- `iot-sensors` — devices, readings, events
- `survey-responses` — respondents, questions, responses

**Custom starter**
- `blank-slate` — minimal schema to fork for your own domain

Explain the three knobs that apply to every template:
- **Template** — pick one above, or start from `blank-slate`
- **Effort** — `quick` (~50-100 rows), `medium` (~1K), `thorough` (~5K+). Scales row counts and
  event density but leaves schema shape intact.
- **Format** — xlsx (multi-sheet workbook with title banners), csv, json, sql, parquet.

Offer to peek inside a template YAML so they see what a schema actually looks like:

```bash
cat <synthdata-generate>/templates/hr-directory.yaml
```

Point out the anatomy: `name:`, `tables:`, per-table `rows:` (with quick/medium/thorough tiers),
`columns:` with typed fields (`id`, `faker`, `choice`, `int`, `float`, `date`), optional
`profiles:` (behavioral segments), `foreign_key:` links between tables, and `writers:` at the
bottom. This is the same format they'll write from scratch in Step 2.

Now generate the chosen template at `medium` effort:

```bash
python3 scripts/generate.py --template hr-directory --effort medium \
    --output /tmp/synthdata-tutorial/hr.xlsx --seed 42
```

Show the user the output — row counts per table, foreign-key integrity (e.g., every
`employee.department_id` resolves to a `departments.department_id`), and column distributions
(department weights, tenure mean, salary lognormal).

Bridge:

> Twelve templates cover a lot of ground, but not everything. What if you need something that isn't
> in the catalog — say, a fleet of delivery drivers with route assignments?

### Step 2: Custom schema — "Write your own YAML"

**Skill used:** `synthdata-generate`
**What it demonstrates:** The schema format is the real product. Templates are just pre-written
YAML — users can write their own for any domain.

Write a minimal two-table schema to a file. Use something concrete that isn't in the catalog so it
feels distinct from Step 1 — delivery drivers + trips works well:

```bash
cat > /tmp/synthdata-tutorial/drivers.yaml << 'EOF'
name: delivery-fleet
tables:
  - name: drivers
    rows: { quick: 20, medium: 100, thorough: 500 }
    columns:
      - { name: driver_id, type: id, prefix: "D", width: 4 }
      - { name: name, type: faker, method: name }
      - { name: license_state, type: choice, values: [CA, TX, NY, FL, WA], weights: [0.3, 0.25, 0.2, 0.15, 0.1] }
      - { name: hire_date, type: date, start: "2020-01-01", end: "2025-06-30" }
      - { name: rating, type: float, distribution: normal, mean: 4.6, sigma: 0.25, min: 1.0, max: 5.0 }
  - name: trips
    foreign_key: { column: driver_id, references: drivers.driver_id, distribution: zipfian, alpha: 1.3 }
    rows_per_parent: { distribution: poisson, lam: 25 }
    columns:
      - { name: trip_id, type: id, prefix: "T", width: 6 }
      - { name: ts, type: timestamp, start: "2025-01-01", end: "2025-12-31" }
      - { name: miles, type: float, distribution: lognormal, mean: 8, sigma: 0.6, min: 0.5 }
      - { name: on_time, type: bool, p: 0.87 }
writers: [xlsx]
EOF
```

Walk the user through what this schema is saying, line by line:

- **`drivers` is a parent table** — no `foreign_key`, so row count comes from `rows:` (100 at medium).
- **Column types** — `id` generates `D0001`..`D0100`; `faker` calls `fake.name()`; `choice` picks
  from a weighted list; `date` samples uniformly in a window; `float` with `distribution: normal`
  draws from a clipped Gaussian.
- **`trips` is a child table** — `foreign_key` links `trips.driver_id` → `drivers.driver_id`.
  `distribution: zipfian` means a few drivers get disproportionately many trips (realistic:
  full-timers vs. occasional drivers).
- **`rows_per_parent`** controls fan-out — Poisson with λ=25 means each driver averages ~25 trips,
  so `trips` will have ~2,500 rows total. The child's row count is *derived*, not specified.
- **`timestamp`** generates time-window events; **`lognormal`** gives a right-skewed distribution
  (most trips short, long tail of long trips); **`bool`** with `p: 0.87` gives 87% on-time rate.

Then run it through the generator with `--schema` instead of `--template`:

```bash
python3 <synthdata-generate>/scripts/generate.py --schema /tmp/synthdata-tutorial/drivers.yaml \
    --effort medium --output /tmp/synthdata-tutorial/drivers.xlsx --seed 42
```

Show the output — ~100 drivers, ~2,500 trips, FK integrity intact, on-time rate near 87%, trip
miles right-skewed.

Point out what else the schema format supports (so they know where to look next):
- **Profiles** — behavioral segments like `high_mileage` / `baseline` with conditional overrides
- **`weights_by_profile` / `p_by_profile`** — child-column values conditioned on parent's profile
- **`formula` columns** — derived from prior columns in the same row (e.g., `miles * 0.6` for pay)
- **`ref` columns** — copy a parent column down to the child via FK
- **More distributions** — `uniform`, `normal`, `lognormal`, `zipf`, `poisson`, `exponential`

Full reference: `<synthdata-generate>/references/schema-spec.md`,
`references/distributions.md`, `references/faker-fields.md`.

Bridge:

> So that's generate — templates for the common cases, custom YAML for everything else. But we've
> got Excel output; what if we need JSON for a downstream tool?

### Step 3: Extract — "Turn that into JSON"

**Skill used:** `synthdata-extract`
**What it demonstrates:** Excel → JSON with auto-detected title rows. Handles the synthdata-generate
convention (row 1 = banner, row 2 = headers) and plain sheets alike.

Extract the workbook from Step 1:

```bash
python3 <synthdata-extract>/scripts/extract.py --input /tmp/synthdata-tutorial/hr.xlsx \
    --output /tmp/synthdata-tutorial/json/
```

Show the user what was produced — one JSON file per sheet, row counts confirmed, ISO-8601 dates.
Open one of the JSON files and show the first 2-3 records so they can see the structure.

Point out:
- Auto-detection: row 1 is recognized as a merged title banner and skipped
- Date serialization: `hire_date` becomes `"2021-03-15"` not a binary Excel epoch
- Compactness flag: `--indent 0` produces one-line-per-record output for large datasets

Bridge:

> Now we have both the Excel and the JSON. What if 1,000 rows isn't enough — can we grow the
> dataset without regenerating from scratch?

### Step 4: Extend — "Add more rows and a new column"

**Skill used:** `synthdata-extend`
**What it demonstrates:** Growing existing datasets without touching original rows. Continues ID
numbering, respects FKs, infers distributions from existing columns.

Add 200 more employees:

```bash
python3 <synthdata-extend>/scripts/extend.py --input /tmp/synthdata-tutorial/hr.xlsx \
    --table employees --add-rows 200 \
    --output /tmp/synthdata-tutorial/hr_v2.xlsx
```

Show before/after row counts and confirm the new employee IDs continue the original sequence
(e.g., original max was `E01000`, new rows start at `E01001`).

Then add a whole new column — say, a `bonus_2026` field that follows a lognormal distribution:

```bash
python3 scripts/extend.py --input /tmp/synthdata-tutorial/hr_v2.xlsx \
    --table employees --add-column bonus_2026 --col-type float \
    --distribution lognormal --mean 5000 --sigma 0.6 \
    --output /tmp/synthdata-tutorial/hr_v3.xlsx
```

Point out two things:
- **ID continuity** — the extend script detected the existing `E#####` pattern and continued it
- **FK safety** — new rows only reference `department_id` values that already exist

Bridge:

> That covers making and growing synthetic data. But what if I already have real data and just want
> to scrub the PII out of it?

### Step 5: Anonymize — "Scrub the PII"

**Skill used:** `synthdata-anonymize`
**What it demonstrates:** Turning real data into safe synthetic data by replacing PII columns with
Faker values, while preserving row counts, numeric distributions, and join keys.

First, create a tiny "real-looking" CSV so the user has something to anonymize:

```bash
cat > /tmp/synthdata-tutorial/real.csv << 'EOF'
customer_id,full_name,email,phone,department,salary,hire_date
C001,Jane Doe,jane.doe@acme.com,555-123-4567,Engineering,95000,2020-03-15
C002,John Smith,john.smith@acme.com,555-987-6543,Sales,82000,2019-07-01
C003,Alice Wong,alice.w@acme.com,555-555-1212,Engineering,110000,2018-01-20
EOF
```

Run the scanner first — detect-only mode shows the user what the tool *thinks* is PII, before
touching the data:

```bash
python3 <synthdata-anonymize>/scripts/anonymize.py --input /tmp/synthdata-tutorial/real.csv --scan
```

Walk through the detection report:
- `full_name` → `name` (name keyword match)
- `email` → `email` (keyword + regex match, HIGH confidence)
- `phone` → `phone_number` (keyword + regex match, HIGH confidence)
- `department`, `salary`, `hire_date` → not flagged (pass through)

Then run the anonymization:

```bash
python3 scripts/anonymize.py --input /tmp/synthdata-tutorial/real.csv \
    --output /tmp/synthdata-tutorial/anon.csv \
    --keep department,salary,hire_date,customer_id --seed 42
```

Show the before/after side-by-side. Point out:
- **Names/emails/phones replaced** with plausible Faker values
- **Salaries preserved** exactly — numeric distributions shouldn't be disturbed by PII scrubbing
- **Department and hire_date preserved** — analytical value retained
- **Customer IDs preserved** — join keys stay stable (use `--preserve-joins` to keep them consistent
  across multiple tables when anonymizing relational datasets)

### Step 6: Serve — "Let Claude query the data directly"

**Skill used:** `synthdata-serve`
**What it demonstrates:** Turning a dataset into a read-only MCP server that Claude can query
using auto-generated tools — no code generation, no manual tool definition.

First, inspect the dataset to see what will be served:

```bash
python3 <synthdata-serve>/scripts/serve.py --inspect \
    --input /tmp/synthdata-tutorial/hr.xlsx
```

Walk through the output — table names, column counts, row counts, dtypes, and a 3-row sample per
table. This is what Claude will be able to query.

Now explain the MCP configuration. To wire this up for Claude Code, you'd add to `.mcp.json`:

```json
{
  "mcpServers": {
    "synthdata-hr": {
      "command": "python3",
      "args": ["<absolute-path-to>/scripts/serve.py", "--input", "<absolute-path-to>/hr.xlsx"]
    }
  }
}
```

After restarting Claude Code, five tools become available:
- **`list_tables`** — see what tables exist and their shapes
- **`describe_table`** — column details, sample values, FK hints
- **`query_table`** — filter rows (e.g., `salary > 100000`), select columns, sort, paginate
- **`sample_rows`** — grab N random rows
- **`get_stats`** — mean, std, percentiles for numeric columns; top values for categorical

Show that you can also export a standalone server that works without the synthdata plugin:

```bash
python3 <synthdata-serve>/scripts/export.py \
    --input /tmp/synthdata-tutorial/hr.xlsx \
    --output /tmp/synthdata-tutorial/hr-server/ \
    --name synthdata-hr-demo
```

Walk through what was produced:
- `server.py` — self-contained MCP server (no synthdata dependency)
- `data/dataset.json` — the dataset embedded as JSON
- `requirements.txt` — just `mcp`, `pandas`, `numpy`, `openpyxl`
- `README.md` — config snippets for Claude Code and Claude Desktop

Point out:
- **Any format works** — xlsx, csv, json, even parquet (with pyarrow installed)
- **Read-only** — the server never modifies your data
- **Dataset-derived name** — the server name defaults to `synthdata-<filename>` so you can run
  multiple servers for different datasets simultaneously

Bridge:

> So now you can generate data *and* make it queryable — Claude can explore it, filter it, pull
> stats, all through MCP tools. That's the full hands-on loop.

---

## Wrapping Up

After Step 6, close with a brief recap:

> That's the core synthdata loop: pick a **template** (or write your own YAML), **generate** a
> dataset, **extract** it to another format, **extend** it without regenerating, **anonymize**
> real data to make it shareable, and **serve** it as an MCP server so Claude can query it live.
>
> **Beyond the basics:** Two more skills round out the toolkit:
> - **synthdata-prompt-builder** — start here for complex scenarios. Describe your domain and
>   what downstream tools need, and it'll plan which generation and computation steps to run.
> - **synthdata-compute** — derive aggregated or scored tables from raw data. Monthly rollups,
>   composite risk scores, percentile ranks, segment summaries — anything that requires reading
>   generated data back and transforming it.
>
> Together, these eight skills cover the full lifecycle: plan what you need, generate raw data,
> compute derived metrics, extract to other formats, extend when you need more, anonymize
> real data to make it safe, and serve any dataset as a queryable MCP server.
>
> Want to try a different template, write a custom schema, or plan a multi-step workflow? Just ask.

## Pacing and Interaction

- Pause between steps for acknowledgment ("yes", "go on", "next")
- If a step's command fails (e.g., dependency missing), diagnose before moving on
- Keep all tutorial outputs in the chosen scratch directory so the user can inspect them afterward
- The whole walkthrough is 12-15 minutes at conversational pace, ~7 minutes if the user is moving fast
- If the user wants to jump straight to one skill, skip the others — this is a guide, not a script
- If the user already knows the template catalog, collapse Step 1's tour and move to the generate command