---
name: position-insert
description: "The 5-gate sequence the Scout runs for EACH candidate position before INSERTing into `positions`: dedup → link verification → JD fetch → permissive filters → INSERT. Skipping any gate fills the DB with duplicates, dead links, or out-of-scope rows that the Analyst then has to drop — wasted Sonnet budget downstream. Owned by the Scout role; pair with `circles-and-sources` (decides WHERE to look) and `scout-coord` (decides WHO looks where)."
allowed-tools: Bash(curl *), Bash(python3 *), Bash(grep *)
---

# position-insert — 5 gates per position

A position is worth inserting only if all five gates pass. The order matters: the cheaper checks come first so the expensive ones (full JD fetch + filtering) run only on viable candidates.

## Gate 1 — Dedup (cheap, mandatory first)

```bash
python3 /app/shared/skills/db_query.py check-url <linkedin_id_or_url>
```

- Output `TROVATA` → **SKIP** (already in DB, possibly different status — never re-insert).
- Output `NON TROVATA` → proceed to Gate 2.

The dedup key is the canonical URL (or LinkedIn job ID for LinkedIn). If the same posting comes from two different sources (e.g. company career page AND a LinkedIn cross-listing), `check-url` deduplicates.

## Gate 2 — Link verification (HTTP + URL)

Two-step `curl` to detect dead postings AND silent redirects to a generic `/careers` page (= job removed but page returns 200).

### Step 2a — status code + final URL

```bash
curl -s -o /dev/null -w "HTTP:%{http_code} URL_FINALE:%{url_effective}" \
  -L -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)' '<URL>'
```

| Result                                        | Action                                         |
|-----------------------------------------------|------------------------------------------------|
| `HTTP:404` / `HTTP:410`                       | SKIP (link morto)                              |
| `HTTP:301/302` to a generic `/careers` or `/jobs` | SKIP (posizione rimossa, redirect generico) |
| `HTTP:200/301/302` final URL = posting page  | proceed to Step 2b                             |

### Step 2b — content signals

```bash
curl -s -L -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)' '<URL>' \
  | grep -i 'no longer accepting\|closed-job\|position has been filled\|expired\|job not found'
```

- Match → SKIP (closed job)
- No match → proceed to Gate 3

### Workable note

For ATS hosted on Workable: there are **two** URLs per posting. Use the right one:
- `apply.workable.com/...` → apply form: returns `302` when the job is closed (looks like a dead link, false positive).
- `jobs.workable.com/...` → canonical JD page: HTTP 200 + valid JSON-LD if the position is alive.

Always verify the **canonical** page (`jobs.workable.com`), not the apply form. Same principle for Greenhouse, Lever, Ashby.

## Gate 3 — Fetch the FULL JD

The DB contract requires `--jd-text` and `--requirements` to be COMPLETE — partial scrapes break the Analyst downstream.

```bash
# tier 1 — curl with browser UA (most cases)
curl -s -L -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)' '<URL>' > $JHT_AGENT_DIR/tmp/jd-raw.html

# tier 2 — JS-heavy pages (Wellfound, some custom careers): use playwright MCP
# tier 3 — fallback: WebFetch / WebSearch
```

Extract the **full text body** (not just the title) and the **requirements section** (skills, years of experience, languages). If the page has a clear "Requirements" / "Must have" / "What you'll bring" section, scrape it verbatim into `--requirements`.

Blocked sites (do NOT use `fetch` MCP, blocked by robots.txt):
- `linkedin.com` → use `linkedin_check.py` (authenticated) or `curl` with browser UA
- `wellfound.com` → use `playwright` or `curl`

## Gate 4 — Permissive Scout-level filters

Apply ONLY the four totally-out-of-scope filters (full table in `circles-and-sources` skill). Skip if:

- Title contains explicitly: `senior`, `lead`, `staff`, `principal`, `head of`, `director`
- Geographic work-auth incompatible (`US-only` / `Canada-only` and candidate has no visa)
- Domain completely outside IT/coding (and candidate is in IT)
- Hard requirement of `> real_years + 3` years of experience

Everything else: pass through to Gate 5. **Don't do the Analyst's job** — adjacent stacks, near-fits, slight gaps are all `checked` material; the Scorer applies the gap penalty.

## Gate 5 — INSERT

```bash
python3 /app/shared/skills/db_insert.py position \
  --title "<TITOLO>" \
  --company "<AZIENDA>" \
  --url "<URL canonica, NON apply form>" \
  --location "<location reale dalla JD>" \
  --remote-type <full_remote|hybrid|on_site> \
  --source <slug fonte: linkedin|greenhouse|lever|indeed|wellfound|remoteok|...> \
  --found-by $MY_ID \
  --jd-text "<TESTO COMPLETO DELLA JD>" \
  --requirements "<stack + requirements estratti dalla JD>"
```

**All flags are mandatory** — `--jd-text` empty or `--url` missing means the Analyst can't do its job. The `db_insert.py` script enforces non-empty values; if it rejects your call, fix the input — never bypass with raw SQL.

## DB write boundary (T05 + role)

The Scout writes ONLY:
- `positions` (INSERT, never UPDATE except for the dup-recovery case below)

NEVER touches:
- `companies` (Analyst territory)
- `scores` (Scorer)
- `applications` (Scrittore)
- `position_highlights` (Analyst)
- positions with `status != 'new'` (already moved downstream, hands off)

### Dup recovery (the only allowed UPDATE)

If you accidentally inserted a duplicate (Gate 1 was wrong, e.g. a normalised URL slipped through), you can mark the duplicate as excluded — but never DELETE:

```bash
python3 /app/shared/skills/db_update.py position <DUP_ID> --status excluded \
  --notes "DUPLICATA di #<ORIGINAL_ID>"
```

`DELETE` / `DROP` SQL is forbidden (T02 + DB safety). Rewinds via `excluded` notes are auditable; deletes are not.

## After the INSERT — notify Analysts

After every batch of 3-5 inserts, ping the Analyst sessions with the ID range. They pick up `status=new` from the DB anyway, but the ping shortens the latency:

```bash
jht-tmux-send ANALISTA-1 "[@$MY_ID -> @analista-1] [INFO] Batch 5 posizioni inserite (IDs: X-Y)"
```

If you have 2 Analysts, alternate the ping target to balance load (Analysts also have `last_checked` claim coordination so it's never wrong, but tmux notification helps responsiveness).

## Anti-patterns

- ❌ Skipping Gate 1 "because it looked new" — `check-url` is cheap, always run it.
- ❌ Inserting with empty `--jd-text` "I'll fill it later" — there is no later, the Analyst processes it next.
- ❌ Verifying with `curl` without `-L` — a 302 to a generic `/careers` looks alive without follow-redirect; you'd insert a dead JD.
- ❌ Verifying the apply form on Workable instead of the canonical JD page — false-positive dead links.
- ❌ Using `fetch` MCP on `linkedin.com` / `wellfound.com` — blocked, gets you a 403 banner instead of the JD.
- ❌ Bypassing the wrapper with `python3 -c "import sqlite3; INSERT ..."` — breaks dedup invariants and `found-by` tracking.
- ❌ Setting `--status` to anything other than the default `new` (the Scout never sets status manually; the wrapper handles it).

## See also

- `circles-and-sources` — what to search WHERE (this skill is what to do AFTER you find a candidate posting).
- `scout-coord` — boot-time partition (this skill is per-position, downstream of partition).
- `db-insert` — the wrapper internals + `position` schema.
- `agents/_manual/anti-collision.md` — broader Scout coordination contract.
- `agents/scout/scout.md` — the orchestrator prompt that calls this skill in the main loop.