--- name: linkedin-job-scraper description: Scrape LinkedIn job listings using the user's own logged-in browser session and compile them into a master Excel tracker. Use this skill whenever the user wants to search, scrape, collect, track, or monitor jobs on LinkedIn — including phrases like "find me jobs", "scrape LinkedIn", "job hunt", "look for openings", "update my job tracker", or any request to gather job postings by keyword, title, or location into a spreadsheet. Also use it when the user asks to re-run or refresh a previous LinkedIn job search. --- # LinkedIn Job Scraper Collect LinkedIn job listings (company, title, location, posting date, link, Easy Apply) through the user's own logged-in Chrome session, and append them to a master spreadsheet (`linkedin_jobs.xlsx`) that deduplicates across runs. ## Why this skill works the way it does LinkedIn actively detects bot-like behavior and may restrict accounts that trip its limits. Everything below — bulk extraction instead of clicking every card, randomized delays, session caps — exists to keep the user's account safe. Do not "optimize" these guardrails away, even if the user is impatient. If the user asks to exceed the caps, explain the account risk and get explicit confirmation first. The strategy: navigate to the **search results page** (which already contains all the data we need for ~25 jobs at once) and extract everything in one JavaScript pass per page. Never click into individual job postings — it's slow, expensive, and looks like a bot. ## Guardrails (non-negotiable defaults) - **Default 10 jobs per session.** The user may ask for any amount they wish, up to a hard limit of **100 per session** (≈4 result pages) — never exceed 100, even on request; explain the account risk instead. When the cap is below a full page (~25 jobs), still extract the whole page in one pass but keep only the newest jobs up to the cap (`sortBy=DD` puts newest first). - **Randomized 5–15 second delay between page navigations.** Compute the delay with code (e.g., `Math.random()*10+5` seconds), don't pick a round number. - **At most 2 scrape sessions per day.** If the master spreadsheet shows a scrape from earlier today, mention it before running another. - Scroll the results list gradually (the extraction script does this) rather than jumping instantly. ## Workflow ### Step 1: Gather search parameters Ask the user (skip anything they already provided): - **Keywords / job titles** (e.g., "product designer", "UX researcher") - **Location** (e.g., "Toronto, Ontario, Canada", "United States", "Remote") - Optional overrides: date-posted filter (default: **past week**), number of jobs (default: **10**, any amount up to the hard limit of 100), remote/hybrid/on-site filter Multiple keyword searches in one session are fine, but the session cap applies to the **total** across all searches. ### Step 2: Choose the browser interface **Default: Claude in Chrome** (`mcp__Claude_in_Chrome__*` tools — load via ToolSearch if deferred). It uses the user's real Chrome profile, so their LinkedIn session is already logged in. If Claude in Chrome is unavailable (extension not installed / not connected), check for a Playwright MCP and follow `references/playwright.md`. If neither exists, tell the user what to install — don't fall back to raw HTTP requests, which would bypass the logged-in session and look like scraping infrastructure to LinkedIn. ### Step 3: Build the search URL ``` https://www.linkedin.com/jobs/search/?keywords=KEYWORDS&location=LOCATION&f_TPR=r604800&sortBy=DD ``` - `keywords`, `location`: URL-encoded user input - `f_TPR`: date filter — `r86400` (24h), `r604800` (week, default), `r2592000` (month). Omit for "any time". - `sortBy=DD`: most recent first (important — keeps repeat runs hitting new jobs before old ones) - `f_WT`: workplace type, comma-separated — `1` on-site, `2` remote, `3` hybrid (e.g., `f_WT=2%2C3`) - Pagination: append `&start=25`, `&start=50`, … (25 jobs per page) ### Step 4: Navigate and verify login Navigate to the search URL. Get the page text and verify (a) the user is logged in (no "Sign in" wall) and (b) results loaded. If hit with a login wall or CAPTCHA, stop and ask the user to handle it manually in their browser — never attempt to log in or solve challenges yourself. ### Step 5: Extract in bulk via JavaScript (scroll–extract–merge loop) Read `references/extraction.js` first — its header documents four constraints discovered on a live run that will break a naive approach: 1. **No top-level `await`, and async continuations die when the JS call returns.** All extraction code must be synchronous. 2. **The results list is virtualized** — cards far from the viewport are de-rendered (id present, content empty). No single pass sees all ~25 jobs. 3. **Programmatic scrolling often doesn't trigger hydration.** Use real mouse-wheel scrolls (the computer tool's `scroll` action over the list, ~5 ticks), which is also more human-like. 4. **JS tool output truncates (~1000 chars).** Results accumulate in `window.__liAcc`; read out at the end via `substring` slices of ~950 chars. The loop per results page (batch the steps into one tool call where possible): 1. Run `extraction.js` once — installs `window.__liExtract()` and resets the accumulator. 2. Repeat: real-scroll down ~5 ticks over the list → wait ~2s → call `window.__liExtract()` (returns `new=X, total=Y, cardsInDom=Z`). Continue until `total` ≈ `cardsInDom` or two consecutive rounds add nothing. Hydration can lag a scroll — a final extra wait+extract catches stragglers. 3. Read out: `window.__liOut = JSON.stringify(Object.values(window.__liAcc))`, then `substring` slices. **If extraction captures 0 jobs:** LinkedIn has likely changed its DOM (it does this regularly). Don't give up — inspect the page (`get_page_text` / `read_page` or a small exploratory JS snippet), identify the current job-card selectors, and adapt. The data is always on the results page; only the selectors drift. The fallback chains in `extraction.js` show which selector families LinkedIn has used. ### Step 6: Paginate (respecting guardrails) If more jobs are needed to reach the cap and the page reported ~25 results, wait the randomized 5–15s delay, navigate to the next `start=` offset, and repeat Step 5. Stop when: cap reached, no new results, or all searches done. ### Step 7: Save and update the spreadsheet 1. Write all extracted jobs to a JSON file in the working directory (one array, include a `searchQuery` field on each job noting which keywords/location found it). 2. Run `scripts/update_spreadsheet.py`: ```bash python scripts/update_spreadsheet.py ``` The script appends to the master file (creating it with formatting on first run), **deduplicates by LinkedIn job ID**, converts relative dates ("3 days ago") to real dates, and prints a JSON summary (`added`, `duplicates_skipped`, `total_in_sheet`). The master file lives in the user's connected folder if one exists, otherwise the outputs folder — same location every run so dedupe works. Update the sheet **after each page** of extraction, not only at the end, so results accumulate in real time and nothing is lost if a session is interrupted. ### Step 8: Report Tell the user: how many new jobs were added, how many duplicates were skipped, total rows in the tracker, and present the spreadsheet file. Mention 2–3 notable postings (newest, or best keyword matches). Offer to schedule this as a recurring task if the user seems to want ongoing monitoring. ## Spreadsheet schema Columns (in order): Job ID, Company, Job Title, Location, Workplace Type, Easy Apply, Posted Date, Posted (raw), Job Link, Search Query, Date Scraped, Status, Notes. "Status" and "Notes" are left empty for the user's own application tracking — never overwrite them on existing rows. ## Edge cases - **Relative dates**: prefer the `