---
name: forge-web-search
description: Using web search inside AI agents. Query construction, source triage (the credibility hierarchy), anti-SEO-spam, fetch-vs-snippet decision, citation hygiene, freshness signals, fetch budget. Contains domain-credibility table + worked agent loop. Use whenever an LLM agent calls a search tool.
license: MIT
---

# forge-web-search

You are an agent (or a developer of one) calling a web search tool. Default agent-written search loops issue a single broad query, take the top three results at face value, and quote SEO-spam content as if it were authoritative. This skill exists to turn web search from a noise source into a signal source.

The mental model: **a search result is a hypothesis, not a fact.** The snippet is a teaser; the actual content can be different. The source's credibility is part of the result, not a separate question.

## Quick reference (the things you must never ship)

1. A search loop with no cap on iterations.
2. A search loop with no cap on `fetch()` calls.
3. Quoting the search-result snippet without fetching the actual page.
4. Citing a URL the agent never visited.
5. Trusting a content-farm result equally to the official docs.
6. Single-search synthesis on a consequential question (no counter-search).
7. Ignoring the publication date on fast-moving topics.
8. SEO listicles ("Top 10 Best X in 2025") accepted as research.
9. Citing the LLM itself ("Claude told me...") as a source.
10. The same search re-issued multiple times within one session (no caching).

## Hard rules

### Query construction

**1. Specific terms over natural-language questions.** `Postgres 17 json_table example` beats `how do I use the new json feature in Postgres?`.

**2. Quote known phrases.** `"json_table"` is exact-match.

**3. Use site filters when you know the canonical source.** `site:postgresql.org`, `site:developer.mozilla.org`, `site:github.com`.

**4. Use date filters for fast-moving topics.** `after:2025`, `before:2024`.

**5. Avoid vague qualifiers.** `best`, `top`, `popular` invite SEO spam. Replace with concrete criteria.

**6. Iterate based on results.** First query reveals vocabulary; second uses it. Three rounds is normal for a meaningful question.

### Source triage

**7. Triage before deep-reading.** Look at domain, snippet, date. Discard SEO-spam patterns: AI-content farms, listicles, "ultimate guide" titles, recently-registered domains.

### Domain credibility hierarchy

| Tier | Examples | Trust signal |
| --- | --- | --- |
| **Highest** | official docs (postgresql.org, mdn, w3.org), language/framework foundations, primary repos on GitHub | Direct source of truth |
| **High** | Stripe / Cloudflare / Vercel engineering blogs, expert personal blogs (Julia Evans, Hillel Wayne), peer-reviewed papers | Domain expert, often a maintainer |
| **Medium** | Stack Overflow (high-voted answers), Hacker News discussions, GitHub issues | Community signal, varies |
| **Low** | dev.to, Medium, generic blog farms | Often AI-summarized, often shallow |
| **Skip** | SEO listicles, "top 10" sites, freshly-created domains, content farms | Adversarial signal |

**8. Prefer multiple corroborating sources for non-trivial claims.** A single Stack Overflow answer is one data point. Same answer from docs + SO + maintainer blog is strong signal.

### Fetching vs snippet-reading

**9. Read the snippet first. Fetch the full page only when the snippet is insufficient.** Many questions are answered by snippets alone.

**10. Fetch the page when you need: code samples, exact quotes, structured tables, anything that does not fit in a snippet.**

**11. Strip noise after fetching.** Cookie banners, nav, ad inserts, "related articles" pollute. Most fetchers do basic noise removal.

**12. Respect robots.txt and rate limits.** Slamming a site with 50 fetches in 5 seconds is hostile.

### Citation hygiene

**13. Quote URLs verbatim, not paraphrased.** A "from the Postgres docs" without a link is not a citation.

**14. Note publication or access date.** Especially for fast-moving topics. `[Postgres 17 docs, accessed 2026-05-22]`.

**15. If you quote, quote exactly.** A close paraphrase wearing quote marks is misattribution.

**16. Verify URLs resolve before citing.** A 404 in your citation is embarrassing.

### Anti-hallucination

**17. Cited URLs were actually visited or are in the search results.** An LLM agent can fabricate plausible URLs; surface in audit logs and verify.

**18. Quoted text appears in the source.** Cross-check before quoting.

**19. Statistics from "the source" are in the source.** "According to a 2024 study" without a link is no source.

### Freshness

**20. Software / API topics: prefer content under 12 months old.** Frameworks change.

**21. Mathematics, physics, established protocols: age less important.** A 2010 paper on TCP is fine.

**22. "Current state of the world" (laws, prices): always check the date.**

**23. When you do not know the date and it matters, search for it.**

### Adversarial searches

**24. For consequential conclusions, run a counter-search.** "Why X is good" AND "why X is bad." Cheapest hedge against confirmation bias.

**25. Read the comments on contested claims.** Sometimes the comments contain the nuance the post got wrong.

**26. Check who is funded by whom.** A vendor's blog post about their competitor is signal-but-biased.

### Output

**27. Pull the answer out of the noise.** Do not return "here are 10 search results, decide for yourself." Synthesize.

**28. List sources used, not all sources visited.**

**29. Surface conflicts when found.** "Source A says X; Source B says Y; the docs do not address this directly."

### Cost and latency

**30. Search is rate-limited and costs money. Cache within a session.** Cap searches per task.

**31. Cap fetches per task too.** 30 pages to answer one question is overkill; 3-5 is typical.

**32. Use search APIs that return structured data when possible.** Brave Search, Tavily, Exa. Cleaner than scraping Google.

### Specific failure modes

| Failure | What's happening | Defense |
| --- | --- | --- |
| Snippet says X, full page says nothing about X | Bing/Google sometimes machine-generates snippets | Fetch to verify |
| Top result is SEO content farm | Spam optimized for ranking | Skip; second page if needed |
| Correct in 2022, wrong in 2026 | Outdated | Date filter |
| Source is AI-laundered (LLM output republished) | Low-value, often wrong | Check author + publication |

## Worked example: agent search loop with budgets

```ts
const MAX_SEARCHES = 5;
const MAX_FETCHES  = 6;

async function research(question: string): Promise<{ answer: string; sources: string[] }> {
  let searches = 0;
  let fetches  = 0;
  const visitedUrls = new Set<string>();
  const sources: { url: string; title: string; excerpt: string }[] = [];

  // Initial broad query
  let query = await rewriteForSearch(question);

  while (searches < MAX_SEARCHES) {
    const results = await webSearch(query, { limit: 10 });
    searches++;

    // Triage: drop SEO spam, listicles, content farms
    const filtered = results.filter((r) => {
      if (/top \d+ best/i.test(r.title)) return false;
      if (/ultimate guide/i.test(r.title)) return false;
      if (visitedUrls.has(r.url)) return false;
      return true;
    });

    // Fetch the top 2 high-credibility results we have not visited
    for (const r of filtered.slice(0, 3)) {
      if (fetches >= MAX_FETCHES) break;
      if (visitedUrls.has(r.url)) continue;
      visitedUrls.add(r.url);
      try {
        const text = await fetchAndExtract(r.url);
        fetches++;
        sources.push({ url: r.url, title: r.title, excerpt: text.slice(0, 1000) });
      } catch {
        // 404 or unreachable; skip
      }
    }

    // Decide: have we answered? If not, refine the query
    const verdict = await checkSufficient(question, sources);
    if (verdict.sufficient) break;
    query = verdict.refined_query;
  }

  // Counter-search if the question is consequential
  if (await isConsequential(question) && fetches < MAX_FETCHES) {
    const counterQuery = await generateCounterQuery(question);
    const counterResults = await webSearch(counterQuery, { limit: 5 });
    searches++;
    for (const r of counterResults.slice(0, 2)) {
      if (fetches >= MAX_FETCHES) break;
      if (visitedUrls.has(r.url)) continue;
      visitedUrls.add(r.url);
      const text = await fetchAndExtract(r.url);
      fetches++;
      sources.push({ url: r.url, title: r.title, excerpt: text.slice(0, 1000) });
    }
  }

  const answer = await synthesize(question, sources);
  return { answer, sources: sources.map((s) => s.url) };
}
```

What this demonstrates: explicit search + fetch caps (rules 30-31); triage of SEO spam (rule 7); visited-URL deduplication (rule 31); iterative refinement (rule 6); counter-search for consequential questions (rule 24); synthesizes, does not dump raw results (rule 27).

## Workflow

When using web search in an agent loop:

1. **Build the query.** Specific, with site or date filter if helpful.
2. **Issue the search. Read snippets.**
3. **Triage: discard spam, identify primary sources.**
4. **Fetch one or two high-credibility pages if needed.**
5. **Cross-check the claim against a second source for consequential conclusions.**
6. **Synthesize. Cite. Surface uncertainty.**

## Verification

```bash
bash skills/research/forge-web-search/verify/check_web_search.sh path/to/agent.ts
```

Flags: search loops without iteration cap, fetch loops without cap, code that quotes without verifying URL existence.

## When to skip this skill

- Searches inside private indexes (your docs, your code) - that is RAG; see [`forge-rag`](../../llm/forge-rag/SKILL.md).
- Trivial fact lookups where the snippet is obviously sufficient.
- Tasks where the user provided the source material.

## Related skills

- [`forge-research`](../forge-research/SKILL.md) - the methodology layer.
- [`forge-citation`](../forge-citation/SKILL.md) - the citation layer.
- [`forge-rag`](../../llm/forge-rag/SKILL.md) - retrieval over a private corpus.
- [`forge-tool-use`](../../llm/forge-tool-use/SKILL.md) - the agent loop the search lives in.