---
name: defend-llm-top-10
description: "Defensive playbook for the OWASP Top 10 for LLM Applications (2025), centered on prompt injection (MITRE ATLAS AML.T0051, direct and indirect), RAG retrieval abuse (LLM08), and improper output handling (LLM05). Prompt injection has no patch, so defense is logging, blast-radius design, and breaking the lethal-trifecta exfiltration path, as seen in EchoLeak (CVE-2025-32711). Keys off LLM app telemetry: prompt and completion logs, tool-call audit events, RAG retrieval logs, guardrail verdicts, the Purview unified audit log for Copilot, and Azure OpenAI diagnostics. Ships a canary-token tripwire, an egress/exfiltration detection, a tool-call fan-out rule, required logging, and a mitigation table. Use when threat-modeling an LLM, RAG, or agent feature, deciding what to log, writing detections for prompt injection or system-prompt leakage, hunting copilot data exfiltration, or triaging an EchoLeak-style incident."
---

# OWASP LLM Top 10 - Detection and Response for Defenders

Your organization is shipping LLM features whether security signed off or not: chatbots, copilots, RAG over internal documents, agents with tool access. This skill reads the OWASP Top 10 for LLM Applications (2025) as a defender's checklist and ships the detections that work today. It is defensive: it describes injection classes only to the depth needed to log, detect, and contain them, and provides no working injection payloads, jailbreak strings, or evasion techniques.

## TL;DR for defenders

Treat every LLM in your stack as an untrusted component that will eventually follow someone else's instructions. There is no patch for prompt injection: the model reads system prompt, user message, retrieved document, and tool output as tokens in one context window, so instruction-following is a statistical tendency, not a security boundary. What works: least-privilege tool scopes, output handling as strict as you apply to user input, breaking the data-exfiltration leg (egress controls on what the model can render or call), a **canary token** in the system prompt as a leak tripwire, and logging prompts, completions, and tool calls as first-class audit events. The canary (rule 1 below) is the highest-precision, lowest-effort detection; the egress rule (rule 2) catches the EchoLeak-style exfiltration it misses.

## Why this belongs in your threat model now

LLM-integrated apps stopped being experimental the moment they got real data and real actions. The clearest proof point is **EchoLeak** ([CVE-2025-32711](https://nvd.nist.gov/vuln/detail/CVE-2025-32711), CVSS 9.3): a zero-click flaw in Microsoft 365 Copilot disclosed June 2025 where a crafted email made Copilot exfiltrate the user's mail, files, and chat history with no click. Microsoft fixed it server-side, but the class (an "LLM scope violation") is the point: untrusted content steering a model that holds privileged access. Indirect prompt injection was described systematically by Greshake et al. in early 2023 and has topped every edition of the OWASP LLM Top 10 since. The gap between "known since 2023" and "still shipping exploitable in a 2025 flagship product" is why detection and blast-radius design outrank waiting for a fix.

The 2025 list in production terms, with the primary defense for each:

| Risk | What it looks like in production | Primary defense |
|---|---|---|
| LLM01 Prompt Injection | Untrusted content (user input, a retrieved doc, an email) steers behavior | Least privilege + egress control; assume it happens |
| LLM02 Sensitive Information Disclosure | Model reveals other users' data, internal docs, or given credentials | Data minimization; do not give the model secrets |
| LLM03 Supply Chain | Poisoned or backdoored third-party model, adapter, or dataset | Pin versions, verify sources, inventory models like packages |
| LLM04 Data and Model Poisoning | Attacker-influenced training or fine-tuning data changes behavior | Provenance and validation on anything you train on |
| LLM05 Improper Output Handling | Model output flows into HTML, SQL, or a shell without encoding | Treat output as untrusted input; encode and parameterize |
| LLM06 Excessive Agency | Model can call tools or APIs with more permission than the task needs | Scope tools per task; human approval for consequential actions |
| LLM07 System Prompt Leakage | Instructions, and any secrets inside them, get echoed out | No secrets in prompts; canary tripwire (below) |
| LLM08 Vector and Embedding Weaknesses | RAG pulls attacker-planted or cross-tenant content into context | Access control at retrieval time, source tagging |
| LLM09 Misinformation | Confident wrong answers drive real decisions | Grounding, citations, human review where it matters |
| LLM10 Unbounded Consumption | Token-burning denial of service or a shocking bill | Rate limits, quotas, per-identity cost caps |

Half of these are old vulnerability classes wearing a new interface: LLM05 is unescaped output, LLM03 is supply chain, LLM10 is resource exhaustion. The genuinely new ones are LLM01, LLM06, and LLM08, and they compound each other.

A useful frame for when indirect injection becomes a real breach is the **lethal trifecta** (Simon Willison): an AI system that combines (1) access to private data, (2) exposure to untrusted content, and (3) a way to communicate externally. All three together means attacker instructions can reach private data and carry it out. EchoLeak was exactly this inside Copilot: the crafted email was the untrusted content, the user's mail and files were the private data, and image/link rendering was the exfiltration channel. You usually cannot remove legs one and two. Defense concentrates on leg three.

## Log sources required

You cannot detect what you do not log, and most LLM apps today log almost nothing security-relevant. LLM applications do not share a standard log format yet, so adapt the field names below to your own telemetry. Whether you build or buy, insist on these as structured, retained, queryable events:

| Event | Fields you lean on | Why it matters |
|---|---|---|
| Prompt + completion | user identity, session ID, model and version, hash of the system prompt in effect, prompt text, response text | The record of what was asked and answered. This is sensitive data: scope retention and access accordingly. |
| Tool / function call | tool name, arguments, session ID, allowed/denied verdict | The agent's process-creation log equivalent; it is where impact happens. |
| RAG retrieval | document ID, source, retrieving query, session ID | "What did the model read" is the first incident question and is unanswerable without this. |
| Guardrail / filter verdict | verdict (block/flag/rewrite), rule, session ID, whether the request still proceeded | Your IDS alerts for the LLM; today most go nowhere. |

Microsoft ecosystem specifics: Copilot interactions surface in the **Microsoft Purview unified audit log** (operations such as `CopilotInteraction`), and **Azure OpenAI** deployments emit request logs through Azure diagnostic settings. Check what you actually collect; the default is usually not enough, and for vendor-hosted copilots you get what Purview exposes, not raw model input/output.

## Detection

Three layered detections, highest precision first. These schemas are illustrative, not canonical Sigma: LLM telemetry has no shared format, so map the field names to yours. No LLM-specific rule ships in the UMBRASEC [sigma-pack](https://umbrasec.dev/tools/sigma-pack/) yet, so nothing below is a reused pack rule.

### 1. Canary token in the system prompt (near-zero false positives)

The honeypot-SPN trick, ported to LLMs. Plant a unique, meaningless marker in your system prompt and alert if it ever appears in model output, because no legitimate completion has a reason to contain it. One marker gives two strong signals at once: your system prompt is leaking (LLM07), and the model is disclosing context it was told not to, which usually means an injection is steering it.

```text
# System prompt (excerpt)
# Integrity marker. Never include the following token in any response:
# UMB-CANARY-3f91c2a8

# Detection (pseudo-rule over completion logs)
alert when:
    response.text contains "UMB-CANARY-"
severity: high
action: capture full session (prompt chain, retrieved docs, tool calls)
        for review; rotate the canary after any hit
```

The same idea extends to RAG: seed a **decoy document** no honest query should retrieve, and alert when it enters a context window. That is your honeypot for LLM08-style retrieval abuse.

### 2. Egress: watch what the model is allowed to render or call

EchoLeak exfiltrated through a rendered image URL, the data leaving in the query string of a request the victim's client fired automatically. That channel generalizes: markdown images, auto-fetched links, and tool calls that take URLs are the standard exfiltration legs of the trifecta. Lock them down by policy, and alert on the attempts.

```text
# Output policy (enforce before rendering / before the HTTP client fires)
- strip or neutralize markdown images and links in model output
  unless the host is on an explicit allowlist
- block tool calls whose URL/domain argument is not allowlisted

# Detection (pseudo-rule over output + tool-call logs)
alert when:
    output.contains_image_or_link AND destination.host not in allowlist
or:
    toolcall.name in ("http_get", "browse", "send_email")
    AND toolcall.argument_domain not in allowlist
    AND session.context_included_external_content == true
```

That last condition is the high-signal one: the model read untrusted content, then immediately tried to reach an unfamiliar destination. Legitimate sessions do this rarely; injected ones do it by construction. This is the MITRE D3FEND **Outbound Traffic Filtering (D3-OTF)** countermeasure applied to the model's rendering and tool-call surface.

### 3. Behavioral: tool-call fan-out and sequence anomalies

The agent equivalent of a Kerberoasting fan-out rule. A copilot answering a question touches one or two tools; a hijacked agent enumerates, reading many documents and then calling a send/export/write tool it rarely uses. Baseline per tool and per identity, then alert on outliers.

```text
# Pseudo-aggregation over tool-call logs
alert when, within 5m, a single session:
    calls >= N distinct tools            # fan-out (baseline N first)
    or reads >= M distinct documents     # bulk context-stuffing
    or invokes a "consequential" tool    # send_email, create_rule,
       it has never used before          # export, delete, payment
severity: medium - this is a triage feed, not a pager rule
```

## Tuning and false positives

- **Rule 1 (canary).** Use a random value per deployment, rotate it on a schedule and after every hit, and make sure it is excluded from any prompt text you intentionally publish. A hit is near-binary; treat it as an incident, not an alert to batch.
- **Rule 2 (egress).** Build the host allowlist from your real integrations first, or every legitimate outbound link becomes noise. The compound condition (untrusted content in context AND unfamiliar destination) is what keeps precision high; the bare "link to non-allowlisted host" variant is noisier and belongs in report-only until tuned.
- **Rule 3 (behavioral).** Run it report-only for a couple of weeks exactly like a SIEM correlation rule: find your noisy legitimate automations, allowlist them, and set `N` and `M` above your noisiest honest session. These thresholds are dials, not gospel.

## Threat hunt

The canary catches leakage, not all injection; an attack that quietly biases an answer or exfiltrates without echoing the prompt walks past rule 1. Hunt the quiet cases:

1. Over a long window, join RAG retrieval logs to completion and tool-call logs by session ID. Flag sessions where the context included externally sourced or user-supplied content and the same session then invoked a network-touching or consequential tool.
2. Rank those sessions by the privilege of the identity the agent acted as and the sensitivity of the documents retrieved. An agent running with a broad mailbox or export scope that read untrusted content outweighs any volume signal.
3. Review guardrail verdicts that fired but let the request proceed. A cluster of flags against one session or one data source is a lead even when nothing was blocked.
4. For Microsoft 365 Copilot, pull `CopilotInteraction` events from the Purview unified audit log and look for interactions whose source content came from external mail or shared files immediately before sensitive-data access.

## Mitigation

Design beats detection here: more than in any other area, the architecture is the control. Mappings below follow the control codes used in the UMBRASEC [identity and access guide](https://umbrasec.dev/guide/identity-and-access.html) and [detection and response guide](https://umbrasec.dev/guide/detection-and-response.html); a cell is "-" where no single listed control maps cleanly, and a D3FEND countermeasure is named where one fits. The joint NCSC/CISA Guidelines for Secure AI System Development and the NIST AI RMF are the formal backing when you write policy.

| Control | What it does | Essential Eight | ISO 27001:2022 | NIST CSF 2.0 | D3FEND |
|---|---|---|---|---|---|
| Scope tools to the task, not the product (LLM06) | A summarizer gets read access to one document, not a mailbox-wide search tool and an email sender | Restrict administrative privileges (ML2+) | A.8.2 Privileged access rights | PR.AA-05 Least privilege | - |
| Human between the model and consequential actions | Approval for sending mail, moving money, changing configs, deleting data; show the real action, not the model's summary | - | A.5.15 Access control | PR.AA-05 Least privilege | - |
| Break the exfiltration leg (egress allowlisting) | No unrestricted outbound fetches, no remote image rendering from output, allowlisted domains for every network tool | - | A.8.16 Monitoring activities | PR.DS | D3-OTF Outbound Traffic Filtering |
| Keep secrets out of prompts (LLM07) | System prompts leak, so API keys, internal hostnames, and customer data do not belong there; the canary is the only intentional "secret" | - | A.5.17 Authentication information | PR.DS | - |
| Treat model output as untrusted input (LLM05) | Encode before HTML, parameterize before SQL, never pipe to a shell | - | - | PR.PS Platform security | - |
| Centralized logging of prompts, tool calls, and retrieval | The telemetry every rule above keys off | - | A.8.15 Logging; A.8.16 Monitoring activities | DE.CM Continuous monitoring; DE.AE Adverse events | - |
| Inventory and pin models like dependencies (LLM03) | Version-pin, verify sources, track models and adapters as supply chain | - | A.5.9 Inventory of assets; A.5.19-A.5.22 Supplier relationships | ID.AM Asset Management; GV.SC Supply chain | - |

## Honest limitations

- There is no deterministic fix for prompt injection. Guardrail models and prompt classifiers are probabilistic: useful as a filter layer, bypassable by a motivated attacker. Anyone selling a product that "solves" injection is selling you LLM09.
- These detections are younger than the rest of defensive tradecraft. Windows eventing has decades of field testing; LLM telemetry has a couple of years and no standard schema. Expect to adapt every schema here to your own logging, and expect it to age faster than an Active Directory detection.
- The canary catches leakage, not all injection. That is why the egress and behavioral layers exist, and why blast-radius design (least privilege, human-in-the-loop, breaking leg three of the trifecta) outranks all three rules.
- Vendor-hosted copilots limit visibility. For M365 Copilot you get what Purview exposes, not raw model I/O. Push vendors on logging the way you would push any SaaS provider on audit APIs.

## ATT&CK and references

- MITRE ATLAS: [AML.T0051 LLM Prompt Injection](https://atlas.mitre.org/techniques/AML.T0051) (the AI matrix that is MITRE's counterpart to ATT&CK; direct and indirect variants). ATLAS, not enterprise ATT&CK, is where prompt injection is catalogued.
- MITRE D3FEND: Outbound Traffic Filtering (D3-OTF) for the egress-break countermeasure in rule 2.
- UMBRASEC writeup: [Prompt Injection and the OWASP LLM Top 10: A Field Guide for Defenders](https://umbrasec.dev/research/owasp-llm-top-10-for-defenders.html) (2026-06-11).
- Microsoft Learn - Search the audit log / Copilot activities in the Purview unified audit log (the `CopilotInteraction` operation); Azure OpenAI monitoring and diagnostic logs.
- OWASP - Top 10 for LLM Applications (2025).
- NVD - [CVE-2025-32711](https://nvd.nist.gov/vuln/detail/CVE-2025-32711) (M365 Copilot "EchoLeak" information disclosure, CVSS 9.3); Microsoft MSRC advisory.
- Greshake et al. - Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023).
- Simon Willison - The lethal trifecta for AI agents (Jun 2025).
- NCSC / CISA - Guidelines for Secure AI System Development (Nov 2023); NIST - AI Risk Management Framework.

**Scope note.** This is a defensive skill. It explains the OWASP LLM injection classes only to the depth a defender needs to log, detect, and contain them, and deliberately ships no working injection payloads, jailbreak strings, or evasion tooling. UMBRASEC publishes defense, not offense.