--- name: suricata description: "Expert agent for Suricata IDS/IPS/NSM. Covers rule writing, EVE JSON logging, AF_PACKET/DPDK capture, protocol parsers, JA3/JA4 fingerprinting, file extraction, datasets, suricata-update, performance tuning, and IPS inline mode. WHEN: \"Suricata\", \"EVE JSON\", \"suricata rule\", \"suricata-update\", \"ET rules\", \"Emerging Threats\", \"JA3\", \"JA4\", \"AF_PACKET\", \"Suricata IPS\", \"suricata.yaml\", \"threshold.config\"." license: MIT --- # Suricata This skill covers Suricata, the open-source multi-threaded IDS/IPS/NSM engine. It has deep knowledge of Suricata's rule language, EVE JSON output, capture methods, protocol parsers, performance tuning, and operational management including suricata-update and ruleset management. ## How to Approach Tasks When you receive a request: 1. **Classify** the request: - **Rule writing/tuning** -- Load `references/best-practices.md` - **Architecture/performance** -- Load `references/architecture.md` - **Deployment/installation** -- Apply operational guidance from architecture.md - **EVE log analysis** -- Load `references/best-practices.md` - **Troubleshooting** -- Apply diagnostic methodology below 2. **Identify version** -- Suricata 7.x is current stable; 8.0 adds significant features. Ask if unclear; some EVE fields and keywords differ. 3. **Gather context** -- Throughput (Gbps), deployment mode (IDS/IPS), OS (Linux preferred), NIC type, capture method (AF_PACKET/DPDK/PF_RING/NFQ), existing ruleset. 4. **Analyze** -- Apply Suricata-specific reasoning. Performance issues are often capture layer problems before rule evaluation problems. 5. **Recommend** -- Provide specific suricata.yaml configuration snippets, rule syntax, or operational commands. ## Core Expertise ### Rule Syntax Suricata rules follow the format: ``` action proto src_ip src_port direction dst_ip dst_port (options;) ``` **Actions:** `alert`, `drop` (IPS only), `reject`, `pass` **Protocols:** `tcp`, `udp`, `icmp`, `ip`, `http`, `tls`, `dns`, `smtp`, `ftp`, `ssh`, `dcerpc`, `smb`, and 50+ application-layer protocols **Basic rule example:** ```suricata alert http $HOME_NET any -> $EXTERNAL_NET any ( msg:"ET MALWARE Suspicious User-Agent"; http.user_agent; content:"EvilBot/1.0"; classtype:trojan-activity; sid:9000001; rev:1; ) ``` **Key rule options:** - `content:"..."` -- Byte string match (case-sensitive by default) - `nocase` -- Case-insensitive content match - `pcre:"/pattern/flags"` -- Perl-compatible regex - `flow:established,to_server` -- Flow direction and state - `flowbits:set,name` / `flowbits:isset,name` -- Track state across packets - `threshold:type limit,track by_src,count 5,seconds 60` -- Rate limiting ### Sticky Buffers (Suricata-Specific) Sticky buffers replace the legacy `content` + `http_*` modifier approach. They set the detection context once and all subsequent keywords apply to that buffer: ```suricata # Modern sticky buffer approach (preferred) alert http any any -> any any ( msg:"Suspicious URI"; http.uri; content:"/admin/upload"; nocase; content:"../../"; sid:9000002; rev:1; ) # Legacy approach (still works but deprecated for new rules) alert http any any -> any any ( msg:"Suspicious URI"; uricontent:"/admin/upload"; nocase; sid:9000003; rev:1; ) ``` **Common sticky buffers:** | Buffer | Matches | |---|---| | `http.uri` | Normalized URI | | `http.uri.raw` | Raw (un-normalized) URI | | `http.method` | HTTP method | | `http.host` | HTTP Host header | | `http.user_agent` | User-Agent header | | `http.request_body` | HTTP request body | | `http.response_body` | HTTP response body | | `tls.sni` | TLS SNI field | | `tls.cert_subject` | Certificate subject | | `dns.query` | DNS query name | | `smb.named_pipe` | SMB named pipe | | `ssh.software` | SSH software version | ### Suricata-Specific Keywords Keywords not in Snort rules that leverage Suricata's capabilities: **JA3/JA4 fingerprinting:** ```suricata alert tls any any -> any any ( msg:"Suspicious TLS Fingerprint (Cobalt Strike Default)"; ja3.hash; content:"72a589da586844d7f0818ce684948eea"; sid:9000004; rev:1; ) alert tls any any -> any any ( msg:"Known C2 JA4 Fingerprint"; ja4.hash; content:"t13d1516h2_8daaf6152771_02713d6af862"; sid:9000005; rev:1; ) ``` **Datasets (IP/domain/hash reputation lists):** ```suricata # Load a file of malicious IPs alert ip $EXTERNAL_NET any -> $HOME_NET any ( msg:"Connection from Threat Intel IP"; iprep:any,MaliciousIPs,>,30; sid:9000006; rev:1; ) # Dataset with file hash matching alert http any any -> any any ( msg:"Known Malware Hash Download"; filemd5:malware-md5.lst; sid:9000007; rev:1; ) ``` **Lua scripting for complex detection:** ```suricata alert http any any -> any any ( msg:"Custom Lua Detection"; lua:detect_custom.lua; sid:9000008; rev:1; ) ``` ### EVE JSON Output EVE JSON is Suricata's structured logging system. All event types are written to a single JSON log (or multiple files). **Key event types:** - `alert` -- Rule matches - `dns` -- DNS queries and responses - `http` -- HTTP transactions - `tls` -- TLS handshakes and certificates - `flow` -- Connection flow records (at connection close) - `fileinfo` -- Files extracted from network sessions - `anomaly` -- Protocol violations - `ssh` -- SSH handshakes - `smb` -- SMB sessions - `ftp` -- FTP sessions - `dhcp` -- DHCP transactions (8.0+) **Sample alert EVE event:** ```json { "timestamp": "2024-01-15T14:23:45.123456+0000", "event_type": "alert", "src_ip": "10.1.2.3", "src_port": 54321, "dest_ip": "192.168.1.10", "dest_port": 445, "proto": "TCP", "alert": { "action": "allowed", "gid": 1, "signature_id": 2027250, "rev": 2, "signature": "ET EXPLOIT Possible SMB Exploit Attempt", "category": "Attempted Administrator Privilege Gain", "severity": 1 }, "flow": { "pkts_toserver": 5, "pkts_toclient": 2, "bytes_toserver": 450, "bytes_toclient": 120, "start": "2024-01-15T14:23:44.000000+0000" } } ``` **Configuring EVE in suricata.yaml:** ```yaml outputs: - eve-log: enabled: yes filetype: regular filename: /var/log/suricata/eve.json types: - alert: payload: yes payload-printable: yes packet: yes metadata: yes http-body: yes http-body-printable: yes - http: extended: yes - dns: version: 2 - tls: extended: yes - files: force-magic: yes - flow - anomaly: enabled: yes ``` ### IPS Mode Configuration **NFQ (Linux iptables inline) mode:** ```bash # iptables rules to send traffic to Suricata iptables -I FORWARD -j NFQUEUE --queue-num 0 iptables -I INPUT -j NFQUEUE --queue-num 0 iptables -I OUTPUT -j NFQUEUE --queue-num 0 ``` ```yaml # suricata.yaml NFQ configuration nfq: mode: accept fail-open: yes ``` ```bash suricata -c /etc/suricata/suricata.yaml -q 0 ``` **AF_PACKET inline (bump-in-wire) mode:** ```yaml af-packet: - interface: eth0 cluster-id: 99 cluster-type: cluster_flow copy-mode: ips copy-iface: eth1 - interface: eth1 cluster-id: 98 cluster-type: cluster_flow copy-mode: ips copy-iface: eth0 ``` ```bash suricata -c /etc/suricata/suricata.yaml --af-packet ``` ### suricata-update suricata-update manages rulesets from multiple sources. **Basic operations:** ```bash # Initial setup suricata-update update-sources # Fetch available source list suricata-update enable-source et/open # Enable Emerging Threats Open (free) suricata-update enable-source oisf/trafficid # OISF traffic ID rules # For ET Pro (paid subscription): suricata-update enable-source et/pro --set-setting secret-code YOUR_KEY # Update all enabled sources suricata-update # Update and reload Suricata without restart suricata-update && suricatasc -c ruleset-reload-nonblocking ``` **Rule disable/enable customization (`/etc/suricata/disable.conf`):** ``` # Disable noisy rules by SID re:2013028 # Disable entire rule group by SID regex group:emerging-p2p.rules # Disable entire file sid:2019401 # Disable specific SID ``` **Rule modification (`/etc/suricata/modify.conf`):** ``` # Change alert to drop for high-confidence rules 2019401 "alert" "drop" # Add threshold to noisy rule 2027865 "noalert" "threshold:type limit,track by_src,count 1,seconds 300" ``` ### threshold.config Controls alert rate limiting independent of the rule itself: ``` # Global threshold: limit alerts from same src to 1 per 60 seconds threshold gen_id 1, sig_id 0, type limit, track by_src, count 1, seconds 60 # Suppress specific SID entirely from a known-good IP suppress gen_id 1, sig_id 2019401, track by_src, ip 10.0.0.50 # Suppress SID for entire subnet suppress gen_id 1, sig_id 2027250, track by_src, ip 192.168.10.0/24 # Both threshold and track threshold gen_id 1, sig_id 2008578, type both, track by_src, count 5, seconds 300 ``` ### File Extraction ```yaml # suricata.yaml file extraction configuration file-store: version: 2 enabled: yes dir: /var/log/suricata/filestore write-fileinfo: yes write-meta: yes # In rules: extract specific file types alert http any any -> any any ( msg:"PDF Download"; fileext:"pdf"; filestore; sid:9000009; rev:1; ) ``` Files are stored with SHA256 hashes as filenames, enabling automated malware hash lookups. ## Troubleshooting ### Performance Issues **Check for packet drops:** ```bash # Suricata stats suricatasc -c dump-counters | python3 -m json.tool | grep -E "drop|capture" # Kernel drop stats (AF_PACKET) cat /proc/net/packet | column -t # Suricata stats log tail -f /var/log/suricata/stats.log | grep -E "drop|capture" ``` **Key counters to monitor:** - `capture.kernel_drops` -- Kernel dropped packets before Suricata (NIC/ring buffer issue) - `decoder.pkts` -- Total packets decoded - `detect.alert` -- Total alerts generated - `tcp.reassembly_gap` -- TCP stream gaps (indicates drops) **Common causes and fixes:** - High `kernel_drops`: Increase `buffer-size` in AF_PACKET config, or add more workers - CPU bottleneck: Profile which detect threads are saturated; reduce rules or increase CPU - Memory exhaustion: Reduce `memcap` values or add RAM ### Rule Testing ```bash # Test rule against a PCAP without running as daemon suricata -r /path/to/capture.pcap -c /etc/suricata/suricata.yaml -l /tmp/test-output/ # Test specific rule file suricata -r capture.pcap -S my-custom.rules -l /tmp/test/ # Validate rule syntax suricata --list-runmodes # verify version suricata -T -c /etc/suricata/suricata.yaml # config test mode ``` ### EVE Log Analysis ```bash # Jq examples for EVE analysis # All alerts in last hour by signature jq 'select(.event_type=="alert") | .alert.signature' /var/log/suricata/eve.json | sort | uniq -c | sort -rn # Top talkers by source IP in alerts jq 'select(.event_type=="alert") | .src_ip' /var/log/suricata/eve.json | sort | uniq -c | sort -rn | head -20 # DNS queries to a specific domain jq 'select(.event_type=="dns" and .dns.rrname=="evil.example.com")' /var/log/suricata/eve.json # TLS connections with expired certificates jq 'select(.event_type=="tls" and .tls.notafter < "2024-01-01")' /var/log/suricata/eve.json # Files with specific MIME type jq 'select(.event_type=="fileinfo" and .fileinfo.mimetype=="application/x-dosexec")' /var/log/suricata/eve.json ``` ## Common Pitfalls 1. **Running default rules without tuning** -- ET Open has thousands of rules including many noisy ones. Review and disable `emerging-user_agents.rules`, `emerging-p2p.rules`, and similar high-noise categories immediately after deployment. 2. **Not enabling `fail-open` in IPS mode** -- Without fail-open, a Suricata crash or overload causes network outage. Always configure bypass/fail-open for inline deployments. 3. **Underestimating `buffer-size` for AF_PACKET** -- Default ring buffer sizes are often too small for production traffic. Start at 128MB per thread and increase if you see `kernel_drops`. 4. **Using legacy `http_*` modifiers instead of sticky buffers** -- Legacy modifiers work but are not maintained. New rules should use `http.uri`, `http.user_agent`, etc. 5. **Forgetting to reload after suricata-update** -- Rules update on disk but Suricata must reload them. Use `suricatasc -c ruleset-reload-nonblocking` or `kill -USR2 $(pidof suricata)`. 6. **Writing overly broad PCRE** -- Complex PCRE in rules consumes significant CPU. Anchor with `content` matches before PCRE to limit PCRE evaluation to matching candidates. ## Reference Files Load these when you need deep knowledge for a specific area: - `references/architecture.md` -- Multi-threaded internals, runmodes, capture layer (AF_PACKET/DPDK/PF_RING), EVE JSON schema, rule loading pipeline, packet processing path. Read for architecture and performance questions. - `references/best-practices.md` -- Rule writing guidelines, performance tuning, EVE log analysis patterns, threshold configuration, suricata-update workflow, SIEM integration. Read for operational and optimization questions.