fix(cisco-firepower): parse real FTD events and repair two rule contracts

[kryonsx]

Summary

The Firepower filter extracts nothing from real Firepower Threat Defense syslog, so none of the five Firepower rules can fire.

This draft does three things:

• It makes the filter parse the real event shape and the Firepower-specific events (430001 intrusion, 430002/430003 connection, 430007).
• It carries over the mechanically established fixes from the Cisco ASA review (fix(cisco-asa): repair condition syntax, geolocation keys and rule consumers #2682).
• It corrects the two rules that the real data can support.

Built and tested on the latest versions: v11 d2479c1a, go-sdk v1.1.36, and EventProcessor 8a3ade7. That engine revision is the same source as today's eventprocessor:v11.2.14 image and its eventprocessor/base:1.1.7.

Predecessor: #2600 (closed, historical context only).

Evidence (read-only, described without identifying data)

• The Firepower index is empty on every reachable v11 instance.
• One instance receives about 1.25M genuine Firepower Threat Defense syslog records in a month, stored under the Cisco Switch data type because of a customer-side routing setting:
  • 430003: about 1.24M;
  • 430002: 560;
  • 430001: 32;
  • 430007: 4.
• Their header is <PRI>%FTD-<sev>-<id>: with no timestamp and no device name. The payload is Key: Value, Key: Value with 58 distinct keys.
• Replayed locally as this data type, the unchanged filter rejects every header and stores each record with no fields and 98 errors.
• The deployed filter and rules match this repository.
• Cisco's documentation site refused automated access (HTTP 403). Changes rest on genuine records, the filter itself, the engine and go-sdk.

Filter (filters/cisco/firepower.yml, 3.0.3 → 3.1.1)

• Header: a new header step accepts the real header shape.
• Payload parsing: 430001/430002/430003/430007 payloads are parsed into log.* with go-sdk v1.1.36 names (underscores kept, e.g. log.DNS_TTL; spaces removed, e.g. log.PrefilterPolicy).
  • Standard fields are set only where the payload is unambiguous: origin.ip, target.ip, origin.port, target.port (numeric) and protocol.
  • Two syslog messages glued together by a collector yield no invented fields.
• Conditions: 98 raw log.* comparisons become go-sdk helpers. The raw form fails to compile when no log object exists and adds an error per step.
• Typo: the misspelled lgreaterOrEqual becomes greaterOrEqual/lessOrEqual. Before, that step's condition failed on every event.
• Fixes shared with the ASA review:
  • geolocation writes to log.<field>Geolocation instead of replacing the address;
  • 106102/106103 "permitted" hits are no longer overwritten with "denied" (same step-order bug; value strings unchanged);
  • 302013 direction;
  • 302304 protocol;
  • 305011/305012 action;
  • 302017 user trims;
  • the 113009/113011 user overwrite.
• Three LINA steps that could never write on the engine (3.1.1), fixed the same way as in the ASA review. The EventProcessor grok plugin counts an empty match as no match, and then the whole step writes nothing.
  • 302003/302004: the optional to was its own pattern, (to\s)?, which matched empty text on every 302003 line. A 302003 line whose local side is a host name was parsed by no step. The to is now read with the address and trimmed off.
  • 302022/302024/302026: the two steps for a mapped address without a port used {{.data}}, which on its own always matches empty text, so they never wrote. They now read [^/()]+, which only fits when no port follows.
• Outcome values are unchanged; a separate outcome review owns them.

Rules

• Intrusion rule (intrusion_prevention_high_priority_events.yml):
  • It now matches message 430001 with the device's own Priority = 1, or one of four Classification descriptions.
  • The old log.eventType/lowercase names never existed in syslog, and log.severity is the syslog level ("1" on every record).
  • Result: 32 of 32 real intrusion events in 30 days → about 32 alert documents under 7 top-level alerts.
• Non-standard-port rule (c2_nonstandard_port.yml):
  • It reads log.ApplicationProtocol and greaterThan("log.InitiatorPackets", 0). The old equals(..., true) could never match a string.
  • It requires a port and an external destination, and keeps its port lists and 1-hour/5-event history.
  • Fixing names alone would create about 53,000 alert documents in 30 days (internal web services on other ports). This version: about 122–188 under 7–13 top-level alerts.
• Unchanged: the malware, IOC and threat-intelligence rules. Their event types (430004/430005 and security-intelligence keys) never appeared in 30 days of real data, so their mappings are deferred.

Validation

• Playground (engine 8a3ade7): 148 lines, 83 genuine (private) and 65 fabricated.
  • Unchanged filter: 13,049 errors.
  • Changed filter: 0 errors.
  • All intact genuine headers are accepted, and the single-message records get addresses and protocol (ports when present).
  • Glued records get header fields only, and 30/30 fabricated checks pass.
• go-sdk v1.1.36 replay: the intrusion rule matches all 20 sampled real intrusion records and 0 near-misses. The non-standard-port rule matches only its fabricated positives.
• Playground rule run: 27 intrusion alerts equal to the replay, 0 circuit breakers and 0 compile errors.
• Empty-match correction (3.1.1), on engine 8a3ade7: five new fabricated lines show both defects with version 3.1.0 (the 302003 host-name line kept only its result; the 302024 line had no mapped address) and the fix with 3.1.1; the other three lines are identical. Committed replay.py: 63 events (58 plus the five new ones), 0 errors, every field as recorded, the same 10 alerts, no circuit breaker, no history search. None of the 148 private lines is one of these messages. go-sdk v1.1.36 rule replay over the 63 committed events: 21/21 checks.
• Go tests: the full plugins/alerts suite passes (v1.1.36): 51 pass, 11 skip (other technologies' private-evidence tests), 0 fail. Seven of the eight new Firepower tests fail against the original files. The newest one tries every grok pattern alone on texts that start with each printable character and fails when one matches empty text at the start; it and the model test (now with the five new lines) also fail on version 3.1.0.

Deferred

These need Cisco documentation or an owner decision:

• other header shapes;
• file/malware/security-intelligence mappings and the three rules that depend on them;
• severity and device-time sources;
• user, bytes, packets, URL and DNS as standard fields;
• outcome values;
• the intrusion rule's MITRE label (the device tags these as T1190);
• "Unknown" traffic and deduplication in the non-standard-port rule.

For the owner, not a filter change

On one instance the Firepower device logs arrive on the Cisco Switch input, so they never reach this filter. Point that device at the Firepower input. About 1.25% of those records are two syslog messages glued together by the collector.

Limits

• The non-standard-port history search was not executed against OpenSearch.
• Production indexing of the new log.* fields is untested.
• The data covers one customer, one device and one software version.
• Volumes are estimates from real timestamps.

See filters/audits/cisco-firepower.md.

🤖 Generated with Claude Code