fix(sophos-xg): make the field extractors match in the real grok step

[kryonsx]

Problem

The current v11 Sophos XG filter (from #2638) deletes the KV output and rebuilds each field with a grok of three patterns: the text up to key=, the value, and (?:\s+[A-Za-z_][A-Za-z0-9_]*=|$). The EventProcessor grok step trims whitespace before each pattern, treats an empty match as no match, and writes nothing unless every pattern matches. The third pattern therefore never matches, and none of the 208 extractors writes anything. In the real parser, Sophos XG events get no status, type, subtype, addresses, ports or actionResult, so blocked, allowed and failed activity all look the same and threat-intelligence correlation has no address to check.

Two other steps have the same kind of defect. The date step's first pattern matches empty text at the start of the line, so the Sophos clock is never read. The header grok's optional prefix matches empty text when a line starts with its first key, so such lines are not parsed at all.

Change

• Each extractor reads a quoted value, or unquoted words up to the next key=, and checks the end of the value within the same pattern. An empty value, or a quoted value followed directly by text, is left unset instead of taking the next pair.
• The date step consumes the text up to date= and captures the clock without that prefix; the five reformat steps read the new form.
• A second header grok handles lines that start with the first key.
• The current parser (go-sdk v1.1.35 and later) keeps underscores in KV keys, so the original underscored key names are deleted as well. This removes about 120 duplicate raw fields per event. The existing underscore-free names stay in the list for older parsers.
• The outcome rules are unchanged. They were already correct: denial first, then failure, then success.
• Correction (second commit): the first version read unquoted values only up to the next space and did not check where a value ended. Real logs often carry empty values such as src_mac=, tran_src_ip= and tran_dst_ip=, so an empty value took the next pair as its value, which reached origin.mac and target.mac. Multi-word values such as ether_type=Unknown (0x0000) were cut at the first space.

Validation

All results use the latest versions: EventProcessor 8a3ade7, with every parser and rule plugin on go-sdk v1.1.36, and v11 d2479c1a.

• Real records: 42 retained records from one deployment, covering firewall allowed and denied, web filtering allowed and denied, successful and failed authentication, VPN established and terminated, IPS drop and heartbeat. The unchanged filter passed 2/42 on this engine, and also 2/42 on the older 497bf53 engine, so the defect does not come from the go-sdk change. This branch passed 42/42. Denials, allows, failures and the VPN and heartbeat cases got the expected outcome, and the source and destination addresses matched the stored ones, with no event errors. The replay also checks the hardware addresses, translated addresses and ether_type against each raw line: empty values stay unset, and all 16 multi-word ether_type values are complete.
• Edge cases: six fabricated records covered a status at the end of the line, a line without a priority tag, date= as the first key, an unquoted value at the end, empty values, and a quoted value followed directly by text. The unchanged filter passed 0/6, and one case produced event errors. This branch passed 6/6, including the device time.
• Contract test: sophos_xg_contract_test.go now consumes grok patterns in order, as the executor does. On the unchanged filter the repository's 73 contracts fail 42 subtests (27 on actionResult, 11 on statusCode, 12 on addresses). With this change all 79 contracts pass, and the full plugins/alerts suite passes. The two contracts added for the correction fail with the first version.
• Rules: sophos_xg_vpn_auth_failures.yml and sophos_password_guessing_on_administrator_account.yml read actionResult and use history. Their trigger conditions were replayed with go-sdk v1.1.36 on the parser output. The VPN rule matched the one real SSL VPN failed sign-in with this branch and nothing with the unchanged filter. The administrator rule was not exercised, because the sample had no failed administrator sign-in. History queries were not executed.

Production context

The deployment in the sample still runs an older filter. Over three days it stored 2.7 million events as the nonstandard accepted and only about 120,000 as denied, while about 931,000 events have the Denied subtype. Blocked web requests in the sample were stored as accepted. Deploying the current v11 filter without this change would remove outcomes and addresses from Sophos XG events altogether.

In a random sample of 1,500 records, an empty value directly followed by the next key appeared for tran_src_ip in 864 records, tran_dst_ip in 617, src_mac in 193 and dst_mac in 192, and ether_type held several words in 1,099.

Limits

• The playground results apply to the recorded engine build; deployed behavior after rollout remains unverified. No customer configuration was changed. Records and identifiers stay private.

🤖 Generated with Claude Code