Skip to content

Commit 152bdb5

Browse files
kryonsxkryonsx
authored
fix(windows): normalize native fields and preserve authentication correlation (#2628)
Windows filter v3.1.2: IP guard applied at the rename (only real IPs enter origin.ip); promotes origin.user/host/domain/path/port; writes log.authenticationCandidate.* markers; rules correlate on dataSource+authenticationSource* instead of origin.ip (fixes wave-1 nil-placeholder break). Raw-log + SDK-backed tests (SearchRequest.Execute against local mock). Verified: go build + go test ./... pass on go-sdk v1.1.31.
1 parent 4b070d8 commit 152bdb5

20 files changed

Lines changed: 3097 additions & 81 deletions

‎filters/audits/windows.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Windows normalization and correlation review
2+
3+
This draft targets UTMStack `v11`. The SDK Event/Side protobuf in the pinned
4+
`go-sdk v1.1.31` is the schema authority. The local dictionary is supporting
5+
material, not an alternative schema.
6+
7+
## Correction after review
8+
9+
The first draft removed placeholder IPs after promoting them to `origin.ip`,
10+
while authentication rules still required `{{.origin.ip}}` in historical
11+
searches. The SDK returns an error when a placeholder is missing; it returns
12+
before evaluating an `or` fallback. Restoring `-` as a standardized IP would
13+
also aggregate unrelated sources under one placeholder.
14+
15+
The filter now validates `log.data.IpAddress` at the original rename. Invalid,
16+
missing and unspecified addresses stay in their original vendor field and
17+
never enter `origin.ip`. The unspecified-address guard uses CIDR membership,
18+
not literal strings: expanded/compressed IPv6 zero and IPv4-mapped zero spellings
19+
are rejected while the exact original vendor value is preserved. Tests cover
20+
account and workstation fallback, no qualified fallback, and a valid mapped-IPv4
21+
control. These alternate-spelling regressions are synthetic; no additional
22+
customer occurrence is claimed. Authentication correlation chooses a real source IP,
23+
then workstation, then actor account, recorded in `log.authenticationSource`
24+
and `log.authenticationSourceType`. The account fallback identifies an account,
25+
not a network client. Every affected history search scopes that identity by
26+
its kind, domain scope, the agent's `dataSource`, and event code. Brute-force rules also
27+
require the target account; Kerberos searches constrain ticket encryption and,
28+
for AS-REP, preauthentication type. No shared placeholder is an identity. A username-only fallback requires its
29+
domain or an already qualified UPN; an unqualified username without a domain
30+
is not treated as a safe correlation identity.
31+
32+
The native Windows collector sets `dataSource` from its hostname for every
33+
record. The rules reject its documented `unknown`/empty sentinel so account
34+
fallback cannot pool unidentified collectors. They do not require a redundant
35+
`exists(dataSource)` check. The filter and all seven correlation consumers
36+
must be deployed together. Existing indexed records do not have the new
37+
correlation fields, so the updated history windows warm up after deployment.
38+
39+
Golden Ticket's historical query previously depended on `origin.host`, not
40+
`origin.ip`; native Kerberos records can lack that workstation too. Its
41+
correlation now uses the same identity selection. Filter-derived `log.authenticationCandidate.*` markers repeat each exact
42+
trigger predicate so benign events with the same event code cannot satisfy the
43+
historical threshold. The tests assert marker/predicate parity. Success after
44+
failures searches the failed-logon marker. The update preserves each rule's
45+
existing count and time window. It does not claim that the existing
46+
Golden/Silver Ticket heuristics prove forged tickets.
47+
48+
## Standard field promotion
49+
50+
The native agent emits `computer`, `timestamp`/`timeCreated`, and `data.IpPort`.
51+
The filter promotes the event-producing computer to `target.host`, device
52+
time to `deviceTime`, and valid remote ports to `origin.port`. WorkstationName
53+
and the native NTLM `Workstation` alias describe `origin.host`; the recording computer must not overwrite
54+
the remote workstation. Kerberos account names are also promoted to
55+
`origin.user` while the existing `target.user` and vendor aliases remain
56+
available to consumers. Vendor data without a standard counterpart remains
57+
under `log`. `SubjectDomainName` and the authenticated account's domain are
58+
promoted without mixing actor and target roles. A non-placeholder `ProcessName`
59+
is copied to `origin.path`, preserving the vendor alias used by current rules.
60+
61+
NTLM status must also accept the agent's numeric zero: CEL `regexMatch` only
62+
matches strings. The first draft's string-only check would have changed a
63+
successful native 4776 event to failure. The revised check covers numeric zero
64+
and hexadecimal zero strings, with nonzero numeric/string failure regressions.
65+
66+
The original fixes for successful account administration, NTLM status,
67+
placeholder cleanup, and event-versus-alert grouping remain included.
68+
69+
## Validation and limits
70+
71+
- `windows_contract_test.go` is standalone and runs with `go test ./...` in
72+
`plugins/alerts`, without the shared test-runner PR.
73+
- 60 sanitized raw JSON fixtures exercise valid IPv4/IPv6, missing/placeholder
74+
addresses, host/account fallback, valid/invalid ports, host roles and time.
75+
- 50 positive predicate cases cover all seven changed correlation consumers.
76+
Negative identity cases also compile/evaluate all 38 shipped Windows rules.
77+
- The real SDK executes historical requests against a local mock OpenSearch
78+
server, including mapping resolution, placeholder expansion, query creation,
79+
time/count boundaries and separation of different sources, identity kinds,
80+
collectors, account domains, event types and non-candidate history. The old missing-IP regression is reproduced with
81+
the SDK, without a customer connection.
82+
- The shared manifest adds seven nonempty rule assertions to its normalization
83+
cases. Its runner is supplied by draft #2590.
84+
85+
JSON extraction and filter transformations are an offline model based on the
86+
SDK sanitization utility and documented filter operations. These tests do not
87+
execute the closed EventProcessor, a live OpenSearch cluster, alert creation
88+
or delivery. Staging must compare real raw events, normalized results and
89+
created alerts before rollout. No customer configuration was changed.
90+
91+
## Bounded live evidence
92+
93+
The deployed Windows filter was also read without modification. Its relevant
94+
mappings matched the repository baseline: unguarded IpAddress promotion,
95+
WorkstationName alone, no computer/port/process-path promotion, and numeric-zero
96+
NTLM status handled through `equals`. Configuration SHA-256:
97+
`c3a781cc3f2015c3b9e1cb66773da3463186ccec68518051fcf3be533f77cc1c`.
98+
99+
A read-only seven-day sample from three instances yielded 28 distinct records
100+
for the requested authentication event codes. Fifteen had no usable source IP.
101+
All 28 retained the native computer only under `log`, and all 28 had deviceTime
102+
defaulted to ingestion time instead of the raw vendor timestamp. Three NTLM
103+
records supplied `data.Workstation` without a standardized origin host; nine
104+
records supplied a process path without `origin.path`. The private evidence
105+
pack keeps the instance/document anchors without publishing customer payloads.
106+
107+
No missing-IP 4768/4769/4771 events were observed in this seven-day aggregate.
108+
The report's broad claim that Kerberos placeholder IPs were observed is not
109+
supported by this sample. The SDK's missing-placeholder behavior is reproduced
110+
with synthetic Kerberos records; actual missing/placeholder IPs are confirmed
111+
for local logon and credential-validation event types.
112+
113+
Replaying those 28 raw records through the offline filter model and actual CEL
114+
produced two failed-logon candidates and five successful-logon candidates with
115+
resolved history placeholders. This is not proof of historical thresholds or
116+
created alerts: the sample is intentionally bounded and the live filter/rules
117+
were not replaced.
118+
119+
## Sources
120+
121+
- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
122+
- [SDK correlation execution](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/rules.go)
123+
- [Native Windows collector](https://github.com/utmstack/UTMStack/blob/v11/agent/collector/platform/windows_amd64.go)
124+
- [Filter operations](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
125+
- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
126+
127+
Alert grouping for `lastEvent.*` still depends on the shared alert-grouping
128+
fix in #2590, which requires its own staging comparison of alert counts.

0 commit comments

Comments
 (0)