v12 dashboard: Deceptive Bytes

[kryonsx]

Part of: #2696 · Needs first: #2697 (click-through to the Log Explorer and the value/count table)
Related: #1727 (parsers and rules for this integration, owned by the detection team)

Goal

Ship a built-in Deceptive Bytes dashboard that shows, at a glance, how much Deceptive Bytes is sending, whether it is still sending, and what kinds of events they are. The heart of it is the list of Deceptive Bytes logs grouped by event types (widget W7): click one and the Log Explorer opens on exactly those logs.

Where the data comes from

Integration (catalog name)	DECEPTIVE_BYTES
Data type	deceptive-bytes
How the logs arrive	The Deceptive Bytes management console forwards its events over syslog (UDP or TCP port 7010) to the UTMStack collector, which passes each line unchanged to the server; the setup guide says the product sends CEF.
Parser	definitions/filters/antivirus/deceptive-bytes.yaml
What dataSource holds	The IP address the syslog message came from, normally the Deceptive Bytes management console or a relay (or the collector's hostname when the sender is 127.0.0.1). Proof: collectors/forwarder/collector/syslog/listener.go and handler.go set DataSource: resolveRemoteAddr(addr); log-input/ingest/server.go applyDefaults only fills 'unknown'; the parser never writes dataSource. So W6 is 'Logs by sender IP'.
Grouped by	log.eventType (event types)

Why log.eventType: Least-bad choice. log.eventType is the only event-name field the parser writes in more than one line shape: the CEF name when a line is exactly 'CEF:0|...', the value of an 'eventType=' key in key=value lines, and a number in the comma-separated shape. The rules also use it. It is missing for CEF lines with a syslog header in front, and nothing could be checked against real Deceptive Bytes logs (see caveats), so W7 and W8 may stay empty for some customers.

Typical values: Credential theft attempt blocked (CEF name; example made up to match the parser), ProcessBlocked (eventType= key; example made up to match the parser)

Widgets

Standard layout from the parent issue; rows W2, W3 and W9 onward are specific to this integration.

#	Title	Shown as	Query	Why
W1	Total logs	number	logs: count	All Deceptive Bytes logs in the selected time range; works whatever the line format.
W2	Blocked or prevented	number	logs: count; filter actionResult = blocked	Events whose action was blocked or prevented (parser sets actionResult 'blocked'); only lines with an 'action=' key can count.
W3	High severity events	number	logs: count; filter severity = high	Lines with level code C, A or E, which the parser maps to severity 'high'.
W4	Alerts	number	alerts: count	Alerts raised from Deceptive Bytes logs.
W5	Log volume over time	area chart	logs: count over time	Shows gaps and spikes in what the console forwards.
W6	Logs by sender IP	bar chart	logs: top 10 values of dataSource	dataSource is the IP that sends the syslog (the management console or a relay).
W7	Top event types	value and count table	logs: top 25 values of log.eventType; filter log.eventType exists	The main list: event names or types by count; click one to open those logs.
W8	Event types over time (top 5)	line chart	logs: count over time, one line per value of log.eventType (top 5); filter log.eventType exists	When each of the five most common event types happened.
W9	Actions	bar chart	logs: top 10 values of log.action; filter log.action exists	What Deceptive Bytes did (for example blocked or prevented), from the 'action=' key.
W10	Top reporting hosts	bar chart	logs: top 10 values of log.syslogHostIP; filter log.syslogHostIP exists	Host named in the syslog header of each line.
W11	Events by severity	pie chart	logs: top 50 values of severity; filter severity exists	Split into high, medium and low (three values set by the parser).
W13	Alerts by rule	bar chart	alerts: top 10 values of name	Which Deceptive Bytes detection rules fire most.
W14	Alerts by severity	bar chart	alerts: top 50 values of severity	Split of Deceptive Bytes alerts into low, medium and high.
W15	Latest logs	table of latest logs	logs: latest 20 records; columns @timestamp, dataSource, log.eventType, log.action, severity, log.syslogHostIP, raw	The newest records; raw is included because many line shapes keep their content only there.

Fields used and where they come from

• log.eventType: Event name or type, depending on the line shape. Examples: Credential theft attempt blocked, ProcessBlocked. Source: CEF header grok on log.restData (fieldName log.eventType) + trim '|'; kv on log.restData (key eventType); comma-separated grok ',{{.integer}}'.
• log.action: Action from an 'action=' key. Examples: blocked, prevented. Source: kv step on log.restData; the parser's own add step tests the values blocked and prevented.
• actionResult: 'blocked' when log.action is blocked or prevented. Examples: blocked. Source: add actionResult 'blocked' where exists(log.action) && log.action in [blocked, prevented].
• severity: Mapped from the one-letter level code of diagnostic lines. Examples: high, medium, low. Source: add 'high' where log.severityLabelCharacter in [C, A, E]; 'medium' where W; 'low' where in [D, V, N, I, T].
• log.syslogHostIP: Host name or IP in the RFC 5424 syslog header. Examples: host01, 10.0.0.5. Source: header groks, pattern '{{.ipv4}}|{{.ipv6}}|{{.word}}'.

Watch out for

• Not verifiable: no public documentation of Deceptive Bytes' syslog content and no real sample line could be found (the author of v11 pull request fix(deceptive-bytes): align parsed fields and correlation consumers #2666 reports the same after 29 searches). The UTMStack setup guide says the product emits CEF, and SGBox's guide says to choose CEF if available. Every test line used here was made up to match the parser's patterns, so the field names are proven by the parser, but not whether real logs reach them.
• What each line shape produces in the replay: 'CEF:0|vendor|product|version|id|name|severity|extension' gives log.eventType (the CEF name), log.signatureID and log.severity (number); a syslog RFC 5424 header in front of CEF gives only the extension keys (no event name); a BSD header in front of CEF, or CEF with no header, gives nothing but raw; RFC 5424 plus key=value ('... DeceptiveBytes:1 action=blocked eventType=ProcessBlocked processName=evil.exe') gives log.action, log.eventType, log.processName and actionResult; comma-separated and letter-coded diagnostic lines give log.processId, log.eventTime and severity.
• W2 only counts lines with an 'action=' key: the CEF key 'act' is never mapped to action, and as the first extension key it is lost anyway (see parser issues).
• W3 and W11 only count lines with a one-letter level code (C, A, E = high; W = medium; D, V, N, I, T = low); CEF severity stays as a number in log.severity and is not mapped.
• Alerts may stay at zero: the 16 Deceptive Bytes rules read origin.ip, which this parser never writes, and fields such as log.isDecoy, log.deceptionTriggered and log.tacticName, which only exist if real lines happen to carry those exact keys (the parser has no step for them).
• W1, W5, W6 and W15 work whatever the line format, so they are the reliable part of this dashboard until a real sample is available.
• No field used here depends on the engine's underscore rule, but any real key with an underscore (the rules expect some, such as event_type or trap_type) would be named differently on old and new engine builds.
• Checked with a step-by-step replay of this filter (the event processor's own step code and go-sdk rules) over 12 made-up lines, with both engine key rules.

Parser problems found while designing this

These are not dashboard work, but they limit what the dashboard can show. They belong to #1727; raise them there rather than working around them in the dashboard.

• Standard CEF with a BSD syslog header ('<14>Sep 23 12:00:00 host CEF:0|...') or with no header at all is not parsed: every header grok fails and the log keeps only raw.
• With an RFC 5424 header in front of CEF, the CEF header (event name, signature ID, severity) is not extracted; the grok built for 'word: user CEF:0' instead puts the header text into log.message and log.user.
• The kv steps read text that still ends with the CEF header, so the first extension key is glued to the header's last word and severity (for example 'act' becomes log.blocked9act) and its real key is lost.
• The CEF key 'act' is never renamed to action, so actionResult can never be set from CEF lines.
• In the bracketed shape '[date time] [W] [pid/tid] message' the time grok leaves the closing ']' in front of the rest, so the level-code grok never matches and severity is not set.
• The comma-separated shape writes 'command' at the top level, which the event format does not have, so the command line is dropped; it also puts a number into log.processName.
• Three kv steps (on log.restMessageToKv, log.restData and log.pidStatusToKv) fail with 'source does not exist' on most lines, adding error entries to nearly every event.

How to build it

Before you start: parser field names are changing while the parsers are updated for the engine's new underscore handling (see "Field names are about to move" in #2696). Check every field in this issue against the v12 parser at that moment and against real logs, and build against what you find.

1. Add definitions/dashboards/integration-deceptive-bytes.yaml. Keep it in the top folder: the test that checks shipped dashboards (TestEveryShippedDashboardDefinitionIsValid) only reads the top folder.
2. Start from the file below; it follows the table above and passes the same checks as the backend (domain.Spec.Validate).
3. W7 is a value/count table: a category query shown with chart type table. It already renders; v12 dashboards: groundwork for integration dashboards (click-through to the Log Explorer, value/count table, Agents dashboard fix) #2697 makes its rows clickable and its headers readable.
4. Load real logs from this technology on a v12 test server (or replay samples) and check every widget before opening the pull request.

Starting dashboard file

# Dashboard version v1.0.0
#
# System-owned default dashboard for the Deceptive Bytes integration, seeded by
# backend/modules/dashboards/repository/dashboard_bootstrap.go.
# Field names come from definitions/filters/antivirus/deceptive-bytes.yaml.
# Verify every widget against real logs before shipping.
name: "Deceptive Bytes"
description: "What Deceptive Bytes is sending: volume, event types, and the activity worth a look."
widgets:
  - layout: { x: 0, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: metric
      metric:
        agg: count
    config:
      __builder:
        chartType: metric
        title: "Total logs"

  - layout: { x: 3, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: metric
      metric:
        agg: count
      filters:
        - field: "actionResult"
          op: eq
          value: "blocked"
    config:
      __builder:
        chartType: metric
        title: "Blocked or prevented"

  - layout: { x: 6, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: metric
      metric:
        agg: count
      filters:
        - field: "severity"
          op: eq
          value: "high"
    config:
      __builder:
        chartType: metric
        title: "High severity events"

  - layout: { x: 9, y: 0, w: 3, h: 2 }
    spec:
      dataset: alerts
      dataType: "deceptive-bytes"
      chart: metric
      metric:
        agg: count
    config:
      __builder:
        chartType: metric
        title: "Alerts"

  - layout: { x: 0, y: 2, w: 8, h: 4 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: time
      metric:
        agg: count
    config:
      __builder:
        chartType: area
        title: "Log volume over time"

  - layout: { x: 8, y: 2, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "dataSource"
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Logs by sender IP"

  - layout: { x: 0, y: 6, w: 6, h: 6 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "log.eventType"
      filters:
        - field: "log.eventType"
          op: exists
      limit: 25
    config:
      __builder:
        chartType: table
        title: "Top event types"

  - layout: { x: 6, y: 6, w: 6, h: 6 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: time
      metric:
        agg: count
      dimension: "log.eventType"
      filters:
        - field: "log.eventType"
          op: exists
      limit: 5
    config:
      __builder:
        chartType: line
        title: "Event types over time (top 5)"

  - layout: { x: 0, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "log.action"
      filters:
        - field: "log.action"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Actions"

  - layout: { x: 4, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "log.syslogHostIP"
      filters:
        - field: "log.syslogHostIP"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Top reporting hosts"

  - layout: { x: 8, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "severity"
      filters:
        - field: "severity"
          op: exists
    config:
      __builder:
        chartType: pie
        title: "Events by severity"

  - layout: { x: 0, y: 16, w: 6, h: 4 }
    spec:
      dataset: alerts
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "name"
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Alerts by rule"

  - layout: { x: 6, y: 16, w: 6, h: 4 }
    spec:
      dataset: alerts
      dataType: "deceptive-bytes"
      chart: category
      metric:
        agg: count
      dimension: "severity"
    config:
      __builder:
        chartType: bar
        title: "Alerts by severity"

  - layout: { x: 0, y: 20, w: 12, h: 6 }
    spec:
      dataset: logs
      dataType: "deceptive-bytes"
      chart: table
      metric:
        agg: count
      limit: 20
      columns: ["@timestamp", "dataSource", "log.eventType", "log.action", "severity", "log.syslogHostIP", "raw"]
    config:
      __builder:
        chartType: table
        title: "Latest logs"

Done when

• definitions/dashboards/integration-deceptive-bytes.yaml is merged to release/v12.0.0 and go test ./modules/dashboards/... passes in backend/.
• With Deceptive Bytes logs flowing on a v12 test server, every widget shows data. A widget that stays empty while logs arrive means a wrong field or value: fix it, don't ship it.
• Clicking a row, bar or line point opens the Log Explorer on the same logs, and the Log Explorer count matches the widget.
• A time range with no logs shows empty states, not errors.
• A screenshot of the finished dashboard is attached to this issue.