Skip to content

Add custom yara signature - #776

Open
matthewscherer wants to merge 9 commits into
mainfrom
add_custom_yara_signature
Open

Add custom yara signature#776
matthewscherer wants to merge 9 commits into
mainfrom
add_custom_yara_signature

Conversation

@matthewscherer

@matthewscherer matthewscherer commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Change Summary

Added new fields for custom user yara signature alerts.

Sample values

...
        "memory_region": {
          "malware_signature": {
            "secondary": [
              {
                "custom_yara_signature": {
                  "entry_name": "Second user defined entry name",
                  "rule_identifier": "User_Defined_Rule_Identifier_2",
                  "entry_id": "789-012"
                },
                "signature": {
                  "name": "user_yara_scan_test_two",
                  "id": "",
                  "hash": {
                    "sha256": "cc0e0bdf70ec65d98f652e2428e3567013d5413f2725a2905b372fd18da8b9aa"
                  }
                },
                "matches": [
                  "VVNFUllBUkFTQ0FOVE9LRU4="
                ]
              }
            ],
            "identifier": "endpoint-yararules-windows-v1",
            "all_names": "user_yara_scan_test-user_yara_scan_test_two",
            "version": "1.0.1",
            "primary": {
              "custom_yara_signature": {
                "entry_name": "User defined entry name",
                "rule_identifier": "User_Defined_Rule_Identifier_1",
                "entry_id": "123-456"
              },
              "signature": {
                "name": "user_yara_scan_test",
                "id": "",
                "hash": {
                  "sha256": "aa0e0bdf70ec65d98f652e2428e3567013d5413f2725a2905b372fd18da8b9dd"
                }
              },
              "matches": [
                "VVNFUllBUkFTQ0FOVE9LRU4="
              ]
            }
          },
          "bytes_compressed_present": false
        },
...

Sample document:

{
"@timestamp": "2026-07-28T20:29:36.105Z",
"agent":
    {
    "build":
        {
        "original": "version: 9.6.0-SNAPSHOT, compiled: Tue Jul 28 15:00:00 2026, branch: yara_scanning_tests, commit: b5c26561c30feed6324949e52d1c4cfb20aee28f"
        },
    "id": "20260728_152339_869824_pid7208_172.16.32.109_41:05:15:56:5a:69_b4cbb574-1b13-41b3-9607-ca146fffaa8f",
    "type": "endpoint",
    "version": "9.6.0-SNAPSHOT"
    },
"data_stream":
    {
    "dataset": "endpoint.alerts",
    "namespace": "default",
    "type": "logs"
    },
"ecs":
    {
    "version": "8.10.0"
    },
"elastic":
    {
    "agent":
        {
        "id": "20260728_152339_869824_pid7208_172.16.32.109_41:05:15:56:5a:69_b4cbb574-1b13-41b3-9607-ca146fffaa8f"
        }
    },
"Endpoint":
    {
    "policy":
        {
        "applied":
            {
            "artifacts":
                {
                "global":
                    {
                    "channel": "stable",
                    "identifiers":
                        [
                            {
                            "name": "diagnostic-configuration-v1",
                            "sha256": "1aed8b2e7de5adcf41a2628829e2b168f33577f3152beb129b97790dd6560af6"
                            },
                            {
                            "name": "diagnostic-endpointpe-v4-blocklist",
                            "sha256": "338eb3e0d0c2b7efcd291ca270a0a6188d3632a25c7a8dc5d020fdb44293e12f"
                            },
                            {
                            "name": "diagnostic-endpointpe-v4-exceptionlist",
                            "sha256": "535a6e4ddd10248431c92a14e75e6fd801889c585b3794a1c5f66de29a917f2d"
                            },
                            {
                            "name": "diagnostic-endpointpe-v4-model",
                            "sha256": "9f4ef0f516df05ca6b2d27a1e8a1095aa6043b3c19f204cd94b3c066a9225bc7"
                            },
                            {
                            "name": "diagnostic-malware-signature-v1-windows",
                            "sha256": "beeb87ea9263d34c7cd23e0593b1775f40d72c06ecf269cf0c31610f3990594d"
                            },
                            {
                            "name": "diagnostic-ransomware-v1-windows",
                            "sha256": "9636fb1bda16d4dc236d40d2aeb379f30d81276028e9b3048002e3ca6a83dfb5"
                            },
                            {
                            "name": "diagnostic-rules-windows-v1",
                            "sha256": "befe2423c6d007c214b23426ca60b41e1851018e781fa84c41f8e65fd35691e0"
                            },
                            {
                            "name": "endpointpe-v4-blocklist",
                            "sha256": "fd5ba57ee0f28c9424e16898cfdfb05a6e9a52d83013d4d0701b3b7d74effe56"
                            },
                            {
                            "name": "endpointpe-v4-exceptionlist",
                            "sha256": "8c225635fe231f43186d7f3a46aa241b05f2ba7fa86b71e376bad43f7bb331a8"
                            },
                            {
                            "name": "endpointpe-v4-model",
                            "sha256": "6452b718ae142b74b02d1caa7f3546d35b52540eb268067f405ba946b18be712"
                            },
                            {
                            "name": "global-configuration-v1",
                            "sha256": "88a87c3571993b4f7364a384c04c48253b82425a85e5155c87050522a64fcc96"
                            },
                            {
                            "name": "global-eventfilterlist-windows-v1",
                            "sha256": "2a30e0a996468b99cf2bd71629713d866ba7c51c24aa54c57c467122c8d55ab4"
                            },
                            {
                            "name": "global-exceptionlist-windows",
                            "sha256": "4cee734eba97714d8c01040b2c6ad5086c91c428505cd5b536e4122d6923e858"
                            },
                            {
                            "name": "global-trustlist-windows-v1",
                            "sha256": "aaeab218a20828574c347e88ecb4eb411ffca9e8496c484327b2213f49e1dc9e"
                            },
                            {
                            "name": "production-malware-signature-v1-windows",
                            "sha256": "52b59b6dae8cb8c3c6257f97865effa290d68d4c5aae26cc2905e69c01527fba"
                            },
                            {
                            "name": "production-ransomware-v1-windows",
                            "sha256": "e3efc40c7e3fc0a590a074011abf990ab120f9f182b3dbca7b32ee58d0e5c3aa"
                            },
                            {
                            "name": "production-rules-windows-v1",
                            "sha256": "5dd6c5ccc0326f0ddeae9bf0caa57736628ee7386c5789a4fe587d73124a54e9"
                            },
                            {
                            "name": "tamper-protection-config-v1",
                            "sha256": "6fd383db741c2dbbd6f1a0d455c8461e9f78356cbb441c48c6c92e3429415a44"
                            }
                        ],
                    "manifest_type": "stable",
                    "snapshot": "latest",
                    "update_age": 0,
                    "version": "06HYZKR5ZI"
                    },
                "user":
                    {
                    "identifiers":
                        [
                            {
                            "name": "endpoint-blocklist-windows-v1",
                            "sha256": "54428969c0caf8b3ecd837d16aeb58309f0085dd06d148705c197de448698349"
                            },
                            {
                            "name": "endpoint-eventfilterlist-windows-v1",
                            "sha256": "6122af1927f4ab869c8281cb52079c956f49978fea62299af51778cec6441ec9"
                            },
                            {
                            "name": "endpoint-exceptionlist-windows-v1",
                            "sha256": "d12789f634623786e0e72c13a22710afc4dc722f142003008a719b71b9b2164c"
                            },
                            {
                            "name": "endpoint-hostisolationexceptionlist-windows-v1",
                            "sha256": "93c69c537542708e8e1e6eeaad63a818f1b64fb6e84fd06e07d142cb926457a4"
                            },
                            {
                            "name": "endpoint-trusteddevicelist-windows-v1",
                            "sha256": "ca1f73fe84ce93d962dadc64d9c2c80e73b0f511dc2a741c40d781cc66b68419"
                            },
                            {
                            "name": "endpoint-trustlist-windows-v1",
                            "sha256": "0c10a80c47b133248c1517fb0d517b8a387e61fdc392fb022ad069d3d09b926b"
                            },
                            {
                            "name": "endpoint-yararules-windows-v1",
                            "sha256": "ddbb97deec0a44627d056f18af9ed3611c250530bf9d4d33ec3966bb53754447"
                            }
                        ],
                    "version": "1.0.0"
                    }
                },
            "endpoint_policy_version": "eaf-policy-version",
            "id": "5416419d-71dd-48d3-807e-7bd291ac53b2",
            "name": "eaf-default-policy",
            "version": "9U64O04J16"
            }
        }
    },
"event":
    {
    "category":
        [
        "malware",
        "intrusion_detection",
        "process"
        ],
    "code": "memory_signature",
    "created": "2026-07-28T20:29:36.105Z",
    "dataset": "endpoint.alerts",
    "id": "OaYDqzik1vqjnM8a++++++PM",
    "kind": "alert",
    "module": "endpoint",
    "outcome": "success",
    "risk_score": 99,
    "sequence": 3696,
    "severity": 99,
    "type":
        [
        "info",
        "start",
        "allowed"
        ]
    },
"host":
    {
    "architecture": "x86_64",
    "hostname": "DESKTOP-1TK590J",
    "id": "dabadaba-0000-0000-0000-000000000000",
    "ip":
        [
        "172.16.32.109",
        "fe80::944a:36ff:7e0:55f3",
        "127.0.0.1",
        "::1"
        ],
    "mac":
        [
        "00-0c-29-55-05-69"
        ],
    "name": "desktop-1tk590j",
    "os":
        {
        "Ext":
            {
            "variant": "Windows 11 Pro"
            },
        "family": "windows",
        "full": "Windows 11 Pro 21H2 (10.0.22000.2538)",
        "kernel": "21H2 (10.0.22000.2538)",
        "name": "Windows",
        "platform": "windows",
        "type": "windows",
        "version": "21H2 (10.0.22000.2538)"
        }
    },
"message": "Memory Threat Detection Alert: user_yara_scan_test",
"process":
    {
    "args":
        [
        "C:\\Program Files\\VMware\\VMware Tools\\vmtoolsd.exe",
        "-n",
        "vmusr"
        ],
    "args_count": 3,
    "code_signature":
        {
        "exists": true,
        "status": "trusted",
        "subject_name": "VMware, Inc.",
        "trusted": true
        },
    "command_line": "\"C:\\Program Files\\VMware\\VMware Tools\\vmtoolsd.exe\" -n vmusr",
    "entity_id": "uZ/8VqxvbBwCcCV0UirAHg",
    "executable": "C:\\Program Files\\VMware\\VMware Tools\\vmtoolsd.exe",
    "Ext":
        {
        "ancestry":
            [
            "pxyUH4D2TKhVQJyOLDOYTg"
            ],
        "architecture": "x86_64",
        "code_signature":
            [
                {
                "exists": true,
                "status": "trusted",
                "subject_name": "VMware, Inc.",
                "trusted": true
                },
                {
                "exists": true,
                "status": "trusted",
                "subject_name": "VMware, Inc.",
                "trusted": true
                }
            ],
        "memory_region":
            {
            "bytes_compressed_present": false,
            "malware_signature":
                {
                "all_names": "user_yara_scan_test-user_yara_scan_test_two",
                "identifier": "endpoint-yararules-windows-v1",
                "primary":
                    {
                    "custom_yara_signature":
                        {
                        "entry_id": "123-456",
                        "entry_name": "User defined entry name",
                        "rule_identifier": "User_Defined_Rule_Identifier_1"
                        },
                    "matches":
                        [
                        "VVNFUllBUkFTQ0FOVE9LRU4="
                        ],
                    "signature":
                        {
                        "hash":
                            {
                            "sha256": "aa0e0bdf70ec65d98f652e2428e3567013d5413f2725a2905b372fd18da8b9dd"
                            },
                        "id": "",
                        "name": "user_yara_scan_test"
                        }
                    },
                "secondary":
                    [
                        {
                        "custom_yara_signature":
                            {
                            "entry_id": "789-012",
                            "entry_name": "Second user defined entry name",
                            "rule_identifier": "User_Defined_Rule_Identifier_2"
                            },
                        "matches":
                            [
                            "VVNFUllBUkFTQ0FOVE9LRU4="
                            ],
                        "signature":
                            {
                            "hash":
                                {
                                "sha256": "cc0e0bdf70ec65d98f652e2428e3567013d5413f2725a2905b372fd18da8b9aa"
                                },
                            "id": "",
                            "name": "user_yara_scan_test_two"
                            }
                        }
                    ],
                "version": "1.0.1"
                }
            },
        "protection": "",
        "token":
            {
            "domain": "DESKTOP-1TK590J",
            "elevation": false,
            "elevation_type": "limited",
            "integrity_level_name": "medium",
            "sid": "S-1-5-21-1749029863-1064264096-968553628-1001",
            "user": "Default"
            },
        "user": "Default"
        },
    "hash":
        {
        "sha256": "63ead767c40d62d65dd74090b3aae43b96f76e28537a41d8def631273f6e9efe"
        },
    "name": "vmtoolsd.exe",
    "parent":
        {
        "args":
            [
            "C:\\Windows\\Explorer.EXE"
            ],
        "args_count": 1,
        "code_signature":
            {
            "exists": true,
            "status": "trusted",
            "subject_name": "Microsoft Windows",
            "trusted": true
            },
        "command_line": "C:\\Windows\\Explorer.EXE",
        "entity_id": "pxyUH4D2TKhVQJyOLDOYTg",
        "executable": "C:\\Windows\\explorer.exe",
        "Ext":
            {
            "architecture": "x86_64",
            "code_signature":
                [
                    {
                    "exists": true,
                    "status": "trusted",
                    "subject_name": "Microsoft Windows",
                    "trusted": true
                    }
                ],
            "protection": "",
            "user": "Default"
            },
        "hash":
            {
            "sha256": "2e303b5e8791f2538b5b89542d50a6215c4d9ba4075dddd9b0a91f635327a2b7"
            },
        "name": "explorer.exe",
        "pid": 6304,
        "ppid": 6224,
        "start": "2026-07-18T10:52:29.909Z",
        "uptime": 898627
        },
    "pe":
        {
        },
    "pid": 7324,
    "ppid": 6304,
    "start": "2026-07-18T10:52:42.019Z",
    "uptime": 898614
    },
"rule":
    {
    "id": "",
    "name": "user_yara_scan_test",
    "ruleset": "production"
    },
"user":
    {
    "domain": "DESKTOP-1TK590J",
    "name": "Default"
    }
}

Release Target

Q/A

For mapping changes:

  • I ran make after making the schema changes, and committed all changes
  • If these field(s) are "exception"-able, I made a companion PR to Kibana adding it (see Readme)
  • If this is a metadata change, I also updated both transform destination schemas to match

For Transform changes:

  • The new transform successfully starts in Kibana
  • The corresponding transform destination schema was updated if necessary

@matthewscherer
matthewscherer requested a review from a team as a code owner July 27, 2026 15:44
@szwarckonrad
szwarckonrad requested review from gergoabraham and removed request for szwarckonrad July 27, 2026 15:48
@matthewscherer matthewscherer added the enhancement New feature or request label Jul 27, 2026
matthewscherer and others added 4 commits July 27, 2026 14:25
@matthewscherer
fixed a number value.
d6fcdc8
@matthewscherer
Moved the custom_yara_signature keys.
fc06260
@matthewscherer
Merge branch 'add_custom_yara_signature' of github.com:elastic/endpoi…
f3e2e11 
…nt-package into add_custom_yara_signature
@matthewscherer @cursoragent
Fix invalid secondary subset ref and regenerate schemas
b6da2b7 
The alerts memory_protection_event subset referenced
memory_region.malware_signature.secondary as an empty `{}`, but that
custom fieldset's `secondary` has sub-fields, so the ECS generator
failed with "'fields' key expected, not found in subset for secondary"
(breaking `make` in CI). Changed it to `secondary: fields: "*"` to
include the secondary subtree.

Regenerated all artifacts. Because custom_yara_signature.* now lives in
the shared malware_signature fieldset, the api data stream also picks up
these fields via its wildcard memory_region.malware_signature reference.

Co-authored-by: Cursor <cursoragent@cursor.com>
gergoabraham
gergoabraham approved these changes Jul 30, 2026
View reviewed changes

@gergoabraham gergoabraham left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome! 🚀

@matthewscherer
matthewscherer requested a review from a team as a code owner August 6, 2026 16:03
@matthewscherer
matthewscherer enabled auto-merge (squash) August 7, 2026 12:39
@matthewscherer
matthewscherer disabled auto-merge August 7, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joeypoon joeypoon joeypoon approved these changes
@gergoabraham gergoabraham gergoabraham approved these changes
@intxgo intxgo intxgo approved these changes

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@matthewscherer @joeypoon @gergoabraham @intxgo