[FR] Add new unit test for process fields in non process events

[Mikaayenson]

Pull Request

Issue link(s):

This PR adds a new unit test that uncovers a previously-silent bug class affecting EQL rules on logs-endpoint.events.* indices. Targets EQL only since KQL queries are scoped by index and authors naturally filter on category-applicable fields. ES|QL's FROM clause + Kibana field discovery makes the bug hard to write accidentally. Cursory scan didn't find any offenders either.

Important

This PR cannot land until the four currently-failing rules are fixed first. See the "Blocking PRs" list below.

Summary - What I changed

Adds a new unit test TestEQLEventFieldUsage::test_process_fields_present_in_endpoint_schema which fails CI when an EQL rule references a process.* field inside a non-process clause (file where, network where, registry where, library where, dns where) on logs-endpoint.events.* indices and that field is not mapped for the targeted dataset in the endpoint integration schema.

Elastic Defend doesn't populate every `process.*` field on every event category.

For example, process.command_line, process.args, process.working_directory, process.parent.name, and process.pe.* are only present on process events; they're mapping-absent on file, network, registry, library, and dns events. EQL's <event_type> where ... syntax happily lets authors mix category prefixes with field references that don't belong to that category, so predicates referencing those fields never match — silent no-ops. There is no rule-load-time error; the rule just quietly under-fires.

Blocking PRs — must merge before this one

The test currently catches several production-maturity rules with this bug, grouped here by domain so they can be tuned in batched PRs:

• GenAI Rules — [Rule Tuning] Misc GenAI Tuning #6006
  • c0136397-f82a-45e5-9b9f-a3651d77e21a — GenAI Process Accessing Sensitive Files (EQL) — process.command_line inside file where
  • 9050506c-df6d-4bdf-bc82-fcad0ef1e8c1 — GenAI Process Connection to Unusual Domain (KQL) — bundled into the same tuning because it shares the same root cause (process.command_line not populated outside of process events); not caught by this test because the test is EQL-only
• Windows Rules — [Rule Tuning] Misc Windows Tuning #5990
  • 0e79980b-4250-4a50-a509-69294c14e84b — MsBuild Making Network Connections — process.pe.original_file_name inside network where
• Linux Rules — [Rule Tuning] Fixes for Unsupported Fields #6025
  • 7f3e8b9a-2c4d-5e6f-8a1b-9c2d3e4f5a6b — Potential Webshell Deployed via Apache Struts CVE-2023-50164 Exploitation — process.parent.name inside file where
  • 00546494-5bb0-49d6-9220-5f3b4c12f26a — Uncommon Destination Port Connection by Web Server — process.working_directory inside network where

Design notes:

• No CUSTOM_RULES_DIR carve-out. Unlike test_preserve_upstream_protected_rule_id_name and test_elastic_license_only_v2, this test does not skip when CUSTOM_RULES_DIR is set, because the bug class is a property of the Elastic Defend integration schema (universal to anyone running Defend), not a property of the Elastic-shipped corpus. DaC users with the same bug benefit from catching it. DaC users with legitimate custom endpoint fields can register them via RULES_CONFIG.stack_schema_map (custom non-ECS schema YAML).
• Performance. Test body executes in ~0.4s on the full corpus (973 EQL rules → 784 endpoint-EQL → AST walk). The 60s of measurable runtime is the shared BaseRuleTest.setUpClass rule loading, which is @lru_cache'd and amortized across every test class.

How To Test

python3 -m pytest tests/test_all_rules.py::TestEQLEventFieldUsage -v

Bash Output

[github-actions]

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.

Documentation and Context

• Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
• Include additional context or screenshots.
• Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the enhancement.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
• Validate that any rules affected by the enhancement are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the enhancement works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[eric-forte-elastic]

Update looks good, comment addressed .

Reviewed scope and implementation.