Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update security service integrations packages mappings #12624

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Feb 5, 2025

Proposed commit message

Updated failed mappings found while validating the documents based on mappings.

Example of Buildkite build where the failures are raised: https://buildkite.com/elastic/integrations/builds/20794

Packages updated here:

  • box_events
[0] field "threat.enrichments.indicator.first_seen" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
[1] field "threat.enrichments.indicator.last_seen" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
  • claroty_ctd
[0] field "threat.indicator.modified_at" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
  • crowdstrike
[0] field "crowdstrike.assessments.analytics_and_improvements_mac" is undefined: field definition not found
[1] field "crowdstrike.assessments.application_firewall_mac" is undefined: field definition not found
[2] field "crowdstrike.assessments.crendential_dumping_hash_mac" is undefined: field definition not found
...
  • github
field "message" is undefined: actual mapping type (keyword) does not match with ECS definition type: match_only_text
  • mimecast
[0] field "threat.indicator.first_seen" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
[1] field "threat.indicator.modified_at" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
  • sublime_security
[0] field "sublime_security.email_message.headers.hops.fields.position" is undefined: field definition not found
  • teleport
[0] field "teleport.audit.account_id" is undefined: field definition not found
[1] field "teleport.audit.aws_host" is undefined: field definition not found
[2] field "teleport.audit.aws_region" is undefined: field definition not found
[3] field "teleport.audit.aws_service" is undefined: field definition not found
[4] field "teleport.audit.exit_code" is undefined: field definition not found
[5] field "teleport.audit.instance_id" is undefined: field definition not found
[6] field "teleport.audit.region" is undefined: field definition not found
[7] field "teleport.audit.status" is undefined: field definition not found
[8] field "teleport.audit.target" is undefined: field definition not found
  • ti_anomali
[0] field "threat.indicator.modified_at" is undefined: actual mapping type (keyword) does not match with ECS definition type: date
  • ti_custom
[0] field "threat.indicator.url.original" is undefined: actual mapping type (keyword) does not match with ECS definition type: wildcard
  • tychon
[0] field "related.ip" is undefined: actual mapping type (keyword) does not match with ECS definition type: ip
[1] field "server.ip" is undefined: actual mapping type (keyword) does not match with ECS definition type: ip
  • wiz
[0] field "ecs.version" is undefined: actual mapping type (text) does not match with ECS definition type: keyword
[1] field "message" is undefined: actual mapping type (text) does not match with ECS definition type: match_only_text

[0] field "device.id" is undefined: actual mapping type (text) does not match with ECS definition type: keyword
[1] field "ecs.version" is undefined: actual mapping type (text) does not match with ECS definition type: keyword
[2] field "message" is undefined: actual mapping type (text) does not match with ECS definition type: match_only_text
[3] field "related.ip" is undefined: actual mapping type (text) does not match with ECS definition type: ip
[4] field "tags" is undefined: actual mapping type (text) does not match with ECS definition type: keyword
[5] field "vulnerability.reference" is undefined: actual mapping type (text) does not match with ECS definition type: keyword

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • Is it required to update transform settings ?
  • Add changelog entries for all the updated packages

How to test this PR locally

Run elastic-package with mappings validation enabled and run system tests:

cd packages/<package>
elastic-package stack up -v -d --version <kibana_constraint>
export ELASTIC_PACKAGE_FIELD_VALIDATION_TEST_METHOD=mappings
# Add defer-cleanup to check the data ingested in the corresponding indices/data streams
elastic-package test system -v --defer-cleanup 900s

elastic-package stack down -v

Related issues

Screenshots

qcorporation and others added 26 commits February 4, 2025 12:21
…astic#12571)

Updates the following integrations to ECS 8.17.0:

- fortinet_fortiedr
- fortinet_fortigate
- fortinet_fortimail
- fortinet_fortimanager
- fortinet_fortiproxy
- goflow2
- hashicorp_vault
- imperva
- iptables
…astic#12569)

Update the following integrations to ECS 8.17.0:
- arista_ngfw
- cef
- checkpoint
- cisco_aironet
- cisco_asa
- cisco_ftd
- cisco_ios
- cisco_ise
- cisco_nexus
- cisco_secure_email_gateway
- citrix_waf
…astic#12572)

Updated the following integrations to ECS 8.17.0:
- juniper_srx
- modsecurity
- netflow
- osquery
- panw
- pfsense
- proxysg
…astic#12574)

Updated the following integrations to ECS 8.17.0:
- qnap_nas
- snort
- sonicwall_firewall
- sophos
- squid
- stormshield
- suricata
- tcp
- udp
- watchguard_firebox
- zeek
- The time format used "week-based-year" (Y) instead of "year-of-era" (y). The former
gives very different results when calculating the month and date, since it is week-based.
- Regenerated test files with updated dates.
elastic#12612)

* Fix dashboard query in [Metrics System] Host overview

* update pr link
* Add 9.0.0 constraint to sec-windows-platform packages

* Fix typo

* add scanner options to allow small test files

* Update packages/microsoft_exchange_server/data_stream/httpproxy/agent/stream/filestream.yml.hbs

* Update packages/microsoft_exchange_server/data_stream/httpproxy/agent/stream/filestream.yml.hbs

* Update packages/microsoft_exchange_server/data_stream/httpproxy/agent/stream/filestream.yml.hbs

* Update packages/microsoft_exchange_server/data_stream/httpproxy/agent/stream/filestream.yml.hbs

* Update packages/microsoft_exchange_server/data_stream/httpproxy/agent/stream/filestream.yml.hbs
* Docker Enablement for v9.0.0

Signed-off-by: Andreas Gkizas <[email protected]>
…ingress, istio and containerd integrations (elastic#12535)

* Enabling 9.0.0 kibana support for  kubernetes, kubernetes_otel, nginx_ingress, istio and containerd integrations

Signed-off-by: Andreas Gkizas <[email protected]>
@mrodm mrodm self-assigned this Feb 5, 2025
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @mrodm

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
57.2% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants