Elasticsearch

Publisher: Splunk
Connector Version: 2.0.6
Product Vendor: Elastic
Product Name: Elasticsearch
Product Version Supported (regex): ".*"
Minimum Product Version: 5.1.0

This app integrates with an Elasticsearch installation to implement ingestion and investigative actions

Elasticsearch installations can be configured to allow REST API access without any type of authentication. The app therefore marks username and password as optional parameters. If specified, the app will end up using these for generating the basic authentication header used in the various Elasticsearch REST endpoints.
Connection can be configured over HTTP or HTTPS, so if test connectivity fails please check the protocol.

Configuration Variables

The below configuration variables are required for this Connector to operate. These variables are specified when configuring a Elasticsearch asset in SOAR.

VARIABLE	REQUIRED	TYPE	DESCRIPTION
url	required	string	Device URL including the port, e.g. https://myelastic.enterprise.com:9200
verify_server_cert	optional	boolean	Verify server certificate
username	optional	string	Username
password	optional	password	Password
ingest_index	optional	string	Ingestion index
ingest_type	optional	string	Ingestion type
ingest_routing	optional	string	Ingestion routing
ingest_query	optional	string	Ingestion query
ingest_parser	optional	file	Custom Elasticsearch parser

Supported Actions

test connectivity - Validate the asset configuration for connectivity. This action logs into the device to check the connection and credentials
get config - Returns the list of indices and types currently configured on the ElasticSearch instance
run query - Run a search query on the Elasticsearch installation. Please escape any quotes that are part of the query string
on poll - Run a query in elasticsearch and ingest the results

action: 'test connectivity'

Validate the asset configuration for connectivity. This action logs into the device to check the connection and credentials

Type: test
Read only: True

Action Parameters

No parameters are required for this action

Action Output

No Output

action: 'get config'

Returns the list of indices and types currently configured on the ElasticSearch instance

Type: investigate
Read only: True

Action Parameters

No parameters are required for this action

Action Output

DATA PATH	TYPE	CONTAINS
action_result.status	string
action_result.data.*.index	string	elasticsearch index
action_result.data.*.types	string	elasticsearch type
action_result.summary.total_indices	numeric
action_result.message	string
summary.total_objects	numeric
summary.total_objects_successful	numeric

action: 'run query'

Run a search query on the Elasticsearch installation. Please escape any quotes that are part of the query string

Type: investigate
Read only: True

The action executes the query on an Elasticsearch installation by doing a POST on the REST endpoint 'base_url/index/type/_search' with the input query as the data. Please see the Elasticseach website for query format and documentation.
The routing parameter is appended as a parameter in the REST call if specified.
As an e.g. the following query returns only the id and name of all the items in an index of the specified type
{ "query": { "match_all": {} }, "_source": ["id", "name"]}.

Action Parameters

PARAMETER	REQUIRED	DESCRIPTION	TYPE	CONTAINS
index	required	Index to query on	string	elasticsearch index
type	optional	Type to query on	string	elasticsearch type
routing	optional	Shards to query on (routing value)	string
query	required	Query to run (in ElasticSearch language)	string	elasticsearch query

Action Output

DATA PATH	TYPE	CONTAINS
action_result.status	string
action_result.parameter.index	string	elasticsearch index
action_result.parameter.query	string	elasticsearch query
action_result.parameter.routing	string
action_result.parameter.type	string	elasticsearch type
action_result.data.*._shards.failed	numeric
action_result.data.*._shards.successful	numeric
action_result.data.*._shards.total	numeric
action_result.data.*.hits.hits.*._id	string
action_result.data.*.hits.hits.*._index	string
action_result.data.*.hits.hits.*._score	numeric
action_result.data.*.hits.hits.*._source	numeric
action_result.data.*.hits.hits.*._type	string
action_result.data.*.hits.hits.*.fields	numeric
action_result.data.*.hits.max_score	numeric
action_result.data.*.hits.total	numeric
action_result.data.*.timed_out	boolean
action_result.data.*.took	numeric
action_result.summary.timed_out	boolean
action_result.summary.total_hits	numeric
action_result.message	string
summary.total_objects	numeric
summary.total_objects_successful	numeric

action: 'on poll'

Run a query in elasticsearch and ingest the results

Type: ingest
Read only: True

This will run a query in elasticsearch using the index, type, routing, and query configured in the app settings and ingest the results. The query is not modified by Phantom in any way before being requested in elasticsearch. This means that the query must account for relative time between ingestion runs, query limits, and page sizes.

The raw JSON response from elasticsearch is passed to a parser script which returns a list of containers and artifacts. If a custom parsing script is not provided, the default parsing script is used:

def ingest_parser(data):
results = []
if not isinstance(data, dict):
return results

hits = data\.get\('hits', \{\}\)\.get\('hits', \[\]\)
for hit in hits\:
    container = \{\}
    artifacts = \[\]

    \# anything printed to stdout will be added to the phantom debug logs
    print\('Found hit \{\}\. Building container'\.format\(hit\['\_id'\]\)\)

    container\['run\_automation'\] = False
    container\['source\_data\_identifier'\] = hit\['\_id'\]
    container\['name'\] = 'Elasticsearch\: \{\} \{\} \{\}'\.format\(hit\['\_index'\],
                                                         hit\['\_type'\],
                                                         hit\['\_id'\]\)

    artifacts\.append\(\{
        \# always True since there is only one
        'run\_automation'\: True,
        'label'\: 'event',
        'name'\: 'elasticsearch event',
        'cef'\: hit\.get\('\_source'\),
        'source\_data\_identifier'\: hit\['\_id'\]
    \}\)

    results\.append\(\{
        'container'\: container,
        'artifacts'\: artifacts
    \}\)

return results

.

Action Parameters

PARAMETER	REQUIRED	DESCRIPTION	TYPE	CONTAINS
container_id	optional	Limit ingestion to these container IDs	string
start_time	optional	Start of time range in epoch time (default: 10 days ago)	numeric
end_time	optional	End of time range in epoch time (default: now)	numeric
container_count	optional	Maximum number of containers to create	numeric
artifact_count	optional	Maximum number of artifacts to create per container	numeric

Action Output

No Output

action: 'modify query'

Modifies the selected index mapping with a field specified by the user

Type: investigate
Read only: True

The action executes the query on an Elasticsearch installation by doing a PUT on the REST endpoint 'base_url/index/_mapping' with the input query as the data. Please see the Elasticseach website for query format and documentation.

Action Parameters

PARAMETER	REQUIRED	DESCRIPTION	TYPE	CONTAINS
index	required	Index to query on	string	elasticsearch index
query	required	Query to run (in ElasticSearch language)	string	elasticsearch query

Action Output

DATA PATH	TYPE	CONTAINS
action_result.status	string
action_result.data.*.index	string	elasticsearch index
action_result.parameter.query	string	elasticsearch query
action_result.parameter.index	string	elasticsearch index
action_result.message	string

action: 'get mapping'

GET query action on mapping on the Elasticsearch installation to obtain current look for specified index mapping

Type: investigate
Read only: True

The action executes the query on an Elasticsearch installation by doing a GET on the REST endpoint 'base_url/index/_mapping. Please see the Elasticseach website for query format and documentation.

Action Parameters

PARAMETER	REQUIRED	DESCRIPTION	TYPE	CONTAINS
index	required	Index to query on	string	elasticsearch index

Action Output

DATA PATH	TYPE	CONTAINS
action_result.status	string
action_result.data.*.index	string	elasticsearch index
action_result.data.*.properties	string	elasticsearch mapping
action_result.message	string