Split filter - logstash - No example on the web - OR it does not work - need assistance.....

[jordansissel]

(This issue was originally filed by @MKA-MKA at elastic/logstash#2345)

---

Hello,
I am trying to set up and pull JSON data from S3 to Logstash in to ElasticSearch with split filter and I have some issues. The S3 data has 25 events in a single line separated by curly braces like { } and I want to use split filter to separate them during the pull from S3 using config file, I have very limited examples for split filter. can you guys help me.

Thanks
Karan

[dhermans]

Hi,
I guess you are trying to pull cloudtrail data into logstash?
Refer here:
http://techblog.mdsol.com/2014/01/27/parsing-amazon-cloudtrail-json-logs-with-a-customized-logstash-build.html

for a patched 1.4 logstash that achieves this split and refiltering correctly..

I have tried to follow the flow from pull request 793 but can't see how to do this in 1.5 even though it is marked as fixed...

i use the 'jq method' (referenced same page) which uses jq to split the single large array into elements before injecting:

gzip -dc cloudtrail-log.json.gz | jq -r -M .Records[] -c | logstash -f /conf/file.conf