Skip to content
This repository was archived by the owner on Feb 2, 2021. It is now read-only.

Allow piping metrics into the conversion script #25

Closed
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -64,3 +64,13 @@ When `--netcat` is used with InfluxDB output, data will be sent to port 8086. Th
```
./json2graphite.rb ~/Downloads/logdump/puppetserver/*.json --convert-to influxdb --netcat localhost --influx-db pe-metrics
```

### Pipe data into json2graphite.rb

You can pipe the output of the metrics script from pe_metric_curl_cron_jobs directly into json2graphite.rb such as:

```
/opt/puppetlabs/pe_metric_curl_cron_jobs/scripts/tk_metrics --metrics_type puppetdb | ./json2graphite.rb
```

However, you cannot pipe metrics data in and read a file in the same invocation of the script.
28 changes: 27 additions & 1 deletion json2graphite.rb
Original file line number Diff line number Diff line change
@@ -65,6 +65,10 @@ def close
def parse_file(filename)
data = JSON.parse(File.read(filename))

parse_input(data)
end

def parse_input( data )
# Newer versions of the log tool insert a timestamp field into the JSON.
if data['timestamp']
timestamp = Time.parse(data.delete('timestamp'))
@@ -316,9 +320,17 @@ def influx_metrics(data, timestamp, parent_key = nil)
$net_output = NetworkOutput.new(url)
end

data_files = ARGV
data_files = []
data_files += Dir.glob($options[:pattern]) if $options[:pattern]

#http://ruby-doc.org/core-1.9.3/ARGF.html#method-i-filename
while ARGF.filename != '-'
filename = ARGF.filename
data_files += [filename]
ARGF.skip
break if filename == ARGF.filename
end

data_files.each do |filename|
begin
converted_data = parse_file(filename)
@@ -333,4 +345,18 @@ def influx_metrics(data, timestamp, parent_key = nil)
end
end

if ARGF.filename == '-'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is hanging because ruby always has the option to read from STDIN. So - will be present even if you are not piping anything in. It hangs in this repo because we do not pipe anything in. Perhaps we can gate reading from STDIN on a command line argument?

begin
converted_data = parse_input( JSON.parse(ARGF.read) )

if $options[:host]
$net_output.write(converted_data)
else
STDOUT.write(converted_data)
end
rescue => e
STDERR.puts "ERROR: During read from STDIN: #{e.message}"
end
end

$net_output.close if $options[:host]