You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+21-19Lines changed: 21 additions & 19 deletions
Original file line number
Diff line number
Diff line change
@@ -4,11 +4,13 @@ puppet-splunk_hec
4
4
Description
5
5
-----------
6
6
7
-
This is a report processor designed to send a report summary of useful information to the [Splunk HTTP Endpoint Collector "HEC"](http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) service. These summaries are designed to be informative but also not too verbose to make logging a burden to the enderuser. The summaries are meant to be small but sufficient to determine if a puppet run was successful on a node, and to include metadata such as code-id, transaction-id, and other details to allow for more detailed actions to be done.
7
+
Puppet collects a wide variety of useful information about the servers it manages. When you have key Puppet data in Splunk, you can save time and make richer analyses. For example, send an alert if Puppet sees unexpected change on a server. To make this possible, this is a Puppet report processor designed to send a report summary of useful information to the [Splunk HTTP Endpoint Collector "HEC"](http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) service. These summaries are designed to be informative but also not too verbose to make logging a burden to the enderuser. The summaries are meant to be small but sufficient to determine if a Puppet run was successful on a node, and to include metadata such as code-id, transaction-id, and other details to allow for more detailed actions to be done.
8
8
9
-
It is best used the Splunk Addon [Puppet Report Viewer](https://splunkbase.splunk.com/app/4413/) which adds sourcetypes to make ingesting this data easier into Splunk (sourcetypes can be associated with specifc HEC tokens to make event viewing/processing easier). The Report Viewer also adds an actionable alert for Puppet Enterprise Users: using the data from a `puppet:summary` event, the Detailed Report Builder actionable alert will create a new event with the type of `puppet:detailed` containing information such as the node in questions facts, the resource_events from the node, and links to relavant reports in the Puppet Enterprise Console.
9
+
It is best used with the Splunk Addon [Puppet Report Viewer](https://splunkbase.splunk.com/app/4413/), which adds sourcetypes to make ingesting this data easier into Splunk. Sourcetypes can also be associated with specifc HEC tokens to make event viewing/processing easier.
10
10
11
-
There are also two tasks included in this module, `splunk_hec:bolt_apply` and `splunk_hec:bolt_result` designed to provide similar data formats to allow for Bolt Plans to be written that submit data to Splunk. Also included are plans showing example useage of the tasks.
11
+
Because Puppet and Bolt excel at taking action, the Report Viewer also adds an actionable alert for Puppet Enterprise users: using the data from a `puppet:summary` event, the Detailed Report Builder actionable alert will create a new event with the type of `puppet:detailed` containing information such as the node in questions facts, the resource_events from the node, and links to relavant reports in the Puppet Enterprise Console.
12
+
13
+
There are two Tasks included in this module, `splunk_hec:bolt_apply` and `splunk_hec:bolt_result` that are designed to provide similar data formats to allow for Bolt Plans that submit data to Splunk. Also included are Plans showing example useage of the Tasks.
12
14
13
15
14
16
Requirements
@@ -22,30 +24,30 @@ This was tested on both Puppet Enterprise 2018.1.4 & Puppet 6, using stock gems
22
24
Report Processor Installation & Usage
23
25
--------------------
24
26
25
-
The steps below will help one install and troubleshoot the report processor on a single Puppet Master, including manual steps to configure a puppet-server, and to use the included splunk_hec class.
27
+
The steps below will help install and troubleshoot the report processor on a single Puppet Master, including manual steps to configure a puppet-server, and to use the included splunk_hec class.
26
28
27
-
1. Install the Puppet Report Viewer Addon in Splunk. This will import the needed sourcetypes that make setting up the HEC easier in the next steps, and also some overview dashboards that make it a lot easier to see if you're getting reports into Splunk.
29
+
1. Install the Puppet Report Viewer Addon in Splunk. This will import the needed sourcetypes that make setting up the HEC easier in the next steps, and also some overview dashboards that make it a lot easier to see if you're sending Puppet run reports into Splunk.
28
30
29
31
2. Create a Splunk HEC Token (preferably named `puppet:summary` and using the sourcetype `puppet:summary` from the Report Viewer addon). Follow the steps provided by Splunk's [Getting Data In Guide](http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) if you are new to HTTP Endpoint Collectors.
30
32
31
-
3. Install this module in the environment your Puppet Server's are using (probably `production`)
33
+
3. Install this Puppet module in the environment that manages your Puppet Servers are using (probably `production`)
32
34
33
-
4. Run `puppet plugin download` on your Puppet Master to sync the new
35
+
4. Run `puppet plugin download` on your Puppet Server to sync the content
34
36
35
37
5. Create a `/etc/puppetlabs/puppet/splunk_hec.yaml` (see examples directory for one) adding your Splunk Server & Token from step 1
36
38
- You can add 'timeout' as an optional parameter, default value is 2 for both open and read sessions, so take value x2 for real world use
37
-
- The same is true for port, defaults to 8088 if none provided
38
-
- Provide a 'puppetdb\_callback\_hostname' variable if the hostname that Splunk will use to lookup further information about a report is different than the puppetserver processing the reports (ie, multiple servers, load balancer, external dns name vs internal, etc.) Defaults to the certname of the puppetserver processing the report. This feature is yet to be enabled in the Puppet Report Viewer.
39
+
- The same is true for port, defaults to 8088 if none is provided
40
+
- Provide a `puppetdb\_callback\_hostname` variable if the hostname that Splunk will use to lookup further information about a report is different than the Puppet Server processing the reports (i.e., multiple servers, load balancer, external dns name vs internal, etc.) This defaults to the certname of the Puppet Server processing the report. Note that this feature has yet to be enabled in the Puppet Report Viewer.
39
41
40
42
```
41
43
---
42
44
"server" : "splunk-dev.testing.internal"
43
45
"token" : "13311780-EC29-4DD0-A796-9F0CDC56F2AD"
44
46
```
45
47
46
-
6. Run `puppet apply -e 'notify { "hello world": }' --reports=splunk_hec` from the puppet server, this will load the report processor and test your configuration settings without actually modifying your puppet servers running configuration. If you are using the Puppet Report Viewer app in Splunk, you will see the page update with new data. If not, you will want to perform a search by the sourcetype you provided with your HEC configuration.
48
+
6. Run `puppet apply -e 'notify { "hello world": }' --reports=splunk_hec` from the Puppet Server, this will load the report processor and test your configuration settings without actually modifying your Puppet Server's running configuration. If you are using the Puppet Report Viewer app in Splunk then you will see the page update with new data. If not, perform a search by the sourcetype you provided with your HEC configuration.
47
49
48
-
7. Provide the working parameters / values to the splunk_hec class and use it in a profile or add it to the PE Masters subgroup of PE Infrastructure in the classification section of the console. Run puppet on the MoM first (because it is the Puppet Server all the other compile masters are using) before running puppet on the other compile masters. This will restart the puppet-server processor, so stagger the runs to prevent an outage.
50
+
7. Provide the working parameters / values to the splunk_hec class and use it in a profile or add it to the PE Masters subgroup of PE Infrastructure in the classification section of the console. Run Puppet on the MoM first (because it is the Puppet Server all the other compile masters are using) before running puppet on the other compile masters. This will restart the puppet-server processor, so stagger the runs to prevent an outage.
- Restart the puppetserver process for it to reload the configuration and the plugin
63
+
- Restart the puppet-server process for it to reload the configuration and the plugin
62
64
63
-
- Run `puppet agent -t` somewhere, if you are using the suggested name, use `source="http:puppet-report-summary"` in your splunk search field to show the reports as they arrive
65
+
- Run `puppet agent -t` somewhere, if you are using the suggested name, use `source="http:puppet-report-summary"` in your Splunk search field to show the reports as they arrive
64
66
65
67
66
68
SSL Support
67
69
-----------
68
70
Configuring SSL support for this report processor and tasks requires that the Splunk HEC service being used has a [properly configured SSL certificate](https://docs.splunk.com/Documentation/Splunk/latest/Security/AboutsecuringyourSplunkconfigurationwithSSL). Once the HEC service has a valid SSL certificate, the CA will need to be made available to the report processor to load. One could add the CA to Puppet's trust, or just make the CA file available on the puppet-server (/etc/puppetlabs/puppet/splunk\_hec/splunk\_ca.cert works). Either option is supported.
69
71
70
-
One can update the splunk_hec.yaml file with the below settings
72
+
One can update the splunk_hec.yaml file with these settings:
Or create a profile that copies the splunk_ca.cert as part of invoking the splunk_hec class.
80
+
Or create a profile that copies the `splunk_ca.cert` as part of invoking the splunk_hec class:
79
81
80
82
```
81
83
class profile::splunk_hec {
@@ -98,13 +100,13 @@ class profile::splunk_hec {
98
100
Tasks
99
101
-----
100
102
101
-
Two tasks are provided for submitting data from a Bolt plan to Splunk. It is suggested to use a different HEC token to distinguish between events from Puppet runs and those generated by Bolt. The Puppet Report Viewer addon includes a puppet:bolt sourcetype to faciltate this. Currently SSL validation for bolt communications to Splunk is not supported.
103
+
Two tasks are provided for submitting data from a Bolt plan to Splunk. For clarity, we recommend using a different HEC token to distinguish between events from Puppet runs and those generated by Bolt. The Puppet Report Viewer addon includes a puppet:bolt sourcetype to faciltate this. Currently SSL validation for Bolt communications to Splunk is not supported.
102
104
103
-
`splunk_hec::bolt_apply`: A task that uses the remote task option of Bolt to submit a Bolt Apply report in a similar format to the puppet:summary. Unlike the summary, this includes the facts from a target because those are available to bolt at execution time and added to the report data before submission to Splunk.
105
+
`splunk_hec::bolt_apply`: A task that uses the remote task option to submit a Bolt Apply report in a similar format to the puppet:summary. Unlike the summary, this includes the facts from a target because those are available to bolt at execution time and added to the report data before submission to Splunk.
104
106
105
107
`splunk_hec::bolt_result`: A task that sends the result of a function to Splunk. Since the format is freeform and dependent on the individual function/tasks being called, formatting of the data is best done in the plan itself prior to submitting the result hash to the task.
106
108
107
-
To setup, one needs to add the splunk_hec endpoint as a remote target in the inventory.yml:
109
+
To setup, add the splunk_hec endpoint as a remote target in `inventory.yml`:
108
110
109
111
```
110
112
---
@@ -118,7 +120,7 @@ nodes:
118
120
port: 8088
119
121
```
120
122
121
-
See the plans/ directory for working examples of apply and result usage.
123
+
See the `plans/` directory for working examples of apply and result usage.
0 commit comments