Http output plugin

../../../../logstash/docs/include/plugin_header.asciidoc

Description

This output lets you send events to a generic HTTP(S) endpoint.

This output will execute up to 'pool_max' requests in parallel for performance. Consider this when tuning this plugin for performance.

Additionally, note that when parallel execution is used strict ordering of events is not guaranteed!

Beware, this gem does not yet support codecs. Please use the 'format' option for now.

Retry policy

This output has two levels of retry: library and plugin.

Library retry

The library retry applies to IO related failures. Non retriable errors include SSL related problems, unresolvable hosts, connection issues, and OS/JVM level interruptions happening during a request.

The options for library retry are:

automatic_retries. Controls the number of times the plugin should retry after failures at the library level.
retry_non_idempotent. When set to false, GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be retried.

Plugin retry

The options for plugin level retry are:

retry_failed. When set to true, the plugin retries indefinitely for HTTP error response codes defined in the retryable_codes option (429, 500, 502, 503, 504) and retryable exceptions (socket timeout/ error, DNS resolution failure and client protocol exception).
retryable_codes. Sets http response codes that trigger a retry.

Note	The `retry_failed` option does not control the library level retry.

Http Output Configuration Options

This plugin supports the following configuration options plus the [plugins-outputs-http-common-options] described later.

Note	As of version `6.0.0` of this plugin, a number of previously deprecated settings related to SSL have been removed. Please check out HTTP Output Obsolete Configuration Options for details.

Setting	Input type	Required
`automatic_retries`	number	No
`connect_timeout`	number	No
`content_type`	string	No
`cookies`	boolean	No
`follow_redirects`	boolean	No
`format`	string, one of `["json", "json_batch", "form", "message"]`	No
`headers`	hash	No
`http_compression`	boolean	No
`http_method`	string, one of `["put", "post", "patch", "delete", "get", "head"]`	Yes
`ignorable_codes`	number	No
`keepalive`	boolean	No
`mapping`	hash	No
`message`	string	No
`pool_max`	number	No
`pool_max_per_route`	number	No
`proxy`	<<,>>	No
`request_timeout`	number	No
`retry_failed`	boolean	No
`retry_non_idempotent`	boolean	No
`retryable_codes`	number	No
`socket_timeout`	number	No
`ssl_certificate`	path	No
`ssl_certificate_authorities`	list of path	No
`ssl_cipher_suites`	list of string	No
`ssl_enabled`	boolean	No
`ssl_keystore_password`	password	No
`ssl_keystore_path`	path	No
`ssl_keystore_type`	string	No
`ssl_supported_protocols`	string	No
`ssl_truststore_password`	password	No
`ssl_truststore_path`	path	No
`ssl_truststore_type`	string	No
`ssl_verification_mode`	string, one of `["full", "none"]`	No
`url`	string	Yes
`validate_after_inactivity`	number	No

Also see [plugins-outputs-http-common-options] for a list of options supported by all output plugins.

`automatic_retries`

Value type is number
Default value is 1

How many times should the client retry a failing URL. We recommend setting this option to a value other than zero if the keepalive option is enabled. Some servers incorrectly end keepalives early, requiring a retry. See Retry Policy for more information.

`connect_timeout`

Value type is number
Default value is 10

Timeout (in seconds) to wait for a connection to be established. Default is 10s

`content_type`

Value type is string
There is no default value for this setting.

Content type

If not specified, this defaults to the following:

if format is "json", "application/json"
if format is "json_batch", "application/json". Each Logstash batch of events will be concatenated into a single array and sent in one request.
if format is "form", "application/x-www-form-urlencoded"

`cookies`

Value type is boolean
Default value is true

Enable cookie support. With this enabled the client will persist cookies across requests as a normal web browser would. Enabled by default

`follow_redirects`

Value type is boolean
Default value is true

Should redirects be followed? Defaults to true

`format`

Value can be any of: json, json_batch, form, message
Default value is "json"

Set the format of the http body.

If json_batch, each batch of events received by this output will be placed into a single JSON array and sent in one request. This is particularly useful for high throughput scenarios such as sending data between Logstash instaces.

If form, then the body will be the mapping (or whole event) converted into a query parameter string, e.g. foo=bar&baz=fizz…

If message, then the body will be the result of formatting the event according to message

Otherwise, the event is sent as json.

`headers`

Value type is hash
There is no default value for this setting.

Custom headers to use format is headers ⇒ ["X-My-Header", "%{host}"]

`http_compression`

Value type is boolean
Default value is false

Enable request compression support. With this enabled the plugin will compress http requests using gzip.

`http_method`

This is a required setting.
Value can be any of: put, post, patch, delete, get, head
There is no default value for this setting.

The HTTP Verb. One of "put", "post", "patch", "delete", "get", "head"

`ignorable_codes`

Value type is number
There is no default value for this setting.

If you would like to consider some non-2xx codes to be successes enumerate them here. Responses returning these codes will be considered successes

`keepalive`

Value type is boolean
Default value is true

Turn this on to enable HTTP keepalive support. We highly recommend setting automatic_retries to at least one with this to fix interactions with broken keepalive implementations.

`mapping`

Value type is hash
There is no default value for this setting.

This lets you choose the structure and parts of the event that are sent.

For example:

   mapping => {"foo" => "%{host}"
              "bar" => "%{type}"}

`message`

Value type is string
There is no default value for this setting.

`pool_max`

Value type is number
Default value is 50

Max number of concurrent connections. Defaults to 50

`pool_max_per_route`

Value type is number
Default value is 25

Max number of concurrent connections to a single host. Defaults to 25

`proxy`

Value type is string
There is no default value for this setting.

If you’d like to use an HTTP proxy . This supports multiple configuration syntaxes:

Proxy host in form: http://proxy.org:1234
Proxy host in form: {host ⇒ "proxy.org", port ⇒ 80, scheme ⇒ 'http', user ⇒ 'username@host', password ⇒ 'password'}
Proxy host in form: {url ⇒ 'http://proxy.org:1234', user ⇒ 'username@host', password ⇒ 'password'}

`request_timeout`

Value type is number
Default value is 60

This module makes it easy to add a very fully configured HTTP client to logstash based on [Manticore](https://github.com/cheald/manticore). For an example of its usage see https://github.com/logstash-plugins/logstash-input-http_poller Timeout (in seconds) for the entire request

`retry_failed`

Value type is boolean
Default value is true

Note that this option controls plugin-level retries only. It has no affect on library-level retries.

Set this option to false if you want to disable infinite retries for HTTP error response codes defined in the retryable_codes or retryable exceptions (Timeout, SocketException, ClientProtocolException, ResolutionFailure and SocketTimeout). See Retry policy for more information.

`retry_non_idempotent`

Value type is boolean
Default value is false

When this option is set to false and automatic_retries is enabled, GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be retried.

When set to true and automatic_retries is enabled, this will cause non-idempotent HTTP verbs (such as POST) to be retried. See Retry Policy for more information.

`retryable_codes`

Value type is number
Default value is [429, 500, 502, 503, 504]

If the plugin encounters these response codes, the plugin will retry indefinitely. See Retry Policy for more information.

`socket_timeout`

Value type is number
Default value is 10

Timeout (in seconds) to wait for data on the socket. Default is 10s

`ssl_certificate`

Value type is path
There is no default value for this setting.

SSL certificate to use to authenticate the client. This certificate should be an OpenSSL-style X.509 certificate file.

Note	This setting can be used only if `ssl_key` is set.

`ssl_certificate_authorities`

Value type is a list of path
There is no default value for this setting

The .cer or .pem CA files to validate the server’s certificate.

`ssl_cipher_suites`

Value type is a list of string
There is no default value for this setting

The list of cipher suites to use, listed by priorities. Supported cipher suites vary depending on the Java and protocol versions.

`ssl_enabled`

Value type is boolean
Default value is true

Enable SSL/TLS secured communication. It must be true for other ssl_ options to take effect.

`ssl_key`

Value type is path
There is no default value for this setting.

OpenSSL-style RSA private key that corresponds to the ssl_certificate.

Note	This setting can be used only if `ssl_certificate` is set.

`ssl_keystore_password`

Value type is password
There is no default value for this setting.

Set the keystore password

`ssl_keystore_path`

Value type is path
There is no default value for this setting.

The keystore used to present a certificate to the server. It can be either .jks or .p12

`ssl_keystore_type`

Value can be any of: jks, pkcs12
If not provided, the value will be inferred from the keystore filename.

The format of the keystore file. It must be either jks or pkcs12.

`ssl_supported_protocols`

Value type is string
Allowed values are: 'TLSv1.1', 'TLSv1.2', 'TLSv1.3'
Default depends on the JDK being used. With up-to-date Logstash, the default is ['TLSv1.2', 'TLSv1.3']. 'TLSv1.1' is not considered secure and is only provided for legacy applications.

List of allowed SSL/TLS versions to use when establishing a connection to the HTTP endpoint.

For Java 8 'TLSv1.3' is supported only since 8u262 (AdoptOpenJDK), but requires that you set the LS_JAVA_OPTS="-Djdk.tls.client.protocols=TLSv1.3" system property in Logstash.

Note

If you configure the plugin to use 'TLSv1.1' on any recent JVM, such as the one packaged with Logstash, the protocol is disabled by default and needs to be enabled manually by changing jdk.tls.disabledAlgorithms in the $JDK_HOME/conf/security/java.security configuration file. That is, TLSv1.1 needs to be removed from the list.

`ssl_truststore_password`

Value type is password
There is no default value for this setting.

Set the truststore password

`ssl_truststore_path`

Value type is path
There is no default value for this setting.

The truststore to validate the server’s certificate. It can be either .jks or .p12.

`ssl_truststore_type`

Value can be any of: jks, pkcs12
If not provided, the value will be inferred from the truststore filename.

The format of the truststore file. It must be either jks or pkcs12.

`ssl_verification_mode`

Value type is string
Supported values are: full, none
Default value is full

Controls the verification of server certificates. The full option verifies that the provided certificate is signed by a trusted authority (CA) and also that the server’s hostname (or IP address) matches the names identified within the certificate.

The none setting performs no verification of the server’s certificate. This mode disables many of the security benefits of SSL/TLS and should only be used after cautious consideration. It is primarily intended as a temporary diagnostic mechanism when attempting to resolve TLS errors. Using none in production environments is strongly discouraged.

`url`

This is a required setting.
Value type is string
There is no default value for this setting.

URL to use

`validate_after_inactivity`

Value type is number
Default value is 200

How long to wait before checking if the connection is stale before executing a request on a connection using keepalive. You may want to set this lower, possibly to 0 if you get connection errors regularly Quoting the Apache commons docs (this client is based Apache Commmons): 'Defines period of inactivity in milliseconds after which persistent connections must be re-validated prior to being leased to the consumer. Non-positive value passed to this method disables connection validation. This check helps detect connections that have become stale (half-closed) while kept inactive in the pool.' See these docs for more info

HTTP Output Obsolete Configuration Options

Warning

As of version 6.0.0 of this plugin, some configuration options have been replaced. The plugin will fail to start if it contains any of these obsolete options.

Setting	Replaced by
cacert	`ssl_certificate_authorities`
client_cert	`ssl_certificate`
client_key	`ssl_key`
keystore	`ssl_keystore_path`
keystore_password	`ssl_keystore_password`
keystore_type	`ssl_keystore_password`
truststore	`ssl_truststore_path`
truststore_password	`ssl_truststore_password`
truststore_type	`ssl_truststore_type`

../../../../logstash/docs/include/output.asciidoc

Files

index.asciidoc

Latest commit

History

index.asciidoc

File metadata and controls

Http output plugin

Description

Retry policy

Library retry

Plugin retry

Http Output Configuration Options

automatic_retries

connect_timeout

content_type

cookies

follow_redirects

format

headers

http_compression

http_method

ignorable_codes

keepalive

mapping

message

pool_max

pool_max_per_route

proxy

request_timeout

retry_failed

retry_non_idempotent

retryable_codes

socket_timeout

ssl_certificate

ssl_certificate_authorities

ssl_cipher_suites

ssl_enabled

ssl_key

ssl_keystore_password

ssl_keystore_path

ssl_keystore_type

ssl_supported_protocols

ssl_truststore_password

ssl_truststore_path

ssl_truststore_type

ssl_verification_mode

url

validate_after_inactivity