-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Generate component ID * test id change * lint: package comment * feat(resource): Connectors for configuration v2 (#141) * add connectors * support delete * move readRolloutOptions() * WIP: We need component ID to be generated before apply * generate connector id * Remove test file * feat: Configuration V2 with routing (#143) * support delete * WIP: We need component ID to be generated before apply * Remove test file * rough working concept * test some advanced routing * remove route id, bindplane can generate it * input validation * resolve validation func * linter * fix issue with telemetry type and route id not persisting * feat: Configuration V2 Processors (#144) * support delete * WIP: We need component ID to be generated before apply * test some advanced routing * remove route id, bindplane can generate it * input validation * resolve validation func * fix issue with telemetry type and route id not persisting * WIP: We need component ID to be generated before apply * implement create * implement read * sync tests * chore: Test against v1.74.0 - current (#145) * lets see which versions work :) * tighten up versions * add version notice * doc processor groups on config v2 (#146) * feat: Config v2 connectors (#147) * add connector to config v2 schema * fix tests from bad rebase upstream * support connectors in the configuration * chore: Config v2 route handling refactor (#148) * refactor route parsing * Ensure routes returned are not nil * refactor config v2 read routes * typo * fix: Config v2 routes should use TypeSet to avoid ordering requirements (#149) * Fix an issue where the config v2 read func was not comparing route_id correctly * refactor RoutesToState further * use TypeSet for routes in order to avoid ordering constraint * ci * Route (#150) * unify route schema * implement route_id and enable connector support * add license * linter fixes * doc * fix bad rebase
- Loading branch information
Showing
19 changed files
with
3,234 additions
and
51 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,265 @@ | ||
| --- | ||
| subcategory: "Pipeline" | ||
| description: |- | ||
| A Configuration is a combination of sources, processors, and destinations | ||
| used by BindPlane OP to generate an agent configuration. | ||
| Configuration V2 is the latest iteration of the BindPlane Configuration resource. | ||
| It supports advanced routing and OpenTelemetry Connectors. | ||
| --- | ||
|
|
||
| > [!NOTE] | ||
| > bindplane_configuration_v2 resources are supported by BindPlane OP version 1.85.0 and later. | ||
| # bindplane_configuration_v2 | ||
|
|
||
| The `bindplane_configuration_v2` resource creates a BindPlane configuration that can be applied | ||
| to one or more managed agents. Configurations are a combination of [sources](./bindplane_source.md), | ||
| [destinations](./bindplane_destination.md), [processors](./bindplane_processor.md), and [connectors](./bindplane_connector.md). | ||
| Explicit routes can be defined between components to control how telemetry data flows through the configuration. | ||
|
|
||
| Configuration V2 builds upon [Configuration V1](./bindplane_configuration.md) by introducing the following features: | ||
| - **Routing**: Routes can be defined between all components. This allows for | ||
| more granular control over how telemetry data flows through the configuration. | ||
| - **Connectors**: [OpenTelemetry Connectors](https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md) | ||
| can be defined in the configuration. Connectors are used to route telemetry between pipelines, such as counting the number of logs | ||
| and emitting a metric based on that count. | ||
| - **Processors**: Processors can be defined in the configuration without attaching them directly to sources or destinations. | ||
| This allows for more flexibility in how processors are used within a configuration. Processors can still be attached to sources | ||
| or destinations just like before. | ||
|
|
||
| ## Options | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------ | --------------- | -------- | --------------------------------------------------------------------------- | | ||
| | `name` | string | required | The source name. | | ||
| | `platform` | string | required | The platform the configuration supports. See the [supported platforms](./bindplane_configuration.md#supported-platforms) section. | | ||
| | `labels` | map | optional | Key value pairs representing labels to set on the configuration. | | ||
| | `source` | block | optional | One or more source blocks. See the [source block](./bindplane_configuration.md#source-block) section. | | ||
| | `processor_group` | block | optional | One or more processor group blocks. See the [processor group block](./bindplane_configuration.md#processor-group-block) section. | | ||
| | `destination` | block | optional | One or more destination blocks. See the [destination block](./bindplane_configuration.md#destination-block) section. | | ||
| | `extensions` | list(string) | optional | One or more extension names to attach to the configuration. | | ||
| | `rollout` | bool | required | Whether or not updates to the configuration should trigger an automatic rollout of the configuration. | | ||
| | `rollout_options` | block (single) | optional | Options for configuring the rollout behavior of the configuration. See the [rollout options block](./bindplane_configuration.md#rollout-options-block) section. | | ||
|
|
||
| ### Source Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `name` | string | required | The source name. | | ||
| | `processors` | list(string) | optional | One or more processor names to attach to the source. | | ||
| | `route` | string | optional | One or more routes to attach to the source. See the [route block](./bindplane_configuration.md#route-block) section. | | ||
|
|
||
| ### Processor Group Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `route_id` | string | required | An arbitrary string that can be used to configure routes to this processor group. | | ||
| | `processors` | list(string) | optional | One or more processor names to attach to the processor group. | | ||
| | `route` | string | optional | One or more routes to attach to the processor group. See the [route block](./bindplane_configuration.md#route-block) section. | | ||
|
|
||
| ### Destination Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `route_id` | string | required | An arbitrary string that can be used to configure routes to this destination. | | ||
| | `name` | string | required | The source name. | | ||
| | `processors` | list(string) | optional | One or more processor names to attach to the destination. | | ||
|
|
||
| ### Rollout Options Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `type` | string | required | The type of rollout to perform. Valid values are 'standard' and 'progressive'. | | ||
| | `parameters` | list(block) | optional | One or more parameters for the rollout. See the [parameters block](./bindplane_configuration.md#parameters-block) section. | | ||
|
|
||
| ### Parameters Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `name` | string | required | The name of the parameter. | | ||
| | `value` | any | required | The value of the parameter. | | ||
|
|
||
| ### Route Block | ||
|
|
||
| | Option | Type | Default | Description | | ||
| | ------------------- | ------------ | -------- | ---------------------------- | | ||
| | `route_id` | string | required | A unique string that identifies the route. | | ||
| | `telemetry_type` | enum | `logs+metrics+traces` | The telemetry type to route. | | ||
| | `components` | list(string) | required | One or more components to route the telemetry to. | | ||
|
|
||
| Telemetry types include the following | ||
| - `logs` | ||
| - `metrics` | ||
| - `traces` | ||
| - `logs+metrics` | ||
| - `logs+traces` | ||
| - `metrics+traces` | ||
| - `logs+metrics+traces` (default) | ||
|
|
||
| The following example shows two route blocks, one for all telemetry types and one for traces only. | ||
|
|
||
| ```tf | ||
| # Route all telemetry types to the Datadog destination | ||
| route { | ||
| components = [ | ||
| "destinations/${bindplane_destination.datadog.id}" | ||
| ] | ||
| } | ||
| # Route only traces to the Google destination | ||
| route { | ||
| telemetry_type = "traces" | ||
| components = [ | ||
| "destinations/${bindplane_destination.google.id}" | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ### Supported Platforms | ||
|
|
||
| This table should be used as a reference for supported `platform` values. | ||
|
|
||
| | Platform | Value | | ||
| | ---------------------- | ----------------------- | | ||
| | Linux | `linux` | | ||
| | Windows | `windows` | | ||
| | macOS | `macos` | | ||
| | Kubernetes DaemonSet | `kubernetes-daemonset` | | ||
| | Kubernetes Deployment | `kubernetes-deployment` | | ||
| | OpenShift DaemonSet | `openshift-daemonset` | | ||
| | OpenShift Deployment | `openshift-deployment` | | ||
|
|
||
| ## Examples | ||
|
|
||
| This example shows the creation of a `bindplane_configuration_v2` which uses the following resources: | ||
| - two [bindplane_source](./bindplane_source.md) resources | ||
| - one [bindplane_destination](./bindplane_destination.md) resource | ||
| - two [bindplane_processor](./bindplane_processor.md) resources | ||
|
|
||
| Routes are configured on the sources to send telemetry data to the destination. | ||
|
|
||
| ```hcl | ||
| resource "bindplane_source" "host" { | ||
| rollout = true | ||
| name = "my-host" | ||
| type = "host" | ||
| parameters_json = jsonencode( | ||
| [ | ||
| { | ||
| "name": "collection_interval", | ||
| "value": 30 | ||
| }, | ||
| { | ||
| "name": "enable_process", | ||
| "value": false | ||
| } | ||
| ] | ||
| ) | ||
| } | ||
| resource "bindplane_source" "journald" { | ||
| rollout = true | ||
| name = "my-journald" | ||
| type = "journald" | ||
| } | ||
| resource "bindplane_destination" "google" { | ||
| rollout = true | ||
| name = "my-google" | ||
| type = "googlecloud" | ||
| } | ||
| resource "bindplane_processor" "batch" { | ||
| rollout = true | ||
| name = "my-batch" | ||
| type = "batch" | ||
| } | ||
| resource "bindplane_extension" "pprof" { | ||
| rollout = true | ||
| name = "my-pprof" | ||
| type = "pprof" | ||
| parameters_json = jsonencode( | ||
| [ | ||
| { | ||
| "name": "listen_address", | ||
| "value": "0.0.0.0" | ||
| }, | ||
| { | ||
| "name": "tcp_port", | ||
| "value": 5000, | ||
| } | ||
| ] | ||
| ) | ||
| } | ||
| resource "bindplane_configuration_v2" "configuration" { | ||
| // When removing a component from a configuration and deleting that | ||
| // component during the same apply, we want to update the configuration | ||
| // before the component is deleted. | ||
| lifecycle { | ||
| create_before_destroy = true | ||
| } | ||
| rollout = true | ||
| name = "my-config" | ||
| platform = "linux" | ||
| labels = { | ||
| environment = "production" | ||
| managed-by = "terraform" | ||
| } | ||
| source { | ||
| name = bindplane_source.host.name | ||
| route { | ||
| components = [ | ||
| "destinations/${bindplane_destination.google.id}" | ||
| ] | ||
| } | ||
| } | ||
| source { | ||
| name = bindplane_source.journald.name | ||
| route { | ||
| components = [ | ||
| "destinations/${bindplane_destination.google.id}" | ||
| ] | ||
| } | ||
| } | ||
| destination { | ||
| name = bindplane_destination.google.name | ||
| processors = [ | ||
| bindplane_processor.batch.name | ||
| ] | ||
| } | ||
| extensions = [ | ||
| bindplane_extension.pprof.name | ||
| ] | ||
| rollout_options { | ||
| type = "progressive" | ||
| parameters = [ | ||
| { | ||
| name = "stages" | ||
| value = [ | ||
| { | ||
| labels = { | ||
| env = "stage" | ||
| } | ||
| name = "stage" | ||
| }, | ||
| { | ||
| labels = { | ||
| env = "production" | ||
| } | ||
| name = "production" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ``` |
Oops, something went wrong.