Skip to content

Expose type/parameters on inline connector and processor_group blocks in bindplane_configuration_v2 #210

@jamesmewton

Description

@jamesmewton

Is your feature request related to a problem? Please describe.

We've been moving a couple of production Bindplane configs from the UI over to Terraform and hit a wall that took a bit to track down: inline routing:3 connectors and processor group telemetry_types get dropped by the provider on terraform apply, so configs that render fine in the UI come out broken in SaaS. Figured I'd open this up since I think the fix is pretty contained and the workaround isn't pleasant.

The short version: the inline connector block in bindplane_configuration_v2 only accepts route_id, name, and route. There's no way to set type or parameters on it. For a library-referenced connector (name points at a separately defined bindplane_connector resource), that's fine. For an inline routing connector that carries its own OTTL conditions in the config spec, there's nowhere to put them, and the SaaS UI then can't resolve the connector and the downstream pipeline renders as a pile of disconnected processor group nodes with no incoming edges. Same story for processor group telemetry_types (e.g. ["Logs"]): no schema field, the provider doesn't send it, processor groups end up under the wrong telemetry section.

I poked around the code a bit to confirm. In internal/configuration/configuration.go, ResourceConfig (L27-40) only has Name, Processors, RouteID, Routes:

type ResourceConfig struct {
    Name       string
    Processors []string
    RouteID    string
    Routes     *model.Routes
}

And withResourcesByName (L227-258) builds model.ResourceConfiguration from that, so there's no path for Type or Parameters to flow through even if the schema did expose them. The model.ResourceConfiguration on the API side already supports all of this (Type, Parameters, etc.) — the UI-created configs carry it fine, it's just that the provider doesn't.

For context on the real-world impact: we had three inline routing:3 connectors handling the main fan-out on one config (~50 processor groups across 30+ routes). After terraform apply, all three had empty type and parameters in the deployed config spec. The other three connectors on the same config were library-referenced (bindplane_connector resource) and those deployed fine, which is how we figured out it was specifically the inline path. Ended up running a null_resource post-apply to PATCH the fields back via /v1/apply so we could ship, but obviously that's not a real answer.

Describe the solution you'd like

I think this splits pretty cleanly. The contained bit is to add the fields to ResourceConfig and the two inline block schemas so they can round-trip:

  1. ResourceConfig (L27-40): add Type string and Parameters []model.Parameter (matching what model.ResourceConfiguration already takes).
  2. withResourcesByName (L227-258): set them on the outer resource (line ~244).
  3. connector schema in provider/resource_configuration_v2.go (L105-127): add type (string, Optional) and parameters_json (string, Optional, with DiffSuppressFunc: suppressEquivalentJSONDiffs). Same shape you already use on the standalone bindplane_connector resource (L46-58 of resource_connector.go), so schema-wise it's the same mental model, just on the inline block.
  4. processor_group schema (L128-151): add parameters_json (same pattern). Mostly so telemetry_types can get through.
  5. Create path: read the new fields in the connector and processor_group loops (around L336-389) and populate ResourceConfig.Type / ResourceConfig.Parameters.
  6. Read path: in the connector and processor_group sections (L544-596), serialise c.Type / pg.Parameters back into state using the same JSON round-trip pattern the other resources use.

The readRolloutOptions / withRolloutOptions pair in resource_configuration_generic.go is basically a reference implementation of this — type + parameters already flow through there, so we're just extending the same shape to two more blocks.

The part I want to sanity check with you

Inner processors inside a processor_group lose their type, parameters, and displayName too. Same root cause — withResourcesByName builds each inner processor as model.ResourceConfiguration{Name: name} at L234, only the name is set. Fixing that properly would mean changing processor_group.processors from list(string) to a structured block with name + type + parameters_json. That's a breaking schema change for existing users, which is why I didn't want to just stuff it into this issue.

Options I can see:

  • Ship the connector + processor_group parameters_json fix now (non-breaking), open a separate discussion for the inner-processor question.
  • Add a parallel processor block alongside the existing processors = [names] and gradually deprecate the string list.
  • Break it in a major version.

Which of those lines up best with how you'd want to tackle it? Happy to go with your call and scope this issue / PR accordingly.

Describe alternatives you've considered

  • Post-apply null_resource with local-exec PATCHing the fields back — what we're actually running right now. Works, keeps the pipeline wired, but obviously fragile and not something we want as a permanent part of the config.
  • Fork the provider — doable but not great for upgrades.
  • Use the advanced block — only maps to agent-level telemetry settings, not per-component parameters.

Additional context

  • Tested against BindPlane SaaS (Google Edition) and localhost BindPlane OSS; the same generated TF renders correctly on localhost OSS (it POSTs the spec verbatim via API) but is stripped by the provider on apply, which is what makes it SaaS/Terraform-specific.
  • Checked PR feat: Support resource display name and description metadata fields #115 (display_name / description support) before opening this — that one's scoped to component resources and processor_bundle, so the changes here shouldn't collide with it. Happy to rebase around it if you'd rather land that first.
  • Provider v1.8.0, BindPlane v1.99.

Happy to put up a draft PR against this once you've had a look, if that's useful.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions