feat(configuration_v2): carry inline type/parameters on connector, processor_group and inner processors

[jamesmewton]

Closes #210.

Adds schema and wiring so inline components on bindplane_configuration_v2 round-trip through terraform apply without losing their identity. Covers the three wiring-critical field categories that are currently stripped (connector type+parameters, processor_group parameters, inner processor type+parameters), which in our testing is everything needed to restore SaaS pipeline rendering for the cases we've seen in the wild.

What's in this PR

• internal/configuration/configuration.go

  • Two new fields on ResourceConfig: Type and Parameters, populated onto model.ResourceConfiguration.ParameterizedSpec in withResourcesByName.
  • New ProcessorRef struct and ProcessorRefs []ProcessorRef field on ResourceConfig, appended into the same inner processor list as the existing name-only Processors.

• provider/resource_configuration_v2.go

  • connector block: new type (string, optional) and parameters_json (string, optional, DiffSuppressFunc: suppressEquivalentJSONDiffs).
  • processor_group block: new parameters_json (for fields like telemetry_types).
  • processor_group block: new processor block (singular) for inline inner processors with name (required) + type (optional) + parameters_json (optional). processors = [names] is kept as an additive non-breaking alternative and relaxed from Required to Optional.
  • Schema pattern matches the standalone bindplane_connector resource so the mental model stays consistent across resource types.

• internal/configuration/configuration_test.go

  • TestWithConnectorsByName_InlineTypeAndParameters
  • TestWithProcessorGroups_Parameters
  • TestWithProcessorGroups_InlineProcessorRefs
  • TestWithProcessorGroups_MixedProcessorsAndRefs

• docs/resources/bindplane_configuration_v2.md

  • Adds the missing Connector Block section (it wasn't in the options reference previously).
  • Documents the new processor block nested under processor_group.
  • Adds an "Inline Routing Connector" example that exercises an inline routing:3 connector, an inline batch:3 processor on one processor group, and the name-only form on another.

Schema design notes

A couple of choices worth flagging for review:

1. Inner processors as an additive processor block rather than breaking the processors string list. Breaking the list form would have been simpler in the wiring but would force every existing user to rewrite their TF. The additive block keeps everyone's current config working unchanged. Happy to go the other way if you'd prefer to break it in a major.

2. processors relaxed from Required to Optional. Needed so users can express a processor group using only the processor block form without a stub empty list. Strictly additive — no existing config breaks.

3. Read-path classification uses len(Parameters) > 0, not the presence of Type. When a library-referenced processor or connector hits the server, the server commonly populates Type from the referenced resource. If we reclassified on Type != "", that would flip a processors = ["my-filter"] config into a processor { name = ..., type = ... } block on every refresh and create spurious drift. Using Parameters as the discriminator avoids that: library-referenced components stay in the simpler form; inline ones (which always have Parameters in practice for routing/batch/etc.) round-trip through the rich form. Same logic applied to connectors for symmetry. Happy to discuss state-aware Read logic if you'd prefer a more principled approach.

Known limitations

Calling these out explicitly rather than hiding them:

• Inline components with type but no parameters_json will be reclassified as library-referenced on Read and the type field cleared from state, causing drift. In practice this affects anyone who declares type = "batch:3" with no parameters (relying on defaults). Workaround: declare at least one parameter, or use the processors = [names] list form. The drift-avoidance trade-off described in design note 3 is why. Open to a state-aware Read if you'd rather avoid this.
• parameters_json drift on "" vs "[]". suppressEquivalentJSONDiffs treats only-one-empty as different, so users who declare parameters_json = jsonencode([]) (an explicit empty parameters list) will see drift since the Read path returns "" for empty parameters. Users who omit the attribute entirely are unaffected. Minor — happy to extend the diff-suppress if you want it bundled in here.
• Inner processor displayName is not covered. The related restoration we had to run as a workaround also re-patched displayName on inner processors; this PR does not carry it through. The effect is cosmetic only — the SaaS canvas falls back to the internal name. PR feat: Support resource display name and description metadata fields #115 adds display_name on standalone resources; I didn't want this PR to collide with that surface, so display_name on inline inner processors is deliberately a separate follow-up.

What's explicitly out of scope

• display_name support on any resource. PR feat: Support resource display name and description metadata fields #115 is in flight on the standalone-resource surface; inner-processor display_name can pick up from there.
• processor block support on source and destination. The defect we've been chasing is scoped to processor_group inner processors, so I scoped the same way. Happy to extend if you'd like symmetry.

Testing

• Unit tests added in internal/configuration for the new plumbing, following the style of TestReadRolloutOptions / the existing table-driven TestNewV2 tests.
• gofmt clean on the four files touched.
• I couldn't run go build / go test locally: github.com/observiq/bindplane-op-enterprise is private and I don't have SSH access. CI has the secret and will validate. If there's a documented path for external contributors to build this locally, happy to follow it; otherwise I'll iterate on CI output and review comments.
• Spec shape validated against BindPlane OP v1.99.1 on localhost. Posted a Configuration whose spec matches what the patched provider would emit (inline connector with type=routing:2 + two parameters; processor group with telemetry_types parameter + an inner processor with type=batch:2 + two parameters), GETed it back, and verified every field round-trips. 0 failures across connector.type, connector.parameters, processor_group.parameters, inner_processor.type, inner_processor.parameters. This confirms BindPlane accepts and preserves the shape; it does not reproduce the SaaS-specific render bug (localhost OP renders specs more leniently than SaaS), which is a distinct layer.
• End-to-end fix behaviour (provider no longer strips → SaaS renders correctly) is gated on CI validation of the Go code and whatever acceptance tests you have around bindplane_configuration_v2. I don't have a way to run those locally.

Evidence

We hit this on a couple of production Bindplane Cloud configs with inline routing:3 connectors and a few inline batch:3 processors embedded on processor groups (~50 processor groups across 30+ routes on the largest). After terraform apply, the SaaS UI rendered the downstream pipeline as disconnected processor groups with no incoming edges. Library-referenced components on the same config deployed fine.

We confirmed the root cause by running a null_resource post-apply local-exec that re-PATCHes the stripped fields via POST /v1/apply: with connector type/parameters, processor group telemetry_types, and inner processor type/parameters restored, the SaaS UI resolves everything correctly and the pipeline renders 1:1 with the original UI config. HAR captures of the SaaS graph before and after the workaround made the causal chain unambiguous. The example in the docs is a synthetic minimal repro; happy to share more specific patterns privately if it would help review.

Compatibility

• All new schema attributes are Optional and all changes are additive. Existing bindplane_configuration_v2 configs that don't set the new fields are unaffected.
• processors on processor_group is relaxed from Required to Optional. Existing configs that set processors = [...] continue to work; new configs can omit it when using processor blocks instead.
• Read path is conservative for library-referenced components: only emits rich-block form when Parameters are non-empty, so upgrading the provider doesn't introduce drift on existing state for configs that don't touch the new fields.
• suppressEquivalentJSONDiffs prevents spurious drift on JSON re-serialisation, matching the standalone resource.

Happy to iterate on naming, descriptions, schema shape, or anything else on review.

[jsirianni]

Hi @jamesmewton. I appreciate the pull request. Would you be able to break it up into several smaller, focused changes?

We have been super busy lately, it will be easier to review smaller changes. Overall, I think this looks good, and I see no reason to reject them.

[jmewton2-SecOps]

Hi @jsirianni

No problems will split this up into smaller changes and advise