Skip to content

[processor/transform] Add support for flat configuration style#37444

Merged
evan-bradley merged 6 commits intoopen-telemetry:mainfrom
edmocosta:add-config-flat-statements-support
Feb 3, 2025
Merged

[processor/transform] Add support for flat configuration style#37444
evan-bradley merged 6 commits intoopen-telemetry:mainfrom
edmocosta:add-config-flat-statements-support

Conversation

@edmocosta
Copy link
Copy Markdown
Contributor

Description

This PR is part of #29017, and a split from #36888. It changes the transformprocessor, adding support for flat configuration styles.

Change log:

  • It now supports an additional configuration style, where statements are expressed as a list of strings, being the path's context required, and the context inferred from them thanks to the context inferrer & ottl.ParserCollection. For example:
    log_statements:
     - set(log.body, "bear") where log.attributes["http.path"] == "/animal"
     - set(resource.attributes["name"], "bear")
  • It does support mixed configuration styles.
  • The context's cache values are only shared among flat statements
  • Structured configuration cache values are still isolated, which means that a cache written using a structured configuration style will only be available for that configuration group's statements, and won't be shared with flat statements and/or other structured configuration groups, for example:
    log_statements:
    - set(resource.cache["flat"], "value")
    
    -  statements:
       - set(resource.cache["name"], "bear")
       - set(resource.attributes["name"], resource.cache["name"]) # OK
       - set(resource.attributes["name"], resource.cache["flat"]) # Fail(not set by this group of statements)
    
    - set(resource.attributes["name"], resource.cache["name"]) # Fail(not set by a flat statement)
    - set(resource.attributes["flat"], resource.cache["flat"]) # OK
    
    -  statements:
       - set(resource.attributes["name"], resource.cache["name"]) # Fail(set by another group)

Link to tracking issue

#29017

Testing

Unit tests

Comment thread processor/transformprocessor/internal/common/config.go Outdated
Comment thread processor/transformprocessor/config.go Outdated
Comment thread processor/transformprocessor/config.go
Comment thread processor/transformprocessor/config.go Outdated
Comment thread processor/transformprocessor/internal/common/cache.go Outdated
Comment thread processor/transformprocessor/internal/common/config.go Outdated
@TylerHelmuth
Copy link
Copy Markdown
Member

@evan-bradley please review

Copy link
Copy Markdown
Contributor

@evan-bradley evan-bradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the long delay. The way we're handling the config makes sense to me.

Comment thread processor/transformprocessor/testdata/config.yaml Outdated
Comment thread processor/transformprocessor/internal/logs/processor_test.go
Comment thread processor/transformprocessor/internal/metrics/processor_test.go
Comment thread processor/transformprocessor/internal/traces/processor_test.go
edmocosta and others added 2 commits February 3, 2025 22:42
Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
@evan-bradley evan-bradley merged commit 56866f4 into open-telemetry:main Feb 3, 2025
@github-actions github-actions bot added this to the next release milestone Feb 3, 2025
yiquanzhou added a commit to dash0hq/opentelemetry-collector-contrib that referenced this pull request Feb 5, 2025
* main: (392 commits)
  fix(deps): update module golang.org/x/text to v0.22.0 (open-telemetry#37686)
  [exporter/bmchelix] Second PR of New component: BMC Helix Exporter (open-telemetry#37350)
  chore(deps): update otel/opentelemetry-collector-contrib docker tag to v0.119.0 (open-telemetry#37688)
  [chore] fix codeowners allowlist (open-telemetry#37684)
  chore(deps): update otel/opentelemetry-collector docker tag to v0.119.0 (open-telemetry#37687)
  Update jpkroehling's affiliation (open-telemetry#37683)
  fix(deps): update module github.com/clickhouse/clickhouse-go/v2 to v2.30.3 (open-telemetry#37655)
  fix(deps): update all opentelemetry collector contrib packages to v0.119.0 (open-telemetry#37666)
  fix(deps): update module github.com/elastic/go-docappender/v2 to v2.4.0 (open-telemetry#37667)
  fix(deps): update all golang.org/x packages (open-telemetry#37680)
  [exporter/prometheusremotewrite] Fix WAL deadlock (open-telemetry#37630)
  fix(deps): update opentelemetry-go monorepo (open-telemetry#37673)
  fix(deps): update module github.com/shirou/gopsutil/v4 to v4.25.1 (open-telemetry#37671)
  fix(deps): update module github.com/spf13/pflag to v1.0.6 (open-telemetry#37658)
  fix(deps): update all github.com/aws packages (open-telemetry#37661)
  [chore] Prepare release 0.119.0 (open-telemetry#37660)
  make update-otel OTEL_VERSION=v0.119.0 OTEL_STABLE_VERSION=v1.25.0 (open-telemetry#37656)
  add documentation and warning log for deprecating AccessTokenPassthrough (open-telemetry#37575)
  chore: add myself, echlebek, as a codeowner (open-telemetry#37650)
  [processor/transform] Add support for flat configuration style (open-telemetry#37444)
  ...
yiquanzhou added a commit to dash0hq/opentelemetry-collector-contrib that referenced this pull request Feb 5, 2025
* main: (392 commits)
  fix(deps): update module golang.org/x/text to v0.22.0 (open-telemetry#37686)
  [exporter/bmchelix] Second PR of New component: BMC Helix Exporter (open-telemetry#37350)
  chore(deps): update otel/opentelemetry-collector-contrib docker tag to v0.119.0 (open-telemetry#37688)
  [chore] fix codeowners allowlist (open-telemetry#37684)
  chore(deps): update otel/opentelemetry-collector docker tag to v0.119.0 (open-telemetry#37687)
  Update jpkroehling's affiliation (open-telemetry#37683)
  fix(deps): update module github.com/clickhouse/clickhouse-go/v2 to v2.30.3 (open-telemetry#37655)
  fix(deps): update all opentelemetry collector contrib packages to v0.119.0 (open-telemetry#37666)
  fix(deps): update module github.com/elastic/go-docappender/v2 to v2.4.0 (open-telemetry#37667)
  fix(deps): update all golang.org/x packages (open-telemetry#37680)
  [exporter/prometheusremotewrite] Fix WAL deadlock (open-telemetry#37630)
  fix(deps): update opentelemetry-go monorepo (open-telemetry#37673)
  fix(deps): update module github.com/shirou/gopsutil/v4 to v4.25.1 (open-telemetry#37671)
  fix(deps): update module github.com/spf13/pflag to v1.0.6 (open-telemetry#37658)
  fix(deps): update all github.com/aws packages (open-telemetry#37661)
  [chore] Prepare release 0.119.0 (open-telemetry#37660)
  make update-otel OTEL_VERSION=v0.119.0 OTEL_STABLE_VERSION=v1.25.0 (open-telemetry#37656)
  add documentation and warning log for deprecating AccessTokenPassthrough (open-telemetry#37575)
  chore: add myself, echlebek, as a codeowner (open-telemetry#37650)
  [processor/transform] Add support for flat configuration style (open-telemetry#37444)
  ...
chengchuanpeng pushed a commit to chengchuanpeng/opentelemetry-collector-contrib that referenced this pull request Feb 8, 2025
…telemetry#37444)

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
zeck-ops pushed a commit to zeck-ops/opentelemetry-collector-contrib that referenced this pull request Apr 23, 2025
…telemetry#37444)

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

processor/transform Transform processor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants