Skip to content
Open
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
3571707
chore(otel): add support for otel metrics [Part 1]
mabdinur Nov 4, 2025
2d64146
consolidate tests
mabdinur Nov 4, 2025
5ad9754
add support for exporting otlp
mabdinur Nov 4, 2025
9eb0a24
add docs and justify minimum supported versions
mabdinur Nov 5, 2025
d9cd08d
use testagent for otlp tests
mabdinur Nov 5, 2025
eb5c3a2
use default instead of default_proc in otel configs
mabdinur Nov 5, 2025
d6bbff9
consolidate agent resolutation logic
mabdinur Nov 5, 2025
4883393
simplify settings
mabdinur Nov 5, 2025
f1cb22c
clean up testagent helpers
mabdinur Nov 5, 2025
1818637
clean up docs
mabdinur Nov 5, 2025
2e8cdc1
Merge branch 'master' into munir/add-otel-metrics-configs
mabdinur Nov 5, 2025
060757d
clean up gemfile
mabdinur Nov 5, 2025
270a3e9
fix metric export timeout
mabdinur Nov 5, 2025
e138f38
use datadog_env
mabdinur Nov 5, 2025
2eefa81
make http protobuf the default
mabdinur Nov 6, 2025
65bcf2c
use agent resolver
mabdinur Nov 6, 2025
a894798
use opentelemetry configure
mabdinur Nov 6, 2025
c9abc8a
fix spacing
mabdinur Nov 6, 2025
94b46d8
fix spacing
mabdinur Nov 6, 2025
9f3751f
update config tests to validate the exporter that is set
mabdinur Nov 6, 2025
dc158de
track hostname
mabdinur Nov 6, 2025
fdc674b
clean up exporter init
mabdinur Nov 6, 2025
8fea838
break circular import
mabdinur Nov 6, 2025
88c8aee
run metrics tests everywhere
mabdinur Nov 6, 2025
9bcd1a9
Merge branch 'master' into munir/add-otel-metrics-configs
mabdinur Nov 7, 2025
31b7dd4
add version gating for unsupported ruby versions
mabdinur Nov 6, 2025
4c54c4e
clean up docker compose and docs
mabdinur Nov 13, 2025
074a638
update configs
mabdinur Nov 13, 2025
72d5b2f
Merge branch 'master' into munir/add-otel-metrics-configs
mabdinur Nov 15, 2025
9c4a9da
Merge 72d5b2f97cbdc7e5e4a381c2d4789edf6461a237 into 49cee89ecc8776876…
mabdinur Nov 15, 2025
9b876c4
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 15, 2025
4c04601
create an exporter to collect telemetry
mabdinur Nov 15, 2025
531565e
reduce min version 0.4.0
mabdinur Nov 16, 2025
cf17ae2
set default temporality to delta
mabdinur Nov 16, 2025
87ce493
Merge remote-tracking branch 'origin/master' into munir/add-otel-metr…
mabdinur Nov 19, 2025
e7f942d
Merge 87ce493d34fc951232a0124a0f539a439362bb45 into 5853a3ddf4ceb3be8…
mabdinur Nov 19, 2025
e75d535
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 19, 2025
65de9e4
init otel metrics in components.rb
mabdinur Nov 19, 2025
6378761
lint exporter files and tests
mabdinur Nov 19, 2025
8fc1cca
fix exporting logic
mabdinur Nov 19, 2025
530dc78
reset configs between tests
mabdinur Nov 19, 2025
ab9b76e
fix ci
mabdinur Nov 20, 2025
f078a1f
fix ci
mabdinur Nov 20, 2025
2db89f8
fix 3.5 failures
mabdinur Nov 20, 2025
3b1fcca
Merge 2db89f81a4520f4e657e788a7db71276e376c71a into fde6d187f95eb2de7…
mabdinur Nov 20, 2025
f070383
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 20, 2025
9eddf40
lint 4
mabdinur Nov 20, 2025
a8244c0
update appriasal to set min otel common
mabdinur Nov 20, 2025
0a59da7
Merge a8244c0e903a8cc4d3ba1ac1d3923bdb0978b277 into d4c61e8b329e697d6…
mabdinur Nov 20, 2025
6ecf21f
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 20, 2025
ea277d1
address review comments
mabdinur Nov 20, 2025
22028d6
Merge remote-tracking branch 'origin/master' into munir/add-otel-metr…
mabdinur Nov 20, 2025
5216665
simplify logic
mabdinur Nov 20, 2025
f822b5c
remove unnessary casting
mabdinur Nov 20, 2025
2e6e569
Merge f822b5c2068427109b19f30fb20ed42c6a9d4804 into 2c84fc089f429cc1d…
mabdinur Nov 20, 2025
b2b05cf
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 20, 2025
428d1bc
fix test runs in ci
mabdinur Nov 21, 2025
85b5b13
update how configurator is applied
mabdinur Nov 21, 2025
6e44693
lint
mabdinur Nov 21, 2025
bb8502a
ivos comments
mabdinur Nov 21, 2025
0277207
Merge bb8502adac766e9cda35e89479c5835b4ff008b5 into 2c84fc089f429cc1d…
mabdinur Nov 21, 2025
d282e0f
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 21, 2025
ff85daf
fix steep check
mabdinur Nov 21, 2025
3faa913
simplify try_wait_until logic
mabdinur Nov 21, 2025
4aa5529
revert change to ruby 3.2
mabdinur Nov 21, 2025
ca22f30
Update lib/datadog/opentelemetry.rb
mabdinur Nov 21, 2025
09059fd
Merge ca22f30a104db040c764335ccd327a5d5a8d5546 into 2c84fc089f429cc1d…
mabdinur Nov 21, 2025
7384a8b
[🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/r…
dd-apm-ecosystems-autobot[bot] Nov 21, 2025
447d925
fix how otlp headers are parsed
mabdinur Nov 21, 2025
0feb1be
clean up resource attribute creation
mabdinur Nov 21, 2025
fe8d703
address comments round 3
mabdinur Nov 25, 2025
59bb33b
lint
mabdinur Nov 25, 2025
a13a5a7
update docs to include configuration issue
mabdinur Nov 25, 2025
bee73bb
more clean ups, address all open comments
mabdinur Nov 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ DD_AGENT_HOST=testagent
DD_API_KEY=00000000000000000000000000000000
DD_METRIC_AGENT_PORT=8125
DD_TRACE_AGENT_PORT=9126
OTLP_HTTP_PORT=4318
OTLP_GRPC_PORT=4317

# Values are used for proxying from APM Test Agent to real Datadog Agent
DD_REAL_AGENT_HOST=ddagent
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/_unit_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ jobs:
TEST_MYSQL_HOST: mysql
DD_AGENT_HOST: agent
DD_TRACE_AGENT_PORT: '9126'
OTLP_GRPC_PORT: '4317'
OTLP_HTTP_PORT: '4318'
DATADOG_GEM_CI: 'true'
TEST_DATADOG_INTEGRATION: '1'
JRUBY_OPTS: "--dev" # Faster JVM startup: https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag
Expand All @@ -141,6 +143,8 @@ jobs:
LOG_LEVEL: DEBUG
TRACE_LANGUAGE: ruby
PORT: '9126'
OTLP_GRPC_PORT: '4317'
OTLP_HTTP_PORT: '4318'
DD_POOL_TRACE_CHECK_FAILURES: 'true'
DD_DISABLE_ERROR_RESPONSES: 'true'
ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
Expand Down Expand Up @@ -211,6 +215,8 @@ jobs:
LOG_LEVEL: DEBUG
TRACE_LANGUAGE: ruby
PORT: '9126'
OTLP_GRPC_PORT: '4317'
OTLP_HTTP_PORT: '4318'
DD_POOL_TRACE_CHECK_FAILURES: 'true'
DD_DISABLE_ERROR_RESPONSES: 'true'
ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
Expand Down
2 changes: 2 additions & 0 deletions Steepfile
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ target :datadog do
ignore 'lib/datadog/opentelemetry/api/context.rb'
ignore 'lib/datadog/opentelemetry/api/trace/span.rb'
ignore 'lib/datadog/opentelemetry/sdk/configurator.rb'
ignore 'lib/datadog/opentelemetry/sdk/exporter.rb'
ignore 'lib/datadog/opentelemetry/sdk/metrics_exporter.rb'
ignore 'lib/datadog/opentelemetry/sdk/id_generator.rb'
ignore 'lib/datadog/opentelemetry/sdk/propagator.rb'
ignore 'lib/datadog/opentelemetry/sdk/span_processor.rb'
Expand Down
2 changes: 2 additions & 0 deletions appraisal/ruby-3.1.rb
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@

appraise 'opentelemetry' do
gem 'opentelemetry-sdk', '~> 1.1'
gem 'opentelemetry-metrics-sdk', '>= 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '>= 0.4'
end

appraise 'opentelemetry_otlp' do
Expand Down
2 changes: 2 additions & 0 deletions appraisal/ruby-3.2.rb
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,8 @@

appraise 'opentelemetry' do
gem 'opentelemetry-sdk', '~> 1.1'
gem 'opentelemetry-metrics-sdk', '>= 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '>= 0.4'
end

appraise 'opentelemetry_otlp' do
Expand Down
2 changes: 2 additions & 0 deletions appraisal/ruby-3.3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,8 @@

appraise 'opentelemetry' do
gem 'opentelemetry-sdk', '~> 1.1'
gem 'opentelemetry-metrics-sdk', '>= 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '>= 0.4'
end

appraise 'opentelemetry_otlp' do
Expand Down
2 changes: 2 additions & 0 deletions appraisal/ruby-3.4.rb
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,8 @@

appraise 'opentelemetry' do
gem 'opentelemetry-sdk', '~> 1.1'
gem 'opentelemetry-metrics-sdk', '>= 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '>= 0.4'
end

appraise 'opentelemetry_otlp' do
Expand Down
4 changes: 4 additions & 0 deletions appraisal/ruby-3.5.rb
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,10 @@

appraise 'opentelemetry' do
gem 'opentelemetry-sdk', '~> 1.1'
gem 'opentelemetry-metrics-sdk', '>= 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '>= 0.4'
# opentelemetry-metrics-sdk 0.11+ requires opentelemetry-common >= 0.23.0 (for time_in_nanoseconds)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is time_in_nanoseconds needed? I can't find it in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's defined in opentelemetry-common and used in the opentelemetry metrics sdk: https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-ruby%20time_in_nanoseconds&type=code. It's not directly used in our library.

This is just note for us so we do not install incompatible opentelemetry libraries.

gem "opentelemetry-common", ">= 0.23.0"
end

appraise 'opentelemetry_otlp' do
Expand Down
4 changes: 4 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,8 @@ services:
image: ghcr.io/datadog/dd-apm-test-agent/ddapm-test-agent:v1.37.0
ports:
- "127.0.0.1:${DD_TRACE_AGENT_PORT}:9126"
- "127.0.0.1:${OTLP_GRPC_PORT}:4317"
- "127.0.0.1:${OTLP_HTTP_PORT}:4318"
depends_on:
- ddagent
env_file: ./.env
Expand All @@ -271,6 +273,8 @@ services:
- TRACE_LANGUAGE=ruby
- DD_TRACE_AGENT_URL=http://${DD_REAL_AGENT_HOST}:${DD_REAL_AGENT_PORT}
- PORT=${DD_TRACE_AGENT_PORT}
- OTLP_GRPC_PORT=${OTLP_GRPC_PORT}
- OTLP_HTTP_PORT=${OTLP_HTTP_PORT}
- DD_POOL_TRACE_CHECK_FAILURES=true
- DD_DISABLE_ERROR_RESPONSES=true
- ENABLED_CHECKS=trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
Expand Down
57 changes: 54 additions & 3 deletions docs/OpenTelemetry.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# OpenTelemetry

**Supported tracing frameworks**:
**Supported OpenTelemetry features**:

| Type | Documentation | datadog version | Gem version support |
| ------------- | ---------------------------------------------------- | --------------- | ------------------- |
| OpenTelemetry | https://github.com/open-telemetry/opentelemetry-ruby | 1.9.0+ | >= 1.1.0 |
| Tracing | https://github.com/open-telemetry/opentelemetry-ruby | 1.9.0+ | >= 1.1.0 |
| Metrics SDK | https://rubygems.org/gems/opentelemetry-metrics-sdk | 2.25.0+ | >= 0.8 |
| OTLP Metrics Exporter | https://rubygems.org/gems/opentelemetry-exporter-otlp-metrics | 2.25.0+ | >= 0.4 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 2.25.0? Are we not planning to ship this with the next 2.23.0 release, or at least the 2.24.0 after that? 👀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure when this PR would get merged so I picked a release that would likely go out at the end of Q4. Since we are close to getting this merged we can set it to 2.23.0 😄


## Configuring OpenTelemetry
## Configuring OpenTelemetry Tracing

1. Add the `datadog` gem to your Gemfile:

Expand Down Expand Up @@ -44,6 +46,55 @@

[Integration instrumentations](#integration-instrumentation) and OpenTelemetry [Automatic instrumentations](https://opentelemetry.io/docs/instrumentation/ruby/automatic/) are also supported.

## Configuring OpenTelemetry Metrics

1. Add required gems to your Gemfile:

```ruby
gem 'datadog'
gem 'opentelemetry-metrics-sdk', '~> 0.8'
gem 'opentelemetry-exporter-otlp-metrics', '~> 0.4'
```

1. Install gems with `bundle install`

1. Enable metrics export:

```ruby
# Set environment variable before initializing metrics support
ENV['DD_METRICS_OTEL_ENABLED'] = 'true'
require 'opentelemetry/sdk'
require 'opentelemetry-metrics-sdk'
require 'opentelemetry/exporter/otlp_metrics'
require 'datadog/opentelemetry'
Comment on lines 64 to 67
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered turning these 5 lines into a datadog/opentelemetry/metrics_autoload (or boot or some name) or something like that?

That way customers would only need to copy-paste the simpler require :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually only opentelemetry/sdk and datadog/opentelemetry are required. opentelemetry/sdk is required to call OpenTelemetry::SDK.configure and 'datadog/opentelemetry' is required to enable ddtrace-opentelemetry support for both metrics and traces.

We do not need to import 'opentelemetry/exporter/otlp_metrics' and 'opentelemetry/exporter/otlp_metrics' here. This is handled by 'datadog/opentelemetry'. I added these require statements to signal to users that they NEED to install the metrics-sdk and the metrics-exporter. I'll remove these imports and make the description a bit clearer.

# IMPORTANT: Call Datadog.configure before OpenTelemetry::SDK.configure
# and keep both in the same file to ensure proper initialization order.
Datadog.configure do |c|
# Configure Datadog settings here
end
# Initialize OpenTelemetry SDK (required for metrics)
OpenTelemetry::SDK.configure
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Datadog.configure or OpenTelemetry::SDK.configure have to be called before OpenTelemetry metrics start being emitted, or does it just work if I follow these instructions?

    ENV['DD_METRICS_OTEL_ENABLED'] = 'true'
    require 'opentelemetry/sdk'
    require 'opentelemetry-metrics-sdk'
    require 'opentelemetry/exporter/otlp_metrics'
    require 'datadog/opentelemetry'

    require 'opentelemetry/metrics'
    provider = OpenTelemetry.meter_provider
    meter = provider.meter('my-app')
    counter = meter.create_counter('requests')
    counter.add(1)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenTelemetry::SDK.configure is required here. Otherwise we will get the NoopMeterProvider. Good catch. I will fix these docs

1. Use the [OpenTelemetry Metrics API](https://opentelemetry.io/docs/languages/ruby/instrumentation/#metrics) to create and record metrics.

**Note:** Call `Datadog.configure` before `OpenTelemetry::SDK.configure` and keep both in the same file. Configuration changes require calling `OpenTelemetry::SDK.configure` again to take effect.

**Configuration Options:**

- `DD_METRICS_OTEL_ENABLED` - Enable metrics export (default: false)
- `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL` - Protocol: `http/protobuf` (default); `grpc` and `http/json` are not yet supported.
- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` - Custom endpoint (defaults to the Datadog agent otlp endpoint)
- `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` - `delta` (default) or `cumulative`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mabdinur does cumulative work with the Datadog agent?

Copy link
Member

@marcotc marcotc Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mabdinur does cumulative work with the Datadog agent?

Just to confirm, does it? If it doesn't them we have to document it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cumulative is support, it just requires additional processing in the agent/collector. We have this doc that discusses it: https://docs.datadoghq.com/opentelemetry/guide/otlp_delta_temporality/?tab=python.

- `OTEL_METRIC_EXPORT_INTERVAL` - Export interval in milliseconds (default: 10000)

[General OTLP settings](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/) (`OTEL_EXPORTER_OTLP_*`) serve as defaults if metrics-specific settings are not provided.

**Note:** Minimum `opentelemetry-metrics-sdk` is v0.8.0 (contains critical bug fixes). Minimum `opentelemetry-exporter-otlp-metrics` is v0.4.0. Use the latest versions for best support. If you spot any issue with the OpenTelemetry API affecting the `datadog` gem, [please do open a GitHub issue](https://github.com/DataDog/dd-trace-rb/issues).


## Limitations

There are a few limitations to OpenTelemetry Tracing when the APM integration is activated:
Expand Down
2 changes: 2 additions & 0 deletions gemfiles/ruby_3.1_opentelemetry.gemfile

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 22 additions & 3 deletions gemfiles/ruby_3.1_opentelemetry.gemfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions gemfiles/ruby_3.2_opentelemetry.gemfile

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 27 additions & 3 deletions gemfiles/ruby_3.2_opentelemetry.gemfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions gemfiles/ruby_3.3_opentelemetry.gemfile

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 22 additions & 3 deletions gemfiles/ruby_3.3_opentelemetry.gemfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions gemfiles/ruby_3.4_opentelemetry.gemfile

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading