Skip to content

Internal telemetry config options for OTLP export #5986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions content/en/docs/collector/internal-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,69 @@ Note that the `tracer_provider` section there corresponds to `traces` here.
[kitchen-sink-config]:
https://github.com/open-telemetry/opentelemetry-configuration/blob/main/examples/kitchen-sink.yaml

### Self-monitoring

The Collector can be configured to push its own telemetry to an
[OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver)
Comment on lines +184 to +185
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, could we link to docs page, maybe docs/collector/configuration/#receivers. That page links out to the repo's receiver folder.

@theletterf @tiffany76 @open-telemetry/docs-approvers WDYT?

Copy link
Contributor

@tiffany76 tiffany76 Mar 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me, but I think @jade-guiton-dd's comment below is worth addressing first. This PR's content was purposely removed from the page in a previous PR, and there's been no call to add it back.

and send the data through configured pipelines. In the following example, the
Collector is configured to push metrics, traces, and logs every 10s using OTLP
gRPC to `localhost:14317`:

```yaml
receivers:
otlp/internal:
protocols:
grpc:
endpoint: localhost:14317
exporters:
debug:
service:
pipelines:
metrics:
receivers: [otlp/internal]
exporters: [debug]
traces:
receivers: [otlp/internal]
exporters: [debug]
telemetry:
metrics:
readers:
- periodic:
interval: 10000
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
traces:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
logs:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
```

{{% alert title="Caution" color="warning" %}}

When self-monitoring, the Collector collects its own telemetry and sends it to
the desired backend for analysis. This can be a risky practice. If the Collector
is underperforming, its self-monitoring capability could be impacted. As a
result, the self-monitored telemetry might not reach the backend in time for
critical analysis.

Moreover, sending internal telemetry through the Collector's own pipelines can
create a continuous loop of spans, metric points, or logs, putting undue strain
on the Collector's performance. This setup should not be used in production.
Comment on lines +233 to +241
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When self-monitoring, the Collector collects its own telemetry and sends it to
the desired backend for analysis. This can be a risky practice. If the Collector
is underperforming, its self-monitoring capability could be impacted. As a
result, the self-monitored telemetry might not reach the backend in time for
critical analysis.
Moreover, sending internal telemetry through the Collector's own pipelines can
create a continuous loop of spans, metric points, or logs, putting undue strain
on the Collector's performance. This setup should not be used in production.
Self-monitoring risks impacting the Collector's performance: routing internal
telemetry through the Collector's pipelines can create feedback loops, further
stressing the system. Avoid self-monitoring in production.


{{% /alert %}}

## Types of internal telemetry

The OpenTelemetry Collector aims to be a model of observable service by clearly
Expand Down