Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal telemetry config options for OTLP export #5986

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions content/en/docs/collector/internal-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,69 @@ Note that the `tracer_provider` section there corresponds to `traces` here.
[kitchen-sink-config]:
https://github.com/open-telemetry/opentelemetry-configuration/blob/main/examples/kitchen-sink.yaml

### Self-monitoring

The Collector can be configured to push its own telemetry to an
[OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver)
and send the data through configured pipelines. In the following example, the
Collector is configured to push metrics, traces, and logs every 10s using OTLP
gRPC to `localhost:14317`:

```yaml
receivers:
otlp/internal:
protocols:
grpc:
endpoint: localhost:14317
exporters:
debug:
service:
pipelines:
metrics:
receivers: [otlp/internal]
exporters: [debug]
traces:
receivers: [otlp/internal]
exporters: [debug]
telemetry:
metrics:
readers:
- periodic:
interval: 10000
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
traces:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
logs:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
```

{{% alert title="Caution" color="warning" %}}

When self-monitoring, the Collector collects its own telemetry and sends it to
the desired backend for analysis. This can be a risky practice. If the Collector
is underperforming, its self-monitoring capability could be impacted. As a
result, the self-monitored telemetry might not reach the backend in time for
critical analysis.

Moreover, sending internal telemetry through the Collector's own pipelines can
create a continuous loop of spans, metric points, or logs, putting undue strain
on the Collector's performance. This setup should not be used in production.

{{% /alert %}}

## Types of internal telemetry

The OpenTelemetry Collector aims to be a model of observable service by clearly
Expand Down