Skip to content

Commit b3223b8

Browse files
theletterfcarsonip
theletterf
and
carsonip
authored
Add ECS conversion tables (#1186)
Fixes elastic/apm-server#15509 Take the content in https://github.com/elastic/observability-docs/issues/4843, converts it to Markdown and apply edits for style and structure. +CC @harshithsaiv --------- Co-authored-by: Carson Ip <[email protected]>
1 parent f6d9994 commit b3223b8

File tree

6 files changed

+136
-54
lines changed

6 files changed

+136
-54
lines changed

‎redirects.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,8 @@ redirects:
3333
'solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md': 'solutions/observability/apm/upstream-opentelemetry-collectors-language-sdks.md'
3434
'solutions/observability/apps/collect-metrics.md': 'solutions/observability/apm/collect-metrics.md'
3535
'solutions/observability/apps/limitations.md': 'solutions/observability/apm/limitations.md'
36-
'solutions/observability/apps/resource-atrributes.md': 'solutions/observability/apm/resource-attributes.md'
36+
'solutions/observability/apps/resource-atrributes.md': 'solutions/observability/apm/attributes.md'
37+
'solutions/observability/apps/resource-attributes.md': 'solutions/observability/apm/attributes.md'
3738
'solutions/observability/apps/apm-k8s-attacher.md': 'solutions/observability/apm/apm-k8s-attacher.md'
3839
'solutions/observability/apps/monitoring-aws-lambda-functions.md': 'solutions/observability/apm/monitor-aws-lambda-functions.md'
3940
'solutions/observability/apps/integrate-with-jaeger-deprecated.md': 'solutions/observability/apm/jaeger.md'

‎solutions/observability/apm/attributes.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
mapped_pages:
3+
- https://www.elastic.co/guide/en/observability/current/apm-open-telemetry-resource-attributes.html
4+
- https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry-resource-attributes.html
5+
applies_to:
6+
stack:
7+
serverless:
8+
---
9+
10+
# Attributes and labels [apm-open-telemetry-resource-attributes]
11+
12+
In OpenTelemetry, an attribute is a key-value pair. Attributes are similar to [labels](/solutions/observability/apm/metadata.md#apm-data-model-labels) in that they add metadata to transactions, spans, and other entities.
13+
14+
Resource attributes are a type of attribute that contains information about the entities that produce telemetry. Resource attributes map to Elastic Common Schema (ECS) fields like `service.*`, `cloud.*`, `process.*`, and so on. These fields describe the service and its environment.
15+
16+
For example, Elastic APM maps the OpenTelemetry `deployment.environment` field to the ECS `service.environment` field on ingestion.
17+
18+
## Setting resource attributes
19+
20+
You can set resource attributes through environment variables or by editing the configuration of the resource processor of the OpenTelemetry Collector.
21+
22+
### OpenTelemetry agent
23+
24+
Use the `OTEL_RESOURCE_ATTRIBUTES` environment variable to pass resource attributes at process invocation. For example:
25+
26+
```bash
27+
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
28+
```
29+
30+
### OpenTelemetry Collector
31+
32+
Use the [resource processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourceprocessor) to set or apply changes to resource attributes when using the OTel Collector.
33+
34+
```yaml
35+
...
36+
processors:
37+
resource:
38+
attributes:
39+
- key: deployment.environment
40+
action: insert
41+
value: production
42+
...
43+
```
44+
45+
## Handling of unmapped attributes
46+
47+
When sending telemetry to Elastic APM, only a subset of OpenTelemetry attributes are directly mapped to Elastic APM document fields, such as ECS fields. If an attribute doesn't have a predefined mapping, the system stores it under `labels.*`, with dots replaced by underscores.
48+
49+
Unmapped resource attributes are treated as global labels in Elastic APM, meaning they apply to all telemetry data from the resource. In contrast, unmapped record-level attributes, such as those specific to a log record, span, or data point, are stored as normal labels associated only with that specific record.
50+
51+
For example, if an OpenTelemetry resource contains:
52+
53+
```json
54+
{
55+
"service.name": "user-service",
56+
"deployment.environment": "production",
57+
"otel.library.name": "my-lib",
58+
"custom.attribute.with.dots": "value"
59+
}
60+
```
61+
62+
Elastic APM stores the following:
63+
64+
```json
65+
{
66+
"service.name": "user-service",
67+
"service.environment": "production",
68+
"labels": {
69+
"otel_library_name": "my-lib",
70+
"custom_attribute_with_dots": "value"
71+
}
72+
}
73+
```
74+
75+
## Scope attributes translation
76+
77+
Scope attributes are translated as follows:
78+
79+
| OpenTelemetry attribute | Elastic APM field |
80+
|-------------------------|-------------------|
81+
| scope.name | service.framework.name |
82+
| scope.version | service.framework.version |
83+
84+
Unmapped scope attributes are ignored.

‎solutions/observability/apm/resource-attributes.md

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎solutions/observability/apm/spans.md

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -475,4 +475,49 @@ Support for span compression is available in the following agents and can be con
475475
| **Java agent** | [`span_compression_same_kind_max_duration`](apm-agent-java://reference/config-huge-traces.md#config-span-compression-same-kind-max-duration) | [`span_compression_exact_match_max_duration`](apm-agent-java://reference/config-huge-traces.md#config-span-compression-exact-match-max-duration) |
476476
| **.NET agent** | [`SpanCompressionSameKindMaxDuration`](apm-agent-dotnet://reference/config-core.md#config-span-compression-exact-match-max-duration) |
477477
| **Node.js agent** | [`spanCompressionSameKindMaxDuration`](apm-agent-nodejs://reference/configuration.md#span-compression-exact-match-max-duration) |
478-
| **Python agent** | [`span_compression_same_kind_max_duration`](apm-agent-python://reference/configuration.md#config-span-compression-exact-match-max_duration) |
478+
| **Python agent** | [`span_compression_same_kind_max_duration`](apm-agent-python://reference/configuration.md#config-span-compression-exact-match-max_duration) |
479+
480+
## OpenTelemetry and Elastic APM spans
481+
482+
OpenTelemetry spans are mapped to Elastic APM transactions and spans as follows:
483+
484+
- Root spans, such as entry points, are mapped to APM transactions.
485+
- Child spans, such as internal operations and DB queries, are mapped to APM spans.
486+
487+
The following table summarizes the mapping between OpenTelemetry span kinds and Elastic APM entities.
488+
489+
| OpenTelemetry span kind | Mapped to APM | Example |
490+
|-------------------------|---------------|---------|
491+
| `SERVER` | Transaction | Incoming HTTP request (`GET /users/{id}`) |
492+
| `CONSUMER` | Transaction | Message queue consumer event |
493+
| `CLIENT` | Span | Outgoing database query (`SELECT * FROM users`) |
494+
| `PRODUCER` | Span | Sending a message to a queue |
495+
| `INTERNAL` | Span | Internal function execution |
496+
497+
The following example shows OpenTelemetry spans:
498+
499+
```json
500+
[
501+
{
502+
"traceId": "abcd1234",
503+
"spanId": "root5678",
504+
"parentId": null,
505+
"name": "GET /users/{id}",
506+
"kind": "SERVER"
507+
},
508+
{
509+
"traceId": "abcd1234",
510+
"spanId": "db1234",
511+
"parentId": "root5678",
512+
"name": "SELECT FROM users",
513+
"kind": "CLIENT"
514+
}
515+
]
516+
```
517+
518+
The previous OTel spans are stored by Elastic APM as follows:
519+
520+
```
521+
Transaction: GET /users/{id}
522+
├── Span: SELECT FROM users
523+
```

‎solutions/observability/apm/upstream-opentelemetry-collectors-language-sdks.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \
175175
1. [preview] The OpenTelemetry logs intake via APM Server is currently in technical preview.
176176

177177
`OTEL_RESOURCE_ATTRIBUTES`
178-
: Fields that describe the service and the environment that the service runs in. See [resource attributes](/solutions/observability/apm/resource-attributes.md) for more information.
178+
: Fields that describe the service and the environment that the service runs in. See [attributes](/solutions/observability/apm/attributes.md) for more information.
179179

180180
`OTEL_EXPORTER_OTLP_ENDPOINT`
181181
: APM Server URL. The host and port that APM Server listens for events on.
@@ -216,7 +216,7 @@ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \
216216
1. [preview] The OpenTelemetry logs intake via Elastic is currently in technical preview.
217217

218218
`OTEL_RESOURCE_ATTRIBUTES`
219-
: Fields that describe the service and the environment that the service runs in. See [resource attributes](/solutions/observability/apm/resource-attributes.md) for more information.
219+
: Fields that describe the service and the environment that the service runs in. See [attributes](/solutions/observability/apm/attributes.md) for more information.
220220

221221
`OTEL_EXPORTER_OTLP_ENDPOINT`
222222
: Elastic URL. The host and port that Elastic listens for APM events on.
@@ -265,5 +265,5 @@ For more information on how APM Server services gRPC requests, see [Muxing gRPC
265265
## Next steps [apm-open-telemetry-direct-next]
266266

267267
* [Collect metrics](/solutions/observability/apm/collect-metrics.md)
268-
* Add [Resource attributes](/solutions/observability/apm/resource-attributes.md)
268+
* Add [resource attributes](/solutions/observability/apm/attributes.md)
269269
* Learn about the [limitations of this integration](/solutions/observability/apm/limitations.md)

‎solutions/toc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ toc:
131131
- file: observability/apm/upstream-opentelemetry-collectors-language-sdks.md
132132
- file: observability/apm/collect-metrics.md
133133
- file: observability/apm/limitations.md
134-
- file: observability/apm/resource-attributes.md
134+
- file: observability/apm/attributes.md
135135
- file: observability/apm/apm-k8s-attacher.md
136136
- file: observability/apm/monitor-aws-lambda-functions.md
137137
- file: observability/apm/jaeger.md

0 commit comments

Comments
 (0)