Merge branch 'main' into patch-1

content/en/blog/2025/observing-lambdas/index.md

@@ -0,0 +1,124 @@
+---
+title: Observing Lambdas using the OpenTelemetry Collector Extension Layer
+author: '[Dominik Süß](https://github.com/theSuess) (Grafana)'
+linkTitle: Observing Lambdas
+date: 2025-02-05
+sig: FaaS
+issue: 5961
+cSpell:ignore: Dominik
+---
+
+Getting telemetry data out of modern applications is very straightforward (or at
+least it should be). You set up a collector which either receives data from your
+application or asks it to provide an up-to-date state of various counters. This
+happens every minute or so, and if it’s a second late or early, no one really
+bats an eye. But what if the application isn’t around for long? What if every
+second waiting for the data to be collected is billed? Then you’re most likely
+thinking of Function-as-a-Service (FaaS) environments, the most well-known being
+AWS Lambda.
+
+In this execution model, functions are called directly, and the environment is
+frozen afterward. You’re only billed for actual execution time and no longer
+need a server to wait for incoming requests. This is also where the term
+serverless comes from. Keeping the function alive until metrics can be collected
+isn’t really an option and even if you were willing to pay for that, different
+invocations will have a completely separate context and not necessarily know
+about all the other executions happening simultaneously. You might now be
+saying: "I'll just push all the data at the end of my execution, no issues
+here!", but that doesn’t solve the issue. You’ll still have to pay for the time
+it takes to send the data and with many invocations, this adds up.
+
+But there is another way! Lambda extension layers allow you to run any process
+alongside your code, sharing the execution runtime and providing additional
+services. With the
+[opentelemetry-lambda](https://github.com/open-telemetry/opentelemetry-lambda/blob/main/collector/README.md)
+extension layer, you get a local endpoint to send data to while it keeps track
+of the Lambda lifecycle and ensures your telemetry gets to the storage layer.
+
+## How does it work?
+
+When your function is called for the first time, the extension layer starts an
+instance of the OpenTelemetry Collector. The Collector build is a stripped down
+version, providing only components necessary in the context of Lambda. It
+registers with the Lambda
+[Extensions API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-extensions-api.html)
+and
+[Telemetry API](https://docs.aws.amazon.com/lambda/latest/dg/telemetry-api.html).
+By doing this, it receives notifications whenever your function is executed,
+emits a logline, or the execution context is about to be shut down.
+
+### This is where the magic happens
+
+Up until now, this just seems like extra work for nothing. You'll still have to
+wait for the Collector to export the data, right? This is where the special
+`decouple` processor comes in. It separates the receiving and exporting
+components while interfacing with the Lambda lifecycle. This allows for the
+Lambda to return, even if not all data has been sent. At the next invocation (or
+on shutdown) the Collector continues exporting the data while your function does
+its thing.
+
+{{< figure src="diagram-execution-timing.svg" caption="Diagram showcasing how execution timing differs with and without a Collector">}}
+
+## How can I use it?
+
+As of November 2024, the opentelemetry-lambda project publishes
+[releases of the Collector extension layer](https://github.com/open-telemetry/opentelemetry-lambda/releases/tag/layer-collector%2F0.12.0).
+It can be configured through a configuration file hosted either in an S3 bucket
+or on an arbitrary HTTP server. It is also possible to bundle the configuration
+file with your Lambda code. In both cases, you have tradeoffs to consider.
+Remote configuration files add to the cold start duration as an additional
+request needs to be made, while bundling the configuration increases the
+management overhead when trying to control the configuration for multiple
+Lambdas.
+
+The simplest way to get started is with an embedded configuration. For this, add
+a file called `collector.yaml` to your function. This is a regular Collector
+configuration file. To take advantage of the Lambda specific extensions, they
+need to be configured. As an example, the configuration shown next receives
+traces and logs from the Telemetry API and sends them to another endpoint.
+
+```yaml
+receivers:
+  telemetryapi:
+exporters:
+  otlphttp/external:
+    endpoint: 'external-collector:4318'
+processors:
+  batch:
+  decouple:
+service:
+  pipelines:
+    traces:
+      receivers: [telemetryapi]
+      processors: [batch, decouple]
+      exporters: [otlphttp/external]
+    logs:
+      receivers: [telemetryapi]
+      processors: [batch, decouple]
+      exporters: [otlphttp/external]
+```
+
+The `decouple` processor is configured by default if omitted. It is explicitly
+added in this example to illustrate the entire pipeline. For more information,
+see
+[Autoconfiguration](https://github.com/open-telemetry/opentelemetry-lambda/tree/main/collector#auto-configuration).
+
+Afterward, set the `OPENTELEMETRY_COLLECTOR_CONFIG_URI` environment variable to
+`/var/task/collector.yaml`. Once the function is redeployed, you’ll see your
+function logs appear! You can see this in action in the video below.
+
+<p>
+  <video controls style="width: 100%">
+    <source src="./video-lambda-real-time.webm" />
+  </video>
+</p>
+
+Every log line your Lambda produces will be sent to the `external-collector`
+endpoint specified. You don't need to modify the code at all! From there,
+telemetry data flows to your backend as usual. Since the transmission of
+telemetry data might be frozen when the lambda is not active, logs can arrive
+delayed. They'll either arrive during the next execution or during the shutdown
+interval.
+
+If you want further insight into your applications, also see the
+[language specific auto instrumentation layers](https://github.com/open-telemetry/opentelemetry-lambda/?tab=readme-ov-file#extension-layer-language-support).

content/en/docs/collector/internal-telemetry.md

@@ -10,6 +10,18 @@ configure it to help you
 [monitor](#use-internal-telemetry-to-monitor-the-collector) and
 [troubleshoot](/docs/collector/troubleshooting/) the Collector.
 
+{{% alert title="Important" color="warning" %}} The Collector uses the
+OpenTelemetry SDK
+[declarative configuration schema](https://github.com/open-telemetry/opentelemetry-configuration)
+for configuring how to export its internal telemetry. This schema is still under
+[development](/docs/specs/otel/document-status/#lifecycle-status) and may
+undergo **breaking changes** in future releases. We intend to keep supporting
+older schemas until a 1.0 schema release is available, and offer a transition
+period for users to update their configurations before dropping pre-1.0 schemas.
+For details and to track progress see
+[issue #10808](https://github.com/open-telemetry/opentelemetry-collector/issues/10808).
+{{% /alert %}}
+
 ## Activate internal telemetry in the Collector
 
 By default, the Collector exposes its own telemetry in two ways:

data/ecosystem/distributions.yaml

@@ -106,12 +106,12 @@
 - name: Embrace Android Distribution for OpenTelemetry
   url: https://github.com/embrace-io/embrace-android-sdk
   docsUrl: https://embrace.io/docs/open-telemetry/integration/?android-language=java
-  components: [Android]
-- name: Embrace Android Distribution for OpenTelemetry
-  url: https://github.com/embrace-io/embrace-android-sdk
-  docsUrl: https://embrace.io/docs/open-telemetry/integration/?android-language=java
-  components: [Java]
+  components: [Android, Java]
 - name: Embrace Apple Distribution for OpenTelemetry
   url: https://github.com/embrace-io/embrace-apple-sdk
   docsUrl: https://embrace.io/docs/open-telemetry/integration/#apple
   components: [Swift]
+- name: Embrace React Native Distribution for OpenTelemetry
+  url: https://github.com/embrace-io/embrace-react-native-sdk
+  docsUrl: https://embrace.io/docs/open-telemetry/integration/#react-native
+  components: [React Native, Javascript]

data/registry/collector-exporter-alertmanager.yml

@@ -17,7 +17,7 @@ authors:
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/alertmanagerexporter
-  version: v0.118.0
+  version: v0.119.0
 urls:
   repo: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/alertmanagerexporter
 createdAt: 2023-12-05

data/registry/collector-exporter-alibaba-cloud-log-service.yml

@@ -17,4 +17,4 @@ createdAt: 2021-02-24
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/alibabacloudlogserviceexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-aws-xray.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awsxrayexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-awscloudwatchlogs.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awscloudwatchlogsexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-awsemf.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awsemfexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-awss3.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awss3exporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-azure-monitor.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-azuredataexplorer.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuredataexplorerexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-carbon.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/carbonexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-cassandra.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/cassandraexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-clickhouse.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-coralogix.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/coralogixexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-datadog.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/datadogexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-dataset.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/datasetexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-doris.yml

@@ -18,4 +18,4 @@ createdAt: 2024-11-18
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/dorisexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-elasticsearch.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-file.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/fileexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-googlecloud.yml

@@ -18,4 +18,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlecloudexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-googlecloudpubsub.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlecloudpubsubexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-googlemanagedprometheus.yml

@@ -19,4 +19,4 @@ createdAt: 2022-10-27
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlemanagedprometheusexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-honeycombmarker.yml

@@ -18,4 +18,4 @@ createdAt: 2023-10-17
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-influxdb.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/influxdbexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-kafka.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-kinetica.yml

@@ -18,4 +18,4 @@ createdAt: 2023-09-19
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kineticaexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-load-balancing.yml

@@ -16,4 +16,4 @@ createdAt: 2020-10-22
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-logicmonitor.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/logicmonitorexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-logzio.yml

@@ -16,4 +16,4 @@ createdAt: 2020-10-22
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/logzioexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-loki.yml

@@ -16,4 +16,4 @@ createdAt: 2020-10-22
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/lokiexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-mezmo.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/mezmoexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-opencensus.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/opencensusexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-opensearch.yml

@@ -18,4 +18,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/opensearchexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-otelarrow.yml

@@ -21,4 +21,4 @@ createdAt: 2024-02-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/otelarrowexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-prometheus-remote-write.yml

@@ -17,4 +17,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-prometheus.yml

@@ -16,4 +16,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-pulsar.yml

@@ -18,4 +18,4 @@ createdAt: 2022-10-27
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/pulsarexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-rabbitmq.yml

@@ -17,4 +17,4 @@ createdAt: 2024-04-18
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/rabbitmqexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-sapm.yml

@@ -18,4 +18,4 @@ createdAt: 2020-11-05
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/sapmexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-sentry.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/sentryexporter
-  version: v0.118.0
+  version: v0.119.0

data/registry/collector-exporter-signalfx.yml

@@ -16,4 +16,4 @@ createdAt: 2020-06-06
 package:
   registry: go-collector
   name: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter
-  version: v0.118.0
+  version: v0.119.0