feat(azure-functions): support Azure Functions programming model v4 (#4426)

See https://learn.microsoft.com/en-ca/azure/azure-functions/functions-node-upgrade-v4
This builds on the work in #4178
by @qzxlkj, which provide most of the runtime code fix.

The rest of this is adding testing and updated examples and docs.

Refs: #4178
Closes: #3185

Author: trentm

.github/dependabot.yml

@@ -48,7 +48,15 @@ updates:
         - "eslint*"
 
   - package-ecosystem: "npm"
-    directory: "/test/instrumentation/azure-functions/fixtures/AJsAzureFnApp"
+    directory: "/test/instrumentation/azure-functions/fixtures/azfunc3"
+    schedule:
+      interval: "weekly"
+    open-pull-requests-limit: 5
+    reviewers:
+      - "elastic/apm-agent-node-js"
+
+  - package-ecosystem: "npm"
+    directory: "/test/instrumentation/azure-functions/fixtures/azfunc4"
     schedule:
       interval: "weekly"
     open-pull-requests-limit: 5

CHANGELOG.asciidoc

@@ -41,12 +41,15 @@ See the <<upgrade-to-v4>> guide.
 [float]
 ===== Features
 
+* Support instrumentation of Azure Functions using the https://learn.microsoft.com/en-ca/azure/azure-functions/functions-node-upgrade-v4[v4 Node.js programming model].
+  ({pull}4426[#4426])
+
 [float]
 ===== Bug fixes
 
 * Fix instrumentation of `@aws-sdk/client-s3`, `@aws-sdk/client-sqs`, and
-  `@aws-sdk/client-sns` for versions 3.723.0 and later. (Internally the AWS SDK
-  clients updated to `@smithy/smithy-client@4`.)
+  `@aws-sdk/client-sns` for versions 3.723.0 and later. Internally the AWS SDK
+  clients updated to `@smithy/smithy-client@4`. ({pull}4398[#4398])
 
 [float]
 ===== Chores

docs/azure-functions.asciidoc

@@ -9,7 +9,7 @@ endif::[]
 
 === Monitoring Node.js Azure Functions
 
-The Node.js APM Agent can trace function invocations in an https://learn.microsoft.com/en-us/azure/azure-functions/[Azure Functions] app.
+The Node.js APM Agent can trace function invocations in an https://learn.microsoft.com/en-us/azure/azure-functions/[Azure Functions] app, using either v3 or https://learn.microsoft.com/en-us/azure/azure-functions/functions-node-upgrade-v4[v4 of the Node.js programming model].
 
 
 [float]
@@ -36,7 +36,7 @@ will result in unreasonably large deployments that will be very slow to publish
 and will run your Azure Function app VM out of disk space.
 ====
 
-You can also take a look at and use this https://github.com/elastic/apm-agent-nodejs/tree/main/examples/an-azure-function-app/[Azure Functions example app with Elastic APM already integrated].
+You can also take a look at and use this https://github.com/elastic/apm-agent-nodejs/tree/main/examples/azure-function-app/[Azure Functions example app with Elastic APM already integrated].
 
 [float]
 [[azure-functions-setup]]
@@ -69,17 +69,14 @@ require('elastic-apm-node').start({
 ----
 <1> Optional <<configuration,configuration options>> can be added here.
 
-2. Add a "main" entry to your package.json pointing to the app init file.
+2. Change the "main" entry in your "package.json" to point to the initapm.js file.
 +
 [source,json]
 ----
 ...
-  "main": "initapm.js",
+  "main": "{initapm.js,src/functions/*.js}",
 ...
 ----
-+
-If your application already has a "main" init file, you can instead add the
-`require('elastic-apm-node').start()` to top of that file.
 
 
 [float]
@@ -103,7 +100,7 @@ For example:
 
 image::./images/azure-functions-configuration.png[Configuring the APM Agent in the Azure Portal]
 
-For local testing via `func start` you can set these environment variables in
+For local testing via `func start`, you can set these environment variables in
 your terminal, or in the "local.settings.json" file. See the
 <<configuration,agent configuration guide>> for full details on supported
 configuration variables.
@@ -127,13 +124,11 @@ of time, so allow a minute or so for data to appear.
 [[azure-functions-limitations]]
 ==== Limitations
 
-This instrumentation does not send an APM transaction or error to APM server when
-a handler has an `uncaughtException` or `unhandledRejection`.
-The Azure Functions Node.js reference https://learn.microsoft.com/en-ca/azure/azure-functions/functions-reference-node#use-async-and-await[has a section] with best practices for avoiding these cases.
+Distributed tracing for incoming HTTP requests to Azure Functions (using v4 of the programming model) does *not* work, because of a issue with Azure's handling of trace-context. See https://github.com/elastic/apm-agent-nodejs/pull/4426#issuecomment-2596922653[this] for details.
+
+Azure Functions instrumentation currently does _not_ collect system metrics in the background because of a concern with unintentionally increasing Azure Functions costs (for Consumption plans).
 
-Azure Functions instrumentation currently does _not_ collect system metrics in
-the background because of a concern with unintentionally increasing Azure
-Functions costs (for Consumption plans).
+Elastic APM's <<central-config,central configuration>> is not supported for Azure Functions.
 
 
 [float]

docs/supported-technologies.asciidoc

@@ -63,7 +63,7 @@ These are the frameworks that we officially support:
 |=======================================================================
 | Framework             | Version | Note
 | <<lambda,AWS Lambda>> | N/A |
-| <<azure-functions,Azure Functions>> | ~4 | See https://learn.microsoft.com/en-ca/azure/azure-functions/set-runtime-version[the guide on Azure Functions runtime versions].
+| <<azure-functions,Azure Functions>> | v3, v4 | https://learn.microsoft.com/en-us/azure/azure-functions/functions-node-upgrade-v4[Node.js programming model v3 and v4]
 | <<express,Express>>   | ^4.0.0 |
 | <<fastify,Fastify>>   | >=1.0.0 | See also https://www.fastify.io/docs/latest/Reference/LTS/[Fastify's own LTS documentation]
 | <<hapi,@hapi/hapi>>   | >=17.9.0 <22.0.0 |

eslint.config.js

@@ -30,7 +30,7 @@ module.exports = [
 
       'examples/esbuild/dist/**',
       'examples/typescript/dist/**',
-      'examples/an-azure-function-app/**',
+      'examples/azure-function-app/**',
       'lib/opentelemetry-bridge/opentelemetry-core-mini/**',
       'test/babel/out.js',
       'test/lambda/fixtures/esbuild-bundled-handler/hello.js',

examples/an-azure-function-app/Bye/index.js

@@ -0,0 +1,15 @@
+APP_NAME=$(USER)-example-azure-function-app
+
+.PHONY: print-app-name
+print-app-name:
+	@echo "APP_NAME: $(APP_NAME)"
+
+.PHONY: publish
+publish:
+	func azure functionapp publish "$(APP_NAME)"
+
+# Note that the Azure Functions log stream is extremely flaky. Don't expect it
+# to reliably be able to show logs from the deployed function app.
+.PHONY: logstream
+logstream:
+	func azure functionapp logstream "$(APP_NAME)"

examples/an-azure-function-app/Hi/function.json

@@ -1,10 +1,10 @@
-This directory holds a very small Azure Function App implemented in Node.js
-and setup to be traced by the Elastic APM agent. The App has two "functions":
-
-1. `Hi` - an HTTP-triggered function that will call the `Bye` function, then
-   respond with `{"hi":"there"}`.
-2. `Bye` - an HTTP-triggered function that will respond with `{"good":"bye"}`.
+This directory holds a simple Azure Function (using v4 of the Node.js
+programming model) implemented in Node.js and setup to be traced by the Elastic
+APM agent. The App has a single function:
 
+- `Hello`: an HTTP-triggered function that will call worldtimeapi.org to get
+   the current time in Vancouver and respond with
+   `{"hello": "world", "current time in Vancouver": "..."}`
 
 # Testing locally
 
@@ -13,14 +13,7 @@ and setup to be traced by the Elastic APM agent. The App has two "functions":
 
 2. Install the [Azure Functions Core Tools](https://github.com/Azure/azure-functions-core-tools),
    which provide a `func` CLI tool for running Azure Functions locally for
-   development, and for publishing an Function App to Azure. One way to
-   install is via:
-
-        npm install -g azure-functions-core-tools@4
-
-    It is recommended that you **not** install it in the local `./node_modules`
-    folder, because its large install size will get in the way of publishing to
-    Azure.
+   development, and for publishing an Function App to Azure.
 
 3. Set environment variable to configure the APM agent, for example:
 
@@ -29,12 +22,12 @@ and setup to be traced by the Elastic APM agent. The App has two "functions":
     export ELASTIC_APM_SECRET_TOKEN=...
     ```
 
-4. `npm start`
+4. `npm start` (This calls `func start` to run the Azure Function app locally.)
 
 5. In a separate terminal, call the Azure Function via:
 
     ```
-    curl -i http://localhost:7071/api/Hi
+    curl -i http://localhost:7071/api/Hello
     ```
 
 
@@ -52,7 +45,7 @@ and setup to be traced by the Elastic APM agent. The App has two "functions":
 4. Call your functions:
 
     ```
-    curl -i https://<APP_NAME>.azurewebsites.net/api/hi
+    curl -i https://<APP_NAME>.azurewebsites.net/api/hello
     ```
 
 The result (after a minute for data to propagate) should be a `<APP_NAME>` service

examples/an-azure-function-app/Hi/index.js

@@ -0,0 +1,15 @@
+{
+  "version": "2.0",
+  "logging": {
+    "applicationInsights": {
+      "samplingSettings": {
+        "isEnabled": true,
+        "excludedTypes": "Request"
+      }
+    }
+  },
+  "extensionBundle": {
+    "id": "Microsoft.Azure.Functions.ExtensionBundle",
+    "version": "[4.*, 5.0.0)"
+  }
+}

examples/an-azure-function-app/initapm.js

@@ -0,0 +1 @@
+require('elastic-apm-node').start()

examples/an-azure-function-app/package.json

@@ -5,7 +5,7 @@
     "AzureWebJobsStorage": "",
 
     "REGION_NAME": "test-region-name",
-    "WEBSITE_SITE_NAME": "an-azure-function-app",
+    "WEBSITE_SITE_NAME": "example-azure-function-app",
     "WEBSITE_INSTANCE_ID": "test-website-instance-id"
   }
 }

examples/an-azure-function-app/.gitignore examples/azure-function-app/.gitignore

@@ -0,0 +1,18 @@
+{
+  "name": "example-azure-function-app",
+  "version": "2.0.0",
+  "description": "An example Azure Function app showing Elastic APM integration for tracing/monitoring",
+  "private": true,
+  "main": "{initapm.js,src/functions/*.js}",
+  "engines": {
+    "node": ">=20"
+  },
+  "scripts": {
+    "start": "func start",
+    "dev:sync-local-apm-agent-changes": "rsync -av ../../lib/ ./node_modules/elastic-apm-node/lib/"
+  },
+  "dependencies": {
+    "@azure/functions": "^4.0.0",
+    "elastic-apm-node": "^4.11.0"
+  }
+}

examples/an-azure-function-app/.npmrc examples/azure-function-app/.npmrc

@@ -0,0 +1,24 @@
+const { app } = require('@azure/functions');
+
+app.http('Hello', {
+  methods: ['GET'],
+  authLevel: 'anonymous',
+  handler: async (_request, _context) => {
+    const url = new URL('http://worldtimeapi.org/api/timezone/America/Vancouver');
+    const timeRes = await fetch(url, { signal: AbortSignal.timeout(5000) });
+    const timeBody = await timeRes.json();
+
+    const body = JSON.stringify({
+      hello: 'world',
+      'current time in Vancouver': timeBody.datetime
+    });
+    return {
+      status: 200,
+      headers: {
+        'Content-Type': 'application/json',
+        'Content-Length': Buffer.byteLength(body)
+      },
+      body
+    };
+  },
+});