From 63818f3ed6942ce88e79c026af4c8e41f1dcc7d0 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Wed, 3 Jun 2026 15:17:48 -0500 Subject: [PATCH 1/4] Initial checkin of new docs --- .../configuration-reference.md | 4 +- .../yaml-schema/tracing-configuration.md | 5 + docs/observability/tracing/content-capture.md | 150 ++++++++++++++++++ docs/observability/tracing/index.md | 4 +- 4 files changed, 161 insertions(+), 2 deletions(-) create mode 100644 docs/observability/tracing/content-capture.md diff --git a/docs/configure-rails/configuration-reference.md b/docs/configure-rails/configuration-reference.md index fa42058bbc..d3e9492436 100644 --- a/docs/configure-rails/configuration-reference.md +++ b/docs/configure-rails/configuration-reference.md @@ -761,9 +761,11 @@ tracing: adapters: - name: FileSystem span_format: opentelemetry - enable_content_capture: false + enable_content_capture: false # records prompts/responses on spans; may include PII ``` +For the environment-variable overrides, output formats, and privacy guidance that apply to `enable_content_capture`, see [](../observability/tracing/content-capture.md). + ### Streaming ```{deprecated} v0.20.0 diff --git a/docs/configure-rails/yaml-schema/tracing-configuration.md b/docs/configure-rails/yaml-schema/tracing-configuration.md index ffd1ad7718..3ae593f905 100644 --- a/docs/configure-rails/yaml-schema/tracing-configuration.md +++ b/docs/configure-rails/yaml-schema/tracing-configuration.md @@ -48,6 +48,8 @@ tracing: |--------|-------------|---------| | `enabled` | Enable or disable tracing | `false` | | `adapters` | List of tracing adapters | `[]` | +| `span_format` | Span structure used by the LLMRails adapter: `opentelemetry` or `legacy` | `opentelemetry` | +| `enable_content_capture` | Record prompts, responses, and rail inputs on spans. Captures potentially sensitive data (PII); see [](../../observability/tracing/content-capture.md) | `false` | ## Tracing Adapters @@ -121,6 +123,9 @@ Traces capture the following information: | **Errors** | Error conditions and debugging information | | **Timing** | Duration of each operation | +By default, traces capture only metadata such as timing and token counts. +Prompt and response content is recorded only when content capture is enabled; see [](../../observability/tracing/content-capture.md). + ## Example Configurations ### Development Configuration diff --git a/docs/observability/tracing/content-capture.md b/docs/observability/tracing/content-capture.md new file mode 100644 index 0000000000..90d080f5d3 --- /dev/null +++ b/docs/observability/tracing/content-capture.md @@ -0,0 +1,150 @@ +--- +title: + page: Capturing Message Content + nav: Content Capture +description: Capture prompts, responses, and rail inputs on guardrails spans for debugging, with privacy controls and OpenTelemetry GenAI formats. +topics: +- Observability +- AI Safety +tags: +- Tracing +- OpenTelemetry +- Content Capture +- Privacy +- GenAI +content: + type: how_to + difficulty: technical_intermediate + audience: + - engineer + - DevOps Engineer + - AI Engineer +--- + +(content-capture)= + +# Capturing Message Content + +By default, guardrails spans carry only metadata: durations, token counts, finish reasons, and which rails activated. +The prompts and responses themselves are not recorded. +Content capture is an opt-in feature that adds the actual user inputs, model outputs, and rail inputs to your spans, so you can debug blocked prompts, investigate false positives in safety rails, and see exactly what a model received and returned. + +:::{admonition} Experimental Feature +The inline content-capture behavior described on this page is emitted by the opt-in IORails engine. +To enable IORails, set `NEMO_GUARDRAILS_IORAILS_ENGINE=1`. +IORails is an early-release feature, and the captured attribute and event names follow the OpenTelemetry GenAI semantic conventions, which are still under active development and can change. +::: + +:::{warning} +Enabling content capture writes user inputs and model outputs to your telemetry backend. +This may include personally identifiable information (PII) and other sensitive data. +Only enable it when necessary, restrict access to the backend that receives the spans, and ensure compliance with your data-protection obligations. +::: + +## Enabling Content Capture + +Content capture is controlled by the `enable_content_capture` field in the `tracing` section of `config.yml`: + +```yaml +tracing: + enabled: true + enable_content_capture: true # default: false + adapters: + - name: OpenTelemetry +``` + +Content is only captured when tracing is also enabled — there is no point recording content onto spans that are never exported. + +### Environment Variable Override + +The `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` environment variable overrides the config field in **both** directions. +This gives operators a single OpenTelemetry-standard switch to flip capture across all services, regardless of what each deployed `config.yml` says. + +| `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | Result | +|------------------------------------------------------|--------| +| `true` or `1` | Capture is forced **on**, even if `enable_content_capture: false`. | +| `false` or `0` | Capture is forced **off**, even if `enable_content_capture: true`. | +| Unset, empty, or any other value | Falls through to the `enable_content_capture` config field. | + +Values are case-insensitive, and surrounding whitespace is ignored. + +## Output Format + +Captured content is emitted in one of two forms, selected by the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable. +This variable holds a comma-separated list of opt-in tokens. + +| `OTEL_SEMCONV_STABILITY_OPT_IN` | Format | What is emitted | +|---------------------------------|--------|-----------------| +| Contains `gen_ai_latest_experimental` | **JSON span attributes** | Structured, JSON-encoded span attributes following the latest experimental OpenTelemetry GenAI conventions. | +| Unset, or does not contain the token (default) | **Legacy span events** | One span event per message, following the earlier GenAI event conventions. | + +### JSON Span Attributes + +When `OTEL_SEMCONV_STABILITY_OPT_IN` contains `gen_ai_latest_experimental`, content is recorded as JSON-encoded span attributes: + +| Attribute | Contents | +|-----------|----------| +| `gen_ai.input.messages` | The non-system input messages, each as `{"role": ..., "parts": [{"type": "text", "content": ...}]}`. | +| `gen_ai.output.messages` | The assistant output, as a single role-wrapped message. | +| `gen_ai.system_instructions` | The system messages, as a flat list of `{"type": "text", "content": ...}` parts (no role wrapper, per the specification). | + +Each attribute is set only when it has content, so a backend can distinguish "no system instructions" from an empty string. + +### Legacy Span Events + +By default — when `OTEL_SEMCONV_STABILITY_OPT_IN` is unset or does not include the token — content is recorded as span events instead: + +| Event | Emitted for | +|-------|-------------| +| `gen_ai.system.message` | Each system message. | +| `gen_ai.user.message` | Each user message. | +| `gen_ai.assistant.message` | Each assistant message in the input. | +| `gen_ai.choice` | The assistant output (the response). | + +Roles outside this set (for example, `tool`) are skipped; tool and function-call events are not yet captured. + +```{note} +This format selection is independent of the `tracing.span_format` config field. +`span_format` (`opentelemetry` or `legacy`) selects the span structure used by the LLMRails post-hoc tracing adapter, whereas `OTEL_SEMCONV_STABILITY_OPT_IN` selects how IORails encodes captured content on its inline spans. +``` + +## Where Content Is Captured + +When capture is active, content lands on the following spans. + +| Span | Kind | Captured content | +|------|------|------------------| +| `guardrails.request` | SERVER | The request input messages, and the final output delivered to the caller, using the `gen_ai.*` attribute or event names above. | +| `gen_ai.*` | CLIENT | The input messages and output of every LLM call — both the main LLM and the per-rail-action LLM calls (for example, content-safety models). | +| `guardrails.rail` | INTERNAL | `guardrails.rail.input` — the JSON-encoded rail input (`{"messages": [...], "bot_response": ...}`). On a rail that blocks, `guardrails.rail.reason` also carries the human-readable block reason. | + +Because the request span and the LLM spans share the same `gen_ai.*` attribute and event names, a backend can correlate the outer guardrails request with the inner model calls by name alone. + +## Streaming + +For streamed responses, output chunks are accumulated and the captured output is written once, at the end of the stream. +The recorded output is exactly what reached the consumer. +If an output rail blocks mid-stream, the captured output reflects the truncated stream plus any injected error response — not text the caller never received. +When nothing is delivered, no output is recorded. + +## Engine Support + +| Engine | Content capture | +|--------|-----------------| +| **IORails** | Preview support. The full behavior on this page — environment-variable resolution, JSON-attribute or legacy-event format selection, and capture on the request, LLM, and rail spans — is emitted by the opt-in `IORails` engine. | +| **LLMRails** | The `enable_content_capture` field is honored by the LLMRails post-hoc tracing adapter, but content is emitted through that adapter's own span extractors and attribute names. The `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` and `OTEL_SEMCONV_STABILITY_OPT_IN` controls described here apply to IORails. | + +## Important Considerations + +- **Privacy first.** + Captured spans contain raw prompts and responses. + Treat the receiving backend as holding sensitive data, and prefer enabling capture in development or in scoped investigations rather than broadly in production. +- **No truncation.** + Content is captured in full; there is no size limit or truncation knob. + Size your exporters and backend accordingly, especially for large inputs or long streamed responses. +- **Evolving GenAI standards.** + The [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) are still under active development. + Attribute names, event names, and structures can change as the specification matures. +- **Performance.** + Extensive telemetry collection can affect performance, especially with large inputs and outputs. + The hot-path cost is dominated by SDK-level batching and export, which your application controls. diff --git a/docs/observability/tracing/index.md b/docs/observability/tracing/index.md index d4f64c072b..262888860e 100644 --- a/docs/observability/tracing/index.md +++ b/docs/observability/tracing/index.md @@ -84,7 +84,7 @@ The following are the key differences between the supported span formats. **Development Status**: The [OpenTelemetry semantic conventions for GenAI](https://opentelemetry.io/docs/specs/semconv/gen-ai/) are currently in development and may undergo changes. Consider the following risks: - **Evolving Standards**: Conventions may change as they mature, potentially affecting existing implementations -- **Data Privacy**: The `enable_content_capture` option captures user inputs and model outputs, which may include sensitive information (PII). Only enable when necessary and ensure compliance with data protection regulations. See [GenAI Events documentation](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) for details +- **Data Privacy**: The `enable_content_capture` option records user inputs and model outputs onto spans, which may include sensitive information (PII). Only enable when necessary and ensure compliance with data protection regulations. See [](content-capture.md) for the full behavior, environment-variable controls, and privacy guidance - **Performance Impact**: Extensive telemetry collection may impact system performance, especially with large inputs/outputs ### Migration Path @@ -100,6 +100,7 @@ Existing configurations will continue to work. However, it is strongly recommend - [](quick-start.md) - Minimal setup to enable tracing using the OpenTelemetry SDK - [](adapter-configurations.md) - Detailed configuration for FileSystem, OpenTelemetry, and Custom adapters +- [](content-capture.md) - Capture prompts, responses, and rail inputs on spans, with privacy controls and output formats - [](opentelemetry-integration.md) - Production-ready OpenTelemetry setup and ecosystem compatibility - [](opentelemetry-logs.md) - Forward guardrails Python logs to OpenTelemetry with trace correlation - [](troubleshooting.md) - Common issues and solutions @@ -114,6 +115,7 @@ Existing configurations will continue to work. However, it is strongly recommend Quick Start Adapters +Content Capture OpenTelemetry OpenTelemetry Logs Troubleshooting From f7ea5da6d283bb77bb5a05b2a1bde5c4075a196b Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Wed, 3 Jun 2026 15:33:11 -0500 Subject: [PATCH 2/4] Address coderabbit feedback --- docs/observability/tracing/content-capture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/observability/tracing/content-capture.md b/docs/observability/tracing/content-capture.md index 90d080f5d3..a84c380486 100644 --- a/docs/observability/tracing/content-capture.md +++ b/docs/observability/tracing/content-capture.md @@ -118,7 +118,7 @@ When capture is active, content lands on the following spans. | `gen_ai.*` | CLIENT | The input messages and output of every LLM call — both the main LLM and the per-rail-action LLM calls (for example, content-safety models). | | `guardrails.rail` | INTERNAL | `guardrails.rail.input` — the JSON-encoded rail input (`{"messages": [...], "bot_response": ...}`). On a rail that blocks, `guardrails.rail.reason` also carries the human-readable block reason. | -Because the request span and the LLM spans share the same `gen_ai.*` attribute and event names, a backend can correlate the outer guardrails request with the inner model calls by name alone. +Because the request span and the LLM spans belong to the same trace, a backend correlates the outer guardrails request with the inner model calls through trace and span context (the `trace_id` and parent-child `span_id` relationships). The shared `gen_ai.*` attribute and event names do not establish that link — names repeat across requests — but they make the captured content easier to interpret once the spans are correlated. ## Streaming From 20c274fcf9154df8847253ce367da7889d8b4fb1 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Wed, 3 Jun 2026 15:49:49 -0500 Subject: [PATCH 3/4] Address Greptile / coderabbit feedback --- docs/configure-rails/yaml-schema/tracing-configuration.md | 2 +- docs/observability/tracing/content-capture.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/configure-rails/yaml-schema/tracing-configuration.md b/docs/configure-rails/yaml-schema/tracing-configuration.md index 3ae593f905..d6499fab66 100644 --- a/docs/configure-rails/yaml-schema/tracing-configuration.md +++ b/docs/configure-rails/yaml-schema/tracing-configuration.md @@ -49,7 +49,7 @@ tracing: | `enabled` | Enable or disable tracing | `false` | | `adapters` | List of tracing adapters | `[]` | | `span_format` | Span structure used by the LLMRails adapter: `opentelemetry` or `legacy` | `opentelemetry` | -| `enable_content_capture` | Record prompts, responses, and rail inputs on spans. Captures potentially sensitive data (PII); see [](../../observability/tracing/content-capture.md) | `false` | +| `enable_content_capture` | Record prompts and responses on spans. Captures potentially sensitive data (PII); see [](../../observability/tracing/content-capture.md) for full behavior and engine-specific differences | `false` | ## Tracing Adapters diff --git a/docs/observability/tracing/content-capture.md b/docs/observability/tracing/content-capture.md index a84c380486..65d025959c 100644 --- a/docs/observability/tracing/content-capture.md +++ b/docs/observability/tracing/content-capture.md @@ -57,6 +57,8 @@ Content is only captured when tracing is also enabled — there is no point reco ### Environment Variable Override +*Applies to the IORails engine only — see [Engine Support](#engine-support) below.* + The `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` environment variable overrides the config field in **both** directions. This gives operators a single OpenTelemetry-standard switch to flip capture across all services, regardless of what each deployed `config.yml` says. @@ -70,6 +72,8 @@ Values are case-insensitive, and surrounding whitespace is ignored. ## Output Format +*Applies to the IORails engine only — see [Engine Support](#engine-support) below.* + Captured content is emitted in one of two forms, selected by the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable. This variable holds a comma-separated list of opt-in tokens. From e7c0d256775178cd800af1060addabfaadc7e559 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Fri, 5 Jun 2026 12:03:43 -0500 Subject: [PATCH 4/4] Update docs to reflect final merged state of the code PR #1972 --- docs/observability/tracing/content-capture.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/observability/tracing/content-capture.md b/docs/observability/tracing/content-capture.md index 65d025959c..ecbb89a4b4 100644 --- a/docs/observability/tracing/content-capture.md +++ b/docs/observability/tracing/content-capture.md @@ -74,8 +74,9 @@ Values are case-insensitive, and surrounding whitespace is ignored. *Applies to the IORails engine only — see [Engine Support](#engine-support) below.* -Captured content is emitted in one of two forms, selected by the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable. +The `gen_ai.*` content captured on the LLM calls is emitted in one of two forms, selected by the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable. This variable holds a comma-separated list of opt-in tokens. +The format selector applies only to this `gen_ai.*` content; the `guardrails.request.*` and `guardrails.rail.*` attributes described under [Where Content Is Captured](#where-content-is-captured) are always emitted as plain span attributes regardless of its value. | `OTEL_SEMCONV_STABILITY_OPT_IN` | Format | What is emitted | |---------------------------------|--------|-----------------| @@ -103,9 +104,10 @@ By default — when `OTEL_SEMCONV_STABILITY_OPT_IN` is unset or does not include | `gen_ai.system.message` | Each system message. | | `gen_ai.user.message` | Each user message. | | `gen_ai.assistant.message` | Each assistant message in the input. | +| `gen_ai.tool.message` | Each tool message in the input. | | `gen_ai.choice` | The assistant output (the response). | -Roles outside this set (for example, `tool`) are skipped; tool and function-call events are not yet captured. +Roles outside this set (for example, the legacy `function` role) are skipped; function-call events are not yet captured. ```{note} This format selection is independent of the `tracing.span_format` config field. @@ -118,11 +120,16 @@ When capture is active, content lands on the following spans. | Span | Kind | Captured content | |------|------|------------------| -| `guardrails.request` | SERVER | The request input messages, and the final output delivered to the caller, using the `gen_ai.*` attribute or event names above. | -| `gen_ai.*` | CLIENT | The input messages and output of every LLM call — both the main LLM and the per-rail-action LLM calls (for example, content-safety models). | +| `guardrails.request` | SERVER | `guardrails.request.input` — the JSON-encoded caller input messages — and `guardrails.request.output` — the plain-text response actually delivered to the caller (the refusal message when a rail blocks). These are always plain span attributes, independent of the `OTEL_SEMCONV_STABILITY_OPT_IN` format selector. | +| `gen_ai.*` | CLIENT | The input messages and output of every LLM call — both the main LLM and the per-rail-action LLM calls (for example, content-safety models) — using the `gen_ai.*` attribute or event names above. | | `guardrails.rail` | INTERNAL | `guardrails.rail.input` — the JSON-encoded rail input (`{"messages": [...], "bot_response": ...}`). On a rail that blocks, `guardrails.rail.reason` also carries the human-readable block reason. | -Because the request span and the LLM spans belong to the same trace, a backend correlates the outer guardrails request with the inner model calls through trace and span context (the `trace_id` and parent-child `span_id` relationships). The shared `gen_ai.*` attribute and event names do not establish that link — names repeat across requests — but they make the captured content easier to interpret once the spans are correlated. +The request span deliberately uses its own `guardrails.request.*` attributes rather than the `gen_ai.*` names. +On a block path the two diverge: the LLM CLIENT span records the raw model response, while the SERVER span records what the caller actually received — the refusal message. +Reusing `gen_ai.output.messages` on both would put different values under the same name and confuse a backend correlating them. + +Because the request span and the LLM spans belong to the same trace, a backend correlates the outer guardrails request with the inner model calls through trace and span context (the `trace_id` and parent-child `span_id` relationships). +The shared `gen_ai.*` names across the LLM CLIENT spans do not establish that link — names repeat across requests — but they make the captured content easier to interpret once the spans are correlated. ## Streaming