Cloudflare: Support Custom Server-to-Server Trace IDs

[parisholley]

Problem Statement

I use Inngest to orchestrate all of my event processing tasks. A feature of their platform is being able to break a single workflow up into multiple "steps", all of which are tagged with a single "run id". For tracing, I would like to keep each "sub step" together inside of a single trace but the library only supports reading trace ids from the header (sentry-trace/baggage).

Solution Brainstorm

Allow for overriding the trace id instead of relying on Sentry.continueTrace(). My current workaround is running this code within my inngest middleware:

Sentry.getActiveSpan()['_traceId'] = v5(ctx.runId, '92f4ea30-d9e0-4750-9d47-69d8c729d79a').replaceAll('-', '');

The only way around this would be to NOT use the out-of-the-box wrappers and roll my own but that means going down an unsupported path.

[mydea]

The way to do this is to use continueTrace. In how far does that not work for you? Can you share your actual code where you invoke/use Sentry?

[parisholley]

sentry-javascript/packages/core/src/tracing/trace.ts

Lines 225 to 229 in b720e15

	return withScope(scope => {
	const propagationContext = propagationContextFromHeaders(sentryTrace, baggage);
	scope.setPropagationContext(propagationContext);
	return callback();
	});

continueTrace still expects a "sentry managed" value vs. an arbitrary UUID that the user owns (eg: many companies have correlation id mechanisms built out already).

continueTrace also only supports "wrapping" a method invocation, in my case, the run id isn't known until a middleware hook is fired within the inngest framework (they provide their own request handler, similar to remix and other cloudflare compatible libs).

[parisholley]

Here is an example of what my middleware looks like:

https://www.inngest.com/docs/features/middleware/create

  return new InngestMiddleware({
    name: 'Sentry',
    init() {
      return {
        onFunctionRun({ ctx, fn }) {
          Sentry.setTag('runId', ctx.runId);

          const span = Sentry.getActiveSpan();

          if (span) {
            Sentry.updateSpanName(span, fn.name);

            // hack to allow chaining inngest runs together by a single run id
            // @ts-ignore
            span['_traceId'] = v5(ctx.runId, '92f4ea30-d9e0-4750-9d47-69d8c729d79a').replaceAll('-', '');
          }

          return {
            transformInput() {
              return {
                ctx: {
                  span<T>(name: string, fn: () => Promise<T>) {
                    return Sentry.startSpan({ name, op: 'function' }, () => fn());
                  },
                },
              };
            },
            finished({ result }) {
              if (result.error) {
                Sentry.captureException(result.error);
              }
            },
          };
        },
      };
    },
  });

[s1gr1d]

Hello, adding an API for overriding the trace ID is a door to open various problems and a subject to break things as spans may not be continued correctly anymore.

What you could do is creating your own handler middleware which is starting/continuing a span just like the Sentry Cloudflare middleware is doing: https://github.com/getsentry/sentry-javascript/blob/develop/packages/cloudflare/src/handler.ts#L56-L94

Can you share how your Sentry setup looks like right now?

[parisholley]

Enforcing the user to rely on sentry-managed data generation (sentry-trace, baggage) is the core issue that isn't impacted by how I have things setup. I've worked in numerous enterprise environments where trace/correlation id patterns have been well established (eg: tying requests from backend to frontend and making joining them via logs/observability) and it isn't feasible to expect a team to rebuild how they do distributed tracing and do it the "sentry way".

If i want to trace a message through a multi-step message queue or due to vendor/technology restrictions have to rely on webhooks/requests (eg: using a MessageSID from twilio) in which I cannot control headers (but can pull an id out of query parameters or payloads), there is no mechanism to tie these spans together.

I would imagine that using out-of-the-box handlers supplied by the SDK gives me benefits (eg: I noticed color coded response status codes in the sentry UI), that I don't want to keep up with every time there is an update. There simply needs to be way to leverage a user-owned value for tying requests together.