Add live graph backend (merge to feature branch)

[wildum]

PR Description

This is the backend part of the new live graph.

The approach is similar to live debugging, except that it adds the callback to all components in the given module.

The live debugging feed is now a struct that contains data needed to build the new graph. Because the data string is not needed for the graph, it's passed around as a function. This way we can use the same struct for the live debugging and the live graph with limited performance overhead.

Which issue(s) this PR fixes

Fixes #2608

Notes to the Reviewer

This was tested manually via curl:

• curl -N http://localhost:12345/api/v0/web/graph to get the data from all components at the root
• curl -N http://localhost:12345/api/v0/web/graph/{moduleID} to get data from a particular module

PR Checklist

• [na] CHANGELOG.md updated (will be done in the feature branch)
• [na] Documentation added (will be done in another branch to the feature branch)
• Tests updated
• [na] Config converters updated

[thampiotr]

Looking good! some comments, but nothing big

[thampiotr]

Looks much, much cleaner without the callbacks to LiveDebugging(...) function from the service. Few more comments, but we're almost there!

[thampiotr]

For all the 3 telemetry signals, can we use Go generics to simplify the code here? Seems like the variation would be only really on plogs.Logs and otelconsumer.Logs types.

[wildum]

that works for plogs.Logs but I don't think that we can use a generic type for otelconsumers.Logs because we call ConsumeLogs with it. That means we will still need an interceptor type for each telemetry signal and each will still have its corresponding ConsumeMetrics|ConsumeLogs|ConsumeTraces func

[thampiotr]

I see. Maybe that could be avoided with a func but if not, then we'll need to live with this duplication :(

[wildum]

I couldn't find a way to reduce the duplication while keeping the complexity of the code low. Having a bit of duplication for Otel (whether it's in the Otel col project or in Alloy) in the logic between metrics/logs/traces seems to be common. (see the fanout consumer for example). Maybe we should keep it simple and follow the same pattern with a bit of duplication?

[thampiotr]

I don't think nextLogs needs to be otelcol.Consumer, you only ever use them to extract component ID.

[wildum]

True but it's quite handy for the components because it's easy to provide.

Would you prefer that I change it to:
PublishLogsIfActive(debugDataPublisher livedebugging.DebugDataPublisher, componentID string, ld plog.Logs, componentIDs []string)
and that I export the extractIDs function?

Or do you have a different idea in mind? I'd like to keep the code as little intrusive as possible in the components