Loki datasource reading from subprocess stdout

[sysvinit]

Request

I would like a new Loki data source which allows reading log data from the standard output of a subprocess. When such a source is configured, Alloy should spawn the program given in the configuration as a subprocess, and then forward lines read from the subprocess's standard output to other Loki receivers in the Alloy configuration.

Use case

Concretely, in order to process kernel audit logs received from go-audit. I've written about this previously in the community Discourse: https://community.grafana.com/t/bringing-logs-from-linux-audit-subsystem-into-loki/138209. These logs are potentially quite sensitive, and I'd like to get them into a Loki server as directly as possible, without passing them through intermediaries such as systemd-journald or saving them in a file on disk first (which isn't 100% tamper-proof).

Thus far in my demo environment I've been using go-audit's GELF writer and a GELF listener in Alloy bound to the loopback interface, but an open port on the loopback network interface is tricky to secure against other applications sending fake log data to Alloy and potentially poisoning the logs. Extending go-audit with a writer for the Loki push API would be relatively easy at surface level, but more difficult to make robust from a distributed systems perspective – and this is functionality which already exists in Alloy, which would be a duplication of effort.

More generally, this would be a useful feature in cases where the interface to a given log source is a dedicated reader process (kernel interfaces in particular come to mind) where it's undesired for these logs to be handled in the regular system logging pipeline (journal/syslog/etc), but which should still be collected and aggregated for central analysis.

Having looked at how the other Loki sources in Alloy are implemented I would be interested in drafting up an implementation of this idea, but I unfortunately don't have the spare time presently to put any work into that.