The syslog message generated by Logstash syslog output plugin is not RFC5424 compliant

[tfidecki]

Logstash information:

Please include the following information:

1. Logstash version (e.g. bin/logstash --version) - 8.15.4
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) docker, or tar
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) either systemd or docker
4. How was the Logstash Plugin installed: built-in

JVM (e.g. java -version): bundled JDK

OS version (uname -a if on a Unix-like system):
$ uname -a Linux aacf4bb8c884 5.15.167.4-microsoft-standard-WSL2 #1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
According to the RFC5424 definition, each message HEADER should contain a TIMESTAMP, which can be a value of NILVALUE, or a time stamp in accordance with RFC3339.
NILVALUE is defined as the hyphen character i.e. ASCII value dec 45 / hex 2D

If the @timestamp field is removed from the document before sending out the syslog message, message is generated without TIMESTAMP or NILLVALUE (which is expected)

Following pipeline config was used to test it:

filter {
  mutate {
    remove_field => ["@timestamp", "@version"]
  }

  mutate {
    add_field => {
      "host" => ""
      "message" => "%{clusterId},%{namespace},%{processType},%{processInstance},%{correlationID},%{subsystem},%{id},%{resource},%{severity},%{cause},%{detailedInformation},%{repairAction}"
    }
  }
}

output {
  syslog {
    id => "syslog_event_exporter_no_ssl"
    appname => "TITAN.IUM"
    host => "${NTI_TELEMETRY_SYSLOGHOST}"
    port => "${NTI_TELEMETRY_SYSLOGPORT:6514}"
    protocol => "tcp"
    rfc => "rfc5424"
    sourcehost => ""
    ssl_verify => false
    severity => "%{syslogSeverity}"
    use_labels => true
  }
}

On the following screenshot you can see that either the timestamp or the hyphen character is missing. There is a space character (hex 20) instead.

Steps to reproduce:

1. Configure syslog output to send messages in rfc5424 format
2. Remove @timestamp field before sending the message

Provide logs (if relevant):
See attached pcap file with captured syslog traffic
cap.zip

[tfidecki]

I expect the question: why do you remove the timestamp from the source message?

The answer is that RFC allows you not to send the timestamp in the syslog message. This is the normal situation from the RFC point of view. Then the timestamp is set by the syslog server that receives the message. And in this case, the timestamp field tells us when the message was delivered, not when it was generated at the origin.