make SocketAppender support server list

[lu-xiaoshuang]

Typically, applications generate massive amounts of log data, and these data should be sent to backend systems for storage, analysis and so on.

At present, a SocketAppender only supports one address-port. Seems that remote server represented by this address-port is very likely the bottleneck (network traffic) and single point failure endpoint (without gateway).

Would it be possible that SocketAppender supports server list?

[swebb2066]

Have you considered sending logging events via the fluent-bit Forward Upstream plugin?

[lu-xiaoshuang]

Have you considered sending logging events via the fluent-bit Forward Upstream plugin?

Good idea. Thank you for your thoughts. @swebb2066

It would be great if there are no additional prerequisites.

[ppkarwasz]

@lu-xiaoshuang,

At present, a SocketAppender only supports one address-port. Seems that remote server represented by this address-port is very likely the bottleneck (network traffic) and single point failure endpoint (without gateway).

It would be great if there are no additional prerequisites.

The Socket appender is based on the basic Java socket classes. These classes don't provide support for connection pooling, so we would need additional libraries (like Apache MINA or Netty) or a lot of custom code to handle multiple connections.

Since TCP does not guarantee a reliable delivery without message loss, it is probably better to look at message queue appenders. Some of them do support multiple servers.

[vy]

@lu-xiaoshuang, curious: what is the log sink you use where you encounter this problem?

[lu-xiaoshuang]

@lu-xiaoshuang, curious: what is the log sink you use where you encounter this problem?

logstash

[lu-xiaoshuang]

@lu-xiaoshuang,

At present, a SocketAppender only supports one address-port. Seems that remote server represented by this address-port is very likely the bottleneck (network traffic) and single point failure endpoint (without gateway).

It would be great if there are no additional prerequisites.

The Socket appender is based on the basic Java socket classes. These classes don't provide support for connection pooling, so we would need additional libraries (like Apache MINA or Netty) or a lot of custom code to handle multiple connections.

Since TCP does not guarantee a reliable delivery without message loss, it is probably better to look at message queue appenders. Some of them do support multiple servers.

Thanks, @ppkarwasz

KafkaAppender is not capable of sending logs generated by org.apache.kafka:kafka-clients:jar, is it?

By the way, If community decides to make SocketAppender support server list, what about creating multiple sockets? When append method is invoked, log4j2 can choose a socket randomly. I think this simple solution is free of connection pool stuff.

[ppkarwasz]

KafkaAppender is not capable of sending logs generated by org.apache.kafka:kafka-clients:jar, is it?

Right, that usually causes problems with recursive logging.

By the way, If community decides to make SocketAppender support server list, what about creating multiple sockets? When append method is invoked, log4j2 can choose a socket randomly. I think this simple solution is free of connection pool stuff.

Log4j is highly pluggable, so I would recommend to:

1. Create and publish a new appender (let's say MultiSocketAppender) in your personal or company repo under Apache-2.0.
2. Test the plugin in your test environment.
3. Announce it on our dev@logging.apache.org mailing list. We will add a section for it in our Appenders page, so users can easily find it.
4. If the appender is well received by users we will consider adding it to the standard distribution.

Note that we are adopting the same strategy for our own plugins, e.g. my copernik-eu/log4j-tomcat is waiting for some user feedback.

[lu-xiaoshuang]

1. in your personal or company repo under Apache-2.0.

Good idea.

[vy]

@lu-xiaoshuang, in #3482, I see you randomly choose hosts to write to. Can't you simply define multiple socket appenders and choose them using scripted routes:

<Configuration xmlns="https://logging.apache.org/xml/ns"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="
                   https://logging.apache.org/xml/ns
                   https://logging.apache.org/xml/ns/log4j-config-2.xsd">
  <Appenders>
    <Routing name="ROUTING">
      <Script language="groovy">
        staticVariables.hosts = [
            'h1.example.com',
            'h2.example.com',
            'h3.example.com'
        ];
        staticVariables.random = new Random();
      </Script>
      <Routes>
        <Script language="groovy">
          int hostIndex = staticVariables.random.nextInt(staticVariables.hosts.length);
          String host = staticVariables.hosts[hostIndex];
          return configuration.properties['host'] = host;
        </Script>
        <Route>
          <Socket name="${host}"
                  protocol="TCP"
                  host="${host}"
                  port="500">
            <JsonTemplateLayout/>
          </Socket>
        </Route>
      </Routes>
    </Routing>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="ROUTING"/>
    </Root>
  </Loggers>
</Configuration>

[lu-xiaoshuang]

@lu-xiaoshuang, in #3482, I see you randomly choose hosts to write to. Can't you simply define multiple socket appenders and choose them using scripted routes:

<Script language="groovy">
staticVariables.hosts = [
'h1.example.com',
'h2.example.com',
'h3.example.com'
];
staticVariables.random = new Random();
</Script>

<Script language="groovy">
int hostIndex = staticVariables.random.nextInt(staticVariables.hosts.length);
String host = staticVariables.hosts[hostIndex];
return configuration.properties['host'] = host;
</Script>

Thanks for your reply, @vy

I'm afraid Routing is not an elegant way. It seems a little bit ugly instead.

It would be great if the framework could hide the underlying complexity (i.e. transparent to users).

I think offering a new SocketAppender (assuming SocketAppenderV2) may be a possible solution. Because origin design of SocketAppender is not easy to support multiple sockets.

When SocketAppenderV2 is ready, users can migrate from old version to new version. At last, log4j can deprecate SocketAppender.