Cyclone DDS "finer" tracing improves performance compared to tracing "none"

[gianlucapargaetzi]

Hello

I am currently working on implementing an Multimedia Streaming Pipeline between an Nvidia Jetson Orin NX 16GB and a GPU Server using Intel E810 network cards with 25 Gbit/s transceivers.

With the following parametrisation, I reach 13.5 Gbit/s udp throughput with 0% package loss using iperf3:

sudo ip link set dev $INTERFACE_NAME mtu 9000
sudo ethtool --set-ring $INTERFACE_NAME rx 8160 tx 8160
sudo ip link set dev $INTERFACE_NAME txqueuelen 10000

sudo sysctl -w net.core.rmem_default=73400320
sudo sysctl -w net.core.rmem_max=2147483647
sudo sysctl -w net.core.wmem_default=73400320
sudo sysctl -w net.core.wmem_max=73400320
sudo sysctl -w net.ipv4.ipfrag_time=3
sudo sysctl -w net.ipv4.ipfrag_high_thresh=134217728
sudo sysctl -w net.core.netdev_max_backlog=3000

My Cyclone DDS config looks as followed:

<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
  <Domain Id="any">
    <General>
      <Interfaces>
        <NetworkInterface name="enp3s0f0np0" priority="2" multicast="default" />
      </Interfaces>
      <AllowMulticast>default</AllowMulticast>
      <MaxMessageSize>9000B</MaxMessageSize>
    </General>
    <Internal>
      <SocketReceiveBufferSize min="10MB"/>
      <Watermarks>
        <WhcHigh>500kB</WhcHigh>
      </Watermarks>
    </Internal>
  </Domain>
</CycloneDDS>

I am trying to stream an Color Image 1080p30 (16bpp YUYV) from a Intel RealSense D435I, but am not able to reach full framerate on the receiving side.
While also experiencing the same issue as describe in Issue #2139, I tried logging the DDS to see what is going wrong.

To my suprise, I suddenly reached the full framerate upon changing logging verbosity from none / info to finer with a single subscriber on the receiver platform.

Any reason why increasing logging improves the performance? Out of intuition, I thought it should be the other way around?

Looking forward to suggestions and inputs!