Degraded PointCloud2 performance over 100 Mbps Ethernet with ROS 2

[mariamercade]

Hello,

I’m running into communication issues when sending a ~3.15 MB PointCloud2 at ~10 Hz over a 100 Mbps Ethernet link between two Ubuntu 24.04 machines running ROS 2 Jazzy and Fast DDS.

Setup:

Robot: Intel i7, 8 GB RAM, Ubuntu 24.04 Server, IP: 192.168.33.198

Dev VM: Ubuntu 24.04 Desktop in VirtualBox on Windows, 8 GB RAM, 6 CPUs, IP: 192.168.33.197

Ethernet: 100 Mbps, same subnet

Topic: PointCloud2, ~3.15 MB per message, 10 Hz

QoS: sensor data profile

    History: keep_last(5)

    Reliability: best_effort

    Durability: volatile

Notes:

• On a single machine, the publisher maintains ~10 Hz even with multiple subscribers.
• When subscribing from the VM over Ethernet, the rate drops to ~5.5 Hz.
• Discovery works; topics appear and subscribers connect.
• I followed the ROS 2 DDS tuning guide
• In ROS 1 on the same hardware and Ethernet link the Pointcloud2 transmission setup worked fine at 10 Hz, so I believe it should also be achievable in ROS 2 with proper tuning.

Config:

Here is the fastrtps.xml I’ve been testing with:

<?xml version="1.0" encoding="UTF-8"?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>UDP_transport</transport_id>
            <type>UDPv4</type>
            <interfaceWhiteList>
                <address>192.168.33.198</address>
                <address>192.168.33.197</address>
            </interfaceWhiteList>
            <maxMessageSize>65000</maxMessageSize>
            <sendBufferSize>1048576</sendBufferSize>
    	    <receiveBufferSize>1048576</receiveBufferSize>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="participant_profile_ros2" is_default_profile="true">
        <rtps>
            <userTransports>
                <transport_id>UDP_transport</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
            <sendSocketBufferSize>1048576</sendSocketBufferSize>
            <listenSocketBufferSize>1048576</listenSocketBufferSize>
            <builtin>
                <initialPeersList>
                    <locator>
                        <udpv4>
                            <address>192.168.33.198</address>
                        </udpv4>
                    </locator>
                    <locator>
                        <udpv4>
                            <address>192.168.33.197</address>
                        </udpv4>
                    </locator>
                </initialPeersList>
            </builtin>
        </rtps>
    </participant>
</profiles>

Question:

• Given that this worked in ROS 1 over 100 Mbps, what settings or approaches would you recommend to achieve similar performance in ROS 2 with Fast DDS?

Thanks for your help!

[cferreiragonz]

Hello @mariamercade,

Fast DDS has its own tuning guide for Large Data rates with extra info about how to tune Fast DDS.

In your case, I would directly recommend to increase system buffer sizes. I would also increase the buffer sizes on the XML to twice the message size you are sending, something like 6.5 MB will do.

Could you try again with system buffer sizes and Fast DDS buffers of 6.5 MB?

[mariamercade]

Hello, @cferreiragonz ,

Thanks for the reply.

With the new setup, it seems I cannot see the ROS 2 topic frequency (hz) through the CLI. When I subscribe, this is what I get:

maria@maria ~$ ros2 topic echo /pc --no-arr
A message was lost!!!
total count change:1
total count: X---

(with X incrementing over time)

Just to be clear, these are my current sysctl parameters:

net.ipv4.ipfrag_time=3
net.ipv4.ipfrag_high_thresh=134217728
net.core.rmem_max=6815744
net.core.rmem_default=6815744
net.core.wmem_max=6815744
net.core.wmem_default=6815744

And here is the new Fast DDS config file I am using:

<?xml version="1.0" encoding="UTF-8"?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>UDP_transport</transport_id>
            <type>UDPv4</type>
            <interfaceWhiteList>
                <address>192.168.33.198</address>
                <address>192.168.33.197</address>
            </interfaceWhiteList>
            <maxMessageSize>65000</maxMessageSize>
            <sendBufferSize>6815744</sendBufferSize>
    	    <receiveBufferSize>6815744</receiveBufferSize>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="participant_profile_ros2" is_default_profile="true">
        <rtps>
            <userTransports>
                <transport_id>UDP_transport</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
            <sendSocketBufferSize>6815744</sendSocketBufferSize>
            <listenSocketBufferSize>6815744</listenSocketBufferSize>
            <builtin>
                <initialPeersList>
                    <locator>
                        <udpv4>
                            <address>192.168.33.198</address>
                        </udpv4>
                    </locator>
                    <locator>
                        <udpv4>
                            <address>192.168.33.197</address>
                        </udpv4>
                    </locator>
                </initialPeersList>
            </builtin>
        </rtps>
    </participant>
</profiles>

Am I doing something wrong? Also, as I think I have not said it before, besides the point cloud I have other topics running (e.g., cmd_vel, tf, imu, etc.).

[cferreiragonz]

Hi @mariamercade,

Your sysctl configuration should be enough for your use case. It would be great if you confirm that the XML profiles are being properly loaded on your nodes. Also, could you confirm that you are also loading this participant profile configuration onto the ROS 2 daemon? Otherwise, the subscriber created by the ROS 2 CLI will keep the default (smaller) buffers and could become a bottleneck.

In principle, increasing buffer sizes should not affect discovery, so if you were able to pinpoint the discovery issue to this new configuration I would ask you to share a minimal reproducer with your environment in order to try it on our side. Trying to isolate the publisher and subscriber to test them on their own seems like a good approach to figure out if there exists a real issue regarding this configuration.

Lastly, a small NIT: there is no need to use <initialPeersList> for this scenario. Multicast discovery is sufficient for nodes to find each other.