Seastar: per-shard TCP listeners not binding or exiting silently on non-0 shards

[somesh-m]

I’m trying to start a per-shard TCP listener in Seastar, where each shard binds to its own port (6010 + shard_id).
On shard 0, everything works — the listener binds and accepts connections fine.
But on other shards, the listener either exits silently or never shows as bound.
No exceptions are thrown, and the accept loop doesn’t seem to run. I had earlier used keep_doing but then changed to repeat just to make sure accept loop retries. Has anyone faced the same issue. Assuming this is possible to do, please help me out.

The specific use case is that I want to send requests to a particular shard based on a hash function. Since Seastar doesn’t provide a direct way to route incoming network requests to a specific shard, I’m experimenting with port-based routing, where each shard listens on a unique port representing its shard ID.

Here is the sample code used:

#include <boost/iterator/counting_iterator.hpp>
#include <boost/range/irange.hpp>
#include <chrono>
#include <coroutine>
#include <seastar/core/app-template.hh>
#include <seastar/core/future.hh>
#include <seastar/core/reactor.hh>
#include <seastar/core/sleep.hh>

#include <seastar/net/api.hh>

using namespace seastar;

seastar::future<> init();
seastar::future<> start();

int main(int argc, char **argv) {
    app_template app;
    try {
        return app.run(argc, argv, init);
    } catch (...) {
        std::cerr << "Couldn't start application: " << std::current_exception()
                  << "\n";
        return 1;
    }
    return 0;
}

seastar::future<> init() {
    return seastar::parallel_for_each(
        boost::irange<unsigned>(0, seastar::smp::count),
        [](unsigned c) { return seastar::smp::submit_to(c, &start); });
}

future<> start() {
    const auto sid = seastar::this_shard_id();
    const auto port = uint16_t(6010 + sid);
    std::cout << "Starting server on shard " << sid << " port " << port << "\n";
    seastar::listen_options lo;
    lo.reuse_address = true;
    return seastar::do_with(
        seastar::listen(make_ipv4_address({port}), lo),
        [port, sid](seastar::server_socket &listener) {
            return seastar::repeat(
                [&listener, sid,
                 port]() -> seastar::future<seastar::stop_iteration> {
                    return listener.accept()
                        .then([sid, port](seastar::accept_result ar) {
                            std::cout << "Accepted connection on shard " << sid
                                      << " port " << port << "\n";
                        })
                        .then([] {
                            return seastar::stop_iteration::no; // keep
                                                                // accepting
                        });
                });
        });
}

OS: Ubuntu 24.04.3 LTS

Comand used:
sudo strace -ff -ttT -e trace=bind,listen,accept -o strace.log ./testsocket --smp 2

Things I’ve already checked

• Each shard uses a unique port (no collisions)
• reuse_address = true
• No firewall or permission issues
• Tried both keep_doing and repeat
• Tested with multiple --smp values
• No visible exceptions in logs or console
• Tried to check the system calls via strace and I could find only one system call for the base port 6010 in this case. There were no other socket binding system calls. cat strace.log.36918 07:06:15.075703 bind(13, {sa_family=AF_INET, sin_port=htons(6010), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 <0.000014> 07:06:15.075809 listen(13, 100) = 0 <0.000010>

Expected Behaviour:

Each shard should independently bind to a unique TCP port (6010 + shard_id) and stay in an active accept loop.
For example, on an 8-core setup (--smp=8), the following should occur:

Shard 0 → Port 6010

Shard 1 → Port 6011

Shard 2 → Port 6012

…

Shard 7 → Port 6017

[niekbouman]

1st tip: use coroutines, easier to read .
2nd tip: use
```cpp
(code here)
```
to get GitHub syntax highlighting
3rd tip, use fixed_cpu load balancing strategy, first make a working single-threaded implementation, then you can use sharded class to run on multiple threads.
https://docs.seastar.io/master/structseastar_1_1listen__options.html
https://docs.seastar.io/master/classseastar_1_1sharded.html
4rd tip: look at the examples / unit test for api use

[niekbouman]

for every port, you need to call listen & accept on each shard; at least shard 0 must listen on all ports (because under the hood, all connections are accepted on shard0, and then xferred to the actual target shard)

[somesh-m]

Thanks, @niekbouman — this did the trick. I had shard 0 listen on all ports and mapped each shard to its respective port. With fixed_cpu load balancing enabled, everything worked smoothly. I’m drafting an article to share these learnings with the community and will post it here soon.

[somesh-m]

Here is the sample code https://github.com/somesh-m/seastar-samples/tree/master/all_shard_listener