Proposal: Using eBPF to Control Core Load Balancing

[Big-Dan-Plus]

If DPDK is disabled, Seastar must switch to POSIX mode. I found that the function posix_reuseport_available always returns false due to "heavy load imbalance", which decreases server performance. Currently, I have to use a single core to receive data via udp_channel, and then transmit the received data to other cores using submit_to or similar functions to handle cross-core processing.
I propose that using an eBPF socket option could improve this situation:

1. Register an eBPF callback in the kernel using setsockopt with SO_ATTACH_REUSEPORT_EBPF.
2. Periodically update the idle state of each core via a BPF map (shared memory).
3. Let the kernel compute and choose the socket index to emit data when network data arrives.
4. The chosen socket index corresponds to a specific core, ensuring that the same core will receive the data at the user level.

This plan still requires a sustainable algorithm to coordinate core workloads without user-level intervention. Implementing this in the kernel is a lower-level approach, which will require additional adaptation to support common systems efficiently.

    // FIXME: reuseport currently leads to heavy load imbalance.
    // Until we fix that, just disable it unconditionally.
    bool posix_reuseport_available() const { return false; }

This is just my personal opinion.