[Kranzes]

Linux 7.3 can serve the reads and writes of an open with zero-copy over io_uring. Instead of copying a READ or WRITE's data into a payload buffer, the kernel registers the client's pages in the server's io_uring buffer table, and the server moves the data between them and a file with IORING_OP_READ_FIXED and IORING_OP_WRITE_FIXED. It needs buffer pools (#281) and CAP_SYS_ADMIN, and the file system asks for it per open with FOPEN_IO_URING_ZERO_COPY. This is the io_uring side of the zero-copy reads and writes that #17 asks for. #268 covers the kernel passthrough side.

The first commit adds the uAPI and two hooks in core. OpenOptions::IO_URING_ZERO_COPY is the open flag, and abi::fuse_uring gets the ADD_QUEUE and request flags and a setter for the buffer table slot at REGISTER. The server hands a WRITE's data to the file system as a Reader, which transports can't replace. So Reader::with_zero_copy() attaches data that only a ZeroCopySource from the transport can move, and Reader::read_to_at() hands it over. FileReadWriteVolatile::raw_fd() gives the transport the file to move it to. Plain reads, and files without a descriptor, get a copy through memory instead.

The second commit adds passthrough::Config::uring_zero_copy, which asks for zero-copy on the files PassthroughFs opens. Its reads and writes already go through write_from() and read_to() on the backing file. Files it opens with O_DIRECT don't get it, because their IO goes through an aligned bounce buffer, which zero-copy data reaches only through a slower copy in memory.

The third commit serves it in UringFuseServing with UringConfig::zero_copy. It adds the queues with FUSE_URING_ZERO_COPY and gives each worker ring an empty buffer table with a slot per entry, and each entry names its slot at REGISTER. A zero-copy READ gets a reply body that write_from_at() fills with READ_FIXED, and a zero-copy WRITE gets a Reader whose data WRITE_FIXED moves into the file. Both run on the worker's ring, because the kernel registered the pages in that ring's table, and entry completions that arrive meanwhile join the worker's queue. If the kernel refuses zero-copy with EPERM, or with EINVAL because it predates it, UringFuseServing adds the queues without it. Data that a file system reads or writes through memory, or with a file without a descriptor, takes a copy through a memory file per worker, so PassthroughFs's O_DIRECT bounce buffer keeps working on zero-copy opens.

The fourth commit makes that copy cheaper. Each worker maps its memory file once, at the size of a payload buffer, and the file system reads or writes the mapped bytes. A copy through memory now costs the fixed-buffer operation and one copy, with no buffer to allocate and no extra pread() or pwrite(). Reader::read_to() also stops copying through memory when its destination has a file descriptor. Without an offset, WRITE_FIXED writes at the file's position.

Served from a tmpfs as root, with 4 workers and 4 fio jobs doing direct random IO at depth 16, zero-copy made 1 MiB reads 30 to 50% faster (13896 to 15202 MiB/s against 9966 to 10651) and 1 MiB writes 35 to 55% faster (12299 to 12883 MiB/s against 8164 to 9070). At 128 KiB the two settings overlapped. docs/fuse-uring-performance.md has the setup and the table, and the benchmark daemon takes --uring-zero-copy and --no-backing-direct-io to compare.

The mount test runs with zero-copy too. Without CAP_SYS_ADMIN it covers the fallback, and as root it requires zero-copy, checks that requests went through it, and turns O_DIRECT on with fcntl() for one file to drive the copy through memory. It passed both ways on Linux 7.3-rc7, and the rest of the suite passes in a user namespace on both async runtimes, apart from the 14 passthrough tests that need real root and also fail on master. This PR carries #279 and #281, because zero-copy changes the write_from_at() that #279 fixes and needs #281's buffer pools.

The transport here serves the synchronous handlers. Zero-copy for AsyncFileSystem handlers needs async methods on ZeroCopySource and comes with the async io_uring transport, which builds on #267.