Repository navigation
Conversation
Kranzes
force-pushed
the
async-tokio-io
branch
from
October 19, 2026 16:42
a1aa8b6 to
75ff4d6
Compare
Kranzes
marked this pull request as ready for review
October 19, 2026 16:49
Kranzes
requested review from
bergwolf,
eryugey,
imeoer,
jiangliu and
liubogithub
as code owners
October 19, 2026 16:49
Kranzes
force-pushed
the
async-tokio-io
branch
from
October 19, 2026 17:19
ac1e92a to
4197d2f
Compare
The crate detects the runtime type on first use, and only the FUSE_BACKEND_RS_ASYNC_RUNTIME environment variable overrides it. A daemon that drives FuseDevTask from its own tokio runtime has to pick tokio, because that runtime can't poll tokio-uring files. Setting the variable from the daemon needs std::env::set_var, which is unsafe once other threads may read the environment. Add async_runtime::set_runtime_type() to select the type in code. It overrides the variable and fails if the type in effect is a different one. Add async_runtime::runtime_type() to read the type, and make RuntimeType public for both. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
With the tokio runtime, FuseDevTask reads the nonblocking fuse device with plain preadv(). While no request is pending, the read fails with EAGAIN, and poll_handler() yields and reads again right away, so an idle task keeps a CPU busy. Register the fd with the runtime's IO driver and wait until it is readable before each read and after EAGAIN. Wait for errors too, because the kernel reports a torn down connection with EPOLLERR alone. A 100ms timer wakes the wait, so an idle task still returns once its state flag is set. Without the timer, stopping a worker whose unmount fails would hang. An error that isn't an OS error, such as the runtime shutting down, ends the task. The io_uring runtime is unchanged. AsyncFd::register_with_interest() needs tokio 1.53.3, which deprecates with_interest(). Registering panics outside of a tokio runtime, so a task that another executor polls keeps retrying reads as before. The docs say that dropping poll_handler() to stop an idle task is only safe with the tokio runtime. tokio-uring leaves its read pending in the kernel, which then copies the next request into the freed buffer. One test checks that an idle task uses little CPU and returns on unmount. Before this change it used all of the 500ms it idled. Another checks that the state flag stops an idle task while the mount stays up. A third serves a mount without any tokio runtime. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
A daemon can hand its FUSE device fds to a new process, which then serves the connection without a FUSE_INIT. The new Server can't learn what the old one negotiated, so it parses requests as if no option were enabled, such as the FUSE_EXT_GROUPS extension of create requests. Add Server::init_state() to read the protocol version and the options enabled at INIT, and Server::restore_init() to set them on the new server. Add abi::fuse_abi::UNIQUE_RESEND, the bit the kernel sets in requests that notify_resend() makes it send again. The server already replies with the unique id unchanged, and a test now covers that. Document that a thread that serves requests must not call notify_inval_entry(). It waits for the parent's lock, which a lookup in that directory holds until its reply. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
Add tests that serve a mount with FuseDevTask on a current-thread tokio runtime, from a file system that can make a lookup wait. They cover two cases: - A waiting lookup doesn't hold up another lookup in the same directory, and notify_inval_entry() from another thread works while the task serves. - A second server takes over the connection from one that stopped with a lookup in flight. It restores the INIT state, sends notify_resend() and answers the lookup that the kernel sends again. The file system enables FUSE_PARALLEL_DIROPS, because without it the kernel serializes lookups in a directory. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
The async server logged a read error it didn't know and read again. Now that a tokio task waits for the device to become readable, a device that stays readable but fails every read made the task spin at full CPU and log on every pass. EINVAL for a buffer smaller than the kernel's requests does that. Retry ENOENT, which the kernel returns when a request was interrupted before the read, and stop the task on any other error. The synchronous server does the same. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
AsyncFileReadWriteVolatile used tokio_uring only for IoBuf::bytes_total(), so call FileVolatileBuf::cap() instead. The vfs async handlers used the Linux-only libc::stat64 and VfsOptions::no_open, so use fuse_abi::stat64 and check no_open on Linux only. The umbrella crate now forwards async-io to the transports with pkg?/feature, so async-io alone no longer pulls in the Linux-only virtiofs transport. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
async-io failed to build on macOS, because it pulled in io-uring on every platform and io-uring only builds on Linux. The code that uses io_uring was already Linux-only. Make io-uring a Linux-only dependency, like tokio-uring. Other platforms run async-io on tokio. The macfuse example gets the AsyncFileSystem impl that async-io requires, and the Makefile builds and tests async-io on macOS. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
When the io-uring probe failed, set_runtime_type(RuntimeType::Uring) returned "async runtime type is Tokio, not Uring". The probe only logged its reason, so an application couldn't tell a low RLIMIT_MEMLOCK from a seccomp filter. Keep the probe's error and return it from set_runtime_type(), with its kind. The fallback warning includes it too. The test blocks io_uring_setup() with a seccomp filter, and skips where it can't install one, such as under QEMU. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
async_io_microbench times writes and reads between a file and 1, 16 or 256 page-sized buffers, as a virtio-fs descriptor chain hands them over. It also times PassthroughFs READ and WRITE of 4 KiB, 128 KiB and 1 MiB with the blocking thread pool off and on. Its READ reply buffer refuses the inline page cache attempt, so READ measures the cache miss path. The daemon gets a --thread-pool flag for the async mode, and bench_sync_async.sh gets a pool mode that passes it. Both micro-benchmarks share their PassthroughFs setup in benches/common. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
The tokio runtime serves macOS, Linux without io_uring, and applications that select it. On it, every async file operation is a blocking preadv() or pwritev(). Still, async_read_vectored_at_volatile() and async_write_vectored_at_volatile() split the buffers into one operation each, which adds no concurrency. A virtio-fs WRITE of 1 MiB in 256 pages makes 256 pwritev() calls. Make one call for all buffers, unless the file goes through io_uring, which keeps the split. preadv() and pwritev() make one call per 1024 buffers, the most one call takes on Linux and macOS. The new RuntimeType::file_io_blocks() tells callers such as PassthroughFs whether file IO blocks the calling thread. In async_io_microbench on tokio, with the data in the page cache, 256 buffers of 4 KiB take 22.5 µs instead of 144 µs to write and 20 µs instead of 67 µs to read. 16 buffers take 1.4 µs instead of 5.0 µs to write and 1.3 µs instead of 4.0 µs to read. With the page cache dropped, reading 16 buffers takes 549 µs instead of 645 µs. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
On the tokio runtime, PassthroughFs serves READ cache misses and every WRITE with a blocking preadv() or pwritev() on the runtime thread, even with enable_async_thread_pool(). A slow backing file then stalls every other request of the worker. The O_DIRECT and WRITE_KILL_PRIV requests that it relays to the synchronous handlers block the runtime thread on any runtime. With the pool enabled, serve these requests with the synchronous handlers on a pool thread. The pool thread never touches the transport buffer, because the request frees it when its future is dropped, for example for a takeover. A READ reads into a buffer that the pool task owns, and the runtime thread copies it into the reply. A READ larger than the reply buffer fails before the task allocates. A WRITE copies its payload out of the request before the task starts. A task that no pool thread has started when its request is dropped doesn't run, so the old copy of a WRITE can't overwrite newer data after a takeover. Measured with one worker on the tokio runtime and the backing files on ext4: - With a READ waiting in a FUSE file system below, reading another file through the mount takes 1.6 ms instead of 2 s. - fio random 4 KiB reads with 4 jobs and a cold page cache reach 34.5K IOPS instead of 12.1K. - Cached fio random 4 KiB writes reach 424K IOPS instead of 433K. The copy and the thread handoff add about 3.5 µs to a 4 KiB READ miss or WRITE. READ cache hits stay inline on file systems with RWF_NOWAIT. On others, such as tmpfs, overlayfs and FUSE, every READ counts as a miss. io_uring keeps its native path for buffered READ and WRITE. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
lseek() took the handle's write lock, which READ and WRITE hold as readers for the length of their IO. With the thread pool, an LSEEK on a handle with a pooled READ or WRITE in flight blocked the runtime thread until the IO ended. Take the read lock instead. READ and WRITE use positioned IO, so the file position doesn't matter to them. do_readdir() still seeks and reads a directory under the write lock, so an lseek() can't move the position in between. lseek() now also drops the handle's cached readdir cookie, which a seek makes stale. The kernel sends LSEEK only for regular files, so both only guard against other clients. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
async_read() looks up the handle data for its inline RWF_NOWAIT attempt. On a miss it ran read() on the pool, which looked the handle up again. In no_open mode a lookup opens the file, so a pooled READ opened it twice. Move the body of read() into do_read(), which takes the handle data, and pass the data that async_read() already has to the pool. An O_DIRECT READ now looks up its handle in async_read() too, so in no_open mode it opens the file on the runtime thread, as a buffered READ already did. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
A pooled O_DIRECT READ read into the synchronous handler's page-aligned bounce buffer, copied that into the pool task's buffer, and the runtime thread then copied it into the reply. A pooled O_DIRECT WRITE took the same two copies the other way. Start the pool task's buffer at a page boundary, and let do_read() and the new do_write() do the direct IO on it, without the bounce buffer. Like read_direct() and write_direct(), they read until EOF and write the whole payload. The buffer is a Vec that is one page larger than needed. Allocating it with posix_memalign() instead made a pooled 1 MiB READ four times slower. If the allocation fails, the request gets ENOMEM, where the WRITE payload's Vec used to abort. In async_io_microbench on tokio, with O_DIRECT cases added locally, a pooled O_DIRECT READ of 4 KiB takes 23.4 instead of 24.7 µs and a WRITE 20.2 instead of 22.1 µs. Device IO dominates the larger sizes. A buffered 1 MiB READ on the pool takes 37.8 instead of 43.2 µs. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
If a request future is dropped while a pool thread serves its LOOKUP, OPEN or CREATE, for example for a takeover, the pool thread still creates a lookup count or a handle that the kernel never learns of. OPEN leaks the handle and its fd. LOOKUP leaks a lookup count, which keeps the inode and its O_PATH fd until the file system is dropped. CREATE leaks both. The pool task now returns its result in a guard that undoes it if nobody claims it. OPEN releases the handle, LOOKUP forgets the lookup count, and CREATE does both. The JoinHandle hands the result back, so the undo runs on the pool thread if the request is gone when the task ends, and otherwise on the thread that drops the request. Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
Kranzes
force-pushed
the
async-tokio-io
branch
from
October 28, 2026 15:05
4197d2f to
f696277
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NEEDS #270. This branch is stacked on #270, which is stacked on #267. The commits of this PR start at "benchmark: measure vectored IO and the passthrough thread pool".
On the tokio runtime, every async file operation is a blocking
preadv()orpwritev()on the runtime thread. That runtime serves macOS, Linux without io_uring, and applications that driveFuseDevTaskfrom their own tokio runtime (#267). This makes those calls fewer, and moves the ones that block onto the blocking thread pool whenPassthroughFs::enable_async_thread_pool(true)is set.async_io_microbenchtimes vectored writes and reads of 1, 16 and 256 page-sized buffers, and PassthroughFs READ and WRITE with the pool off and on. The daemon gets--thread-pool, andbench_sync_async.shgets apoolmode.pwritev()calls. They now make one call per 1024 buffers. io_uring keeps the split. The newRuntimeType::file_io_blocks()tells callers whether file IO blocks the calling thread.lseek()takes the handle's read lock instead of its write lock, so an LSEEK no longer waits for a pooled READ or WRITE on the same handle. It also drops the handle's cached readdir cookie, which a seek makes stale.no_openmode that saves oneopen()per READ.Vecused to abort.Measured on an AMD Ryzen AI 9 HX 370 with Linux 7.2.7, one worker on tokio, backing files on ext4:
The pool costs one copy and a thread handoff per request, about 3.5 µs for 4 KiB. A pooled 4 KiB O_DIRECT READ takes 23.4 µs instead of 24.7 µs with two copies, and a pooled buffered 1 MiB READ 37.8 µs instead of 43.2 µs.
Known limitations, which I left for later:
RWF_NOWAIT. On tmpfs, overlayfs and FUSE every READ goes to the pool.O_APPENDfile can land twice, and anO_EXCLCREATE fails withEEXISTbecause the first one's file stays. Stopping the serving task through its state flag avoids that, and the docs say so.