Sync internal 2026-04-17#864
Merged
Merged
Conversation
…ng graceful upgrades
Add BodyWriter task API (send_body_task, write_current_body_task, send_finish_task, write_current_finish_task) and HeaderWriter for cancel-safe writes that can be used in tokio::select! loops.
Using the proxy task API allows polling for the upstream rx task at the same time, so that upstream cache writes can continue even while serving downstream. proxy_h2 and h2 downstream (as well as custom) is a todo.
This was previously counting the response header bytes as well, which is incorrect.
Add LruUnit::peek_lru(), Lru::peek_lru(shard), and Manager::peek_lru(shard) to peek at the least-recently-used item in a shard without evicting it. Returns None for empty shards or out-of-bounds shard indices. This enables callers to report the eviction frontier — the age of the item that would be evicted next — for cache observability metrics.
Update bench_lru to test at production-level data sizes (~100K and ~500K items/shard). The original benchmark only tested 100 items across 10 shards (10 per shard), which made promote_top_n appear 42% faster. At larger scales, promote() is actually 20-25% faster because the read-lock scan rarely finds hot items near the head. Add heavy-hitter benchmarks (10 and 100 items at 10,000x weight) to test whether extremely concentrated access patterns benefit from promote_top_n. Result: promote() still ties or wins even with heavy hitters, because with few hot items spread across 32 shards, most shards have 0-1 hot items and the scan is wasted on cold accesses. Each benchmark variant uses a fresh LRU and a thread barrier to avoid state contamination and staggered starts. The 16M-item config is gated behind BENCH_LARGE=1 to avoid OOM on CI. Add a performance warning to promote_top_n() docs recommending promote() for large-scale workloads.
Also temp ignore the active RUSTSECs until the internal dependency bumps are synced.
andrewhavck
approved these changes
Apr 18, 2026
johnhurt
approved these changes
Apr 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.