fix: PDX-2459 avoid parallel cache-misses on the same token#17
fix: PDX-2459 avoid parallel cache-misses on the same token#17
Conversation
github-actions
Bot
added
🐞 bug
arnecls
added
🐞 bug
github-actions
Bot
added
the
🔀 metadata-server
There was a problem hiding this comment.
Pull request overview
Introduces a per-token FIFO ticket lock to prevent parallel cache misses for the same token, reducing thundering-herd behavior and downstream rate limiting.
Changes:
- Added a
TicketLock(FIFO lock with context-aware cancellation) and auint64min-heap for tracking canceled tickets. - Updated access token handler to serialize same-token cache-miss fetches and return
429on lock acquisition timeout. - Extended token cache with a per-token inflight lock map.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/shared/ticketlock.go | Adds FIFO ticket lock with canceled-ticket tracking to avoid parallel cache misses. |
| internal/shared/intheap.go | Adds HeapUint64 to support min-heap operations for canceled tickets. |
| internal/shared/ticketlock_test.go | Adds a basic unit test for ticket allocation, cancellation, and unlock behavior. |
| cmd/metadata-server/tokencache.go | Adds inflight lock map and accessor to retrieve per-token locks. |
| cmd/metadata-server/tokenhandlers.go | Uses per-token lock on cache miss and returns 429 when lock can’t be acquired. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Arne Claus <arne.claus@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Arne Claus <arne.claus@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Merging this branch will not change overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Arne Claus <arne.claus@gmail.com>
Merging this branch will not change overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
1 similar comment
Merging this branch will not change overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
When a lot of requests for the same token come in at the same time, hitting a cold cache, each request will trigget a cache miss.
In severe cases, this can lead to rate limiting.
This change introduces a FIFO, lock based queue based on the token ID to tackle this problem. The locks may time out based on request context and return a 429 (overloaded, come back later).