feat(kvevents): reference-count duplicate KV-event removals

[Change72]

What

Adds an eventDedupFilter in pkg/kvevents that reference-counts
BlockStored/BlockRemoved announcements so a duplicate remove does not evict a
block that another announcement still references.

Why

vLLM's OffloadingConnector publishes self-describing, block-granular KV events.
In chunk mode (block_size > engine block size), when a shared prefix is not
aligned to the offloaded chunk size, two overlapping chunks legitimately list the
same constituent block hash, so the same hash is Stored and Removed more than
once on the wire.

Today BlockRemoved is forwarded straight to index.Evict per hash, so the
first of these duplicate removes evicts a block the sibling chunk still
references — premature eviction that under-credits the lower tier in routing.
There is currently no de-duplication anywhere in pkg/kvevents. This mirrors the
role of Dynamo's EventDedupFilter (ai-dynamo/dynamo#8012), which llm-d has no
equivalent for.

How

A small filter that mirrors the wire event stream (not index state):

• every BlockStored increments the count for its hashes (duplicates included),
  after the store has been applied to the index;
• a BlockRemoved is forwarded to index.Evict only once its count returns to
  zero; removes for never-seen hashes pass through defensively (an unknown evict
  is a no-op) and a count never goes negative;
• AllBlocksCleared resets a pod's counts in lockstep with the index's pod-wide
  clear.

The reference-count scope is (podIdentifier, deviceTier, groupIdx), matching the
dimensions of the PodEntry identity that an eviction actually targets, so
gpu/cpu copies and distinct KV-cache groups of the same hash are counted
independently.

Data-parallel rank

The scope carries a dataParallelRank field that is fixed to a sentinel on
main, because the current index identity (PodEntry) is pod-level and does
not distinguish data-parallel ranks — so reference counts correctly aggregate
across ranks (a block is still resident on the pod until every rank has removed
it). This is intentionally not a change to DP semantics. Once a DP-aware
index lands (e.g. #370, which adds DataParallelRank to PodEntry and
EventBatch), the rank can be sourced from the event at the call site so the
dedup scope becomes DP-aware in lockstep, with no change to the filter itself (a
TODO(#370) marks the single line).

Scope of this change

• No changes to EventBatch or the engine adapters — DP-rank propagation is left
  to Add DP-aware routing support to KVEvents and indexing pipeline #370, which owns it.
• Behavior is unchanged for the non-duplicate case (a single store + single
  remove forwards exactly as before).

Observability

Because a suppressed BlockRemoved is otherwise invisible, two counters are added
to pkg/kvcache/metrics following the existing global-collector pattern (counts
are at block-hash granularity, not BlockRemoved events):

• kvcache_kvevents_dedup_removed_hashes_suppressed_total
• kvcache_kvevents_dedup_removed_hashes_forwarded_total

A TRACE log is also emitted when removals are suppressed. A gauge for the
outstanding per-pod reference-count map size is a planned follow-up.

Testing

go build ./...
go vet ./pkg/kvevents/...
golangci-lint run ./pkg/kvevents/...                          # 0 issues
go test -race ./pkg/kvevents/... ./pkg/kvcache/kvblock/...     # all pass

New tests in pkg/kvevents/event_dedup_filter_test.go:

• Filter unit tests: duplicate store → first remove suppressed / second forwards;
  aggregation across sources; gpu/cpu tier independence; KV-cache group
  independence; defensive unknown-remove pass-through; partial forward; clear
  resets and isolates pods; nil-safe.
• Pool-level (pool_test.go, processEventBatch + real InMemoryIndex):
  duplicate store survives the first remove and is evicted by the second;
  CPU/offload duplicate removal through the empty-token device-tier path (the
  GPU copy stays untouched); a device-tier update that resolves no keys is not
  reference-counted; AllBlocksCleared resets the filter.

AI assistance

Developed with AI assistance. The submitter has reviewed every changed line, ran
the tests, and owns the change.

[Copilot]

Pull request overview

This PR adds a small reference-counting de-duplication layer to KV event processing so that duplicate BlockRemoved events (legitimately produced by vLLM chunk-mode offloading for overlapping chunks) do not prematurely evict blocks that are still referenced by another outstanding announcement.

Changes:

• Introduces an eventDedupFilter that reference-counts (pod, deviceTier, groupIdx, dpRank-sentinel, blockHash) across BlockStored / BlockRemoved.
• Wires the filter into Pool.processEventBatch: stores increment after successful index add; removes are filtered so eviction is forwarded only when the refcount reaches zero; AllBlocksCleared resets counts for the pod.
• Adds focused unit + pool-level integration tests covering duplicate store/remove behavior, tier/group isolation, defensive unknown removes, and clear/reset behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
pkg/kvevents/pool.go	Normalizes device tier consistently, tracks stores/removes through the new dedup filter, and clears filter state on AllBlocksCleared.
pkg/kvevents/event_dedup_filter.go	Implements the reference-counting filter keyed by pod/tier/group/dp-rank-sentinel/hash with defensive pass-through behavior.
pkg/kvevents/event_dedup_filter_test.go	Adds unit + end-to-end tests validating duplicate-remove suppression, isolation across tiers/groups, and clear/reset behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yankay]

Thanks, this looks consistent with Dynamo’s dedup behavior and the current llm-d index semantics. The tier/group/offload coverage looks good to me.

/lgtm

[vMaroon]

/approve