Skip to content

Commit

Permalink
kvs-watch: only fetch new data for appends
Browse files Browse the repository at this point in the history
Problem: The FLUX_KVS_WATCH_APPEND flag is implemented inefficiently
in kvs-watch.  Every time new data is appended, the entire contents
of the watched key are retrieved and only new data calculated via
an offset is sent to the watcher.  This significantly slows down performance
of large appended data (such as stdio).

Solution: Instead of retrieving the entire contents of the watched key,
fetch the tree object for the key from the KVS.  With access to the tree
object's blobref array, we need only access the new appended blobrefs from
the content store.  This significantly cuts down on the amount of data
transfered during a kvs-watch.

Fixes flux-framework#6414
  • Loading branch information
chu11 committed Nov 21, 2024
1 parent 469fc14 commit f1a8a62
Showing 1 changed file with 271 additions and 65 deletions.
Loading

0 comments on commit f1a8a62

Please sign in to comment.