Commit f1a8a62
committed
kvs-watch: only fetch new data for appends
Problem: The FLUX_KVS_WATCH_APPEND flag is implemented inefficiently
in kvs-watch. Every time new data is appended, the entire contents
of the watched key are retrieved and only new data calculated via
an offset is sent to the watcher. This significantly slows down performance
of large appended data (such as stdio).
Solution: Instead of retrieving the entire contents of the watched key,
fetch the tree object for the key from the KVS. With access to the tree
object's blobref array, we need only access the new appended blobrefs from
the content store. This significantly cuts down on the amount of data
transfered during a kvs-watch.
Fixes flux-framework#64141 parent 469fc14 commit f1a8a62
1 file changed
Lines changed: 271 additions & 65 deletions
0 commit comments