Optimize UTXO handling with in-memory caching #417
Conversation
- Added `utxo_cache` in `WalletStore` to store UTXOs from `listunspent` RPC calls, reducing redundant network requests. - Implemented `update_utxo_cache` to update and maintain the cache efficiently using `RwLock`. - Overrode `PartialEq` for `WalletStore` since `RwLock` does not implement it, ensuring correct equality checks. - Updated `get_all_utxo` to leverage caching, reducing unnecessary RPC calls while keeping UTXO data fresh. This improves wallet performance and reduces RPC overhead. 🚀
|
Fixes #143 |
|
Currently, the UTXO state is stored in Should we add a periodic refresh mechanism to ensure the UTXO cache stays updated? |
The utxo cache should update with every sync call. The makerd syncs in loop every 15 mins. Also after each spend tx, and coinswap it syncs. So it should be updated as long as we update the cache at syncs. Add a sync call at |
There was a problem hiding this comment.
Approach looks good. Some things needs to be changed.
- Don't use RwLock, not needed.
- Update the cache at
wallet::synccalls. - The cache should include
<ListUnspentResultEntry, UTXOSpendInfo>. Checkj thelist_all_utxo_spend_infofunction for how to generate the spend infos from the utxos. - Add a sync call at maker's
SendToAddresandRedeemFidelityrpc handlers.
src/wallet/api.rs
Outdated
| let cache = self.store.utxo_cache.read().unwrap(); | ||
| if !cache.is_empty() { | ||
| log::info!("Using cached UTXO data."); | ||
| return Ok(cache.values().cloned().collect()); | ||
| } | ||
| } // Read lock is dropped here to allow a write lock later |
There was a problem hiding this comment.
We don't need to check, if its empty it will return empty vec, which is fine.
src/wallet/api.rs
Outdated
| self.rpc.unlock_unspent_all()?; | ||
| let all_utxos = self | ||
| .rpc | ||
| .list_unspent(Some(0), Some(9999999), None, None, None)?; | ||
|
|
||
| // Acquire write lock to update the cache | ||
| self.store.update_utxo_cache(all_utxos.clone()); | ||
|
|
There was a problem hiding this comment.
Update the store in wallet::sync() function, not here.
src/wallet/storage.rs
Outdated
| @@ -40,6 +42,9 @@ pub(crate) struct WalletStore { | |||
| pub(super) last_synced_height: Option<u64>, | |||
|
|
|||
| pub(super) wallet_birthday: Option<u64>, | |||
|
|
|||
| #[serde(default)] // Ensures deserialization works if `utxo_cache` is missing | |||
| pub(super) utxo_cache: RwLock<HashMap<OutPoint, ListUnspentResultEntry>> | |||
There was a problem hiding this comment.
We don't need a RwLock here, as the wallet is already wrapped in it at maker's multithreaded code. So just simple hashmap will work.
There was a problem hiding this comment.
yeah makes sense, since only one makerd can run from a single wallet.
- Added `utxo_cache` in `WalletStore` as HashMap<OutPoint, (ListUnspentResultEntry, UTXOSpendInfo)> - `update_utxo_cache_internal` fn for updating the cache - `build_utxo_list_from_cache` to build original list of utxo from cache(mathches the output format for rpc call `list_unspent`) - `refresh_utxo_cache` added inside `wallet/api.rs` to refresh the cache with wallet's internal spend info - added wallet sync in `SendToAddress` and `ReedemFidelity`.
- Added `utxo_cache` in `WalletStore` as HashMap<OutPoint, (ListUnspentResultEntry, UTXOSpendInfo)> - `update_utxo_cache_internal` fn for updating the cache - `build_utxo_list_from_cache` to build original list of utxo from cache(mathches the output format for rpc call `list_unspent`) - `refresh_utxo_cache` added inside `wallet/api.rs` to refresh the cache with wallet's internal spend info - added wallet sync in `SendToAddress` and `ReedemFidelity`.
yeah having the cache updates inside the wallet sync call keeps the utxo list updated |
|
utxo_cacheinWalletStoreto store UTXOs fromlistunspentRPC calls, reducing redundant network requests.update_utxo_cacheto update and maintain the cache efficiently usingRwLock.PartialEqforWalletStoresinceRwLockdoes not implement it, ensuring correct equality checks.get_all_utxoto leverage caching, reducing unnecessary RPC calls while keeping UTXO data fresh.