statement-store: make encode/hash faster #10882
Open
+101
−36
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
P1sar
approved these changes
Jan 22, 2026
alexggh
added
the
T8-polkadot
AndreiEres
approved these changes
Jan 23, 2026
Contributor
AndreiEres
left a comment
AndreiEres
left a comment
There was a problem hiding this comment.
Should we reduce calls of hash() then and pass it into arguments where it's possible?
| queue_sender, | ||
| metrics: None, | ||
| initial_sync_timeout: Box::pin(tokio::time::sleep(INITIAL_SYNC_BURST_INTERVAL).fuse()), | ||
| initial_sync_timeout: Box::pin(pending().fuse()), |
Contributor
Author
There was a problem hiding this comment.
benchmark was crashing because tokio time wants a tokio runtime.
AndreiEres
reviewed
Jan 23, 2026
| self.num_topics as u32; | ||
|
|
||
| let mut output = Vec::new(); | ||
| // Calculate capacity for preallocation using size_of for type sizes: |
Contributor
There was a problem hiding this comment.
Rust size or SCALE-encoded size?
Contributor
Author
There was a problem hiding this comment.
the SCALE-encoded is what we want and this is a close approximation without actually scale encoding.
Contributor
There was a problem hiding this comment.
Could you put it in a comment please?
Co-authored-by: Andrei Eres <eresav@me.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By reserving the memory in advance we halve the encoding speed which ultimately speeds up the statement.hash() function which gets called in a lot of places.
More importantly, when we start being connected to more nodes the hash function gets called a lot for the same statement because we might receive the same statement from all peers we are connected to.
For example on versi on_statements ate a lot of time when running with 15 nodes, see #10814 (comment).
Modified the statement_network benchmark to also be parameterizable by the number of times we might receive a statement and if we receive it from 16 peers, we notice a speed up with this PR of ~16%, which I consider not negligible, so I consider this an worthy improvement.