Add prefetch trie nodes #1871
Add prefetch trie nodes #1871
Conversation
core/txpool/legacypool/legacypool.go
Outdated
| header := pool.currentHead.Load() | ||
| pool.mu.RUnlock() | ||
|
|
||
| if header == nil || len(txs) == 0 { |
There was a problem hiding this comment.
Shall we move the txs len check on top of the function?
Just an optimization to reduce lock contention. Since txs is an input and doesn’t depend on pool state, you can fast-return before touching any shared data.
core/blockchain.go
Outdated
| storageCacheHitMeter = metrics.NewRegisteredMeter("chain/storage/reads/cache/process/hit", nil) | ||
| storageCacheMissMeter = metrics.NewRegisteredMeter("chain/storage/reads/cache/process/miss", nil) | ||
|
|
||
| accountCacheHitPrefetchMeter = metrics.NewRegisteredMeter("chain/account/reads/cache/prefetch/hit", nil) |
There was a problem hiding this comment.
Shall we remove all the unused metrics here? Lint's complaining
| defer cancel() | ||
|
|
||
| if followupInterrupt == nil { | ||
| followupInterrupt = &atomic.Bool{} |
There was a problem hiding this comment.
Not sure. I didn't make this change.
| vmCfg := bc.cfg.VmConfig | ||
| vmCfg.Tracer = nil | ||
| bc.prefetcher.Prefetch(block, throwaway, vmCfg, followupInterrupt) | ||
| // go func(start time.Time, throwaway *state.StateDB, block *types.Block) { |
There was a problem hiding this comment.
Is this worth deleting?
Or - alternatively - have a comment to explain why it's disabled
There was a problem hiding this comment.
I commented it out so that we don't run multiple prefetches together. I'll probably remove this once we finalize all the optimizations.
| return nil, 0 | ||
| } | ||
| vmCfg := bc.cfg.VmConfig | ||
| vmCfg.Tracer = nil |
There was a problem hiding this comment.
why is this required? Same for PrefetchFromTxpool
| // Append cache hit/miss percentages | ||
| if bc != nil { | ||
| if total := bc.lastProcAccHit + bc.lastProcAccMiss; total > 0 { | ||
| pct := int64(100 * bc.lastProcAccHit / total) |
There was a problem hiding this comment.
seems like this (and others in the same func) conversion is pleonastic
There was a problem hiding this comment.
Yeah right. I don't think these will make it to develop anyway.
|
|
||
| // Note: Change the go image version in Dockerfile if you change this. | ||
| go 1.24.6 | ||
| go 1.24.8 |
There was a problem hiding this comment.
worth updating in the CI yml fles as well.
Also, since we're bumping, maybe we can use 1.24.9 to solve the govulncheck issues (I have that on my upstream PR as well)
There was a problem hiding this comment.
Could do yeah but anyway this isn't getting merged to develop just yet.
core/analyzer.sh
Outdated
| @@ -0,0 +1,102 @@ | |||
| # ./log_analyzer.sh "3 days ago" # Scans the last 3 days | |||
There was a problem hiding this comment.
This should be excluded from this PR I guess?
core/state_prefetcher.go
Outdated
| // Aggregate touched pre-state from stateCpy and populate the process reader cache directly | ||
| acc, stor := stateCpy.PrefetchTouched() | ||
| for a := range acc { | ||
| reader.Account(a) | ||
| } | ||
| for a, bucket := range stor { | ||
| for k := range bucket { | ||
| reader.Storage(a, k) | ||
| } | ||
| } |
There was a problem hiding this comment.
What's the rationale of this change? IntermediateRoot also fetches touched account/storages.
There was a problem hiding this comment.
Yes but it also calculates state root which we strictly don't need. Pre-execution should fill up the accounts/storages. Do you think it still should be there ?
There was a problem hiding this comment.
It should be there. Simply loading account/storage isn't sufficient, because in order to calculate the final state root hash, some sibling nodes will need to be retrieved, even though they aren't related to touched accounts.
core/blockchain.go
Outdated
| vmCfg.Tracer = nil | ||
| synthetic := types.NewBlockWithHeader(header).WithBody(types.Body{Transactions: txs}) | ||
| var noop atomic.Bool | ||
| bc.prefetcher.Prefetch(synthetic, warmdb, vmCfg, &noop) |
There was a problem hiding this comment.
This call execute transactions without taking fees from transactions, which introduces a DoS vector. Malicious users can submit resource-expensive transaction and replace them with a bit higher gas continuously without paying the gas fee.
For example, if the current base gas price is 50Gwei, I can submit a transaction with 32M gas of SLOAD, with a initial gas price of 1Gwei, and continuously replace the existing tx by bumping up the gas by 10% for 41 times before reaching the 50Gwei base price (1Gwei * 1.1^41 = 49Gwei). This will result in the node to fully execute this transaction 41 times before the transaction is actually included in a block.
There was a problem hiding this comment.
Good catch. Though I am planning to remove PrefetchFromTxpool altogether as I didn't see any impact of prefetching from txpool. Perhaps running it in worker (which is present currently) is more deterministic ?
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## src-optimizations #1871 +/- ##
====================================================
Coverage ? 47.92%
====================================================
Files ? 844
Lines ? 143880
Branches ? 0
====================================================
Hits ? 68959
Misses ? 70402
Partials ? 4519 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Description
This PR enables prefetching of trie nodes before execution of block. It adds a
waitforwarmconfig which if enabled will block execution of block until the warming finishes.Changes
Checklist
Testing