-
Notifications
You must be signed in to change notification settings - Fork 927
Description
Description
Lighthouse presently prunes all knowledge of non-canonical blocks, from disk and fork choice, once they conflict with finalization.
The pruning is not always immediate, fork choice currently prunes once the number of fork choice nodes reaches a threshold of 256. However, @dapplion is keen to remove this and just prune every finalization (I agree this is simpler).
pub const DEFAULT_PRUNE_THRESHOLD: usize = 256; |
lighthouse/consensus/proto_array/src/proto_array.rs
Lines 681 to 693 in e5b4983
/// Update the tree with new finalization information. The tree is only actually pruned if both | |
/// of the two following criteria are met: | |
/// | |
/// - The supplied finalized epoch and root are different to the current values. | |
/// - The number of nodes in `self` is at least `self.prune_threshold`. | |
/// | |
/// # Errors | |
/// | |
/// Returns errors if: | |
/// | |
/// - The finalized epoch is less than the current one. | |
/// - The finalized epoch is equal to the current one, but the finalized root is different. | |
/// - There is some internal error relating to invalid indices inside `self`. |
The database is pruned more consistently with the default configuration. Every block that does not descend from the finalized block is deleted:
lighthouse/beacon_node/beacon_chain/src/migrate.rs
Lines 701 to 727 in e5b4983
// Blocks both finalized and unfinalized are in the same DB column. We must only | |
// prune blocks from abandoned forks. Note that block pruning and state pruning differ. | |
// The blocks DB column is shared for hot and cold data, while the states have different | |
// columns. Thus, we only prune unviable blocks or from abandoned forks. | |
let should_prune = if finalized_and_descendant_block_roots_of_finalized_checkpoint | |
.contains(&block_root) | |
{ | |
// Keep unfinalized blocks descendant of finalized checkpoint + finalized block | |
// itself Note that we anchor this set on the finalized checkpoint instead of the | |
// finalized block. A diagram above shows a relevant example. | |
false | |
} else if newly_finalized_blocks.contains(&(block_root, slot)) { | |
// Keep recently finalized blocks | |
false | |
} else if slot < newly_finalized_states_min_slot { | |
// Keep recently finalized blocks that we know are canonical. Blocks with slots < | |
// that `newly_finalized_blocks_min_slot` we don't have canonical information so we | |
// assume they are part of the finalized pruned chain | |
// | |
// Pruning these would risk breaking the DB by deleting canonical blocks once the | |
// HDiff grid advances. If the pruning routine is correct this condition should | |
// never be hit. | |
false | |
} else { | |
// Everything else, prune | |
true | |
}; |
Steps to resolve
It would be nice to develop a simple (easy to reason about) system for handling messages relating to blocks that are non-canonical. We already have a system for handling old canonical blocks, which we could consider combining with the non-canonical block tracker, if that would be simple. That system for pre-finalization blocks lives here:
One option would be to write all non-canonical block roots to their own column on disk. Then we could check whether a block root is non-canonical with a single DB lookup. We could rely on the DB's cache, or add our own in-memory cache in front of this (the DB cache should be assumed sufficient unless benchmarks show otherwise).
Related Issues
We had a bug related to this very recently: