Summary
sanity/slots/pyspec_tests/historical_accumulator takes ~23-24s for gloas and heze on
the mainnet preset, vs ~1-6s for every pre-gloas fork. The sanity slots timeout is
30000ms (packages/beacon-node/test/spec/presets/sanity.test.ts:49), so these two cases
run at 76-81% of the budget and are a timeout-flake risk on slower CI runners.
Measurements
mainnet preset, sanity/slots/pyspec_tests/historical_accumulator, local run @ v1.7.0-alpha.13:
| fork |
duration |
| phase0 |
1003ms |
| bellatrix |
2241ms |
| deneb |
2598ms |
| electra |
3032ms |
| fulu |
3142ms |
| altair |
5003ms |
| capella |
5847ms |
| gloas |
22948ms |
| heze |
24234ms |
gloas is 7.3x fulu; heze is 7.7x fulu. heze is consistently ~1.3s above gloas.
On the minimal preset all forks finish in <=27ms (gloas 26ms, heze 27ms), so this is
mainnet-only.
Why mainnet only
The test runs process_slots(state, state.slot + SLOTS_PER_HISTORICAL_ROOT) — 8192 slot
transitions on mainnet vs 64 on minimal.
Likely cause
Two gloas-specific changes make each of those 8192 slot transitions more expensive:
packages/state-transition/src/slot/index.ts:34-40 — every post-gloas slot writes into
executionPayloadAvailability, a BitVector[SLOTS_PER_HISTORICAL_ROOT] (8192 bits on
mainnet):
if (fork >= ForkSeq.gloas) {
// Unset the next payload availability
(state as CachedBeaconStateGloas).executionPayloadAvailability.set(
(state.slot + 1) % SLOTS_PER_HISTORICAL_ROOT,
false
);
}
That is 8192 ViewDU writes into a large bitvector, each dirtying nodes that must be
re-merkleized on commit.
- The gloas BeaconState is a ProgressiveContainer (EIP-7688), which has a different
merkleization cost profile than the fixed-depth containers used pre-gloas. heze widens
it further, consistent with heze being slower than gloas.
Not yet profiled — the split between (1) and (2) is unknown.
Impact
- Timeout-flake risk: 76% (gloas) / 81% (heze) of the 30s budget.
- Inflates total mainnet spec-test wall clock.
- Any future fork inherits the regression.
Reproduce
pnpm vitest run --project spec-mainnet
packages/beacon-node/test/spec/presets/sanity.test.ts -t historical_accumulator
Proposed short term
Skip the two cases with a pointer to this issue, rather than raising the timeout —
raising it hides the regression. See #9763 discussion.
Investigation ideas
- Profile to attribute cost between the bitvector write and progressive-container hashing.
- Consider deferring/batching the executionPayloadAvailability reset so process_slots
does not commit a large bitvector every slot.
- Compare against a fixed-depth BitVector control to isolate the EIP-7688 contribution.
Summary
sanity/slots/pyspec_tests/historical_accumulatortakes ~23-24s for gloas and heze onthe mainnet preset, vs ~1-6s for every pre-gloas fork. The sanity
slotstimeout is30000ms (
packages/beacon-node/test/spec/presets/sanity.test.ts:49), so these two casesrun at 76-81% of the budget and are a timeout-flake risk on slower CI runners.
Measurements
mainnet preset,
sanity/slots/pyspec_tests/historical_accumulator, local run @ v1.7.0-alpha.13:gloas is 7.3x fulu; heze is 7.7x fulu. heze is consistently ~1.3s above gloas.
On the minimal preset all forks finish in <=27ms (gloas 26ms, heze 27ms), so this is
mainnet-only.
Why mainnet only
The test runs
process_slots(state, state.slot + SLOTS_PER_HISTORICAL_ROOT)— 8192 slottransitions on mainnet vs 64 on minimal.
Likely cause
Two gloas-specific changes make each of those 8192 slot transitions more expensive:
packages/state-transition/src/slot/index.ts:34-40— every post-gloas slot writes intoexecutionPayloadAvailability, aBitVector[SLOTS_PER_HISTORICAL_ROOT](8192 bits onmainnet):
merkleization cost profile than the fixed-depth containers used pre-gloas. heze widens
it further, consistent with heze being slower than gloas.
Not yet profiled — the split between (1) and (2) is unknown.
Impact
Reproduce
pnpm vitest run --project spec-mainnet
packages/beacon-node/test/spec/presets/sanity.test.ts -t historical_accumulator
Proposed short term
Skip the two cases with a pointer to this issue, rather than raising the timeout —
raising it hides the regression. See #9763 discussion.
Investigation ideas
does not commit a large bitvector every slot.