Skip to content

feat(forge): add argument to set iteration of fuzz run #13140

@avniculae

Description

@avniculae

Component

Forge

Describe the feature you would like

In scenarios where fuzz runs are relying on vm.random* function instead of input arguments, it would be useful to:

  1. Log the rng seed for the failed testcase -- this is useful in case seed is not set
  2. Be able to pass an additional forge cli argument to set the fuzz run
  3. Maybe save seed and fuzz run iteration in cache to be abel to quiclly reproduce it with forge t --rerun

This is useful in situations where fuzzing campaigns take longer, and counterexamples are only found after long periods of time. Otherwise, debugging such tests is quite difficult. For example, test below fails after a few thousands runs if seed is not set, and fails after exactly 5932 runs if seed is set to 0x1111. POC provided runs fast, but in case testcase is complex, waiting times can be > 3-5 minutes, making debugging difficult.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.28;

import {Test} from 'forge-std/Test.sol';

contract RunsTest is Test {
  function test_fuzz(uint256) public view {
    uint256 ratioBps = bound(vm.randomUint(), 100_00, 200_00);
    uint256 amountA = bound(vm.randomUint(), 0, 1e30);
    uint256 amountB = amountA * ratioBps / 100_00;
    uint256 effectiveRatioBps = effectiveRatioBps(amountA, amountB);
    uint256 effectiveRatioWad = effectiveRatioWad(amountA, amountB);
    assertEq(effectiveRatioBps < 111_11, effectiveRatioWad < 1.111099999999999999e18);
  }

  function effectiveRatioBps(uint256 amountA, uint256 amountB) internal pure returns (uint256) {
    return amountB * 100_00 / amountA;
  }

  function effectiveRatioWad(uint256 amountA, uint256 amountB) internal pure returns (uint256) {
    return amountB * 1e18 / amountA;
  }
}

Additional context

At the moment I am overcoming this by

  • setting a rng seed
  • waiting for test to revert
  • read the number of runs logged by forge test
  • populate (runs-1) here
  • use locally-built forge for debug purposes

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions