Skip to content

[fuzzer] Root-cause and document issue #383's "Key $N does not exists in frame." finding#389

Open
PhilippGrulich wants to merge 1 commit into
mainfrom
claude/github-issue-383-jbwurh
Open

[fuzzer] Root-cause and document issue #383's "Key $N does not exists in frame." finding#389
PhilippGrulich wants to merge 1 commit into
mainfrom
claude/github-issue-383-jbwurh

Conversation

@PhilippGrulich

@PhilippGrulich PhilippGrulich commented Jul 9, 2026

Copy link
Copy Markdown
Member

Summary

Issue #383 tracked a fuzzer finding — Frame::getValue throwing "Key $N does not exists in frame." — with a 2026-07-09 audit that couldn't reproduce it and left it tolerated-but-latent, hypothesizing it was the same class of merged-value register/variable-binding bug as #321/#322 (getResultRegister/bindResult), just in a Kind::If-nested-in-Kind::Call shape those fixes don't cover.

This PR corrects that diagnosis and confirms the finding is very much live:

  • Reproduced it with a wider (non-coverage-guided, ~2M-program) search across the bc/tbc/asmjit backends after building the fuzzer locally — the original PR-gate/weekly-sweep searches just hadn't drawn the right byte sequence.
  • Root-caused it, with an instrumented build of SSACreationPhase.cpp, to a bug one layer upstream of any lowering provider: Frame<K,V>, getResultRegister, bindResult, and the cpp backend's getVariable are all fine — every compiling backend (cpp, bc, tbc, asmjit) and every IR-pass-permutation config throws identically for the same minimized input, which is exactly what you'd expect from a bug in the shared trace→SSA→IR pipeline that runs once before any backend-specific lowering.
  • Minimized the reproducer down to a Kind::Loop whose trip count traces to (but isn't literally) zero, with a Kind::Call in its body taking a Kind::LoopContinue argument — no Kind::If required at all: 
    loop(count=((0i8 + 0i8) + 0i8), init=0i8, body=sum3(0i8, cont(cond=0i8, value=0i8), 0i8))
    SSACreationPhaseContext::processBlock's worklist only walks backward from the recorded Return via Block::predecessors. With the trip count resolved to zero, the only recorded Return is reached through the loop's exit edge, never through the body — the tracer never recorded a back-edge closing the body block's speculative exploration (needed even at trip-count-zero, since the compiled function must stay correct for whatever trip count a later call computes) back to the loop's condition check. The body block is real (a genuine CMP successor with real operations) but orphaned from the backward walk, so its free values never get threaded as block arguments. TraceToIRConversionPhase visits it anyway (it walks every CMP/JMP successor unconditionally) and Frame::getValue throws.

This points the real bug at the tracer's loop/back-edge recording during speculative branch exploration — the same fragile machinery already implicated by two other tolerated findings (Tag/Snapshot collision, empty-return-list) — not at any lowering provider's bookkeeping. A full fix belongs there and is out of scope to attempt blind in this change, given how sensitive that code already is.

Changes

  • nautilus/test/fuzz/regressions/finding3-loop-call-continue.bin: a minimized, directly replayable 23-byte reproducer (nautilus-fuzz-replay test/fuzz/regressions/finding3-loop-call-continue.bin).
  • nautilus/test/fuzz/README.md: updates the "Known findings" add support for std lib #3 write-up with the corrected root cause, confirmed-live status, and a pointer to the pinned reproducer.
  • nautilus/test/fuzz/ReplayMain.cpp: updates the isKnownPreExistingFinding tolerance's doc comment to match (the tolerance itself is unchanged — this is confirmed real, not something to stop tolerating).

Test plan

  • Built nautilus-fuzz-replay locally (Clang 18, Release, MLIR disabled) and confirmed the pinned finding3-loop-call-continue.bin reproduces the exact "Key $35 does not exists in frame." exception on every compiling backend (cpp/bc/tbc/asmjit) and every IR-pass config permutation.
  • Ran the default 2000-input deterministic smoke corpus (nautilus-fuzz-replay) after the change — still passes, tolerating the same known findings as before.
  • clang-format-18 --dry-run --Werror clean on the modified ReplayMain.cpp.

🤖 Generated with Claude Code

Generated by Claude Code

@claude
Root-cause and document issue #383's Key$-missing fuzzer finding
cf7d4d4 
Confirmed the "Key $N does not exists in frame." finding is still live
(a wider search across ~2M generated programs reproduces it, contradicting
the 2026-07-09 non-reproduction audit), and corrected its root cause: it is
not a register/variable-binding bug in any lowering provider (Frame<K,V>,
getResultRegister, bindResult, the cpp backend's getVariable are all fine),
but a tracer bug in loop back-edge recording for a provably-zero-trip
Kind::Loop whose body contains a Kind::Call argument with a
Kind::LoopContinue -- one layer upstream of SSACreationPhase's
return-anchored backward walk, which never visits the loop body block
because it never reaches the recorded Return.

Pins a minimized, directly replayable 23-byte reproducer and updates
README.md's "Known findings" and the ReplayMain.cpp tolerance comment
with the corrected diagnosis so a future fix attempt starts from accurate
information instead of the original (incorrect) lowering-provider theory.
github-actions[bot]
github-actions Bot reviewed Jul 9, 2026
View reviewed changes

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracing Benchmark

Details
Benchmark suite Current: cf7d4d4 Previous: f41374a Ratio
tiered_twotier_addOne 3.89183 ms (± 276.824) 3.42561 ms (± 107.294) 1.14
tiered_singletier_addOne 3.79669 ms (± 312.734) 3.34111 ms (± 162.817) 1.14
single_compile_mlir_addOne 3.70734 ms (± 150.325) 3.27521 ms (± 105.829) 1.13
trace_add 2.81485 us (± 150.6) 2.59873 us (± 399.055) 1.08
completing_trace_add 2.8351 us (± 241.39) 2.58653 us (± 383.618) 1.10
trace_ifThenElse 9.52325 us (± 2.01718) 9.3891 us (± 1.23953) 1.01
completing_trace_ifThenElse 5.24392 us (± 485.512) 4.93568 us (± 642.891) 1.06
trace_deeplyNestedIfElse 27.4857 us (± 4.87133) 27.1045 us (± 2.97687) 1.01
completing_trace_deeplyNestedIfElse 14.1225 us (± 993.643) 13.5802 us (± 1.99339) 1.04
trace_loop 9.3308 us (± 1.53707) 9.40226 us (± 1.33047) 0.99
completing_trace_loop 5.42042 us (± 614.577) 5.02606 us (± 728.008) 1.08
trace_ifInsideLoop 17.9408 us (± 2.3845) 18.401 us (± 2.70805) 0.97
completing_trace_ifInsideLoop 9.62967 us (± 1.53735) 8.6913 us (± 1.09544) 1.11
trace_loopDirectCall 9.53691 us (± 1.61526) 9.37847 us (± 1.34723) 1.02
completing_trace_loopDirectCall 5.48664 us (± 710.041) 5.11186 us (± 645.682) 1.07
trace_pointerLoop 15.5888 us (± 990.133) 15.3655 us (± 3.39147) 1.01
completing_trace_pointerLoop 11.6708 us (± 1.79482) 15.3243 us (± 4.30168) 0.76
trace_staticLoop 8.44011 us (± 865.793) 8.01932 us (± 1.12244) 1.05
completing_trace_staticLoop 8.36197 us (± 624.982) 7.80501 us (± 781.477) 1.07
trace_fibonacci 11.0775 us (± 2.57814) 10.6667 us (± 1.65863) 1.04
completing_trace_fibonacci 6.87313 us (± 763.368) 6.44976 us (± 925.086) 1.07
trace_gcd 8.46128 us (± 984.721) 8.46586 us (± 1.23074) 1.00
completing_trace_gcd 4.55818 us (± 455.332) 4.29953 us (± 516.939) 1.06
trace_nestedIf10 45.6665 us (± 4.5113) 44.4188 us (± 6.58941) 1.03
completing_trace_nestedIf10 46.2687 us (± 5.25239) 45.4956 us (± 7.90785) 1.02
trace_nestedIf100 1.76613 ms (± 26.1494) 1.80642 ms (± 35.4136) 0.98
completing_trace_nestedIf100 1.78251 ms (± 31.1246) 1.82227 ms (± 58.1152) 0.98
trace_chainedIf10 101.602 us (± 4.13656) 106.176 us (± 8.27667) 0.96
completing_trace_chainedIf10 50.6779 us (± 5.08057) 49.7741 us (± 7.91397) 1.02
trace_chainedIf100 4.474 ms (± 31.6044) 4.93623 ms (± 205.022) 0.91
completing_trace_chainedIf100 2.16515 ms (± 28.8402) 2.21114 ms (± 44.4422) 0.98
e2e_tiered_bc_to_mlir 3778.37 us (± 193098) 3463.76 us (± 199089) 1.09
e2e_single_mlir 6.32515 ms (± 187.713) 5.74879 ms (± 432.889) 1.10
comp_mlir_add 6.32798 ms (± 191.417) 5.896 ms (± 346.457) 1.07
comp_mlir_ifThenElse 6.92117 ms (± 179.531) 6.30767 ms (± 363.573) 1.10
comp_mlir_deeplyNestedIfElse 5.81937 ms (± 116.705) 5.13365 ms (± 140.976) 1.13
comp_mlir_loop 7.7212 ms (± 292.219) 7.35789 ms (± 336.052) 1.05
comp_mlir_ifInsideLoop 27.9926 ms (± 438.343) 30.2558 ms (± 1.18806) 0.93
comp_mlir_loopDirectCall 12.1294 ms (± 361.183) 13.0919 ms (± 1.20147) 0.93
comp_mlir_pointerLoop 27.0311 ms (± 561.337) 28.9496 ms (± 792.212) 0.93
comp_mlir_staticLoop 5.68496 ms (± 148.241) 5.14729 ms (± 237.354) 1.10
comp_mlir_fibonacci 10.9647 ms (± 396.875) 11.0728 ms (± 508.797) 0.99
comp_mlir_gcd 9.81871 ms (± 195.676) 10.0482 ms (± 497.236) 0.98
comp_mlir_nestedIf10 11.0165 ms (± 263.134) 10.951 ms (± 319.645) 1.01
comp_mlir_nestedIf100 25.6075 ms (± 441.476) 26.0312 ms (± 553.3) 0.98
comp_mlir_chainedIf10 11.4734 ms (± 311.424) 10.1778 ms (± 856.013) 1.13
comp_mlir_chainedIf100 60.1631 ms (± 417.784) 21.6828 ms (± 604.969) 2.77
comp_cpp_add 21.1942 ms (± 428.597)
comp_cpp_ifThenElse 21.7004 ms (± 423.434)
comp_cpp_deeplyNestedIfElse 22.5909 ms (± 513.634)
comp_cpp_loop 21.8306 ms (± 531.664)
comp_cpp_ifInsideLoop 22.7895 ms (± 551.669)
comp_cpp_loopDirectCall 21.7518 ms (± 556.929)
comp_cpp_pointerLoop 22.3557 ms (± 501.728)
comp_cpp_staticLoop 21.6737 ms (± 508.104)
comp_cpp_fibonacci 21.9258 ms (± 621.484)
comp_cpp_gcd 21.5256 ms (± 545.455)
comp_cpp_nestedIf10 24.6523 ms (± 547.069)
comp_cpp_nestedIf100 57.4411 ms (± 644.884)
comp_cpp_chainedIf10 27.3166 ms (± 637.859)
comp_cpp_chainedIf100 87.2085 ms (± 699.943)
comp_bc_add 11.9114 us (± 2.27281)
comp_bc_ifThenElse 14.5934 us (± 2.69072)
comp_bc_deeplyNestedIfElse 21.8047 us (± 4.66943)
comp_bc_loop 14.6017 us (± 3.22126)
comp_bc_ifInsideLoop 18.1975 us (± 3.32281)
comp_bc_loopDirectCall 14.7646 us (± 2.37848)
comp_bc_pointerLoop 16.2022 us (± 2.56188)
comp_bc_staticLoop 14.0048 us (± 2.75048)
comp_bc_fibonacci 14.7556 us (± 2.17538)
comp_bc_gcd 14.3323 us (± 3.20531)
comp_bc_nestedIf10 41.1725 us (± 5.8168)
comp_bc_nestedIf100 286.664 us (± 9.09791)
comp_bc_chainedIf10 53.8813 us (± 6.71512)
comp_bc_chainedIf100 491.464 us (± 10.0852)
comp_tbc_add 1.98026 us (± 128.571)
comp_tbc_ifThenElse 4.41192 us (± 292.236)
comp_tbc_deeplyNestedIfElse 10.1268 us (± 705.735)
comp_tbc_loop 4.41564 us (± 347.143)
comp_tbc_ifInsideLoop 7.98552 us (± 1.23757)
comp_tbc_loopDirectCall 4.69987 us (± 414.141)
comp_tbc_pointerLoop 5.98067 us (± 629.773)
comp_tbc_staticLoop 5.20299 us (± 391.972)
comp_tbc_fibonacci 4.82226 us (± 271.558)
comp_tbc_gcd 3.89393 us (± 234.424)
comp_tbc_nestedIf10 30.8008 us (± 2.23916)
comp_tbc_nestedIf100 297.874 us (± 7.19124)
comp_tbc_chainedIf10 39.7416 us (± 3.61995)
comp_tbc_chainedIf100 420.702 us (± 10.5901)
comp_asmjit_add 18.6179 us (± 3.0711)
comp_asmjit_ifThenElse 26.1585 us (± 7.07955)
comp_asmjit_deeplyNestedIfElse 36.6491 us (± 7.11392)
comp_asmjit_loop 26.3432 us (± 7.09873)
comp_asmjit_ifInsideLoop 34.2837 us (± 6.95361)
comp_asmjit_loopDirectCall 29.4217 us (± 6.96811)
comp_asmjit_pointerLoop 33.7815 us (± 7.68617)
comp_asmjit_staticLoop 24.1445 us (± 6.58516)
comp_asmjit_fibonacci 29.2874 us (± 6.90155)
comp_asmjit_gcd 27.7035 us (± 6.88498)
comp_asmjit_nestedIf10 62.8205 us (± 8.74549)
comp_asmjit_nestedIf100 498.831 us (± 14.7417)
comp_asmjit_chainedIf10 79.4392 us (± 9.8348)
comp_asmjit_chainedIf100 640.389 us (± 24.1573)
exec_mlir_add 12.3329 ns (± 1.01969) 10.7864 ns (± 1.33064) 1.14
exec_mlir_fibonacci 26.3538 us (± 2.70548) 13.0123 us (± 1.86314) 2.03
exec_mlir_sum 658.302 us (± 37.1366) 550.158 us (± 69.58) 1.20
exec_cpp_add 5.29222 ns (± 0.673361) 4.6233 ns (± 0.526639) 1.14
exec_cpp_fibonacci 76.2491 us (± 3.53284) 48.1806 us (± 4.0411) 1.58
exec_cpp_sum 16.5118 ms (± 103.811) 11.4272 ms (± 257.321) 1.44
exec_bc_add 38.9839 ns (± 2.43671) 48.3058 ns (± 6.38573) 0.81
exec_bc_fibonacci 338.372 us (± 8.21211) 441.338 us (± 53.589) 0.77
exec_bc_sum 72.0934 ms (± 2.12303) 98.0111 ms (± 2.56027) 0.74
exec_tbc_add 26.5884 ns (± 1.7461) 27.5168 ns (± 3.91019) 0.97
exec_tbc_fibonacci 158.86 us (± 5.22091) 142.737 us (± 8.24385) 1.11
exec_tbc_sum 42.985 ms (± 132.061) 36.538 ms (± 42.7287) 1.18
exec_asmjit_add 2.99333 ns (± 0.378735) 3.49501 ns (± 0.262662) 0.86
exec_asmjit_fibonacci 13.7725 us (± 1.59431) 14.226 us (± 1.04001) 0.97
exec_asmjit_sum 3.76088 ms (± 18.9826) 2.8109 ms (± 60.563) 1.34
exec_bc_add_passesOff 39.4864 ns (± 5.54265) 47.9748 ns (± 6.08804) 0.82
exec_bc_add_passesOn 39.0699 ns (± 3.0346) 48.9161 ns (± 7.81767) 0.80
exec_bc_fibonacci_passesOff 339.709 us (± 10.9605) 441.259 us (± 53.0676) 0.77
exec_bc_fibonacci_passesOn 309.889 us (± 13.0826) 391.024 us (± 34.5844) 0.79
exec_bc_sum_passesOff 72.2006 ms (± 2.5732) 97.8564 ms (± 1.5048) 0.74
exec_bc_sum_passesOn 62.968 ms (± 2.61131) 82.1116 ms (± 661.324) 0.77
exec_tbc_add_passesOff 26.3215 ns (± 2.08497) 27.6939 ns (± 4.71754) 0.95
exec_tbc_add_passesOn 26.89 ns (± 2.25725) 27.5584 ns (± 4.32763) 0.98
exec_tbc_fibonacci_passesOff 158.175 us (± 3.52649) 142.074 us (± 7.24783) 1.11
exec_tbc_fibonacci_passesOn 158.722 us (± 3.87055) 142.502 us (± 6.73604) 1.11
exec_tbc_sum_passesOff 42.999 ms (± 141.737) 36.5595 ms (± 187.939) 1.18
exec_tbc_sum_passesOn 35.4345 ms (± 167.331) 33.9826 ms (± 58.5671) 1.04
exec_asmjit_add_passesOff 2.93487 ns (± 0.134747) 3.55598 ns (± 0.456682) 0.83
exec_asmjit_add_passesOn 2.99685 ns (± 0.300825) 3.5902 ns (± 0.767078) 0.83
exec_asmjit_fibonacci_passesOff 13.7307 us (± 1.6891) 14.829 us (± 3.08758) 0.93
exec_asmjit_fibonacci_passesOn 13.7308 us (± 1.65186) 14.5345 us (± 2.25004) 0.94
exec_asmjit_sum_passesOff 3.75739 ms (± 12.5755) 2.8089 ms (± 34.4276) 1.34
exec_asmjit_sum_passesOn 3.7612 ms (± 12.0788) 2.80937 ms (± 38.1438) 1.34
exec_bc_add_noRegAlloc 39.0215 ns (± 2.81391) 48.2529 ns (± 6.59167) 0.81
exec_bc_add_regAlloc 40.0181 ns (± 5.94689) 47.2023 ns (± 3.44363) 0.85
exec_bc_fibonacci_noRegAlloc 327.339 us (± 7.34802) 431.414 us (± 13.9557) 0.76
exec_bc_fibonacci_regAlloc 328.578 us (± 8.30056) 436.805 us (± 45.1169) 0.75
exec_bc_sum_noRegAlloc 72.7452 ms (± 457.424) 98.1295 ms (± 2.75429) 0.74
exec_bc_sum_regAlloc 71.7167 ms (± 1.83957) 97.7088 ms (± 966.809) 0.73
exec_bc_add_call 38.4361 ns (± 0.678215) 49.0665 ns (± 9.37831) 0.78
exec_bc_add_switch 38.6932 ns (± 5.17137) 41.6378 ns (± 5.14195) 0.93
exec_bc_add_threaded 35.9815 ns (± 1.77184) 40.1203 ns (± 7.02462) 0.90
exec_bc_fibonacci_call 330.273 us (± 10.6347) 440.685 us (± 50.1327) 0.75
exec_bc_fibonacci_switch 288.795 us (± 6.69886) 339.99 us (± 45.9944) 0.85
exec_bc_fibonacci_threaded 395.005 us (± 6.09501) 439.042 us (± 14.2326) 0.90
exec_bc_sum_call 72.1544 ms (± 2.22991) 97.6201 ms (± 341.948) 0.74
exec_bc_sum_switch 65.5365 ms (± 1.7441) 69.4814 ms (± 1.9765) 0.94
exec_bc_sum_threaded 82.9514 ms (± 1.23976) 86.9319 ms (± 763.881) 0.95
exec_bc_add_threaded_noReuse 35.9537 ns (± 1.49864) 40.5489 ns (± 7.84818) 0.89
exec_bc_add_threaded_reuse 33.0799 ns (± 3.66686) 36.0746 ns (± 8.31595) 0.92
exec_bc_fibonacci_threaded_noReuse 397.916 us (± 12.5494) 437.056 us (± 10.5284) 0.91
exec_bc_fibonacci_threaded_reuse 393.638 us (± 4.90645) 438.152 us (± 13.241) 0.90
exec_bc_sum_threaded_noReuse 82.9266 ms (± 979.119) 87.3435 ms (± 1.85246) 0.95
exec_bc_sum_threaded_reuse 82.7329 ms (± 47.7488) 87.0172 ms (± 1.09887) 0.95
exec_bc_add_threaded_noSuperinstr 37.0564 ns (± 5.12579) 40.7211 ns (± 6.79945) 0.91
exec_bc_add_threaded_superinstr 35.945 ns (± 1.17006) 40.3912 ns (± 5.84741) 0.89
exec_bc_addOne 34.0865 ns (± 3.14764) 41.1053 ns (± 6.36012) 0.83
exec_mlir_addOne 415.891 ns (± 11.4479) 285.832 ns (± 6.75401) 1.46
exec_cpp_addOne 3.80948 ns (± 1.3055) 4.02278 ns (± 0.663561) 0.95
exec_interpreted_addOne 37.0559 ns (± 3.6125) 40.2643 ns (± 3.84524) 0.92
ir_add 848.588 ns (± 75.7083) 776.301 ns (± 79.8048) 1.09
ir_ifThenElse 1.72598 us (± 108.193) 1.66552 us (± 197.026) 1.04
ir_deeplyNestedIfElse 3.8274 us (± 446.891) 3.38832 us (± 281.408) 1.13
ir_loop 1.76443 us (± 109.613) 1.68975 us (± 214.503) 1.04
ir_ifInsideLoop 3.20546 us (± 181.558) 2.90519 us (± 314.265) 1.10
ir_loopDirectCall 2.08444 us (± 266.487) 1.78207 us (± 120.577) 1.17
ir_pointerLoop 2.23248 us (± 135.964) 2.0415 us (± 219.885) 1.09
ir_staticLoop 1.57852 us (± 111.207) 1.48883 us (± 158.059) 1.06
ir_fibonacci 1.9082 us (± 143.161) 1.77205 us (± 161.076) 1.08
ir_gcd 1.65241 us (± 111.85) 1.53376 us (± 184.337) 1.08
ir_nestedIf10 8.84943 us (± 905.452) 7.83858 us (± 877.414) 1.13
ir_nestedIf100 95.2961 us (± 2.45052) 90.8768 us (± 5.49107) 1.05
ir_chainedIf10 13.6107 us (± 975.727) 11.8792 us (± 995.547) 1.15
ir_chainedIf100 173.559 us (± 3.36985) 171.998 us (± 8.29716) 1.01
ssa_add 196.349 ns (± 17.216) 181.424 ns (± 19.4974) 1.08
ssa_ifThenElse 443.911 ns (± 37.4019) 458.523 ns (± 51.2103) 0.97
ssa_deeplyNestedIfElse 1.26466 us (± 71.8245) 1.17222 us (± 167.162) 1.08
ssa_loop 466.698 ns (± 31.6352) 488.313 ns (± 62.795) 0.96
ssa_ifInsideLoop 976.55 ns (± 77.3689) 938.76 ns (± 102.535) 1.04
ssa_loopDirectCall 481.23 ns (± 31.0013) 494.181 ns (± 48.2228) 0.97
ssa_pointerLoop 559.032 ns (± 45.7635) 583.022 ns (± 51.4938) 0.96
ssa_staticLoop 390.445 ns (± 34.4728) 409.059 ns (± 49.1153) 0.95
ssa_fibonacci 526.795 ns (± 93.3617) 509.911 ns (± 52.4863) 1.03
ssa_gcd 446.307 ns (± 29.3984) 456.237 ns (± 46.5592) 0.98

This comment was automatically generated by workflow using github-action-benchmark.

@PhilippGrulich PhilippGrulich marked this pull request as ready for review July 9, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@PhilippGrulich @claude