Skip to content

Recover val<> variable names via DWARF on the host binary#266

Closed
PhilippGrulich wants to merge 3 commits into
mainfrom
claude/llvm-debuginfo-variables-F2Qt3
Closed

Recover val<> variable names via DWARF on the host binary#266
PhilippGrulich wants to merge 3 commits into
mainfrom
claude/llvm-debuginfo-variables-F2Qt3

Conversation

@PhilippGrulich

Copy link
Copy Markdown
Member

Introduces an opt-in pipeline that captures the source-level variable
name of every val at its declaration site and attaches it as
metadata on the corresponding IR operation, enabling downstream DWARF
emission for the JITted code.

The capture chain:

  • val constructors take a defaulted std::source_location that
    resolves at the user's declaration site.
  • Tracing hooks forward the location to a per-ValueRef side-table on
    ExecutionTrace, gated by the new "debug.captureVarNames" option so
    the default-off path is provably inert.
  • TraceToIRConversionPhase consults a new DebugSymbolResolver that
    builds an llvm::DWARFContext over /proc/self/exe and indexes every
    DW_TAG_variable / DW_TAG_formal_parameter by (file, line, column).
  • Recovered names are arena-interned onto Operation::debugName_ and
    rendered as "// " trailers in both the text and GraphViz IR
    dumps when present.

Coverage:

  • Works for named locals in arithmetic, bool, and enum val
    specializations. Pointer wrappers and moved/assigned targets are
    deliberately untouched (their names come from the original
    constructor site).
  • Gracefully degrades to no names when the host binary lacks DWARF,
    when the platform is not Linux, or when the MLIR backend isn't
    linked in (DWARF libraries come with it).
  • Byte-identical IR dumps when the option is off; all 183 existing
    tests still pass.

Also adds two dedicated Catch2 tests under test/debug-info-tests/
that exercise the resolver directly and an end-to-end trace →
compileToIR → debugName lookup.

https://claude.ai/code/session_01UiVm99oMEXioaGjWdGtM2u

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracing Benchmark

Details
Benchmark suite Current: 9879e2f Previous: fbd0008 Ratio
trace_add 2.08482 us (± 127) 2.29783 us (± 194.868) 0.91
completing_trace_add 2.09097 us (± 131.714) 2.28293 us (± 195.657) 0.92
trace_ifThenElse 8.47188 us (± 995.019) 8.39651 us (± 1.51334) 1.01
completing_trace_ifThenElse 4.32527 us (± 567.778) 4.536 us (± 579.837) 0.95
trace_deeplyNestedIfElse 25.3608 us (± 2.57285) 24.7254 us (± 3.66263) 1.03
completing_trace_deeplyNestedIfElse 11.9689 us (± 1.66708) 12.5865 us (± 1.72963) 0.95
trace_loop 8.24525 us (± 817.867) 8.36665 us (± 1.57258) 0.99
completing_trace_loop 4.3297 us (± 416.778) 4.57594 us (± 541.226) 0.95
trace_ifInsideLoop 16.1367 us (± 1.49889) 16.1426 us (± 3.65332) 1.00
completing_trace_ifInsideLoop 7.78275 us (± 589.12) 8.41512 us (± 1.23609) 0.92
trace_loopDirectCall 8.31288 us (± 667.768) 8.27417 us (± 1.47989) 1.00
completing_trace_loopDirectCall 4.47746 us (± 375.371) 4.68052 us (± 712.241) 0.96
trace_pointerLoop 13.2086 us (± 1.0578) 13.824 us (± 2.78924) 0.96
completing_trace_pointerLoop 9.22973 us (± 954.992) 10.199 us (± 1.51772) 0.90
trace_staticLoop 6.93762 us (± 409.064) 7.46664 us (± 792.799) 0.93
completing_trace_staticLoop 6.99985 us (± 499.058) 7.4373 us (± 743.801) 0.94
trace_fibonacci 9.52013 us (± 698.868) 9.62993 us (± 1.75064) 0.99
completing_trace_fibonacci 5.82631 us (± 654.264) 5.98329 us (± 775.486) 0.97
trace_gcd 7.68087 us (± 641.276) 7.56161 us (± 1.08357) 1.02
completing_trace_gcd 3.72609 us (± 307.439) 3.86545 us (± 481.362) 0.96
trace_nestedIf10 35.6398 us (± 3.21098) 39.6688 us (± 7.93865) 0.90
completing_trace_nestedIf10 38.9283 us (± 28.1862) 39.4348 us (± 7.44999) 0.99
trace_nestedIf100 1.32094 ms (± 35.8194) 1.47933 ms (± 29.4589) 0.89
completing_trace_nestedIf100 1.36919 ms (± 89.989) 1.49096 ms (± 30.1932) 0.92
trace_chainedIf10 96.2871 us (± 5.91038) 96.2765 us (± 10.9483) 1.00
completing_trace_chainedIf10 45.7938 us (± 3.37993) 50.8909 us (± 9.56979) 0.90
trace_chainedIf100 4.40587 ms (± 66.7602) 4.45426 ms (± 60.0025) 0.99
completing_trace_chainedIf100 2.11489 ms (± 33.3494) 2.29528 ms (± 43.0008) 0.92
exec_bc_addOne 34.7005 ns (± 5.00391) 37.2303 ns (± 3.51708) 0.93
exec_mlir_addOne 288.839 ns (± 16.2656) 271.771 ns (± 6.65227) 1.06
exec_cpp_addOne 4.07081 ns (± 0.766171) 3.75101 ns (± 0.429435) 1.09
exec_interpreted_addOne 38.0839 ns (± 1.9882) 38.531 ns (± 1.84049) 0.99
exec_mlir_add 11.1885 ns (± 1.36881) 10.7525 ns (± 0.679524) 1.04
exec_mlir_fibonacci 13.1283 us (± 1.54774) 16.9187 us (± 1.67676) 0.78
exec_mlir_sum 513 us (± 15.9868) 560.379 us (± 61.6932) 0.92
exec_cpp_add 4.73428 ns (± 0.75402) 4.45106 ns (± 0.302066) 1.06
exec_cpp_fibonacci 96.3456 us (± 8.31833) 109.859 us (± 8.67151) 0.88
exec_cpp_sum 35.97 ms (± 163.296) 23.6387 ms (± 305.914) 1.52
exec_bc_add 42.5766 ns (± 3.937) 44.1067 ns (± 7.20227) 0.97
exec_bc_fibonacci 786.595 us (± 13.311) 625.821 us (± 11.0899) 1.26
exec_bc_sum 168.833 ms (± 1.74905) 141.515 ms (± 1.57658) 1.19
exec_asmjit_add 3.58767 ns (± 0.551613) 3.57412 ns (± 0.404158) 1.00
exec_asmjit_fibonacci 20.6982 us (± 1.96162) 22.2244 us (± 1.57474) 0.93
exec_asmjit_sum 4.85053 ms (± 18.8958) 5.29957 ms (± 34.5193) 0.92
exec_bc_add_noRegAlloc 42.9662 ns (± 3.90677) 44.5689 ns (± 5.76884) 0.96
exec_bc_add_regAlloc 42.6584 ns (± 3.24016) 44.7931 ns (± 6.39022) 0.95
exec_bc_fibonacci_noRegAlloc 785.887 us (± 14.3287) 635.532 us (± 22.5573) 1.24
exec_bc_fibonacci_regAlloc 790.873 us (± 23.4258) 624.517 us (± 11.8616) 1.27
exec_bc_sum_noRegAlloc 175.339 ms (± 8.00078) 142.153 ms (± 2.08175) 1.23
exec_bc_sum_regAlloc 170.677 ms (± 4.23663) 142.506 ms (± 4.38093) 1.20
ir_add 770.004 ns (± 35.7543) 714.188 ns (± 61.9818) 1.08
ir_ifThenElse 1.57292 us (± 144.427) 1.56722 us (± 100.633) 1.00
ir_deeplyNestedIfElse 3.41493 us (± 215.96) 3.32578 us (± 194.785) 1.03
ir_loop 1.663 us (± 149.002) 1.58636 us (± 91.8031) 1.05
ir_ifInsideLoop 2.88122 us (± 271.682) 2.8972 us (± 459.129) 0.99
ir_loopDirectCall 1.81796 us (± 206.882) 1.74208 us (± 107.84) 1.04
ir_pointerLoop 1.98802 us (± 115.901) 1.90251 us (± 88.3959) 1.04
ir_staticLoop 1.50439 us (± 108.951) 1.4572 us (± 92.1571) 1.03
ir_fibonacci 1.77094 us (± 176.095) 1.71397 us (± 107.39) 1.03
ir_gcd 1.5056 us (± 119.829) 1.42298 us (± 126.153) 1.06
ir_nestedIf10 7.77628 us (± 526.379) 7.81615 us (± 432.449) 0.99
ir_nestedIf100 90.3355 us (± 4.90439) 93.3493 us (± 3.75176) 0.97
ir_chainedIf10 12.0265 us (± 1.24157) 12.0695 us (± 774.013) 1.00
ir_chainedIf100 170.743 us (± 9.26689) 173.153 us (± 6.55219) 0.99
e2e_tiered_bc_to_mlir 55.6813 us (± 9.38588) 45.8324 us (± 15.2855) 1.21
e2e_single_mlir 5.53721 ms (± 40.9647) 6.1304 ms (± 147.647) 0.90
ssa_add 189.971 ns (± 11.4025) 179.069 ns (± 9.40586) 1.06
ssa_ifThenElse 477.644 ns (± 91.6252) 435.753 ns (± 26.2967) 1.10
ssa_deeplyNestedIfElse 1.17822 us (± 77.9459) 1.11537 us (± 91.3448) 1.06
ssa_loop 489.957 ns (± 32.969) 452.529 ns (± 24.0127) 1.08
ssa_ifInsideLoop 921.761 ns (± 71.1568) 843.793 ns (± 40.7455) 1.09
ssa_loopDirectCall 493.69 ns (± 33.4005) 462.963 ns (± 24.3206) 1.07
ssa_pointerLoop 590.211 ns (± 45.4837) 560.052 ns (± 40.0071) 1.05
ssa_staticLoop 454.665 ns (± 31.5446) 430.417 ns (± 18.362) 1.06
ssa_fibonacci 509.793 ns (± 30.9847) 481.922 ns (± 35.7511) 1.06
ssa_gcd 450.461 ns (± 30.2704) 420.96 ns (± 26.8031) 1.07
comp_mlir_add 5.55725 ms (± 37.6726) 6.377 ms (± 172.925) 0.87
comp_mlir_ifThenElse 6.14918 ms (± 49.1722) 7.01691 ms (± 248.474) 0.88
comp_mlir_deeplyNestedIfElse 5.0534 ms (± 53.8726) 5.78417 ms (± 190.535) 0.87
comp_mlir_loop 7.16962 ms (± 50.0534) 8.35393 ms (± 324.38) 0.86
comp_mlir_ifInsideLoop 28.7597 ms (± 120.99) 30.2604 ms (± 582.671) 0.95
comp_mlir_loopDirectCall 11.92 ms (± 57.9022) 12.7503 ms (± 263.994) 0.93
comp_mlir_pointerLoop 27.9455 ms (± 104.029) 29.5681 ms (± 948.096) 0.95
comp_mlir_staticLoop 5.04314 ms (± 36.4919) 5.64512 ms (± 145.239) 0.89
comp_mlir_fibonacci 10.5093 ms (± 51.6781) 11.307 ms (± 187.951) 0.93
comp_mlir_gcd 9.54589 ms (± 53.3729) 10.1921 ms (± 258.79) 0.94
comp_mlir_nestedIf10 10.5872 ms (± 56.7083) 11.3222 ms (± 231.663) 0.94
comp_mlir_nestedIf100 25.0258 ms (± 150.218) 26.7072 ms (± 515.304) 0.94
comp_mlir_chainedIf10 9.6167 ms (± 60.783) 10.7337 ms (± 406.502) 0.90
comp_mlir_chainedIf100 20.2308 ms (± 110.552) 21.7938 ms (± 565.933) 0.93
comp_cpp_add 25.2546 ms (± 1.36918) 28.7531 ms (± 408.965) 0.88
comp_cpp_ifThenElse 25.9442 ms (± 731.213) 28.4886 ms (± 1.18424) 0.91
comp_cpp_deeplyNestedIfElse 26.7402 ms (± 482.952) 29.5467 ms (± 432.148) 0.91
comp_cpp_loop 26.7027 ms (± 796.838) 29.1782 ms (± 4.03728) 0.92
comp_cpp_ifInsideLoop 28.3044 ms (± 642.793) 29.672 ms (± 404.492) 0.95
comp_cpp_loopDirectCall 27.6679 ms (± 498.089) 29.0444 ms (± 513.671) 0.95
comp_cpp_pointerLoop 27.2811 ms (± 725.781) 28.8736 ms (± 550.805) 0.94
comp_cpp_staticLoop 25.5695 ms (± 428.511) 28.3266 ms (± 504.336) 0.90
comp_cpp_fibonacci 25.7136 ms (± 316.945) 28.83 ms (± 600.848) 0.89
comp_cpp_gcd 25.7358 ms (± 432.448) 28.6621 ms (± 571.361) 0.90
comp_cpp_nestedIf10 29.1245 ms (± 672.469) 31.517 ms (± 1.02842) 0.92
comp_cpp_nestedIf100 62.5974 ms (± 471.817) 65.1716 ms (± 1.53225) 0.96
comp_cpp_chainedIf10 31.8998 ms (± 750.768) 33.5113 ms (± 452.851) 0.95
comp_cpp_chainedIf100 93.2578 ms (± 2.84926) 94.182 ms (± 436.859) 0.99
comp_bc_add 14.9156 us (± 2.49322) 14.3438 us (± 2.26851) 1.04
comp_bc_ifThenElse 19.1065 us (± 4.06862) 16.1772 us (± 2.83898) 1.18
comp_bc_deeplyNestedIfElse 22.647 us (± 3.81301) 20.3351 us (± 4.1405) 1.11
comp_bc_loop 18.4418 us (± 3.11443) 16.6135 us (± 3.54299) 1.11
comp_bc_ifInsideLoop 21.6527 us (± 3.84768) 18.7632 us (± 3.64657) 1.15
comp_bc_loopDirectCall 19.4513 us (± 3.23805) 16.8706 us (± 3.56488) 1.15
comp_bc_pointerLoop 20.5312 us (± 3.30095) 17.8911 us (± 3.89168) 1.15
comp_bc_staticLoop 17.6438 us (± 3.58075) 16.4539 us (± 3.34504) 1.07
comp_bc_fibonacci 18.8966 us (± 2.93959) 17.0151 us (± 4.24733) 1.11
comp_bc_gcd 18.8804 us (± 3.35427) 16.3401 us (± 3.44193) 1.16
comp_bc_nestedIf10 36.8133 us (± 4.8887) 31.9 us (± 5.1301) 1.15
comp_bc_nestedIf100 206.541 us (± 26.4103) 200.356 us (± 10.6207) 1.03
comp_bc_chainedIf10 53.6699 us (± 9.76106) 44.7201 us (± 10.3866) 1.20
comp_bc_chainedIf100 307.22 us (± 22.7885) 315.43 us (± 12.9589) 0.97
comp_asmjit_add 21.2922 us (± 4.60285) 17.3357 us (± 5.7874) 1.23
comp_asmjit_ifThenElse 33.2272 us (± 4.84418) 27.4672 us (± 7.46295) 1.21
comp_asmjit_deeplyNestedIfElse 57.1593 us (± 8.2439) 48.3402 us (± 10.4588) 1.18
comp_asmjit_loop 35.224 us (± 5.40623) 29.0349 us (± 5.82163) 1.21
comp_asmjit_ifInsideLoop 56.4719 us (± 7.11622) 49.227 us (± 11.1378) 1.15
comp_asmjit_loopDirectCall 48.9311 us (± 8.76696) 32.377 us (± 6.76528) 1.51
comp_asmjit_pointerLoop 48.6896 us (± 7.73305) 35.1061 us (± 5.90692) 1.39
comp_asmjit_staticLoop 28.5454 us (± 5.11744) 24.0122 us (± 5.12412) 1.19
comp_asmjit_fibonacci 43.0948 us (± 6.2184) 31.5818 us (± 5.64908) 1.36
comp_asmjit_gcd 35.733 us (± 6.06582) 29.06 us (± 6.34809) 1.23
comp_asmjit_nestedIf10 105.361 us (± 13.6884) 96.5661 us (± 12.9167) 1.09
comp_asmjit_nestedIf100 1050.94 us (± 23810.199999999997) 1070.25 us (± 74274.6) 0.98
comp_asmjit_chainedIf10 156.129 us (± 12.5399) 151.378 us (± 17.7476) 1.03
comp_asmjit_chainedIf100 2.17187 ms (± 35.4027) 2.27398 ms (± 236.764) 0.96
tiered_compile_addOne 56.146 us (± 12.4318) 46.9348 us (± 22.4931) 1.20
single_compile_mlir_addOne 3.32114 ms (± 84.0705) 3.58179 ms (± 119.249) 0.93
single_compile_cpp_addOne 25.3831 ms (± 552.177) 27.3309 ms (± 434.848) 0.93
single_compile_bc_addOne 56.4305 us (± 12.958) 45.9956 us (± 14.7115) 1.23
tiered_compile_sumLoop 75.1701 us (± 8.91457) 63.9913 us (± 18.5359) 1.17
single_compile_mlir_sumLoop 5.36342 ms (± 56.3378) 5.65544 ms (± 147.111) 0.95
single_compile_cpp_sumLoop 25.609 ms (± 372.082) 28.1095 ms (± 448.351) 0.91
single_compile_bc_sumLoop 76.1174 us (± 8.53903) 64.9117 us (± 17.2028) 1.17

This comment was automatically generated by workflow using github-action-benchmark.

@PhilippGrulich
PhilippGrulich force-pushed the claude/llvm-debuginfo-variables-F2Qt3 branch from 7690c35 to 3b999d8 Compare April 26, 2026 14:28
claude added 2 commits April 26, 2026 15:12
…ation framework

Rebases on main, which added trace-time source-location capture to IR
operations in #271. That PR's SourceLocationResolver symbolises return
addresses to (file, function, line, column) frames using backward.hpp;
this commit layers DWARF variable-name recovery on top.

Concretely:
- New nautilus::debug::DwarfVariableResolver, sibling to
  tracing::SourceLocationResolver. Given a (file, line, column) — or
  the SourceFrame produced by the source-location resolver — it queries
  the host binary's DWARF for a DW_TAG_variable / DW_TAG_formal_parameter
  whose decl coordinates match, and returns DW_AT_name. That recovers
  the user-declared local name (e.g. "sum", "factor") that the trace
  itself can't know.
- IRPrintOptions gains showVariableNames + variableResolver. When both
  are set the formatter appends a [var=<name>] tail to the existing
  "; at <file>:<line> (<func>)" trailer at the innermost user frame.
- LegacyCompiler::compileToIR honours a new dump.variableNames option
  that auto-implies dump.sourceLocations, since variable resolution
  needs a resolved frame to query against.
- Resolver is gated on Linux + ENABLE_MLIR_BACKEND (LLVM DWARF libs).
  Other platforms get a stub that returns nullptr from getInstance(),
  the same shape as SourceLocationResolver's fallback when
  ENABLE_STACKTRACE is off. The IRGraph formatter silently omits the
  [var=...] annotation when the resolver is null.
- Tests verify (a) the resolver handles namespace-scope and local
  declarations on the running test binary, (b) every traced IR op has a
  non-null sourceTag from main's capture, (c) IRPrintOptions defaults
  preserve byte-identical legacy output, (d) the trailer-shape regex
  matches the documented format. The end-to-end "render and inspect"
  test is intentionally narrow because of a pre-existing lifetime issue
  in #271 — see the comment in IRGraph.cpp.

Lifetime caveat carried over from #271: TagRecorder is stack-local in
the trace contexts, so Operation::sourceTag is dangling once tracing
returns. The compiler-internal "after_ir_creation" dump runs while the
trace is still in scope, so the engine path works; out-of-band
toString(opts) callers (tests, future post-trace inspection) need the
trie relocated into longer-lived storage. Documented in IRGraph.cpp;
fix is a follow-up to #271 rather than this commit.
…rceTag

Addresses the lifetime issue introduced in #271: Operation::sourceTag
held a pointer into TagRecorder's trie, which was stack-allocated in
{Lazy,ExceptionBased}TraceContext::startTrace and destroyed when
tracing returned. Calling toString() with showSourceLocations=true
after compileToIR therefore dereferenced dangling pointers — the
compiler-internal dump worked only because DumpHandler gates its
lambda invocation on the matching dump option being on.

The new design resolves the trace's tag chain into a fully-baked
OperationDebugInfo (frames + optional variable name) once during
TraceToIRConversionPhase, while the live TagRecorder is still in
scope, and parks it on the IRGraph keyed by OperationIdentifier. After
conversion the IR is self-contained: the trace arena (and the trie)
can be torn down, passes run unaffected, and post-compile callers can
freely render with showSourceLocations=true.

Concretely:
- IRGraph gains setDebugInfo / getDebugInfo / getDebugInfoMap accessors
  and an unordered_map<OperationIdentifier, OperationDebugInfo>
  side-table. Empty when no resolver was supplied (the default), so
  the option-off path stays zero-overhead.
- IRPrintOptions drops the resolver / variableResolver fields. They
  carried live pointers consumed at print time; resolution now happens
  at conversion. Only the boolean show* flags remain.
- TraceToIRConversionPhase::apply gains a DebugInfoResolvers
  parameter. Each process* call site is rewired from
  BasicBlock::addTaggedOperation to plain addOperation +
  bakeDebugInfo(irOp, traceOp). The processCMP and processJMP
  bespoke setSourceTag calls become bakeDebugInfo too.
- Operation::sourceTag, Operation::setSourceTag, getSourceTag, and
  BasicBlock::addTaggedOperation are removed entirely. Operation
  drops the tracing::Tag forward declaration and its mutable Tag*
  field (still 32 B on a 64-bit target — the field was occupying
  tail padding).
- ConstantFoldingAndCopyPropagationPass copies the
  OperationDebugInfo from the original op to its folded replacement
  via the IRGraph side-table, preserving provenance across the
  rewrite.
- LegacyCompiler now constructs the resolvers up-front when
  dump.sourceLocations / dump.variableNames is set and passes them
  into TraceToIRConversionPhase. The resolvers can be released
  immediately after compileToIR returns; the IR keeps the baked data.
- Tests are restored to strict end-to-end shape: a fresh fixture
  compiles a program, the dump path is exercised after compile (no
  more lifetime workaround), and the trailer "; at <file>:<line>
  (<func>) [var=<name>]" is checked with the documented regex.

The lifetime warning comment in IRGraph.cpp is gone — the dangling-
pointer hazard it described no longer exists.
@PhilippGrulich
PhilippGrulich force-pushed the claude/llvm-debuginfo-variables-F2Qt3 branch from 32a4205 to d1c6914 Compare April 26, 2026 15:18
Extends OperationDebugInfo from a single innermost variable name to a
parallel vector of names — one per frame in the captured stack. When a
value flows through user-side helper functions (e.g.
`val<int> sum = compute(x, y);` where `compute` itself contains
`val<int> acc = a + b;`), each frame in the trace's tag chain gets its
own `[var=<name>]` annotation in the dump, recovered from DWARF at that
frame's source coordinates.

- OperationDebugInfo::variableName (optional<string>) is replaced by
  variableNames (vector<optional<string>>), kept in lock-step with
  frames. Element i holds the variable that received the value at
  frames[i], or nullopt when no DIE matches at that site.
- bakeDebugInfo loops over every frame and queries the DWARF resolver
  for each, instead of only the innermost. The resolver caches lookups
  per (file, line, column) so per-frame resolution is effectively free
  after the first occurrence on each line.
- The IRGraph dump formatter pairs each frame with its name: innermost
  on the same line as the op, inlined-from frames each on their own
  line, each carrying a [var=...] suffix when one was recovered.
- IRGraph::setDebugInfo merges the new vector field analogously to the
  old scalar — empty/no-name vectors don't overwrite a populated
  existing entry, so callers can attach frames and names independently.
- Added VarNameMetadata test that exercises the multi-frame path
  through a [[gnu::noinline]] user-side helper. The test cleanly skips
  on builds without ENABLE_STACKTRACE / DWARF / inlined helpers, but
  asserts both "acc" and "sum" appear when all three line up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants