Harden CUDA backend for production use#276
Open
PhilippGrulich wants to merge 1 commit into
Open
Conversation
- CUDALoweringProvider: replace assert(false) on unknown IR type with a
RuntimeException so failures are caught in release builds.
- Emit a header-only `nautilus_cuda_check` that wraps cudaGetLastError /
cudaDeviceSynchronize after kernel launches when the new
`gpu.cuda.errorCheck` option is set; default off keeps reference codegen
byte-identical.
- GPULoweringProviderBase: validate that setGrid/setBlock intrinsics receive
exactly three arguments instead of indexing past the end of the vector.
- CUDACompiler:
* Look for nvcc via PATH directly (no shell), CUDA_HOME / CUDA_PATH, and
common install paths; allow `gpu.cuda.nvccPath` override.
* Make the nvcc invocation configurable: `gpu.cuda.arch`,
`gpu.cuda.optLevel` (default -O3), `gpu.cuda.fastMath`,
`gpu.cuda.debug` (-g -G -lineinfo), `gpu.cuda.extraFlags`.
* Quote all interpolated paths so temp directories with spaces or shell
metacharacters can't break the command.
* Use a unique temp-file stem (PID + counter + random) so concurrent
compilations of the same IR id don't clobber each other.
* On failure, surface the path to the offending .cu so users can
reproduce; opt-out via `gpu.cuda.keepArtifacts=false`.
* Stop manually unlinking the .so after load — SharedLibrary's destructor
already does it; remove the redundant double-unlink.
* Replace std::runtime_error with RuntimeException for consistency with
the rest of the codebase.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Tracing Benchmark
Details
| Benchmark suite | Current: 5215eba | Previous: 10a6635 | Ratio |
|---|---|---|---|
trace_add |
2.14193 us (± 241.418) |
2.49638 us (± 295.356) |
0.86 |
completing_trace_add |
2.07941 us (± 160.366) |
2.4736 us (± 334.61) |
0.84 |
trace_ifThenElse |
8.22276 us (± 871.086) |
9.30328 us (± 1.67908) |
0.88 |
completing_trace_ifThenElse |
4.24021 us (± 454.07) |
4.70958 us (± 574.319) |
0.90 |
trace_deeplyNestedIfElse |
24.7444 us (± 2.77825) |
26.0099 us (± 3.07346) |
0.95 |
completing_trace_deeplyNestedIfElse |
11.6508 us (± 1.62897) |
13.1091 us (± 1.70257) |
0.89 |
trace_loop |
8.29836 us (± 1.3598) |
8.71348 us (± 1.22788) |
0.95 |
completing_trace_loop |
4.29911 us (± 475.927) |
4.77327 us (± 516.624) |
0.90 |
trace_ifInsideLoop |
16.1344 us (± 2.22654) |
17.3334 us (± 2.90873) |
0.93 |
completing_trace_ifInsideLoop |
7.80822 us (± 851.052) |
8.82408 us (± 1.28846) |
0.88 |
trace_loopDirectCall |
8.75312 us (± 1.88931) |
8.96005 us (± 1.49533) |
0.98 |
completing_trace_loopDirectCall |
4.36941 us (± 427.279) |
5.1524 us (± 762.558) |
0.85 |
trace_pointerLoop |
12.9485 us (± 1.84169) |
14.2471 us (± 2.16149) |
0.91 |
completing_trace_pointerLoop |
9.26741 us (± 967.185) |
10.4746 us (± 1.84546) |
0.88 |
trace_staticLoop |
6.82796 us (± 551.67) |
7.6142 us (± 1.0114) |
0.90 |
completing_trace_staticLoop |
6.88286 us (± 703.306) |
7.46691 us (± 872.687) |
0.92 |
trace_fibonacci |
9.45589 us (± 1.16794) |
10.1531 us (± 1.57566) |
0.93 |
completing_trace_fibonacci |
5.531 us (± 508.298) |
6.17947 us (± 858.737) |
0.90 |
trace_gcd |
7.53 us (± 1.12838) |
8.05058 us (± 1.02295) |
0.94 |
completing_trace_gcd |
3.58555 us (± 298.261) |
4.06246 us (± 448.501) |
0.88 |
trace_nestedIf10 |
35.6223 us (± 4.68492) |
37.7878 us (± 6.23274) |
0.94 |
completing_trace_nestedIf10 |
35.2874 us (± 4.25188) |
38.9354 us (± 7.34432) |
0.91 |
trace_nestedIf100 |
1.34016 ms (± 51.9094) |
1.34644 ms (± 53.7282) |
1.00 |
completing_trace_nestedIf100 |
1.34357 ms (± 21.784) |
1.35177 ms (± 56.6128) |
0.99 |
trace_chainedIf10 |
96.6318 us (± 8.83748) |
99.7738 us (± 12.6143) |
0.97 |
completing_trace_chainedIf10 |
47.0374 us (± 7.6243) |
49.9159 us (± 9.0803) |
0.94 |
trace_chainedIf100 |
4.39303 ms (± 31.0124) |
4.48434 ms (± 79.2886) |
0.98 |
completing_trace_chainedIf100 |
2.20295 ms (± 36.5426) |
2.26925 ms (± 73.0211) |
0.97 |
comp_mlir_add |
5.53652 ms (± 149.231) |
5.52896 ms (± 150.264) |
1.00 |
comp_mlir_ifThenElse |
6.13808 ms (± 149.771) |
6.15129 ms (± 143.859) |
1.00 |
comp_mlir_deeplyNestedIfElse |
5.03609 ms (± 123.402) |
5.04123 ms (± 67.5269) |
1.00 |
comp_mlir_loop |
7.22426 ms (± 246.93) |
7.23834 ms (± 348.192) |
1.00 |
comp_mlir_ifInsideLoop |
29.156 ms (± 469.694) |
28.78 ms (± 822.48) |
1.01 |
comp_mlir_loopDirectCall |
11.9438 ms (± 370.641) |
12.4205 ms (± 456.737) |
0.96 |
comp_mlir_pointerLoop |
28.3016 ms (± 515.343) |
28.18 ms (± 795.83) |
1.00 |
comp_mlir_staticLoop |
4.99304 ms (± 132.209) |
4.98862 ms (± 45.2884) |
1.00 |
comp_mlir_fibonacci |
10.5078 ms (± 196.246) |
10.5174 ms (± 254.747) |
1.00 |
comp_mlir_gcd |
9.44779 ms (± 183.967) |
9.43739 ms (± 66.7206) |
1.00 |
comp_mlir_nestedIf10 |
10.4734 ms (± 137.331) |
10.9118 ms (± 562.598) |
0.96 |
comp_mlir_nestedIf100 |
25.0817 ms (± 233.428) |
25.5045 ms (± 614.911) |
0.98 |
comp_mlir_chainedIf10 |
9.58567 ms (± 173.005) |
9.77351 ms (± 299.372) |
0.98 |
comp_mlir_chainedIf100 |
20.3462 ms (± 318.434) |
20.2026 ms (± 289.302) |
1.01 |
comp_cpp_add |
24.9347 ms (± 393.281) |
25.3523 ms (± 536.121) |
0.98 |
comp_cpp_ifThenElse |
25.4642 ms (± 372.209) |
25.6938 ms (± 472.16) |
0.99 |
comp_cpp_deeplyNestedIfElse |
26.8425 ms (± 451.891) |
26.2772 ms (± 280.811) |
1.02 |
comp_cpp_loop |
26.2579 ms (± 571.716) |
25.3249 ms (± 302.8) |
1.04 |
comp_cpp_ifInsideLoop |
27.73 ms (± 523.59) |
26.0316 ms (± 253.783) |
1.07 |
comp_cpp_loopDirectCall |
26.549 ms (± 551.827) |
25.5171 ms (± 227.43) |
1.04 |
comp_cpp_pointerLoop |
26.5721 ms (± 429.797) |
25.7051 ms (± 267.651) |
1.03 |
comp_cpp_staticLoop |
26.2457 ms (± 677.671) |
25.1416 ms (± 369.29) |
1.04 |
comp_cpp_fibonacci |
26.36 ms (± 537.517) |
25.9456 ms (± 429.397) |
1.02 |
comp_cpp_gcd |
26.1743 ms (± 359.475) |
25.3955 ms (± 444.048) |
1.03 |
comp_cpp_nestedIf10 |
29.2668 ms (± 314.615) |
28.4827 ms (± 851.989) |
1.03 |
comp_cpp_nestedIf100 |
63.1243 ms (± 741.327) |
62.4417 ms (± 958.238) |
1.01 |
comp_cpp_chainedIf10 |
31.8151 ms (± 504.364) |
31.448 ms (± 927.552) |
1.01 |
comp_cpp_chainedIf100 |
92.0869 ms (± 560.315) |
92.2053 ms (± 608.636) |
1.00 |
comp_bc_add |
14.3923 us (± 2.264) |
14.582 us (± 1.75527) |
0.99 |
comp_bc_ifThenElse |
18.5174 us (± 3.51217) |
18.6811 us (± 3.58857) |
0.99 |
comp_bc_deeplyNestedIfElse |
22.168 us (± 4.76928) |
22.7163 us (± 4.31134) |
0.98 |
comp_bc_loop |
18.698 us (± 4.08432) |
18.9731 us (± 4.056) |
0.99 |
comp_bc_ifInsideLoop |
21.585 us (± 4.27995) |
21.2198 us (± 3.74251) |
1.02 |
comp_bc_loopDirectCall |
19.1437 us (± 3.33248) |
18.9753 us (± 3.3135) |
1.01 |
comp_bc_pointerLoop |
20.5751 us (± 3.90964) |
20.1783 us (± 3.72759) |
1.02 |
comp_bc_staticLoop |
17.1818 us (± 4.01158) |
17.3933 us (± 3.71277) |
0.99 |
comp_bc_fibonacci |
19.0605 us (± 3.72124) |
18.4986 us (± 2.35865) |
1.03 |
comp_bc_gcd |
18.3764 us (± 3.47277) |
17.9795 us (± 2.53701) |
1.02 |
comp_bc_nestedIf10 |
35.2973 us (± 4.56463) |
34.8719 us (± 4.27966) |
1.01 |
comp_bc_nestedIf100 |
194.799 us (± 11.5642) |
191.397 us (± 12.8541) |
1.02 |
comp_bc_chainedIf10 |
51.7933 us (± 10.302) |
49.1121 us (± 5.49462) |
1.05 |
comp_bc_chainedIf100 |
302.074 us (± 14.8114) |
297.447 us (± 12.8065) |
1.02 |
comp_asmjit_add |
21.517 us (± 4.57154) |
20.7886 us (± 3.14001) |
1.04 |
comp_asmjit_ifThenElse |
33.112 us (± 5.21107) |
32.8265 us (± 5.22408) |
1.01 |
comp_asmjit_deeplyNestedIfElse |
55.4894 us (± 8.09718) |
56.0766 us (± 9.15709) |
0.99 |
comp_asmjit_loop |
35.6537 us (± 5.69414) |
35.0292 us (± 5.17887) |
1.02 |
comp_asmjit_ifInsideLoop |
56.8754 us (± 9.78529) |
57.2267 us (± 11.6151) |
0.99 |
comp_asmjit_loopDirectCall |
45.1682 us (± 8.20607) |
44.056 us (± 5.42689) |
1.03 |
comp_asmjit_pointerLoop |
48.562 us (± 9.57276) |
48.1419 us (± 8.30256) |
1.01 |
comp_asmjit_staticLoop |
27.7981 us (± 4.70184) |
28.3138 us (± 5.3941) |
0.98 |
comp_asmjit_fibonacci |
42.4897 us (± 7.70104) |
43.0434 us (± 8.60098) |
0.99 |
comp_asmjit_gcd |
35.267 us (± 5.71499) |
35.0731 us (± 6.56689) |
1.01 |
comp_asmjit_nestedIf10 |
113.935 us (± 24.3674) |
104.565 us (± 14.2612) |
1.09 |
comp_asmjit_nestedIf100 |
1032.36 us (± 16945.6) |
1068.05 us (± 146903) |
0.97 |
comp_asmjit_chainedIf10 |
155.324 us (± 15.2157) |
154.231 us (± 15.4403) |
1.01 |
comp_asmjit_chainedIf100 |
2.15389 ms (± 36.8769) |
2.13529 ms (± 28.6135) |
1.01 |
exec_mlir_add |
10.4381 ns (± 0.965989) |
10.7701 ns (± 1.25718) |
0.97 |
exec_mlir_fibonacci |
13.4741 us (± 1.59366) |
13.3631 us (± 1.67729) |
1.01 |
exec_mlir_sum |
562.467 us (± 19.8394) |
513.173 us (± 15.8951) |
1.10 |
exec_cpp_add |
4.5432 ns (± 0.45645) |
4.84074 ns (± 0.975486) |
0.94 |
exec_cpp_fibonacci |
96.4863 us (± 9.44943) |
97.148 us (± 14.5938) |
0.99 |
exec_cpp_sum |
36.1241 ms (± 727.387) |
35.9689 ms (± 194.582) |
1.00 |
exec_bc_add |
43.4638 ns (± 5.07882) |
43.5894 ns (± 5.2732) |
1.00 |
exec_bc_fibonacci |
752.523 us (± 15.9853) |
753.817 us (± 16.9593) |
1.00 |
exec_bc_sum |
162.42 ms (± 5.38065) |
163.437 ms (± 1.52509) |
0.99 |
exec_asmjit_add |
3.47111 ns (± 0.287785) |
3.59554 ns (± 0.522867) |
0.97 |
exec_asmjit_fibonacci |
20.6396 us (± 2.5395) |
20.7515 us (± 1.80854) |
0.99 |
exec_asmjit_sum |
4.86177 ms (± 30.2713) |
4.84254 ms (± 26.3078) |
1.00 |
exec_bc_add_noRegAlloc |
43.2599 ns (± 4.91202) |
44.3788 ns (± 6.39372) |
0.97 |
exec_bc_add_regAlloc |
42.7576 ns (± 4.45358) |
43.9551 ns (± 5.99039) |
0.97 |
exec_bc_fibonacci_noRegAlloc |
750.608 us (± 8.14254) |
751.654 us (± 14.3443) |
1.00 |
exec_bc_fibonacci_regAlloc |
750.687 us (± 9.47664) |
751.761 us (± 11.6866) |
1.00 |
exec_bc_sum_noRegAlloc |
163.116 ms (± 301.428) |
163.167 ms (± 446.848) |
1.00 |
exec_bc_sum_regAlloc |
162.845 ms (± 5.21806) |
163.266 ms (± 1.17473) |
1.00 |
e2e_tiered_bc_to_mlir |
80.9789 us (± 14.9499) |
56.5271 us (± 14.5799) |
1.43 |
e2e_single_mlir |
5.51656 ms (± 222.49) |
5.53847 ms (± 193.361) |
1.00 |
tiered_compile_addOne |
53.3415 us (± 9.08245) |
55.7034 us (± 13.3607) |
0.96 |
single_compile_mlir_addOne |
3.24884 ms (± 108.304) |
3.28567 ms (± 103.998) |
0.99 |
single_compile_cpp_addOne |
24.4974 ms (± 271.136) |
25.0471 ms (± 510.067) |
0.98 |
single_compile_bc_addOne |
59.3384 us (± 13.4253) |
56.0618 us (± 12.9099) |
1.06 |
tiered_compile_sumLoop |
74.9239 us (± 8.89201) |
77.4694 us (± 16.0654) |
0.97 |
single_compile_mlir_sumLoop |
5.28337 ms (± 120.682) |
5.30892 ms (± 116.288) |
1.00 |
single_compile_cpp_sumLoop |
25.3077 ms (± 320.227) |
25.7075 ms (± 518.009) |
0.98 |
single_compile_bc_sumLoop |
76.8915 us (± 15.3143) |
77.1814 us (± 14.9321) |
1.00 |
ssa_add |
190.582 ns (± 15.5952) |
204.202 ns (± 28.1667) |
0.93 |
ssa_ifThenElse |
477.218 ns (± 39.1005) |
484.779 ns (± 51.85) |
0.98 |
ssa_deeplyNestedIfElse |
1.2019 us (± 135.91) |
1.20907 us (± 105.009) |
0.99 |
ssa_loop |
499.016 ns (± 30.3365) |
533.363 ns (± 80.3068) |
0.94 |
ssa_ifInsideLoop |
925.078 ns (± 64.5929) |
959.38 ns (± 91.9966) |
0.96 |
ssa_loopDirectCall |
505.608 ns (± 36.937) |
538.332 ns (± 83.9372) |
0.94 |
ssa_pointerLoop |
610.703 ns (± 48.1879) |
612.761 ns (± 60.7414) |
1.00 |
ssa_staticLoop |
455.876 ns (± 19.1457) |
475.509 ns (± 52.7257) |
0.96 |
ssa_fibonacci |
518.436 ns (± 49.0584) |
530.832 ns (± 47.7214) |
0.98 |
ssa_gcd |
460.562 ns (± 31.4093) |
468.824 ns (± 43.3429) |
0.98 |
exec_bc_addOne |
35.05 ns (± 2.98841) |
37.3468 ns (± 8.00576) |
0.94 |
exec_mlir_addOne |
255.114 ns (± 4.8993) |
258.029 ns (± 4.30464) |
0.99 |
exec_cpp_addOne |
3.59918 ns (± 0.390714) |
3.6505 ns (± 0.452295) |
0.99 |
exec_interpreted_addOne |
38.019 ns (± 1.82535) |
38.3209 ns (± 4.07534) |
0.99 |
ir_add |
728.573 ns (± 43.7856) |
772.195 ns (± 71.3318) |
0.94 |
ir_ifThenElse |
1.5266 us (± 278.515) |
1.61196 us (± 182.266) |
0.95 |
ir_deeplyNestedIfElse |
3.32416 us (± 348.072) |
3.38468 us (± 292.836) |
0.98 |
ir_loop |
1.58451 us (± 118.791) |
1.65684 us (± 243.329) |
0.96 |
ir_ifInsideLoop |
2.77388 us (± 266.158) |
2.89326 us (± 320.554) |
0.96 |
ir_loopDirectCall |
1.80664 us (± 167.903) |
1.81889 us (± 168.556) |
0.99 |
ir_pointerLoop |
1.93372 us (± 141.546) |
2.05088 us (± 272.65) |
0.94 |
ir_staticLoop |
1.484 us (± 111.631) |
1.51351 us (± 200.357) |
0.98 |
ir_fibonacci |
1.67708 us (± 115.62) |
1.77363 us (± 155.603) |
0.95 |
ir_gcd |
1.43875 us (± 76.5798) |
1.56065 us (± 170.558) |
0.92 |
ir_nestedIf10 |
7.55705 us (± 358.199) |
8.02456 us (± 1.15304) |
0.94 |
ir_nestedIf100 |
86.4448 us (± 4.63005) |
87.6189 us (± 6.58879) |
0.99 |
ir_chainedIf10 |
11.5731 us (± 672.134) |
11.6437 us (± 1.02792) |
0.99 |
ir_chainedIf100 |
162.859 us (± 6.6144) |
164.697 us (± 8.40094) |
0.99 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RuntimeException so failures are caught in release builds.
nautilus_cuda_checkthat wraps cudaGetLastError /cudaDeviceSynchronize after kernel launches when the new
gpu.cuda.errorCheckoption is set; default off keeps reference codegenbyte-identical.
exactly three arguments instead of indexing past the end of the vector.
common install paths; allow
gpu.cuda.nvccPathoverride.gpu.cuda.arch,gpu.cuda.optLevel(default -O3),gpu.cuda.fastMath,gpu.cuda.debug(-g -G -lineinfo),gpu.cuda.extraFlags.metacharacters can't break the command.
compilations of the same IR id don't clobber each other.
reproduce; opt-out via
gpu.cuda.keepArtifacts=false.already does it; remove the redundant double-unlink.
the rest of the codebase.