bootstrap_impl=script breaks subprocess.run #3187
Description
Is this a regression?
Compared to the flag not being flipped, yes. Compared to previous versions, I don't know.
Description
I recently tried enabling --@rules_python//python/config_settings:bootstrap_impl=script in https://github.com/wpilibsuite/allwpilib/tree/2027. When I did that, I got a rather nasty failure.
ERROR: /home/austin/local/allwpilib3/hal/BUILD.bazel:36:13: Action hal/_gendir failed: (Exit 1): linux-sandbox failed: error executing Action command
(cd /dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__ && \
exec env - \
TMPDIR=/tmp \
/home/austin/.cache/bazel/_bazel_austin/install/0fdbfcdc2c6c2270badcaff8eec2742a/linux-sandbox -t 15 -w /dev/shm -w /dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__ -w /tmp -M /dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/_hermetic_tmp -m /tmp -S /dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/stats.out -N -D /dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/debug.out -- bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb --output_directory bazel-out/k8-opt/bin/hal/_gendir/main/native/cpp/mrc/protobuf --nanopb bazel-out/k8-opt-exec-ST-a828a81199fe/bin/wpiutil/nanopb_generator)
/tmp/tmp.ItiQOKZwcE/bin/python3: can't open file '/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/wpiutil/_nanopb_generator_stage2_bootstrap.py': [Errno 2] No such file or directory
Traceback (most recent call last):
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/_generate_nanopb_stage2_bootstrap.py", line 444, in <module>
main()
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/_generate_nanopb_stage2_bootstrap.py", line 438, in main
_run_py_path(main_filename, args=sys.argv[1:])
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/_generate_nanopb_stage2_bootstrap.py", line 270, in _run_py_path
runpy.run_path(main_filename, run_name="__main__")
File "/home/austin/.cache/bazel/_bazel_austin/2acd1000f84633b416b0e8cbeff6e0aa/execroot/__main__/external/python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/home/austin/.cache/bazel/_bazel_austin/2acd1000f84633b416b0e8cbeff6e0aa/execroot/__main__/external/python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/austin/.cache/bazel/_bazel_austin/2acd1000f84633b416b0e8cbeff6e0aa/execroot/__main__/external/python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/generate_nanopb.py", line 87, in <module>
main()
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/generate_nanopb.py", line 83, in main
generate_nanopb(args.nanopb, args.output_directory, args.proto_directory)
File "/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/hal/generate_nanopb.runfiles/__main__/hal/generate_nanopb.py", line 22, in generate_nanopb
subprocess.run(
File "/home/austin/.cache/bazel/_bazel_austin/2acd1000f84633b416b0e8cbeff6e0aa/execroot/__main__/external/python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bazel-out/k8-opt-exec-ST-a828a81199fe/bin/wpiutil/nanopb_generator', '-I/home/austin/local/allwpilib3/hal/src/mrc/proto', '-D/dev/shm/bazel-sandbox.4d3c097c0a97f2da7a46adb4ecfab883763059ef5c0a7cd7d6c5786cd12fe713/linux-sandbox/2005/execroot/__main__/bazel-out/k8-opt/bin/hal/_gendir/main/native/cpp/mrc/protobuf', '-S.cpp', '-e.npb', PosixPath('/home/austin/local/allwpilib3/hal/src/mrc/proto/MrcComm.proto')]' returned non-zero exit status 2.
The code in question is a genrule. In that genrule, a py_binary takes a command line argument to another py_binary. The first py_binary uses subproces.run to call the second py_binary. Without the bootstrap_impl flag flipped, it correctly works. With it flipped, it fails.
🌍 Your Environment
Operating System:
Debian Bookworm (x86_64)
Output of bazel version:
$ bazel version
Build label: 8.3.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Jun 30 16:23:40 2025 (1751300620)
Build timestamp: 1751300620
Build timestamp as int: 1751300620
Rules_python version:
1.4.1
Anything else relevant?
The following patch fixed it for me:
diff --git a/wpimath/generate_nanopb.py b/wpimath/generate_nanopb.py
index d8bf6d5b7..a2c39d5b1 100755
--- a/wpimath/generate_nanopb.py
+++ b/wpimath/generate_nanopb.py
@@ -21,14 +21,14 @@ def generate_nanopb(nanopb: Path, output_directory: Path, proto_dir: Path):
absolute_filename = path.absolute()
subprocess.check_call(
[
nanopb,
f"-I{absolute_filename.parent}",
f"-D{output_directory.absolute()}",
"-S.cpp",
"-e.npb",
absolute_filename,
- ]
+ ],
+ env={"TMPDIR": os.environ["TMPDIR"]},
)
java_files = (output_directory).glob("*")
for java_file in java_files:
I'm pretty sure the RUNFILES_DIR variable is ending up being set from the calling bootstrap script instead of the one being executed, which results in it being wrong, and unable to find the bootstrap script. The python bootstrap apparently doesn't have this issue.