Use `shell=False` for `Popen` on Windows #1324

michaelosthege · 2025-03-27T16:48:03Z

Description

I'm having hard times creating new environments with working compiler dependencies.

My debugging attempts show that shell=False could fix this, however this observation contradicts the comments next to that line.
~~Further testing may be required.~~ Tested on another machine.

I also removed an old workaround for Python 2..

Related Issue

Hints?

Checklist

Checked that the pre-commit linting/style checks pass
Included tests that prove the fix is effective or that the new feature works
~~Added necessary documentation (docstrings and/or example notebooks)~~
If you are a pro: each commit corresponds to a relevant logical change

Type of change

📚 Documentation preview 📚: https://pytensor--1324.org.readthedocs.build/en/1324/

codecov · 2025-03-27T17:31:20Z

Codecov Report

Attention: Patch coverage is 85.91549% with 10 lines in your changes missing coverage. Please review.

Project coverage is 82.08%. Comparing base (f1514eb) to head (fa88367).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
pytensor/link/c/cmodule.py	85.91%	7 Missing and 3 partials ⚠️

❌ Your patch check has failed because the patch coverage (85.91%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1324      +/-   ##
==========================================
+ Coverage   82.05%   82.08%   +0.02%     
==========================================
  Files         203      206       +3     
  Lines       48863    49171     +308     
  Branches     8695     8718      +23     
==========================================
+ Hits        40093    40360     +267     
- Misses       6619     6654      +35     
- Partials     2151     2157       +6

Files with missing lines	Coverage Δ
pytensor/link/c/cmodule.py	`61.10% <85.91%> (+0.22%)`	⬆️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

michaelosthege · 2025-03-28T08:55:14Z

@Armavica you also worked on these components - what do you think about shell=False? Does it work on your machine?

Armavica · 2025-03-28T11:23:00Z

I will have a look this weekend or next week

michaelosthege · 2025-04-08T18:44:07Z

I will have a look this weekend or next week

@Armavica did you already get a chance?

Armavica · 2025-04-10T20:48:48Z

I don't think I worked specifically on this besides cosmetic changes, and I never tried to install PyTensor on Windows.
However, I think if we can make things work with shell=False it's probably for the best. Have you tried without the list to string conversion that follows, that is supposed to be only needed for shell=True?

The backslashes in `__file__` created invalid escape sequences.

michaelosthege · 2025-04-12T14:31:08Z

Have you tried without the list to string conversion that follows

Without the list to string conversion already the first test failed.

ricardoV94 · 2025-04-18T20:32:44Z

Should we have a try/except where this is called that tries the shell trick if it fails without it, just in case this is actually doing something for someone out there?

michaelosthege · 2025-04-18T21:01:15Z

I don't think that's needed. To me it looks like the recent changes to compilation dependencies (switching to g++, no longer using mkl-toolchain and so on) simply don't work with shell=True.

And this is only going to affect future versions which will be in that new configuration..

ricardoV94 · 2025-04-18T21:08:35Z

We didn't switch to g++ recently or anything like that.

michaelosthege · 2025-04-18T21:20:50Z

We didn't switch to g++ recently or anything like that.

I'm mean this: f37380f

Related conda-forge/pytensor-suite-feedstock#133 (comment)

ricardoV94 · 2025-04-18T21:40:26Z

What about non-conda users?

michaelosthege · 2025-04-19T07:33:36Z

Do you have a specific scenario in mind?

~~I can also introduce a subprocess_shell config variable.~~

michaelosthege · 2025-04-21T18:07:51Z

can we get to a conclusion on this one?

from my point of view this change fixes all Windows setups I can come up with; all of which were broken before..
If there are other relevant setups we should test, please let me know.

jessegrabowski · 2025-04-21T18:21:43Z

It all seems good to me, but I'm gun shy about approving since I haven't touched this part of the codebase before. You're saying the recent changes to the tooling makes that comment about the conda g++.batobsolete? @maresb might also weigh in, since he knows a lot about the conda tooling stuff

That's the only possible definition, and the type checker is freaking out.

Convert lines when `parse=False` from generator to list. The result when `parse=False` is only used in a warning message, and I don't think the generator would even print properly.

...in print statements

maresb · 2025-04-22T01:55:53Z

I took this further because halfway workarounds are very difficult for me to reason about.

Have you tried without the list to string conversion that follows

Without the list to string conversion already the first test failed.

Is this in reference to the following?

pytensor/pytensor/utils.py

Lines 140 to 145 in 2928211

    
           # "If shell is True, it is recommended to pass args as a string rather than as a sequence." (cite taken from https://docs.python.org/2/library/subprocess.html#frequently-used-arguments) 
        
           # In case when command arguments have spaces, passing a command as a list will result in incorrect arguments break down, and consequently 
        
           # in "The filename, directory name, or volume label syntax is incorrect" error message. 
        
           # Passing the command as a single string solves this problem. 
        
           if isinstance(command, list): 
        
               command = " ".join(command)  # type: ignore[assignment]

I'd love to get rid of that too, but I'm not quite feeling that motivated.

maresb

@michaelosthege maybe a quick glance at my commits?

(Please ping me in case of CI failure.)

michaelosthege · 2025-04-22T08:05:11Z

Thanks @maresb, your commits look good!
I hesitated about that part of the code, but it looks like you brought the right tools to the party.

Running the tests now..

maresb · 2025-04-22T08:44:45Z

Hmm, this may be an important occasion to make codecov happy. This is a situation where the tests are basically the only indication we're not breaking stuff.

Just about to take off now though. ✈️

michaelosthege · 2025-04-22T12:29:23Z

Unsurprisingly I get failing tests when running things locally (even pip install -e . doesn't work because I can't compile the scan module..), but with this branch it's better than on main, so that's nice.

91 failed, 8397 passed, 455 skipped, 91 xfailed, 2704 warnings, 40 errors in 864.99s

List of failing tests

FAILED tests/compile/test_mode.py::test_get_target_language - AssertionError: assert ('py',) == ('c', 'py')
FAILED tests/compile/test_shared.py::TestSharedVariable::test_ctors - AssertionError: TensorType(int64, shape=())
FAILED tests/link/c/test_cmodule.py::test_compiler_error - pytensor.link.c.exceptions.MissingGXX: g++ not available! We can't compile c code.
FAILED tests/link/c/test_cmodule.py::test_cache_versioning - UserWarning: `pytensor.config.cxx` is not an identifiable `g++` compiler. PyTensor will disable compiler optimizations specific to `g++`. At worst, this c...
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Working_CXX-Linux] - AssertionError: assert {''} == {'-liomp5', '..., '-lpthread'}    
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Working_CXX-Windows] - AssertionError: assert {''} == {'-liomp5', '..., '-lpthread'}  
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Working_CXX-Darwin] - AssertionError: assert {''} == {'-liomp5', '..., '-lpthread'}   
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_intel-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags_conda_windows[mkl_intel] - assert {''} == {'-L"C:\\User..., '-lpthread'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Working_CXX-Linux] - AssertionError: assert {''} == {'-lgomp', '-..., '-lpthread'}      
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Working_CXX-Windows] - AssertionError: assert {''} == {'-lgomp', '-..., '-lpthread'}    
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Working_CXX-Darwin] - AssertionError: assert {''} == {'-lgomp', '-..., '-lpthread'}     
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[mkl_gnu-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) 
were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags_conda_windows[mkl_gnu] - assert {''} == {'-L"C:\\User..., '-lpthread'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[accelerate-Working_CXX-Darwin] - AssertionError: assert {''} == {'-framework', 'Accelerate'}    
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[accelerate-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[accelerate-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[accelerate-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Working_CXX-Linux] - AssertionError: assert {''} == {'-fopenmp', ... '-lopenblas'}     
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Working_CXX-Windows] - AssertionError: assert {''} == {'-fopenmp', ... '-lopenblas'}   
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Working_CXX-Darwin] - AssertionError: assert {''} == {'-fopenmp', ... '-lopenblas'}    
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) 
were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[openblas-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags_conda_windows[openblas] - assert {''} == {'-L"C:\\User... '-lopenblas'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Working_CXX-Linux] - AssertionError: assert {''} == {'-lblas', '-...apack', '-lm'}       
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Working_CXX-Windows] - AssertionError: assert {''} == {'-lblas', '-...apack', '-lm'}     
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Working_CXX-Darwin] - AssertionError: assert {''} == {'-lblas', '-...apack', '-lm'}      
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) 
were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[lapack-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags_conda_windows[lapack] - assert {''} == {'-L"C:\\User...apack', '-lm'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Working_CXX-Linux] - AssertionError: assert {''} == {'-lblas', '-lcblas'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Working_CXX-Windows] - AssertionError: assert {''} == {'-lblas', '-lcblas'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Working_CXX-Darwin] - AssertionError: assert {''} == {'-lblas', '-lcblas'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[blas-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags_conda_windows[blas] - assert {''} == {'-L"C:\\User...s', '-lcblas'}
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[no_blas-Broken_CXX-Linux] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[no_blas-Broken_CXX-Windows] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
FAILED tests/link/c/test_cmodule.py::test_default_blas_ldflags[no_blas-Broken_CXX-Darwin] - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) 
were emitted.
FAILED tests/link/c/test_cmodule.py::test_cache_race_condition - AttributeError: Can't get local object 'test_cache_race_condition.<locals>.f_build'
FAILED tests/link/c/test_op.py::test_ExternalCOp_c_code_cache_version - assert 2 == 0
FAILED tests/link/c/test_type.py::TestEnumTypes::test_op_with_cenumtype_debug - NotImplementedError
FAILED tests/link/test_vm.py::test_partial_function[cvm] - pytensor.link.c.exceptions.MissingGXX: lazylinker will not be imported if pytensor.config.cxx is not set.
FAILED tests/link/test_vm.py::test_partial_function_with_output_keys[cvm] - TypeError: Loop.__call__() got an unexpected keyword argument 'output_subset'      
FAILED tests/link/test_vm.py::test_partial_function_with_updates[cvm] - TypeError: Loop.__call__() got an unexpected keyword argument 'output_subset'
FAILED tests/scalar/test_math.py::test_gammainc_nan_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammainc(ScalarFromTensor(<Scalar(float64, shape=())...
FAILED tests/scalar/test_math.py::test_gammainc_inf_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammainc(ScalarFromTensor(<Scalar(float64, shape=())...
FAILED tests/scalar/test_math.py::test_gammaincc_nan_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammaincc(ScalarFromTensor(<Scalar(float64, shape=()...
FAILED tests/scalar/test_math.py::test_gammaincc_inf_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammaincc(ScalarFromTensor(<Scalar(float64, shape=()...
FAILED tests/scalar/test_math.py::test_gammal_nan_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammal(ScalarFromTensor(<Scalar(float64, shape=())>)...
FAILED tests/scalar/test_math.py::test_gammau_nan_c - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(gammau(ScalarFromTensor(<Scalar(float64, shape=())>)...
FAILED tests/scalar/test_math.py::test_betainc[c] - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(betainc(ScalarFromTensor(a), ScalarFromTensor(b), Sc...
FAILED tests/scan/test_basic.py::test_output_storage_reuse[cvm] - ModuleNotFoundError: No module named 'pytensor.scan.scan_perform'
FAILED tests/scan/test_rewriting.py::TestSaveMem::test_while_scan_taps - OverflowError: Python integer 1000 out of bounds for int8
FAILED tests/scan/test_rewriting.py::TestSaveMem::test_while_scan_taps_and_map - OverflowError: Python integer 200 out of bounds for int8
FAILED tests/tensor/rewriting/test_basic.py::TestLocalCanonicalizeAlloc::test_inconsistent_shared[True] - AssertionError: Regex pattern did not match.
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-None-floatX-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-None-int32-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-0-floatX-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-0-int32-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-1-floatX-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-1-int32-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-axis3-floatX-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-axis3-int32-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-axis4-floatX-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/rewriting/test_elemwise.py::TestFusion::test_CAReduce_single_input[sum-sum-axis4-int32-cvm] - ValueError: too many values to unpack (expected 1)
FAILED tests/tensor/test_basic.py::TestAlloc::test_alloc_of_view_linker - UserWarning: `pytensor.config.cxx` is not an identifiable `g++` compiler. PyTensor will disable compiler optimizations specific to `g++`. At worst, this c...zations specific to `g++`. At worst, this c...                                                                                                                                                                     ++`. At worst, this c...
FAILED tests/tensor/test_basic.py::TestAlloc::test_runtime_broadcast[mode1] - UserWarning: `pytensor.config.cxx` is not an identifiable `g++` compiler. PyTensor will disable compiler optimizations specific to `g At worst, this c...timizations specific to `g++`. At worst, this c...
FAILED tests/tensor/test_basic.py::TestJoinAndSplit::test_split_view[c] - UserWarning: `pytensor.config.cxx` is not an identifiable `g++` compiler. PyTensor will disable compiler optimizations specific to `g++`.zations specific to `g++`. At worst, this c...                                                                                                                                                                     e=(?,))>))')
FAILED tests/tensor/test_basic.py::TestLongTensor::test_fit_int64 - AssertionError: assert 'int64' == 'int32'                                                                                                      Use `specify_broadcastable`...     
FAILED tests/tensor/test_blas.py::test_batched_dot_blas_flags - AssertionError: assert False
FAILED tests/tensor/test_elemwise.py::TestDimShuffle::test_c_views - pytensor.link.c.exceptions.MissingGXX: ("g++ not available! We can't compile c code.", 'FunctionGraph(ExpandDims{axis=0}(<Vector(float64, shaps=0}(<Vector(float64, shape=(?,))>))')
FAILED tests/tensor/test_extra_ops.py::test_broadcast_shape_symbolic_one_symbolic - AssertionError: Could not broadcast dimensions. Broadcasting is only allowed along axes that have a statically known length 1. tatically known length 1. Use `specify_broadcastable`...
FAILED tests/tensor/test_interpolate.py::test_interpolate_scalar_extrapolate[linear] - TypeError: 'numpy.int64' object does not support item assignment
FAILED tests/tensor/test_interpolate.py::test_interpolate_scalar_extrapolate[nearest] - TypeError: 'numpy.int64' object does not support item assignment
FAILED tests/tensor/test_interpolate.py::test_interpolate_scalar_extrapolate[first] - TypeError: 'numpy.int64' object does not support item assignment
FAILED tests/tensor/test_interpolate.py::test_interpolate_scalar_extrapolate[last] - TypeError: 'numpy.int64' object does not support item assignment                                                              .]], [[0.]], [[-...
FAILED tests/tensor/test_interpolate.py::test_interpolate_scalar_extrapolate[mean] - TypeError: 'numpy.int64' object does not support item assignment
FAILED tests/tensor/test_math.py::TestInvBroadcast::test_grad - pytensor.gradient.GradientError: GradientError: numeric gradient and analytic gradient exceed tolerance:
FAILED tests/tensor/test_math.py::TestMinMax::test_uint[uint64]
FAILED tests/tensor/test_math.py::TestMinMax::test_uint64_special_value
FAILED tests/tensor/test_math_scipy.py::TestHyp2F1Broadcast::test_grad - AttributeError: ("'Scratchpad' object has no attribute 'ufunc'\nApply node that caused the error: Scalarloop(Composite{...}.3, [[0.]], [[0posite{...}.3, [[0.]], [[0.]], [[0.]], [[-...
FAILED tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_hyp2f1_grad_stan_cases - AttributeError: 'Scratchpad' object has no attribute 'ufunc'
FAILED tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_unused_grad_loop_opt[wrt6] - ValueError: not enough values to unpack (expected 2, got 1)
ERROR tests/compile/function/test_types.py::test_minimal_random_function_call_benchmark[True]
ERROR tests/compile/function/test_types.py::test_minimal_random_function_call_benchmark[False]
ERROR tests/graph/test_basic.py::TestTruncatedGraphInputs::test_single_pass_per_node
ERROR tests/scan/test_basic.py::TestExamples::test_reordering
ERROR tests/scan/test_basic.py::TestExamples::test_scan_as_tensor_on_gradients
ERROR tests/scan/test_basic.py::TestExamples::test_multiple_outs_taps
ERROR tests/scan/test_rewriting.py::TestPushOutAddScan::test_pregreedy_optimizer
ERROR tests/scan/test_rewriting.py::TestSaveMem::test_savemem_opt
ERROR tests/tensor/rewriting/test_elemwise.py::TestFusion::test_eval_benchmark
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:()-mu:()]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:()-mu:(1000,)]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:()-mu:(4, 1000)]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(1000,)-mu:()]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(1000,)-mu:(1000,)]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(1000,)-mu:(4, 1000)]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(4, 1000)-mu:()]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(4, 1000)-mu:(1000,)]
ERROR tests/tensor/test_blockwise.py::test_batched_mvnormal_logp_and_dlogp[cov:(4, 1000)-mu:(4, 1000)]
ERROR tests/tensor/test_elemwise.py::TestDimShuffle::test_benchmark[True]
ERROR tests/tensor/test_elemwise.py::TestDimShuffle::test_benchmark[False]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=0]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=1]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=2]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=(0, 1)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=(0, 2)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=(1, 2)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=True-axis=None]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=0]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=1]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=2]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=(0, 1)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=(0, 2)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=(1, 2)]
ERROR tests/tensor/test_elemwise.py::test_c_careduce_benchmark[c_contiguous=False-axis=None]
ERROR tests/tensor/test_math_scipy.py::test_gammaincc_ddk_performance
ERROR tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_benchmark[a-case0]
ERROR tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_benchmark[a-case1]
ERROR tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_benchmark[all-case0]
ERROR tests/tensor/test_math_scipy.py::TestHyp2F1Grad::test_benchmark[all-case1]
ERROR tests/tensor/test_shape.py::TestReshape::test_benchmark

They could get tested independently. This was mostly done to understand their purpose.

Such that its name can be more informative.

michaelosthege · 2025-04-22T22:43:30Z

Ran a bunch of tests, did some more refactoring and fixed some tests for Windows.

Everything appears to be working fine.

michaelosthege added bug Something isn't working C-backend labels Mar 27, 2025

michaelosthege and others added 2 commits April 12, 2025 15:48

Simplify test to be Windows-compatible

fb8842b

The backslashes in `__file__` created invalid escape sequences.

Remove old workarounds from subprocess_Popen

c78cef1

michaelosthege force-pushed the fix-windows-popen branch from 1c15c12 to c78cef1 Compare April 12, 2025 13:49

michaelosthege marked this pull request as ready for review April 12, 2025 14:00

michaelosthege requested review from Armavica, ricardoV94 and jessegrabowski April 12, 2025 14:31

maresb added 7 commits April 21, 2025 19:37

Define get_lines outside of the if statement

0bf03ce

That's the only possible definition, and the type checker is freaking out.

Add type annotation for get_lines

0083aa1

Clean up a few types

39c8b7c

Convert lines when `parse=False` from generator to list. The result when `parse=False` is only used in a warning message, and I don't think the generator would even print properly.

Use shlex.join for converting commands to strings

b9f13c7

...in print statements

Use lists instead of strings for commands

dc17b91

Tighten type annotations to remove str-based commands

8d7a8bc

Remove superfluous shell=True arguments from subprocess.Popen

2928211

maresb force-pushed the fix-windows-popen branch from 98b81fb to 2928211 Compare April 22, 2025 01:43

maresb approved these changes Apr 22, 2025

View reviewed changes

Extract local functions and type them

d2ef2b0

They could get tested independently. This was mostly done to understand their purpose.

michaelosthege force-pushed the fix-windows-popen branch from b4f7eb3 to 5e03d50 Compare April 22, 2025 21:32

michaelosthege added 2 commits April 22, 2025 23:35

Fix Windows compatibility of some tests

5bcb2ff

Refactor function fixture as parametrize

fa88367

Such that its name can be more informative.

michaelosthege force-pushed the fix-windows-popen branch from 5e03d50 to fa88367 Compare April 22, 2025 21:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `shell=False` for `Popen` on Windows #1324

Use `shell=False` for `Popen` on Windows #1324

michaelosthege commented Mar 27, 2025 •

edited

Loading

codecov bot commented Mar 27, 2025 •

edited

Loading

michaelosthege commented Mar 28, 2025

Armavica commented Mar 28, 2025

michaelosthege commented Apr 8, 2025

Armavica commented Apr 10, 2025 •

edited

Loading

michaelosthege commented Apr 12, 2025

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 18, 2025

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 18, 2025 •

edited

Loading

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 19, 2025 •

edited

Loading

michaelosthege commented Apr 21, 2025

jessegrabowski commented Apr 21, 2025

maresb commented Apr 22, 2025

maresb left a comment •

edited

Loading

michaelosthege commented Apr 22, 2025

maresb commented Apr 22, 2025

michaelosthege commented Apr 22, 2025 •

edited

Loading

michaelosthege commented Apr 22, 2025

Use shell=False for Popen on Windows #1324

Are you sure you want to change the base?

Use shell=False for Popen on Windows #1324

Conversation

michaelosthege commented Mar 27, 2025 • edited Loading

Description

Related Issue

Checklist

Type of change

codecov bot commented Mar 27, 2025 • edited Loading

Codecov Report

michaelosthege commented Mar 28, 2025

Armavica commented Mar 28, 2025

michaelosthege commented Apr 8, 2025

Armavica commented Apr 10, 2025 • edited Loading

michaelosthege commented Apr 12, 2025

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 18, 2025

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 18, 2025 • edited Loading

ricardoV94 commented Apr 18, 2025

michaelosthege commented Apr 19, 2025 • edited Loading

michaelosthege commented Apr 21, 2025

jessegrabowski commented Apr 21, 2025

maresb commented Apr 22, 2025

maresb left a comment • edited Loading

Choose a reason for hiding this comment

michaelosthege commented Apr 22, 2025

maresb commented Apr 22, 2025

michaelosthege commented Apr 22, 2025 • edited Loading

michaelosthege commented Apr 22, 2025

Use `shell=False` for `Popen` on Windows #1324

Use `shell=False` for `Popen` on Windows #1324

michaelosthege commented Mar 27, 2025 •

edited

Loading

codecov bot commented Mar 27, 2025 •

edited

Loading

Armavica commented Apr 10, 2025 •

edited

Loading

michaelosthege commented Apr 18, 2025 •

edited

Loading

michaelosthege commented Apr 19, 2025 •

edited

Loading

maresb left a comment •

edited

Loading

michaelosthege commented Apr 22, 2025 •

edited

Loading