py35 and py36 fail in pipes

[cbm755]

py27.exe is working fine for me, but the Python 3 version are failing when I run them with popen2.

The relevant part of the error is:

Traceback (most recent call last):
  File "C:\projects\pyexe\pyexe.py", line 1176, in <module>
  File "C:\Python36\lib\os.py", line 1017, in fdopen
ValueError: Must have exactly one of read or write mode
[13316] Failed to execute script pyexe

I don't have a minimum working example yet but it involves talking to py.exe with a pipe. My tests using system() in Octave work fine. But when I open it with popen2, I get this error message.

I can dig deeper later if needed, but thought I'd report first in case its an easy fix.

[cbm755]

FWIW, this comment shows how to use socat to similar the pipes thing. No idea if this is do-able on Windows.

[manthey]

This sounds similar to the change in Python 2 to 3 in how unbuffered stdin/stdout access is performed. In Python2, specifying a buffer size of 0 works to make a file point unbuffered, but using 0 in Python 3 shows a similar error (whereas using a buffer size of 1 works successfully). Does passing an explicit bufsize of a positive number to popen2 work?

Also, I'm having trouble reproducing this, so a minimal example (or even a not-so-minimal example) would help.

[cbm755]

Ok, not so minimal but here goes:

1. Install Octave-4.2.2. Specifically, I think I used octave-4.2.2-w64-installer.exe from https://ftp.gnu.org/gnu/octave/windows/
2. Run Octave (GUI or cmd line)
3. "cd where/ever/pyexe/is/
4. type the following:

[fin, fout, pid] = popen2('py36_v14dev.exe', '-i')

The output for me is:

fin =  4
fout =  3
pid =  14592
>>
>>
>>
>> Traceback (most recent call last):
  File "C:\projects\pyexe\pyexe.py", line 1176, in <module>
  File "C:\Python36\lib\os.py", line 1017, in fdopen
ValueError: Must have exactly one of read or write mode
[8912] Failed to execute script pyexe

(I pressed enter a couple times which made the ">>" prompts but this wasn't necessary.)

I don't know anything about system calls on Windows so I'm not how hard it is to test with a real popen2 (in C). Or maybe we could try reproducing with Python2's deprecated popen2 command...

[cbm755]

Does passing an explicit bufsize of a positive number to popen2 work?

Unfortunately, I don't think I have direct control of that in Octave.

[manthey]

I can reproduce with Octave (but not with python 2's popen2). So far I've managed to change the error pyexe outputs, but not avoid it. I can get a similar error with something like [in, out, pid] = popen2("\\python36\\python", {"\\temp\\pyexe.py", "-i"});, where pyexe.py is an output artifact from the appveyor build (with some fussing). I'm hoping if I can figure out how to workaround that, it will apply to the built exe.

[cbm755]

In case it matters, Its possible Octave uses its own popen2 implementation on Windows (rather than a POSIX system call). This stuff is in oct-syscalls.cc and wrappers/octave-popen2.c

None of this is urgent from my point of view: I can simply ship py27.exe.