Porting micropython : close all open files.

[mibi88]

I help to port MicroPython to CASIO calculators with gint, but MicroPython is not closing files correctly and that is causing file locks. I want to now if there is a function that can close all open files ?

[robert-hh]

Do you know, which files stay open?

[mibi88]

The .py files imported as modules.

[jimmo]

@mibi88 Sorry about the delay responding...

It's impossible to provide more information without some more details. How does the port work, is it available (e.g. on github). Is this the official one that Casio ship with their calculators?

Do you use the VFS, etc?

[mibi88]

No it's not the official port. It's a port that's not official : https://gitea.planet-casio.com/Lephenixnoir/PythonExtra
The goal of this port is to add some functionalities that the official one hasn't.
We use the standard functions to manage files, and I need a function that closes all open files, that I want to call when the user exits the add-in, because they are not closed so it causes file locks and that's annoying because I need to connect the calculator to the computer to unlock the files.
Sorry for my bad english.

[jimmo]

Your file objects should implement finalisers, and then when your add-in exits, you can call gc_sweep_all() to empty the heap and run all finalisers.

See for example how this is done for the FAT filesystem in MicroPython -- https://github.com/micropython/micropython/blob/master/extmod/vfs_fat_file.c#L166 and https://github.com/micropython/micropython/blob/master/extmod/vfs_fat_file.c#L137

[lephe]

Hi, I'm the main developer for that port (and the unikernel it's using). Currently we use the POSIX reader to access .py files imported as modules, and we haven't enabled open() yet. AFAIK this should close all files automatically, at least during normal execution. Is there any documentation about the timing of module files being closed by MicroPython (after parsing, after compiling, after running...)?

A possible culprit is that the calculator's OS design allows users to leave the application any time they have keyboard input, including mid-execution. In addition, the OS will not close open files automatically, and leaving them open causes issues. We have control of the execution when such an early exit happens, but we don't know whether MicroPython has facilities to close all files early or where in MicroPython code we could hook that feature. Note that we could be missing a finalization mechanism (eg. an __attribute__((destructor)) function) since the standard runtime is, you guessed it, custom-made (the OS doesn't provide any).

[jimmo]

Sorry I'm not really sure I quite understand. The only way that a file will be open is if it was opened from Python and the Python code hasn't yet closed it. But a gc_sweep_all() will handle this via the posix file type's finaliser.

As far as files opened for loading .py files (e.g. for import) via the POSIX reader, the reader will be automatically closed at the end of parsing, which closes the underlying file descriptor.

mp_parse -> mp_lexer_free -> lex->reader.close -> mp_reader_posix_close -> close(fd)

[lephe]

It's been a while, but since this issue has come up again I wanted to leave a trace here for future Internet users.

The problem actually occurred specifically when an error was raised during an import. I found that mp_parse() did not free the lexer in this case, which has been fixed a few months ago by 5e122b1. Question answered, I believe. :)