Enables caching of the compiled code if tmp_dir is set.

[moorepants]

Fixes #107

A hash of the unique evaluation code is stored in its C file and the files present in the tmp_dir are checked to see if they contain the hash. If one does, then that compiled module is loaded if possible. This saves having to recompile the C code if the differential algebraic equations passed to Problem or ConstraintCollocator have not changed.

TODO:

• Add some unit tests.
• Add documentation of the feature.
• Include it in an example that has more than one execution of Problem. can add later with human gait update
• Handle parallel vs not, as that is not present in the eval_code string.
• Deal with const being populated or not.

[moorepants]

I manually tested some with the gait2d model and the Jacobian function which has about 4000 subexpressions seems to generate different hash sometimes, i.e. it doesn't seem deterministic.

[moorepants]

Maybe our _forward_jacobian function is not deterministic.

[moorepants]

Maybe it would be deterministic if I hash the SymPy equations of motion: hash(s.equations_of_motion.as_immutable()).

EDIT: hash() is probably not repeatable across new interpreters.

[moorepants]

The generated code for the Jacobian has slight differences, maybe in the ordering of the subexpressions:

$ diff --color codegen/ufuncify_matrix_1_c.c codegen/ufuncify_matrix_3_c.c 
1c1
< // opty_code_hash=3fc4ac84638576bd605a5b8df0c14c2caf3301beb7b6bdd514f5301dfcdfc966
---
> // opty_code_hash=0144e43378d3a13238d5bfacb371d6d8a917688d8992924fbcd900ddb150f83f
4c4
< #include "ufuncify_matrix_1_h.h"
---
> #include "ufuncify_matrix_3_h.h"
554,556c554,556
<     double z1171_ = mc_ + md_;
<     double z1176_ = mb_*xb_*z802_ - mb_*yb_*z824_ + mc_*z1111_ + md_*z1124_ + z1169_*z1171_;
<     double z1177_ = mb_*xb_*z803_ - mb_*yb_*z825_ + mc_*z1112_ + md_*z1125_ + z1170_*z1171_;
---
>     double z1172_ = mc_ + md_;
>     double z1176_ = mb_*xb_*z802_ - mb_*yb_*z824_ + mc_*z1111_ + md_*z1124_ + z1169_*z1172_;
>     double z1177_ = mb_*xb_*z803_ - mb_*yb_*z825_ + mc_*z1112_ + md_*z1125_ + z1170_*z1172_;
561,563c561,563
<     double z1201_ = mf_ + mg_;
<     double z1205_ = me_*xe_*z818_ - me_*ye_*z830_ + mf_*z1141_ + mg_*z1154_ + z1198_*z1201_;
<     double z1206_ = me_*xe_*z819_ - me_*ye_*z831_ + mf_*z1142_ + mg_*z1155_ + z1199_*z1201_;
---
>     double z1200_ = mf_ + mg_;
>     double z1205_ = me_*xe_*z818_ - me_*ye_*z830_ + mf_*z1141_ + mg_*z1154_ + z1198_*z1200_;
>     double z1206_ = me_*xe_*z819_ - me_*ye_*z831_ + mf_*z1142_ + mg_*z1155_ + z1199_*z1200_;

[moorepants]

cse has this kwarg but we are just using the default:

    order : string, 'none' or 'canonical'
        The order by which Mul and Add arguments are processed. If set to
        'canonical', arguments will be canonically ordered. If set to 'none',
        ordering will be faster but dependent on expressions hashes, thus
        machine dependent and variable. For large expressions where speed is a
        concern, use the setting order='none'.

[moorepants]

I think the only unit test I can write for this is to run Problem twice with tmp_dir and make sure only two sets of Cython files are ecreated. It is difficult to test anything else given that it falls back to compiling.

[moorepants]

I get this error on the path test:

>       assert os.path.exists(os.path.join(tmp_dir, 'ufuncify_matrix_0_c.c'))
E       AssertionError: assert False
E        +  where False = <function exists at 0x10103e980>('/var/folders/xc/cl1fyykn2pj4ryhdw6r1mqtc0000gn/T/tmpettkbpazopty_cache_test/ufuncify_matrix_0_c.c')
E        +    where <function exists at 0x10103e980> = <module 'posixpath' (frozen)>.exists
E        +      where <module 'posixpath' (frozen)> = os.path
E        +    and   '/var/folders/xc/cl1fyykn2pj4ryhdw6r1mqtc0000gn/T/tmpettkbpazopty_cache_test/ufuncify_matrix_0_c.c' = <function join at 0x10103f600>('/var/folders/xc/cl1fyykn2pj4ryhdw6r1mqtc0000gn/T/tmpettkbpazopty_cache_test', 'ufuncify_matrix_0_c.c')
E        +      where <function join at 0x10103f600> = <module 'posixpath' (frozen)>.join
E        +        where <module 'posixpath' (frozen)> = os.path

I'm confused why it says False = <function...

[moorepants]

The issue is that in the pytest runs the module counter has already been advanced:

opty/tests/test_direct_collocation.py:447: AssertionError
----------------------------- Captured stdout call -----------------------------
['ufuncify_matrix_5_setup.py', 'ufuncify_matrix_4_setup.py', 'ufuncify_matrix_5.cpython-313-darwin.so', 'ufuncify_matrix_5_h.h', 'ufuncify_matrix_4_h.h', 'ufuncify_matrix_4_c.c', 'ufuncify_matrix_4.c', 'ufuncify_matrix_5_c.c', 'ufuncify_matrix_4.cpython-313-darwin.so', 'ufuncify_matrix_5.c', 'ufuncify_matrix_4.pyx', 'build', 'ufuncify_matrix_5.pyx']

[moorepants]

Over in gait-closed-loop-id I'm getting a failed to import even though it finds a match:

08:31:53 INFO utils - ufuncify_matrix: Changed directory to /home/moorepants/src/gait-closed-loop-id/gait_codegen
08:31:53 INFO utils - ufuncify_matrix: Checking ufuncify_matrix_4 for cached code.
08:31:53 INFO utils - ufuncify_matrix: Checking ufuncify_matrix_3 for cached code.
08:31:53 INFO utils - ufuncify_matrix: Checking ufuncify_matrix_2 for cached code.
08:31:53 INFO utils - ufuncify_matrix: Checking ufuncify_matrix_1 for cached code.
08:31:53 INFO utils - ufuncify_matrix: Checking ufuncify_matrix_0 for cached code.
08:31:53 INFO utils - ufuncify_matrix: ufuncify_matrix_0 matches!
08:31:53 INFO utils - ufuncify_matrix: Failed to import ufuncify_matrix_0.
08:31:53 INFO utils - ufuncify_matrix: Compiling matrix evaluation.

[moorepants]

Interesting if I open IPython and run src/solve.py the module loads but if I python src/solve.py it fails to load.

[moorepants]

I also tried cd src and python solve.py and it failed to load the existing modules.

[moorepants]

ipython src/solve.py works.

[moorepants]

Maybe ipython does something different with the working directory: https://stackoverflow.com/questions/77767321/importing-module-works-with-ipython-but-it-doesnt-if-im-not-using-ipython

[moorepants]

Also: https://www.devasking.com/issue/import-fails-when-running-python-as-script-but-not-in-ipython

[moorepants]

Interesting I have the opposite result here: #109 (comment)