Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emscripten.cmake should set CMAKE_<LANG>_FLAGS_INIT rather than CMAKE_<LANG>_FLAGS #104

Open
tttapa opened this issue Feb 19, 2025 · 3 comments

Comments

@tttapa
Copy link

tttapa commented Feb 19, 2025

From https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_FLAGS_INIT.html:

CMAKE_<LANG>_FLAGS_INIT
Value used to initialize the CMAKE_<LANG>_FLAGS cache entry the first time a build tree is configured for language <LANG>. This variable is meant to be set by a toolchain file. CMake may prepend or append content to the value based on the environment and target platform. For example, the contents of a xxxFLAGS environment variable will be prepended, where xxx will be language-specific but not necessarily the same as <LANG>; (e.g. CXXFLAGS for CXX, FFLAGS for Fortran, and so on).

Setting CMAKE_<LANG>_FLAGS directly breaks compatibility with other toolchain files (such as the ones generated by Conan), and prevents users from appending their own flags to the default ones (https://discourse.cmake.org/t/strictly-appending-to-cmake-lang-flags/6478).

@ryanking13
Copy link
Member

Thanks for the report. That is a good to know. Let me see if that change breaks any existing package.

@ryanking13
Copy link
Member

There were a few test failures when I updated the toolchain file, so I guess this can be a breaking change for a few recipes. I'll take a look at each failures when I have a bandwidth.

Meanwhile, you can overwrite the toolchain file by passing CMAKE_TOOLCHAIN_FILE env variable when running pyodide build-recipes or pyodide build.

@tttapa
Copy link
Author

tttapa commented Feb 21, 2025

Thanks a lot for the quick response!

Tests from two packages seem to be failing:

  1. cpp-exceptions-test:
    There's a problem importing /usr/lib/cpp-exceptions-test-{catch,throw}.so, the result of pyodide._module.LDSO.loadedLibsByName["/usr/lib/cpp-exceptions-test-{catch,throw}.so"] is undefined:
    https://github.com/ryanking13/pyodide/blob/53fbc52ea3e9ae09395fdd1c672f3836eccc47d8/packages/cpp-exceptions-test/test_cpp_exceptions.py#L10
    https://github.com/ryanking13/pyodide/blob/53fbc52ea3e9ae09395fdd1c672f3836eccc47d8/src/tests/test_stack_switching.py#L264
    This error is the cause of all test failures in test_cpp_exceptions.py and test_stack_switching.py.
    The extension modules that are being loaded for these tests are built manually (https://github.com/ryanking13/pyodide/blob/53fbc52ea3e9ae09395fdd1c672f3836eccc47d8/packages/cpp-exceptions-test/meta.yaml#L14-L17), they do not (should not) interact with CMake in any way.
  2. magic:
    python-magic/test_magic.py::magic fails on Chrome with the error
    ModuleNotFoundError: There was a problem with unpickling the return value/exception from your pyodide environment. This usually means the type of the return value does not exist in your host environment. The original message is: No module named 'magic'. I'm not sure what's going on here: magic itself is a pure-Python package, and its only dependency, libmagic, is built using Makefiles, so it shouldn't be affected by a CMake-only change either.

Finally, there's the build failure of SciPy's scipy/sparse/sparsetools/csr.cxx file in build-packages-numpy-dependents, which produces no actual error messages. SciPy uses Meson, not CMake.


The workaround I'm currently using is to have a simple wrapper around pyodide-build's Emscripten.cmake that sets CMAKE_<LANG>_FLAGS_INIT and unsets CMAKE_<LANG>_FLAGS:
https://github.com/tttapa/py-build-cmake-example/blob/45c0ee1f7849dc44f1c37ef64f6c71722085b261/scripts/ci/pyodide.toolchain.cmake
(pyodide.toolchain.cmake is included as a user_toolchain in the actual CMake toolchain file generated by Conan.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants