predict_version_str() not general enough for setup.py #33

jhgoebbert · 2024-10-31T16:46:23Z

Hi,

if I try to install ebrains-drive from source it fails in setup.py->__version__ = predict_version_str() because the version cannot be found by version_query->query_package_folder() here:
https://github.com/mbdevpl/version-query/blob/v1.5.5/version_query/py_query.py#L38

As the egg-info is located at <...>/ebrainsdrive/ebrains-drive-0.6.0/ebrains_drive.egg-info
my assumption is here that ebrains-drive-0.6.0 is written with - and ebrains_drive.egg-info with _.

This is related to mbdevpl/version-query#2

Details:
I install ebrains-drive from the source on pypi using the following command

python -m pip install --prefix=/p/usersoftware/swmanage/goebbert1/stage2024/jupyterlab-test/easybuild/jusuf/software/tvb-ext-bucket/2.0.0-GCCcore-12.3.0  --no-deps  --ignore-installed  --no-index  --no-build-isolation  .

It fails with in __version__ = predict_version_str() from here https://github.com/HumanBrainProject/ebrains-storage/blob/master/setup.py#L4 with

        File "/p/usersoftware/swmanage/goebbert1/stage2024/jupyterlab-test/easybuild/jusuf/software/tvb-ext-bucket/2.0.0-GCCcore-12.3.0/lib/python3.11/site-packages/version_query/query.py", line 36, in query_folder
          return query_package_folder(path, search_parent_directories=search_parent_directories)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/p/usersoftware/swmanage/goebbert1/stage2024/jupyterlab-test/easybuild/jusuf/software/tvb-ext-bucket/2.0.0-GCCcore-12.3.0/lib/python3.11/site-packages/version_query/py_query.py", line 44, in query_package_folder
          raise ValueError(paths, metadata_json_paths, pkg_info_paths)
      ValueError: ([PosixPath('/dev/shm/goebbert1/jusuf/tvbextbucket/2.0.0/GCCcore-12.3.0/ebrainsdrive/ebrains-drive-0.6.0'), PosixPath('/dev/shm/goebbert1/jusuf/tvbextbucket/2.0.0/GCCcore-12.3.0/ebrainsdrive'), PosixPath('/dev/shm/goebbert1/jusuf/tvbextbucket/2.0.0/GCCcore-12.3.0'), PosixPath('/dev/shm/goebbert1/jusuf/tvbextbucket/2.0.0'), PosixPath('/dev/shm/goebbert1/jusuf/tvbextbucket'), PosixPath('/dev/shm/goebbert1/jusuf'), PosixPath('/dev/shm/goebbert1'), PosixPath('/dev/shm'), PosixPath('/dev'), PosixPath('/')], [], [])
      [end of output]

The text was updated successfully, but these errors were encountered:

liadomide · 2024-11-04T21:17:32Z

I confirm this issue also, when trying to install from Pypi ebrains-drive-0.6.0.tar.gz
when installing from the .whl file it works

mbdevpl · 2025-02-16T08:37:44Z

I get a different error:

First, I run this to get a 100% clean environment:

docker run -it python:3.13 bash

then, inside the above container, this to install ebrains-drive from source (i.e. tar.gz):

pip install --no-binary ebrains-drive ebrains-drive==0.6.0

Collecting ebrains-drive==0.6.0
  Downloading ebrains-drive-0.6.0.tar.gz (21 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
          ~~~~^^
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-dtd18g4e/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dtd18g4e/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
          ~~~~~~~~~~~~~~^^
        File "/tmp/pip-build-env-dtd18g4e/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dtd18g4e/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
          ~~~~^^^^^^^^^^^^^^^^
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'version_query'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I think the package is not packaged correctly - build system requirements are not specified anywhere, and therefore not met.

At least a minimal pyproject.toml should be present that specifies what packages are needed in order to execute setup.py, as described here:

https://peps.python.org/pep-0518/#build-system-table

The only workaround is to preinstall the build requirements and then install from source without build isolation, like this:

pip install setuptools version_query
pip install --no-build-isolation --no-binary ebrains-drive ebrains-drive==0.6.0

Then, I encounter another this error:

Collecting ebrains-drive==0.6.0
  Using cached ebrains-drive-0.6.0.tar.gz (21 kB)
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [28 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
          ~~~~^^
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/usr/local/lib/python3.13/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
          self.run_setup()
          ~~~~~~~~~~~~~~^^
        File "/usr/local/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
          ~~~~^^^^^^^^^^^^^^^^
        File "<string>", line 4, in <module>
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 96, in predict_version_str
          return predict_caller(2).to_str()
                 ~~~~~~~~~~~~~~^^^
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 81, in predict_caller
          here = _caller_folder(stack_level + 1)
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 22, in _caller_folder
          assert here.is_file(), here
                 ~~~~~~~~~~~~^^
      AssertionError: /tmp/pip-install-lucah5iq/ebrains-drive_28b39b508150410ba3966883977e13f7/<string>
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@liadomide Is the last error just above similar to yours, or rather to the one posted by @jhgoebbert ?

mbdevpl · 2025-02-16T09:39:31Z

After I additionally install wheel (on a new clean container):

pip install setuptools version_query wheel
pip install --no-build-isolation --no-binary ebrains-drive ebrains-drive==0.6.0

I think I'm able to reproduce the first mentioned issue:

Collecting ebrains-drive==0.6.0
  Downloading ebrains-drive-0.6.0.tar.gz (21 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
          exec(compile('''
          ~~~~^^^^^^^^^^^^
          # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ...<31 lines>...
          exec(compile(setup_py_code, filename, "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          ''' % ('/tmp/pip-install-2nztl_a5/ebrains-drive_8b4ecb06d8a74e92a09a3a151b344924/setup.py',), "<pip-setuptools-caller>", "exec"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-2nztl_a5/ebrains-drive_8b4ecb06d8a74e92a09a3a151b344924/setup.py", line 4, in <module>
          __version__ = predict_version_str()
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 96, in predict_version_str
          return predict_caller(2).to_str()
                 ~~~~~~~~~~~~~~^^^
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 82, in predict_caller
          return predict_folder(here, True)
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 76, in predict_folder
          return query_folder(path, search_parent_directories=search_parent_directories)
        File "/usr/local/lib/python3.13/site-packages/version_query/query.py", line 36, in query_folder
          return query_package_folder(path, search_parent_directories=search_parent_directories)
        File "/usr/local/lib/python3.13/site-packages/version_query/py_query.py", line 44, in query_package_folder
          raise ValueError(paths, metadata_json_paths, pkg_info_paths)
      ValueError: ([PosixPath('/tmp/pip-install-2nztl_a5/ebrains-drive_8b4ecb06d8a74e92a09a3a151b344924'), PosixPath('/tmp/pip-install-2nztl_a5'), PosixPath('/tmp'), PosixPath('/')], [], [])
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I think the issue is that version-query expects the version to be defined within the Python package, not one level above it in a file such as setup.py -- I think that indeed this is an issue in version-query. I've opened a PR in version-query that when done should fix the above issue: mbdevpl/version-query#26

mbdevpl · 2025-02-16T14:42:33Z

As I posted in mbdevpl/version-query#2:

I think this issue has been fixed in mbdevpl/version-query#26 -- which has been released just now in version 1.6.0. Please update and check again, the issue should be gone.

jhgoebbert mentioned this issue Oct 31, 2024

ValueError on Conda in windows environment mbdevpl/version-query#2

Closed

apdavison linked a pull request Nov 5, 2024 that will close this issue

Switch to pyproject.toml-based build #34

Open

mbdevpl mentioned this issue Feb 16, 2025

improve Python package metadata finding when running outside of git repositories mbdevpl/version-query#26

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

predict_version_str() not general enough for setup.py #33

predict_version_str() not general enough for setup.py #33

jhgoebbert commented Oct 31, 2024 •

edited

Loading

liadomide commented Nov 4, 2024

mbdevpl commented Feb 16, 2025 •

edited

Loading

mbdevpl commented Feb 16, 2025

mbdevpl commented Feb 16, 2025

predict_version_str() not general enough for setup.py #33

predict_version_str() not general enough for setup.py #33

Comments

jhgoebbert commented Oct 31, 2024 • edited Loading

liadomide commented Nov 4, 2024

mbdevpl commented Feb 16, 2025 • edited Loading

mbdevpl commented Feb 16, 2025

mbdevpl commented Feb 16, 2025

jhgoebbert commented Oct 31, 2024 •

edited

Loading

mbdevpl commented Feb 16, 2025 •

edited

Loading