Use Python "build" package as build frontend, don't directly run setup.py (#287)

* Use Python "build" package as build frontend, don't directly run setup.py

Running setup.py directly is deprecated, from 2025-10-31 it won't be
supported. The Python ecosystem is moving toward seperation of build frontend
and backend. This commit switches to the "build" frontend, and keeps
setuptools as the backend.

Two `setup()` keywords have been altered

`packages` was incomplete, but happened to work. Setuptools now warns about
such a condition, which added noise in the build output.

`test_suite` was unnnecessary because pytest is used for testing. It has been
deprecated since by setuptools 41.5.0 (Oct 2019), and its presence emitted a
warning that added noise to build output.

This change should
- allow wasmtime-py wheels to be built, published, and installed.

It should not
- alter versioning behaviour (specifically .devNNN tagging)
- alter wheel contents

It does not
- allow building of an sdist (source tar.gz distribution)

Risks
- `-C=--build-option=--plat-name -C=--build-option=...` is ugly and
  possibly fragile. A future release of build or setuptools may break it.
- No version constraints are placed on setuptools in pyproject.toml.

I think this is the minimum viable change that fixes #286. Other improvements
can be made seperately (e.g. migrating more metadata from setup.py to
pyproject.toml).

See
- https://build.pypa.io/en/stable/
- https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
- https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-test-suite
- pypa/pip#11859
- pypa/setuptools#2491
- pypa/setuptools#3882

* Revert change to packages keyword in setup.py

CI was failing because the "Setup wasmtime-py CI" action in .github/actions/
setup.yml installs `.[testing]` before ci/build-rust.py has been run. Thus
wasmtime/bindgen/generated and wasmtime/bindgen/generated/imports didn't exist
yet.

See
- https://github.com/bytecodealliance/wasmtime-py/actions/runs/15640817846/job/44067456534

* Build source distributions (sdist) alongside wheels (bdist_wheel)

I have integrated running of the ci/download-wasmtime.py and ci/build-rsyt.py
into the PEP-517 `build_wheel()` step. This allows a wheel to be built from
just a source distribution (as opposed to a git checkout), so e.g.

```
pip install --no-binary=:any: wasmtime-py
```

has a chance of working.

The integration mechanism is ci/_custom_build/backend.py, an in-tree PEP-517
build backend that extends the build backend provided by setuptools. This file
is excluded from pytest discovery/scanning for expediency and to keep the
scope of this change smaller. A future PR could remove this restriction to
improve test and style guide coverage.

I elected to keep most project metadata in setup.py for now, again to minimize
the size of the PR. I recommend a future PR to move the metadata to
pyproject.toml.

This change incorporates the changes begun in PR #285, and supercedes it. As
with that PR wheels builtin in wasmtime-pr should be unchanged - so in the
common case end users do not require a rust compiler, or to build from source.

Author: moreati

.github/workflows/main.yml

@@ -48,55 +48,41 @@ jobs:
         # publishes to `test.pypi.org` works. (see `git` command in `setup.py`)
         fetch-depth: 0
     - uses: ./.github/actions/setup
-    - run: pip install setuptools wheel
+    - run: pip install build
     # If this is a tagged build use real version numbers
     - run: echo "PROD=true" >> $GITHUB_ENV
       if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
+    - name: Build sdist (source distribution)
+      run: |
+        git clean -fdx wasmtime build
+        python -m build --sdist
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py linux x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name manylinux1-x86_64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=manylinux1-x86_64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py linux aarch64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name manylinux2014_aarch64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=manylinux2014_aarch64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py darwin x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name macosx-10-13-x86_64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=macosx-10-13-x86_64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py darwin arm64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name macosx-11-0-arm64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=macosx-11-0-arm64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py win32 x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name win-amd64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=win-amd64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py win32 arm64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name win-arm64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=win-arm64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py musl x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name musllinux_1_2_x86_64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=musllinux_1_2_x86_64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py android x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name android_26_x86_64
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=android_26_x86_64
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py android aarch64
-        python ci/build-rust.py
-        python setup.py bdist_wheel --plat-name android_26_arm64_v8a
+        python -m build --wheel -C=--build-option=--plat-name -C=--build-option=android_26_arm64_v8a
 
     # Build an "any" wheel with:
     #
@@ -109,13 +95,11 @@ jobs:
     # shared libraries.
     - run: |
         git clean -fdx wasmtime build
-        python ci/download-wasmtime.py win32 x86_64
-        python ci/build-rust.py
-        python setup.py bdist_wheel
+        python -m build --wheel -C=--wasmtime-py-mingw-any
 
     - uses: actions/upload-artifact@v4
       with:
-        name: wheels
+        name: dists
         path: dist
 
   docs:
@@ -164,7 +148,7 @@ jobs:
       with:
         path: generated-docs
 
-  upload_wheels:
+  upload_dists:
     needs: build
     runs-on: ubuntu-latest
     steps:
@@ -176,7 +160,7 @@ jobs:
     - uses: ./.github/actions/setup
     - uses: actions/download-artifact@v4
       with:
-        name: wheels
+        name: dists
         path: dist
     - run: find .
 

MANIFEST.in

@@ -1 +1,31 @@
-recursive-include wasmtime *.py *.wasm *.so *.dll *.dylib
+# Specify files to be included in source distributions (sdist).
+# Some files are included by default already (e.g. LICENSE, README*).
+# https://setuptools.pypa.io/en/stable/userguide/miscellaneous.html#controlling-files-in-the-distribution
+
+graft ci
+graft examples
+graft rust
+graft tests
+
+prune rust/target
+prune tests/bindgen/generated
+prune tests/codegen/generated
+
+# Wasmtime C headers. Downloaded by ci/download-wasmtime.py
+prune wasmtime/include
+
+# WebAssmbly modules, python bindings. Generated by ci/build-rust.py
+recursive-exclude wasmtime/bindgen/generated *.py *.wasm
+
+include .flake8
+include CONTRIBUTING.md
+include mypy.ini
+include pytest.ini
+
+# Wasmtime shared library. Downloaded by ci/download-wasmtime.py
+# Included in binary distributions
+global-exclude *.dll
+global-exclude *.dylib
+global-exclude *.py[cdo]
+global-exclude *.so
+global-exclude *~

ci/_custom_build/backend.py

@@ -0,0 +1,92 @@
+"""
+An in-tree PEP-517 build backend that downloads wasmtime and generates Python
+bindings. It extends the PEP-517 backend provided by setuptools.
+
+See
+- https://peps.python.org/pep-0517/#build-backend-interface
+- https://setuptools.pypa.io/en/latest/build_meta.html#dynamic-build-dependencies-and-other-build-meta-tweaks
+"""
+
+import subprocess
+import sys
+
+from typing import Union
+
+# `from ... import *` is intentional and necessary, so that any PEP-517 hooks
+# not overridden here are still available to build frontends.
+from setuptools import build_meta as build_meta_orig
+from setuptools.build_meta import *
+
+
+def get_plat_name(config_settings:Union[dict, None]) -> Union[str, None]:
+    """
+    Return the value of any --plat-name passed to the build frontend,
+    otherwise None.
+
+    >>> get_plat_name({'--build-option': ['--plat-name', 'manylinux1-x86_64']})
+    'manylinux1-x86_64'
+    >>> print(get_plat_name(None))
+    None
+    >>> print(get_plat_name({}))
+    None
+    >>> print(get_plat_name({'--build-option': []}))
+    None
+    """
+    if config_settings is None:
+        return None
+
+    try:
+        build_options = config_settings['--build-option']
+    except KeyError:
+        return None
+
+    try:
+        plat_name_option_idx = build_options.index('--plat-name')
+    except ValueError:
+        return None
+
+    plat_name = build_options[plat_name_option_idx+1]
+    return plat_name
+
+
+def plat_name_to_download_args(plat_name:str) -> list[str]:
+    "Return a list of download-wasmtime.py arguments for a given platform name."
+    mapping = {
+        'android_26_arm64_v8a':         ['android', 'aarch64'],
+        'android_26_x86_64':            ['android', 'x86_64'],
+        'macosx-10-13-x86_64':          ['darwin', 'x86_64'],
+        'macosx-11-0-arm64':            ['darwin', 'arm64'],
+        'manylinux1-x86_64':            ['linux', 'x86_64'],
+        'manylinux2014_aarch64':        ['linux', 'aarch64'],
+        'musllinux_1_2_x86_64':         ['musl', 'x86_64'],
+        'win-amd64':                    ['win32', 'x86_64'],
+        'win-arm64':                    ['win32', 'arm64'],
+    }
+    return mapping[plat_name]
+
+
+def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
+    """
+    Download a pre-compiled build of wasmtime and generate bindings, both to be
+    included in the output Python wheel.
+
+    This is a PEP-517 build backend function.
+    """
+    plat_name = get_plat_name(config_settings)
+    if plat_name is not None:
+        download_args = plat_name_to_download_args(plat_name)
+    elif config_settings is not None and '--wasmtime-py-mingw-any' in config_settings:
+        download_args = ['win32', 'x86_64']
+    else:
+        download_args = []
+
+    subprocess.run(
+        [sys.executable, 'ci/download-wasmtime.py', *download_args],
+        check=True,
+    )
+    subprocess.run(
+        [sys.executable, 'ci/build-rust.py'],
+        check=True,
+    )
+
+    return build_meta_orig.build_wheel(wheel_directory, config_settings, metadata_directory)

pyproject.toml

@@ -0,0 +1,9 @@
+[build-system]
+requires = [
+    "setuptools",
+]
+# Use an in-tree build backend, to customise the wheels we generate
+build-backend = "backend"
+backend-path = [
+    "ci/_custom_build",
+]

pytest.ini

@@ -1,4 +1,5 @@
 [pytest]
 addopts = --doctest-modules --mypy --ignore-glob=tests/bindgen/*/app.py
 norecursedirs = 
+    ci/_custom_build
     tests/bindgen/generated/*

setup.py

@@ -11,7 +11,9 @@
 
 # Give unique version numbers to all commits so our publication-on-each commit
 # works on main
-if 'PROD' not in os.environ:
+# TODO Temporarily disable '.devNNN' version suffix while porting to PEP-517
+# https://github.com/bytecodealliance/wasmtime-py/pull/287#issuecomment-2979288710
+if False and 'PROD' not in os.environ:
     res = subprocess.run(['git', 'rev-list', 'HEAD', '--count'], capture_output=True, encoding="utf8")
     version += '.dev' + res.stdout.strip()
 
@@ -30,12 +32,28 @@
         "Documentation": "https://bytecodealliance.github.io/wasmtime-py/",
         "Source Code": "https://github.com/bytecodealliance/wasmtime-py",
     },
-    packages=['wasmtime'],
+    packages=[
+        'wasmtime',
+        'wasmtime.bindgen',
+    ],
     install_requires=['importlib_resources>=5.10'],
     include_package_data=True,
-    package_data={"wasmtime": ["py.typed"]},
+    package_data={
+        "wasmtime": [
+            # Wasmtime shared library. Downloaded by ci/download-wasmtime.py
+            "*-*/*.dll",
+            "*-*/*.dylib",
+            "*-*/*.so",
+            "py.typed",
+        ],
+        "wasmtime.bindgen": [
+            # WebAssmbly modules, python bindings. Generated by ci/build-rust.py
+            "generated/*.py",
+            "generated/*.wasm",
+            "generated/imports/*.py",
+        ],
+    },
     python_requires='>=3.9',
-    test_suite="tests",
     extras_require={
         'testing': [
             'coverage',