Skip to content

Commit bc205fa

Browse files
graebmgraebm
authored
Add pyproject.toml, and stop calling setup.py directly (#629)
**Issue:** For a few years now, we've been seen the occasional deprecation warning about setup.py. Including: ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** and ******************************************************************************** Ensure that any custom bdist_wheel implementation is a subclass of setuptools.command.bdist_wheel.bdist_wheel. By 2025-Oct-15, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/wheel#631 for details. ******************************************************************************** 2025-Oct is on its way, so let's finally do something about it... **Description of changes:** Follow advice from ["Is setup.py deprecated?"](https://packaging.python.org/en/latest/discussions/setup-py-deprecated/) page at packaging.python.org tldr; having a setup.py file **is not** deprecated, but directly invoking `python setup.py` **is** deprecated. - Add [pyproject.toml](https://packaging.python.org/en/latest/specifications/pyproject-toml/) file (the modern way to configure your package) - Move data from `setup.py` -> `pyproject.toml` wherever possible (can't move dynamic stuff, like build steps for C extension) - Move dev dependencies from `requirements-dev.txt` -> `pyproject.toml [project.optional-dependencies]` - Now you call `pip install ".[dev]"`, instead of `pip install -r requirements-dev.txt`. The dev dependencies will be installed by the same command that builds and installs our project. - This change isn't 100% necessary, but seemed like the modern way to do things. It's one less step to set things up on your machine. However, you can't install dependencies without ALSO building/installing the package (we can roll this back if we end up hating it) - Use `python3 -m build` instead of `python3 setup.py sdist bdist_wheel` (the modern way to build a wheel for distribution) - Sadly, [build](https://pypi.org/project/build/) is a non-core package, so might need to be pip installed before using. Fortunately it's been [preinstalled on manylinux images since 2021](https://github.com/pypa/manylinux/blob/6b97f150fc9ede1cb4f25de84a939538a1019a58/docker/build_scripts/requirements3.8.txt#L7), so our Linux release process will be OK. - I updated our MacOS and Windows machines to have `build` installed for all python versions, so our release process should be OK. - STOP calling `pip install --upgrade setuptools` before `pip install .` - Modern pip installs the "build dependencies" declared in `pyproject.toml` (setuptools & wheel) in a special isolated environment before it builds our project. We declared `setuptools>=75.3.1`, so we're guaranteed it will get installed, and be a recent version.
1 parent 94811c4 commit bc205fa

26 files changed

+117
-169
lines changed

‎.builder/actions/aws_crt_python.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,10 @@ def run(self, env):
3131
env.shell.setenv('AWS_TEST_S3', '1')
3232

3333
actions = [
34+
[self.python, '--version'],
3435
[self.python, '-m', 'pip', 'install', '--upgrade', 'pip'],
35-
[self.python, '-m', 'pip', 'install', '--upgrade', '--requirement', 'requirements-dev.txt'],
3636
Builder.SetupCrossCICrtEnvironment(),
37-
[self.python, '-m', 'pip', 'install', '--verbose', '.'],
37+
[self.python, '-m', 'pip', 'install', '--verbose', '.[dev]'],
3838
# "--failfast" because, given how our leak-detection in tests currently works,
3939
# once one test fails all the rest usually fail too.
4040
[self.python, '-m', 'unittest', 'discover', '--verbose', '--failfast'],

‎.github/workflows/ci.yml

+6-6
Original file line numberDiff line numberDiff line change
@@ -337,8 +337,7 @@ jobs:
337337
submodules: true
338338
- name: Run tests
339339
run: |
340-
python3 -m pip install --upgrade --requirement requirements-dev.txt
341-
python3 -m pip install . --verbose
340+
python3 -m pip install ".[dev]" --verbose
342341
python3 -m unittest discover --failfast --verbose
343342
344343
package-source:
@@ -356,22 +355,23 @@ jobs:
356355
submodules: true
357356
- name: Package source + install
358357
run: |
359-
python3 setup.py sdist
358+
python3 -m pip install build
359+
python3 -m build --sdist
360360
cd dist
361361
python3 -m pip install -v awscrt-1.0.0.dev0.tar.gz
362362
python3 -c "import awscrt.io"
363363
364364
# check that docs can still build
365365
check-docs:
366-
runs-on: ubuntu-22.04 # latest
366+
runs-on: ubuntu-22.04 # getting errors from sphinx using ubuntu-24.04, so stuck on 22.04 till we diagnose it
367367
steps:
368368
- uses: actions/checkout@v4
369369
with:
370370
submodules: true
371371
- name: Check docs
372372
run: |
373-
python3 -m pip install -r requirements-dev.txt
374-
python3 -m pip install --verbose .
373+
python3 -m pip install --upgrade pip
374+
python3 -m pip install --verbose ".[dev]"
375375
./scripts/make-docs.py
376376
377377
check-submodules:

‎.github/workflows/docs.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ on:
99

1010
jobs:
1111
update-docs-branch:
12-
runs-on: ubuntu-22.04 # latest
12+
runs-on: ubuntu-22.04 # getting errors from sphinx using ubuntu-24.04, so stuck on 22.04 till we diagnose it
1313
permissions:
1414
contents: write # allow push
1515
steps:
@@ -20,8 +20,8 @@ jobs:
2020

2121
- name: Update docs branch
2222
run: |
23-
python3 -m pip install -r requirements-dev.txt
24-
python3 -m pip install --verbose .
23+
python3 -m pip install --upgrade pip
24+
python3 -m pip install --verbose ".[dev]"
2525
./scripts/make-docs.py
2626
2727
- name: Commit

‎.github/workflows/lint.yml

+8-17
Original file line numberDiff line numberDiff line change
@@ -7,28 +7,19 @@ on:
77
- 'docs'
88

99
jobs:
10-
clang-format:
11-
12-
runs-on: ubuntu-24.04 # latest
13-
14-
steps:
15-
- name: Checkout Sources
16-
uses: actions/checkout@v4
17-
18-
- name: clang-format lint
19-
run: |
20-
./format-check.py
21-
22-
autopep8:
10+
lint:
2311
runs-on: ubuntu-24.04 # latest
2412

2513
steps:
2614
- name: Checkout Source
2715
uses: actions/checkout@v4
2816

29-
- name: Build and Test
17+
- name: Format and Check Diff
18+
# run formatting script, which edits files in place
19+
# then do `git diff` to show what changed (if anything)
20+
# then do `git diff --quiet` to fail if anything changed
3021
run: |
31-
python3 -m pip install --upgrade setuptools
32-
python3 -m pip install --upgrade wheel
3322
python3 -m pip install --upgrade autopep8
34-
python3 -m autopep8 --exit-code --diff --recursive awscrt test .builder setup.py
23+
python3 scripts/format-all.py
24+
git diff
25+
git diff --quiet

‎.lgtm.yml

-19
This file was deleted.

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ git submodule update --init
3232
python3 -m pip install .
3333
```
3434

35-
To use from your Python application, declare `awscrt` as a dependency in your `setup.py` file.
35+
To use from your Python application, declare `awscrt` as a dependency.
3636

3737
### OpenSSL and LibCrypto (Unix only)
3838

‎codebuild/cd/manylinux-x64-build.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@ phases:
1212
build:
1313
commands:
1414
- echo Build started on `date`
15-
- /opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
15+
- /opt/python/cp38-cp38/bin/python -m build
1616
- auditwheel repair --plat manylinux1_x86_64 dist/awscrt-*cp38-cp38-linux_x86_64.whl
17-
- /opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
17+
- /opt/python/cp39-cp39/bin/python -m build
1818
- auditwheel repair --plat manylinux1_x86_64 dist/awscrt-*cp39-cp39-linux_x86_64.whl
1919
# python 3.9 is the last version manylinux1 will ever receive
2020
- cp -r wheelhouse ../dist

‎codebuild/cd/manylinux-x86-build.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@ phases:
1212
build:
1313
commands:
1414
- echo Build started on `date`
15-
- /opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
15+
- /opt/python/cp38-cp38/bin/python -m build
1616
- auditwheel repair --plat manylinux1_i686 dist/awscrt-*cp38-cp38-linux_i686.whl
17-
- /opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
17+
- /opt/python/cp39-cp39/bin/python -m build
1818
- auditwheel repair --plat manylinux1_i686 dist/awscrt-*cp39-cp39-linux_i686.whl
1919
# python 3.9 is the last version manylinux1 will ever receive
2020
- cp -r wheelhouse ../dist

‎codebuild/cd/publish_to_prod_pypi.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ phases:
66
- sudo apt-get update -y
77
- sudo apt-get install python3 python3-pip -y
88
- python3 -m pip install --user --upgrade pip
9-
- python3 -m pip install --user --upgrade twine setuptools wheel boto3 PyOpenSSL six
9+
- python3 -m pip install --user --upgrade twine boto3
1010
pre_build:
1111
commands:
1212
- export CC=gcc

‎codebuild/cd/publish_to_test_pypi.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ phases:
66
- sudo apt-get update -y
77
- sudo apt-get install python3 python3-pip -y
88
- python3 -m pip install --user --upgrade pip
9-
- python3 -m pip install --user --upgrade twine setuptools wheel boto3 PyOpenSSL six
9+
- python3 -m pip install --user --upgrade twine boto3
1010
pre_build:
1111
commands:
1212
- export CC=gcc

‎codebuild/cd/test_version_exists.yml

-3
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,6 @@ version: 0.2
55
phases:
66
install:
77
commands:
8-
- sudo apt-get update -y
9-
- sudo apt-get install python3 python3-pip -y
10-
- pip3 install --upgrade setuptools
118
pre_build:
129
commands:
1310
build:

‎codebuild/linux-integration-tests.sh

+1-2
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,5 @@ git submodule update --init
1414
cd $CODEBUILD_SRC_DIR
1515

1616
export AWS_TEST_S3=YES
17-
python -m pip install --upgrade --requirement requirements-dev.txt
18-
python -m pip install --verbose .
17+
python -m pip install --verbose ".[dev]"
1918
python -m unittest discover --failfast --verbose 2>&1 | tee /tmp/tests.log

‎continuous-delivery/build-wheels-manylinux2014-aarch64.sh

+6-11
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,22 @@ set -ex
44

55
/opt/python/cp39-cp39/bin/python ./continuous-delivery/update-version.py
66

7-
/opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
7+
/opt/python/cp38-cp38/bin/python -m build
88
auditwheel repair --plat manylinux2014_aarch64 dist/awscrt-*cp38*.whl
99

10-
/opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
10+
/opt/python/cp39-cp39/bin/python -m build
1111
auditwheel repair --plat manylinux2014_aarch64 dist/awscrt-*cp39*.whl
1212

13-
/opt/python/cp310-cp310/bin/python setup.py sdist bdist_wheel
13+
/opt/python/cp310-cp310/bin/python -m build
1414
auditwheel repair --plat manylinux2014_aarch64 dist/awscrt-*cp310*.whl
1515

16-
/opt/python/cp311-cp311/bin/python setup.py sdist bdist_wheel
16+
/opt/python/cp311-cp311/bin/python -m build
1717
auditwheel repair --plat manylinux2014_aarch64 dist/awscrt-*cp311*.whl
1818

19-
# Don't need to build wheels for Python 3.12.
20-
# The 3.11 wheel uses the stable ABI, so it works with newer versions too.
19+
# Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
2120

2221
# We are using the Python 3.13 stable ABI from Python 3.13 onwards because of deprecated functions.
23-
# Manylinux images don't contain setuptools from Python 3.13, so we need to install it.
24-
# Install in a custom location due to access issues.
25-
/opt/python/cp313-cp313/bin/python -m pip install --target ./local -r requirements-dev.txt
26-
export PYTHONPATH=./local:$PYTHONPATH
27-
/opt/python/cp313-cp313/bin/python setup.py sdist bdist_wheel
22+
/opt/python/cp313-cp313/bin/python -m build
2823
auditwheel repair --plat manylinux2014_aarch64 dist/awscrt-*cp313*.whl
2924

3025
rm dist/*.whl

‎continuous-delivery/build-wheels-manylinux2014-x86_64.sh

+6-11
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,22 @@ set -ex
44

55
/opt/python/cp39-cp39/bin/python ./continuous-delivery/update-version.py
66

7-
/opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
7+
/opt/python/cp38-cp38/bin/python -m build
88
auditwheel repair --plat manylinux2014_x86_64 dist/awscrt-*cp38*.whl
99

10-
/opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
10+
/opt/python/cp39-cp39/bin/python -m build
1111
auditwheel repair --plat manylinux2014_x86_64 dist/awscrt-*cp39*.whl
1212

13-
/opt/python/cp310-cp310/bin/python setup.py sdist bdist_wheel
13+
/opt/python/cp310-cp310/bin/python -m build
1414
auditwheel repair --plat manylinux2014_x86_64 dist/awscrt-*cp310*.whl
1515

16-
/opt/python/cp311-cp311/bin/python setup.py sdist bdist_wheel
16+
/opt/python/cp311-cp311/bin/python -m build
1717
auditwheel repair --plat manylinux2014_x86_64 dist/awscrt-*cp311*.whl
1818

19-
# Don't need to build wheels for Python 3.12.
20-
# The 3.11 wheel uses the stable ABI, so it works with newer versions too.
19+
# Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
2120

2221
# We are using the Python 3.13 stable ABI from Python 3.13 onwards because of deprecated functions.
23-
# Manylinux images don't contain setuptools from Python 3.13, so we need to install it.
24-
# Install in a custom location due to access issues.
25-
/opt/python/cp313-cp313/bin/python -m pip install --target ./local -r requirements-dev.txt
26-
export PYTHONPATH=./local:$PYTHONPATH
27-
/opt/python/cp313-cp313/bin/python setup.py sdist bdist_wheel
22+
/opt/python/cp313-cp313/bin/python -m build
2823
auditwheel repair --plat manylinux2014_x86_64 dist/awscrt-*cp313*.whl
2924

3025
rm dist/*.whl

‎continuous-delivery/build-wheels-musllinux-1-1-aarch64.sh

+6-11
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,22 @@ set -ex
44

55
/opt/python/cp39-cp39/bin/python ./continuous-delivery/update-version.py
66

7-
/opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
7+
/opt/python/cp38-cp38/bin/python -m build
88
auditwheel repair --plat musllinux_1_1_aarch64 dist/awscrt-*cp38*.whl
99

10-
/opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
10+
/opt/python/cp39-cp39/bin/python -m build
1111
auditwheel repair --plat musllinux_1_1_aarch64 dist/awscrt-*cp39*.whl
1212

13-
/opt/python/cp310-cp310/bin/python setup.py sdist bdist_wheel
13+
/opt/python/cp310-cp310/bin/python -m build
1414
auditwheel repair --plat musllinux_1_1_aarch64 dist/awscrt-*cp310*.whl
1515

16-
/opt/python/cp311-cp311/bin/python setup.py sdist bdist_wheel
16+
/opt/python/cp311-cp311/bin/python -m build
1717
auditwheel repair --plat musllinux_1_1_aarch64 dist/awscrt-*cp311*.whl
1818

19-
# Don't need to build wheels for Python 3.12.
20-
# The 3.11 wheel uses the stable ABI, so it works with newer versions too.
19+
# Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
2120

2221
# We are using the Python 3.13 stable ABI from Python 3.13 onwards because of deprecated functions.
23-
# Manylinux images don't contain setuptools from Python 3.13, so we need to install it.
24-
# Install in a custom location due to access issues.
25-
/opt/python/cp313-cp313/bin/python -m pip install --target ./local -r requirements-dev.txt
26-
export PYTHONPATH=./local:$PYTHONPATH
27-
/opt/python/cp313-cp313/bin/python setup.py sdist bdist_wheel
22+
/opt/python/cp313-cp313/bin/python -m build
2823
auditwheel repair --plat musllinux_1_1_aarch64 dist/awscrt-*cp313*.whl
2924

3025
rm dist/*.whl

‎continuous-delivery/build-wheels-musllinux-1-1-x86_64.sh

+6-11
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,22 @@ set -ex
44

55
/opt/python/cp39-cp39/bin/python ./continuous-delivery/update-version.py
66

7-
/opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
7+
/opt/python/cp38-cp38/bin/python -m build
88
auditwheel repair --plat musllinux_1_1_x86_64 dist/awscrt-*cp38*.whl
99

10-
/opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
10+
/opt/python/cp39-cp39/bin/python -m build
1111
auditwheel repair --plat musllinux_1_1_x86_64 dist/awscrt-*cp39*.whl
1212

13-
/opt/python/cp310-cp310/bin/python setup.py sdist bdist_wheel
13+
/opt/python/cp310-cp310/bin/python -m build
1414
auditwheel repair --plat musllinux_1_1_x86_64 dist/awscrt-*cp310*.whl
1515

16-
/opt/python/cp311-cp311/bin/python setup.py sdist bdist_wheel
16+
/opt/python/cp311-cp311/bin/python -m build
1717
auditwheel repair --plat musllinux_1_1_x86_64 dist/awscrt-*cp311*.whl
1818

19-
# Don't need to build wheels for Python 3.12.
20-
# The 3.11 wheel uses the stable ABI, so it works with newer versions too.
19+
# Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
2120

2221
# We are using the Python 3.13 stable ABI from Python 3.13 onwards because of deprecated functions.
23-
# Manylinux images don't contain setuptools from Python 3.13, so we need to install it.
24-
# Install in a custom location due to access issues.
25-
/opt/python/cp313-cp313/bin/python -m pip install --target ./local -r requirements-dev.txt
26-
export PYTHONPATH=./local:$PYTHONPATH
27-
/opt/python/cp313-cp313/bin/python setup.py sdist bdist_wheel
22+
/opt/python/cp313-cp313/bin/python -m build
2823
auditwheel repair --plat musllinux_1_1_x86_64 dist/awscrt-*cp313*.whl
2924

3025
rm dist/*.whl

‎continuous-delivery/build-wheels-osx.sh

+7-8
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,18 @@
11
#!/bin/bash
22
#before running this, you'll need cmake3 and a compiler. These python versions are just
3-
#using the default python installers from python.org. Each version needs updated pip, wheel, and setuptools
3+
#using the default python installers from python.org. Each version needs updated: pip, build
44
set -ex
55

66
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 ./continuous-delivery/update-version.py
77

8-
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3 setup.py sdist bdist_wheel
9-
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 setup.py sdist bdist_wheel
10-
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3 setup.py sdist bdist_wheel
11-
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3 setup.py sdist bdist_wheel
8+
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3 -m build
9+
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 -m build
10+
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3 -m build
11+
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3 -m build
1212

13-
# Don't need to build wheels for Python 3.12.
14-
# The 3.11 wheel uses the stable ABI, so it works with newer versions too.
13+
# Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
1514

1615
# We are using the Python 3.13 stable ABI from Python 3.13 onwards because of deprecated functions.
17-
/Library/Frameworks/Python.framework/Versions/3.13/bin/python3 setup.py sdist bdist_wheel
16+
/Library/Frameworks/Python.framework/Versions/3.13/bin/python3 -m build
1817

1918
#now you just need to run twine (that's in a different script)

‎continuous-delivery/build-wheels-win32.bat

+6-7
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
11

22
"C:\Program Files (x86)\Python39-32\python.exe" .\continuous-delivery\update-version.py || goto error
33

4-
"C:\Program Files (x86)\Python38-32\python.exe" setup.py sdist bdist_wheel || goto error
5-
"C:\Program Files (x86)\Python39-32\python.exe" setup.py sdist bdist_wheel || goto error
6-
"C:\Program Files (x86)\Python310-32\python.exe" setup.py sdist bdist_wheel || goto error
7-
"C:\Program Files (x86)\Python311-32\python.exe" setup.py sdist bdist_wheel || goto error
4+
"C:\Program Files (x86)\Python38-32\python.exe" -m build || goto error
5+
"C:\Program Files (x86)\Python39-32\python.exe" -m build || goto error
6+
"C:\Program Files (x86)\Python310-32\python.exe" -m build || goto error
7+
"C:\Program Files (x86)\Python311-32\python.exe" -m build || goto error
88

9-
:: Don't need to build wheels for Python 3.12.
10-
:: The 3.11 wheel uses the stable ABI, so it works with newer versions too.
9+
:: Don't need to build wheels for Python 3.12, it works with the 3.11 stable ABI wheel
1110

1211
:: We are using the 3.13 stable ABI from 3.13 onwards because of deprecated functions.
13-
"C:\Program Files (x86)\Python313-32\python.exe" setup.py sdist bdist_wheel || goto error
12+
"C:\Program Files (x86)\Python313-32\python.exe" -m build || goto error
1413

1514
goto :EOF
1615

0 commit comments

Comments
 (0)