Skip to content

Commit 307eddb

Browse files
nicbusnicbus
committed
refactor, update and add multi-platform support
 * switch to manylinux for linux wheels * support for linux aarch64 * support for darwin x86_64 and arm * support windows amd64 * fix sdist * update to poetry 2.x * handle native build with build.py (hooked in pyproject.toml) * automate package builds with build_packages.py * split README.md dev part into CONTRIBUTING.md
1 parent 14fc195 commit 307eddb

20 files changed

+660
-178
lines changed

‎.dockerignore‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎.gitignore‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
**/__pycache__
1+
**/.vim/
2+
**/__pycache__/
23
/dist/
34
/poetry.lock
4-
/src/rgb_lib/_rgb_lib/librgblibuniffi.so
5-
/src/rgb_lib/_rgb_lib/rgb_lib.py
5+
/rgb_lib/*rgblibuniffi.*
6+
/rgb_lib/rgb_lib.py

‎.gitmodules‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
[submodule "rgb-lib"]
22
path = rgb-lib
33
url = https://github.com/RGB-Tools/rgb-lib.git
4+
[submodule "cross"]
5+
path = cross
6+
url = https://github.com/cross-rs/cross

‎CONTRIBUTING.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# RGB Lib Python bindings development
2+
3+
The goal of this project is to produce Python bindings for [rgb-lib], which is
4+
included as a submodule. The bindings are created by the [rgb-lib-uniffi]
5+
project, which is located inside the rgb-lib submodule.
6+
7+
## Build
8+
9+
In order to build the bindings package(s), clone the project (`git clone
10+
https://github.com/RGB-Tools/rgb-lib-python --recurse-submodules`), enter the
11+
project root (`cd rgb-lib-python`) and follow the next instructions.
12+
13+
Always make sure the submodules are up-to-date:
14+
15+
```sh
16+
git submodule update --init --recursive
17+
```
18+
19+
The builds of the native Rust library and of the python bindings are carried
20+
out in Docker, using [cross].
21+
22+
### Requirements
23+
24+
- [poetry] (version 2)
25+
- [docker]
26+
- [cargo]
27+
- [cross] (`cargo install cross`)
28+
- cc (`apt install gcc`)
29+
30+
### Local project
31+
32+
In order to build or install the project on the local machine, cross needs to
33+
be properly configured first. To do so, run:
34+
35+
```sh
36+
poetry run python -c 'import build_packages; build_packages.setup_cross()'
37+
```
38+
39+
or, if the project has already been installed (see below):
40+
41+
```sh
42+
poetry run setup-cross
43+
```
44+
45+
The package can then be built with:
46+
47+
```sh
48+
poetry build
49+
```
50+
51+
or installed with:
52+
53+
```sh
54+
poetry install
55+
```
56+
57+
Once the package has been installed, the following scripts are available to
58+
ease development:
59+
60+
- build-packages (builds all the packages, see the [packages] section)
61+
- setup-cross (configure cross to build project/packages)
62+
63+
### Packages
64+
65+
In order to build all the packages for the project, one wheel per supported
66+
platform plus the sdist, if the project has already been installed run:
67+
68+
```sh
69+
poetry run build-packages
70+
```
71+
72+
else run:
73+
74+
```sh
75+
poetry run python build_packages.py
76+
```
77+
78+
The package build script will build the required docker image and use it to
79+
build the wheels for all supported platforms, then builds the sdist. Once the
80+
build completes, the archives will be available in the `dist/` directory. The
81+
script will also cleanup the build artifacts and restore the submodules to
82+
their initial conditions.
83+
84+
### Wheel tags
85+
86+
The `build.py` script contains the wheel tags associated to the supported
87+
platforms. See Python's [platform compatibility tags] for details. You can use
88+
`sysconfig.get_platform()` and `platform.machine()` to get the data for the
89+
running system.
90+
91+
Special care is necessary for macosx wheels, as there
92+
`sysconfig.get_platform()` returns what the Python interpreter was built for
93+
(e.g. the system python reports `macosx-10.9-universal2`, while homebrew python
94+
3.12 might return `macosx-14.0-arm64`) and the operating system (e.g. version
95+
15.1.1) version is not used in a numerical comparison. To have wheels
96+
successfully install it's important to tag them with the correct major version
97+
but use `0` as the minor version (e.g. `macosx-12.0-arm64` will install on OSX
98+
12.7 but `macosx-12.3-arm64` won't).
99+
100+
## Format
101+
102+
To format the code of the build scripts, from the project root run:
103+
104+
```sh
105+
poetry run black *.py
106+
```
107+
108+
## Lint
109+
110+
To lint the code of the build scripts, from the project root run:
111+
112+
```sh
113+
poetry run flake8 *.py
114+
poetry run pylint *.py
115+
```
116+
117+
## Publish
118+
119+
Publishing to PyPI is handled with [poetry] (version 2.0 or later).
120+
121+
Make sure the `dist/` directory only contains the expected packages (e.g. using
122+
`poetry build` produces a wheel that's not meant to be published). The best way
123+
do to this is to empty the `dist/` directory and then build all the [packages].
124+
To check what would be published use `poetry publish --dry-run`.
125+
126+
To configure the access token, which only needs to be done once, run:
127+
128+
```sh
129+
poetry config pypi-token.pypi <token>
130+
```
131+
132+
To publish a new release run:
133+
134+
```sh
135+
poetry publish
136+
```
137+
138+
### Test PyPI
139+
140+
To use the test PyPI instance, the repository needs to be configured in poetry:
141+
142+
```sh
143+
poetry config repositories.test-pypi https://test.pypi.org/legacy/
144+
```
145+
146+
An access token then needs to also be set:
147+
148+
```sh
149+
poetry config pypi-token.test-pypi <token>
150+
```
151+
152+
Publishing needs to specify the registry:
153+
154+
```sh
155+
poetry publish -r test-pypi
156+
```
157+
158+
To install the package from test PyPI:
159+
160+
```sh
161+
pip install --index-url https://test.pypi.org/simple/ rgb-lib
162+
```
163+
164+
[cargo]: https://github.com/rust-lang/cargo
165+
[cross]: https://github.com/cross-rs/cross
166+
[docker]: https://docs.docker.com/engine/install/
167+
[packages]: #packages
168+
[platform compatibility tags]: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/
169+
[poetry]: https://github.com/python-poetry/poetry

‎Cross.toml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
[target.x86_64-unknown-linux-gnu]
2+
image = "ghcr.io/cross-rs/x86_64-unknown-linux-gnu:local"
3+
4+
[target.aarch64-unknown-linux-gnu]
5+
image = "ghcr.io/cross-rs/aarch64-unknown-linux-gnu:local"
6+
7+
[target.x86_64-apple-darwin]
8+
image = "ghcr.io/cross-rs/x86_64-apple-darwin-cross:local"
9+
10+
[target.aarch64-apple-darwin]
11+
image = "ghcr.io/cross-rs/aarch64-apple-darwin-cross:local"
12+
13+
[target.x86_64-pc-windows-gnu]
14+
image = "ghcr.io/cross-rs/x86_64-pc-windows-gnu:local"
15+
pre-build = [
16+
"apt-get update && apt-get --assume-yes install nasm"
17+
]

‎Dockerfile‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎README.md‎

Lines changed: 29 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,51 @@
11
# RGB Lib Python bindings
22

3-
This project builds a Python library, `rgb-lib`, for the [rgb-lib]
4-
Rust library, which is included as a git submodule. The bindings are created by
5-
the [rgb-lib-uniffi] project, which is located inside the rgb-lib submodule.
3+
Python bindings for the [rgb-lib] Rust library.
64

7-
## Install from PyPI
5+
## Installation
86

9-
Install the [latest release] by running:
10-
```shell
7+
### From PyPI (wheel)
8+
9+
Install the [latest stable release] from PyPI by running:
10+
11+
```sh
1112
pip install rgb-lib
1213
```
1314

14-
## Demo
15+
### From source distribution (sdist)
1516

16-
The `demo/` directory contains a demonstration of the most common operations in
17-
the form of a Jupyter notebook. See the included `README.md` file for more
18-
details.
17+
Installation from source distribution (tested on Linux) has the following
18+
requirements:
1919

20-
## Install locally
21-
22-
### Requirements
20+
- [docker]
2321
- [cargo]
24-
- [poetry] 1.4+
25-
26-
In order to install the project locally, run:
27-
```shell
28-
# Update the submodule
29-
git submodule update --init
22+
- [cross]
23+
- cc
3024

31-
# Generate the bindings
32-
./generate.sh
25+
The process is quite long and requires several GB of disk space, due to the
26+
builds of the required Docker image and the rgb-lib rust library.
3327

34-
# Build the source and wheels archives
35-
poetry build
28+
## Usage
3629

37-
# Install the wheel (replacing <version> with built version)
38-
pip install ./dist/rgb_lib-<version>-py3-none-any.whl
30+
Once installed, you can import the `rgb_lib` module and call its APIs.
3931

40-
# or install the sdist (replacing <version> with built version)
41-
pip install ./dist/rgb_lib-<version>.tar.gz
42-
```
32+
As an example:
4333

44-
## Build in Docker
45-
In order to build the project in a Docker container, run:
46-
```shell
47-
# Update the submodule
48-
git submodule update --init
34+
```python
35+
import rgb_lib
4936

50-
# run the build script
51-
./build_in_docker.sh
37+
keys = rgb_lib.generate_keys(rgb_lib.BitcoinNetwork.REGTEST)
38+
print(keys.account_xpub)
5239
```
5340

54-
The `build_in_docker.sh` script will build the docker image and use it to first
55-
generate the bindings, then build the source and wheel archives. Once the build
56-
completes, archives will be available in the `dist/` directory as if they were
57-
built locally.
58-
59-
## Publish
60-
61-
Publishing to PyPI is handled with Poetry.
62-
63-
To configure the access token, which only needs to be done once, run:
64-
```shell
65-
poetry config pypi-token.pypi <token>
66-
```
67-
68-
To publish a new release run:
69-
```shell
70-
poetry publish
71-
```
41+
## Demo
7242

43+
The `demo/` directory contains a demonstration of the most common operations in
44+
the form of a Jupyter notebook. See the included `README.md` file for more
45+
details.
7346

7447
[cargo]: https://github.com/rust-lang/cargo
75-
[rgb-lib]: https://github.com/RGB-Tools/rgb-lib
48+
[docker]: https://docs.docker.com/engine/install/
49+
[latest stable release]: https://pypi.org/project/rgb-lib/
7650
[rgb-lib-uniffi]: https://github.com/RGB-Tools/rgb-lib/tree/master/bindings/uniffi
77-
[latest release]: https://pypi.org/project/rgb-lib/
78-
[poetry]: https://github.com/python-poetry/poetry
51+
[rgb-lib]: https://github.com/RGB-Tools/rgb-lib

0 commit comments

Comments
 (0)