Skip to content

multipy: README updates for Beta + CHANGELOG #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

multipy: README updates for Beta + CHANGELOG #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# CHANGELOG

## multipy-0.1.0

This is the initial Beta release of `torch::deploy`.

* PyTorch 1.13 support
* Python 3.7-3.10 support
* `torch::deploy` is now suitable for use in production environments.
* `torch::deploy` uses the current Python environment and no longer
requires building PyTorch, Python and C extensions from source.
* C extensions can be installed via standard `pip`/`conda` and will be
dynamically loaded at runtime. Popular PyTorch extensions have been tested but
there may be some libraries that are incompatible. If you run into an
incompatible library please file an issue.
* Prototype aarch64 support
* Improved performance and memory usage when keeping an InterpreterSession alive
for a long time.
* Supports all PyTorch core backends (CPU/CUDA/ROCm).
58 changes: 39 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,30 @@
[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](LICENSE)
[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](LICENSE) ![Runtime Tests](https://github.com/pytorch/multipy/actions/workflows/runtime_tests.yaml/badge.svg)


# \[experimental\] MultiPy
# `torch::deploy` (MultiPy)

> :warning: **This is project is still a prototype.** Only Linux x86 is supported, and the API may change without warning. Furthermore, please **USE PYTORCH NIGHTLY** when using `multipy::runtime`!

`MultiPy` (formerly `torch::deploy` and `torch.package`) is a system that allows you to run multi-threaded python code in C++. It offers `multipy.package` (formerly `torch.package`) in order to package code into a mostly hermetic format to deliver to `multipy::runtime` (formerly `torch::deploy`) which is a runtime which takes packaged
code and runs it using multiple embedded Python interpreters in a C++ process without a shared global interpreter lock (GIL). For more information on how `MultiPy` works
`torch::deploy` (MultiPy for non-PyTorch use cases) is a C++ library that enables you to run eager mode PyTorch models in production without any modifications to your model to support tracing. `torch::deploy` provides a way to run using multiple independent Python interpreters in a single process without a shared global interpreter lock (GIL). For more information on how `torch::deploy` works
internally, please see the related [arXiv paper](https://arxiv.org/pdf/2104.00254.pdf).

To learn how to use `torch::deploy` see [Installation](#installation) and [Examples](#examples).

Requirements:

* PyTorch 1.13+ or PyTorch nightly
* Linux (ELF based)
* x86_64 (Beta)
* arm64/aarch64 (Prototype)

> ℹ️ This is project is in Beta. `torch::deploy` is ready for use in production environments but may have some rough edges that we're continuously working on improving. We're always interested in hearing feedback and usecases that you might have. Feel free to reach out!

## Installation

### Building `multipy::runtime` via Docker
### Building via Docker

The easiest way to build multipy, along with fetching all interpreter dependencies, is to do so via docker.
The easiest way to build deploy and install the interpreter dependencies is to do so via docker.

```shell
git clone https://github.com/pytorch/multipy.git
git clone --recurse-submodules https://github.com/pytorch/multipy.git
cd multipy
export DOCKER_BUILDKIT=1
docker build -t multipy .
Expand All @@ -32,17 +40,23 @@ docker run --rm multipy multipy/runtime/build/test_deploy

### Installing via `pip install`

We support installing both python modules and the runtime libs using `pip install`, with the caveat of having to manually install the C++ dependencies first. This also serves as a single-command source build, essentially being a wrapper around `python setup.py develop`, once all the dependencies have been installed.
We support installing both python modules and the runtime libs using `pip
install`, with the caveat of having to manually install the C++ dependencies
first. This serves as a single-command source build, essentially being a wrapper
around `python setup.py develop`, once all the dependencies have been installed.


To start with, the multipy repo should be cloned first:

To start with, the multipy repo should be cloned:
```shell
git clone https://github.com/pytorch/multipy.git
git clone --recurse-submodules https://github.com/pytorch/multipy.git
cd multipy

# (optional) if using existing checkout
git submodule sync && git submodule update --init --recursive
```


#### Installing system dependencies
#### Installing System Dependencies

The runtime system dependencies are specified in `build-requirements.txt`. To install them on Debian-based systems, one could run:

Expand All @@ -51,7 +65,7 @@ sudo apt update
xargs sudo apt install -y -qq --no-install-recommends <build-requirements.txt
```

#### Installing environment encapsulators
#### Python Environment Setup

We support both `conda` and `pyenv`+`virtualenv` to create isolated environments to build and run in. Since `multipy` requires a position-independent version of python to launch interpreters with, for `conda` environments we use the prebuilt `libpython-static=3.x` libraries from `conda-forge` to link with at build time, and for `virtualenv`/`pyenv` we compile python with `-fPIC` to create the linkable library.

Expand Down Expand Up @@ -125,16 +139,22 @@ pip install -e . --install-option="--cmakeoff"

> **NOTE** As of 10/11/2022 the linking of prebuilt static fPIC versions of python downloaded from `conda-forge` can be problematic on certain systems (for example Centos 8), with linker errors like `libpython_multipy.a: error adding symbols: File format not recognized`. This seems to be an issue with `binutils`, and the steps in https://wiki.gentoo.org/wiki/Project:Toolchain/Binutils_2.32_upgrade_notes/elfutils_0.175:_unable_to_initialize_decompress_status_for_section_.debug_info can help. Alternatively, the user can go with the `virtualenv`/`pyenv` flow above.

## Development

### Running `multipy::runtime` build steps from source
### Manually building `multipy::runtime` from source

Both `docker` and `pip install` options above are wrappers around the `cmake build` of multipy's runtime. If the user wishes to run the build steps manually instead, as before the dependencies would have to be installed in the user's (isolated) environment of choice first. After that the following steps can be executed:
Both `docker` and `pip install` options above are wrappers around the `cmake`
build of multipy's runtime. For development purposes it's often helpful to
invoke `cmake` separately.

#### Building
See the install section for how to correctly setup the Python environment.

```bash
# checkout repo
git checkout https://github.com/pytorch/multipy.git
git clone --recurse-submodules https://github.com/pytorch/multipy.git
cd multipy

# (optional) if using existing checkout
git submodule sync && git submodule update --init --recursive

cd multipy
Expand Down