You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+61-43
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ This repository holds PyTorch bindings maintained by Intel for the Intel® oneAP
6
6
7
7
[PyTorch](https://github.com/pytorch/pytorch) is an open-source machine learning framework.
8
8
9
-
[Intel® oneCCL](https://github.com/oneapi-src/oneCCL) (collective communications library) is a library for efficient distributed deep learning training implementing such collectives like `allreduce`, `allgather`, `alltoall`. For more information on oneCCL, please refer to the [oneCCL documentation](https://spec.oneapi.com/versions/latest/elements/oneCCL/source/index.html) and [oneCCL specification](https://spec.oneapi.com/versions/latest/elements/oneCCL/source/index.html).
9
+
[Intel® oneCCL](https://github.com/oneapi-src/oneCCL) (collective communications library) is a library for efficient distributed deep learning training implementing such collectives like `allreduce`, `allgather`, `alltoall`. For more information on oneCCL, please refer to the [oneCCL documentation](https://spec.oneapi.com/versions/latest/elements/oneCCL/source/index.html).
10
10
11
11
`oneccl_bindings_for_pytorch` module implements PyTorch C10D ProcessGroup API and can be dynamically loaded as external ProcessGroup and only works on Linux platform now.
12
12
@@ -16,28 +16,28 @@ The table below shows which functions are available for use with CPU / Intel dGP
16
16
17
17
|| CPU | GPU |
18
18
| :--------------- | :---: | :---: |
19
-
|`send`| × |×|
20
-
|`recv`| × |×|
19
+
|`send`| × |√|
20
+
|`recv`| × |√|
21
21
|`broadcast`| √ | √ |
22
22
|`all_reduce`| √ | √ |
23
23
|`reduce`| √ | √ |
24
24
|`all_gather`| √ | √ |
25
25
|`gather`| √ | √ |
26
26
|`scatter`| × | × |
27
-
|`reduce_scatter`|×|×|
27
+
|`reduce_scatter`|√|√|
28
28
|`all_to_all`| √ | √ |
29
29
|`barrier`| √ | √ |
30
30
31
31
32
32
## Pytorch API Align
33
33
34
-
We recommend Anaconda as Python package management system. The following is the corresponding branches (tags) of `oneccl_bindings_for_pytorch` and supported Pytorch.
34
+
We recommend using Anaconda as Python package management system. The followings are the corresponding branches (tags) of `oneccl_bindings_for_pytorch` and supported Pytorch.
The usage details can be found in the README of corresponding branch. The following part is about the usage of v1.9 tag. if you want to use other version of torch-ccl please checkout to that branch(tag). For pytorch-1.5.0-rc3, the [#PR28068](https://github.com/pytorch/pytorch/pull/28068) and [#PR32361](https://github.com/pytorch/pytorch/pull/32361) are need to dynamicall register external ProcessGroup and enable `alltoall` collective communication primitive. The patch file about these two PRs is in `patches` directory and you can use it directly.
52
52
53
53
## Requirements
54
54
55
-
- Python 3.6 or later and a C++17 compiler
55
+
- Python 3.8 or later and a C++17 compiler
56
56
57
-
- PyTorch v1.13.0
57
+
- PyTorch v2.0.1
58
58
59
59
## Build Option List
60
60
61
61
The following build options are supported in Intel® oneCCL Bindings for PyTorch*.
**Note:** To run the torch-ccl without oneCCL library installed, Please make sure you have installed oneCCL in the oneAPI basekit from https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html#base-kit
104
+
105
+
### Install PreBuilt Wheel
106
+
107
+
Wheel files are avaiable for the following Python versions.
Wheel files are avaiable for the following Python versions.
124
+
- If oneccl_bindings_for_pytorch is built without oneCCL and use oneCCL in system, dynamic link oneCCl from oneAPI basekit (recommended usage):
125
+
126
+
```bash
127
+
source$basekit_root/ccl/latest/env/vars.sh
128
+
```
129
+
130
+
Note: Make sure you have installed [basekit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html#base-kit) when using Intel® oneCCL Bindings for Pytorch\* on Intel® GPUs.
For Point-to-point communication, directly call dist.send/recv after initializing the process group in launch script will trigger runtime error. Because all ranks of the group are expected to participate in this call to create communicators in our current implementation, while dist.send/recv only has a pair of ranks' participation. As a result, dist.send/recv should be used after collective call, which ensures all ranks' participation. The further solution for supporting directly call dist.send/recv after initializing the process group is still under investigation.
0 commit comments