|
1 | 1 | Customize Process Group Backends Using Cpp Extensions
|
2 | 2 | =====================================================
|
3 | 3 |
|
4 |
| -**Author**: `Howard Huang <https://github.com/H-Huang>`, `Feng Tian <https://github.com/ftian1>`__, `Shen Li <https://mrshenli.github.io/>`__, `Min Si <https://minsii.github.io/>`__ |
| 4 | +**Author**: `Howard Huang <https://github.com/H-Huang>`__, `Feng Tian <https://github.com/ftian1>`__, `Shen Li <https://mrshenli.github.io/>`__, `Min Si <https://minsii.github.io/>`__ |
5 | 5 |
|
6 | 6 | .. note::
|
7 | 7 | |edit| View and edit this tutorial in `github <https://github.com/pytorch/tutorials/blob/main/intermediate_source/process_group_cpp_extension_tutorial.rst>`__.
|
@@ -100,7 +100,7 @@ repository for the full implementation.
|
100 | 100 | // The collective communication APIs without a custom implementation
|
101 | 101 | // will error out if invoked by application code.
|
102 | 102 | };
|
103 |
| - |
| 103 | +
|
104 | 104 | class WorkDummy : public Work {
|
105 | 105 | public:
|
106 | 106 | WorkDummy(
|
@@ -266,8 +266,8 @@ After installation, you can conveniently use the ``dummy`` backend when calling
|
266 | 266 | `init_process_group <https://pytorch.org/docs/stable/distributed.html#torch.distributed.init_process_group>`__
|
267 | 267 | as if it is an builtin backend.
|
268 | 268 |
|
269 |
| -We can specify dispatching based on backend by changing the ``backend`` argument of ``init_process_group``. We |
270 |
| -can dispatch collective with CPU tensor to ``gloo`` backend and dispatch collective with CUDA tensor to ``dummy`` backend by |
| 269 | +We can specify dispatching based on backend by changing the ``backend`` argument of ``init_process_group``. We |
| 270 | +can dispatch collective with CPU tensor to ``gloo`` backend and dispatch collective with CUDA tensor to ``dummy`` backend by |
271 | 271 | specifying ``cpu:gloo,cuda:dummy`` as the backend argument.
|
272 | 272 |
|
273 | 273 | To send all tensors to ``dummy`` backend, we can simply specify ``dummy`` as the backend argument.
|
|
0 commit comments