Skip to content

Commit 9269e73

Browse files
aporialiaofacebook-github-bot
authored andcommitted
Update correct DMP link on Tutorial Notebook (#2722)
Summary: Pull Request resolved: #2722 Noticed this link was not updated from recent update to docs when i went through the tutorial again Reviewed By: iamzainhuda Differential Revision: D68924131 fbshipit-source-id: c6434b08cd2a5c953e342341037c3d08132683f0
1 parent fa52570 commit 9269e73

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

TorchRec_Interactive_Tutorial_Notebook_OSS_version.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1818,7 +1818,7 @@
18181818
"source": [
18191819
"### DistributedModelParallel\n",
18201820
"\n",
1821-
"We have now explored sharding a single EmbeddingBagCollection! We were able to take the `EmbeddingBagCollectionSharder` and use the unsharded `EmbeddingBagCollection` to generate a `ShardedEmbeddingBagCollection` module. This workflow is fine, but typically when doing model parallel, [`DistributedModelParallel`](https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel) (DMP) is used as the standard interface. When wrapping your model (in our case `ebc`), with DMP, the following will occur:\n",
1821+
"We have now explored sharding a single EmbeddingBagCollection! We were able to take the `EmbeddingBagCollectionSharder` and use the unsharded `EmbeddingBagCollection` to generate a `ShardedEmbeddingBagCollection` module. This workflow is fine, but typically when doing model parallel, [`DistributedModelParallel`](https://pytorch.org/torchrec/model-parallel-api-reference.html#model-parallel) (DMP) is used as the standard interface. When wrapping your model (in our case `ebc`), with DMP, the following will occur:\n",
18221822
"\n",
18231823
"1. Decide how to shard the model. DMP will collect the available ‘sharders’ and come up with a ‘plan’ of the optimal way to shard the embedding table(s) (i.e, the EmbeddingBagCollection)\n",
18241824
"2. Actually shard the model. This includes allocating memory for each embedding table on the appropriate device(s).\n",

0 commit comments

Comments
 (0)