state_dict checkpoint before tensor storage serialization part

Author: motus

load/state_dict.ipynb

@@ -6,15 +6,18 @@
    "source": [
     "# PyTorch model (de)serialization\n",
     "\n",
+    "At the top level, serialization in PyTorch has two methods, `torch.save()` and `torch.load()`, implemented in [torch/serialization.py](https://github.com/pytorch/pytorch/blob/master/torch/serialization.py).\n",
+    "\n",
     "## Saving the model\n",
     "\n",
-    "In this example we will explore the serialization and deserialization of PyTorch model. We'll use the [MNIST model](https://github.com/pytorch/examples/tree/master/mnist) from previous examples, augmented with `torch.save()` call at the end.\n",
+    "Below we will explore the serialization and deserialization of PyTorch model.\n",
+    "We'll use the [MNIST model](https://github.com/pytorch/examples/tree/master/mnist) from PyTorch examples, augmented with `torch.save()` call at the end.\n",
     "\n",
     "We save the trained model like this:\n",
     "\n",
     "```python\n",
     "torch.save({\n",
-    "    'epoch': args.epochs, # == 10\n",
+    "    'epoch': args.epochs,  # == 10\n",
     "    'model_state_dict': model.state_dict(),\n",
     "    'optimizer_state_dict': optimizer.state_dict()\n",
     "}, './mnist-model.pt')\n",
@@ -59,7 +62,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "So the `torch.load()` function just reads back the dictionary that was passed to `torch.save()`, and for basic Python types it is not different from Python standard [`pickle`](https://docs.python.org/3.5/library/pickle.html) module (in fact, it *is* a pickle). The most interesting part here are the model's and optimizer's parameters, as returned from [`torch.nn.Module.state_dict()`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module.state_dict) method. Let's take a closer look."
+    "So the `torch.load()` function just reads back the dictionary that was passed to `torch.save()`, and for basic Python types it is not different from Python standard [pickle](https://docs.python.org/3.5/library/pickle.html) module (in fact, it *is* a pickle). The most interesting part here are the model's and optimizer's parameters, as returned from [`torch.nn.Module.state_dict()`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module.state_dict) method. Let's take a closer look."
    ]
   },
   {
@@ -140,7 +143,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Remember, that after the model instantiation its parameters are initialized with random values, e.g."
+    "Remember that after the model instantiation its parameters are initialized with random values, e.g."
    ]
   },
   {
@@ -208,11 +211,22 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "## Serialization across devices\n",
+    "\n",
+    "PyTorch documentation has a [good tutorial](https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-across-devices) on that."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Tensor serialization\n",
+    "\n",
+    "The model and optimizer serialization in PyTorch is built on the standard Python [pickle](https://docs.python.org/3.5/library/pickle.html) functionality - except for the tensor storage itself. That part is implemented in "
+   ]
   }
  ],
  "metadata": {