mnist/1_mnist_load.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch MNIST example dissected\n",
    "\n",
    "In this notebook we'll explore the components of the\n",
    "[PyTorch MNIST example](https://github.com/pytorch/examples/tree/master/mnist)\n",
    "one-by-one.\n",
    "\n",
    "* Part 1: [Loading the data](1_mnist_load.ipynb) <-- **you are here**\n",
    "* Part 2: [Model components and forward propagation](2_mnist_model.ipynb)\n",
    "* Part 3: [Autodiff and backpropagation](3_mnist_backprop.ipynb)\n",
    "* Part 4: [Training the model](4_mnist_train.ipynb)\n",
    "* Part 5: [Visualizing the results](5_mnist_visualize.ipynb)\n",
    "\n",
    "In this notebook:\n",
    "\n",
    "* [1.1 Loading the data](#1.1-Loading-the-data)\n",
    "* [1.2 Transforming the data](#1.2-Transforming-the-data)\n",
    "* [1.3 Other available datasets and transforms](#1.3-Other-available-datasets-and-transforms)\n",
    "   * [1.3.1 Note on PIL performance](#1.3.1-Note-on-PIL-performance)\n",
    "* [1.4 Dataset and DataLoader](#1.4-Dataset-and-DataLoader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 Training data\n",
    "\n",
    "We start with the obligatory imports:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torchvision import datasets, transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torchvision` package has a lot of goodies related to image processing. Its `datasets` module already has a MNIST dataset class, and `transforms` has helpers to convert MNIST data to the format that can be used to train our model.\n",
    "\n",
    "**P.S.** There is also a `torchtext` package for NLP-related tasks.\n",
    "\n",
    "### 1.1 Loading the data\n",
    "\n",
    "This is how the MNIST example loads the training data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = datasets.MNIST(\n",
    "    '../data', train=True, download=True,\n",
    "    transform=transforms.Compose([\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize((0.1307,), (0.3081,))]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So it downloads the data and caches it in the `../data` directory. Set `train=False` to get the test data.\n",
    "\n",
    "We can also apply some transformations to the data as we load it. We'll play with it later. For now, we can just remember that `transform` parameter specifies the features' transformations, and the `target_transform` those for the labels.\n",
    "\n",
    "Here's how the data looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 60000\n",
       "    Split: train\n",
       "    Root Location: ../data\n",
       "    Transforms (if any): Compose(\n",
       "                             ToTensor()\n",
       "                             Normalize(mean=(0.1307,), std=(0.3081,))\n",
       "                         )\n",
       "    Target Transforms (if any): None"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The actual data is in `dataset.train_data` and `dataset.train_labels` fields:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  data: torch.Size([60000, 28, 28])\n",
      "labels: torch.Size([60000])\n"
     ]
    }
   ],
   "source": [
    "print(\"  data:\", dataset.train_data.size())\n",
    "print(\"labels:\", dataset.train_labels.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "i.e. each training example is 28x28 greyscale image, with 1 byte per pixel. Note that we do not apply the transformations to that (internal) data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,  51, 159, 253, 159,  50,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          48, 238, 252, 252, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  54,\n",
       "         227, 253, 252, 239, 233, 252,  57,   6,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  10,  60, 224,\n",
       "         252, 253, 252, 202,  84, 252, 253, 122,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 163, 252, 252,\n",
       "         252, 253, 252, 252,  96, 189, 253, 167,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  51, 238, 253, 253,\n",
       "         190, 114, 253, 228,  47,  79, 255, 168,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,  48, 238, 252, 252, 179,\n",
       "          12,  75, 121,  21,   0,   0, 253, 243,  50,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,  38, 165, 253, 233, 208,  84,\n",
       "           0,   0,   0,   0,   0,   0, 253, 252, 165,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   7, 178, 252, 240,  71,  19,  28,\n",
       "           0,   0,   0,   0,   0,   0, 253, 252, 195,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,  57, 252, 252,  63,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0, 253, 252, 195,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0, 198, 253, 190,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0, 255, 253, 196,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  76, 246, 252, 112,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0, 253, 252, 148,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  85, 252, 230,  25,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   7, 135, 253, 186,  12,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  85, 252, 223,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   7, 131, 252, 225,  71,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  85, 252, 145,   0,   0,   0,   0,   0,\n",
       "           0,   0,  48, 165, 252, 173,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  86, 253, 225,   0,   0,   0,   0,   0,\n",
       "           0, 114, 238, 253, 162,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  85, 252, 249, 146,  48,  29,  85, 178,\n",
       "         225, 253, 223, 167,  56,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  85, 252, 252, 252, 229, 215, 252, 252,\n",
       "         252, 196, 130,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,  28, 199, 252, 252, 253, 252, 252, 233,\n",
       "         145,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,  25, 128, 252, 253, 252, 141,  37,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
       "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0]],\n",
       "       dtype=torch.uint8)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.train_data[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADqJJREFUeJzt3X+sVPWZx/HPIy3+ACQiFxYteinixh+Jl82EbKLZsKk2sDZBohiIEtYQaQioNfVXMKbGaCLrtghxJV4WIsSWtqG48odZq6YRm9TGEUwR2d0avPIz3EuE1Gq0/Hj2j3tobvHOd4aZM3OG+7xfyc3MnOd873ky8LlnZr4z8zV3F4B4zim6AQDFIPxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4L6RisPNnbsWO/s7GzlIYFQenp6dPjwYatl34bCb2YzJK2UNEzSf7r706n9Ozs7VS6XGzkkgIRSqVTzvnU/7DezYZL+Q9JMSVdLmmdmV9f7+wC0ViPP+adJ+sjdd7v7XyT9XNKsfNoC0GyNhP9SSXsH3N6XbfsbZrbIzMpmVu7r62vgcADy1Ej4B3tR4WufD3b3bncvuXupo6OjgcMByFMj4d8naeKA29+SdKCxdgC0SiPhf1fSFDObZGbDJc2VtCWftgA0W91Tfe5+3MyWSnpN/VN969x9Z26dAWiqhub53f1VSa/m1AuAFuLtvUBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E1dIlujH07N27N1lfuXJlxdqKFSuSY++///5k/b777kvWJ06cmKxHx5kfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JqaJ7fzHokfSbphKTj7l7Koym0j/379yfrU6dOTdaPHj1asWZmybHPPvtssr5+/fpkva+vL1mPLo83+fyzux/O4fcAaCEe9gNBNRp+l/RrM3vPzBbl0RCA1mj0Yf/17n7AzMZJet3M/sfdtw7cIfujsEiSLrvssgYPByAvDZ353f1Adtkr6WVJ0wbZp9vdS+5e6ujoaORwAHJUd/jNbISZjTp1XdJ3JX2QV2MAmquRh/3jJb2cTdd8Q9LP3P2/c+kKQNPVHX533y3puhx7QQE++eSTZH369OnJ+pEjR5L11Fz+6NGjk2PPPffcZL23tzdZ3717d8Xa5Zdfnhw7bNiwZH0oYKoPCIrwA0ERfiAowg8ERfiBoAg/EBRf3T0EHDt2rGKt2lTejBkzkvVqX83diK6urmT9qaeeStZvuOGGZH3KlCkVa93d3cmxCxcuTNaHAs78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU8/xDwIMPPlix9txzz7WwkzPz1ltvJeuff/55sj579uxkffPmzRVr27dvT46NgDM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTFPP9ZoNpn6l966aWKNXdv6NjV5tJvvfXWZP3OO++sWJs4cWJy7FVXXZWsP/zww8n6pk2bKtYavV+GAs78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxCUVZvvNLN1kr4nqdfdr822jZH0C0mdknok3e7u6bWaJZVKJS+Xyw22PPTs378/Wb/uuvRK6EePHq372HfccUeyvmbNmmT9ww8/TNa3bdtWsTZ37tzk2AsuuCBZrya1zPaIESOSY3fu3JmsV3uPQlFKpZLK5XLlddEHqOXM/6Kk01d2eETSm+4+RdKb2W0AZ5Gq4Xf3rZI+PW3zLEnrs+vrJd2Sc18Amqze5/zj3f2gJGWX4/JrCUArNP0FPzNbZGZlMyv39fU1+3AAalRv+A+Z2QRJyi57K+3o7t3uXnL3UkdHR52HA5C3esO/RdKC7PoCSa/k0w6AVqkafjPbKOl3kv7ezPaZ2UJJT0u6ycz+KOmm7DaAs0jVz/O7+7wKpe/k3MuQdfjw4WR9+fLlyfqRI+m3UIwfP75ibdKkScmxixcvTtaHDx+erHd1dTVUL8oXX3yRrD/zzDPJ+qpVq/JspxC8ww8IivADQRF+ICjCDwRF+IGgCD8QFF/dnYPjx48n6w888ECynvrqbUkaPXp0sv7aa69VrF1xxRXJsceOHUvWo/r444+LbqHpOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDM8+dgz549yXq1efxq3nnnnWT9yiuvrPt3n3/++XWPxdmNMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU8fw6WLFmSrFdbBn327NnJeiPz+JGdPHmyYu2cc9LnvWr/ZkMBZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKrqPL+ZrZP0PUm97n5ttu1xSXdL6st2W+burzaryXawffv2irWtW7cmx5pZsj5nzpy6ekJaai6/2r9JqVTKu522U8uZ/0VJMwbZvsLdu7KfIR18YCiqGn533yrp0xb0AqCFGnnOv9TM/mBm68zsotw6AtAS9YZ/taTJkrokHZT040o7mtkiMyubWbmvr6/SbgBarK7wu/shdz/h7iclrZE0LbFvt7uX3L3U0dFRb58AclZX+M1swoCbsyV9kE87AFqllqm+jZKmSxprZvsk/UjSdDPrkuSSeiR9v4k9AmiCquF393mDbF7bhF7a2pdfflmx9tVXXyXHXnLJJcn6zTffXFdPQ93x48eT9VWrVtX9u2+77bZkfdmyZXX/7rMF7/ADgiL8QFCEHwiK8ANBEX4gKMIPBMVXd7fAeeedl6yPHDmyRZ20l2pTeatXr07WH3rooWS9s7OzYu3RRx9Njh0+fHiyPhRw5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoJjnb4H58+cX3UJh9u/fX7G2fPny5Njnn38+Wb/rrruS9TVr1iTr0XHmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgmOevkbvXVZOkF198MVl/7LHH6mmpLWzcuDFZv+eeeyrWjhw5khx77733JusrVqxI1pHGmR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqo6z29mEyVtkPR3kk5K6nb3lWY2RtIvJHVK6pF0u7unJ27PYmZWV02S9u3bl6w/8cQTyfrChQuT9VGjRlWs7dy5Mzn2hRdeSNbffvvtZL2npydZnzx5csXa3Llzk2OrzfOjMbWc+Y9L+qG7XyXpHyUtMbOrJT0i6U13nyLpzew2gLNE1fC7+0F335Zd/0zSLkmXSpolaX2223pJtzSrSQD5O6Pn/GbWKWmqpN9LGu/uB6X+PxCSxuXdHIDmqTn8ZjZS0q8k/cDd/3QG4xaZWdnMyn19ffX0CKAJagq/mX1T/cH/qbtvzjYfMrMJWX2CpN7Bxrp7t7uX3L3U0dGRR88AclA1/Nb/UvZaSbvc/ScDSlskLciuL5D0Sv7tAWiWWj7Se72k+ZJ2mNn72bZlkp6W9EszWyhpj6Q5zWnx7HfixIlkvdpU39q1a5P1MWPGVKzt2LEjObZRM2fOTNZnzJhRsbZ06dK828EZqBp+d/+tpEoT2d/Jtx0ArcI7/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dXdNbrmmmsq1m688cbk2DfeeKOhY1f7SHBqGexqxo1LfyRj8eLFyfrZ/LXj0XHmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgmOev0YUXXlixtmnTpuTYDRs2JOvN/IrqJ598Mlm/++67k/WLL744z3bQRjjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQ5u4tO1ipVPJyudyy4wHRlEollcvl9JrxGc78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBU1fCb2UQz+42Z7TKznWZ2X7b9cTPbb2bvZz//0vx2AeSlli/zOC7ph+6+zcxGSXrPzF7Paivc/d+b1x6AZqkafnc/KOlgdv0zM9sl6dJmNwaguc7oOb+ZdUqaKun32aalZvYHM1tnZhdVGLPIzMpmVu7r62uoWQD5qTn8ZjZS0q8k/cDd/yRptaTJkrrU/8jgx4ONc/dudy+5e6mjoyOHlgHkoabwm9k31R/8n7r7Zkly90PufsLdT0paI2la89oEkLdaXu03SWsl7XL3nwzYPmHAbrMlfZB/ewCapZZX+6+XNF/SDjN7P9u2TNI8M+uS5JJ6JH2/KR0CaIpaXu3/raTBPh/8av7tAGgV3uEHBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IqqVLdJtZn6RPBmwaK+lwyxo4M+3aW7v2JdFbvfLs7XJ3r+n78loa/q8d3Kzs7qXCGkho197atS+J3upVVG887AeCIvxAUEWHv7vg46e0a2/t2pdEb/UqpLdCn/MDKE7RZ34ABSkk/GY2w8z+18w+MrNHiuihEjPrMbMd2crD5YJ7WWdmvWb2wYBtY8zsdTP7Y3Y56DJpBfXWFis3J1aWLvS+a7cVr1v+sN/Mhkn6P0k3Sdon6V1J89z9w5Y2UoGZ9UgquXvhc8Jm9k+S/ixpg7tfm237N0mfuvvT2R/Oi9z94Tbp7XFJfy565eZsQZkJA1eWlnSLpH9Vgfddoq/bVcD9VsSZf5qkj9x9t7v/RdLPJc0qoI+25+5bJX162uZZktZn19er/z9Py1XorS24+0F335Zd/0zSqZWlC73vEn0VoojwXypp74Db+9ReS367pF+b2XtmtqjoZgYxPls2/dTy6eMK7ud0VVdubqXTVpZum/uunhWv81ZE+Adb/aedphyud/d/kDRT0pLs4S1qU9PKza0yyMrSbaHeFa/zVkT490maOOD2tyQdKKCPQbn7geyyV9LLar/Vhw+dWiQ1u+wtuJ+/aqeVmwdbWVptcN+104rXRYT/XUlTzGySmQ2XNFfSlgL6+BozG5G9ECMzGyHpu2q/1Ye3SFqQXV8g6ZUCe/kb7bJyc6WVpVXwfdduK14X8iafbCrjWUnDJK1z96da3sQgzOzb6j/bS/2LmP6syN7MbKOk6er/1NchST+S9F+SfinpMkl7JM1x95a/8Faht+nqf+j615WbTz3HbnFvN0h6W9IOSSezzcvU//y6sPsu0dc8FXC/8Q4/ICje4QcERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+IKj/BxmeJtv9WSKzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1f5ba9904e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.imshow(dataset.train_data[1], interpolation='none', aspect='equal', cmap='Greys')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To actually apply the transformations, we use `dataset.__getitem__()`, i.e."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[[-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242,  0.2249,  1.5996,  2.7960,  1.5996,  0.2122, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "            0.1867,  2.6051,  2.7833,  2.7833,  2.7833,  2.5924, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.2631,\n",
       "            2.4651,  2.7960,  2.7833,  2.6178,  2.5415,  2.7833,  0.3013,\n",
       "           -0.3478, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.2969,  0.3395,  2.4269,\n",
       "            2.7833,  2.7960,  2.7833,  2.1469,  0.6450,  2.7833,  2.7960,\n",
       "            1.1286, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242,  1.6505,  2.7833,  2.7833,\n",
       "            2.7833,  2.7960,  2.7833,  2.7833,  0.7977,  1.9814,  2.7960,\n",
       "            1.7014, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242,  0.2249,  2.6051,  2.7960,  2.7960,\n",
       "            1.9942,  1.0268,  2.7960,  2.4778,  0.1740,  0.5813,  2.8215,\n",
       "            1.7141, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242,  0.1867,  2.6051,  2.7833,  2.7833,  1.8541,\n",
       "           -0.2715,  0.5304,  1.1159, -0.1569, -0.4242, -0.4242,  2.7960,\n",
       "            2.6687,  0.2122, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242,  0.0595,  1.6759,  2.7960,  2.5415,  2.2233,  0.6450,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  2.7960,\n",
       "            2.7833,  1.6759, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.3351,  1.8414,  2.7833,  2.6306,  0.4795, -0.1824, -0.0678,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  2.7960,\n",
       "            2.7833,  2.0578, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "            0.3013,  2.7833,  2.7833,  0.3777, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  2.7960,\n",
       "            2.7833,  2.0578, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "            2.0960,  2.7960,  1.9942, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  2.8215,\n",
       "            2.7960,  2.0705, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.5431,\n",
       "            2.7069,  2.7833,  1.0013, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  2.7960,\n",
       "            2.7833,  1.4596, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6577,\n",
       "            2.7833,  2.5033, -0.1060, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.3351,  1.2941,  2.7960,\n",
       "            1.9432, -0.2715, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6577,\n",
       "            2.7833,  2.4142, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.3351,  1.2432,  2.7833,  2.4396,\n",
       "            0.4795, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6577,\n",
       "            2.7833,  1.4214, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242,  0.1867,  1.6759,  2.7833,  1.7778, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6704,\n",
       "            2.7960,  2.4396, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242,  1.0268,  2.6051,  2.7960,  1.6378, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6577,\n",
       "            2.7833,  2.7451,  1.4341,  0.1867, -0.0551,  0.6577,  1.8414,\n",
       "            2.4396,  2.7960,  2.4142,  1.7014,  0.2886, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,  0.6577,\n",
       "            2.7833,  2.7833,  2.7833,  2.4906,  2.3124,  2.7833,  2.7833,\n",
       "            2.7833,  2.0705,  1.2305, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.0678,\n",
       "            2.1087,  2.7833,  2.7833,  2.7960,  2.7833,  2.7833,  2.5415,\n",
       "            1.4214, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.1060,  1.2050,  2.7833,  2.7960,  2.7833,  1.3705,  0.0467,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242],\n",
       "          [-0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242,\n",
       "           -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242, -0.4242]]]),\n",
       " tensor(0))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is, for each training example it returns a **pair** of (features, label).\n",
    "\n",
    "Note the type and size of the data after the transformations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features size: torch.Size([1, 28, 28]) of type: torch.float32\n",
      "Label is: tensor(0) of type: torch.int64\n"
     ]
    }
   ],
   "source": [
    "(features, label) = dataset[1]\n",
    "\n",
    "print(\"Features size: %s of type: %s\" % (features.size(), features.dtype))\n",
    "print(\"Label is: %s of type: %s\" % (label, label.dtype))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can plot that data, too - so there is actually no need to reach into `dataset.train_data`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAD8CAYAAADJwUnTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF1lJREFUeJzt3X+0XWV95/H3J9cLgYAITYE0BKOSlKFOS/QapFhBWTiBuoys+gOmC5HaxrZiQZmpyJqlLGd1FroUK8MvrxIBi2BXQczMiiCl6QAtYH4YCXClpGmQazKJCQw/BELuvd/54+zYc+455zn73nvu3XuffF5r7ZVz9nfvZz+e4DfP8+xnP1sRgZlZlcwqugJmZhPlxGVmlePEZWaV48RlZpXjxGVmlePEZWaV48RlZpXjxGVmlePEZWaV85qZvNgBOjBmM2cmL2m2X3mFX/Jq7NFUyvhP75oTu58ZzXXs+kf23B0Ry6ZyvcmYUuKStAz4GtAHfDMirkgdP5s5nKTTp3JJM0t4OO6dchm7nxnlR3cfm+vYvnlPzp3yBSdh0olLUh9wDXAGMAyslbQqIh7vVuXMbOYFMMZY0dVImkqLaymwOSK2AEi6DVgOOHGZVVgQ7I18XcWiTCVxzQeervs+DJw0/iBJK4AVALM5eAqXM7OZ0sstrlYDgE1r5ETEIDAI8Fod4TV0zEouCEZLvtzVVBLXMLCg7vsxwLapVcfMymCsuQ1SKlNJXGuBRZLeAPwcOAf4z12plZkVJoDRXk1cETEi6ULgbmrTIVZGxGNdq5mZFaaXW1xExGpgdZfqYmYlEMDeHh7jMrMeFETvdhXNrEcFjJY7bzlxmVmj2sz5cnPiMrNxxGjLaZrl4cRlZg1qg/NOXGZWIbV5XE5cZlYxY25xmVmVVKHF5aWbzaxBIEaZlWvrRNICSWskDUl6TNJFLY45TdJzkjZm2+c6lesWl5k16WJXcQS4JCI2SDoUWC/pnhYLjt4fEe/NW6gTl5k1CMSr0dedsiK2A9uzzy9IGqK2lt+UFhx1V9HMGtQmoM7KtU2EpIXAEuDhFuGTJf1E0g8k/VanstziMrMmExicnytpXd33wWzx0AaSDgFuBy6OiOfHhTcAr4+IFyWdBdwJLEpd1InLzBpEiNHI3ZraFREDqQMk9VNLWrdExB3N1/v3RBYRqyVdK2luROxqV6YTl5k1GevSdAhJAm4AhiLiyjbHHA3siIiQtJTaENbuVLlOXGbWoDY437XUcApwHrBJ0sZs32XAsQARcT3wAeDPJI0ALwPnRKQXBHPiMrMG+wbnu1JWxAO0frFO/TFXA1dPpFwnLjNrMupHfsysSvbNnC8zJy4zazKW/65iIZy4zKxB7SFrJy7rYSPvfmsyvv3P97SN/eTkm5Ln/s6D5yfjv3HNAcl435oNybi1Foi9XXrkZ7o4cZlZgwgmMgG1EE5cZjaOujYBdbo4cZlZg8AtLjOrIA/Om1mlBPKa82ZWLbXXk5U7NZS7dmZWgB5/IaykrcALwCgw0mldHquesVOXJONXrUw/G3tcf/v/xDq95v3HJ38rGX9iYDQZ/68L397hCtZKsH/MnH9XasEvM6uenm5xmVnviVDPt7gC+KGkAL7eaq1pM6uW2uB8bz/yc0pEbJN0JHCPpJ9GxH31B0haAawAmM3BU7ycmU2/Ca05X4gp1S4itmV/7gS+ByxtccxgRAxExEA/B07lcmY2A2qD88q1FWXSiUvSnOzNtEiaA7wHeLRbFTOz4owyK9dWlKl0FY8Cvld7iQevAb4TEXd1pVZmVpienjkfEVuA3+liXawAe9+Tnnr3l9d+Oxlf3J9eE2ssMVtry969yXOfG0sPLSzpMPKw58y3tY0dtGZT8tyxV15JF97juvWyjOni6RBm1iAC9o45cZlZhdS6ik5cZlYxnjlvZpWybzpEmZW7PWhmBah1FfNsHUuSFkhaI2lI0mOSLmpxjCRdJWmzpEckvaVTuW5xmVmTLq45PwJcEhEbsnmf6yXdExGP1x1zJrAo204Crsv+bMuJqwf0vfa1bWO/fOfxyXM/9dXvJOPvOujFDleffKP9xmd/Nxm/99qTk/F/uvyqZPyeb17fNnbC31yYPPeNn3kwGe9ltbuK3XlWMSK2A9uzzy9IGgLmA/WJazlwc0QE8JCk10mal53bkhOXmTWY4ATUuZLW1X0fbLfYgqSFwBLg4XGh+cDTdd+Hs31OXGaW3wS6irvyLCAq6RDgduDiiHh+fLjFKZEqz4nLzBp0+66ipH5qSeuWiLijxSHDwIK678cA21Jl+q6imTXp4l1FATcAQxFxZZvDVgEfye4uvh14LjW+BW5xmdk4EWKkezPnTwHOAzZJ2pjtuww4tnatuB5YDZwFbAZeAi7oVKgTl5k16VZXMSIeoPUYVv0xAXxiIuU6cZlZgyrMnHfi6gHDN89vG1v7tmtmsCYT84Uj1ybjdx2Snud1wdb3JOM3Lfz7trHXnrA7ee7+zonLzCqlpxcSNLPe1cVHfqaFE5eZNYiAES8kaGZV466imVWKx7jMrJLCicvMqsaD8zZlI+9+azJ+64lXt43NIv36sE4ueOr0ZHzd3/+HZHzTx9rXbc3Ls5PnHrnu5WR887Pptcb6/8eatrFZ5f7/ZaEiPMZlZpUjRn1X0cyqxmNcZlYpflbRzKonauNcZebEZWZNfFfRzColPDhvZlVU+a6ipJXAe4GdEfHmbN8RwHeBhcBW4EMR8ez0VbO3jZ26JBm/amX7uVAAx/W3/2scYyx57vt+enYy3veBXybjr/v99H/hJ3y7/fsLF1/zdNsYwKynf5yMH35/MszevxptG7v9t1cmz/2jd/1FMt63ZkP64hVX9ruKedqDNwLLxu27FLg3IhYB92bfzawHRNQSV56tKB0TV0TcBzwzbvdy4Kbs803A+7tcLzMr0Fgo11aUyY5xHbXv9UERsV3SkV2sk5kVrPJjXFMlaQWwAmA2B0/35cxsigIxVvK7ipOt3Q5J8wCyP3e2OzAiBiNiICIG+jlwkpczs5kUObeiTDZxrQLOzz6fD3y/O9Uxs8L1wuC8pFuBB4HflDQs6WPAFcAZkp4Ezsi+m1mv6FKTS9JKSTslPdomfpqk5yRtzLbP5alexzGuiDi3TSi9UJP9it76W8n4rk+n151a3J9eU2v9nvaxf3jxhOS5u29bkIz/2rMPJuOH/c1D6XgiNpI8c3od1Zcetth98UvJ+JHtl/rqCV1sTd0IXA3cnDjm/oh470QK9cx5M2sQwNhYdxJXRNwnaWFXCqtT7lsHZjbzAgjl27rjZEk/kfQDSenuScYtLjNrMoF5XHMlrav7PhgRgxO41Abg9RHxoqSzgDuBRZ1OcuIys2b5E9euiBiY9GUinq/7vFrStZLmRsSu1HlOXGY2zsxNdZB0NLAjIkLSUmrDV7s7nefEZWbNujS7NJtOdRq1LuUw8HmgHyAirgc+APyZpBHgZeCciM4dVSeuLph1cPpRppEvPZ+MP3T8Hcn4v428mox/+rJL2sYOv/9nyXOPnNP2oQcA2i8M09uWznsqGd86M9UoRkB0765iu+lU++JXU5suMSFOXGbWQrnX43LiMrNm+/vqEGZWQU5cZlYp+yaglpgTl5k12e8XEjSzCurSXcXp4sRlZk3kFlfve/nU9HOhdx9/7ZTK/+OLPpWMH3pn+6Vlilw6xiqq6OVNc3DiMrNxurryw7Rw4jKzZm5xmVnlpF+AXjgnLjNr5HlcZlZFvqtoZtVT8sTlNefNrHLc4uqC3/7vG5PxWR3+fbjgqfSb3g6680cTrpNBv/raxvZ2aFH0lb2vNM3K/j/ficvMGgV+5MfMKsgtLjOrGncVzax6nLjMrHKcuMysShTuKppZFVX9rqKklcB7gZ0R8eZs3+XAnwC/yA67LCJWT1cly+D/nXdy29h/O+rLyXPHOCAZX//DE5LxY/nnZNxa2xvt3wo51uEp4ruG0n8ni9gwqTpVRdlbXHlmzt8ILGux/6sRcWK29XTSMtvvRM6tIB0TV0TcBzwzA3UxszKIfx/n6rR1ImmlpJ2SHm0Tl6SrJG2W9Iikt+Sp4lSeVbwwu9BKSYdPoRwzK5vutbhupHWPbZ8zgUXZtgK4Lk+hk01c1wFvAk4EtgNfaXegpBWS1klat5c9k7ycmc0kjeXbOsnRY1sO3Bw1DwGvkzSvU7mTSlwRsSMiRiNiDPgGsDRx7GBEDETEQD8HTuZyZlZec/c1TLJtxQTPnw88Xfd9ONuXNKnpEJLmRcT27OvZQMv+q5lVVP6B910RMTCFK7Wad9Hx6nmmQ9wKnEYtsw4DnwdOk3RidoGtwMcnUlMzK7GZnYA6DCyo+34MsK3TSR0TV0Sc22L3Dfnr1RtGDmofO2xWep7Wg6+ku8hvvDn997S/vhtx1sEHJ+M//fKbO5Swvm3kD7ecmTzz+Iv+LRlvP0OsR8xc4lpF7UbfbcBJwHN1vbm2PHPezJp1KXG16bH1A0TE9cBq4CxgM/AScEGecp24zKyByHfHMI82Pbb6eACfmGi5Tlxm1sgPWZtZJTlxmVnlOHGZWdW4q2jsHj0kGR/ZsnVmKlIynaY7PHHFf0zGf7r86mT8By8d1ja27Zrjkuce+uxDyXjPc+Iys0qJ7t1VnC5OXGbWzC0uM6saj3GZWfU4cZlZpRS8LHMeTlxm1kC4q2hmFeTEZfyXf/pgMr44sfxK1Y2duqRtbOenX06eOzSQnqd1+qYPJ+Nzlm1pGzuU/XyeVidOXGZWOU5cZlYpXh3CzCrJicvMqsaP/JhZ5biraGbV4gmoZlZJTlw9otVrKzOzOrwQ/GvvuDUZv4bFk6lRKTz1hZOT8ds/cmXb2OL+9Gvd3vKj85Px3zj78WTcJscz582skjRW7szlxGVmjTzGZWZVVPauYnpwxsz2T5Fzy0HSMklPSNos6dIW8Y9K+oWkjdn2x53KdIvLzJp0q8UlqQ+4BjgDGAbWSloVEePvrHw3Ii7MW65bXGbWrHstrqXA5ojYEhGvArcBy6daPScuM2uUveUnz5bDfODpuu/D2b7x/kDSI5L+TtKCToV27CpmhdwMHA2MAYMR8TVJRwDfBRYCW4EPRcSzncqrrMS/LmOk/wZPPWh3Mn7xjW9Nxt/0rXT5/f/3hbaxHaf+evLcIz48nIx/8th7k/EzD06vJbbql0e1jX1k07LkuXO/PicZt+kxwXlccyWtq/s+GBGD44obb3zp/wu4NSL2SPpT4Cbg3amL5hnjGgEuiYgNkg4F1ku6B/gocG9EXJENuF0KfCZHeWZWdpE7c+2KiIFEfBiob0EdA2xrvFTU/8v+DeCLnS7asasYEdsjYkP2+QVgiFpTbzm1zEj25/s7lWVm1aDIt+WwFlgk6Q2SDgDOAVY1XEuaV/f1fdRyTNKE7ipKWggsAR4GjoqI7VBLbpKOnEhZZlZSXZyAGhEjki4E7gb6gJUR8ZikLwDrImIV8BeS3ketd/cMtd5cUu7EJekQ4Hbg4oh4Xko8vNd43gpgBcBsDs57OTMrUDfX44qI1cDqcfs+V/f5s8BnJ1JmrruKkvqpJa1bIuKObPeOfU287M+dbSo9GBEDETHQz4ETqZuZFaSLdxWnRcfEpVrT6gZgKCLqH/VfBex7fP984Pvdr56ZzbigNjifZytInq7iKcB5wCZJG7N9lwFXAH8r6WPAz4D0O7j2Y7OV/pmHzrg+GX/g92Yn40/uObpt7ILDtibPnaqLtv1eMn7XP5/YNrboIr8irKzK/qxix8QVEQ/QfjWq07tbHTMrhaonLjPbv3ghQTOrnggvJGhmFVTuvOXEZWbN3FU0s2oJwF1FM6uccuctJ668jvrHlg8GAPCZj6df0fXFox+c0rXfOfvVZPwds7dOuuwf70nPQT73/6xIxhdfkF7WZhGeq1VF7iqaWeX4rqKZVYtfT2ZmVVObgFruzOXEZWbNClz5IQ8nLjNr4haXmVWLx7jMrHr8rGLPGP2Xf20be/KDC5PnnvDJTybjj3/of06mSrkcv/rPk/HfvPalZHzxj9PztKxHuatoZpUSxS7LnIcTl5k1c4vLzCqn3HnLicvMmmms3H1FJy4zaxR4AqqZVYuI0k9AzfVCWDPbz3TxvYqSlkl6QtJmSZe2iB8o6btZ/GFJCzuV6RZXF4xs2ZqMH/epdPx9n3pb9yozzmLWJuPl/nfVCtOlFpekPuAa4AxgGFgraVVEPF532MeAZyPiOEnnAF8EPpwq1y0uM2u0b4wrz9bZUmBzRGyJiFeB24Dl445ZDtyUff474HRJ7d7lCrjFZWYtTOCu4lxJ6+q+D0bEYN33+cDTdd+HgZPGlfGrYyJiRNJzwK8Bu9pd1InLzMbJP34F7IqIgUS8VctpfOF5jmngrqKZNQq6OTg/DCyo+34MsK3dMZJeAxwGPJMq1InLzJp1b4xrLbBI0hskHQCcA6wad8wq4Pzs8weAf4hIZ0V3Fc2sSbfmcWVjVhcCdwN9wMqIeEzSF4B1EbEKuAH4tqTN1Fpa53Qq14nLzJp1cQJqRKwGVo/b97m6z68AH5xImR27ipIWSFojaUjSY5IuyvZfLunnkjZm21kTubCZlVQEjI7l2wqSp8U1AlwSERskHQqsl3RPFvtqRHx5+qpnZoUo+SM/HRNXRGwHtmefX5A0RG3ehZn1qpInrgndVcyeIVoCPJztulDSI5JWSjq8zTkrJK2TtG4ve6ZUWTObAQGMRb6tILkTl6RDgNuBiyPieeA64E3AidRaZF9pdV5EDEbEQEQM9HNgF6psZtMrIMbybQXJdVdRUj+1pHVLRNwBEBE76uLfAP73tNTQzGZWUOjAex557iqK2jyLoYi4sm7/vLrDzgYe7X71zKwQXVzWZjrkaXGdApwHbJK0Mdt3GXCupBOp5eetwMenpYZmNvNKPjif567iA7R+CHJ1i31mVnnFtqby8Mx5M2sUgF+WYWaV4xaXmVVLlP6uohOXmTUKiALnaOXhxGVmzQqcFZ+HE5eZNfMYl5lVSoTvKppZBbnFZWbVEsToaNGVSHLiMrNG+5a1KTEnLjNr5ukQZlYlAYRbXGZWKRFucZlZ9ZR9cF4dXhjb3YtJvwCeqts1F9g1YxWYmLLWraz1AtdtsrpZt9dHxK9PpQBJd1GrUx67ImLZVK43GTOauJouLq2LiIHCKpBQ1rqVtV7guk1WmetWVhN6y4+ZWRk4cZlZ5RSduAYLvn5KWetW1nqB6zZZZa5bKRU6xmVmNhlFt7jMzCaskMQlaZmkJyRtlnRpEXVoR9JWSZskbZS0ruC6rJS0U9KjdfuOkHSPpCezPw8vUd0ul/Tz7LfbKOmsguq2QNIaSUOSHpN0Uba/0N8uUa9S/G5VMuNdRUl9wL8AZwDDwFrg3Ih4fEYr0oakrcBARBQ+50fSO4EXgZsj4s3Zvi8Bz0TEFVnSPzwiPlOSul0OvBgRX57p+oyr2zxgXkRskHQosB54P/BRCvztEvX6ECX43aqkiBbXUmBzRGyJiFeB24DlBdSj9CLiPuCZcbuXAzdln2+i9h/+jGtTt1KIiO0RsSH7/AIwBMyn4N8uUS+boCIS13zg6brvw5TrLy+AH0paL2lF0ZVp4aiI2A61/yMARxZcn/EulPRI1pUspBtbT9JCYAnwMCX67cbVC0r2u5VdEYmr1Vuxy3Rr85SIeAtwJvCJrEtk+VwHvAk4EdgOfKXIykg6BLgduDgini+yLvVa1KtUv1sVFJG4hoEFdd+PAbYVUI+WImJb9udO4HvUurZlsiMbK9k3ZrKz4Pr8SkTsiIjRqL3b6hsU+NtJ6qeWHG6JiDuy3YX/dq3qVabfrSqKSFxrgUWS3iDpAOAcYFUB9WgiaU42aIqkOcB7gEfTZ824VcD52efzge8XWJcG+5JC5mwK+u0kCbgBGIqIK+tChf527epVlt+tSgqZgJrd7v1roA9YGRF/NeOVaEHSG6m1sqC25M93iqybpFuB06g9qb8D+DxwJ/C3wLHAz4APRsSMD5K3qdtp1Lo7AWwFPr5vTGmG6/YO4H5gE7BvYanLqI0nFfbbJep1LiX43arEM+fNrHI8c97MKseJy8wqx4nLzCrHicvMKseJy8wqx4nLzCrHicvMKseJy8wq5/8Dj+lHisSqpgMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1f5b52962e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(features[0])\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Transforming the data\n",
    "\n",
    "We can apply the transforms to the raw input data manually, just to see how they work.\n",
    "\n",
    "Each transform implements the `.__call__()` method that actually transforms the data. Note that this means that the transform can also have a state that cnahges from one input value to the next. (This can be useful e.g. for some sliding window smoothing etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class ToTensor in module torchvision.transforms.transforms:\n",
      "\n",
      "class ToTensor(builtins.object)\n",
      " |  Convert a ``PIL Image`` or ``numpy.ndarray`` to tensor.\n",
      " |  \n",
      " |  Converts a PIL Image or numpy.ndarray (H x W x C) in the range\n",
      " |  [0, 255] to a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0].\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __call__(self, pic)\n",
      " |      Args:\n",
      " |          pic (PIL Image or numpy.ndarray): Image to be converted to tensor.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Tensor: Converted image.\n",
      " |  \n",
      " |  __repr__(self)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(transforms.ToTensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `dataset.train_data` is a PyTorch tensor, and `ToTensor()` accepts either a PIL image or a numpy array. I guess that the MNIST dataset performs that conversion implicitly (i.e. it uses `torch.tensor` just for storage). Anyway, we have to call `.numpy()` to make things work here.\n",
    "\n",
    "Also note that transforms work on **batch** data. Here we convert *two* first examples into a `float32` tensor: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data size after the transformation: torch.Size([28, 2, 28])\n",
      "\n",
      "tensor([[[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]],\n",
      "\n",
      "        [[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]],\n",
      "\n",
      "        [[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]],\n",
      "\n",
      "        ...,\n",
      "\n",
      "        [[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]],\n",
      "\n",
      "        [[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]],\n",
      "\n",
      "        [[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]]])\n"
     ]
    }
   ],
   "source": [
    "transformed_data = transforms.ToTensor()(dataset.train_data[:2].numpy())\n",
    "\n",
    "print(\"Data size after the transformation: %s\\n\" % str(transformed_data.size()))\n",
    "print(transformed_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So each (`unit8`) pixel is converted into a `float32` value of `[0.0 ... 1.0]` range.\n",
    "\n",
    "(I have no idea why the batch size dimension slips into the middle between image height and width.)\n",
    "\n",
    "Transforms are composable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data size after the transformation: torch.Size([28, 2, 28])\n",
      "\n",
      "tensor([[[-0.4242, -0.4242, -0.4242,  ..., -0.4242, -0.4242, -0.4242],\n",
      "         [-0.4242, -0.4242, -0.4242,  ..., -0.4242, -0.4242, -0.4242]],\n",
      "\n",
      "        [[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
      "         [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],\n",
      "\n",
      "        [[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
      "         [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],\n",
      "\n",
      "        ...,\n",
      "\n",
      "        [[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
      "         [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],\n",
      "\n",
      "        [[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
      "         [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],\n",
      "\n",
      "        [[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],\n",
      "         [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]]])\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose([\n",
    "    transforms.ToTensor(),\n",
    "    transforms.Normalize((0.1307,), (0.3081,))])\n",
    "\n",
    "transformed_data = transform(dataset.train_data[:2].numpy())\n",
    "\n",
    "print(\"Data size after the transformation: %s\\n\" % str(transformed_data.size()))\n",
    "print(transformed_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we end up with the same data as we get from `dataset.__getitem__()` (modulo the weird dimensions).\n",
    "\n",
    "`transforms.Compose()` applies the transforms as they go in the list, i.e. in our example it first invokes `ToTensor`, and then feeds the resulting tensor into `Normalize`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Other available datasets and transforms\n",
    "\n",
    "There are many other prepackaged datasets and transforms available in `torchvision`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('LSUN',\n",
       " 'LSUNClass',\n",
       " 'ImageFolder',\n",
       " 'DatasetFolder',\n",
       " 'FakeData',\n",
       " 'CocoCaptions',\n",
       " 'CocoDetection',\n",
       " 'CIFAR10',\n",
       " 'CIFAR100',\n",
       " 'EMNIST',\n",
       " 'FashionMNIST',\n",
       " 'MNIST',\n",
       " 'STL10',\n",
       " 'SVHN',\n",
       " 'PhotoTour',\n",
       " 'SEMEION',\n",
       " 'Omniglot')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets.__all__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['CenterCrop',\n",
       " 'ColorJitter',\n",
       " 'Compose',\n",
       " 'FiveCrop',\n",
       " 'Grayscale',\n",
       " 'Lambda',\n",
       " 'LinearTransformation',\n",
       " 'Normalize',\n",
       " 'Pad',\n",
       " 'RandomAffine',\n",
       " 'RandomApply',\n",
       " 'RandomChoice',\n",
       " 'RandomCrop',\n",
       " 'RandomGrayscale',\n",
       " 'RandomHorizontalFlip',\n",
       " 'RandomOrder',\n",
       " 'RandomResizedCrop',\n",
       " 'RandomRotation',\n",
       " 'RandomSizedCrop',\n",
       " 'RandomVerticalFlip',\n",
       " 'Resize',\n",
       " 'Scale',\n",
       " 'TenCrop',\n",
       " 'ToPILImage',\n",
       " 'ToTensor']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[clazz for clazz in dir(transforms) if clazz[0].isupper()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's also a bunch of functions defined in `transforms.functional` that can come in handy if you have to write your own image transforms.\n",
    "\n",
    "#### 1.3.1 Note on PIL performance\n",
    "\n",
    "By default, `torchvision` uses [Pillow](https://python-pillow.org) package as modern replacement of the good old `PIL` Python Image Manipulation library (see [torchvision image backend](https://github.com/pytorch/vision#image-backend)).\n",
    "\n",
    "If you have a newer generation Intel or AMD chip, you can instead use a drop-in replacement for `Pillow`, called [Pillow-SIMD](https://github.com/uploadcare/pillow-simd), that can utilize AVX and SSE instructions.\n",
    "\n",
    "To see what instructions are supported on your PC, in Linux you can look at `/proc/cpuinfo` and check gcc capabilities with `gcc --help=target`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 Dataset and DataLoader\n",
    "\n",
    "To feed data into the model, PyTorch uses `torch.utils.data.DataLoader` class. It shuffles and batches up the data and parallelizes the loading process. It also loads data into GPU, if it is available.\n",
    "\n",
    "The data loader's input is an instance of `torch.utils.data.Dataset`. You will inherit from this class if you have your own dataset in some proprietary format. Each dataset must provide random access and iteration over the data, and return the number of batches in `len()` call. In other words, it has to implement `.__getitem__()` and `.__len__()` methods.\n",
    "\n",
    "The `DataLoader` is also an iterable over the batches; each batch is a pair of (features, labels):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "938"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_loader = torch.utils.data.DataLoader(dataset, batch_size=64, shuffle=True)\n",
    "\n",
    "len(data_loader)  # Number of batches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features: torch.Size([64, 1, 28, 28])\n",
      "  Labels: torch.Size([64])\n"
     ]
    }
   ],
   "source": [
    "data_iter = iter(data_loader)\n",
    "\n",
    "(features, labels) = data_iter.next()\n",
    "\n",
    "print(\"Features:\", features.size())\n",
    "print(\"  Labels:\", labels.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Dimension of `1` in the features means that the image is greyscale; it can be, e.g., `3` for RGB pictures.\n",
    " \n",
    " In [Part 2](2_mnist_model.ipynb) we'll see how training data flows through the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}