exercises/01_penguin_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 1: Classifying penguin species with PyTorch\n",
    "\n",
    "<img src=\"https://allisonhorst.github.io/palmerpenguins/reference/figures/lter_penguins.png\" width=\"750\" />\n",
    "\n",
    "\n",
    "Artwork by @allison_horst\n",
    "\n",
    "In this exercise, we will use the python package [``palmerpenguins``](https://github.com/mcnakhaee/palmerpenguins) to supply a toy dataset containing various features and measurements of penguins.\n",
    "\n",
    "We have already created a PyTorch dataset which yields data for each of the penguins, but first we should examine the dataset and see what it contains."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1: look at the data\n",
    "In the following code block, we import the ``load_penguins`` function from the ``palmerpenguins`` package.\n",
    "\n",
    "- Call this function, which returns a single object, and assign it to the variable ``data``.\n",
    "  - Print ``data`` and recognise that ``load_penguins`` has returned a ``pandas.DataFrame``.\n",
    "- Consider which features it might make sense to use in order to classify the species of the penguins.\n",
    "  - You can print the column titles using ``pd.DataFrame.keys()``\n",
    "  - You can also obtain useful information using ``pd.DataFrame.Series.describe()``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from palmerpenguins import load_penguins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now discuss the features we will use to classify the penguins' species, and populate the following list together:\n",
    "- ...\n",
    "- ...\n",
    "- ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2: creating a ``torch.utils.data.Dataset``\n",
    "\n",
    "The penguin data reading and processing can be encapsulated in a PyTorch dataset class.\n",
    "\n",
    "- Why is a class representation helpful?\n",
    "\n",
    "All PyTorch dataset objects are subclasses of the ``torch.utils.data.Dataset`` class. To make a custom dataset, create a class which inherits from the ``Dataset`` class, implement some methods (the Python magic (or dunder) methods ``__len__`` and ``__getitem__``) and supply some data.\n",
    "\n",
    "Spoiler alert: we've done this for you already below (see ``src/ml_workshop/_penguins.py`` for a more sophisticated implementation)\n",
    "\n",
    "- Open the file ``src/ml_workshop/_penguins.py``.\n",
    "- Let's examine, and discuss, each of the methods together.\n",
    "  - ``__len__``\n",
    "    - What does the ``__len__`` method do?\n",
    "    - ...\n",
    "  - ``__getitem__``\n",
    "    - What does the ``__getitem__`` method do?\n",
    "    - ...\n",
    "- Review and discuss the class arguments.\n",
    "  - ``input_keys``— ...\n",
    "  - ``target_keys``— ...\n",
    "  - ``train``— ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List, Tuple, Any\n",
    "\n",
    "# import some useful functions here, see https://pytorch.org/docs/stable/torch.html\n",
    "# where `tensor` and `eye` are used for constructing tensors,\n",
    "# and using a lower-precision float32 is advised for performance\n",
    "# Task 4: add imports here\n",
    "# from torch import tensor, eye, float32\n",
    "\n",
    "from torch.utils.data import Dataset\n",
    "\n",
    "from palmerpenguins import load_penguins\n",
    "\n",
    "\n",
    "class PenguinDataset(Dataset):\n",
    "    \"\"\"Penguin dataset class.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    input_keys : List[str]\n",
    "        The column titles to use in the input feature vectors.\n",
    "    target_keys : List[str]\n",
    "        The column titles to use in the target feature vectors.\n",
    "    train : bool\n",
    "        If ``True``, this object will serve as the training set, and if\n",
    "        ``False``, the validation set.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    The validation split contains 10 male and 10 female penguins of each\n",
    "    species.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(\n",
    "        self,\n",
    "        input_keys: List[str],\n",
    "        target_keys: List[str],\n",
    "        train: bool,\n",
    "    ):\n",
    "        \"\"\"Build ``PenguinDataset``.\"\"\"\n",
    "        self.input_keys = input_keys\n",
    "        self.target_keys = target_keys\n",
    "\n",
    "        data = load_penguins()\n",
    "        data = (\n",
    "            data.loc[~data.isna().any(axis=1)]\n",
    "            .sort_values(by=sorted(data.keys()))\n",
    "            .reset_index(drop=True)\n",
    "        )\n",
    "        # Transform the sex field into a float, with male represented by 1.0, female by 0.0\n",
    "        data.sex = (data.sex == \"male\").astype(float)\n",
    "        self.full_df = data\n",
    "\n",
    "        valid_df = self.full_df.groupby(by=[\"species\", \"sex\"]).sample(\n",
    "            n=10,\n",
    "            random_state=123,\n",
    "        )\n",
    "        # The training items are simply the items *not* in the valid split\n",
    "        train_df = self.full_df.loc[~self.full_df.index.isin(valid_df.index)]\n",
    "\n",
    "        self.split = {\"train\": train_df, \"valid\": valid_df}[\n",
    "            \"train\" if train is True else \"valid\"\n",
    "        ]\n",
    "\n",
    "    def __len__(self) -> int:\n",
    "        \"\"\"Return the length of requested split.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        int\n",
    "            The number of items in the dataset.\n",
    "\n",
    "        \"\"\"\n",
    "        return len(self.split)\n",
    "\n",
    "    def __getitem__(self, idx: int) -> Tuple[Any, Any]:\n",
    "        \"\"\"Return an input-target pair.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        idx : int\n",
    "            Index of the input-target pair to return.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        in_feats : Any\n",
    "            Inputs.\n",
    "        target : Any\n",
    "            Targets.\n",
    "\n",
    "        \"\"\"\n",
    "        # get the row index (idx) from the dataframe and\n",
    "        # select relevant column features (provided as input_keys)\n",
    "        feats = tuple(self.split.iloc[idx][self.input_keys])\n",
    "\n",
    "        # this gives a 'species' i.e. one of ('Gentoo',), ('Chinstrap',), or ('Adelie',)\n",
    "        tgts = tuple(self.split.iloc[idx][self.target_keys])\n",
    "\n",
    "        # Task 4 - Exercise #1: convert the features to PyTorch Tensors\n",
    "\n",
    "        # Task 4 - Exercise #2: convert target to a 'one-hot' vector.\n",
    "\n",
    "        return feats, tgts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3: Obtaining training and validation datasets\n",
    "\n",
    "- Instantiate the penguin dataloader.\n",
    "  - Make sure you supply the correct column titles for the features and the targets.\n",
    "- Iterate over the dataset\n",
    "    - Hint:\n",
    "        ```python\n",
    "        for features, targets in dataset:\n",
    "            # print the features and targets here\n",
    "        ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_set = PenguinDataset(\n",
    "    input_keys=[\"bill_length_mm\", \"body_mass_g\"],\n",
    "    target_keys=[\"species\"],\n",
    "    train=True,\n",
    ")\n",
    "\n",
    "\n",
    "for features, target in data_set:\n",
    "    # print the features and targets here\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Can we give these items to a neural network, or do they need to be transformed first?\n",
    "  - Short answer: no, we can't just pass tuples of numbers or strings to a neural network.\n",
    "    - We must represent these data as ``torch.Tensor``s. This is the fundamental data abstraction used by PyTorch; they are the PyTorch equivalent to Numpy arrays, while also providing support for GPU acceleration. See [pytorch tensors documentation](https://pytorch.org/tutorials/beginner/introyt/tensors_deeper_tutorial.html).\n",
    "    - The targets are tuples of strings i.e. ('Gentoo', )\n",
    "      - One idea is to represent as ordinal values i.e.  [1] or [2] or [3]. But this implies that the class encoded by value 1 is closer to 2 than 1 is to 3. This is not desirable for categorical data. One-hot encoding avoids this by representing each species independently.\\\n",
    "      \"A\" — [1, 0, 0]\\\n",
    "      \"B\" — [0, 1, 0]\\\n",
    "      \"C\" — [0, 0, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 4: Applying transforms to the data\n",
    "\n",
    "Modify the `PenguinDataset` class above so that the tuples of numbers are converted to PyTorch `torch.Tensor` s and the string targets are converted to one-hot vectors.\n",
    "\n",
    "- Begin by importing relevant PyTorch functions.\n",
    "- Apply transformations inside `__getitem__()` function above.\n",
    "\n",
    "Then create a training and validation set.\n",
    "\n",
    "  - We allow the model to learn directly from the training set—i.e. we fit the function to these data.\n",
    "  - During training, we monitor the model's performance on the validation set in order to check how it's doing on unseen data. Normally, people use the validation performance to determine when to stop the training process.\n",
    "  \n",
    "For the validation set, we choose ten males and ten females of each species. This means the validation set is less likely to be biased by sex and species, and is potentially a more reliable measure of performance. You should always be _very_ careful when choosing metrics and splitting data.\n",
    "\n",
    "- Is this solution general?\n",
    "\n",
    "A common way of transforming inputs to neural networks is to apply a series of transforms using `torchvision.transforms.Compose`. The [ `Compose` ](https://pytorch.org/vision/stable/generated/torchvision.transforms.Compose.html) object takes a list of callable objects and applies them to the incoming data. See how this is done more generally in the `src/ml_workshop/_penguins.py` file. \n",
    "\n",
    "These transforms can be very useful for mapping between file paths and tensors of images, etc.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apply transforms we need to PenguinDataset to convert input data and target class to tensors. \n",
    "# See Task 4 exercise comments above.\n",
    "\n",
    "# Create train_set\n",
    "\n",
    "# Create valid_set\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Optional) Task 4b: \n",
    "\n",
    "Apply the `torchvision.transforms.Compose` transformations instead of hardcoding as above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision.transforms import Compose\n",
    "\n",
    "# import some useful functions here, see https://pytorch.org/docs/stable/torch.html\n",
    "# where `tensor` and `eye` are used for constructing tensors,\n",
    "# and using a lower-precision float32 is advised for performance\n",
    "from torch import tensor, eye, float32\n",
    "\n",
    "# Apply the transforms we need to the PenguinDataset to get out input\n",
    "# targets as Tensors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 5: Creating ``DataLoaders``—and why\n",
    "\n",
    "- Once we have created a ``Dataset`` object, we wrap it in a ``DataLoader``.\n",
    "  - The ``DataLoader`` object allows us to put our inputs and targets in mini-batches, which makes for more efficient training.\n",
    "    - Note: rather than supplying one input-target pair to the model at a time, we supply \"mini-batches\" of these data at once (typically a small power of 2, like 16 or 32).\n",
    "    - The number of items we supply at once is called the batch size.\n",
    "      - Q. What number should we choose for the batch size?\n",
    "  - The ``DataLoader`` can also randomly shuffle the data each epoch (when training). This avoids accidental patterns in the data harming the fitting process. Consider providing lots of the positive class followed by the negative class,\n",
    "the network will only learn by saying yes all the time. Therefore need to intersperse positives and negatives.\n",
    "\n",
    "  - The ``DataLoader`` also allows us to load different mini-batches in parallel, which can be very useful for larger datasets and images that can't all fit in memory at once.\n",
    "\n",
    "\n",
    "Note: we are going to use batch normalisation layers in our network, which don't work if the batch size is one. This can happen on the last batch, if we don't choose a batch size that evenly divides the number of items in the data set. To avoid this, we can set the ``drop_last`` argument to ``True``. The last batch, which will be of size ``len(data_set) % batch_size`` gets dropped, and the data are reshuffled. This is only relevant during the training process - validation will use population statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "# Create training and validation DataLoaders."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 6: Creating a neural network in PyTorch\n",
    "\n",
    "Here we will create our neural network in PyTorch, and have a general discussion on clean and messy ways of going about it.\n",
    "\n",
    "&emsp; The module `torch.nn` contains different classes that help you build neural network models. All models in PyTorch inherit from the subclass `nn.Module`, which has useful methods like `parameters()`, `__call__()` and others.\n",
    "\n",
    "&emsp; `torch.nn` also has various layers that you can use to build your neural network. For example, we will use `nn.Linear` in our code below, which constructs a fully connected layer. `torch.nn.Linear` is a subclass of `torch.nn.Module`. \n",
    "\n",
    "&emsp; What exactly is a \"layer\"? It is essentially a step in the neural network computation. i.e. The `nn.Linear` layer computes the linear transformation of the input vector `$x$`:  `$y$ = $W^T x + b$`. Where `W` is the matrix of tunable parameters and `b` is a bias vector.\n",
    "\n",
    "We can also think of the ReLU activation as a \"layer\". However, there are no tunable parameters associated with the ReLU activation function.\n",
    "\n",
    "&emsp; The `__init__()` method is where we typically define the attributes of a class. In our case, all the \"sub-components\" of our model should be defined here.\n",
    "\n",
    "&emsp; The `forward` method is called when we use the neural network to make a prediction. Another term for \"making a prediction\" is running the forward pass, because information flows forward from the input through the hidden layers to the output. This builds a computational graph. To compute parameter updates, we run the backward pass by calling the function `loss.backward()`. During the backward pass, `autograd` traverses this graph to compute the gradients, which are then used to update the model's parameters.\n",
    "\n",
    "&emsp; The `forward` method is called from the `__call__()` function of `nn.Module`, so that when we run `model(batch)`, the `forward` method is called. \n",
    "- First, we will create quite an ugly network to highlight how to make a neural network in PyTorch on a very basic level.\n",
    "- We will then utilise `torch.nn.Sequential` as a neater approach.\n",
    "- Finally, we will discuss how the best approach would be to write a class where various parameters (e.g. number of layers, dropout probabilities, etc.) are passed as arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.nn import Module\n",
    "from torch.nn import BatchNorm1d, Linear, ReLU, Dropout\n",
    "from torch import Tensor\n",
    "\n",
    "\n",
    "class FCNet(Module):\n",
    "    \"\"\"Fully-connected neural network.\"\"\"\n",
    "\n",
    "    # define __init__ function - model defined here.\n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "    # define forward function which calls network\n",
    "    def forward(self, batch: Tensor) -> Tensor:\n",
    "        pass\n",
    "\n",
    "\n",
    "# define a model and print and test (try with torch.rand() function)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 7: Selecting a loss function\n",
    "\n",
    "- Binary cross-entropy is about the most common loss function for classification.\n",
    "  - Details on this loss function are available in the [PyTorch docs](https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html).\n",
    "- Let's instantiate it together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.nn import BCELoss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 8: Selecting an optimiser\n",
    "\n",
    "While we talked about stochastic gradient descent in the slides, most people use the so-called [Adam optimiser](https://pytorch.org/docs/stable/generated/torch.optim.Adam.html).\n",
    "\n",
    "You can think of it as a more complex and improved implementation of SGD.\n",
    "\n",
    "Here we will tell the optimiser what parameters to fit in order to minimise the loss. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an optimiser and give it the model's parameters.\n",
    "from torch.optim import Adam"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have a go at importing the model weights for a large model like ResNet50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 9: Writing basic training and validation loops\n",
    "\n",
    "- Before we jump in and write these loops, we must first choose an activation function to apply to the model's outputs so that they compared to our targets i.e. `[0, 0, 1]`. We chose not to include this in the network itself.\n",
    "  - Here we are going to use the softmax activation function: see [the PyTorch docs](https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html). It can be seen as a generalization of both the logits and sigmoid functions to handle multi-class classification tasks\n",
    "  - For those of you who've studied physics, you may be reminded of the partition function in thermodynamics.\n",
    "  - This activation function is good for classification when the result is one of ``A or B or C``.\n",
    "    - It's bad if you even want to assign two classification to a single image—say a photo of a dog _and_ a cat.\n",
    "  - It turns the raw outputs, or logits, into \"psuedo probabilities\", and we take our prediction to be the most probable class.\n",
    "\n",
    "- Have a go at writing these loops. Read the comments below for help.\n",
    "\n",
    "TIPS:\n",
    "\n",
    "- The model needs to be configured for training and validation.\n",
    "- We need to tell the softmax function over what dimension we should sum the probabilities over in order to equal 1. This should be along the column axis. \n",
    "- The automatic behaviour of the optimiser is to accumulate gradients during training.\n",
    "\n",
    "- Extracting metrics: \n",
    "  - Define a dictionary `metrics = {\"loss\": [], \"accuracy\" : []}`\n",
    "  - Append the loss `loss.item()` which is a 1x1 tensor. We do not need gradients.\n",
    "  - Get the accuracy by writing a function `get_batch_accuracy(preds: Tensor, targets: Tensor)`.\n",
    "    - A decision can be computed as follows: `decision = preds.argmax(dim=1)`\n",
    "  - We need to supply the metrics as `means` over each epoch.\n",
    "  - The metrics should be a dictionary containing \"loss\" and \"accuracy\" as keys and lists as values which we append to each iteration. We can then use dictionary comprehension to get epoch statistics. \n",
    "  ```\n",
    "    metrics = {\"loss \" : [1.0, 2.0, 3.0], \"accuracy\" : [0.7, 0.8, 0.9]}\n",
    "    return {k : mean(v) for k, v in metrics.items() }\n",
    "  ```\n",
    "  - If the validation performance gets really poor this is a sign that we have possibly overfit. \n",
    "\n",
    "- Utilise `@no_grad` where possible. It temporarily disables gradient calculation, which is beneficial during evaluation phases when gradient updates are not required. \n",
    "\n",
    "\n",
    "NOTE: In PyTorch, `requires_grad=True` is set automatically for the parameters of layers defined using `torch.nn.Module` subclasses. Examine the following example:\n",
    "```\n",
    "x = ones(10, requires_grad=True)\n",
    "y = 2*x.exp()\n",
    "print(y)\n",
    "```\n",
    "- Why use BCELoss?\n",
    "  - It may seem odd to be using BCELoss for a multi-class classification problem. In this case, BCELoss treats each element of the prediction vector as an independent binary classification problem. For each class, it compares the predicted probability against the target and computes the loss. It might be better to use `CrossEntropyLoss` instead (ground truth does not need to be one-hot encoded). `CrossEntropyLoss` combines softmax and negative log likelihood. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict\n",
    "\n",
    "\n",
    "def train_one_epoch(\n",
    "    model: Module,\n",
    "    train_loader: DataLoader,\n",
    "    optimiser: Adam,\n",
    "    loss_func: BCELoss,\n",
    ") -> Dict[str, float]:\n",
    "    \"\"\"Train ``model`` for once epoch.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    model : Module\n",
    "        The neural network.\n",
    "    train_loader : DataLoader\n",
    "        Training dataloader.\n",
    "    optimiser : Adam\n",
    "        The optimiser.\n",
    "    loss_func : BCELoss\n",
    "        Binary cross-entropy loss function.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Dict[str, float]\n",
    "        A dictionary of metrics.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    # setup the model for training. IMPORTANT!\n",
    "\n",
    "    # setup loss and accuracy metrics dictionary\n",
    "\n",
    "    # iterate over the batch, targets in the train_loader\n",
    "    for batch, targets in train_loader:\n",
    "        pass\n",
    "\n",
    "        # zero the gradients (otherwise gradients accumulate)\n",
    "\n",
    "        # run forward model and compute proxy probabilities over dimension 1 (columns of tensor).\n",
    "\n",
    "        # compute loss\n",
    "        # e.g. pred = [0.2, 0.7, 0.1] and target = [0, 1, 0]\n",
    "\n",
    "        # compute gradients\n",
    "\n",
    "        # nudge parameters in direction of steepest descent c\n",
    "\n",
    "        # append metrics\n",
    "\n",
    "\n",
    "def validate_one_epoch(\n",
    "    model: Module,\n",
    "    valid_loader: DataLoader,\n",
    "    loss_func: BCELoss,\n",
    ") -> Dict[str, float]:\n",
    "    \"\"\"Validate ``model`` for a single epoch.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    model : Module\n",
    "        The neural network.\n",
    "    valid_loader : DataLoader\n",
    "        Validation dataloader.\n",
    "    loss_func : BCELoss\n",
    "        Binary cross-entropy loss function.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Dict[str, float]\n",
    "        Metrics of interest.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    for batch, targets in valid_loader:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 10: Training, extracting and plotting metrics\n",
    "\n",
    "- Now we can train our model for a specified number of epochs.\n",
    "  - During each epoch the model \"sees\" each training item once.\n",
    "- Append the training and validation metrics to a list.\n",
    "- Turn them into a ``pandas.DataFrame``\n",
    "  - Note: You can turn a ``List[Dict[str, float]]``, say ``my_list`` into a ``DataFrame`` with ``DataFrame(my_list)``.\n",
    "- Use Matplotlib to plot the training and validation metrics as a function of the number of epochs.\n",
    "\n",
    "We will begin the code block together before you complete it independently.  \n",
    "After some time we will go through the solution together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 3\n",
    "\n",
    "# define train_metrics and valid_metrics lists. \n",
    "\n",
    "for _ in range(epochs):\n",
    "\n",
    "    # append output of train_one_epoch() to train_metrics\n",
    "\n",
    "    # append output of valid_one_epoch() to valid_metrics\n",
    "\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 11: Visualise some results\n",
    "\n",
    "Let's do this part together—though feel free to make a start on your own if you have completed the previous exercises."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus: Run the net on 'new' inputs\n",
    "\n",
    "We have built and trained a net, and evaluated and visualised its performance. However, how do we now utilise it going forward?\n",
    "\n",
    "Here we construct some 'new' input data and use our trained net to infer the species. Whilst this is relatively straightforward there is still some work required to transform the outputs from the net to a meaningful result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import no_grad\n",
    "\n",
    "# Construct a tensor of inputs to run the model over\n",
    "\n",
    "# Place model in eval mode and run over inputs with no_grad\n",
    "\n",
    "# Print the raw output from the net\n",
    "\n",
    "# Transform the raw output back to human-readable format"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}