worked-solutions/01_penguin_classification_solutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 1: Classifying penguin species with PyTorch\n",
    "\n",
    "<img src=\"https://allisonhorst.github.io/palmerpenguins/reference/figures/lter_penguins.png\" width=\"750\" />\n",
    "\n",
    "\n",
    "Artwork by @allison_horst\n",
    "\n",
    "In this exercise, we will use the python package [``palmerpenguins``](https://github.com/mcnakhaee/palmerpenguins) to supply a toy dataset containing various features and measurements of penguins.\n",
    "\n",
    "We have already created a PyTorch dataset which yields data for each of the penguins, but first we should examine the dataset and see what it contains."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1: look at the data\n",
    "In the following code block, we import the ``load_penguins`` function from the ``palmerpenguins`` package.\n",
    "\n",
    "- Call this function, which returns a single object, and assign it to the variable ``data``.\n",
    "  - Print ``data`` and recognise that ``load_penguins`` has returned a ``pandas.DataFrame``.\n",
    "- Consider which features it might make sense to use in order to classify the species of the penguins.\n",
    "  - You can print the column titles using ``pd.DataFrame.keys()``\n",
    "  - You can also obtain useful information using ``pd.DataFrame.Series.describe()``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       bill_length_mm  bill_depth_mm  flipper_length_mm  body_mass_g  \\\n",
      "count      342.000000     342.000000         342.000000   342.000000   \n",
      "mean        43.921930      17.151170         200.915205  4201.754386   \n",
      "std          5.459584       1.974793          14.061714   801.954536   \n",
      "min         32.100000      13.100000         172.000000  2700.000000   \n",
      "25%         39.225000      15.600000         190.000000  3550.000000   \n",
      "50%         44.450000      17.300000         197.000000  4050.000000   \n",
      "75%         48.500000      18.700000         213.000000  4750.000000   \n",
      "max         59.600000      21.500000         231.000000  6300.000000   \n",
      "\n",
      "              year  \n",
      "count   344.000000  \n",
      "mean   2008.029070  \n",
      "std       0.818356  \n",
      "min    2007.000000  \n",
      "25%    2007.000000  \n",
      "50%    2008.000000  \n",
      "75%    2009.000000  \n",
      "max    2009.000000  \n",
      "Index(['species', 'island', 'bill_length_mm', 'bill_depth_mm',\n",
      "       'flipper_length_mm', 'body_mass_g', 'sex', 'year'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "from palmerpenguins import load_penguins\n",
    "\n",
    "data = load_penguins()\n",
    "\n",
    "# Note: ``pd.DataFrame.describe`` is a useful function for giving an overview\n",
    "# of what a ``pd.DataFrame`` contains.\n",
    "print(data.describe())\n",
    "\n",
    "print(data.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now discuss the features we will use to classify the penguins' species, and populate the following list together:\n",
    "\n",
    "#### Let's use:\n",
    "\n",
    "- ``\"bill_length_mm\"``\n",
    "    - Biologically relevant and unambiguous.\n",
    "- ``\"bill_depth_mm\"``\n",
    "    - Biologically relevant.\n",
    "- ``flipper_length_mm``\n",
    "    - Biologically relevant.\n",
    "- ``\"body_mass_g\"``\n",
    "    - Biologically relevant.\n",
    "- ``\"sex\"``\n",
    "    - While a potential source of bias, it is likely informative and biologically relevant.\n",
    "\n",
    "#### Let's reject\n",
    "- ``\"island\"``\n",
    "    - While island is likely to be predictive, it seems potentially misleading to use this feature. One island could be heavily dominated by one species of penguin, while other species abide there in much smaller numbers. Such a situation could result in a model giving too much weight to this feature, and confounding the results.\n",
    "- ``\"year\"``\n",
    "    - This feature could also be important: then behaviour of certain species may be changing in response to time-dependent environmental factors such as melting ice. It does however seem like the least biologically-relevant feature, and the most likely source of bias, so we reject it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2: creating a ``torch.utils.data.Dataset``\n",
    "\n",
    "The penguin data reading and processing can be encapsulated in a PyTorch dataset class.\n",
    "\n",
    "- Why is a class representation helpful?\n",
    "  - Modularity - Separation of concerns makes the cde easier to understand, maintain and test.\n",
    "  - Maintainability - Changes are localised, therefore we only need to change a single file to update. \n",
    "  - Abstraction - Users do not need to know how the data is read or processed, they only need to know how to interact with the class. \n",
    "\n",
    "All PyTorch dataset objects are subclasses of the ``torch.utils.data.Dataset`` class. To make a custom dataset, create a class which inherits from the ``Dataset`` class, implement some methods (the Python magic (or dunder) methods ``__len__`` and ``__getitem__``) and supply some data.\n",
    "\n",
    "Spoiler alert: we've done this for you already below (see ``src/ml_workshop/_penguins.py`` for a more sophisticated implementation)\n",
    "\n",
    "- Open the file ``src/ml_workshop/_penguins.py``.\n",
    "- Let's examine, and discuss, each of the methods together.\n",
    "  - ``__len__``\n",
    "    - What does the ``__len__`` method do?\n",
    "      - The ``__len__`` method is a so-called \"magic method\", which tells python to do if the ``len`` function is called on the object containing it.\n",
    "  - ``__getitem__``\n",
    "    - What does the ``__getitem__`` method do?\n",
    "      - The ``__getitem__`` method is another magic method which tells python what to do if we try and index the object containing it (i.e. ``my_object[idx]``).\n",
    "- Review and discuss the class arguments.\n",
    "  - ``input_keys``— A sequence of strings telling the data set which objects to return as inputs to the model.\n",
    "  - ``target_keys``— Same as ``input_keys`` but specifying the targets.\n",
    "  - ``train``— A boolean variable determining if the model returns the training or validation split (``True`` for training)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List, Tuple, Any\n",
    "\n",
    "# import some useful functions here, see https://pytorch.org/docs/stable/torch.html\n",
    "# where `tensor` and `eye` are used for constructing tensors,\n",
    "# and using a lower-precision float32 is advised for performance\n",
    "from torch import tensor, float32, eye\n",
    "\n",
    "from torch.utils.data import Dataset\n",
    "\n",
    "from palmerpenguins import load_penguins\n",
    "\n",
    "\n",
    "class PenguinDataset(Dataset):\n",
    "    \"\"\"Penguin dataset class.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    input_keys : List[str]\n",
    "        The column titles to use in the input feature vectors.\n",
    "    target_keys : List[str]\n",
    "        The column titles to use in the target feature vectors.\n",
    "    train : bool\n",
    "        If ``True``, this object will serve as the training set, and if\n",
    "        ``False``, the validation set.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    The validation split contains 10 male and 10 female penguins of each\n",
    "    species.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(\n",
    "        self,\n",
    "        input_keys: List[str],\n",
    "        target_keys: List[str],\n",
    "        train: bool,\n",
    "    ):\n",
    "        \"\"\"Build ``PenguinDataset``.\"\"\"\n",
    "        self.input_keys = input_keys\n",
    "        self.target_keys = target_keys\n",
    "\n",
    "        data = load_penguins()\n",
    "        data = (\n",
    "            data.loc[~data.isna().any(axis=1)]\n",
    "            .sort_values(by=sorted(data.keys()))\n",
    "            .reset_index(drop=True)\n",
    "        )\n",
    "        # Transform the sex field into a float, with male represented by 1.0, female by 0.0\n",
    "        data.sex = (data.sex == \"male\").astype(float)\n",
    "        self.full_df = data\n",
    "\n",
    "        valid_df = self.full_df.groupby(by=[\"species\", \"sex\"]).sample(\n",
    "            n=10,\n",
    "            random_state=123,\n",
    "        )\n",
    "        # The training items are simply the items *not* in the valid split\n",
    "        train_df = self.full_df.loc[~self.full_df.index.isin(valid_df.index)]\n",
    "\n",
    "        self.split = {\"train\": train_df, \"valid\": valid_df}[\n",
    "            \"train\" if train is True else \"valid\"\n",
    "        ]\n",
    "\n",
    "    def __len__(self) -> int:\n",
    "        \"\"\"Return the length of requested split.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        int\n",
    "            The number of items in the dataset.\n",
    "\n",
    "        \"\"\"\n",
    "        return len(self.split)\n",
    "\n",
    "    def __getitem__(self, idx: int) -> Tuple[Any, Any]:\n",
    "        \"\"\"Return an input-target pair.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        idx : int\n",
    "            Index of the input-target pair to return.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        in_feats : Any\n",
    "            Inputs.\n",
    "        target : Any\n",
    "            Targets.\n",
    "\n",
    "        \"\"\"\n",
    "        # get the row index (idx) from the dataframe and\n",
    "        # select relevant column features (provided as input_keys)\n",
    "        feats = tuple(self.split.iloc[idx][self.input_keys])\n",
    "\n",
    "        # this gives a 'species' i.e. one of ('Gentoo',), ('Chinstrap',), or ('Adelie',)\n",
    "        tgts = tuple(self.split.iloc[idx][self.target_keys])\n",
    "\n",
    "        # Task 4 - Exercise #1: convert the features to PyTorch Tensors\n",
    "        feats = tensor(feats, dtype=float32)\n",
    "\n",
    "        # Task 4 - Exercise #2: convert target to a 'one-hot' vector.\n",
    "        target_names = sorted(self.full_df.species.unique())\n",
    "        tgts = eye(len(target_names))[target_names.index(tgts[0])]\n",
    "\n",
    "        return feats, tgts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3: obtaining training and validation datasets\n",
    "\n",
    "- Instantiate the penguin dataloader.\n",
    "  - Make sure you supply the correct column titles for the features and the targets.\n",
    "- Iterate over the dataset\n",
    "    - Hint:\n",
    "        ```python\n",
    "        for features, targets in dataset:\n",
    "            # print the features and targets here\n",
    "        ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([  42.9000,   13.1000, 5000.0000,  215.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  46.1000,   13.2000, 4500.0000,  211.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  44.9000,   13.3000, 5100.0000,  213.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.3000,   13.4000, 4400.0000,  209.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  42.0000,   13.5000, 4150.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  46.5000,   13.5000, 4550.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  44.0000,   13.6000, 4350.0000,  208.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  40.9000,   13.7000, 4650.0000,  214.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  42.6000,   13.7000, 4950.0000,  213.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  42.7000,   13.7000, 3950.0000,  208.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  45.3000,   13.7000, 4300.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  47.2000,   13.7000, 4925.0000,  214.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  45.2000,   13.8000, 4750.0000,  215.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.6000,   13.9000, 4900.0000,  217.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.8000,   13.9000, 4300.0000,  208.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  45.5000,   13.9000, 4200.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  45.7000,   13.9000, 4400.0000,  214.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.3000,   14.0000, 4575.0000,  208.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  47.5000,   14.0000, 4875.0000,  212.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  46.2000,   14.1000, 4375.0000,  217.0000,    0.0000]) tensor([0., 0., 1.])\n"
     ]
    }
   ],
   "source": [
    "features = [\n",
    "    \"bill_length_mm\",\n",
    "    \"bill_depth_mm\",\n",
    "    \"body_mass_g\",\n",
    "    \"flipper_length_mm\",\n",
    "    \"sex\",\n",
    "]\n",
    "\n",
    "target_names = sorted(data.species.unique())\n",
    "\n",
    "data_set = PenguinDataset(\n",
    "    input_keys=features,\n",
    "    target_keys=[\"species\"],\n",
    "    train=True,\n",
    ")\n",
    "\n",
    "for _, (input_feats, target) in zip(range(20), data_set):\n",
    "    print(input_feats, target)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Can we give these items to a neural network, or do they need to be transformed first?\n",
    "  + Short answer: no, we can't just pass tuples of numbers or strings to a neural network.\n",
    "    - We must represent these data as ``torch.Tensor``s. This is the fundamental data abstraction used by PyTorch; they are the PyTorch equivalent to Numpy arrays. See [pytorch tensors documentation](https://pytorch.org/tutorials/beginner/introyt/tensors_deeper_tutorial.html) . \n",
    "    - The targets are tuples of strings i.e. ('Gentoo', )\n",
    "      - One idea is to represent as ordinal values i.e.  [1] or [2] or [3]. But this implies that the class encoded by value 1 is closer to 2 than 1 is to 3. This is not desirable for categorical data. One-hot encoding avoids this by representing each species independently.\\\n",
    "    \"A\" — [1, 0, 0]\\\n",
    "    \"B\" — [0, 1, 0]\\\n",
    "    \"C\" — [0, 0, 1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 4: Applying transforms to the data\n",
    "\n",
    "Modify the `PenguinDataset` class above so that the tuples of numbers are converted to PyTorch `torch.Tensor` s and the string targets are converted to one-hot vectors.\n",
    "\n",
    "- Begin by importing relevant PyTorch functions.\n",
    "- Apply transformations inside `__getitem__()` function above.\n",
    "\n",
    "Then create a training and validation set.\n",
    "\n",
    "  - We allow the model to learn directly from the training set—i.e. we fit the function to these data.\n",
    "  - During training, we monitor the model's performance on the validation set in order to check how it's doing on unseen data. Normally, people use the validation performance to determine when to stop the training process.\n",
    "  \n",
    "For the validation set, we choose ten males and ten females of each species. This means the validation set is less likely to be biased by sex and species, and is potentially a more reliable measure of performance. You should always be _very_ careful when choosing metrics and splitting data.\n",
    "\n",
    "- Is this solution general?\n",
    "  - No. The transformations have been hardcoded. A more flexible way of transforming inputs to neural networks is to apply a series of transforms using `torchvision.transforms.Compose`. The [ `Compose` ](https://pytorch.org/vision/stable/generated/torchvision.transforms.Compose.html) object takes a list of callable objects and applies them to the incoming data. See how this is done more generally in the `src/ml_workshop/_penguins.py` file. \n",
    "\n",
    "These transforms can be very useful for mapping between file paths and tensors of images, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([  42.9000,   13.1000, 5000.0000,  215.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  46.1000,   13.2000, 4500.0000,  211.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  44.9000,   13.3000, 5100.0000,  213.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.3000,   13.4000, 4400.0000,  209.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  42.0000,   13.5000, 4150.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n"
     ]
    }
   ],
   "source": [
    "# Apply transforms we need to PenguinDataset to convert input data and target class to tensors. \n",
    "# See Task 4 exercise comments above.\n",
    "\n",
    "\n",
    "# Create train_set\n",
    "train_set = PenguinDataset(\n",
    "    input_keys=features,\n",
    "    target_keys=[\"species\"],\n",
    "    train=True,\n",
    ")\n",
    "\n",
    "# Create valid_set\n",
    "valid_set = PenguinDataset(\n",
    "    input_keys=features,\n",
    "    target_keys=[\"species\"],\n",
    "    train=False,\n",
    ")\n",
    "\n",
    "\n",
    "for _, (input_feats, target) in zip(range(5), train_set):\n",
    "    print(input_feats, target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Optional) Task 4b: \n",
    "\n",
    "Apply the `torchvision.transforms.Compose` transformations instead of hardcoding as above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([  42.9000,   13.1000, 5000.0000,  215.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  46.1000,   13.2000, 4500.0000,  211.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  44.9000,   13.3000, 5100.0000,  213.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  43.3000,   13.4000, 4400.0000,  209.0000,    0.0000]) tensor([0., 0., 1.])\n",
      "tensor([  42.0000,   13.5000, 4150.0000,  210.0000,    0.0000]) tensor([0., 0., 1.])\n"
     ]
    }
   ],
   "source": [
    "from torchvision.transforms import Compose\n",
    "\n",
    "from ml_workshop import PenguinDataset\n",
    "\n",
    "# import some useful functions here, see https://pytorch.org/docs/stable/torch.html\n",
    "# where `tensor` and `eye` are used for constructing tensors,\n",
    "# and using a lower-precision float32 is advised for performance\n",
    "from torch import tensor, float32, eye\n",
    "\n",
    "\n",
    "# Apply the transforms we need to the PenguinDataset to get out input\n",
    "# targets as Tensors.\n",
    "\n",
    "\n",
    "def get_input_transforms() -> Compose:\n",
    "    \"\"\"Return transforms which map from raw inputs to tensors.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Compose\n",
    "        A composition of transforms (callable functions) to map the tuple\n",
    "        of input features (``Tuple[float, ...]``) to a ``torch.Tensor``.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    To create a ``torch.Tensor`` we can use ``torch.tensor([1.0, 2.0, ...])``\n",
    "\n",
    "    \"\"\"\n",
    "    return Compose([lambda x: tensor(x, dtype=float32)])\n",
    "\n",
    "\n",
    "def get_target_tfms() -> Compose:\n",
    "    \"\"\"Return transforms which map from the raw target strings to tensor.\n",
    "    Returns\n",
    "    -------\n",
    "    Compose\n",
    "        A composition of transforms (callable functions) to map the tuple\n",
    "        of input features (``Tuple[str]``) to a ``torch.Tensor``.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    Suppose we have three labels, \"A\", \"B\" and \"C\". We want to encoder each\n",
    "    distinct label as a one-hot-encoded vector. A natural way to do this is:\n",
    "    - \"A\" — [1, 0, 0]\n",
    "    - \"B\" — [0, 1, 0]\n",
    "    - \"C\" — [0, 0, 1]\n",
    "\n",
    "    The transforms this function produces will return these vectors as tensors.\n",
    "    Note also, in the example we have just given, A's vector was the first row\n",
    "    in the identity matrix, B's the second, etc.\n",
    "\n",
    "    \"\"\"\n",
    "    return Compose([lambda x: eye(len(target_names))[target_names.index(x[0])]])\n",
    "\n",
    "\n",
    "train_set = PenguinDataset(\n",
    "    input_keys=features,\n",
    "    target_keys=[\"species\"],\n",
    "    train=True,\n",
    "    x_tfms=get_input_transforms(),\n",
    "    y_tfms=get_target_tfms(),\n",
    ")\n",
    "\n",
    "\n",
    "valid_set = PenguinDataset(\n",
    "    input_keys=features,\n",
    "    target_keys=[\"species\"],\n",
    "    train=False,\n",
    "    x_tfms=get_input_transforms(),\n",
    "    y_tfms=get_target_tfms(),\n",
    ")\n",
    "\n",
    "\n",
    "for _, (input_feats, target) in zip(range(5), train_set):\n",
    "    print(input_feats, target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 5: Creating ``DataLoaders``—and why\n",
    "\n",
    "- Once we have created a ``Dataset`` object, we wrap it in a ``DataLoader``.\n",
    "  - The ``DataLoader`` object allows us to put our inputs and targets in mini-batches, which makes for more efficient training.\n",
    "    - Note: rather than supplying one input-target pair to the model at a time, we supply \"mini-batches\" of these data at once (typically a small power of 2, like 16 or 32).\n",
    "    - The number of items we supply at once is called the batch size.\n",
    "  - The ``DataLoader`` can also randomly shuffle the data each epoch (when training).\n",
    "  - It allows us to load different mini-batches in parallel, which can be very useful for larger datasets and images that can't all fit in memory at once.\n",
    "\n",
    "\n",
    "Note: we are going to use batch normalisation layers in our network, which don't work if the batch size is one. This can happen on the last batch, if we don't choose a batch size that evenly divides the number of items in the data set. To avoid this, we can set the ``drop_last`` argument to ``True``. The last batch, which will be of size ``len(data_set) % batch_size`` gets dropped, and the data are reshuffled. This is only relevant during the training process - validation will use population statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([16, 5]) torch.Size([16, 3])\n",
      "torch.Size([16, 5]) torch.Size([16, 3])\n",
      "torch.Size([16, 5]) torch.Size([16, 3])\n",
      "torch.Size([12, 5]) torch.Size([12, 3])\n"
     ]
    }
   ],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "batch_size = 16\n",
    "\n",
    "train_loader = DataLoader(\n",
    "    train_set,\n",
    "    batch_size=batch_size,\n",
    "    shuffle=True,\n",
    "    drop_last=True,\n",
    ")\n",
    "\n",
    "valid_loader = DataLoader(\n",
    "    valid_set,\n",
    "    batch_size=batch_size,\n",
    "    shuffle=False,\n",
    ")\n",
    "\n",
    "for batch, targets in valid_loader:\n",
    "    print(batch.shape, targets.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 6: Creating a neural network in PyTorch\n",
    "\n",
    "Here we will create our neural network in PyTorch, and have a general discussion on clean and messy ways of going about it.\n",
    "\n",
    "- First, we will create quite an ugly network to highlight how to make a neural network in PyTorch on a very basic level.\n",
    "- We will then discuss a trick for making the print-out nicer.\n",
    "- Finally, we will discuss how the best approach would be to write a class where various parameters (e.g. number of layers, dropout probabilities, etc.) are passed as arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FCNet(\n",
      "  (_fwd_seq): Sequential(\n",
      "    (0): BatchNorm1d(5, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (1): Linear(in_features=5, out_features=16, bias=True)\n",
      "    (2): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (3): Dropout(p=0.1, inplace=False)\n",
      "    (4): LeakyReLU(negative_slope=0.1)\n",
      "    (5): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (6): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (7): Dropout(p=0.1, inplace=False)\n",
      "    (8): LeakyReLU(negative_slope=0.1)\n",
      "    (9): Linear(in_features=16, out_features=3, bias=True)\n",
      "  )\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from torch.nn import Module\n",
    "from torch.nn import BatchNorm1d, Linear, LeakyReLU, Dropout, Sequential\n",
    "from torch import Tensor\n",
    "\n",
    "\n",
    "class FCNet(Module):\n",
    "    \"\"\"Fully-connected neural network.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    in_feats : int\n",
    "        The number of input features to the model.\n",
    "    out_feats : int\n",
    "        The number of output features (or classes) the model should produce.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    The ``torch.nn.Sequential`` class allows us to \"chain\" multiple layers, rather\n",
    "    than manually passing the output of one to the next in the forward\n",
    "    function.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, in_feats: int, out_feats: int):\n",
    "        \"\"\"Build the neural network.\"\"\"\n",
    "        super().__init__()\n",
    "        self._fwd_seq = Sequential(\n",
    "            BatchNorm1d(in_feats),\n",
    "            Linear(in_feats, 16),\n",
    "            BatchNorm1d(16),\n",
    "            Dropout(0.1),\n",
    "            LeakyReLU(0.1),\n",
    "            Linear(16, 16),\n",
    "            BatchNorm1d(16),\n",
    "            Dropout(0.1),\n",
    "            LeakyReLU(0.1),\n",
    "            Linear(16, out_feats),\n",
    "        )\n",
    "\n",
    "    def forward(self, batch: Tensor) -> Tensor:\n",
    "        \"\"\"Pass ``batch`` through the model.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        batch : Tensor\n",
    "            A mini-batch of inputs.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        Tensor\n",
    "            The result of passing ``batch`` through the model.\n",
    "\n",
    "        \"\"\"\n",
    "        return self._fwd_seq(batch)\n",
    "\n",
    "\n",
    "model = FCNet(len(features), len(target_names))\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 7: Selecting a loss function\n",
    "\n",
    "- Binary cross-entropy is about the most common loss function for classification.\n",
    "  - Details on this loss function are available in the [PyTorch docs](https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html).\n",
    "- Let's instantiate it together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.nn import BCELoss\n",
    "\n",
    "loss_func = BCELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 8: Selecting an optimiser\n",
    "\n",
    "While we talked about stochastic gradient descent in the slides, most people use the so-called [Adam optimiser](https://pytorch.org/docs/stable/generated/torch.optim.Adam.html).\n",
    "\n",
    "You can think of it as a more complex and improved implementation of SGD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an optimiser and give it the model's parameters.\n",
    "from torch.optim import Adam\n",
    "\n",
    "optimiser = Adam(model.parameters())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 9: Writing basic training and validation loops\n",
    "\n",
    "- Before we jump in and write these loops, we must first choose an activation function to apply to the model's outputs.\n",
    "  - Here we are going to use the softmax activation function: see [the PyTorch docs](https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html).\n",
    "  - For those of you who've studied physics, you may be remininded of the partition function in thermodynamics.\n",
    "  - This activation function is good for classifcation when the result is one of ``A or B or C``.\n",
    "    - It's bad if you even want to assign two classification to one images—say a photo of a dog _and_ a cat.\n",
    "  - It turns the raw outputs, or logits, into \"psuedo probabilities\", and we take our prediction to be the most probable class.\n",
    "\n",
    "- We will write the training loop together, then you can go ahead and write the (simpler) validation loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict\n",
    "\n",
    "from numpy import mean\n",
    "\n",
    "from torch import no_grad\n",
    "\n",
    "\n",
    "def train_one_epoch(\n",
    "    model: Module,\n",
    "    train_loader: DataLoader,\n",
    "    optimiser: Adam,\n",
    "    loss_func: BCELoss,\n",
    ") -> Dict[str, float]:\n",
    "    \"\"\"Train ``model`` for once epoch.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    model : Module\n",
    "        The neural network.\n",
    "    train_loader : DataLoader\n",
    "        Training dataloader.\n",
    "    optimiser : Adam\n",
    "        The optimiser.\n",
    "    loss_func : BCELoss\n",
    "        Binary cross-entropy loss function.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Dict[str, float]\n",
    "        A dictionary of metrics.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    - The ``model.train()`` is very important:\n",
    "        - it turns on the dropout layers.\n",
    "        - it tells the batch norm layers to use the incoming\n",
    "          statistics, and let them contribute to their \"memory\".\n",
    "\n",
    "    \"\"\"\n",
    "    model.train()\n",
    "    metrics: Dict[str, float] = {\"loss\": [], \"accuracy\": []}\n",
    "\n",
    "    for batch, targets in train_loader:\n",
    "        optimiser.zero_grad()\n",
    "\n",
    "        preds = model(batch).softmax(dim=1)\n",
    "\n",
    "        loss = loss_func(preds, targets)\n",
    "\n",
    "        loss.backward()\n",
    "\n",
    "        optimiser.step()\n",
    "\n",
    "        metrics[\"loss\"].append(loss.item())\n",
    "        metrics[\"accuracy\"].append(batch_level_accuracy(preds, targets))\n",
    "\n",
    "    return {key: mean(val) for key, val in metrics.items()}\n",
    "\n",
    "\n",
    "@no_grad()\n",
    "def validate_one_epoch(\n",
    "    model: Module,\n",
    "    valid_loader: DataLoader,\n",
    "    loss_func: BCELoss,\n",
    ") -> Dict[str, float]:\n",
    "    \"\"\"Validate ``model`` for a single epoch.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    model : Module\n",
    "        The neural network.\n",
    "    valid_loader : DataLoader\n",
    "        Training dataloader.\n",
    "    loss_func : BCELoss\n",
    "        Binary cross-entropy loss function.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    Dict[str, float]\n",
    "        Metrics of interest.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    - The ``model.eval()`` is also very important:\n",
    "        - It turns off the dropout layers, which are likely to impair the\n",
    "          validation performance and render it unrealistically poor.\n",
    "        - It tells the batchnorm layers to _not_ use the batch's statistics,\n",
    "          and to instead use the stats it has built up from the training set.\n",
    "          The model should not \"remember\" anything from the validation set.\n",
    "    - We also protect this function with ``torch.no_grad()``, because having\n",
    "      gradients enabled while validating is a pointless waste of\n",
    "      resources — they are only needed for training.\n",
    "\n",
    "    \"\"\"\n",
    "    model.eval()\n",
    "    metrics: Dict[str, float] = {\"loss\": [], \"accuracy\": []}\n",
    "\n",
    "    for batch, targets in valid_loader:\n",
    "        preds = model(batch).softmax(dim=1)\n",
    "\n",
    "        loss = loss_func(preds, targets)\n",
    "\n",
    "        metrics[\"loss\"].append(loss.item())\n",
    "        metrics[\"accuracy\"].append(batch_level_accuracy(preds, targets))\n",
    "\n",
    "    return {key: mean(val) for key, val in metrics.items()}\n",
    "\n",
    "\n",
    "@no_grad()\n",
    "def batch_level_accuracy(preds: Tensor, targets: Tensor):\n",
    "    \"\"\"Compute the batch-level accuracy.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    preds : Tensor\n",
    "        The model's predictions.\n",
    "    targets : Tensor\n",
    "        The corresponding labels.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    float\n",
    "        The batch-level accuracy.\n",
    "\n",
    "    Notes\n",
    "    -----\n",
    "    - This function assumes the ``preds`` have had the softmax\n",
    "      applied to them along dimension 1, and that the predicted\n",
    "      class is therefore ``preds.argmax(dim=1)``.\n",
    "\n",
    "    \"\"\"\n",
    "    return (preds.argmax(dim=1) == targets.argmax(dim=1)).float().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 10: Training, extracting and plotting metrics\n",
    "\n",
    "- Now we can train our model for a specified number of epochs.\n",
    "  - During each epoch the model \"sees\" each training item once.\n",
    "- Append the training and validation metrics to a list.\n",
    "- Turn them into a ``pandas.DataFrame``\n",
    "  - Note: You can turn a ``List[Dict{str, float}]``, say ``my_list`` into a ``DataFrame`` with ``DataFrame(my_list)``.\n",
    "- Use Matplotlib to plot the training and validation metrics as a function of the number of epochs.\n",
    "\n",
    "We will begin the code block together before you complete it independently.  \n",
    "After some time we will go through the solution together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0-25 time: 8.108920 seconds\n",
      "Epoch 25-50 time: 8.245825 seconds\n",
      "Epoch 50-75 time: 7.894095 seconds\n",
      "Epoch 75-100 time: 8.292500 seconds\n",
      "Epoch 100-125 time: 7.116918 seconds\n",
      "Epoch 125-150 time: 6.541059 seconds\n",
      "Epoch 150-175 time: 7.708282 seconds\n",
      "\n",
      "\n",
      "     loss_train  accuracy_train  loss_valid  accuracy_valid\n",
      "0      0.614220        0.452206    0.668509        0.375000\n",
      "1      0.524949        0.698529    0.527703        0.703125\n",
      "2      0.460917        0.786765    0.463121        0.781250\n",
      "3      0.380868        0.886029    0.396204        0.828125\n",
      "4      0.347903        0.878676    0.337664        0.859375\n",
      "..          ...             ...         ...             ...\n",
      "195    0.050222        0.966912    0.013005        0.984375\n",
      "196    0.036788        0.985294    0.012601        1.000000\n",
      "197    0.033748        0.970588    0.011316        1.000000\n",
      "198    0.038716        0.988971    0.020271        0.984375\n",
      "199    0.015950        0.988971    0.019603        0.984375\n",
      "\n",
      "[200 rows x 4 columns]\n"
     ]
    }
   ],
   "source": [
    "from time import perf_counter\n",
    "\n",
    "from pandas import DataFrame\n",
    "\n",
    "epochs = 200\n",
    "print_interval = 25\n",
    "\n",
    "\n",
    "train_metrics, valid_metrics = [], []\n",
    "\n",
    "\n",
    "for epoch in range(epochs):\n",
    "\n",
    "    if epoch % print_interval == 0:\n",
    "        if epoch != 0:\n",
    "            print(\n",
    "                f\"Epoch {max(epoch - print_interval, 0)}-{epoch} time: {perf_counter() - tic:.6f} seconds\"\n",
    "            )\n",
    "        tic = perf_counter()\n",
    "\n",
    "    train_metrics.append(train_one_epoch(model, train_loader, optimiser, loss_func))\n",
    "\n",
    "    valid_metrics.append(validate_one_epoch(model, valid_loader, loss_func))\n",
    "\n",
    "    stop_time = perf_counter()\n",
    "\n",
    "\n",
    "print(\"\\n\")\n",
    "\n",
    "train_metrics = DataFrame(train_metrics)\n",
    "valid_metrics = DataFrame(valid_metrics)\n",
    "\n",
    "metrics = train_metrics.join(valid_metrics, lsuffix=\"_train\", rsuffix=\"_valid\")\n",
    "\n",
    "print(metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 11: Visualise some results\n",
    "\n",
    "Let's do this part together—though feel free to make a start on your own if you have completed the previous exercises."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAGGCAYAAADmRxfNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAACUG0lEQVR4nOzdd3wUdfrA8c9szW56T0ilBhDpCIhSPBTUU7GDHcupBzburL87y3l3WE7FUxTrYeEQK3oiotIEBOkgAgEhgZCQhNRN2Wyd3x+bbLIkgYQk7Aae9+vFi83slGdmk53vM9+mqKqqIoQQQgghhBBtoPF3AEIIIYQQQojOTxILIYQQQgghRJtJYiGEEEIIIYRoM0kshBBCCCGEEG0miYUQQgghhBCizSSxEEIIIYQQQrSZJBZCCCGEEEKINpPEQgghhBBCCNFmklgIIYQQQggh2kwSCyGEEEIIIUSbBWRi8eOPP3LJJZfQpUsXFEVh4cKFx91mxYoVDB48GKPRSI8ePZg7d26HxymEEOLkk3uEEEIEpoBMLKqqqhgwYACzZ89u0fpZWVlcfPHFjBs3jq1bt3L//fdz++23s2TJkg6OVAghxMkm9wghhAhMiqqqqr+DOBZFUfjiiy+YNGlSs+s8/PDDLFq0iB07dniXTZ48mbKyMr799tuTEKUQQgh/kHuEEEIEjoCssWittWvXMn78eJ9lEyZMYO3atX6KSAghRKCQe4QQQpwcOn8H0B7y8/OJj4/3WRYfH4/FYsFqtWIymRptY7PZsNls3p/dbjclJSVER0ejKEqHxyyEEB1BVVUqKiro0qULGs0p8eyozeQeIYQQHh19jzglEosTMXPmTJ566il/hyGEEB0iJyeH5ORkf4fRack9QghxKuuoe8QpkVgkJCRQUFDgs6ygoICwsLAmn0QBPProo8yYMcP7c3l5OampqeTk5BAWFtah8QohREexWCykpKQQGhrq71ACRsDcI8py4fXhtT8ogArGMJi+EQxm+Oh6yFrpeVsbBPdv9yz//knY+HbtdjrAWb+P+7bBh1dB8Z4GB9IAbtCZ4NZv4c0x9W8Fx8Jda0BrgJcHgq3Ms9wYAec/BV/fB4Cq6FBUJypaFFyeZbVRF6rh3B40i7m3jST8zaFonZUndDlURYdy72Z4dTi4rPXLUVBQKVNN/Eu5nb/zylHnFUT1Dd9injvWZ38OVYNecTd6rQITHc/zD81bDNXu4Wh15+V53fB86183v28NCm6f/dlVLWNsL/Kd8SFCFRtNsasaDEr9dg8F/51vihO9Pz+o+4ibdd8D4EaLpjYOtwqaJirM6j6vhr8fDbdryXUC2H3R5/Tc/Tra/d83irOpOMoJIej8xzF+/xAALlVBq6hUqkbG2F7ChpEuEUF8dMdIckqqSIk0c9GsH1iqnU6w4vAe142CBhWM4TB9AxjM/GfNfl74bm+j+B/W/ZcbdUuPeY6qCpfyAk+43/B+5g406Kn/nVAAguOouGU5WRaVv329k92HK7z7CA/S8cOfxmIyaKmscbLxQAlbDpbx7uosju6c/IDuE27TfetzbUrVEMbbnsOGkbQoEx/cNoKLX/mRihoXUWY9d47pzszFuwFIizLx6d2jCNJruGz2avYfqW50TgBjNZt51dBg0IpbFkPimd4fq2qc7CuqpGdcKKZ1s2DNS7XveL5vHPpw9k76mpFDBnbYPeKUSCxGjhzJN99847Ps+++/Z+TIkc1uYzQaMRqNjZaHhYVJYiGE6PSkuU69gLlHlO0EowLBcTD+SfjyjxBsApMWctfB4TWe9wFwQPFWMIZAzg+e5WMehrP+AEd2w8c3Q3URbHwVKveCUUPN1f/lYJWW1Ohggj64GBQnbJ/bYJ9Ayhlg0sEvC4Dy+vciIuHwajAq2HtezDk7LyVRLaDKnEJk9T5A4TLtGm7QLWOzK52yGjsfvfUPHtFWgbbB/jVGuG4+fPdXKPy10SUojx/BjEPn8obuX+gUF471/0avq0ENT+DWqruptLnoZyziCV6nym3ibPcvhOkU6HkBnPsgvHs+Kk6+/vDvXGdU2OpO5x+OGwCFvSTRk0OAwuGgNH64IR7NV/dgsGRzl/Y7xmj3olUUFvV5jgnqanS7vwJgk7sHzzgmAwpVYT2YPymCr7cf5l9boIdyiLAgPZus8d59NzzO364eQZ+vLj7qLN38WfcNSTo7BWo4awY8x/idfyHMWQTAQudInnTeTE8O8aDuY87S7uF853LWGa+hp3IIjQKX6LYTplF4xX0VV935FAX7t/LKst+Ith3kOcM7ADhVuE95mEKbgVJTN76+MRk1Ip1De7aQEmVi1jYNv2xcxTuG5wlSPIXvOc6LmeO8pNG5PGmcxxlksWfJa5yp+xGTUeFh+20sUYfRk0NEmg3EdetP5o6NgEI/bTZP6D+g0B2K7relxBoVvnSO4BPXGD40PotWhUjcJCp5HLDG8fDsubjdUGhM50bDchL1Tg6rkdxrnw4opCr5vGB4C0LMnr+HI79gLtpPlFFHmlLIATWOnsohAC7Ve67N846rmOcaTy8lh9cM/yZGqfD5FG5zLvZ+5nfYH2CDO6N2Hwq/167lZt0P7DGmce2rq+mhHCIchShjknedvWoS61YvARSeWueme+3ySGPSUTEp/F6/jTCNwr8dl2EhhL/o55HvDicYLQOVbKhW+OTnKPqyD4IU9rqS2LppDVHGcM8+qhUKSvuiFuwitmIXZUfFUff6Em3t30OtI1vmo43sRnH2dgrKq7lnaQ09lENEmQ3Mjl6KzqhQ0O8PBMd3I2TpI2S7jdzxwUag4+4RATkqVGVlJb/99hsAgwYN4sUXX2TcuHFERUWRmprKo48+Sm5uLu+//z7gGUqwX79+TJs2jVtvvZVly5Zx7733smjRIiZMmNCiY1osFsLDwykvL5fEQgjRaZ0O32Wd9h6x+X346h7ofh5c8SY83xOOfvap0UFkOhT/1nj7yK5w90+eWoz/PQCb3vW+pWqDOMv1FkdqtESbdawx/4mgypz6bXUmcFob77OJ5VXGOIaUP0sNvonVaM023jc8C/g+Oa9SDZgVu+eZqN7M9us2k773fcJ/+icA1Rgx43lyX6SGcY7tZd7RP88o7U7vvp36UPpV/JsajERTxgbjH9EonifPigKEJmK9awPKG+cSZMnybleihnC27ZVGsY7sFs07Nw/lixfu4nr7J97lNlXHANtbTA7ZxpPOWQAUq6GMsv3bu49go5YqW+On4AAhRi2Vte+FGHX8+OA4gt4djblkFwBWVY+pwZN4i2riLNtrPKhb4H2ineeO4nzni1S5DUzR/sBM/buNrmmdbHccE+31n0UMpWwwTkNRwKoaGGR7w/veXy7qw7NLduNwqaRHm7HanRRU2HlL/zzna7cAcMAdxwR748/2Nu0i/qqf57NsvzuBi+0zsWKka7SZrOL6p+hJHGG18T6UBp9RnjuKi+1/Z4NxOjrF7T2fhufV8HXdtan7zDcG/bGuHs9bg+RWFTSK6rOs/trEM9H+DDUY+Yfuba7XLQM8v4/Bit27Xt1nbsPo/WsbpfzCPOPMRjG17LUnJjeNR0DKdsfxZ83DfMqfWraPBst8z7vp7equdcNzbG7dOgfccczQ/5VPnfegKFBWA5HPWjrsHhGQPfs2btzIoEGDGDRoEAAzZsxg0KBBPP744wAcPnyYgwcPetfv2rUrixYt4vvvv2fAgAG88MILvP322y2+YQghhOg8Ou09oqi2GU5MLyg9SKOkAsDthOheTW9fmgWFngIsCWf6vKW4auhi8xS4i6udfF2W7rttU0lFM8uDbYVkKJ6kxKirLyZUqPXNxhoWXoJrkwoAxVHN429+zKM/1jcBqksqAGIUCxlKDtvUHj7H1DkqvMdMVoq8+/c+VK04zO3Pv89XJak+20UplWQoOWiPKk1lF1cx+vnl/FwZ57PcqDjJUHLYVRXiXRat1B8baJRUmA3a+u0bXA+36mbcv5bzaWGSd1nDpAIgTLGSoeSQ6U7xLuuiKWH5jXEsnDYKU2w37/Kmmjilawp9YktSir3XxKTYfd579ttdOFxq7flXU1DhKXj+onb3rpOmKWRM+BEAEkINxIYYPOu7Exodu5smn161+2+YVACMiHN646j7v4umhCuCf0VX23Sq7nwanlfD13XXRqN4PnPv7xAN11cbLauTrinwnn+usf4cGyYVUP+Z1/21pUWZqSKoyZha9tqzp6YK0OmaQt4cr2mw7nH20WCZ73k3/bruWjc8x+bWrZOmKaSr9RfvtnXH7igBmViMHTsWVVUb/aubKXXu3LmsWLGi0TZbtmzBZrOxb98+brnllpMetxBCiI7Xae8RRbXtxWN6QlwfiEgDPH0K6uQq8VjHz8Sqi/Auc6m170d192wHMGCyp0lVrWx3HJlqfeF1pnMKRWp94dkdkQZhdQXgBqWPiDQIr91O8RQJcjVdvPuyOT2FxOnjelAR3ptcd3RtTA1OS5cA4Z4C/353AplqCsuc/chzR9XGFu99Xff+a85LKKH+aelBErzHzFRTOOiO9Tn3AySyqSaRZ5yTfc7LGpLK3/5wDRee6Tvq1+HyGooq7XznHkq+Gl5/fYknU01hm9rdey773Qlce9EFxIToffZR18LL7a4/2eKq+sSh2u6mvMbJc85rKaw9Rg7xFKn1bddzas/rK/fZHMFzDdyR3YjrPoiBKREE9zyXw6pnecN0xl37Ge13J7CX+s/VGpnh+SwbXMs6Dt+uHl5vOi+iQI3wxKMkMvOua1k4bRTLHzyPJfePITE8iDVqP298rgbHLgvp0Wh/XaPNpPYZRs5Rn9FBEpl25z2emjXw/j41PK+GvzfZbs9n4VaP/swbru/Zd8O/kYbxWSN6seAPI7hnxhMQ6kmO8pQEChp85o6wNJ/rNPv6wZSH9ar/XW4YXwteq0pd0VlptMwd2Y2oQZfirv178Im7ifNquKzhx3e8Y2e548lXI5vYb+Nj7Hcn8IN7sPf6qh1c9A/IplD+cDo0HxBCnPrku6xjtMt1/fcgKNkPN38NXc8FezUU7mK7NZKn3/0MUPhF7cbbt43m4flr6WLdDSiUmLqx6KZkgrr08zSDqmOvhrwt/LjnCH9Y5qYGIwmhBtwoFFbYCMLGmYqnf8Rf/nA9Oo3C3l9+5oJRIzCX1za16jKIrKIqNq5fw4WjR6KWZnHW23lYVd+mMq9dP5hxGXFkHipg1ryFbK2K9Lb7NqcN5b1bz+KZDz5n7l6zt5lNEDbP0/raQl2GkoM1ohePXTaEW9/bgEGtj+8XtRs1GBmSGsGmg2XebQ+ocaQphWSqKfVNlRQbfdlHYpiJZ++9GVNwKLOX/cbz32U2edl7RGj4+qpggnRarLH92ZRn5YEF26io9NRUWCMz+PL+83nhu0zeXl3fzGpwagSbD5a16KOti/es4aP4YN1B73k9NHUybp2npqd/vAFT6R5Pclj7OX6/s4B73l9DhpLDqLOGsmH9T4DCpeefx9XdnGSqKaTERrO30NN/oH9yBCZs1OTt4KL5RewvV4kLNVJY0XQncYAIk54aa6X3s/ho2nkMTInwvm+1u8gsqCAjSoupdA/WkBT27dmBNaIX3bvEcdns1eSUWkmLMvHcVQPonxzBpgOl3P7Oj97P6JUJEQw96xxMwaHe32si06D0ANaQFA7t2UJciIFrvigjrPI3EsKC+EXtTrZFJTXKxKFSq+f3QXeIvc5Yn/4FH1+TSK+MftQc3snBEispvQahlGWTqabQKykOU12NUu1x//KTi083H+JMZR83DE/jsot+z5iXf+ZAcTVRwQZWPzQORVHYkX0YU9keuvfqh1K027tvU5nnAYA1oif7d21CUVW69h3qXU5sBpQe8JzfkUzfZXWfbRN/23vVJF4cG0SIUcsfllT59NMINerZ5Uwg2ZXDDSPSuGDMWJ846vrN7FGT+eu7//P+Tb36uyDO6NufJ975jFKrA3tET/48xM2/l+5jr5rk87cThI2nR2qYcNYAwrt067B7xCnReVsIIYQIGLUFfsBT4DiSCS67J6kACE/h17xy1vxWRIghhglnJLBN2w+7041GgSU78sit1pBLXwBuHZRGWVQPNu0u4bzexvqClMEM6aPYvn8vNexhXEYcr10/GIBlmQVMn7eFDWpfQo1aosPDGfuvFTjdOkw/b+bV6wdzdrcYFm3P48+fbgf0PPfrLzx0QQZWtRidBpwNHqH2ig/BZNAysFsXgtKHU/ZrPhtUT3xJZW5qFCPvH4ihBhcRZj1l1Q5ioyLIrDBR43Cj1yo8PPV6BqVGYjJoGZwaycYDpd591Hn0oj78cd5mCiugLPJM7JV2ttnrCz+vTBnIqO6xHCytJiM+1Hstpo5KZ+5P2RyptGHQabDXBp8caeJ/D4whqHY9E3BOz1B+fOg8th8qA2oL6wYt94/vxRdbcimustM12szgtMhmE4vkSE9hOCHMSL7FRg1Gcs19ufO8M/liRykbKvvSNdpMv/TE+s8LIHiIz376JIZSg5Ftag+OZNaQV3s9cteXctU5YxlQu+3wkOgGW5kJSj+LRX/yJASpkWYu+vcq8i01pEWZsDnd5FtspEaZeP6qAfSIDWHCyz+yrbIHXaPNZMT7jgZkMmjrE43gIZiAfmed533/uwfGeBKPBtd7QEq4N+4eccGcO25s/Q4NZkiuPc/gGExAz7M8zQ4X9nB59wV4X9+/YAtLfi1gg9PTpGmXoT+VNidpUWZS+p0LBi1BPUbjbSQYEceAoz+U2uP2SsmmZvMRNqh9ubfvWWAwM6JrFAeKqympsnPxv1fxzX2jGdYrGUj27q/hvsHzu3LGyAvr9x/RoFldcIzv/0e/ro2lp91FfsQQT2IWbWbkeaMBCFq3gm3lYaREmdhQEgY1ns0O05cje8xccGG0Txx116+n3UVp5JnUlFhJizIzasxoTAYtLz8y3Xst3arKlmXf4VRVytQwTzNBt0oNRtL7j4Bg35q59iaJhRBCCNFeivfBnHPBUdXsKra5k7iy8HHvE/gXf9iD2107pKcK89bn+Ky/7ZCFMc8tx+Z0E2rU8rfL+hFpNjC8WzQmg5b9RZ5jDU2P9Bb8Lj6zC8WX2nn8q1+psLm4dPYanLVNeqwON7fN3egtFNc5UmHjif95RnIK0uuotHmGLdVrFdKig73rnZkcxre/5nt/ziuz8s0vh6m2u0gMD2LpjDHsKazE5nBx7ZvrAHC4VMxGnTe+s7pGsfFAqWf/mvpmPH/+ZBvf3jfamzgcLrcy/sWVuFUI0msY3ycBk0FLVG3fgDpmo44fHxpHZkEF76zez/+2HQZgfJ9434J9LZNBy/Bu0T7LQoJ0rH74PG8BbfGOw973jDqNt1lYfJiR7xsUtqf/dzNLdxdSVGXn6jd+4rv7xzRKfJpTVFnfVj6vrMbndWZBhU/NQlPnUPf+8j+PbbLAXnf8VQ+d12hZS/kkHrV0Gg16rYLDpVJUYcdqd7Vov0fvq+71mF5xLPm1fkjoz+4eidXhPqF4e8TW/67+5csdfHvfaCb2S2DBRs+oUlnF1ce9tu3FZNA2mZgt/5Pn82r4N1In+xjxmQxaltzfeH9HX9fBqZGszy4B4Nohyfx3g+c75cFPt7Ng6sD2P9EGJLEQ4hTgcDhwuZoexUR0blqtFr2+Y58wiXa07O/HTCoAjJYsnw7MDQuX4EkudBrFmwjsybd4C7UVNhcPfLwNgMTwIJb9aSxZtYlF15hgn/2cmVzfzrykyvcYgE9SAaDTQLXd8z1SaXN6E4+E8CCcLhV9bfmuf1KEz3Yq8MjnvwBQZXOiKAoDUyKw2l3e0YSOflI+LD3S+7ph34Ds4moOllZ7C0mWGid1XRxqHO5jFgjrClc9YkOBw42O0xINC2i9GsTbJzGMg8VVlFQ76B4b4rPelUOSWbq7EPAUWhvGfzwZ8aHea5QWZQJF4UAT16s1cQONjt9UctAWmQUV3o7iZVZHmwvq5/Sof9rfJSKIXvGhJzwcapC+vmh7oLaQPqJbTLO/ix2tqWtft6zh30jd3/zx4mvJZ3lW1yhvYrEss9C7PLu42tusrqNIYiFEJ2axWCgqKsJma759rej8jEYjMTEx0mci0Kkq5Kw/amHdAJrgVBV0ikqW6tvptk6QTkNNbQIRFqTjn1eeyV0fbKaimaFPD5d7nmo3l1j0Tggj0qyntLq+w/GUYcl8ue2wN4FoSKfUT72XEmkiIyGUfEshOSVWLnr5R765z9PsYmh6lLcwVPfUuq7pkaXG6S1kmgxavrlvdJNPyhsW9I5VoG5Y8G5pgTApsn70que+zeS83k3XWhxP99j6TuJdIoLYfdgCwI7ccp8n9OMy4k640Hr0NYLGtQ2B6EQ+l2OJDTV6C9YVVic1DvcJn3+fxLBGsR3rd9GfGsaVGmlucU3X8TR8qJBvsZEYHsTh8hq6RpvpGdexSZUkFkJ0UhaLhdzcXEJCQoiJiUGv18ukaKcYVVVxOByUl5eTm5sLIMlFIMv/BSyHPLNmT/kv6IK8fSzmrz/Is5vxdqa0Y+Stm4Ywfd5mbLVPfsdkxHqbg5RUO4gPDSLcpKfc6kkMLhmQwNaccnJKPEPEahVPgaysNnFIj/ZNLEwGLZ/cNZLxL/7oXXbF4GTO7xvPre9tAsCoU3j3lrO45d311NTGERNs4LsHxvDu6ix+2FX/JL6phOHlH/awPPOId//pRxUym3u62tIC9YkUCKMatCE/UNK2Zi91hd0f9xzxJn0Nk6cTjbGh49U2BKL2LqhnFlR4a+gqbM42fWbNxdbetTbtpWFcRzfxO1Gje8b6JFef3T3Km7Q4ao5do9pWklgI0UkVFRUREhJCcnKyJBSnMJPJRGhoKIcOHaKoqEgSi0D260LP/93HQY/f1S8PjmHvLxGUkU2Z6vn8THoN5/SIZVTPWJbVNqM5t2csmfkVZNcWBnonhNErPoQN2Z6+CNcOTeO5KyPZmF3MHe9vosbpZvEvniY/ieFBTRbuesSFcmZSOL/klgOeNtbPXFE/B4bNqRJs1JGRGMqOXM8T+eHdPX03bj2nK59syvHG01TCkB4TDLWJxaUDuvDslf1bXMhsaYG6tQXCkd1iSI82Nxl3azQs7FbaXD5PfY/ZAfo00Z7nnBEf2i6fWUfE1hk1lVzVJS2OmuNs3EaSWAjRCTkcDmw2GzExMZJUnAYURSE8PJzc3FwcDof0uQhE9mpY+4rnde4mz88Nhoata65Ux1rbX2BUjxhvYtEnMZTFRxUGesSFsiG7FJ1W4YwuYZgMWs7tFcfEfgks3JrHrB88k+4d3QyqoeFdo7yJRXZxNYqikBpl5mBJfSGub2KYN7EYnOrpl2AyaBvFc7TUqPpznHJWSkA0MWlJ3C2RER9KWrSZA8XVpEeb+bzBU99AOM9TSXt9ZqKev5KrgJwgTwhxbHUdtaWAefqo+6ylk36AOrTBM6QsQFUhC75ZwkfrD2Ct7cuwt7ASgNgQz0hQdU2GGnYu/lNtp+y65kYAPeI87fydLpUrXlvj3d/4vp4J4Spr+18khvnOO9HQ/eN7ERvqeb9rtJn+yREsuX80C6eN8vabOKNLfZvsMxIb10w0V9BLbtCf4dHPf/HG52/Hi7ul+/j2Ps91WnzfaKJCDG3ep2hee3xmwv+kxkKITkxqK04f8lkHOE397XS/O4En1qnUsINZP+zl63vO5VCpp1/El9NHUVhh8z6VbTCpc5PDTJ6ZVN/0rWE/h6Tw+gI9wPe7Cpsd8jMkSMePD45r9DS44XEaDtH58Oe/8G1twnE8McH1Cc2xhsnsrE73JjVCtJbUWAghhBBtVXYQgN+0PbjIPtM7R0W+xcbbqz0T48WEGOgSYfJ5Kls3ug7QZNvyM5Mimny/d2KYTy1FXYfi5hzvabDZ0HiIzpboXTsCT3PxCyFOL1JjIYQQQrRVUSYAGxxdvUlFnTkrPYlFt5iQRpsdb3SdY41ws+je0Vz8yqpmOxS3Ru8mhuhsiUAdxlMI4R+SWAghOrXWNhFKS0sjOzu7XWNIT0/nwIEDqKp6/JXFqaloLwCZrkSig/XMnjKYRb8e5oO1B72rRJqb7hN1vOY2zb0fFWJg2Z/Gtkuhvi0JgjQXEkLUkcRCCNGp3XzzzY2WrV69mn379jFgwAAGDhzo815MTEyj9YVosyLP6Ey/qUmM6hHLiB4xDEiN5PNNuVTVdmjecKC02X4QJ6o9C/WSIAgh2koSCyFEpzZ37txGy2655Rb27dvHpEmTePLJJzs8hqVLl+JwOI6/ojgl5RZbSCjahxbY5+7CxPQowFNQ//d1g7ht7kYASqrsp1znZiGEaEg6bwshRBt1796d3r17+zsM4QdWu4u7X/kMLS6qVCP5RDLnx33eYVfP7hZDSu2QrNK5WQhxqpPEQghx2pg7dy6KovDkk0+yZ88eJk+eTHx8PBqNhoULFwLw22+/8eSTTzJy5EgSEhIwGAwkJydz0003sWfPnib3m56e3qivR3Z2NoqiMHbsWKxWK4888ghpaWkYjUZ69OjBs88+K30yTgGZBRWk2PcBkKXGo6LhUKnVO6qSyaDluwfG+MwZIYQQpyppCiWEOO1kZmYybNgwoqOjGTduHKWlpd4J6N5++22ee+45+vXrx7BhwzAajezcuZMPPviAL7/8klWrVtG/f/8WH8tut3PBBRewc+dOxo4dS1VVFStXruSRRx6hoqKCv//97x11muIkyIjS8rR+LgDpSiFB2EiMjvSpmZC+C0KI04UkFkKI085HH33E9OnTmTVrFlqt7xPkSZMmceedd9K1a1ef5f/5z3+49dZbuf/++1m2bFmLj7V27VrGjBlDVlYWYWGeyc42btzIiBEjeOmll3jkkUcICWk8DKnoHEylezApntqJEKWGr66JIqXfuVIzIYQ4LUliIcQpRlVVrA5Xu+/Xanext7CCnnHtP1a9Sa89qTNLx8bG8uyzzzZKKgBGjBjR5DZTp07lnXfeYcWKFZSXlxMeHt6iY2k0Gt544w1vUgEwdOhQLrzwQr7++ms2btzI2LFjT+g8RACI60MVQQRTgzMoml79zgJJKoQQpylJLIQ4xVgdLvo+vsTfYbTKzr9N8Jn5t6ONHz8es9nc7PuVlZX873//Y+vWrZSUlHhHfDp8+DCqqrJv3z4GDx7comOlpaWRkZHRaHmvXr28+xSdV41i5BdXOiO0u7GPexKdofnfKyGEONVJYiGEOO2kpqY2+96yZcuYPHkyR44caXadioqKFh8rOTm5yeWhoZ42+DabrcX7EoHnSIWNcKUKAFN005+1EEKcLiSxEOIUY9Jr2fm3Ce26T6vdxRWv/8SB4mrSos18fvfZ7TvJl/7kNh0JCgpqcnllZSXXXHMNJSUlPP7440yePJm0tDRMJhOKonDdddcxf/78Vo3mpNHI4HunsiOVNpJq+1gowdF+jkYIIfxLEgshTjGKorR7syKzQce3940ms6CCjPj272MRKFatWkVxcTFXXXUVTz31VKP39+/f74eoRCA7YqmhH7U1WGZJLIQQpzdJLIQQLXI6DJlZWloKNN186bfffmPz5s0nOyQR4ErLSjAotYMlmKL8G4wQQviZ1NELIUStug7Vn3/+uU8fi7KyMm677TZvJ24h6lSVFgBg1wSBdNwWQpzmJLEQQohaQ4cO5fzzz+fgwYP06tWLyy+/nMsvv5yuXbuSl5fHZZdd5u8QRYCpKfckoDX6CP8GIoQQAUASCyGEaODLL7/k//7v/4iNjWXx4sVs2rSJyZMns27dOiIiIvwdnggQVruLrTll2C2FADiNkX6OSAgh/E9RWzO8ySnMYrEQHh5OeXm5z0RWQgSimpoasrKy6Nq1a7MjHIlTS0s/c/ku6xgNr6s+KJjzX1rJoVIrV+lW8S/d6xTFn0PM3Yv8HaYQQhxTR98jpMZCCCGEaIXMggoOlVoBCFctAGhCZEQoIYSQxEIIIYRohYz4UHQaBYCo2jkstMEx/gxJCCECgiQWQgghRCuYDFoGp3r6VETWzmHx6a5qrHaXP8MSQgi/k8RCCCGEaCWb05NERCmVAGRVB5FZUOHPkIQQwu8ksRBCCCFaqdLmBCBeVwWALiSGjPhQf4YkhBB+J4mFEEII0UpVNk+NRe8wOwCPXnkOJoPWnyEJIYTfSWIhhBBCtFKV3VNjYbCXAmAMj/VnOEIIERAksRBCCCFaQVVVqmxOFNxorCWehXqzf4MSQogAIImFEEII0Qo2pxu3CvGUolA7x+yHV4C92r+BCSGEn0liIYQQQrRCXcftczS/1C8s2Q+Fu/wUkRBCBAZJLIQQQohWqKpNLEw6tX5hVHeI6+OniIQQIjBIYiGEEEK0Qt2IUF21xZ4FfS6Fu1aDQfpZCCFOb5JYCCGEEK1QNyJUD02eZ0H6OZJUCCEEklgIIYQQrVLXx6KresizIKanH6MRQojAIYmFEKJTu+6661AUhaeffvq4665fvx5FUYiPj8fpdLbqOLfccguKorBixQqf5WPHjkVRFLKzs1u8r7lz56IoCk8++WSrYhCBodrmQoeTRPdhz4KYXv4NSAghAoQkFkKITu3GG28EYN68ecdd98MPPwRgypQp6HS6Do1LnLqqbE5SlUJ0uEAfDKFd/B2SEEIEBEkshBCd2gUXXEB8fDyZmZls2LCh2fWcTicLFiwA6pOR9vD++++za9cukpKS2m2fIrBV2Z10V2r7V8T0AI3cSoUQAiSxEEJ0clqtlilTpgD1NRJN+e677ygsLKRPnz4MGTKk3Y6fmppK79690ev17bZPEdiqbE4ylIOeH6K6+TcYIYQIIJJYCCE6vRtuuAGABQsW4HK5mlynrqnUDTfcQFlZGa+88goTJkwgLS0No9FIdHQ0EydO5Pvvv2/VsY/Vx2LNmjWMHz+e0NBQIiIimDBhAj///HPrTk4EnJrqKu7ULfL8sH+FzLgthBC1JLEQQnR6Q4YMoU+fPhQUFDSZGFRVVfHll1+iKArXX38969at495772XPnj1kZGRw+eWXk5GRwXfffceECRN499132xzT119/zdixY1m6dCl9+/blwgsvJCcnh9GjR7N27do271/4T6hlD6GK1fODtVRm3BZCiFrSe1GIU42qgqMDnqA6quFIJsRmgL6dx+zXm0FR2rSLG2+8kccee4wPP/yQiRMn+rz3+eefU1VVxZgxY0hLS8PtdrN27VpGjBjhs96WLVs477zzeOCBB7jmmmsICQk5oVgqKiq49dZbcTqdvPvuu0ydOhUAVVV59NFHefbZZ0/sJEVA2K9Jxa5qMSguCE2UGbeFEKKWJBZCnGoc1fDPTjZKzWN5YAhu0y6uv/56/u///o+FCxdSVVVFcHD9/ur6XtQ1meratStdu3ZttI9BgwYxbdo0/vGPf7B8+XIuueSSE4rl008/5ciRI4wePdqbVADeYXHnzZvHoUOHTmjfwv/KHDpqMGDAClMWyOR4QghRSxILIcQpITU1ldGjR7Ny5UoWLlzI9ddfD0BBQQFLly4lKCiIq6++2ru+y+Vi6dKl/PTTTxw+fBibzQbA3r17ff4/EatWrQJg8uTJjd7T6/VcddVVzJo164T3L/zLVVNJWF1TqKjGCaoQQpyuAjaxmD17Ns8//zz5+fkMGDCAV155hbPOOqvZ9WfNmsXrr7/OwYMHiYmJ4aqrrmLmzJkEBQWdxKiFCAB6s6cGoD05quHt86E0CyK7wu3ft29zqHba14033sjKlSv58MMPvYnF/PnzcblcXHHFFYSHhwNw6NAhfv/737Nt27Zm91VRUXHCceTlea5/Wlpak++np6ef8L6Fhz/vEUZrAQBOnRldUNgJn4MQQpxqArLz9oIFC5gxYwZPPPEEmzdvZsCAAUyYMIHCwsIm1//vf//LI488whNPPMGuXbt45513WLBgAY899thJjlyIAKAonmZF7fkvOBbu/gluX+b5Pzi2ffffxv4Vda666iqCgoL44YcfvN8Xdc2gGs5dcfvtt7Nt2zauvPJKfv75Z8rKynC5XKiqyhtvvAF4+kOIwOTve4TZVgSA3RR/wucghBCnooBMLF588UXuuOMOpk6dSt++fZkzZw5ms7nZkVp++uknRo0axXXXXUd6ejoXXHABU6ZMYf369Sc5ciFOYQYzJA8J6Pbk4eHhXHrppTidTubPn8/u3bvZtGkTMTEx3g7dVVVVfP/998THx7NgwQLOOusswsPD0dROcrZ///42x5GYmAjAgQMHmny/ueWiZfx9jwixexIYV0jCCZ+DEEKcigIusbDb7WzatInx48d7l2k0GsaPH9/sEI1nn302mzZt8t4k9u/fzzfffMNFF110UmIWQgSOug7a8+bN885dce2113onsCsvL8ftdpOYmIhWq/XZ1uFw8MUXX7Q5hnPPPReAjz/+uNF7TqeTzz77rM3HOF0Fwj0i3OmpsVBDE09oeyGEOFUFXB+LoqIiXC4X8fG+Vczx8fHs3r27yW2uu+46ioqKOOecc1BVFafTyV133XXMam6bzebtrAlgsVja5wSEEH41ceJEYmJi2LBhA1lZWYBvM6i4uDjCw8PZsWMHa9asYdSoUYCnM/fDDz/Mnj172hzD1VdfzUMPPcSKFSt47733uPnmmwFP86onnniCgwcPtvkYp6tAuEdEOotAC4okFkII4SPgaixOxIoVK/jnP//Ja6+9xubNm/n8889ZtGgRTz/9dLPbzJw5k/DwcO+/lJSUkxixEKKj6PV672hMRUVF9OzZk+HDh3vf1+l0PPTQQzidTsaMGcMFF1zA5MmT6dGjB3PmzGHatGltjiE0NJR33nkHrVbLLbfcwogRI7juuuvo168fzz//PHfccUebjyFarj3vEXanm1hKANBFJJ2U+IUQorMIuMQiJiYGrVZLQUGBz/KCggISEppuz/rXv/6VG2+8kdtvv50zzzyTyy+/nH/+85/MnDkTt9vd5DaPPvoo5eXl3n85OTntfi5CCP9oWENR1zSqoccee4z33nuP/v37s2bNGn744QcGDBjAunXrGDp0aLvEcNlll7F8+XLGjRvHjh07WLRoEYmJiaxcuZKzzz67XY5xOvL3PaLK5iReKQVAHymJhRBCNBRwTaEMBgNDhgxh6dKlTJo0CQC3283SpUuZPn16k9tUV1d7O17WqWs73dzILkajEaPR2H6BCyECxllnnXXcUZ1uuukmbrrppkbL+/fvzy233NJo+dy5c5k7d26j5StWrGj2GOeeey7Lli1rtHzkyJFNHkMcn7/vEdX2+sTCGZwQeDdRIYTwo4CrsQCYMWMGb731Fu+99x67du3i7rvvpqqqyjuD7U033cSjjz7qXf+SSy7h9ddf56OPPiIrK4vvv/+ev/71r1xyySWNOmcKIYTo3Px5jyiprCG+tinUHQv2YrW72u/EhBCikwvIhy3XXnstR44c4fHHHyc/P5+BAwfy7bffejvrHTx40Ofp01/+8hcUReEvf/kLubm5xMbGcskll/CPf/zDX6cghBCig/jzHpGTvZdRiqf51N+qn2ZP7ngGdJVO3EIIAaCoMgsU4BnxIzw8nPLycsLCZCZVEdhqamrIysqia9euMrv8aaKln7l8l3WMuuu65pOXOHvHE97lNbd8T1B68zN+CyFEIOnoe0RANoUSQgghAlGNNtz72h3ZjaAu/fwYjRBCBBZJLIQQQogWstusABzSp6O5e01Az0QvhBAnmyQWQgghRAu5rZ6J8ioMcZJUCCHEUSSxEEIIIVpItZUD4NCH+jkSIYQIPJJYCNGJydgLpw/5rAODUuOpsXAawo+zphBCnH4ksRCiE6obe9/hcPg5EnGy1H3WMjePf2nsnsRCNcqIW0IIcTRJLITohPR6PUajkfLycnmSfRpQVZXy8nKMRiN6vd7f4ZzWdI5Kz4sgqbEQQoijBeQEeUKI44uJiSE3N5dDhw4RHh6OXq9HURR/hyXakaqqOBwOysvLqaysJCkpyd8hnfb0zgoANKYI/wYihBABSBILITqpuoltioqKyM3N9XM0oiMZjUaSkpJkwrsAYHRWghY05gh/hyKEEAFHEgshOrGwsDDCwsJwOBy4XC5/hyM6gFarleZPAcToqgIt6IMj/B2KEEIEHEkshDgF6PV6KXwKcRIEq1UAGIKj/ByJEEIEHum8LYQQQrSQuTaxCAqN9HMkQggReCSxEEIIIVoohBoATGFSYyGEEEeTxEIIIYRoIY3iGd45WBILIYRoRBILIYQQohVqVD3GILO/wxBCiIAjiYUQQgjRChVKsMwZI4QQTZDEQgghhGiFaiXE3yEIIURAksRCCCGEaIVqTbC/QxBCiIAkiYUQQgjRCjW6UH+HIIQQAUkSCyGEEKIV7JJYCCFEkySxEEIIIVrBoZfEQgghmiKJhRBCCNEKLkOYv0MQQoiAJImFEEII0Rp6mcNCCCGaIomFEEII0QrDCj8Be7W/wxBCiIAjiYUQQgjRCmZHKRTu8ncYQggRcCSxEEIIIVqhiAiskb38HYYQQgQcSSyEEEKIVvib/ToyS1z+DkMIIQKOJBZCCCFEKxhNIWTEy5CzQghxNEkshBBCiFa4+qx0TAatv8MQQoiAI4mFEEII0QpGg9HfIQghRECSxEIIIYRoBY1W7+8QhBAiIEliIYQQQrSCVqfzdwhCCBGQJLEQQgghWkGrlf4VQgjRFEksjmYt83cEQgghApg0hRJCiKZJYnG0t88De7W/oxBCCBGgtDpJLIQQoimSWBytshAKd/k7CiGEEAFKJ4mFEEI0SRKLo5ljIa6Pv6MQQggRoDSSWAghRJMksTja2dPAYPZ3FEIIIQKUjAolhBBNk8TiaKrb3xEIIYQIYNIUSgghmiaJxdEcNf6OQAghRACTGgshhGiaJBZHc0piIYQQonk6ncHfIQghRECSxOJoTpu/IxBCCBHAdFJjIYQQTZLE4mhSYyGEEOIYtHrpYyGEEE2RxOJoklgIIUSbvPfee9TUnLrfpXqZeVsIIZokicXRpPO2EEK0ydSpU+nSpQv33HMP27Zt83c47U4jTaGEEKJJklgcTfpYCCFEm9x+++04nU5mz57N4MGDGTFiBO+88w5VVVX+Dq19aCSxEEKIpkhicTSn1d8RCCFEp/bmm29y+PBh3nzzTYYNG8b69ev5wx/+QJcuXbjrrrvYuHGjv0NsG0kshBCiSZJYHE36WAghRJsFBwdz++23s27dOrZv3860adPQ6XS8+eabDB8+nEGDBjFnzhwsFou/Q209SSyEEKJJklgcTfpYCCFEu+rXrx///ve/ycvL48MPP2T06NFs27aNadOm0aVLF2677TY2bdrk7zBbTpFbpxBCNEW+HY8mNRZCCNEhHA4HFRUVVFRUAKCqKg6Hg//85z+cddZZXHXVVZSVlfk3yONwogFF8XcYQggRkCSxOIpbOm8LIUS7WrduHbfddhuJiYn88Y9/ZPv27VxxxRV89913WCwW5s2bx5lnnskXX3zBvffe6+9wj8klt00hhGiWNBQ9SnFZOUa7C5NB6+9QhBCi0yotLeWDDz7grbfeYufOnaiqSkpKCg8//DC33347CQkJ3nWnTJnC1VdfzaBBg/jmm2/8GPXxuZF7gxBCNCdgH73Mnj2b9PR0goKCGD58OOvXrz/m+mVlZUybNo3ExESMRiO9evU6oRuUTrWTWVBxomELIcRp74YbbiApKYkHHniAXbt2ceGFF/LVV1+RlZXFX/7yF5+koo5Op2PYsGGUlpa26Bj+uke4JLEQQohmBWSNxYIFC5gxYwZz5sxh+PDhzJo1iwkTJpCZmUlcXFyj9e12O+effz5xcXF8+umnJCUlceDAASIiIlp9bJNiJy4+tB3OQgghTk///e9/SUhI4NZbb+UPf/gDqampLdru8ssvJy0t7bjr+fMeIYmFEEI0T1FVVfV3EEcbPnw4w4YN49VXXwXA7XaTkpLCPffcwyOPPNJo/Tlz5vD888+ze/du9Hr9CR3TYrEQHh5O+SOhhP2zXDrnCSE6Je93WXk5YWFhfonhs88+47LLLkPXQTNU+/MeceCRZFJn5rQpfiGE8JeOvkcEXFMou93Opk2bGD9+vHeZRqNh/PjxrF27tsltvvrqK0aOHMm0adOIj4+nX79+/POf/8Tlcp1YENKBWwghTtiVV17ZYUmFv+8RbkVqLIQQojkBl1gUFRXhcrmIj4/3WR4fH09+fn6T2+zfv59PP/0Ul8vFN998w1//+ldeeOEF/v73vzd7HJvNhsVi8fnnJbNvCyHECdu8eTMzZsxgw4YNza6zfv16ZsyYwdatW1u1b3/fIySxEEKI5gVcYnEi3G43cXFxvPnmmwwZMoRrr72W//u//2POnDnNbjNz5kzCw8O9/1JSUurflEnyhBDihL366qu89tprpKenN7tO165dee2115g9e3aHx9Oe9wj3qXHbFEKIDhFw35AxMTFotVoKCgp8lhcUFDQ5kghAYmIivXr1Qqutf5LUp08f8vPzsdvtTW7z6KOPUl5e7v2Xk9OgzazUWAghxAlbtWoVgwcPJjY2ttl1YmNjGTx4MCtXrmzVvv19j5AaCyGEaF7AJRYGg4EhQ4awdOlS7zK3283SpUsZOXJkk9uMGjWK3377Dbfb7V22Z88eEhMTMRgMTW5jNBoJCwvz+eclNRZCCHHCcnNzj1lbUSctLY28vLxW7dvf9whVEgshhGhWwCUWADNmzOCtt97ivffeY9euXdx9991UVVUxdepUAG666SYeffRR7/p33303JSUl3HfffezZs4dFixbxz3/+k2nTpp1YAE5JLIQQ4kQZjUbKysqOu57FYvGpRWgpf94jJLEQQojmBeQ8Ftdeey1Hjhzh8ccfJz8/n4EDB/Ltt996O+sdPHgQjaY+J0pJSWHJkiU88MAD9O/fn6SkJO677z4efvjhEwtAEgshhDhhZ5xxBqtXr6akpISoqKgm1ykpKeHHH3+kX79+rd6/P+8R0hRKCCGaF5DzWPiDzzwWt30BPX7n75CEEKLVAmEeizlz5vDHP/6RsWPH8v7775OcnOzzfm5uLjfffDPLly/n5ZdfZvr06X6JszXqruuOJ4ZwxpMb/R2OEEKckI6+RwRkjYXfSY2FEEKcsNtvv5358+ezYsUKevXqxcSJE+nevTsA+/btY8mSJVitVkaNGsVdd93l52hbR5pCCSFE8ySxaILbbg3MzidCCNEJ6HQ6Fi9ezL333st7773HwoULfd7XarVMnTqVl19+ucMm0usobo0kFkII0ZzO9Y1+kthqqjD5OwghhOjEzGYzb7/9Nk8//TQrVqzwDteakpLC2LFjSUxM9HOEJ0hqLIQQolmSWDTBLomFEEK0i8TERKZMmeLvMNqNKjUWQgjRLGnx0wS7tdrfIQghhAhEijyPE0KI5sg3ZBMcNkkshBCiraqrq1m+fDl79+6loqKCpgYhVBSFv/71r36I7sSoGnkeJ4QQzWlTYlFdXU1RURHR0dEEBwd7l5eWlvLss8+yY8cOUlNT+dOf/uQdEaQzcEpiIYQQbTJ37lweeOABLBaLd5mqqiiK0ujnzpRYSI2FEEI0r03fkE8//TTPPfcc69evZ8iQIQDYbDZGjBjBb7/95n069emnn7Jt27ZO01nPabf6OwQhhOi0fvjhB2677TbCw8N57LHHWL58OWvXruWNN95g3759fPHFF+zdu5fp06d77x2dhfSxEEKI5rWpTnfZsmV0797d58bw4YcfsnfvXsaNG8eSJUu49957KSoq4qWXXmpzsCeL2yaJhRBCnKgXXngBRVFYvnw5Tz/9ND179gTgjjvu4JlnnuHXX3/l/vvv59133+10iYUio0IJIUSz2pRYHDx40HvDqPPVV1+hKAr/+c9/OP/885k1axa9evVi8eLFbQr0ZHI7JLEQQogTtWHDBkaMGMGAAQOafF+n0/Gvf/2LuLg4nnjiiZMcXRtppCmUEEI0p02JRWlpKREREd6fVVVl9erV9O/fn5SUFO/yAQMGeMcw7wxUSSyEEOKEVVZWkpqa6v3ZaDQCUFFR4V2m0WgYPnw4q1atOunxtYk0hRJCiGa1KbFISEggKyvL+/OmTZsoLS1lzJgxPus17KzXGSjOGn+HIIQQnVZCQgIlJSXen+v61+3Zs8dnvZKSEqzWTvYgR2oshBCiWW1KLAYOHMj69etZuHAhFRUVPP300yiKwu9//3uf9fbu3UuXLl3aFOjJJImFEEKcuN69e7N3717vz2effTaqqvLcc895B/X46aefWLZsGRkZGf4K84QoWkkshBCiOW1KLB566CEArrzySiIiIvjf//7HgAEDOO+887zrFBQUsG3btk7VQU9xSWIhhBAn6uKLLyYrK4v169cD8Lvf/Y7+/fvz6aefkpSUxJAhQxg3bhxut5v777/fv8G2kiJNoYQQolltSizOPvtsvvjiC8455xx69+7NDTfcwFdffYWmwQRC8+fPJzQ0lIkTJ7Y52JPF5CgDu8xlIYQQJ+Kmm25i8eLFxMfHA57+FIsWLeL888+nsLCQLVu2YDab+fvf/84NN9zg52hbSZpCCSFEsxS1qalQT0MWi4Xw8HDKHwklzKjgjuyG5u41YDD7OzQhhGgx73dZeTlhYWH+DqeR6upqysvLiYuLQ6vtPE//667r9rfu4szbX/d3OEIIcUI6+h7RphqLU5mmdD81eTv8HYYQQnQ6M2bM4Omnn27yPbPZTGJiYqdKKhpSpMZCCCGa1abEoqCggB9//JGCggKf5fv27WPy5Mn069ePiy66iLVr17YpSH/Y704gU005/opCCCF8vPrqq2zfvt3fYXQI6bwthBDNa1Ni8cwzzzBu3DjKy8u9yywWC+eccw6ffPIJO3fu5Ntvv2X8+PE+I4R0BvcFP0+vpDh/hyGEEJ1OcnIybrfb32F0CEkshBCieW1KLFasWEHfvn3p1auXd9ncuXMpKChgypQpZGZm8uKLL2K1WnnhhRfaHOzJ4FQ9l+T9mwZgMnTOqnohhPCnSZMmsXLlSp8J8U4VGmkKJYQQzWpTYpGbm0u3bt18li1atAidTsesWbPo2bMn999/PwMGDGDlypVtCvRkKSMEAEdlkZ8jEUKIzumpp54iNTWViy66iC1btvg7nHaldNK+IUIIcTK06dFLRUUFZnP9qEkul4u1a9cyZMgQYmJivMt79+7N119/3ZZDnTQWJRSooLqs4LjrCiGEaOyyyy7DaDSyZs0ahg4dSmJiIqmpqQQFBTVaV1EUli5d6ocoT4xGmkIJIUSz2vQN2aVLF3bv3u39efXq1VRWVjJ27Fif9ZxOJwaDoS2HOmmqNaEA1JQf8XMkQgjROa1YscL7WlVV8vLyyMvLa3JdRVFOUlTtQ6PV+zsEIYQIWG1KLEaOHMn8+fOZNWsWv/vd7/jLX/6CoihccsklPuvt2rWLpKSkNgV6stTowwGwV0hiIYQQJyIrK8vfIXQY6WMhhBDNa9M35KOPPsrnn3/On/70J8DzZGrcuHGcffbZ3nWys7PZuXMnt912W9siPUnshkiwg1v6WAghxAlJS0vzdwgdRqOTxEIIIZrTpm/IM844g9WrV/Pyyy9TVFTEkCFDePDBB33WWbJkCQMGDGDSpEltOdRJ4zRGgB3U6hJ/hyKEECLAaKTzthBCNKvNj14GDx7Me++91+z7d955J3feeWdbD3PSqKYoqABtTam/QxFCiE7p4MGDrVo/NTW1gyJpf9LHQgghmid1ukfRmCMB0NsksRBCiBORnp7e4k7ZiqLgdDo7OKL2I6NCCSFE89rlG7KgoIB3332XVatWkZubC0BSUhKjR49m6tSpxMfHt8dhTgp9SDQAQQ5JLIQQ4kSMHj26ycTC7XaTk5PDwYMHcbvdjBw5stOMGFhHK4mFEEI0q83fkJ999hm33norlZWVqKrqXf7LL7+wZMkSnnnmGd555x2uvPLKth7qpDCEeebfMLssfo5ECCE6p4bDzTZlz5493H777aiqyuLFi09OUO1EOm8LIUTz2jTz9saNG5kyZQpVVVVcfvnlfPHFF2zZsoWtW7eycOFCrrjiCiorK7nuuuvYuHFje8XcoUxhsQCEuiWxEEKIjtCrVy8+//xzdu7cyRNPPOHvcFpFK30shBCiWW1KLGbOnInL5eKTTz7h008/5bLLLmPAgAH079+fSy+9lE8++YRPPvkEh8PBM888014xd6iQSE9iYaYGHFY/RyOEEKemmJgYhg8fzkcffeTvUFpFmkIJIUTz2pRYrF69mrPPPpvLL7+82XUuv/xyRo0axapVq9pyqJMmLDwKh+oZTrDGIpPkCSFER1FVlYKCAn+H0ToaGW5WCCGa06bEory8vEXDBKamplJeXt6WQ500YSY9pYQCUFlS6OdohBDi1LRlyxZWrlzZ+SbTk5m3hRCiWW36hkxISGDLli3HXW/r1q0kJCS05VAnjaIoWJQw4iijqqyQGH8HJIQQnczf/va3Zt+rrKxkz549LF68GKfT2anmOQJAkRoLIYRoTpsSiwkTJvD222/z2GOP8fTTT6M9akZSVVX561//yu7du7njjjvaFOjJVKUJATdUFbVukichhBDw5JNPoiiKz0iBRzObzTz66KPMmDHjJEbWDqTGQgghmqWox/rmP45Dhw4xaNAgSkpKSE1N5ZprriE9PR2AAwcO8Mknn5CdnU10dDSbN28mOTm5veJudxaLhfDwcPLzDhEyZxDBio0SwjA9uBNTcKi/wxNCiBap+y4rLy8nLCzMLzG89957zb5nMBhITExk2LBhBAcHn8So2sZ7XX/5jrB+5/s7HCGEOCEdfY9o06OX5ORkli1bxvXXX8+OHTt4/vnnvZMi1eUrZ555JvPmzQvopKKh3L1bGazYAIjCwp7MTfQaPNa/QQkhRCdy8803+zuEjiOdt4UQolltrtM988wz2b59OytWrGDVqlXk5eUB0KVLF84991zGjh3b1kOcVEk9B1K2NJgIpQoLZlIyhvg7JCGEEAGixqXgn3ogIYQIfO3WWHTs2LHNJhHvvvsuhw4d4vHHH2+vw3UYU3Aoa7vcxITDr5MTOYIzpBmUEEK0yubNm/nwww+ZMmUKw4YNa3Kd9evX89FHH3HTTTcxcODAkxtgGzz42a/MyRiNySA1F0IIcbQ2DTfbUm+99RZPPfXUyThUu1BDPSNYGRwVfo5ECCE6n1dffZXXXnvN2+euKV27duW1115j9uzZJy+wdnDIYiezQO4NQgjRlJOSWHQ2GnM0AEGOMv8GIoQQndCqVasYPHgwsbGxza4TGxvL4MGDWbly5UmMrO3iw4PJiJeabCGEaIokFk3QhnpmrzA7y/wbiBBCdEK5ubnHrK2ok5aW5u2X11nMmjJUmkEJIUQzJLFogr42sQhxWfwciRBCdD5Go5GysrLjrmexWBrNfxToggx6f4cghBABSxKLJhjD4jz/YwN7tZ+jEUKIzuWMM85g9erVlJSUNLtOSUkJP/74I3379j2JkbUDmSBPCCGaJYlFE0JCw7GrtU/RrM3fGIUQQjR2ww03UFlZyVVXXcWhQ4cavZ+bm8s111xDdXU1119/vR8ibANJLIQQolnyDdmEMJOBUkKJpwyqiyG8c0zuJ4QQgeD2229n/vz5rFixgl69ejFx4kS6d+8OwL59+1iyZAlWq5VRo0Zx1113+TnaVpLEQgghmtWqb8jO1hb2RIUF6clTQ4lXynBUFKFP9HdEQgjReeh0OhYvXsy9997Le++9x8KFC33e12q1TJ06lZdffhmdrpMV1DVS0S+EEM1p1Te6qqonfCBFUU5425MtJEhHqeoZTrCmvBDpqieEEK1jNpt5++23efrpp1mxYgU5OTkApKSkMHbsWBITO+kTG83p8YBNCCFORKsSC7fb3VFxBBStRsGiCQPAVlGEjFguhBAnJjExkSlTpvg7jPYjTaGEEKJZUqfbjGptOADOiiI/RyKEEJ2L2+3GYrHgcDiaXcfhcGCxWDrfAytJLIQQolmSWDSjRh8BgLuq2L+BCCFEJ/PSSy8RGRl5zFm1V65cSWRkJK+88spJjKwdSGIhhBDNCujEYvbs2aSnpxMUFMTw4cNZv359i7b76KOPUBSFSZMmnfCxbYYIz4tqSSyEEKI1vvjiC1JSUhg/fnyz64wfP57k5GQ+++yzEzqG3+4PSkDfNoUQwq8C9htywYIFzJgxgyeeeILNmzczYMAAJkyYQGFh4TG3y87O5s9//jPnnntum47vMEYBoJF5LIQQolX27t3LGWeccdz1+vXrx969e1u9f7/eHzrRQCRCCHGyBWxi8eKLL3LHHXcwdepU+vbty5w5czCbzbz77rvNbuNyubj++ut56qmn6NatW5uO7zZ5EgudrbRN+xFCiNNNeXk54eHhx10vPDyc0tLWf8f6+/4ghBCiaQGZWNjtdjZt2uRTja7RaBg/fjxr165tdru//e1vxMXFcdttt7U9CHMkAEa7JBZCCNEaiYmJbN++/bjrbd++nbi4uFbtOyDuD0IIIZoUkIlFUVERLpeL+Ph4n+Xx8fHk5+c3uc3q1at55513eOutt1p0DJvNhsVi8fnXkMYcA4DJXgL2qhM4CyGEOD2dd9557Nq1iwULFjS7zscff8zOnTsZN25cq/Z9Mu4PcPx7hBBCiMYCMrForYqKCm688UbeeustYmJiWrTNzJkzCQ8P9/5LSUnxed9oDgFAhxNeHwX26naPWwghTkUPPvggBoOBm266ienTp7N9+3aqqqqoqqpi+/btTJ8+nRtvvBGDwcCDDz7YobGcyP0Bjn+PEEII0VhAjpsXExODVquloKDAZ3lBQQEJCQmN1t+3bx/Z2dlccskl3mV1Y6PrdDoyMzPp3r27zzaPPvooM2bM8P5ssVh8bhwJ7gbHLs2Cwl2QPKRN5yWEEKeD3r178/7773PzzTfz+uuv8/rrr/u8r6oqQUFB/Oc//6Ffv36t2vfJuD/A8e8RQgghGgvIGguDwcCQIUNYunSpd5nb7Wbp0qWMHDmy0fq9e/fml19+YevWrd5/l156KePGjWPr1q1N3gyMRiNhYWE+/xpyx/bFqdZenvBkiOvTvicphBCnsKuvvprt27dz55130qNHD4xGI0ajkR49enD33Xezbds2rr322lZPkHcy7g9w/HuEEEKIxgKyxgJgxowZ3HzzzQwdOpSzzjqLWbNmUVVVxdSpUwG46aabSEpKYubMmQQFBTV66hUREQHQ6qdhdUJCQ/lN7UJv5RBc+DwYzG06HyGEON306NGD1157rcn3tmzZwowZM/joo4/Iy8tr1X79fX8QQgjRtIBNLK699lqOHDnC448/Tn5+PgMHDuTbb7/1dtg7ePAgGk3HVbiEBukpVsOBQ9itFRg67EhCCHF6yMnJYd68eXz44Yfs2rULVVVRTmBeCH/fH4QQQjRNUVVV9XcQgcBisRAeHk55eTlhYWHszCtn/+vX8HvtOv5tuJ07/vwcJoPW32EKIcQxHf1d5m8VFRV88sknfPjhh/z444+oqoqqqiQlJXHttdcyZcoUhgwJ/P5rgXZdhRDiRHT0d1nA1lj4W6XNSYkaCoBaXUJmQQUDUyL8G5QQQnQCLpeLb7/9lg8++ID//e9/1NTUUPcMS1EUVqxYwbnnnntCtRV+Z68GJLEQQoimSF1xM85MiqBc8SQWyYZqMuJD/RyREEIEtg0bNnDvvffSpUsXLr30Uj7++GOcTieXXnopn3zyCcOGDQNg9OjRnTOpAHh3ggw/LoQQzZAai2aYDFrCouKhHIbEqdIMSgghmvH3v/+defPmsWfPHm/NxNlnn80NN9zANddcQ1RUFACzZs3yY5TtRIYfF0KIZklicQzmiDgoB6qL/R2KEEIErMcffxxFUUhISOCPf/wj119/Penp6f4Oq2NEdpXhx4UQohnSFOoYwqI9I4zoakr8HIkQQgQ2VVXJz89nyZIlfP/995SVlfk7pI5x6xIZflwIIZohicUxRMd2AcDoKPdzJEIIEbh+/vlnpk2bRnR0NKtXr+auu+4iMTGRK6+8ks8//xyHw+HvENuPJBVCCNEsSSyOITHBk1iEuS24Xa2bHVYIIU4Xw4YN45VXXiEvL48vv/ySq666CkVR+OKLL7j66qtJTEzkzjvvpKCgwN+hCiGE6ECSWBxDQmISAEbFweEi6WchhBDHotPpuOSSS1iwYAH5+fm89dZbnHvuuZSWlvLWW2+xb98+AB555BG2bt3q32CFEEK0O0ksjkFrDMZWO+d2Vs4BP0cjhBCdR1hYGLfddhsrVqwgOzubf/zjH/Tu3RtVVXn++ecZMmQIffr04emnn/Z3qEIIIdqJJBbHYHW4vZPkvf7NBqx2l58jEkKIziclJYVHH32UX3/9lY0bN3LvvfcSFxdHZmYmTz75pL/DE0II0U4ksTiGzIIKStQQALQ1pWQWVPg5IiGE6NwGDx7MSy+9RG5uLosWLWLy5Mn+DkkIIUQ7kXksjiEjPpQdGk9ikaIrk9m3hRCinWg0Gi688EIuvPBCf4cihBCinUiNxTGYsDFQmwXAQ9qPMGHzc0RCCCGEEEIEJkksjqVwF3pXNQDhajkU7vJzQEIIIYQQQgQmSSyOJa4PrqBIACpUE2pcbz8HJIQQQgghRGCSxOJYDGbc4x4HYJu7G2UOvZ8DEkIIIYQQIjBJYnEc+ph0AGIUC/mWGv8GI4QQQgghRICSxOJ4QhMBiFdKyS+XxEIIIYQQQoimSGJxPKEJAEQqlRSWlvs5GCGEEEIIIQKTJBbHExSBQzEAUFl0yM/BCCGEEEIIEZgksTgeRaHKGAuAvTTXz8EIIYQQQggRmCSxaAGHKR4At+WwnyMRQgghhBAiMEli0QJqbT8LTVWBnyMRQgghhBAiMEli0QK6cM/IUMbqAqx2l5+jEUIIIYQQIvBIYtECuoguAES4S5j48o+SXAghhBBCCHEUSSxaoEQTA0A8pRworiazoMLPEQkhhBBCCBFYJLFogcTkrgAkKCV0CQ8iIz7UzxEJIYQQQggRWCSxaAFjZBLgSSyevLArJoPWzxEJIYQQQggRWCSxaImgMACCFRsjv7sU7NV+DkgIIYQQQojAIolFS5TleF+GVudA4S4/BiOEEEIIIUTgkcSiJeL64FJ0AFj0sRDXx88BCSGEEEIIEVgksWgJg5mCsDMB+Dr6VjCY/RyQEEIIIYQQgUUSixZyhng6cCvWEj9HIoQQQgghROCRxKKlQhMAMNYU+jkQIYQQQgghAo8kFi2kr519O8Re5OdIhBBCCCGECDySWLSQKSoZgAhXMW636udohBBCCCGECCySWLRQcIwnsYijFEuNw8/RCCGEEEIIEVgksWihuqZQ8UopRRU2P0cjhBBCCCFEYJHEoqVqO2+bFDulpdLPQgghhBBCiIYksWgpvYlKJQSA6qJDfg5GCCGEEEKIwCKJRStY9DEA5Obsw2p3+TkaIYQQQgghAofO3wF0JpWGWLBns+mXXbx16Ee+uW80JoPW32EJIYQIUA6HA5dLHkSdijQaDXq9HkVR/B2KEAFDEotWqKitsRig+Y1FxaVkFlQwMCXCv0EJIYQIOBaLhaKiImw2GezjVKbVajGbzcTFxWEwGPwdjhB+J4lFK8TEd4FSuEn3A+P0vxITtcHfIQkhhAgwFouF3NxcQkJCiImJkafapyBVVXG5XFitVsrLy8nOziY5ORmz2ezv0ITwK0ksWqFLdKT3dYp6GEr3QPAQP0YkhBAi0BQVFRESEkJycrIkFKe4kJAQoqKiOHDgAEVFRaSmpvo7JCH8Sjpvt4K+x1jv64rgNIjr479ghBBCBByHw4HNZiM8PFySitOEVqslKiqKqqoqnE6nv8MRwq8ksWiNhDO9L19IehkMUuUphBCiXl1Hbb1e7+dIxMlkNBoBJLEQpz1JLFrDFIEtKBaArP2ZMuSsEEKIJkltxelFPm8hPCSxaCUlthcAUdXZXPDSSkkuhBBCCCGEQBKLVrMEdwWguyaPnFIrmQUVfo5ICCGEEEII/5PEopXCUs4AoIeSR6RZT0Z8qJ8jEkIIIYQQwv8ksWglQ3wGAH2VbMb3DJWZt4UQQohmKIrSqn/p6entHkN6err0gRDiJAnoxGL27Nmkp6cTFBTE8OHDWb9+fbPrvvXWW5x77rlERkYSGRnJ+PHjj7n+CYvwjFGdqjnCA3tvBXt1+x9DCCHEMQXk/UE0cvPNNzf61717dwAGDBjQ6L2rrrrKzxELIdoiYCfIW7BgATNmzGDOnDkMHz6cWbNmMWHCBDIzM4mLi2u0/ooVK5gyZQpnn302QUFBPPvss1xwwQX8+uuvJCUltV9gNeXel13cebgLdqJJGdp++xdCCHFMAXt/EI3MnTu30bJbbrmFffv2MWnSJJ588skOj2Hp0qU4HI4OP44QIoBrLF588UXuuOMOpk6dSt++fZkzZw5ms5l33323yfXnzZvHH//4RwYOHEjv3r15++23cbvdLF26tH0Di+uLqgsCoFAN55A+rX33L4QQ4pgC9v4gAlL37t3p3bu3v8MQ4rQQkImF3W5n06ZNjB8/3rtMo9Ewfvx41q5d26J9VFdX43A4iIqKat/gDGaUM68F4CvX2ewuluFmhRDiZAno+4Nok7lz56IoCk8++SR79uxh8uTJxMfHo9FoWLhwIQC//fYbTz75JCNHjiQhIQGDwUBycjI33XQTe/bsaXK/TfWxyM7ORlEUxo4di9Vq5ZFHHiEtLQ2j0UiPHj149tlnUVW1o09ZiFNOQDaFKioqwuVyER8f77M8Pj6e3bt3t2gfDz/8MF26dPG5+TRks9mw2Wzeny0WS8sDTBoEW96jp5LLd3sKObdnrHTiFkKIk+Bk3B+gjfcI0SaZmZkMGzaM6Ohoxo0bR2lpqXcm87fffpvnnnuOfv36MWzYMIxGIzt37uSDDz7gyy+/ZNWqVfTv37/Fx7Lb7VxwwQXs3LmTsWPHUlVVxcqVK3nkkUeoqKjg73//e0edphCnpIBMLNrqmWee4aOPPmLFihUEBQU1uc7MmTN56qmnTuwACZ4vrb6aA9z8cw5rfitm8X2jJbkQQogA15L7A7TxHnEMqqpidbRvTbfV7mJvYQU949p/pEKTXnvSR1T66KOPmD59OrNmzUKr9T2fSZMmceedd9K1a1ef5f/5z3+49dZbuf/++1m2bFmLj7V27VrGjBlDVlYWYWFhAGzcuJERI0bw0ksv8cgjjxASEtL2kxLiNBGQiUVMTAxarZaCggKf5QUFBSQkJBxz23/9618888wz/PDDD8d8avHoo48yY8YM788Wi4WUlJSWBRjXB1XREEs5sZSRXQzbD5UxvFt0y7YXQghxQk7G/QHaeI84BqvDRd/Hl7R5PyfLzr9NwGw4uUWF2NhYnn322UZJBcCIESOa3Gbq1Km88847rFixgvLycsLDw1t0LI1GwxtvvOFNKgCGDh3KhRdeyNdff83GjRsZO3bsCZ2HEKejgOxjYTAYGDJkiE/HurqOdiNHjmx2u+eee46nn36ab7/9lqFDjz1Sk9FoJCwszOdfywM0o0b1AOD32rUEYeOBBVux2qW/hRBCdKSTcX+ANt4jRJuMHz8es9nc7PuVlZXMnz+fhx9+mDvuuINbbrmFW265hcOHD6OqKvv27WvxsdLS0sjIyGi0vFevXgAcPny49ScgxGksIGssAGbMmMHNN9/M0KFDOeuss5g1axZVVVVMnToVgJtuuomkpCRmzpwJwLPPPsvjjz/Of//7X9LT08nPzwcgJCSkQ6oxNfF9oXgPT+g/4Ebt91xUPpPMggoGpkS0+7GEEELUC/T7w7GY9Fp2/m1Cu+3Pandxxes/caC4mrRoM5/ffXa7Nocy6U9+E9/U1NRm31u2bBmTJ0/myJEjza5TUVHR4mMlJyc3uTw0NBTAp5+NEOL4AjaxuPbaazly5AiPP/44+fn5DBw4kG+//dbbYe/gwYNoNPUVLq+//jp2u73R5DpPPPFEx4yTHRzrfdlNk0+GksPuw+VkxMts3EII0ZEC/v5wDIqitGvTIrNBx7f3jSazoOKUuf801/elsrKSa665hpKSEh5//HEmT55MWloaJpMJRVG47rrrmD9/fqtGc2r4eyKEaLuATSwApk+fzvTp05t8b8WKFT4/Z2dnd3xADXU/Dza8BcBhbRcy1RQe+XwHb6zczzfSkVsIITpUQN8fTjKTQXta1JavWrWK4uJirrrqqiY71u/fv98PUQkhGpJU/UQl17fR/fGcedRgBCCruJrMgpZXwwohhBDi+EpLS4Gmmy/99ttvbN68+WSHJIQ4iiQWJyokDkI8I5CMjqv0Lk6PNpMRH+qvqIQQQohTUl2H6s8//9ynj0VZWRm33XYbDofDX6EJIWpJYtEWCWd6/qvei0nvuZSvXjdYmkEJIYQQ7Wzo0KGcf/75HDx4kF69enH55Zdz+eWX07VrV/Ly8rjsssv8HaIQpz1JLNoioR8ASv4vdIv1jCxyuLzGnxEJIYQQp6wvv/yS//u//yM2NpbFixezadMmJk+ezLp164iIiPB3eEKc9hS1NcMnnMIsFgvh4eGUl5e3fLzyHZ/Bp7dC8jDuCX6e/23L49ELe3PnmO4dG6wQQjTjhL7LxHG19LrW1NSQlZVF165djzmztzi1yOcuOouOvkdIjUVbJNTO3Hp4O70iPC+ziqr8Fo4QQgghhBD+IolFW4QmAAq4bNy2fQpB2Nh/RBILIYQQQghx+pHEoi2O7AE8LcnM1sNkKDnslxoLIYQQQghxGpLEoi3i+kB4ivfHvkoWlZUWCizSgVsIIYQQQpxeJLFoC4MZpq2H7ucDMNPwH74xPMq1ry7Danf5OTghhBBCCCFOHkks2spghgHXen/spsknvGKvzL4thBBCCCFOK5JYtIfeF6EqnknxDquRZKopHCiqkloLIYQQQghx2pDEoj0YglH6Xw3ApqBzqMHIfQu2ctHLP0pyIYQQQgghTguSWLSXnhMAGKvZTBA2ALKKq8ksqMBqd7E1p0ySDCGEEEIIccqSxKK9pAwHIMSay/fGhwjChlGnITbEyHkvrGDS7DWc/9JKSS6EEEIIIcQpSRKL9lKR732ZohwhQ8nB5nRz2ezVHC73DD97qNQqnbqFEEIIIcQpSRKL9hLXB4IiAHDqQ8lUPfNbFFXa0SieVYKNWjLiQ/0UoBBCCCGEEB1HEov2YjDDxS8CoAlLJDjYk0DotQpuz+Tc9IgNwWTQ+itCIYQQQgghOowkFu2p62gANMV7+M/kngA4XKr37UOlVr+EJYQQQvjDddddh6IoPP3008ddd/369SiKQnx8PE6ns1XHueWWW1AUhRUrVvgsHzt2LIqikJ2d3eJ9zZ07F0VRePLJJ1sVgxBCEov2FRILUd0AOMO9h1Cjzuft4io7lbbWfVkKIYQQndWNN94IwLx584677ocffgjAlClT0Ol0x1lbCBGIJLFobykjANDuWsiIFFOjtw8WV5/siIQQQgi/uOCCC4iPjyczM5MNGzY0u57T6WTBggVAfTLSHt5//3127dpFUlJSu+1TCNE8SSzaW5dBnv+3zuO5omneOS1Mes+lPlhS5a/IhBBCiJNKq9UyZcoUoL5GoinfffcdhYWF9OnThyFDhrTb8VNTU+nduzd6vb7d9imEaJ4kFu0trIv3ZWTNQTKUHAAUxTM01AGpsRBCCHEaueGGGwBYsGABLlfTcznVNZW64YYbKCsr45VXXmHChAmkpaVhNBqJjo5m4sSJfP/996069rH6WKxZs4bx48cTGhpKREQEEyZM4Oeff27dyQkhfEhi0d66nweGYABc2iAOqHEAVNdOjHegpOWJRWtm7N5bUMHHG3NkAj4hhBABZciQIfTp04eCgoImE4Oqqiq+/PJLFEXh+uuvZ926ddx7773s2bOHjIwMLr/8cjIyMvjuu++YMGEC7777bptj+vrrrxk7dixLly6lb9++XHjhheTk5DB69GjWrl3b5v0LcbqSxKK9Gcww+b8AaF01LDY9ThA2YkIMAGw+UMqH6w6wdl/RMZMAq93Fuc8tY9LsNfzuhRXHXffCl1fx0KfbueAYs3u3JlERQggh2ktdv4mmmkN9/vnnVFVVMXr0aNLS0sjIyGDt2rVkZWXx3Xff8dFHH/HTTz+xadMmwsPDeeCBB6isrDzhWCoqKrj11ltxOp28++67/Pzzz8yfP59ff/2VBx54gDfffPOE9y3E6U4Si45gqJ8EL1Et5KtronjpmoEA7M6v4C8LdzDlrZ+56OUfmy3kZxZUUFRpByCvvIaXl+5pdt1fcstw1k6WkdPM7N5Wu4vfvbCCSbPXMGFW88cVQgjRgVQV7FXt+6/qCGSv9vzf3vtW1eOfUwtcf/31KIrCwoULqary7WtYl2zUNZnq2rUrI0aMaLSPQYMGMW3aNCwWC8uXLz/hWD799FOOHDnC6NGjmTp1qnd53bC4ycnJJ7xvIU53Mp5bR4jrA+HJUH4IgF41v2JKOqPRalnF1Ww/VMbwbtGN3ksIC/L5ec7K/SzZkc83941uNMlehMngfR0VbGhydu/MggryymsAOFhSTWZBBQNTIlp9akIIIdrAUQ3/7HL89QLFY3ne5r1tkZqayujRo1m5ciULFy7k+uuvB6CgoIClS5cSFBTE1Vdf7V3f5XKxdOlSfvrpJw4fPozN5hkIZe/evT7/n4hVq1YBMHny5Ebv6fV6rrrqKmbNmnXC+xfidCaJRUcwmGHaBvjsdshcBN89RmzYmwTxBDUYfVZ96NPtfHt/42Rhd74FgCCdhhqnG/AkIk0lBKXVdu/rCX3jMRm0qKrKwi15mAxaxvSKJSM+FINWwe5SMRu0TSYfzdmZV87iHfncNbo7wUHyKyOEEKL1brzxRlauXMmHH37oTSzmz5+Py+XiiiuuIDw8HIBDhw7x+9//nm3btjW7r4qKxjXzLZWXlwdAWlpak++np6ef8L6FON1JKbGjGMxw1h2exAIIsmQzLuIIi8uSiQ0xcKS2mdOBBrUHvxVWUF7toG+XcDYfKAXg/DPi+d+2wwAkR5qaTAjyLTXe1zm1s3svzyzkgY+3AtA12sw3940m2KjDXu3ApNcSpG9ZKzir3cUVr/1EjdPNxxtzWPHncY2SICGEEC2kN3tqAdqLoxrePh9KsyCyK9z+vecY7aUd93XVVVcxffp0fvjhBwoLC4mLi/M2g2o4d8Xtt9/Otm3buPLKK3nooYfIyMggNDQUjUbDm2++yZ133onaTk20hBDtSxKLjpQyHIIioaYUNDpevHMSd1YaSY00M+rZZVgdLiLNelIiTMxYsJXPt+QCEBdqJC3a82U+ols023LKOFhiZeblZzZZqD9cXp9YZBV52q6uyDxSv6y4ml9yyyitdgCeGcBzy6wkRzZ/w7DaXWQWVGBzuLw1JgUWmzShEkKItlCUdmla5GUIhrt/gsJdnma4hnZMKtpZeHg4l156KR9//DHz589nwoQJbNq0iZiYGCZOnAh4Roj6/vvviY+PZ8GCBWi1vve8/fv3tzmOxMREAA4cONDk+80tF0Icn3Te7kgGM9z+A6CA24np7XMYGG8gKsTAIxf2BiAp3MR5L670JhUAhRU2NmR7aizO6BJGUoTnRlFcZW90CID8BolFXrmVGoeLcqvDuyw1ykyI0TeH3HKwrNmwrXYXo59fzqTZa3jw0/qq6OBWNqESQghxEhjMkDwkoJOKOnUdtOfNm+edu+Laa6/1TmBXXl6O2+0mMTGxUVLhcDj44osv2hzDueeeC8DHH3/c6D2n08lnn33W5mMIcbqSxKKj1ZQDtVW2VYWw8lmwV3NhvwQAdhy2+CQBR7v/o63EhXn6ZeSWWZtc53B5/XJV9XTOzi6qH3Xjhav7e2sr6ny/s+CYI1IdqfB0lDtYUr/vpEiTNIMSQghxwiZOnEhMTAwbNmxgzpw5gG8zqLi4OMLDw9mxYwdr1qzxLne5XDz88MPs2bOnzTFcffXVREdHs2LFCt577z3vclVVeeKJJzh48GCbjyHE6UoSi44W1weiutX/vGYWvH42cUFuzkwK91k1KTKI+DDfzt3ZxdUYdZ6PKa/ZxKLG5+f9RyrZW1g/xvdhi63Rtl9ty2t2uNuesSHe1+Emvfd1VlEVDpe7yRiEEEKI49Hr9d7RmIqKiujZsyfDhw/3vq/T6XjooYdwOp2MGTOGCy64gMmTJ9OjRw/mzJnDtGnT2hxDaGgo77zzDlqtlltuuYURI0Zw3XXX0a9fP55//nnuuOOONh9DiNOVJBYdzWCGu9bAxS/VLyvNgsJdDEzxTSxevHogK/48jgV/GOHtY9E12ky/Lp71jpdYdIv1tNv9amued6ZvgANFVd51+iTWN2WqG2XqaLYGyUPDpk8Ol8qB4qpG659KZBJBIYToWA1rKOqaRjX02GOP8d5779G/f3/WrFnDDz/8wIABA1i3bh1Dhw5tlxguu+wyli9fzrhx49ixYweLFi0iMTGRlStXcvbZZ7fLMYQ4HSmqDK0AgMViITw8nPLycsLCwtr/APZqeGUQVOR7fh55L5WRfZjwbRi51Yp35Ka6pkZ1nacz4kPZkF3CTe+uJyM+lCUPjPbdrdNNxl8Xo6oweVgyH2041OjQVw5OxqBTmL8+hztHd+ONHz2d31IiTXw57RwOllaTER/qPfbufAsTZ3nG+Q42aKlqUMiefd1gLu6f2O6XJxBY7S7OfmYppdWORp+HEJ1Fh3+XnaZael1ramrIysqia9euBAUFNbueOLXI5y46i46+R8ioUCeLwQz3bPYMC1j4K6z9NyHAalM0eye+TsoZQ30KsSaD1jv6UpcIz5dUUzUWhRU1qCoYtBrSY0J83tNrFW8tQ3Bt5+1uscGkRJnIKbHyxCV9Oe+FFZRZHaRHm1lcW5Cu618B+CQVAHsKKriYUyuxqEviso5UevuiNDdniGibQ6XV5JZa6Z8ccVolbQ0fFJxO5y2EEOL0Ik2hTiZDMIx+0GeRYi2m1zfXYHpnjKdWowmJ4SYAKmxOLDW+nbDrRoSKDzdyw/A09FrF+96Irp4ZvbOLq70dvBPDTaRHe5pMrcg8Qlltx/HsBs2iGiYWdermvVifVXxKNROqtjkZ9cxSJs1ew8zFu7zLU6PMHTYC1una3OpQaTXnPruca99c12z/npbqTNew0FLDiJk/MGn2mjaftxBCCBHIJLE42XpN8ExidLSSfZC3pclNgo06IsyeTtSHy+o7alfbnKzY45mvItSoR6tReHnyQO/7Ow97Zu8uqrR5O3N3iQgiJcrTf2NbTpl33Siz3luQLmwisRicGgnA2v0lJ1Q4qrA6WLXnSMAVqr7alktJbS1FYUX9cL6vXT+oQ54sb8gqpt+TS07LQuZ3v+bXjY/WbP+elrDaXYytHQ75/JdWBvw1fGXZXsqtTqBt5y2EEEIEOmkKdbIZzJ7JjPK2gLMGPr0Vaso87310PUyeB10GNRqPPCEsiLJqB9tyysgtq2ZHXjmvLP0Nh8tTVNt52MJFL//IM1f2925TXGUn1KilwuairidNhMlAWm1i8Uuexbtuny7h3oJ0UzUW/ZLC+GlfMdC4mdDBkir2F1YxvFu0T2F804ESvt9ZQGpkMH/9agcut0pSRBA/zBjr05dk92ELIUE6yq0OzmgQx8lgc9Z3MQoN0lFR4ykA1hUE29unm3JxuT3HPN2aW7kb9ObqEhF0wjVCmQUVFNT+jh4qtQb8NSyqrE9Y06M7riZMCCGE8DdJLPzBYIb0UZ7Xk/8Lcy/yvK4p9byO6g53rQZHNez9DmuP33Og2NNM6qHPtje726zaddKjzWQXV5MWZSLf4pskXD3nJ+4f36vRthuzS6hxuAjSa5ussRjZPZr/rMnG4VKJCzV6C0fFlTbGPr8Ct+ppPrTkfk8/jWqbk2veWOctRNfJLavxFgStdhejn1vOkcr64zXs69ESbW27fqi0vvlZpFnvTSwKK2qa26RNHK76p+tdT2Ihs6zazr7CSvqe5MStobwGtW0PT+h9wnFkxId6+w91hkkbf2sw9PMbNw5pt+tvsTr4Na+cgSmR0m9DCCFEQJDEwt+6DPIkEiX76peV7IMt/4UlD4PbiVG5j7vdE/lV6cpKdSA1+M51odMoON0qXaPN9E+OYPF9o8ksqMDmcHHtm+t81s0qrsbpbjwQmM3p5r8/H2DKWWkcaaJQnRIZzBWDkliw8RAX9kvwFmS++SXf+yT6YEn9E/g1+4obJRUARp3GWxDMLKjwSSqgvq/H8Z5AW+0uVuwp4M8fb6fK7mp2FKfjJR5ZDSYSbDgZYOFRCVl7db49Uvv0OtSoPWmjThVaahj5zDJctb8j/hrtquFQxYeaGTq5JUwGLWaDlnKrk+Qoc8AWqiusDnYXWNh/pD6x+DXPQkZC20fhqLY5OeufP1DjcJMcaeL7B8YE7HUQQghx+pDEwt8MZk/tRN4WWHADWEs8yxf/ybuKRrVzr/4rAIrUMM6xvUwNRnpEanhllINkTTH5ugSSG4wsVVcj0DXaTFZxtU/ycU7PGJ8QRnaLZu3+Yv729S5eW74Pc5BnH2FBOiy1T/Djwoyc3SOGBRsPsSWnjEOl1eSUVJNv8S0gHqmwYbW7qGjQybzu2ABmvcbbETwjPtTnPYAIk57USDNbc8qaLcRb7S5+98IK8hpMDNhUs6K9BRVc/tpPVNqczRao9x9pel6OAouNlXsKyS21MvGMRH734gpKqx2trlE5Wl1n+wrbyesX8NnmE29+1Z6jGWU3SCz2NShst5bV7vI2VcsuqsLtVtFolONsdXIt3JLL/Qu2Nlq++WApVwxObvP+5/6UTY3DM99MZ2gOJoQQ4vQgiUUgqGsade2H9c2imhGjWFg6ZB0VcUPJWPsgyg9lAIQCrKttQlXbP8Nk8DwVzyyoIDXS7DNfRaRZT2m1g2CDluuHp7B2v6f/RFGVHWrLfwNSIli1t4ggvYZQo45h6VEAbD9UzjnPLvccQ+/b//+O9zfSNdrMmF6xAFzYL4F/TDqTfUUVXP/Wz5RanWQVVdGtdnbvuqTi/vE9mPXDbySGBzHx5R8prLA1mwxkFlT4JBUA0cEGnyYxTpebu+dtotLm22m2YeHL4XJzsKTpkbgyCyp4d00WAC9+v8c7DG1La1Sak98g7nxLDfGhRvYUVjZZcG+vQv2hkvoCfXyYscVNh8qrHZzz3DIqappPzFrK5VbJaVAj1FxC1xJ55fX7sTnd5JZZSYkys3ZfEdlFVUwalOz3p/efbMrx+VmjePqYrN7rGVWtLfHZnW4+/PmA92etRqFHbMgxthD+IFNEnV7k8xbCQ0aFCiR1zaIAlNqCR2giRKT6rJb06xv0Xn4HSl2n7zol++Cnf8O612H7J3BwA6aDKxjo+pUog5OBKfVzByRHeIaw7REXwu/6JBAf5tu8CmBQbeE5zKSnxuEm0mxAe9STYWvtU9MbhtfHmFVczU+1icr4PvFEhRgYlh7NwBTPyFIbs0uB+qfWkWY91w9PB2BXfoW3j0dzI+j0iA2hLorg2vNJjw7GZNBitbvYcrCU55dk8lthfeG1qU6zOSVNNwsDz7C6dRp2vo0JMZxQm36r3cVPvxVRYavvFL50VwFn1I4QdeFRI0RZ7S7G/WuFd+SjAksN/1mTxYHiqiaHWS201PDdr/lNjpD0c+31BphyVmqLC7X/XX/A2+ekraMZ5VtqsDeY0X3/kUqqbc5jDhnb3JCy+Ucllb8VVlJUYeO6t37m0S92MGHWyR9t6+hYj55z5pwenkQ7u7iKiW2Mb+GWXJ/+Ki63yuebD1FV4+TDdQf4YnPjSTLFyaPVev6+HA7HcdYUpxKbzXPf0unkea04vclfQCCpaxZVuAsi06D0AMT18byXtwXytsJ3jzWxoQJ1A3mumNn0viNS4Y7lcCQTa2x/9tf2Lah7crz43tGM/ddyb9Mng1ahV0Lt8LMWm3fEqab6TRi0Cg9O6M0XW3K9E+rtKfAkDYNSI7zrDUmPZH12Ce+vzaLK7iDE6BlCt0dcCCFGHQadBruzvvAZpNc0WYjPt1hRAaNWw1f3nMPvXljJpoOl/HKojD/O20xOaeP2+/+6ekCjAnVd/4oQo85bs1H3ZLlutK2jXTqgS6ufNn+1NZd7P9raaPnbq/Z7R+vKLq5m+6EyhnfzzD2SWVBBvsVTeDxUauW8F1ZQZXPx1P92ApAaZeL5qwbQPzkCgNHPL6fG4W406lahpcan8/DegpY3QcptcB2Pl1BV1Dj4aH0OSZEmxmXEYTJoycy3sOTXfK4YnMyyXYUAdAkP4rClBkuNk2H/+KHZ/jH5lhrGv7CyyWZsRxfa9x2pJLfM6h3KtmFfn4aaqwHKLbNysLjqhDtB55ZZGfv8chwuT1PDz/84yjvYQp0eccH8uNczNPSBZuJrSawAP+wqaLT+41/9yj+/2UWN043b1nQtnDg59Ho9RqOR8vJyQkNDUZTAaqYn2p/L5aKkpITg4GBJLMRpT/4CAo3BDMlDPK+DG/SFSB/lqdHY8BaUeproEJ4Cl78BVUXwyU3H3m/ZQXixD7js6PVhDHHciVUJ4hdbN08hJ9rN11eYuOC/RURRQReljBjDmd7Njx5xKjHcyOFyzxOaM7qEE27W8+/rBnHb3I3ebYL0GhLCgrw/D0gJB2BHXgU78nZ5J/PrERdCZkGFT1KhAWocbr7YcojLa5u2/JpXzo+ZR4gO8dSunJEURpdwE0F6DTUON9e9ta7Zvgs/7StmW04Z8WFGftfH0/m8Lqk6u3s0S3cV4FI9/T525Tf/ZH5vYev7Bsz7+WCTy48eseu+j7ay/M+epKBXXIg3yVGAqqPO62CJlWvfXEdcqJGnJ/XztrdvOOoWeCZBBNBrFBxulV/zyr37OF5Tq40H6ms6osyGZs8v31LD9W+tY1/t9Yww61l0z7lc8uoa7E43L36/17tu97gQVOBweY03CW2qmdrsZXubbcZ2uIkai+RIk/fnCJO+URKUVVTJRS+vwupw+yQqvxwq49JX16DS+hHJ6sz8Zpc3Ec0qrmbp7gKOzr8X/XKYqGADJVV2FDy1ZcWVNs7uHtPoeNU2J6OeXUZptYOYEAPf3T8Gs1HL/7blkRhuYn9R/e9gTIjBW6NW0+DvR/hXTEwMubm5HDp0iPDwcPR6vSQYpxhVVXG5XFitVsrLy3G73SQmJvo7LCH8ThKLzqThHBhQP9+FvdpTI1F2EBQdqLXNbRQtqA0KpC5PAUTnsPC+8XkAKjER9MNQOLCKVGCj0UCI4lmv+KN/MSP4SnZZw8iKOJsBhjy+vQwy9YNJiY1m3DPfkO46QN6RrljtLs7uFkNSRBC5dc00HFb+9NI7vHjPdZiCQ4kPqU8yoL5WoHtsCBnxod6kJT3azIDkcL7cdpjHvtjBM4t384/Lz+De+dtQwZuQ9EsKJ7OgwluorrC50CrgUvH+H2XWU1Lt4MXv93iPG2rUcteYHnzzSx4AKVEmz01fVY87WtHOPAsbs0tYt7+Y287pBtCocF5tc5KZX0HvxDCC9BqfGgNQGarsBhR2qF19RvjKt9Sw+7CFQWmRHKm0eQunDcuodclGncIKGw99ss37s0mv8XZ+T40089TXvwJg1Gtx2JxkF1dTUePgQHEVk9/82VsjMP8PI8muLbDqtArlVie7GyRYeworGfev5bw8eRD9kyN8znXCc9+S7jpAECnUYKSs2sGFL//okyjWCdJpKTpqJLCmakN2Ha4/9tH9Qupmke8WG8z+I56mYWXV9c1OBtQmIHUDALjdKrfN3ehttleXqHSLCeauDzd5r++x+s80l4Ttya/gm18Oe38ONeq8zZx6J4R6r2GBxca8285ixsdbKaiwc898z99wl/Aglv5prM8+F+/I9/bpKaq0M+ZfyzmnZwyLf8n3iWnuLcPonxzB+S+tpLjKjggcYWGekb+KiorIzc31czSiI2m1WsxmM3FxcRgMzT+AEeJ0IYlFZ9NwDoyGy/74c30TqiOZnuWxGZ7XOz6HjW83ubsQrHBgVf3PSn0BJRoL97r+AwZQa15Hedvz3oCgSHaPe5MV2mlE6irJc0eRs1VPr6REll4O3/xaydPrnSwzPkhkTRX22S/CNe/SO7Y/aVEmDpT4Ft67x4VgMmi9w+RmxIeyMbuEL7d5CmyWGif3zK8vPNclJP26hJMRH8oZYTWcXfU9H7vGUq6GolFgxYPjKK6yU21zcN3b632OV2Fz8fx3md6fP910yNvXoqLGiVmvpdrhKRyO6hHNmt+KCcJGhpJDZlUKN85ZQS8lh0/W9qDcpaes2kGXiCC+nn4un28+xMzFu3Cpnifgb9001KfQ9xfdh9yuWwzAfncCF9lnEh0RQW5tQrNkZz69E8PYdqi+ZqHOJf0TeerSfqzPLuJPtcPsApTX1PfbUFWVS15dRW5ZDbEhRm9NR6XN6U2yPlqzm6UrV+K0dwGMZBVXM2Lm0iZ/P7pEBHnb8+dbbFz75jqfJ/5fbNjLMu10onUVZLvjmGh/lhqM3iZ1R1u3v6hRM7NRPXyf2lfW9r3wxhBuYvOBEjQahYEpkd54RnaNYv+RKnbnV/jUJP1yqIxzn1tGUaWdCJOOGqfbm3wGYWN0eCGpITDm+eXeAjx4msQ1NSJZfnkNE2f9SJnVd1Qwa1UFf3/9LYbgZqfSjSrViN3p8tb0DO8aRbXdycESK12jzQxOi+LGken867v6JLekvJxffl7KLncSV4/IwGzU+YyeBZ7fyaOTivgwI2N7xwHw/QNjuPiVVRwuryHEoMXSeBoa4QdhYWGEhYXhcDhwuU5unx9xcmg0GqmNEuIoklicKpprQhUc46nZ+O07T41GaCJodFCe0/R+mqG4GzwRrSml9+KrqetB3UVTAt9MBiAIuAK4KEhPEJ5Cm6E6H+ZehCmsC9/+YSU7Cu189L9vsBVnk6HkcMbBvZB+OyZjiOdpsb2aYfr9jAo+TGV1FZlqSqO5O8DzxNpEDV+7/4iir+FPuk+YZr+PiqRRpESZSYkys3VfHgOU35rdB3hm2Y4LNVJYYSMt2oyiQHZRNWaDllcmD+aSl77jK+e9RCsVZLvjCVOqiFIqKbaHMMN+Nz/Tl7wyGPnMUmwNntJnF1ezeEf90+yrNcu9SQVAN00+F0Tk8eyMy3j661/57/oc5qzcz4INOfz+zATA0za/rhP65oOlmAxaJvbrwlnpMT59YrwfjVP11hg1nCMkLcpEYYUNM1Ym/zieOzQ17Dd4EhunNghnM31Keid4hgRuOMdHw6ZJ1m0LiVY8T+XTNYXcHfQtr9dMBCBDySG512CuGZnBzXM3AJ6kLjE8iMPlNSSEBZFvqeHn/SWoqoqiKFjtLuatO4DTrRJu0lFudbIlp4zr3/Ekh+nRZvRaz5gTXWODvTHV9f3RaxVKGiQLZVYnQdgYqBwkkWKeMbxNuK0a67uvYa1+Ehr8TjidLiY99ynJjoMURw5k4f3ns+9IJZe+utpbS5RdXM1nmw/RO9xF/89G875SAUY44I7jYudzVLoMfLnVUxPWN1bPowNCyFT70CvJ0+/ktnO68eHPB8gvt9FHyWah4XGMS53EuBO4YNWLfP/QRFb/VgRAuElPubXpDsBnp5rg0CaI60NUiJllfxpLZkEFuw7k88HnX9K6v27RkfR6PXq93t9hCCHESSGJxemgYY1Gw87gzhpY9Gco3Q8RaeByQEUe7vBUnA4HhurDjZtTtVBdUuHDkofp1QEMU2CYoxrqao3XfIX687Mo17zn6bC+7G8E2SzMAzBCnjuSq51/o4taQIymEpcK4VQyb/5e+g+xYHB6CtJBipN3jC9QWfYulCyHn2YzYMt7fGm0s9+dwD2mZ0hx5VBcZWcnnifMAGnRZr64fRBFWVtJyRjCzR/sILuoml5xIUSFGFg+dh+GpXWF5/qOs9FKJe8Zn/c+qccJA5QcbxKj1yrk1g5nO1Szm2f1bwGepk11z7f+6pqNiVu5fHAy/13vKQ6WVjuYV/t64hkJvFromTyxYf+JqBADC6eN4rwXVnrjqWtKdrSoYANPX3YmN/1nPQ/pviRU8Vyvbpp8MpQctrl6+KyvAerSo18OWVgybSi5mZu4aVElpQ4dWg24K/JZsWQ5ifnLoEEXgftYwC3GL9EAoUoNFUVp6FLWeOdT6Rpt5rO7R3GwtJo0YzX3//t9qit07DzYF4tTzx/eXUU390GCSGFYWiw/7D7iE1t2sSfhAxjRLZroYIO3Rig9xkx4kN6ntqcPWXxk/Afhiu91MVUc4ArNj+xVkymLOIO8civTlU+5U1mExgjlVSZKF/+BL/eFYlC7+ySlf1m4g4d18xmqq2+ulaYp5GJW8yWjiKeEu3Vfcdl3awnCzoCo2mGgD+3CdHAdy++5kW179tPjy7swKk7vZxFd9Rvrs4q9tTVf/WEQmdt/5t7ldmowEoSNM5V9DNVk8qf9X8FvNs8ocnetxmQwMzBG5cwPzudiYznhjX4LhBBCiI6nqDL4MgAWi4Xw8HDKy8u97WNPC/Zq34Tj6NchcfDuBLDkepIPtwMseZ5aD7fTUwOCAhV5zSchxjCwWdoUZsPCeEuWN7kPjcFb8+I2hFGYehFodET2PQ/jtzOgpgx3cAJPV15EiLOMan0Ej4yKRr/lPaguPOa+X3dcyE26ZQQrNkoI5UHbHVgIZq+axFma3TxveJdwmrkGE5+jJrIHV398hNDqbAzYsRBMNBZmXDKcR1Y5MJTtISVMzz+vHEBQZS6kjoDKAh5b+Avf5IczVJfF5f2i+NNWT/OYAfpctjmSvAXiBX8Ywb8+Wco71fcQVptY2NFym+1P1GAktOsw7hzfDwCbw81N/1mPDid/0P6PPxu/ROO24zTFMbX8VlLJ52/699AqKqoKigKfaS/iStc3TZ/f7cuwRvYiJ3MTKWk9MWV+Dnt/QM1a4f3sColggeb33Oj6nAilmsPuSG7W/J2uxipWlseiKNCPfei0GnY5u9BTOcRbNw7FGtefUS+sw616mjJd1C+ejzd52rRfb/6Zv7tfPu7vh0MbjNtp8xbyj5avhvOK43JCtU7mOs7DhoGlhj/TTZPfaN1y1UQoVhrN13fmVfDLp57XGoPn76TB30qNquMW+4PcfQYs3lnA/uhxfKx5DMpzOEIkHzrGcLd+UdMJ++3LPLWVix+Gn+dgsamEP1Nx+n2XdbDT9h4hhDildPR3mSQWteSmcQxNJR9HD4fbsH9Hw5qQqO5w05fw9nio9C2IOVHQcaxfv4bPzo+vyB1KjObE51o4Hhc6tHgKn4fdEQAkasqOuY1bpVEh8xDx6FUb8cqxt61zvOSp4TGsqg5QMCkOignjHccFpAfZmHT2GehXP4+iulAVHS5Fj85d37yplFBChk9FbzBgD05k/g/rGGFfTYbmcNMHPUqWO56kEDBU+w6FWoURzXWfYPr8Rqhp3G/kWOrOy6k1o6gutG5bo/N16kL5t3U869x92aOm8OiIIGauq+EczS88r38DU4NkQVW0KKqLctVEuHLsTvrNsWDibcdEZui/wKIamW6/j3HarUzVfXdC+wO8yVlDNrQYaUFNodYAE58DjdaTWDirJbHoIHKPEEKcCjr6uyygJ8ibPXs26enpBAUFMXz4cNavX3/M9T/55BN69+5NUFAQZ555Jt9808wTVNE6df03DOb618ExTS9LHwU9fgd3r/E8Sb1rNUSkwL1b4JZvINIzkhJhSTimbWPvRR9TM/kTCEuqP15Eumfd+7dDWJfahfW/qqrSoO1N7euasK783v4Pct2eeSDcSm0rv9AutJe6pAI8CcW/nZcfd5tGT66BB2x3MMb2EnOdF7TouMd74t7wGCbFiUnxPNWOxsJD+k+5xvU/DKueQal9Qq6oTnQZ5/vsI5IK9D//G1b9C8O3f+Jm5yctTioAumoKcF/6qvdzPKx6GuMEY8P030tbnVQ0PC+dq9qbVBx9vjpnBTP0X/Cx8R9sCbqLa7fewpagu3jVMBuT4qxPW8OSsN69mcvtTzPG9hI57gb9kBpwhybyce9/c8Ad1+T7YViZof8CAEXRsl7tw4Kw23AHN17fHXL8oSeLCWuUVADHTSryifKcm8sOi+6H/90DzmrcaL1J76lO7g9CCBF4AjaxWLBgATNmzOCJJ55g8+bNDBgwgAkTJlBY2HSTlJ9++okpU6Zw2223sWXLFiZNmsSkSZPYsWPHSY5cAL7JSN3P6aPqE47pGzHFptHzrAkE9b4Apm/0JBO3fAN/XOtZNyIFpm/yrP/gXu/7yp/31K/75z1w+zLUO1dhik7ld/Z/cXfQc9ju21Wb2Kyqn808sivc8LnnX0R6fayauiTEd5ZztW65Ru/d3h3ZFYAyUyqL3cPZ707wXfeofbhq/8QctR0R9rsTKI/oR3x0JM84p5BLQn0cDRImR8NEqmEnhrrjNFi31T1gwpLg4hdxhyQcf906oYkQHF9/zLAU9k74AFeE53q4I7sR1O0cmL6Rmlu+Z2bQQ83sqMFXTkQ6NZM/YfnQOWS7PfvOdsdhM9de02ZSqmb6mXvXVo5edulsmL6RPbYItri7U0YY59ufZ9mIuZ6ktkGyq7lrNZdccQNTTS9zte0v3Gh7iCNENH1JqOara6L44v7z0fxxbX1yHNkVbvkGzd2r63/36n6HQhMhMt17PPO01dSEea6hXW3m67guOY7sSs3kT9hz0cdE3PBek1dHg4uXgqY3vZ9TiNwfhBAiMAVsU6jhw4czbNgwXn31VQDcbjcpKSncc889PPLII43Wv/baa6mqquLrr7/2LhsxYgQDBw5kzpw5xz2eVHN3fs1O+NawKVddomOvrp8PJDaj8SznDZc30exrU008V769zTsM7dO3XkJ/U2mjfVgjepKzfycp3fqilGWTqabQK8nzdDuzoIKMKC2mI9vrj1c7VLA1oieH9mwhJcpEUGJf3yGE62I6et0whaAlD3uaoIUmglbvGQkM6vvEhCXBnT96apfs1dQc3EjOkTK6//wXNGUHPOs27CsTkQ6TXvOMLNbw2jScQ+XoawtYqypQ3jiXIEuW77FvXVIfU+0+rHYXl8/6HmNpJrbIDL6YNgpT6R7f/j2RXamZ8BwHLSox6f0pzt7uOd9vH4SybN+4G8Yf2c2TzNYe56KXf/R2IvfO5t3EOVjtLrYfKgOgf7zB8xk5a+Dr++vjb7DvY/6eNdd0sG49ezU1eTu4bF4uYZW/kWiCl8LmoS3L8iQmt31Xv23D/c45B0r2+c5dE9mNgmu+JqFL8in9XXay7w8g9wghxKnhtOxjYbfbMZvNfPrpp0yaNMm7/Oabb6asrIwvv/yy0TapqanMmDGD+++/37vsiSeeYOHChWzbtq3R+keTm4ZojWYLqf52dH+YppKnBgmAz3YN161LZOoSiLbE0rBQ3cy+WpUUHivuo5Kuo+M/3mzjLTqno5OrduITG7Zjn3ddLEfPXdNlEJYa5yn9XeaP+wPIPUIIcWro6O+ygBxutqioCJfLRXx8vM/y+Ph4du/e3eQ2+fn5Ta6fn9945BYAm82GzVbfbru83NMG3GJp2+hF4vTx0dSB7C2soGdcKI6aKhw1/o6oVlhPqJvfIupMz/+uBstrmvkdb7hu3etjrd/SWFpybKBbuKbp69iauOuOd4z4mz1OS7XXtWmCNzZo0TVr6nzrvsMC8JlRuzgZ9weQe4QQ4tTU0feIgEwsToaZM2fy1FNPNVqekpLih2iEEKJ9FRcXEx4uM1qcKLlH/H979x9TVf3/Afx5ufJThQsi3qvIlctQFwKWFrspcwUhDsukOTRn6LKWwdL81dQC7R9bTfuxNcutaa1p6Rq2XJoIgpo3lo6WaGNCKir3goAo/kBAXp8/+nK+3rj8OFy898Z9Pra75P1+n8v7vMbpuRfcey4RDWWPKiM8srEIDw+HVqtFXZ39rSvr6uqg1zt+w6ler1e1fsOGDVi9erXydWdnJ5qamjBq1ChoHN2mhezcunUL48ePx5UrV/iygH5izdRjzdS7efMmoqKiEBYW5u6tPBKuyAeAGeEMXrcDw7qpx5qp96gzwiMbCz8/P0ybNg1FRUXKa2g7OztRVFSE3FzHdzwxm80oKiqyew1tYWEhzGazw/X+/v7w9/e3G9PpdIOxfa8SHBzMi1kl1kw91kw9Hx+PvemfU1yRDwAzYjDwuh0Y1k091ky9R5URHtlYAMDq1auRnZ2N6dOn46mnnsInn3yCO3fuYNmyZQCAV155BePGjcPWrVsBACtXrsSsWbOwbds2ZGRk4LvvvsPp06exc+dOd54GERENMuYDEZFn8tjGIisrC9evX0deXh5sNhumTp2Kw4cPK2/Aq6mpseu2nn76aezZswfvvvsuNm7ciNjYWBw4cABTpkxx1ykQEdEjwHwgIvJMHttYAEBubm6Pf9ouKSnpNrZgwQIsWLDgEe+KgH9eJpCfn9/tpQLUM9ZMPdZMPW+pGfPBc3nLz+BgY93UY83Ue9Q188jPsSAiIiIiov+WofnuPiIiIiIicik2FkRERERE5DQ2FkRERERE5DQ2FtSjzZs3Q6PR2D0mT56szLe2tiInJwejRo3CiBEj8NJLL3X7EKqh7vjx43j++ecxduxYaDQaHDhwwG5eRJCXlweDwYDAwECkpqbiwoULdmuampqwePFiBAcHQ6fT4dVXX8Xt27ddeBau11fdli5d2u1nLz093W6NN9Vt69atePLJJzFy5EhERETgxRdfRGVlpd2a/lyPNTU1yMjIQFBQECIiIrBu3Tp0dHS48lRoCGFG9I0ZoR7zQT1Pygg2FtSruLg4WK1W5XHy5Ell7u2338ZPP/2E/fv3o7S0FLW1tcjMzHTjbl3vzp07SExMxOeff+5w/sMPP8Rnn32GL774AmVlZRg+fDhmz56N1tZWZc3ixYtx7tw5FBYW4uDBgzh+/Dhef/11V52CW/RVNwBIT0+3+9nbu3ev3bw31a20tBQ5OTn47bffUFhYiPb2dqSlpeHOnTvKmr6uxwcPHiAjIwNtbW04deoUvv76a+zevRt5eXnuOCUaIpgRvWNGqMd8UM+jMkKIepCfny+JiYkO55qbm8XX11f279+vjP31118CQCwWi4t26FkASEFBgfJ1Z2en6PV6+eijj5Sx5uZm8ff3l71794qIyPnz5wWA/P7778qaQ4cOiUajkWvXrrls7+7077qJiGRnZ8u8efN6PMbb61ZfXy8ApLS0VET6dz3+/PPP4uPjIzabTVmzY8cOCQ4Olvv377v2BGhIYEaow4xQj/kwMO7MCP7Fgnp14cIFjB07FiaTCYsXL0ZNTQ0A4MyZM2hvb0dqaqqydvLkyYiKioLFYnHXdj3KxYsXYbPZ7GoUEhKCpKQkpUYWiwU6nQ7Tp09X1qSmpsLHxwdlZWUu37MnKSkpQUREBCZNmoQVK1agsbFRmfP2ut28eRMAEBYWBqB/16PFYkF8fLzyIXIAMHv2bNy6dQvnzp1z4e5pKGFGDBwzYuCYD71zZ0Z49AfkkXslJSVh9+7dmDRpEqxWK7Zs2YLk5GRUVFTAZrPBz88POp3O7pgxY8bAZrO5Z8MepqsOD1+kXV93zdlsNkRERNjNDxs2DGFhYV5dx/T0dGRmZiI6OhrV1dXYuHEj5syZA4vFAq1W69V16+zsxKpVqzBjxgzlk6P7cz3abDaHP4tdc0RqMSOcw4wYGOZD79ydEWwsqEdz5sxR/p2QkICkpCQYjUbs27cPgYGBbtwZDXULFy5U/h0fH4+EhATExMSgpKQEKSkpbtyZ++Xk5KCiosLutexE7sCMIHdgPvTO3RnBl0JRv+l0OkycOBFVVVXQ6/Voa2tDc3Oz3Zq6ujro9Xr3bNDDdNXh33ddeLhGer0e9fX1dvMdHR1oampiHR9iMpkQHh6OqqoqAN5bt9zcXBw8eBDHjh1DZGSkMt6f61Gv1zv8WeyaI3IWM0IdZsTgYD78P0/ICDYW1G+3b99GdXU1DAYDpk2bBl9fXxQVFSnzlZWVqKmpgdlsduMuPUd0dDT0er1djW7duoWysjKlRmazGc3NzThz5oyypri4GJ2dnUhKSnL5nj3V1atX0djYCIPBAMD76iYiyM3NRUFBAYqLixEdHW0335/r0Ww24+zZs3aBW1hYiODgYDz22GOuOREa0pgR6jAjBoe35wPgYRkxKG8/pyFpzZo1UlJSIhcvXpRff/1VUlNTJTw8XOrr60VE5I033pCoqCgpLi6W06dPi9lsFrPZ7OZdu1ZLS4uUl5dLeXm5AJDt27dLeXm5XL58WUREPvjgA9HpdPLjjz/Kn3/+KfPmzZPo6Gi5d++e8hzp6eny+OOPS1lZmZw8eVJiY2Nl0aJF7joll+itbi0tLbJ27VqxWCxy8eJFOXr0qDzxxBMSGxsrra2tynN4U91WrFghISEhUlJSIlarVXncvXtXWdPX9djR0SFTpkyRtLQ0+eOPP+Tw4cMyevRo2bBhgztOiYYAZkTfmBHqMR/U86SMYGNBPcrKyhKDwSB+fn4ybtw4ycrKkqqqKmX+3r178uabb0poaKgEBQXJ/PnzxWq1unHHrnfs2DEB0O2RnZ0tIv/cTvC9996TMWPGiL+/v6SkpEhlZaXdczQ2NsqiRYtkxIgREhwcLMuWLZOWlhY3nI3r9Fa3u3fvSlpamowePVp8fX3FaDTKa6+9ZncLPBHvqpujWgGQXbt2KWv6cz1eunRJ5syZI4GBgRIeHi5r1qyR9vZ2F58NDRXMiL4xI9RjPqjnSRmh+b8NERERERERDRjfY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FERERERE5jY0FeS6PR9PlYunSpu7fZp82bN0Oj0WD37t3u3goR0ZDAfCAamGHu3gCRu2VnZ/c4N3PmTBfuhIiIPAnzgUgdNhbk9fibHCIicoT5QKQOXwpFREREREROY2NBpIJGo8GECRPQ1taG/Px8xMTEICAgACaTCXl5eWhtbXV4XGNjI9atW4fY2FgEBAQgLCwM6enpOHLkSI/fq7GxEZs2bUJ8fDyGDx+O4OBgxMfHY/369bBarQ6POXv2LF544QWEhoZi+PDhmDVrFk6dOjUo505ERD1jPhABECIvBUDUXgIAJCoqSubOnSuBgYEyd+5cyczMlJCQEAEgKSkp0tHRYXfM1atXxWQyKcdmZWXJs88+K1qtVgDI9u3bu32f8+fPS2RkpAAQvV4v8+fPl/nz50tcXJwAkIKCAmVtfn6+AJCcnBwJCgqS+Ph4ycrKksTERAEgAQEBcvbs2QHViIjIGzEfiAaGjQV5rYEGBwCJjIyU6upqZby+vl6mTJkiAOTjjz+2O2bu3LkCQF5++WW5f/++Mn7ixAkJCgoSrVYr5eXlynh7e7tMmjRJAMiqVavsjhERqaiokKqqKuXrruAAIJ9++qnd2lWrVgkAWbJkiarzJCLyZswHooFhY0Feq+t/tr09Hv7Nz8PH7Ny5s9vzHTp0SABITEyMMlZdXS0AZMSIEdLY2NjtmNWrVwsAWb58uTL2/fffCwCJi4vr9tstR7qCY8aMGd3mGhoaBIAYjcY+n4eIiP7BfCAaGN4Virxeb7cTjIqKcji+cOHCbmPp6ekIDQ1FdXU1rFYrDAYDTp48qcyFhYV1O2bJkiXYvn07Tpw4oYwdPXoUALB8+XJotdp+n0daWlq3sVGjRiEsLKzH19wSEVHPmA9E6rCxIK+n9naCoaGhGDlypMM5o9GIGzduoLa2FgaDAbW1tQCACRMmOFzfNX7t2jVl7MqVKwCAmJgYVfuKjIx0OD5y5Eg0NTWpei4iImI+EKnFu0IRuZFGoxm05/Lx4eVMRDRUMB/ov4g/aUQq3bhxAy0tLQ7nampqAABjx461++/ly5cdrr906RIAYNy4ccrY+PHjAQDV1dWDsl8iInIN5gN5OzYWRAOwb9++bmNHjhxBU1MTTCYTDAYDAGDmzJkAgMOHD6O5ubnbMd9++y0AIDk5WRlLTU0FAHz11Vfo7Owc7K0TEdEjxHwgb8bGgmgAtmzZovw2CQAaGhqwbt06AEBOTo4ybjKZkJGRgZaWFqxcuRLt7e3KnMViwY4dO6DVau2OyczMxMSJE1FRUYH169fbHQMA586dw99///2IzoyIiJzBfCBvxjdvk9dbunRpj3NRUVF4//33u40lJCQgLi4OKSkp8PX1RXFxMZqbm/HMM8/grbfeslv/5ZdfIjk5Gd988w1KS0thNptx/fp1lJSU4MGDB9i2bRumTp2qrB82bBh++OEHPPfcc9i2bRv27NkDs9kMEcGFCxdQUVGBgoICmEymwSwDERH9C/OBSCV33++WyF3Qj/uUJyYmdjvGaDRKa2urbNy4USZMmCB+fn5iNBpl06ZNcvfuXYffq6GhQdasWSMxMTHi5+cnOp1O0tLS5Jdffulxf3V1dbJ27VqZOHGiBAQESEhIiMTHx8s777wjVqtVWdd1n/Jdu3Y5fB6j0aj6g56IiLwZ84FoYDQiIq5rY4j+2zQaDYxGo92fuYmIiJgPRHyPBRERERERDQI2FkRERERE5DQ2FkRERERE5DS+x4KIiIiIiJzGv1gQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT2FgQEREREZHT/gfxzcrXO8SRIAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from numpy import linspace\n",
    "\n",
    "\n",
    "quantities = [\"loss\", \"accuracy\"]\n",
    "splits = [\"train\", \"valid\"]\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(8, 4))\n",
    "\n",
    "for axis, quant in zip(axes.ravel(), quantities):\n",
    "    for split in splits:\n",
    "        key = f\"{quant}_{split}\"\n",
    "        axis.plot(\n",
    "            linspace(1, epochs, epochs),\n",
    "            metrics[key],\n",
    "            \"-o\",\n",
    "            ms=1.5,\n",
    "            label=split.capitalize(),\n",
    "        )\n",
    "    axis.set_ylabel(quant.capitalize(), fontsize=15)\n",
    "\n",
    "for axis in axes.ravel():\n",
    "    axis.legend(fontsize=15)\n",
    "    axis.set_ylim(bottom=0.0, top=1.0)\n",
    "    axis.set_xlim(left=1, right=epochs)\n",
    "    axis.set_xlabel(\"Epoch\", fontsize=15)\n",
    "\n",
    "fig.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus: Run the net on 'new' inputs\n",
    "\n",
    "We have built and trained a net, and evaluated and visualised its performance. However, how do we now utilise it going forward?\n",
    "\n",
    "Here we construct some 'new' input data and use our trained net to infer the species. Whilst this is relatively straightforward there is still some work required to transform the outputs from the net to a meaningful result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw input:\n",
      "tensor([[4.2900e+01, 1.3100e+01, 5.0000e+03, 2.1500e+02, 0.0000e+00],\n",
      "        [3.3600e+01, 1.1300e+01, 2.0000e+03, 2.1100e+02, 1.0000e+00]])\n",
      "\n",
      "Raw output:\n",
      "tensor([[8.2419e-07, 8.8322e-09, 1.0000e+00],\n",
      "        [6.8586e-01, 4.3171e-06, 3.1413e-01]])\n",
      "\n",
      "Predicted species:\n",
      "['Gentoo', 'Adelie']\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from torch import no_grad\n",
    "\n",
    "# Construct a tensor of inputs to run the model over\n",
    "demo_input = tensor(\n",
    "    [\n",
    "        [42.9, 13.1, 5000.0, 215.0, 0.0],\n",
    "        [33.6, 11.3, 2000.0, 211.0, 1.0],\n",
    "    ]\n",
    ")\n",
    "print(f\"Raw input:\\n{demo_input}\\n\")\n",
    "\n",
    "# Place model in eval mode and run over inputs with no_grad\n",
    "model.eval()\n",
    "with no_grad():\n",
    "    demo_output = model(demo_input).softmax(dim=1)\n",
    "\n",
    "# Print the raw output from the net\n",
    "print(f\"Raw output:\\n{demo_output}\\n\")\n",
    "\n",
    "# Transform the raw output back to human-readable format\n",
    "print(f\"Predicted species:\\n{[target_names[val.argmax()] for val in demo_output]}\\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}