01a_machine_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creative machine learning - Machine Learning\n",
    "\n",
    "### Author: Philippe Esling (esling@ircam.fr)\n",
    "\n",
    "In this course we will cover\n",
    "1. A [first definition](#definition) on the concept of machine learning\n",
    "2. An introduction to a simple problem of [linear regression](#regression)\n",
    "4. A detailed implementation of [simple linear regression](#linear)\n",
    "3. An explanation on [model capacity and overfitting](#capacity)\n",
    "4. Two [exercises](#exercises) to explore regression and classification\n",
    "4. An introduction to the [audio datasets](#audio) that we will use"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"definition\"></a>\n",
    "## Defining machine learning\n",
    "\n",
    "In all natural process, there exists complex relations between sets $\\mathcal{X} \\mapsto \\mathcal{Y}$. This can relate some objects with their names, or a cause to a consequence. In most cases, _we do not know the precise relations_ between these sets, all we have is _observations_ such as pairs $(x,y)$, composed of input data $x \\in \\mathcal{X}$, which have a corresponding expected output $y \\in \\mathcal{Y}$. The overarching goal of machine learning is to approximate such _unknown processes_ as a function $\\mathcal{F}_{\\theta}$, which _transforms_ input data $x$ into output data $y$.\n",
    "\n",
    "\n",
    "<center>\n",
    "<img src=\"images/01_machine_learning_basic.png\" align=\"center\"/>\n",
    "</center>\n",
    "\n",
    "Hence, machine learning aims to understand and model the relationship between some (usually complex and high-dimensional) inputs $\\mathbf{x}\\in\\mathcal{X}\\subset\\mathbb{R}^{\\mathcal{X}}$ and outputs $\\mathbf{y}\\in\\mathcal{Y}\\subset\\mathbb{R}^{\\mathcal{Y}}$, given by a set of data examples $\\mathcal{D}=\\left\\{(x_1,y_1),\\cdots,(x_N,y_N)\\right\\}$. This is achieved by defining a parametric model $f_{\\mathbf{\\theta}}\\in\\mathcal{F}$ inside a family of functions $\\mathcal{F}$, which depends on parameters $\\mathbf{\\theta} \\in \\mathbf{\\Theta}$ and that could approximate the underlying relationship. The _learning_ aspect refers to the adjustment of the parameters $\\mathbf{\\theta}$ in order to obtain the best approximation of the given task\n",
    "$$\n",
    "\\begin{equation}\n",
    "f_{\\mathbf{\\theta}}(\\mathbf{x}) = \\bar{\\mathbf{y}}\\approx \\mathbf{y}.\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "Hence, the major elements that we have to define in any machine learning problems are\n",
    "1. **Dataset** : $\\mathcal{D}=\\left\\{(x_1,y_1),\\cdots,(x_N,y_N)\\right\\}$. This dataset has to be representative of the relation $f:\\mathcal{X} \\mapsto \\mathcal{Y}$ that we are looking to model\n",
    "2. **Model** : Our parametric approximation $\\bar{\\mathbf{y}} = f_{\\mathbf{\\theta}}(\\mathbf{x})$, where the choice of family $f_{\\mathbf{\\theta}}\\in\\mathcal{F}$ is critical\n",
    "3. **Loss** : $\\mathcal{L}\\left( \\bar{\\mathbf{y}}, \\mathbf{y} \\mid f_{\\theta}, \\theta \\right)$ allows to measure the amount of errors made by our model\n",
    "4. **Optimization** : Method to find $\\theta^{*}\\in\\Theta$ so that our model minimizes the loss\n",
    "$$\\theta^{*}= \\underset{\\theta}{\\text{argmin }} \\mathcal{L}\\left( \\bar{\\mathbf{y}}, \\mathbf{y} \\mid f_{\\theta}, \\theta \\right)$$\n",
    "\n",
    "\n",
    "To observe this idea in simple setups, we are going to use the `numpy` library and also initialize the homemade course library `cml` and style for future plotting and exercise. We also set the random generator to a fixed point with `rng = np.random.RandomState(1)`, to ensure reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "ImportError",
     "evalue": "This version of TensorFlow Probability requires TensorFlow version >= 2.11; Detected an installation of version 2.10.0. Please upgrade TensorFlow to proceed.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[3], line 4\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mnumpy\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mnp\u001b[39;00m\n\u001b[1;32m      3\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mmatplotlib\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpyplot\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mplt\u001b[39;00m\n\u001b[0;32m----> 4\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mplot\u001b[39;00m \u001b[39mimport\u001b[39;00m initialize_bokeh\n\u001b[1;32m      5\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpanel\u001b[39;00m \u001b[39mimport\u001b[39;00m initialize_panel\n\u001b[1;32m      6\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mjupyterthemes\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mstylefx\u001b[39;00m \u001b[39mimport\u001b[39;00m set_nb_theme\n",
      "File \u001b[0;32m~/Desktop/creative_ml/cml/__init__.py:25\u001b[0m\n\u001b[1;32m     21\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mrandomness\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     22\u001b[0m     Random\n\u001b[1;32m     23\u001b[0m )\n\u001b[1;32m     24\u001b[0m \u001b[39m# Tasks GUI objects\u001b[39;00m\n\u001b[0;32m---> 25\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtasks\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     26\u001b[0m     MLTask,\n\u001b[1;32m     27\u001b[0m     RegressionPolynomial\n\u001b[1;32m     28\u001b[0m )\n\u001b[1;32m     29\u001b[0m \u001b[39m# Panel-related methods\u001b[39;00m\n\u001b[1;32m     30\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpanel\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     31\u001b[0m     initialize_panel\n\u001b[1;32m     32\u001b[0m )\n",
      "File \u001b[0;32m~/Desktop/creative_ml/cml/tasks/__init__.py:34\u001b[0m\n\u001b[1;32m     18\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtasks\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mregression\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     19\u001b[0m     RegressionTask,\n\u001b[1;32m     20\u001b[0m     RegressionLinear,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     24\u001b[0m     RegressionSwissRoll\n\u001b[1;32m     25\u001b[0m )\n\u001b[1;32m     26\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtasks\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mclassification\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     27\u001b[0m     ClassificationTask,\n\u001b[1;32m     28\u001b[0m     ClassificationLinear,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     32\u001b[0m     ClassificationBlobs\n\u001b[1;32m     33\u001b[0m )\n\u001b[0;32m---> 34\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtasks\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mdensity\u001b[39;00m \u001b[39mimport\u001b[39;00m (\n\u001b[1;32m     35\u001b[0m     DensityGaussian,\n\u001b[1;32m     36\u001b[0m     DensityGaussian2D\n\u001b[1;32m     37\u001b[0m )\n\u001b[1;32m     39\u001b[0m __all__ \u001b[39m=\u001b[39m [\n\u001b[1;32m     40\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mMLTask\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m     41\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mRegressionTask\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     54\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mDensityGaussian2D\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m     55\u001b[0m ]\n",
      "File \u001b[0;32m~/Desktop/creative_ml/cml/tasks/density.py:16\u001b[0m\n\u001b[1;32m     14\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mparam\u001b[39;00m\n\u001b[1;32m     15\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mpanel\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mpn\u001b[39;00m\n\u001b[0;32m---> 16\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mdistrax\u001b[39;00m\n\u001b[1;32m     17\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mrandomness\u001b[39;00m \u001b[39mimport\u001b[39;00m Random\n\u001b[1;32m     18\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mcml\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtasks\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mtask\u001b[39;00m \u001b[39mimport\u001b[39;00m MLTask\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/distrax/__init__.py:18\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[39m\"\"\"Distrax: Probability distributions in JAX.\"\"\"\u001b[39;00m\n\u001b[1;32m     17\u001b[0m \u001b[39m# Bijectors.\u001b[39;00m\n\u001b[0;32m---> 18\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mdistrax\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39m_src\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mbijectors\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mbijector\u001b[39;00m \u001b[39mimport\u001b[39;00m Bijector\n\u001b[1;32m     19\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mdistrax\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39m_src\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mbijectors\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mbijector\u001b[39;00m \u001b[39mimport\u001b[39;00m BijectorLike\n\u001b[1;32m     20\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mdistrax\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39m_src\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mbijectors\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mblock\u001b[39;00m \u001b[39mimport\u001b[39;00m Block\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/distrax/_src/bijectors/bijector.py:23\u001b[0m\n\u001b[1;32m     21\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mdistrax\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39m_src\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mutils\u001b[39;00m \u001b[39mimport\u001b[39;00m jittable\n\u001b[1;32m     22\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mjax\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mnumpy\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mjnp\u001b[39;00m\n\u001b[0;32m---> 23\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtensorflow_probability\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39msubstrates\u001b[39;00m \u001b[39mimport\u001b[39;00m jax \u001b[39mas\u001b[39;00m tfp\n\u001b[1;32m     26\u001b[0m tfb \u001b[39m=\u001b[39m tfp\u001b[39m.\u001b[39mbijectors\n\u001b[1;32m     28\u001b[0m Array \u001b[39m=\u001b[39m jnp\u001b[39m.\u001b[39mndarray\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/__init__.py:20\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[39m\"\"\"Tools for probabilistic reasoning in TensorFlow.\"\"\"\u001b[39;00m\n\u001b[1;32m     17\u001b[0m \u001b[39m# Contributors to the `python/` dir should not alter this file; instead update\u001b[39;00m\n\u001b[1;32m     18\u001b[0m \u001b[39m# `python/__init__.py` as necessary.\u001b[39;00m\n\u001b[0;32m---> 20\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtensorflow_probability\u001b[39;00m \u001b[39mimport\u001b[39;00m substrates\n\u001b[1;32m     21\u001b[0m \u001b[39m# from tensorflow_probability.google import staging  # DisableOnExport\u001b[39;00m\n\u001b[1;32m     22\u001b[0m \u001b[39m# from tensorflow_probability.google import tfp_google  # DisableOnExport\u001b[39;00m\n\u001b[1;32m     23\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtensorflow_probability\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpython\u001b[39;00m \u001b[39mimport\u001b[39;00m \u001b[39m*\u001b[39m  \u001b[39m# pylint: disable=wildcard-import\u001b[39;00m\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/substrates/__init__.py:17\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[39m# Copyright 2019 The TensorFlow Probability Authors.\u001b[39;00m\n\u001b[1;32m      2\u001b[0m \u001b[39m#\u001b[39;00m\n\u001b[1;32m      3\u001b[0m \u001b[39m# Licensed under the Apache License, Version 2.0 (the \"License\");\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[39m# limitations under the License.\u001b[39;00m\n\u001b[1;32m     14\u001b[0m \u001b[39m# ============================================================================\u001b[39;00m\n\u001b[1;32m     15\u001b[0m \u001b[39m\"\"\"TensorFlow Probability alternative substrates.\"\"\"\u001b[39;00m\n\u001b[0;32m---> 17\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtensorflow_probability\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpython\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39minternal\u001b[39;00m \u001b[39mimport\u001b[39;00m all_util\n\u001b[1;32m     18\u001b[0m \u001b[39mfrom\u001b[39;00m \u001b[39mtensorflow_probability\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39mpython\u001b[39;00m\u001b[39m.\u001b[39;00m\u001b[39minternal\u001b[39;00m \u001b[39mimport\u001b[39;00m lazy_loader  \u001b[39m# pylint: disable=g-direct-tensorflow-import\u001b[39;00m\n\u001b[1;32m     21\u001b[0m jax \u001b[39m=\u001b[39m lazy_loader\u001b[39m.\u001b[39mLazyLoader(\n\u001b[1;32m     22\u001b[0m     \u001b[39m'\u001b[39m\u001b[39mjax\u001b[39m\u001b[39m'\u001b[39m, \u001b[39mglobals\u001b[39m(),\n\u001b[1;32m     23\u001b[0m     \u001b[39m'\u001b[39m\u001b[39mtensorflow_probability.substrates.jax\u001b[39m\u001b[39m'\u001b[39m)\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:138\u001b[0m\n\u001b[1;32m    135\u001b[0m \u001b[39mif\u001b[39;00m _tf_loaded():\n\u001b[1;32m    136\u001b[0m   \u001b[39m# Non-lazy load of packages that register with tensorflow or keras.\u001b[39;00m\n\u001b[1;32m    137\u001b[0m   \u001b[39mfor\u001b[39;00m pkg_name \u001b[39min\u001b[39;00m _maybe_nonlazy_load:\n\u001b[0;32m--> 138\u001b[0m     \u001b[39mdir\u001b[39;49m(\u001b[39mglobals\u001b[39;49m()[pkg_name])  \u001b[39m# Forces loading the package from its lazy loader.\u001b[39;00m\n\u001b[1;32m    141\u001b[0m all_util\u001b[39m.\u001b[39mremove_undocumented(\u001b[39m__name__\u001b[39m, _lazy_load \u001b[39m+\u001b[39m _maybe_nonlazy_load)\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py:57\u001b[0m, in \u001b[0;36mLazyLoader.__dir__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     56\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39m__dir__\u001b[39m(\u001b[39mself\u001b[39m):\n\u001b[0;32m---> 57\u001b[0m   module \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_load()\n\u001b[1;32m     58\u001b[0m   \u001b[39mreturn\u001b[39;00m \u001b[39mdir\u001b[39m(module)\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/python/internal/lazy_loader.py:37\u001b[0m, in \u001b[0;36mLazyLoader._load\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     35\u001b[0m \u001b[39m\"\"\"Load the module and insert it into the parent's globals.\"\"\"\u001b[39;00m\n\u001b[1;32m     36\u001b[0m \u001b[39mif\u001b[39;00m callable(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_on_first_access):\n\u001b[0;32m---> 37\u001b[0m   \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_on_first_access()\n\u001b[1;32m     38\u001b[0m   \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_on_first_access \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m\n\u001b[1;32m     39\u001b[0m \u001b[39m# Import the target module and insert it into the parent's namespace\u001b[39;00m\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:59\u001b[0m, in \u001b[0;36m_validate_tf_environment\u001b[0;34m(package)\u001b[0m\n\u001b[1;32m     55\u001b[0m \u001b[39m#   required_tensorflow_version = '1.15'  # Needed internally -- DisableOnExport\u001b[39;00m\n\u001b[1;32m     57\u001b[0m   \u001b[39mif\u001b[39;00m (distutils\u001b[39m.\u001b[39mversion\u001b[39m.\u001b[39mLooseVersion(tf\u001b[39m.\u001b[39m__version__) \u001b[39m<\u001b[39m\n\u001b[1;32m     58\u001b[0m       distutils\u001b[39m.\u001b[39mversion\u001b[39m.\u001b[39mLooseVersion(required_tensorflow_version)):\n\u001b[0;32m---> 59\u001b[0m     \u001b[39mraise\u001b[39;00m \u001b[39mImportError\u001b[39;00m(\n\u001b[1;32m     60\u001b[0m         \u001b[39m'\u001b[39m\u001b[39mThis version of TensorFlow Probability requires TensorFlow \u001b[39m\u001b[39m'\u001b[39m\n\u001b[1;32m     61\u001b[0m         \u001b[39m'\u001b[39m\u001b[39mversion >= \u001b[39m\u001b[39m{required}\u001b[39;00m\u001b[39m; Detected an installation of version \u001b[39m\u001b[39m{present}\u001b[39;00m\u001b[39m. \u001b[39m\u001b[39m'\u001b[39m\n\u001b[1;32m     62\u001b[0m         \u001b[39m'\u001b[39m\u001b[39mPlease upgrade TensorFlow to proceed.\u001b[39m\u001b[39m'\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m     63\u001b[0m             required\u001b[39m=\u001b[39mrequired_tensorflow_version,\n\u001b[1;32m     64\u001b[0m             present\u001b[39m=\u001b[39mtf\u001b[39m.\u001b[39m__version__))\n\u001b[1;32m     66\u001b[0m   \u001b[39mif\u001b[39;00m (package \u001b[39m==\u001b[39m \u001b[39m'\u001b[39m\u001b[39mmcmc\u001b[39m\u001b[39m'\u001b[39m \u001b[39mand\u001b[39;00m\n\u001b[1;32m     67\u001b[0m       tf\u001b[39m.\u001b[39mconfig\u001b[39m.\u001b[39mexperimental\u001b[39m.\u001b[39mtensor_float_32_execution_enabled()):\n\u001b[1;32m     68\u001b[0m     \u001b[39m# Must import here, because symbols get pruned to __all__.\u001b[39;00m\n\u001b[1;32m     69\u001b[0m     \u001b[39mimport\u001b[39;00m \u001b[39mwarnings\u001b[39;00m\n",
      "\u001b[0;31mImportError\u001b[0m: This version of TensorFlow Probability requires TensorFlow version >= 2.11; Detected an installation of version 2.10.0. Please upgrade TensorFlow to proceed."
     ]
    }
   ],
   "source": [
    "# Base imports\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cml.plot import initialize_bokeh\n",
    "from cml.panel import initialize_panel\n",
    "from jupyterthemes.stylefx import set_nb_theme\n",
    "from bokeh.io import show\n",
    "initialize_bokeh()\n",
    "initialize_panel()\n",
    "set_nb_theme(\"onedork\")\n",
    "rng = np.random.RandomState(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"regression\"></a>\n",
    "## Simple learning problem\n",
    "\n",
    "Imagine that a certain process somewhere follows the form of a quadratic relationship\n",
    "\n",
    "$$\n",
    " y = a x^{2} + bx + c \n",
    "$$\n",
    "\n",
    "In this case, all the **unknown parameters** are that of a polynomial model, therefore we have $\\theta = \\{a, b, c\\}$. However, this is clearly an ideal (clean) case, whereas in natural observations, there might be some noise in our observations\n",
    "$$\n",
    " y = a x^{2} + bx + c +\\epsilon \\quad \\mbox{with} \\quad \\epsilon \\in [-0.1, 0.1]\n",
    "$$\n",
    "\n",
    "An example of such noisy observations for different parameters is given below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters to our function\n",
    "eps = 0.1\n",
    "a, b, c = 3, 0, 1\n",
    "# Generating the corresponding data\n",
    "x = np.linspace(0, 1, 100)\n",
    "poly = np.poly1d([a, b, c])\n",
    "epsilon = np.random.uniform(-eps, eps, x.shape)\n",
    "y = poly(x) + epsilon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='1061'>\n",
       "  <div class=\"bk-root\" id=\"aa6b87c5-f1a1-460d-9850-f9f8477c32ee\" data-root-id=\"1061\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"fbf97a3f-9fc2-4623-9ed9-d91f3192b3bc\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00101\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1062\",\"type\":\"Spacer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1040\",\"type\":\"Circle\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00102\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1085\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"1050\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1045\"},\"group\":null,\"major_label_policy\":{\"id\":\"1046\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1019\"}},\"id\":\"1018\",\"type\":\"LinearAxis\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"UHERxOIq8T/h0pohuBTwP1lHGkqnNvE/1mWTOyT78D+LIbeHUqvxP3bUDpU7BfE/ON2ODbV57z866S5F8/jvPw2aAZC1F/E/oS7CvIh48D8EvUv7S4PuP1jjcLZg5fA/sd/42F6D8T97X/8lSSTyP14g3ouHQPE/Ff6e61xn8T8ESehuql/yP2N38/CaUvE/THrq3E1n8D+YTvRnFFrzP8WAb9aG3vI/lfLhyXGl8j81Vsgw8UPyP5wkHfX+FfM/hMKDhYno8j9aQ8GRlkr0PwG0GleYZPM/AUfenB2T9D/cFbiFuX/zP3wvjYqjvvI//ZEvcUJY8z86poxAjJn1P7GqD9k/HPY/CBND9nPh9T8o7ebbHL/2P07RW+grGfc/gb6BdZEv9T/Q82rxfrD1PzgPwxtTePY/hSm0Diev9z+EEYg5wVP2P6E+rgEXcfk/HI+lIkRA+T/JxT91vJ/3Pxx2OuZr1fg/1/VZ+g7j+T+QsAeCgaj5P6XBNBw2Hfw/6maYao1S+j8SqKvCwyX9P/4lZ1U9j/s/lTAAYDvz/D/6gsmBmQv+P7TIrcPX5fw/+VxxgOdz/T8DFg9BTCUAQOTn8413mP8/CWlF4vR6AEC+57qlaXMAQGKtZPu87P8/5syV6UBiAUBkXpsb8ukAQEc3cLHkyQBAb3MMde4RAkDOj1axzZYCQGqOwuv6jgJAI00haufzAkBWSmpoq5gDQDwxFgngrgJA1ZRY00HaA0CjPAwQoDQDQH4laXeYeQRAsCseX5/BBEBwy8ZGMdcFQJq2NR8MlAVA8paS7UU8BkDuwR/FjX4FQE5inqPfiAZA0gqdv3UrB0BiXuzligIHQCsUbGb0VQhAL8HutchQB0CWn3Xdk6EIQOUy50iykwlABQQ8go2LCEDnIAyFvFIKQDGbRpgKAwpAzPaxDujUCkBoxUiAXP0KQMJQPsgUPQtAa1bSTqWIDEDpaRXq8rkLQCgsGtWxIQxAOAOIWOXkDEAhR0aT5aYNQPnLMFCQ3w1A1HcCi3xJDkCG4TguSjsPQLD/OrWVpA9AetA5IrlCEEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1051\"},\"selection_policy\":{\"id\":\"1050\"}},\"id\":\"1036\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1019\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1012\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1008\",\"type\":\"DataRange1d\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1039\",\"type\":\"Circle\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1041\"}]},\"id\":\"1054\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1028\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"reload\":false},\"id\":\"1153\",\"type\":\"panel.models.location.Location\"},{\"attributes\":{},\"id\":\"1049\",\"type\":\"AllLabels\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1054\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1053\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1048\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1051\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1057\",\"type\":\"Circle\"},{\"attributes\":{\"children\":[{\"id\":\"1062\"},{\"id\":\"1003\"},{\"id\":\"1085\"}],\"margin\":[0,0,0,0],\"name\":\"Row00105\"},\"id\":\"1061\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"1015\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1010\",\"type\":\"LinearScale\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1073\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Simple quadratic problem\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1004\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1058\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1006\",\"type\":\"DataRange1d\"},{\"attributes\":{\"axis\":{\"id\":\"1014\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1017\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1046\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1026\",\"type\":\"ResetTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1028\"}},\"id\":\"1024\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1041\"}],\"tooltips\":null},\"id\":\"1055\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1022\",\"type\":\"PanTool\"},{\"attributes\":{\"client_comm_id\":\"6a4bf9f2849f49b1abd69be9ea0d6728\",\"comm_id\":\"f07c0f892c7e4e789e5bd65f9dbcb930\",\"plot_id\":\"1061\"},\"id\":\"1119\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"1025\",\"type\":\"SaveTool\"},{\"attributes\":{\"tools\":[{\"id\":\"1022\"},{\"id\":\"1023\"},{\"id\":\"1024\"},{\"id\":\"1025\"},{\"id\":\"1026\"},{\"id\":\"1027\"},{\"id\":\"1055\"},{\"id\":\"1059\"}]},\"id\":\"1029\",\"type\":\"Toolbar\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1014\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1017\"},{\"id\":\"1021\"},{\"id\":\"1053\"}],\"height\":450,\"left\":[{\"id\":\"1018\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1041\"}],\"title\":{\"id\":\"1004\"},\"toolbar\":{\"id\":\"1029\"},\"toolbar_location\":\"left\",\"x_range\":{\"id\":\"1006\"},\"x_scale\":{\"id\":\"1010\"},\"y_range\":{\"id\":\"1008\"},\"y_scale\":{\"id\":\"1012\"}},\"id\":\"1003\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1036\"},\"glyph\":{\"id\":\"1037\"},\"group\":null,\"hover_glyph\":{\"id\":\"1039\"},\"muted_glyph\":{\"id\":\"1040\"},\"nonselection_glyph\":{\"id\":\"1058\"},\"selection_glyph\":{\"id\":\"1057\"},\"view\":{\"id\":\"1042\"}},\"id\":\"1041\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"axis\":{\"id\":\"1018\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1021\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1037\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1027\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1045\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1073\"},\"renderers\":[{\"id\":\"1041\"}]},\"id\":\"1059\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"source\":{\"id\":\"1036\"}},\"id\":\"1042\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1048\"},\"group\":null,\"major_label_policy\":{\"id\":\"1049\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1015\"}},\"id\":\"1014\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1023\",\"type\":\"WheelZoomTool\"}],\"root_ids\":[\"1061\",\"1119\",\"1153\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"fbf97a3f-9fc2-4623-9ed9-d91f3192b3bc\",\"root_ids\":[\"1061\"],\"roots\":{\"1061\":\"aa6b87c5-f1a1-460d-9850-f9f8477c32ee\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 3,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "1061"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from cml.plot import center_plot, scatter\n",
    "plot = (center_plot(scatter(x, y, title=\"Simple quadratic problem\", toolbar_location=\"left\")))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now our main problem is that this function can follow different types of parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = [[5, -5, 4], [-2, 1, 0], [0.1, 1, 1]]\n",
    "# Generating the x axis\n",
    "x = np.linspace(0, 1, 100)\n",
    "plots = []\n",
    "for p in range(len(params)):\n",
    "    poly = np.poly1d(params[p])\n",
    "    epsilon = np.random.uniform(-eps, eps, x.shape)\n",
    "    y = poly(x) + epsilon\n",
    "    plots.append(scatter(x, y, title=\"Problem \"+(str(p+1))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='1364'>\n",
       "  <div class=\"bk-root\" id=\"da5c6933-1463-43b7-81b8-ed069d711cb3\" data-root-id=\"1364\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"be105c76-13ce-44ad-8a04-06ae645ca838\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"1232\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1327\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1269\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1211\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1233\",\"type\":\"AllLabels\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1357\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1207\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1390\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1268\",\"type\":\"ResetTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1212\"}},\"id\":\"1208\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1209\",\"type\":\"SaveTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1328\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1210\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1292\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1350\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1340\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1270\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1282\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1212\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1293\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1351\",\"type\":\"Selection\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1354\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1353\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1234\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 3\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1304\",\"type\":\"Title\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1398\"},\"renderers\":[{\"id\":\"1341\"}]},\"id\":\"1359\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"1235\",\"type\":\"Selection\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1225\"}],\"tooltips\":null},\"id\":\"1239\",\"type\":\"HoverTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1281\",\"type\":\"Circle\"},{\"attributes\":{\"tools\":[{\"id\":\"1322\"},{\"id\":\"1323\"},{\"id\":\"1324\"},{\"id\":\"1325\"},{\"id\":\"1326\"},{\"id\":\"1327\"},{\"id\":\"1355\"},{\"id\":\"1359\"}]},\"id\":\"1329\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1326\",\"type\":\"ResetTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1394\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"tools\":[{\"id\":\"1264\"},{\"id\":\"1265\"},{\"id\":\"1266\"},{\"id\":\"1267\"},{\"id\":\"1268\"},{\"id\":\"1269\"},{\"id\":\"1297\"},{\"id\":\"1301\"}]},\"id\":\"1271\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1319\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1339\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1296\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1295\",\"type\":\"Legend\"},{\"attributes\":{\"axis\":{\"id\":\"1318\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1321\",\"type\":\"Grid\"},{\"attributes\":{\"tools\":[{\"id\":\"1206\"},{\"id\":\"1207\"},{\"id\":\"1208\"},{\"id\":\"1209\"},{\"id\":\"1210\"},{\"id\":\"1211\"},{\"id\":\"1239\"},{\"id\":\"1243\"}]},\"id\":\"1213\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1323\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1336\"},\"glyph\":{\"id\":\"1337\"},\"group\":null,\"hover_glyph\":{\"id\":\"1339\"},\"muted_glyph\":{\"id\":\"1340\"},\"nonselection_glyph\":{\"id\":\"1358\"},\"selection_glyph\":{\"id\":\"1357\"},\"view\":{\"id\":\"1342\"}},\"id\":\"1341\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1341\"}],\"tooltips\":null},\"id\":\"1355\",\"type\":\"HoverTool\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1278\"},\"glyph\":{\"id\":\"1279\"},\"group\":null,\"hover_glyph\":{\"id\":\"1281\"},\"muted_glyph\":{\"id\":\"1282\"},\"nonselection_glyph\":{\"id\":\"1300\"},\"selection_glyph\":{\"id\":\"1299\"},\"view\":{\"id\":\"1284\"}},\"id\":\"1283\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"1336\"}},\"id\":\"1342\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"1278\"}},\"id\":\"1284\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1287\"},\"group\":null,\"major_label_policy\":{\"id\":\"1288\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1261\"}},\"id\":\"1260\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1224\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1223\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1257\",\"type\":\"BasicTicker\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1225\"}]},\"id\":\"1238\",\"type\":\"LegendItem\"},{\"attributes\":{\"axis\":{\"id\":\"1260\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1263\",\"type\":\"Grid\"},{\"attributes\":{\"overlay\":{\"id\":\"1270\"}},\"id\":\"1266\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1261\",\"type\":\"BasicTicker\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1341\"}]},\"id\":\"1354\",\"type\":\"LegendItem\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1283\"}]},\"id\":\"1296\",\"type\":\"LegendItem\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1220\"},\"glyph\":{\"id\":\"1221\"},\"group\":null,\"hover_glyph\":{\"id\":\"1223\"},\"muted_glyph\":{\"id\":\"1224\"},\"nonselection_glyph\":{\"id\":\"1242\"},\"selection_glyph\":{\"id\":\"1241\"},\"view\":{\"id\":\"1226\"}},\"id\":\"1225\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 2\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1246\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1346\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1288\",\"type\":\"AllLabels\"},{\"attributes\":{\"source\":{\"id\":\"1220\"}},\"id\":\"1226\",\"type\":\"CDSView\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1390\"},\"renderers\":[{\"id\":\"1225\"}]},\"id\":\"1243\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1290\"},\"group\":null,\"major_label_policy\":{\"id\":\"1291\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1257\"}},\"id\":\"1256\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1241\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1348\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1290\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1398\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1229\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00110\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1432\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"1230\",\"type\":\"AllLabels\"},{\"attributes\":{\"children\":[{\"id\":\"1365\"},{\"id\":\"1363\"},{\"id\":\"1432\"}],\"margin\":[0,0,0,0],\"name\":\"Row00113\"},\"id\":\"1364\",\"type\":\"Row\"},{\"attributes\":{\"client_comm_id\":\"a526b2edb74341febe08698c40630260\",\"comm_id\":\"a95a4568f97b4e969fc910e479f7a0c6\",\"plot_id\":\"1364\"},\"id\":\"1532\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1198\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1201\"},{\"id\":\"1205\"},{\"id\":\"1237\"}],\"height\":450,\"left\":[{\"id\":\"1202\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1225\"}],\"title\":{\"id\":\"1188\"},\"toolbar\":{\"id\":\"1213\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1190\"},\"x_scale\":{\"id\":\"1194\"},\"y_range\":{\"id\":\"1192\"},\"y_scale\":{\"id\":\"1196\"}},\"id\":\"1187\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1221\",\"type\":\"Circle\"},{\"attributes\":{\"axis\":{\"id\":\"1202\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1205\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1349\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1291\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1190\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1345\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"K2hyJiqxD0AOivO+TdYPQIztcUVVKg9ASCBssXcwD0DBnypCX0gOQI6IOFEIaA1AMMxH9C0zDkCs9FmFx20NQPzpj+xHzg1ACPgn4jsADEA5vvYTa3cMQA/pxoi1JQxA4bZ2FNrwC0DXwO1/aRYMQHCb4CiZ0wtAVqrBVoqVCkBWw1hUsyUKQEAp660v4QpAVgxqmaSxCUBX7MRQHAIJQIA9wjoHYQlA6D2QFQzqCEDvn5kMDt8IQC/zWaaLxghAVBHWPkcRCUAzlPG4OwQJQCiwA3zhLwhAB8Dta3WOB0Dy0cOqv5UIQORtDpIwXQhADKJo6XDDB0BMrjh6ayQIQAEw1XVHNwdAzxKNdlfnBkCS2hogvEoHQFD+lIQLbgZAX9SLath3B0CIlTPiR/EFQHl/IFuuPgdAV1SjEnedBkBnpWdHzQcHQGA4K//OZAZAxypZQRe+BUAwP9w9aAEGQGDsciPH7wVAnQn5rn3ABUCO0XhnbFQGQIlUsTaPvgVA7ZRwcYB+BUBdT+d5uKIGQLKtVOJgZQVARScORrdGBkD+PZcV+J0GQDcD2/lUegVAR1OWbGprBUAm2vnGz3wGQP1coKQebwVAMpAoCchVBkCCgADoB9UGQG4+o+d3YwZAJE7m8SsIB0Aj9eHZBxcHQI1J1kg6fwZAwMUCf6WzBkD6OT/U38QGQF23yOnkWQdAVh8TdZerBkBV6NK30u8HQHWWFWwY+AZAGhkYdfaeB0DypfHr2R4HQDZyW+dKJQdAPVKxuBOQCEAlan4zdXoIQE1i+bodQghAbiS2aoJvCUBWXnvBx8UIQOCCRy18zAlAvxNUV/w0CUBmqovjJh0KQF/3Rz3lWQlAjAx5BOpHCUAtQ1mzIMYJQDednVCEfgpAcfLlvYeHCkBHCJLYTuMKQIZlda8gvwtAv8yjN/jjC0A1oJ5olrYMQHR644xK7gxAgm54mBpXDEAsQwYfQ7MMQNtwIzZCeg1AY0q+lSwgDUBw4dmnZLMNQPIRap+XLQ5A/L4Aqzp9DkD0/QiLjAwPQKzTyn2wyw9AngGAT18VEEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1235\"},\"selection_policy\":{\"id\":\"1234\"}},\"id\":\"1220\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1287\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1192\",\"type\":\"DataRange1d\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00109\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1365\",\"type\":\"Spacer\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1238\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1237\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1203\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1279\",\"type\":\"Circle\"},{\"attributes\":{\"children\":[{\"id\":\"1187\"},{\"id\":\"1245\"},{\"id\":\"1303\"}]},\"id\":\"1362\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"1325\",\"type\":\"SaveTool\"},{\"attributes\":{\"style\":{\"font-size\":\"250%\"},\"text\":\"Observing different problems\"},\"id\":\"1361\",\"type\":\"Div\"},{\"attributes\":{},\"id\":\"1196\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1265\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"1322\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1199\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1299\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1267\",\"type\":\"SaveTool\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1229\"},\"group\":null,\"major_label_policy\":{\"id\":\"1230\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1203\"}},\"id\":\"1202\",\"type\":\"LinearAxis\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1394\"},\"renderers\":[{\"id\":\"1283\"}]},\"id\":\"1301\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1337\",\"type\":\"Circle\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1232\"},\"group\":null,\"major_label_policy\":{\"id\":\"1233\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1199\"}},\"id\":\"1198\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1283\"}],\"tooltips\":null},\"id\":\"1297\",\"type\":\"HoverTool\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1256\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1259\"},{\"id\":\"1263\"},{\"id\":\"1295\"}],\"height\":450,\"left\":[{\"id\":\"1260\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1283\"}],\"title\":{\"id\":\"1246\"},\"toolbar\":{\"id\":\"1271\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1248\"},\"x_scale\":{\"id\":\"1252\"},\"y_range\":{\"id\":\"1250\"},\"y_scale\":{\"id\":\"1254\"}},\"id\":\"1245\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1314\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1317\"},{\"id\":\"1321\"},{\"id\":\"1353\"}],\"height\":450,\"left\":[{\"id\":\"1318\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1341\"}],\"title\":{\"id\":\"1304\"},\"toolbar\":{\"id\":\"1329\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1306\"},\"x_scale\":{\"id\":\"1310\"},\"y_range\":{\"id\":\"1308\"},\"y_scale\":{\"id\":\"1312\"}},\"id\":\"1303\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"Uq+ZGgiA8D/Bvgvfk4/tP09k8iwcwu4/26Zr7/MZ8D/dUHR9fDLxP4mntSO5avA/dOKRJkFB7z8tLieiY+nxP1FF4bBPifE/eTeQ8NVY8T9nP2qgaHnxP14xRsr6EfI/9JWej8a38j+Ug5GjfhLxP+WJWS72xfM/or9Msezs8D/84HwgAavxP4fjcBd62fE/WR+J4rx38j9DpParkOnxP7jUh1gVWvI/Pox4aoSI9D+SLGL72PTyP39th3Kqx/M/qFoNK2YL9T+LmDkeivLyP+rA2giXwfQ/+EunYmLZ8z8S/EPPZn31PyB5rwDXavQ/M/+IyJYw9T/r1/R+Y4z2P4yv6AXtVvQ/rojCUS5R9T95QtzYeif2P/mXZpvtsPU/TOpWx1uz9T/hDSK+4Or2P5403MjakvY/5bKO0Ddo9T//dVNHelD4P0bZxXCd//Y/yyALb1NT+D+Cd+NzbQb4P+olDaH1s/c/4k8Hsu5F+D8iSp1RTzr2P16O5Rotnvc/1Lk7ryT09j+OxPuJym/4P/iRnkMgIvc/GNps+Mo39z/D0VjfVHz5P8mEeoBGLvg/L5i9MDP6+D/qVAzTu776P6cbxscuSPk/fiZ2cRBe+D+zouFb5g/7P+O6aUOJoPk/7h3wB0Bp+z8UrdW8LDH5P51i8ZLdMvo/Ytf6jXHF+z89uEnFdXD8P95ClbJ1Ofw/ceEWez8E+z9sgrxkj9X8P3b9T4AcuPw/Jca7g8r2/D8m6aJucDL9P1hwN2KNFPs//gF+L0LQ/T9SHroN6zz9P1/femxxGvw/QSsv8n39/D9PmObd12f+P+33XQ/Lt/4/8Cl02/vm/T/RDvRFJNv8P+nxl6fAefw/o4iBYECc/T/1+IwVEYL/P0NqqOkffP4/o0pLSbPy/j/Eg8MftToAQA0A76R0SABAyxXKn+uC/z/OLVPHkgEAQI9gCJvTJQBAfHruMGFd/z81TuNwTlwAQEFZvgVT3ABAmC7OZ0/N/z9d5IVBFQ//P1+P1o5THQBATlishJUHAEA0GpN1NfkAQLBDe8fAsgBAmwATXRo3AUA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1351\"},\"selection_policy\":{\"id\":\"1350\"}},\"id\":\"1336\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"gJc3O2ALnz9iSqExszG1v10pqpKd/aK/1wfl8GpAlr+cOfh5ojmIP7D2068q9b4/AK1L1qwgKj/pjJtNQRioP7wg0mWblIc/tSHfFJkcwT/Qg3SqInmIP8LgbZZ5Tbk/Y2JDvMhAtT90L/UGt/2uP3tYnRvYT60/L+3U5fMosz+0eLTWVBC2P2zU/gTmA6k/LMsqQJENqz/DgLuW1tLCP4aaRuJmL8g/Ef6O+H/Wvz/+TCTLTNvFPwP2dBXKB8E/E1IvV9Knyj+wWKMsP9amP50K4Zz/orA/vFHTUpJTpT+XFqvT8s+9P1haeT1pRKQ/vuQ9JZZhyT+I8nFRZ7y5P3QQ0c127Jw/fLetnW4Olj/mw2L14n+5P9AB5uWlqnc/bbIWwZfYsj8eRSa8aIO/P9wQS02+Np8/ADho+2uaej/wRK4SRGqzPxq2wbbrYrk/gqs/+qiBwj9qj60TBYu9P2S1Dmozgo0/AAf1MDIIVb+vBvMJWSK2P49bycbw774/aI58lnthuD+ZcMnSyouGP6uqc9o3Ori/3CZj2ky+t7/Ecrk8tqm7vykHt5DtWqI/+AzohQ8rqD+Nf+2rsXCmv1SI9OuSrK+/EOJGHWPTw79fM+ZvTOW7v6gjy5BuqLi/e9980no0wL+YT6L5qsXKvzUBokPQ08O/LDUsb44cxL/mMqUFyDTAv/JtXqxf28G/k1FXtoTlxb+cKo/9YoTEv3A1G3YEddC/uRAFOijQ078mDV8f+sLJv67Ag/QolNi/5w3z573A0L/JuSctB5XVv1gEgrHadta/oOZtvi5c07+WxSMJIUjVv9AFhdZ23Nu/es7kqrdj2r/t6Dqc3Lzcv6GiQP6+beC/vAh5OIzQ4b/6urIOuavevxkTxKmF2d+/PPXm8MQd4790FPNED2vjv6EkcJ9NYeS/VTXm4NPJ4r+MKAd1grjnvwshZYcwJee/37hqMjFS579MI82f0vXov5vnvr2wYeu/fcOmenx76r9N7IVgr8zov4X+J/1kYem/3FdMfXgV8L+ZMptfY33uv1+9pKNCde+/SGwMNE1X8b8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1293\"},\"selection_policy\":{\"id\":\"1292\"}},\"id\":\"1278\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1300\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1358\",\"type\":\"Circle\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 1\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1188\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1242\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1306\",\"type\":\"DataRange1d\"},{\"attributes\":{\"children\":[{\"id\":\"1361\"},{\"id\":\"1362\"}]},\"id\":\"1363\",\"type\":\"Column\"},{\"attributes\":{},\"id\":\"1248\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1308\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1194\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1250\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1310\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1252\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1264\",\"type\":\"PanTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1328\"}},\"id\":\"1324\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"axis\":{\"id\":\"1198\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1201\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1254\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1314\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1317\",\"type\":\"Grid\"},{\"attributes\":{\"axis\":{\"id\":\"1256\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1259\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1312\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1315\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1206\",\"type\":\"PanTool\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1348\"},\"group\":null,\"major_label_policy\":{\"id\":\"1349\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1315\"}},\"id\":\"1314\",\"type\":\"LinearAxis\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1345\"},\"group\":null,\"major_label_policy\":{\"id\":\"1346\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1319\"}},\"id\":\"1318\",\"type\":\"LinearAxis\"}],\"root_ids\":[\"1364\",\"1532\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"be105c76-13ce-44ad-8a04-06ae645ca838\",\"root_ids\":[\"1364\"],\"roots\":{\"1364\":\"da5c6933-1463-43b7-81b8-ed069d711cb3\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Column)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "1364"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bokeh.models import Div\n",
    "from bokeh.layouts import column, row\n",
    "plot = center_plot(column(Div(text = \"Observing different problems\", style={'font-size': '250%'}), row(*plots)))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In real-life settings, this function can also have different levels of noise, as exemplified in the following code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = [3, 0, 1]\n",
    "noise_levels = [0.1, 1.0, 8.0]\n",
    "# Generating the x axis\n",
    "x = np.linspace(0, 1, 100)\n",
    "plots = []\n",
    "for p in range(len(noise_levels)):\n",
    "    poly = np.poly1d(params)\n",
    "    epsilon = np.random.uniform(-noise_levels[p],noise_levels[p],x.shape)\n",
    "    y = poly(x) + epsilon\n",
    "    plots.append(scatter(x, y, title=\"Problem \"+(str(p+1))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='1875'>\n",
       "  <div class=\"bk-root\" id=\"34d0416b-f1dc-4d12-b6b1-d3d4b81e0729\" data-root-id=\"1875\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"3d8f8fe7-c0bd-44da-b9f1-8b9466bbc852\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"children\":[{\"id\":\"1876\"},{\"id\":\"1874\"},{\"id\":\"1943\"}],\"margin\":[0,0,0,0],\"name\":\"Row00120\"},\"id\":\"1875\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"1861\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1793\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1837\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1862\",\"type\":\"Selection\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1807\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1806\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1714\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1792\",\"type\":\"Circle\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1789\"},\"glyph\":{\"id\":\"1790\"},\"group\":null,\"hover_glyph\":{\"id\":\"1792\"},\"muted_glyph\":{\"id\":\"1793\"},\"nonselection_glyph\":{\"id\":\"1811\"},\"selection_glyph\":{\"id\":\"1810\"},\"view\":{\"id\":\"1795\"}},\"id\":\"1794\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1839\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1734\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1723\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1850\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1746\",\"type\":\"Selection\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1905\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1865\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1864\",\"type\":\"Legend\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1735\",\"type\":\"Circle\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1794\"}]},\"id\":\"1807\",\"type\":\"LegendItem\"},{\"attributes\":{\"overlay\":{\"id\":\"1723\"}},\"id\":\"1719\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1731\"},\"glyph\":{\"id\":\"1732\"},\"group\":null,\"hover_glyph\":{\"id\":\"1734\"},\"muted_glyph\":{\"id\":\"1735\"},\"nonselection_glyph\":{\"id\":\"1753\"},\"selection_glyph\":{\"id\":\"1752\"},\"view\":{\"id\":\"1737\"}},\"id\":\"1736\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1851\",\"type\":\"Circle\"},{\"attributes\":{\"tools\":[{\"id\":\"1717\"},{\"id\":\"1718\"},{\"id\":\"1719\"},{\"id\":\"1720\"},{\"id\":\"1721\"},{\"id\":\"1722\"},{\"id\":\"1750\"},{\"id\":\"1754\"}]},\"id\":\"1724\",\"type\":\"Toolbar\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1749\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"1748\",\"type\":\"Legend\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1847\"},\"glyph\":{\"id\":\"1848\"},\"group\":null,\"hover_glyph\":{\"id\":\"1850\"},\"muted_glyph\":{\"id\":\"1851\"},\"nonselection_glyph\":{\"id\":\"1869\"},\"selection_glyph\":{\"id\":\"1868\"},\"view\":{\"id\":\"1853\"}},\"id\":\"1852\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"1847\"}},\"id\":\"1853\",\"type\":\"CDSView\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1790\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1722\",\"type\":\"HelpTool\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1852\"}]},\"id\":\"1865\",\"type\":\"LegendItem\"},{\"attributes\":{\"children\":[{\"id\":\"1872\"},{\"id\":\"1873\"}]},\"id\":\"1874\",\"type\":\"Column\"},{\"attributes\":{},\"id\":\"1745\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1836\",\"type\":\"SaveTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1909\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1721\",\"type\":\"ResetTool\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1709\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1712\"},{\"id\":\"1716\"},{\"id\":\"1748\"}],\"height\":450,\"left\":[{\"id\":\"1713\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1736\"}],\"title\":{\"id\":\"1699\"},\"toolbar\":{\"id\":\"1724\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1701\"},\"x_scale\":{\"id\":\"1705\"},\"y_range\":{\"id\":\"1703\"},\"y_scale\":{\"id\":\"1707\"}},\"id\":\"1698\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1860\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1743\"},\"group\":null,\"major_label_policy\":{\"id\":\"1744\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1710\"}},\"id\":\"1709\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1717\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1720\",\"type\":\"SaveTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"w+XZ2hBR8T9SSCAN0eDwP1SDhC5zPe4/aF97nETQ7j9GFMGxHCbxP/nUm45WEe8/VvQxs9zd8D9rFNPEbo3xP9i5cyCCM/E/nPx0BaEo7j86lk3edILwPzEvWwXUKPI/pqs9UMXN7j9f4SN99tPwP6AW8A6HCu8/bjSdO7iw8T8x7eFIV63xPwTpKGzbEfA/s9ALcqKW8T8aNkcNBPDxP95qrNCOM/E/sxP/MKsN8j8mr2MoYiHzP/4JgiDjn/M/gqxWsVEt8j+WkXJEHkDyP6KgTi2zifQ/ZcRdrkys9D8u3vr7y5z0P/ipbUuwX/M/VqOEOv4B8z+VEM9TK+j0P3tNyH56gPM/OhBnDRrT8z+sM3FElAb2P45YavCnifU/27+79xQT9T/xklp24Z73P+y2ilrVz/Y/vB7Fa6zP+D/iEJ+sDxD4P2qS5ub4HPk/vQYmZORP+T/IsTfsu5v6P9xP4b2ScPk/fvsDz0T5+j+IJ9nDkur5P+xF1f/NsPk/xPMwoJcI+z/MXte3exb7P1ypTDigyP0/l2RaPIF1/D/2GNms6VD+P3RupdE7H/8/Q+jczifi/j9JEDxgsJ/9P2YHzFzWVP4/NqMP0skMAEBoJjE8O8P/Pynlqyk2PwFAE4ieZHMyAUAzdg7W92oAQPkuG/l90QFACOIqN0/tAEBdIESJG7ACQHKOzBWTlAJANrn7CYclAkDDrCI2SuwCQKgDEFFNjQJAAqkF/O0aBEA8XqTZn0cDQJxksYrx3QNAuH/ZNJEuBUDUcpReYokFQFM3F6algQVAzDUXVmRZBkCWYGCPiYYGQK63GgaMLAZAAjkZB3QMB0BCTeqA2AMHQMRJhmtq5AZAPUblJlxoCEBNr/QnnCkIQMp3H6nuMwlA1K6eps7OCEBlhg1MGUwJQD+K6XQNPQpA/PGkWCHYCUAOyIRgVrEKQI/5lC37zwpAinPODtBLDEDEegB6BWYMQMIPsMzzbgxA4mdcJUPpDEDOH9VF8EkNQMIInyXXPA5A7xqle35KDkAJ01BWZP0OQFZVophXKg9As3aYdcYKEEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1746\"},\"selection_policy\":{\"id\":\"1745\"}},\"id\":\"1731\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1803\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1781\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1710\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1744\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1707\",\"type\":\"LinearScale\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00116\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1876\",\"type\":\"Spacer\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 1\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1699\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1798\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis\":{\"id\":\"1709\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1712\",\"type\":\"Grid\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"IDXr4QEL+T+IOvB7L3z5P2dNzsDy6P4/EzRPIoEu+D+uNmR/IfX2P0z/amGSPvQ/CnZfVxRg+z8MAc9ls8/xP/WRYZt80/0/SH4pTkij1D/FL0D+mY/zP5yAy/HWwOw/czO4AP8d9j9chKS5U2jrP3j3zk9Jb+A/ur/N42dH8z+ui3YZEHboPzvSIfkShfo/OWLCEkLN8z8EEWXuek7jP2xott9SEuU/3Ky0JIup1T8wgqz5133TP3B7mk2MVs8/ApcDIIma/j9nOM01XRf1Py5quAAqUfQ/5gip8GQq+j87XktCRIH+P4rU79NuX+E/MzsbfHQs+j8e3at/aBwBQJlhQlRvi/c/LvJNfAL6/j+JPX9P09UAQMXlVDqAnv0/sH4YtWyY7z/I/XI8ekLpPy8BNEudUPk/XTBi/PD7AkBRr0hZ4FH6P/xKsmnMnOw/tMIqFsuT/D+4JotyAAz4P6QUNtt4Q+U/DHprh/1H9T9wZFzEMCYCQIKSzj4E6QRAN5G5e3BD/T/2BMyQk5jpP6JuHRRedwBAE7S3bNhK9D98rylm29cAQPTpe29tAQVA7hRAmKP9+D88HV2A9IHvP/U08pbQAQRAKaT5L4Ea/z8sebjRQ0r+P7GI5RKmGPE/vWvc7vSzB0AOYuFQaCkIQKJRsVYkvP4/7bWQjrud+T8fQjtmol4JQMQmDNZ3wwBAZMkvsLVY/z+DbUUiJWkIQNiPY7Dra/c/6vslF4+k/T925lQi3QwDQLlvJ51jtf0/GhV8M0HU/D/vMltSV3MGQOj91XYVRv4/qV1iEAU+BkBnZV1zqREIQCBF7U/bqAxAyBqJsC/2DEDeJazHcqwNQEd386Y7GA5AxLfZftYyDkBgujUDL14FQDti4fWgvwFAdaUtKN9WDkA6LmwdxMQQQHIR+w6OwAdAMAn3KfnDDkC2VWjeesQPQMDRsKvdXw1ABCkThd1ZBEAr2DoMeXwPQBPS/Qy8vg5AoUULOg7hEEDQkJvgrxsSQAJRVpyDMRJAgB373jEjD0CLrUbQeXcTQIgIGGIz3xFA7DUsXChGDEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1804\"},\"selection_policy\":{\"id\":\"1803\"}},\"id\":\"1789\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1718\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1732\",\"type\":\"Circle\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"1736\"}]},\"id\":\"1749\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1752\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1810\",\"type\":\"Circle\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1901\"},\"renderers\":[{\"id\":\"1736\"}]},\"id\":\"1754\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1905\"},\"renderers\":[{\"id\":\"1794\"}]},\"id\":\"1812\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1740\"},\"group\":null,\"major_label_policy\":{\"id\":\"1741\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1714\"}},\"id\":\"1713\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1736\"}],\"tooltips\":null},\"id\":\"1750\",\"type\":\"HoverTool\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1794\"}],\"tooltips\":null},\"id\":\"1808\",\"type\":\"HoverTool\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1767\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1770\"},{\"id\":\"1774\"},{\"id\":\"1806\"}],\"height\":450,\"left\":[{\"id\":\"1771\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1794\"}],\"title\":{\"id\":\"1757\"},\"toolbar\":{\"id\":\"1782\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1759\"},\"x_scale\":{\"id\":\"1763\"},\"y_range\":{\"id\":\"1761\"},\"y_scale\":{\"id\":\"1765\"}},\"id\":\"1756\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00117\",\"sizing_mode\":\"stretch_width\"},\"id\":\"1943\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"1701\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1804\",\"type\":\"Selection\"},{\"attributes\":{\"source\":{\"id\":\"1789\"}},\"id\":\"1795\",\"type\":\"CDSView\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1753\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1811\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1848\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1868\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1759\",\"type\":\"DataRange1d\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"1909\"},\"renderers\":[{\"id\":\"1852\"}]},\"id\":\"1870\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"1859\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1761\",\"type\":\"DataRange1d\"},{\"attributes\":{\"source\":{\"id\":\"1731\"}},\"id\":\"1737\",\"type\":\"CDSView\"},{\"attributes\":{\"axis\":{\"id\":\"1713\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1716\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1763\",\"type\":\"LinearScale\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"1825\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"1828\"},{\"id\":\"1832\"},{\"id\":\"1864\"}],\"height\":450,\"left\":[{\"id\":\"1829\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"1852\"}],\"title\":{\"id\":\"1815\"},\"toolbar\":{\"id\":\"1840\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"1817\"},\"x_scale\":{\"id\":\"1821\"},\"y_range\":{\"id\":\"1819\"},\"y_scale\":{\"id\":\"1823\"}},\"id\":\"1814\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1780\",\"type\":\"HelpTool\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"1852\"}],\"tooltips\":null},\"id\":\"1866\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1817\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1819\",\"type\":\"DataRange1d\"},{\"attributes\":{\"client_comm_id\":\"c5e9b8bf57204aa29914f0fe799ec3be\",\"comm_id\":\"84cd2a9b161341dc8f4643c6dd421a96\",\"plot_id\":\"1875\"},\"id\":\"2043\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"1821\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1768\",\"type\":\"BasicTicker\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 2\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1757\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1838\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1743\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1801\"},\"group\":null,\"major_label_policy\":{\"id\":\"1802\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1768\"}},\"id\":\"1767\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"1869\",\"type\":\"Circle\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1798\"},\"group\":null,\"major_label_policy\":{\"id\":\"1799\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1772\"}},\"id\":\"1771\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1765\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1826\",\"type\":\"BasicTicker\"},{\"attributes\":{\"children\":[{\"id\":\"1698\"},{\"id\":\"1756\"},{\"id\":\"1814\"}]},\"id\":\"1873\",\"type\":\"Row\"},{\"attributes\":{\"axis\":{\"id\":\"1767\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1770\",\"type\":\"Grid\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Problem 3\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"1815\",\"type\":\"Title\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1859\"},\"group\":null,\"major_label_policy\":{\"id\":\"1860\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1826\"}},\"id\":\"1825\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1741\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1857\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1703\",\"type\":\"DataRange1d\"},{\"attributes\":{\"axis\":{\"id\":\"1771\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1774\",\"type\":\"Grid\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"1856\"},\"group\":null,\"major_label_policy\":{\"id\":\"1857\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"1830\"}},\"id\":\"1829\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1823\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1772\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1856\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1705\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1825\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1828\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1802\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis\":{\"id\":\"1829\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"1832\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1776\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1901\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1775\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1830\",\"type\":\"BasicTicker\"},{\"attributes\":{\"overlay\":{\"id\":\"1781\"}},\"id\":\"1777\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"ckWpEsFNIECMGAanSsz4v1RyiufqIx9AHGmSy/HUC8BCKxOBZlkVQGgWbDNmBhhA2LJ54oFqFsDMqmUXAsoDQKwy1b9Lb+Y/kk+wKKFVA8Bc0kK3ACMXQLzTCQ+3ICFAuJg9MXic3T9QAudUdOzhP5CUxcZoyr8/vxsHvrGg8L9XfNHZj4L/v3h/UEPyWfC/X/u4WBTWE8CGwruzokjiP8ByuUV+IBvAtsrUMG9+EcC/woqoIdkTQIQHFbf/+wnAsKjvGiEO7z9k98LKWCEiQNBLiOutNQ1AJNpHiLreDcBtq0nweogTwKbdc5kFJh1ADEI8IyJZF8D+WA041JHrv/tARpQIlRXAomMsdjetHUBeuxL4tR0fQNeu62Nq8vA/6IE8BYOWGkCgvWy4qwsiQEvAIDNs1xZAONnW4cReGEAL13jZ/1QQwDAXAcJeohhAZefbZbOKBUCLO/e/+oYYQKJE1XJLfApASvHh9IxH5j9UaDbLHmgiQGaolKjz/xJAZO4q/IetAsBsO4KoGK3Qv7eVzlfDxP2/8mfLN+/oIEDI5JgSdrUcQOTIrZTc/CBAphzumg5W+7/RzcPDj+AiQFTt0degIiJAeidYPGDBIkAsD2qXBvojQNffjFfPcxPAHnPPt5EYFsBc+sigAl/iv94lruJAuCBAkItSHlpH+r/Qx3KG7XQNwARPRD+3QO2/+HMIkIRjFECEEh2yhJv1v16OHL2duyRAlxWNnobbJEBwSt1mNQfaP+zMhgFLPgzAZKZk/lPpCEAMh4FBW0ggQPL+p1S0MgRAVYqn03vbE8CdyaeJCVseQCjLnb/VLx1AwPS0NE5RC0CsEHZlT5H7PwBHSEfoGK2/AnvOQ4qJEEDVCHjzXdYMwCCbw7SEmCJAmxAe5nEUCUA427WwMMgUQNY5NKvbBRNA+MxSD3610z/qpB1u8kkhQABYD+NcSN0/lIpbp0oeIEAeQxQ4jVQUQED1yPHfZQLAopBB04DDEkCS1/vpkSslQHhVlYDzNdS/8G5s+4L//D+w4AMwwAIFQNjdvhzAqeK/6VDyUlIoJ0A=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"1862\"},\"selection_policy\":{\"id\":\"1861\"}},\"id\":\"1847\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"overlay\":{\"id\":\"1839\"}},\"id\":\"1835\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"tools\":[{\"id\":\"1833\"},{\"id\":\"1834\"},{\"id\":\"1835\"},{\"id\":\"1836\"},{\"id\":\"1837\"},{\"id\":\"1838\"},{\"id\":\"1866\"},{\"id\":\"1870\"}]},\"id\":\"1840\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1778\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1740\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1779\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"1799\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1834\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"1833\",\"type\":\"PanTool\"},{\"attributes\":{\"style\":{\"font-size\":\"250%\"},\"text\":\"Different amounts of noise\"},\"id\":\"1872\",\"type\":\"Div\"},{\"attributes\":{},\"id\":\"1801\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"tools\":[{\"id\":\"1775\"},{\"id\":\"1776\"},{\"id\":\"1777\"},{\"id\":\"1778\"},{\"id\":\"1779\"},{\"id\":\"1780\"},{\"id\":\"1808\"},{\"id\":\"1812\"}]},\"id\":\"1782\",\"type\":\"Toolbar\"}],\"root_ids\":[\"1875\",\"2043\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"3d8f8fe7-c0bd-44da-b9f1-8b9466bbc852\",\"root_ids\":[\"1875\"],\"roots\":{\"1875\":\"34d0416b-f1dc-4d12-b6b1-d3d4b81e0729\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Column)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "1875"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot = center_plot(column(Div(text = \"Different amounts of noise\", style={'font-size': '250%'}), row(*plots)))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To summarize, we will have some observations of a function, and we would like to optimize a function that gets as close as possible to the real function that generated this data. Here, we plot the real function and also _subsample_ our number of observations (having only a few points to find the corresponding function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='2287'>\n",
       "  <div class=\"bk-root\" id=\"30218af2-8fc1-4090-a07b-56a6f1e4796a\" data-root-id=\"2287\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"413a10ed-7b71-46c4-acdb-f24e5a806e60\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"2233\",\"type\":\"HelpTool\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2267\"},\"glyph\":{\"id\":\"2268\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"2270\"},\"nonselection_glyph\":{\"id\":\"2269\"},\"view\":{\"id\":\"2272\"}},\"id\":\"2271\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"2252\",\"type\":\"AllLabels\"},{\"attributes\":{\"source\":{\"id\":\"2267\"}},\"id\":\"2272\",\"type\":\"CDSView\"},{\"attributes\":{\"axis\":{\"id\":\"2220\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2223\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"2254\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"2247\"}]},\"id\":\"2260\",\"type\":\"LegendItem\"},{\"attributes\":{\"source\":{\"id\":\"2242\"}},\"id\":\"2248\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"green\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2270\",\"type\":\"Line\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2242\"},\"glyph\":{\"id\":\"2243\"},\"group\":null,\"hover_glyph\":{\"id\":\"2245\"},\"muted_glyph\":{\"id\":\"2246\"},\"nonselection_glyph\":{\"id\":\"2264\"},\"selection_glyph\":{\"id\":\"2263\"},\"view\":{\"id\":\"2248\"}},\"id\":\"2247\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2245\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2255\",\"type\":\"AllLabels\"},{\"attributes\":{\"label\":{\"value\":\"True function\"},\"renderers\":[{\"id\":\"2271\"}]},\"id\":\"2286\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"2225\",\"type\":\"BasicTicker\"},{\"attributes\":{\"children\":[{\"id\":\"2288\"},{\"id\":\"2209\"},{\"id\":\"2315\"}],\"margin\":[0,0,0,0],\"name\":\"Row00127\"},\"id\":\"2287\",\"type\":\"Row\"},{\"attributes\":{\"axis\":{\"id\":\"2224\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2227\",\"type\":\"Grid\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00124\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2315\",\"type\":\"Spacer\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2254\"},\"group\":null,\"major_label_policy\":{\"id\":\"2255\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2221\"}},\"id\":\"2220\",\"type\":\"LinearAxis\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00123\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2288\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"2218\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"2221\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"2216\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2251\"},\"group\":null,\"major_label_policy\":{\"id\":\"2252\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2225\"}},\"id\":\"2224\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"2214\",\"type\":\"DataRange1d\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2234\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"2212\",\"type\":\"DataRange1d\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"W79SoNavlD8yb2dIzNu5P2WTTTbZZMM/UetXCtT6xT88Q2LezpDIPxPzdobEvM0/ejtDYt7O0D910UUXXXTRP2ApUOtXCtQ/W79SoNav1D9WVVVVVVXVP1HrVwrU+tU/TIFav1Kg1j9GF1100UXXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/HcdxHMdx3D8YXXTRRRfdPxPzdobEvN0/CB988MEH3z998MEHH3zgP3o7Q2LezuA/d4bEvJ0h4T910UUXXXThP3BnSMzbGeI/aEjM2xkS4z9gKVDrVwrkP1gK1PqVAuU/U6DWrxSo5T9ONtlkk03mP0Ri3s6QmOc/OY7jOI7j6D832WSTTTbpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8iMW9nSMzrPx3HcRzHcew/Faj1KwVq7T8T83aGxLztPxA++OCDD+4/Dol5O0Ni7j8L1PqVArXuPwZq/UqBWu8/A7V+pUCt7z8AAAAAAADwPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"y\":{\"__ndarray__\":\"KkoB5v5a8T+xNOWPYA7xPwmKQnIDoPI/baG5rHxu9D8Zs8QozYfxP46U227sGfU/MBs8Q8++9T+WlRexJQDzP9dNo50bXvQ/kBE1XHQF8j9qYC3NpBP0P2V4k3dcTPc/dFeBPtKj+D/cv6Un7QT5P7RKRR121vQ/YRKXhub9+T9syW8J4i33PxUGxgHmqfU/RGyNWNmX+j+QlWUQCQP4P5+7CzjOI/s/KAUaFFfT+D9I6KHiN/n5P7Y0PYr1vfw/vhbQULra/z+cn0WVsHD8P5Keuo3diQBAaK1NrLMrAEB+ZGCKu5QBQK00SvegnQNAjsPKC/QkAkC49pHnb20DQGYgJRd1pQVAtvpubkEhBkCTRmFZMkAGQBIAKxbQjwhA4kxNa115B0DIX7JpgagJQItyx/69PwhAXHAgAA7GCEC2WmYk6H8KQMv/Y4O23gtAKjf9QC1wDUCdCnIvsXMMQFh/oZlsWg5APT6UNMZgDUBIaUJpgZUOQCTkqgs9Sg5AGrglp7txEEAzM4UuDhEPQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"2257\"},\"selection_policy\":{\"id\":\"2256\"}},\"id\":\"2242\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2246\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"2220\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"2223\"},{\"id\":\"2227\"},{\"id\":\"2259\"}],\"height\":450,\"left\":[{\"id\":\"2224\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"2247\"},{\"id\":\"2271\"}],\"title\":{\"id\":\"2210\"},\"toolbar\":{\"id\":\"2235\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"2212\"},\"x_scale\":{\"id\":\"2216\"},\"y_range\":{\"id\":\"2214\"},\"y_scale\":{\"id\":\"2218\"}},\"id\":\"2209\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"AAAAAAAA8D8MJLz1QAHwPy6Q8NYDBfA/aESdo0gL8D+6QMJbDxTwPyKFX/9XH/A/ohF1jiIt8D845gIJbz3wP+YCCW89UPA/q2eHwI1l8D+IFH79X33wP3sJ7SW0l/A/hkbUOYq08D+oyzM54tPwP+GYCyS89fA/Ma5b+hca8T+ZCyS89UDxPxixZGlVavE/rZ4dAjeW8T9b1E6GmsTxPx9S+PV/9fE/+hcaUeco8j/tJbSX0F7yP/d7xsk7l/I/GBpR5yjS8j9QAFTwlw/zP6Auz+SIT/M/B6XCxPuR8z+EYy6Q8NbzPxlqEkdnHvQ/xrhu6V9o9D+JT0N32rT0P2QukPDWA/U/VlVVVVVV9T9fxJKlVan1P397SOHX//U/tnp2CNxY9j8FwhwbYrT2P2tROxlqEvc/6CjSAvRy9z98SOHX/9X3PyiwaJiNO/g/6l9oRJ2j+D/EV+DbLg75P7SX0F5Ce/k/vR85zdfq+T/c7xkn71z6PxIIc2yI0fo/YGhEnaNI+z/FEI65QML7P0EBUMFfPvw/1jmKtAC9/D+AujyTIz79P0CDZ13Iwf0/GpQKE+9H/j8K7SW0l9D+PxKOuUDCW/8/MHfFuG7p/z8y1CSOzjwAQNoQo7UmhgBAi3Hd0r/QAEBJ9tPlmRwBQBKfhu60aQFA6Gv17BC4AUDIXCDhrQcCQLNxB8uLWAJAq6qqqqqqAkCuBwqACv4CQL6IJUurUgNA1y39C42oA0D+9pDCr/8DQDDk4G4TWARAbPXsELixBEC2KrWonQwFQAqEOTbEaAVAagF6uSvGBUDWonYy1CQGQExoL6G9hAZA0FGkBejlBkBeX9VfU0gHQPiQwq//qwdAneZr9ewQCEBPYNEwG3cIQAz+8mGK3ghA07/QiDpHCUCopWqlK7EJQIevwLddHApAc93Sv9CICkBpL6G9hPYKQGylK7F5ZQtAej9ymq/VC0CT/XR5JkcMQLjfM07euQxA6eWuGNctDUAlEObYEKMNQG1e2Y6LGQ5AwNCIOkeRDkAgZ/TbQwoPQIohHHOBhA9AAAAAAAAAEEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"2283\"},\"selection_policy\":{\"id\":\"2282\"}},\"id\":\"2267\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"2256\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":0.6,\"line_color\":\"green\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2268\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"2282\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2243\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2257\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"2283\",\"type\":\"Selection\"},{\"attributes\":{\"client_comm_id\":\"bb97ecb7ff4c4babb587191d8111f182\",\"comm_id\":\"fc3d7566514948f6ba70658a0bd2eba2\",\"plot_id\":\"2287\"},\"id\":\"2355\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"2251\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"tools\":[{\"id\":\"2228\"},{\"id\":\"2229\"},{\"id\":\"2230\"},{\"id\":\"2231\"},{\"id\":\"2232\"},{\"id\":\"2233\"},{\"id\":\"2261\"},{\"id\":\"2265\"}]},\"id\":\"2235\",\"type\":\"Toolbar\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2263\",\"type\":\"Circle\"},{\"attributes\":{\"overlay\":{\"id\":\"2234\"}},\"id\":\"2230\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"2285\"},\"renderers\":[{\"id\":\"2247\"}]},\"id\":\"2265\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"2231\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"2232\",\"type\":\"ResetTool\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"2260\"},{\"id\":\"2286\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"2259\",\"type\":\"Legend\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"green\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2269\",\"type\":\"Line\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2285\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"2247\"}],\"tooltips\":null},\"id\":\"2261\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"2228\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"2229\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2264\",\"type\":\"Circle\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Learning problem with groundtruth\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"2210\",\"type\":\"Title\"}],\"root_ids\":[\"2287\",\"2355\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"413a10ed-7b71-46c4-acdb-f24e5a806e60\",\"root_ids\":[\"2287\"],\"roots\":{\"2287\":\"30218af2-8fc1-4090-a07b-56a6f1e4796a\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "2287"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generating the data and subsampling\n",
    "x_all = np.linspace(0, 1, 100); x_plot = np.linspace(0, 1, 100)\n",
    "rng.shuffle(x_all); x = np.sort(x_all[:50])\n",
    "poly = np.poly1d([3,0,1])\n",
    "# Adding some external noise\n",
    "epsilon = np.random.uniform(-0.2,0.2,x.shape)\n",
    "y = poly(x)+ epsilon\n",
    "p = scatter(x, y, title=\"Learning problem with groundtruth\")\n",
    "p.line(x_plot, poly(x_plot), line_width=6, line_alpha=0.6, color=\"green\", legend_label=r\"True function\")\n",
    "plot = (center_plot(p))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summarizing our observations (interactive)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating the object\n"
     ]
    },
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='2479'>\n",
       "  <div class=\"bk-root\" id=\"edac0c74-f92b-4617-bd0f-6f073080fc56\" data-root-id=\"2479\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"026b811b-902a-4392-98a4-97cdfc2c2f0b\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"children\":[{\"id\":\"2421\"}],\"margin\":[0,0,0,0],\"name\":\"Row00154\"},\"id\":\"2492\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"2440\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"2444\",\"type\":\"ResetTool\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":0.0,\"step\":0.1,\"title\":\"Noise level\",\"value\":0.1},\"id\":\"2486\",\"type\":\"Slider\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2458\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2469\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2457\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2430\",\"type\":\"LinearScale\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"2459\"}]},\"id\":\"2472\",\"type\":\"LegendItem\"},{\"attributes\":{\"children\":[{\"id\":\"2484\"},{\"id\":\"2485\"},{\"id\":\"2486\"},{\"id\":\"2487\"},{\"id\":\"2488\"},{\"id\":\"2489\"},{\"id\":\"2490\"},{\"id\":\"2491\"}],\"margin\":[5,5,5,5],\"min_width\":300,\"name\":\"Problem parameters\",\"width\":300},\"id\":\"2483\",\"type\":\"Column\"},{\"attributes\":{\"source\":{\"id\":\"2454\"}},\"id\":\"2460\",\"type\":\"CDSView\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"2459\"}],\"tooltips\":null},\"id\":\"2473\",\"type\":\"HoverTool\"},{\"attributes\":{\"axis\":{\"id\":\"2436\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2439\",\"type\":\"Grid\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"2515\"},\"target\":{\"id\":\"2515\"}},\"code\":\"\\n    var value = source['disabled'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['readonly'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set readonly on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['readonly'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11502904320,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"2529\",\"type\":\"CustomJS\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2446\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2454\"},\"glyph\":{\"id\":\"2455\"},\"group\":null,\"hover_glyph\":{\"id\":\"2457\"},\"muted_glyph\":{\"id\":\"2458\"},\"nonselection_glyph\":{\"id\":\"2476\"},\"selection_glyph\":{\"id\":\"2475\"},\"view\":{\"id\":\"2460\"}},\"id\":\"2459\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 2\",\"value\":-2.0},\"id\":\"2489\",\"type\":\"Slider\"},{\"attributes\":{},\"id\":\"2433\",\"type\":\"BasicTicker\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} -2.0x^2 + 0.0x^1 + 1.0x^0$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2463\"},\"group\":null,\"major_label_policy\":{\"id\":\"2464\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2437\"}},\"id\":\"2436\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"2426\",\"type\":\"DataRange1d\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"2472\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"2471\",\"type\":\"Legend\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2503\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"margin\":[5,10,5,10],\"name\":\"\",\"text\":\"<b>Problem parameters</b>\"},\"id\":\"2484\",\"type\":\"Div\"},{\"attributes\":{},\"id\":\"2467\",\"type\":\"AllLabels\"},{\"attributes\":{\"children\":[{\"id\":\"2482\"},{\"id\":\"2515\"}],\"margin\":[0,0,0,0],\"name\":\"Column00160\"},\"id\":\"2481\",\"type\":\"Column\"},{\"attributes\":{\"axis\":{\"id\":\"2432\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2435\",\"type\":\"Grid\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 1\",\"value\":0.0},\"id\":\"2490\",\"type\":\"Slider\"},{\"attributes\":{\"children\":[{\"id\":\"2480\"},{\"id\":\"2481\"},{\"id\":\"2516\"}],\"margin\":[0,0,0,0],\"name\":\"Row00162\"},\"id\":\"2479\",\"type\":\"Row\"},{\"attributes\":{\"labels\":[\"Ground truth\"],\"margin\":[5,10,5,10]},\"id\":\"2487\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"2432\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"2435\"},{\"id\":\"2439\"},{\"id\":\"2471\"}],\"height\":450,\"left\":[{\"id\":\"2436\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"2459\"}],\"title\":{\"id\":\"2422\"},\"toolbar\":{\"id\":\"2447\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"2424\"},\"x_scale\":{\"id\":\"2428\"},\"y_range\":{\"id\":\"2426\"},\"y_scale\":{\"id\":\"2430\"}},\"id\":\"2421\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00161\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2516\",\"type\":\"Spacer\"},{\"attributes\":{\"code\":\"def polynomial(\\n        coefficients: jnp.ndarray,\\n        n_observations: int = 100) -> [jnp.ndarray, jnp.ndarray]:\\n    x = jnp.linspace(0, 1, n_observations)\\n    y = jnp.polyval(coefficients, x)\\n    return x, \",\"filename\":\"\",\"height\":150,\"js_property_callbacks\":{\"change:disabled\":[{\"id\":\"2529\"}],\"change:readonly\":[{\"id\":\"2528\"}]},\"language\":\"python\",\"margin\":[5,10,5,10],\"sizing_mode\":\"stretch_both\",\"theme\":\"chaos\"},\"id\":\"2515\",\"type\":\"panel.models.ace.AcePlot\"},{\"attributes\":{},\"id\":\"2468\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"2463\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00131\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2480\",\"type\":\"Spacer\"},{\"attributes\":{\"end\":1000,\"margin\":[5,10,5,10],\"start\":10,\"title\":\"Number observations\",\"value\":100},\"id\":\"2485\",\"type\":\"Slider\"},{\"attributes\":{\"labels\":[\"Error\"],\"margin\":[5,10,5,10]},\"id\":\"2488\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2476\",\"type\":\"Circle\"},{\"attributes\":{\"overlay\":{\"id\":\"2446\"}},\"id\":\"2442\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"2464\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"2441\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"tools\":[{\"id\":\"2440\"},{\"id\":\"2441\"},{\"id\":\"2442\"},{\"id\":\"2443\"},{\"id\":\"2444\"},{\"id\":\"2445\"},{\"id\":\"2473\"},{\"id\":\"2477\"}]},\"id\":\"2447\",\"type\":\"Toolbar\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 0\",\"value\":1.0},\"id\":\"2491\",\"type\":\"Slider\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAALV+JTy1fqU8ED74PLV+JT1i3k49ED54Pd7OkD21fqU9jC66PWLezj05juM9ED74PfN2Bj7ezhA+yiYbPrV+JT6g1i8+jC46PneGRD5i3k4+TjZZPjmOYz4k5m0+ED54Pv1KgT7zdoY+6aKLPt7OkD7U+pU+yiabPr9SoD61fqU+q6qqPqDWrz6WArU+jC66PoFavz53hsQ+bbLJPmLezj5YCtQ+TjbZPkNi3j45juM+L7roPiTm7T4aEvM+ED74PgVq/T79SgE/+OADP/N2Bj/uDAk/6aILP+Q4Dj/ezhA/2WQTP9T6FT/PkBg/yiYbP8W8HT+/UiA/uugiP7V+JT+wFCg/q6oqP6VALT+g1i8/m2wyP5YCNT+RmDc/jC46P4bEPD+BWj8/fPBBP3eGRD9yHEc/bbJJP2dITD9i3k4/XXRRP1gKVD9ToFY/TjZZP0jMWz9DYl4/PvhgPzmOYz80JGY/L7poPylQaz8k5m0/H3xwPxoScz8VqHU/ED54PwrUej8Fan0/AACAPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"ki1uPx9TiT8UHoA/KL2KP9L1ij8FfHg/j02KP4J+iz+OAWc/8EiKP+36aD+Uq24/O3V9P4ckgT9ZE4A/erlcP4O2Xz8H9HI/dix4P2m1WT8cCYA/BBV9PzAeZD+wj1E/e4B4Py12cD+O418/MNdsPxx5Vj+HkTo/PUBRP9jNRz/MKjc/NiZXP98eST/k1ig//Eo4Pz1wPD9dT0c/UUsbPzfZFj9nKB8/ExktP6dtHD8kvx4//RskPwtcGz/0AQ8/aVP6PvePCz8y/wk/7HvwPjHoAj/45ds+kKb8Pu/Zsj5fadI+xBLYPogAxj5I3sM+FNFPPrgFqT5KvlU+1T5CPgRDej6S6zs+l2hLPen1Dz1iq/U9OeAyPSJgWjwm0HQ90AEyOyBkwzsEu/O8sk4avs6D0b0fp/q9GiMnvijpZb4c63K+06y+vucAsb48dt++FNvovhDTDr9JMAy/hZsevwYMCL8dGBC/8rIdvw9GG79AkD2/ofYtv+9MOr9l1kK/4mlxvxL3Vr8PD4C//ciIvw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"2469\"},\"selection_policy\":{\"id\":\"2468\"}},\"id\":\"2454\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"2443\",\"type\":\"SaveTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2455\",\"type\":\"Circle\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"2503\"},\"renderers\":[{\"id\":\"2459\"}]},\"id\":\"2477\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"2515\"},\"target\":{\"id\":\"2515\"}},\"code\":\"\\n    var value = source['readonly'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['disabled'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set disabled on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['disabled'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11502904320,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"2528\",\"type\":\"CustomJS\"},{\"attributes\":{},\"id\":\"2437\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"2428\",\"type\":\"LinearScale\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Polynomial regression problem\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"2422\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2475\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2445\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"2424\",\"type\":\"DataRange1d\"},{\"attributes\":{\"children\":[{\"id\":\"2483\"},{\"id\":\"2492\"}],\"margin\":[0,0,0,0],\"name\":\"Row00157\"},\"id\":\"2482\",\"type\":\"Row\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} x$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2466\"},\"group\":null,\"major_label_policy\":{\"id\":\"2467\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2433\"}},\"id\":\"2432\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"2466\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"client_comm_id\":\"26efee9d8f0a4acc8525fc73b53040d1\",\"comm_id\":\"cce021331434459296bb9d6e9cbca131\",\"plot_id\":\"2479\"},\"id\":\"2552\",\"type\":\"panel.models.comm_manager.CommManager\"}],\"root_ids\":[\"2479\",\"2552\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"026b811b-902a-4392-98a4-97cdfc2c2f0b\",\"root_ids\":[\"2479\"],\"roots\":{\"2479\":\"edac0c74-f92b-4617-bd0f-6f073080fc56\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Column\n",
       "        [0] Row\n",
       "            [0] Param(RegressionPolynomial, name='Problem parameters', widgets={})\n",
       "            [1] ParamMethod(method, _pane=Bokeh)\n",
       "        [1] Ace(height=150, language='python', sizing_mode='stretch_both', theme='chaos', value='def polynomial(\\n        ...)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 9,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "2479"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from cml.tasks import RegressionPolynomial\n",
    "explorer = RegressionPolynomial()\n",
    "explorer.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using learning libraries (`scikit-learn`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get a first grip on what machine learning does, we will rely on the `scikit-learn` library. This contains already coded models and learning procedure, that will allow us to _learn_ the parameters of this unknown function.\n",
    "\n",
    "Here we already know that we want to use a `PolynomialFeatures` model to perfom `LinearRegression` and that this polynomial should be of degree 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.pipeline import make_pipeline\n",
    "# Our data to fit\n",
    "X = x[:, np.newaxis]\n",
    "# Degree of our polynomial\n",
    "degree = 2;\n",
    "# Create our polynomial model for regression\n",
    "model = make_pipeline(PolynomialFeatures(degree), LinearRegression())\n",
    "# Fit the parameters of this model\n",
    "model.fit(X, y);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have trained the model, we can perform _predictions_ from it, meaning that we can infer the output of the function at values that we did not observe originally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model error : 0.10676696330062045\n"
     ]
    },
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='2686'>\n",
       "  <div class=\"bk-root\" id=\"0f8a6fcf-c659-4d6f-8069-c8dc05cbc7e0\" data-root-id=\"2686\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"7357f241-694d-498e-9041-5c4be491f9ff\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"2628\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"source\":{\"id\":\"2641\"}},\"id\":\"2647\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"2666\"}},\"id\":\"2671\",\"type\":\"CDSView\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2684\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2642\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"2619\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"2622\"},{\"id\":\"2626\"},{\"id\":\"2658\"}],\"height\":450,\"left\":[{\"id\":\"2623\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"2646\"},{\"id\":\"2670\"}],\"title\":{\"id\":\"2609\"},\"toolbar\":{\"id\":\"2634\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"2611\"},\"x_scale\":{\"id\":\"2615\"},\"y_range\":{\"id\":\"2613\"},\"y_scale\":{\"id\":\"2617\"}},\"id\":\"2608\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"overlay\":{\"id\":\"2633\"}},\"id\":\"2629\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"2627\",\"type\":\"PanTool\"},{\"attributes\":{\"axis\":{\"id\":\"2619\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2622\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2662\",\"type\":\"Circle\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"2684\"},\"renderers\":[{\"id\":\"2646\"}]},\"id\":\"2664\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2669\",\"type\":\"Line\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABbv1Kg1q+EP1u/UqDWr5Q/CB988MEHnz9bv1Kg1q+kPzJvZ0jM26k/CB988MEHrz9wZ0jM2xmyP1u/UqDWr7Q/RhdddNFFtz8yb2dIzNu5Px3HcRzHcbw/CB988MEHvz96O0Ni3s7AP3BnSMzbGcI/ZZNNNtlkwz9bv1Kg1q/EP1HrVwrU+sU/RhdddNFFxz88Q2LezpDIPzJvZ0jM28k/J5tssskmyz8dx3Ecx3HMPxPzdobEvM0/CB988MEHzz9/pUCtXynQP3o7Q2LeztA/ddFFF1100T9wZ0jM2xnSP2r9SoFav9I/ZZNNNtlk0z9gKVDrVwrUP1u/UqDWr9Q/VlVVVVVV1T9R61cK1PrVP0yBWr9SoNY/RhdddNFF1z9BrV8pUOvXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/J5tssskm2z8iMW9nSMzbPx3HcRzHcdw/GF100UUX3T8T83aGxLzdPw6JeTtDYt4/CB988MEH3z8DtX6lQK3fP3+lQK1fKeA/ffDBBx984D96O0Ni3s7gP3eGxLydIeE/ddFFF1104T9yHMdxHMfhP3BnSMzbGeI/bbLJJpts4j9q/UqBWr/iP2hIzNsZEuM/ZZNNNtlk4z9j3s6QmLfjP2ApUOtXCuQ/XnTRRRdd5D9bv1Kg1q/kP1gK1PqVAuU/VlVVVVVV5T9ToNavFKjlP1HrVwrU+uU/TjbZZJNN5j9MgVq/UqDmP0nM2xkS8+Y/RhdddNFF5z9EYt7OkJjnP0GtXylQ6+c/P/jggw8+6D88Q2LezpDoPzmO4ziO4+g/N9lkk0026T80JObtDInpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8l5u0MiXnrPyIxb2dIzOs/IHzwwQcf7D8dx3Ecx3HsPxsS83aGxOw/GF100UUX7T8VqPUrBWrtPxPzdobEvO0/ED744IMP7j8OiXk7Q2LuPwvU+pUCte4/CB988MEH7z8Gav1KgVrvPwO1fqVAre8/AAAAAAAA8D8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"AAAAAAAA8D8MJLz1QAHwPy6Q8NYDBfA/aESdo0gL8D+6QMJbDxTwPyKFX/9XH/A/ohF1jiIt8D845gIJbz3wP+YCCW89UPA/q2eHwI1l8D+IFH79X33wP3sJ7SW0l/A/hkbUOYq08D+oyzM54tPwP+GYCyS89fA/Ma5b+hca8T+ZCyS89UDxPxixZGlVavE/rZ4dAjeW8T9b1E6GmsTxPx9S+PV/9fE/+hcaUeco8j/tJbSX0F7yP/d7xsk7l/I/GBpR5yjS8j9QAFTwlw/zP6Auz+SIT/M/B6XCxPuR8z+EYy6Q8NbzPxlqEkdnHvQ/xrhu6V9o9D+JT0N32rT0P2QukPDWA/U/VlVVVVVV9T9fxJKlVan1P397SOHX//U/tnp2CNxY9j8FwhwbYrT2P2tROxlqEvc/6CjSAvRy9z98SOHX/9X3PyiwaJiNO/g/6l9oRJ2j+D/EV+DbLg75P7SX0F5Ce/k/vR85zdfq+T/c7xkn71z6PxIIc2yI0fo/YGhEnaNI+z/FEI65QML7P0EBUMFfPvw/1jmKtAC9/D+AujyTIz79P0CDZ13Iwf0/GpQKE+9H/j8K7SW0l9D+PxKOuUDCW/8/MHfFuG7p/z8y1CSOzjwAQNoQo7UmhgBAi3Hd0r/QAEBJ9tPlmRwBQBKfhu60aQFA6Gv17BC4AUDIXCDhrQcCQLNxB8uLWAJAq6qqqqqqAkCuBwqACv4CQL6IJUurUgNA1y39C42oA0D+9pDCr/8DQDDk4G4TWARAbPXsELixBEC2KrWonQwFQAqEOTbEaAVAagF6uSvGBUDWonYy1CQGQExoL6G9hAZA0FGkBejlBkBeX9VfU0gHQPiQwq//qwdAneZr9ewQCEBPYNEwG3cIQAz+8mGK3ghA07/QiDpHCUCopWqlK7EJQIevwLddHApAc93Sv9CICkBpL6G9hPYKQGylK7F5ZQtAej9ymq/VC0CT/XR5JkcMQLjfM07euQxA6eWuGNctDUAlEObYEKMNQG1e2Y6LGQ5AwNCIOkeRDkAgZ/TbQwoPQIohHHOBhA9AAAAAAAAAEEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"2682\"},\"selection_policy\":{\"id\":\"2681\"}},\"id\":\"2666\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.6,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2667\",\"type\":\"Line\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"2646\"}],\"tooltips\":null},\"id\":\"2660\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"2631\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"2650\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2644\",\"type\":\"Circle\"},{\"attributes\":{\"children\":[{\"id\":\"2687\"},{\"id\":\"2608\"},{\"id\":\"2714\"}],\"margin\":[0,0,0,0],\"name\":\"Row00169\"},\"id\":\"2686\",\"type\":\"Row\"},{\"attributes\":{\"tools\":[{\"id\":\"2627\"},{\"id\":\"2628\"},{\"id\":\"2629\"},{\"id\":\"2630\"},{\"id\":\"2631\"},{\"id\":\"2632\"},{\"id\":\"2660\"},{\"id\":\"2664\"}]},\"id\":\"2634\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"2624\",\"type\":\"BasicTicker\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00165\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2687\",\"type\":\"Spacer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2663\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"2651\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2650\"},\"group\":null,\"major_label_policy\":{\"id\":\"2651\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2624\"}},\"id\":\"2623\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"2653\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Training a scikit-learn model\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"2609\",\"type\":\"Title\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"2646\"}]},\"id\":\"2659\",\"type\":\"LegendItem\"},{\"attributes\":{\"axis\":{\"id\":\"2623\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2626\",\"type\":\"Grid\"},{\"attributes\":{\"client_comm_id\":\"0ea1fb1d4f834517829581e0df7cfc37\",\"comm_id\":\"6342e438fc304cea88665b67a33d6a71\",\"plot_id\":\"2686\"},\"id\":\"2754\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"2654\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"2632\",\"type\":\"HelpTool\"},{\"attributes\":{\"label\":{\"value\":\"Trained model\"},\"renderers\":[{\"id\":\"2670\"}]},\"id\":\"2685\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"2630\",\"type\":\"SaveTool\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00166\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2714\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"2681\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"2620\",\"type\":\"BasicTicker\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"2659\"},{\"id\":\"2685\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"2658\",\"type\":\"Legend\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2668\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"2682\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2645\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2633\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2641\"},\"glyph\":{\"id\":\"2642\"},\"group\":null,\"hover_glyph\":{\"id\":\"2644\"},\"muted_glyph\":{\"id\":\"2645\"},\"nonselection_glyph\":{\"id\":\"2663\"},\"selection_glyph\":{\"id\":\"2662\"},\"view\":{\"id\":\"2647\"}},\"id\":\"2646\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"2611\",\"type\":\"DataRange1d\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2666\"},\"glyph\":{\"id\":\"2667\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"2669\"},\"nonselection_glyph\":{\"id\":\"2668\"},\"view\":{\"id\":\"2671\"}},\"id\":\"2670\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2653\"},\"group\":null,\"major_label_policy\":{\"id\":\"2654\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2620\"}},\"id\":\"2619\",\"type\":\"LinearAxis\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"W79SoNavlD8yb2dIzNu5P2WTTTbZZMM/UetXCtT6xT88Q2LezpDIPxPzdobEvM0/ejtDYt7O0D910UUXXXTRP2ApUOtXCtQ/W79SoNav1D9WVVVVVVXVP1HrVwrU+tU/TIFav1Kg1j9GF1100UXXPzxDYt7OkNg/N9lkk0022T8yb2dIzNvZPy0Fav1Kgdo/HcdxHMdx3D8YXXTRRRfdPxPzdobEvN0/CB988MEH3z998MEHH3zgP3o7Q2LezuA/d4bEvJ0h4T910UUXXXThP3BnSMzbGeI/aEjM2xkS4z9gKVDrVwrkP1gK1PqVAuU/U6DWrxSo5T9ONtlkk03mP0Ri3s6QmOc/OY7jOI7j6D832WSTTTbpPzJvZ0jM2+k/L7rooosu6j8tBWr9SoHqPypQ61cK1Oo/J5tssskm6z8iMW9nSMzrPx3HcRzHcew/Faj1KwVq7T8T83aGxLztPxA++OCDD+4/Dol5O0Ni7j8L1PqVArXuPwZq/UqBWu8/A7V+pUCt7z8AAAAAAADwPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]},\"y\":{\"__ndarray__\":\"KkoB5v5a8T+xNOWPYA7xPwmKQnIDoPI/baG5rHxu9D8Zs8QozYfxP46U227sGfU/MBs8Q8++9T+WlRexJQDzP9dNo50bXvQ/kBE1XHQF8j9qYC3NpBP0P2V4k3dcTPc/dFeBPtKj+D/cv6Un7QT5P7RKRR121vQ/YRKXhub9+T9syW8J4i33PxUGxgHmqfU/RGyNWNmX+j+QlWUQCQP4P5+7CzjOI/s/KAUaFFfT+D9I6KHiN/n5P7Y0PYr1vfw/vhbQULra/z+cn0WVsHD8P5Keuo3diQBAaK1NrLMrAEB+ZGCKu5QBQK00SvegnQNAjsPKC/QkAkC49pHnb20DQGYgJRd1pQVAtvpubkEhBkCTRmFZMkAGQBIAKxbQjwhA4kxNa115B0DIX7JpgagJQItyx/69PwhAXHAgAA7GCEC2WmYk6H8KQMv/Y4O23gtAKjf9QC1wDUCdCnIvsXMMQFh/oZlsWg5APT6UNMZgDUBIaUJpgZUOQCTkqgs9Sg5AGrglp7txEEAzM4UuDhEPQA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[50]}},\"selected\":{\"id\":\"2656\"},\"selection_policy\":{\"id\":\"2655\"}},\"id\":\"2641\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"2655\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"2613\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"2615\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"2656\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"2617\",\"type\":\"LinearScale\"}],\"root_ids\":[\"2686\",\"2754\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"7357f241-694d-498e-9041-5c4be491f9ff\",\"root_ids\":[\"2686\"],\"roots\":{\"2686\":\"0f8a6fcf-c659-4d6f-8069-c8dc05cbc7e0\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "2686"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Inference points (not observed)\n",
    "X_plot = x_plot[:, np.newaxis]\n",
    "# Predict the values\n",
    "y_plot = model.predict(X_plot)\n",
    "# Compute the error of our model at observed points\n",
    "Y_model_err = np.sqrt(np.mean(np.square(y-model.predict(X))))\n",
    "print(f'Model error : {Y_model_err}')\n",
    "# Plot the result\n",
    "p = scatter(x, y, title=\"Training a scikit-learn model\")\n",
    "p.line(x_plot, poly(x_plot), line_width=6, line_alpha=0.6, color=\"red\", legend_label=r\"Trained model\")\n",
    "plot = (center_plot(p))\n",
    "plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cml.tasks import RegressionPolynomialSolver\n",
    "explorer = RegressionPolynomialSolver()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='2884'>\n",
       "  <div class=\"bk-root\" id=\"32d05313-4071-411c-9f70-f81ca1383eec\" data-root-id=\"2884\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"f44407df-218c-4fbb-bbec-25ee426feea1\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2854\",\"type\":\"Circle\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00173\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2885\",\"type\":\"Spacer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2845\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"2908\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"tools\":[{\"id\":\"2839\"},{\"id\":\"2840\"},{\"id\":\"2841\"},{\"id\":\"2842\"},{\"id\":\"2843\"},{\"id\":\"2844\"},{\"id\":\"2872\"},{\"id\":\"2876\"}]},\"id\":\"2846\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"2909\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAALV+JTy1fqU8ED74PLV+JT1i3k49ED54Pd7OkD21fqU9jC66PWLezj05juM9ED74PfN2Bj7ezhA+yiYbPrV+JT6g1i8+jC46PneGRD5i3k4+TjZZPjmOYz4k5m0+ED54Pv1KgT7zdoY+6aKLPt7OkD7U+pU+yiabPr9SoD61fqU+q6qqPqDWrz6WArU+jC66PoFavz53hsQ+bbLJPmLezj5YCtQ+TjbZPkNi3j45juM+L7roPiTm7T4aEvM+ED74PgVq/T79SgE/+OADP/N2Bj/uDAk/6aILP+Q4Dj/ezhA/2WQTP9T6FT/PkBg/yiYbP8W8HT+/UiA/uugiP7V+JT+wFCg/q6oqP6VALT+g1i8/m2wyP5YCNT+RmDc/jC46P4bEPD+BWj8/fPBBP3eGRD9yHEc/bbJJP2dITD9i3k4/XXRRP1gKVD9ToFY/TjZZP0jMWz9DYl4/PvhgPzmOYz80JGY/L7poPylQaz8k5m0/H3xwPxoScz8VqHU/ED54PwrUej8Fan0/AACAPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"IDeMP+jsbz+WJYM/l/uKP8uxfD8FqmU/PvZrP7IOgD8tbYY/x62CP2aJdT9BXoA/bD5kP4LkXj94Vns/cRZkP5Zjgj94L2w/oMl2P8LMcT9h1GM/rzt+Pwojdj8q3Gg/Lk9gPyZ6VT8ewVQ/3QpvP8PtRT8ANkw/lPFhPyK3ZT9bOEc/oQ83P3bXOT8nVjY/mm5HP60+Ij9kwEs/KNcrP1O0Kj84JRE/p8MdP54yEj8i2TE/Wd0MP7eLBz/fyxg/rf4XPxtK2j7sLuU+WWfCPtzztz7ISeA+TIP3PoRXuj7Osao+EkKcPguepD7g5ZQ+RuSKPk7ZNT5ElWI+rdgoPuQbaD6RKR0+wUcyPiLLGz765ws+fab3PfDJt7sxGIK9BsiEPL40s73X3S6+H1oSvpyODL7GUx6+q4WOvhqpnb4rA8K+6szHvjh6kL5ufKS+14T6vm8E/b4i+wG/WJUcv+11I7+2mS+/2JIcv2vUN7+t+jS/Fl03vz9pZr/lBFq/Md9Hv2DCbL9Wn4C/dJJ4vw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"2868\"},\"selection_policy\":{\"id\":\"2867\"}},\"id\":\"2853\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"children\":[{\"id\":\"2888\"},{\"id\":\"2898\"}],\"margin\":[0,0,0,0],\"name\":\"Row00201\"},\"id\":\"2887\",\"type\":\"Row\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2856\",\"type\":\"Circle\"},{\"attributes\":{\"children\":[{\"id\":\"2885\"},{\"id\":\"2886\"},{\"id\":\"2926\"}],\"margin\":[0,0,0,0],\"name\":\"Row00206\"},\"id\":\"2884\",\"type\":\"Row\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2857\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"2871\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"2870\",\"type\":\"Legend\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"2925\"},\"target\":{\"id\":\"2925\"}},\"code\":\"\\n    var value = source['readonly'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['disabled'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set disabled on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['disabled'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11527165696,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"2940\",\"type\":\"CustomJS\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2853\"},\"glyph\":{\"id\":\"2854\"},\"group\":null,\"hover_glyph\":{\"id\":\"2856\"},\"muted_glyph\":{\"id\":\"2857\"},\"nonselection_glyph\":{\"id\":\"2875\"},\"selection_glyph\":{\"id\":\"2874\"},\"view\":{\"id\":\"2859\"}},\"id\":\"2858\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"2853\"}},\"id\":\"2859\",\"type\":\"CDSView\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"2858\"}]},\"id\":\"2871\",\"type\":\"LegendItem\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"2925\"},\"target\":{\"id\":\"2925\"}},\"code\":\"\\n    var value = source['disabled'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['readonly'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set readonly on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['readonly'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11527165696,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"2941\",\"type\":\"CustomJS\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"2911\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"children\":[{\"id\":\"2889\"},{\"id\":\"2890\"},{\"id\":\"2891\"},{\"id\":\"2892\"},{\"id\":\"2893\"},{\"id\":\"2894\"},{\"id\":\"2895\"},{\"id\":\"2896\"},{\"id\":\"2897\"}],\"margin\":[5,5,5,5],\"min_width\":300,\"name\":\"Problem parameters\",\"width\":300},\"id\":\"2888\",\"type\":\"Column\"},{\"attributes\":{},\"id\":\"2827\",\"type\":\"LinearScale\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2880\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"2823\",\"type\":\"DataRange1d\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2881\",\"type\":\"Line\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2874\",\"type\":\"Circle\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"2911\"},\"renderers\":[{\"id\":\"2858\"}]},\"id\":\"2876\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"2867\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"2831\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"2834\"},{\"id\":\"2838\"},{\"id\":\"2870\"}],\"height\":450,\"left\":[{\"id\":\"2835\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"2858\"},{\"id\":\"2882\"}],\"title\":{\"id\":\"2821\"},\"toolbar\":{\"id\":\"2846\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"2823\"},\"x_scale\":{\"id\":\"2827\"},\"y_range\":{\"id\":\"2825\"},\"y_scale\":{\"id\":\"2829\"}},\"id\":\"2820\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"2858\"}],\"tooltips\":null},\"id\":\"2872\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"2865\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"children\":[{\"id\":\"2887\"},{\"id\":\"2925\"}],\"margin\":[0,0,0,0],\"name\":\"Column00204\"},\"id\":\"2886\",\"type\":\"Column\"},{\"attributes\":{\"line_alpha\":0.7,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2879\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"2866\",\"type\":\"AllLabels\"},{\"attributes\":{\"margin\":[5,10,5,10],\"name\":\"\",\"text\":\"<b>Problem parameters</b>\"},\"id\":\"2889\",\"type\":\"Div\"},{\"attributes\":{},\"id\":\"2844\",\"type\":\"HelpTool\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} x$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2865\"},\"group\":null,\"major_label_policy\":{\"id\":\"2866\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2832\"}},\"id\":\"2831\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"2875\",\"type\":\"Circle\"},{\"attributes\":{\"client_comm_id\":\"0d793b8fbcfd4437824e8a858756e98d\",\"comm_id\":\"21ac1027e66742a18f67c54899f411c5\",\"plot_id\":\"2884\"},\"id\":\"2968\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"2825\",\"type\":\"DataRange1d\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Polynomial regression problem\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"2821\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"2832\",\"type\":\"BasicTicker\"},{\"attributes\":{\"labels\":[\"Ground truth\"],\"margin\":[5,10,5,10]},\"id\":\"2892\",\"type\":\"CheckboxGroup\"},{\"attributes\":{},\"id\":\"2829\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} -2.0x^2 + 0.0x^1 + 1.0x^0$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"2862\"},\"group\":null,\"major_label_policy\":{\"id\":\"2863\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"2836\"}},\"id\":\"2835\",\"type\":\"LinearAxis\"},{\"attributes\":{\"labels\":[\"Error\"],\"margin\":[5,10,5,10]},\"id\":\"2893\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"axis\":{\"id\":\"2835\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2838\",\"type\":\"Grid\"},{\"attributes\":{\"end\":1000,\"margin\":[5,10,5,10],\"start\":10,\"title\":\"Number observations\",\"value\":100},\"id\":\"2890\",\"type\":\"Slider\"},{\"attributes\":{\"axis\":{\"id\":\"2831\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"2834\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"2836\",\"type\":\"BasicTicker\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":0.0,\"step\":0.1,\"title\":\"Noise level\",\"value\":0.1},\"id\":\"2891\",\"type\":\"Slider\"},{\"attributes\":{},\"id\":\"2863\",\"type\":\"AllLabels\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 0\",\"value\":1.0},\"id\":\"2896\",\"type\":\"Slider\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 2\",\"value\":-2.0},\"id\":\"2894\",\"type\":\"Slider\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"2878\"},\"glyph\":{\"id\":\"2879\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"2881\"},\"nonselection_glyph\":{\"id\":\"2880\"},\"view\":{\"id\":\"2883\"}},\"id\":\"2882\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"2840\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"2839\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"2862\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"end\":30,\"margin\":[5,10,5,10],\"start\":1,\"title\":\"Degree\",\"value\":1},\"id\":\"2897\",\"type\":\"Slider\"},{\"attributes\":{\"overlay\":{\"id\":\"2845\"}},\"id\":\"2841\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"source\":{\"id\":\"2878\"}},\"id\":\"2883\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"2842\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"2843\",\"type\":\"ResetTool\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 1\",\"value\":0.0},\"id\":\"2895\",\"type\":\"Slider\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABn0LLjOZV0P2fQsuM5lYQ/mjiM1dbfjj9n0LLjOZWUP4GEn1yIupk/mjiM1dbfnj9adjynkgKiP2fQsuM5laQ/dCopIOEnpz+BhJ9ciLqpP47eFZkvTaw/mjiM1dbfrj9USQEJP7mwP1p2PKeSArI/YaN3ReZLsz9n0LLjOZW0P2397YGN3rU/dCopIOEntz96V2S+NHG4P4GEn1yIurk/h7Ha+tsDuz+O3hWZL028P5QLUTeDlr0/mjiM1dbfvj/QsuM5lRTAP1RJAQk/ucA/198e2OhdwT9adjynkgLCP90MWnY8p8I/YaN3ReZLwz/kOZUUkPDDP2fQsuM5lcQ/6mbQsuM5xT9t/e2Bjd7FP/GTC1E3g8Y/dCopIOEnxz/3wEbviszHP3pXZL40ccg//u2Bjd4VyT+BhJ9ciLrJPwQbvSsyX8o/h7Ha+tsDyz8KSPjJhajLP47eFZkvTcw/EXUzaNnxzD+UC1E3g5bNPxeibgYtO84/mjiM1dbfzj8ez6mkgITPP9Cy4zmVFNA/En5yIepm0D9USQEJP7nQP5UUkPCTC9E/198e2Ohd0T8Zq62/PbDRP1p2PKeSAtI/nEHLjudU0j/dDFp2PKfSPx/Y6F2R+dI/YaN3ReZL0z+ibgYtO57TP+Q5lRSQ8NM/JQUk/ORC1D9n0LLjOZXUP6mbQcuO59Q/6mbQsuM51T8sMl+aOIzVP2397YGN3tU/r8h8aeIw1j/xkwtRN4PWPzJfmjiM1dY/dCopIOEn1z+19bcHNnrXP/fARu+KzNc/OYzV1t8e2D96V2S+NHHYP7wi86WJw9g//u2Bjd4V2T8/uRB1M2jZP4GEn1yIutk/wk8uRN0M2j8EG70rMl/aP0bmSxOHsdo/h7Ha+tsD2z/JfGniMFbbPwpI+MmFqNs/TBOHsdr62z+O3hWZL03cP8+ppICEn9w/EXUzaNnx3D9SQMJPLkTdP5QLUTeDlt0/1tbfHtjo3T8Xom4GLTveP1lt/e2Bjd4/mjiM1dbf3j/cAxu9KzLfPx7PqaSAhN8/X5o4jNXW3z/QsuM5lRTgP3EYq62/PeA/En5yIepm4D+z4zmVFJDgP1RJAQk/ueA/9K7IfGni4D+VFJDwkwvhPzZ6V2S+NOE/198e2Ohd4T94ReZLE4fhPxmrrb89sOE/uRB1M2jZ4T9adjynkgLiP/vbAxu9K+I/nEHLjudU4j89p5ICEn7iP90MWnY8p+I/fnIh6mbQ4j8f2OhdkfniP8A9sNG7IuM/YaN3ReZL4z8BCT+5EHXjP6JuBi07nuM/Q9TNoGXH4z/kOZUUkPDjP4WfXIi6GeQ/JQUk/ORC5D/GautvD2zkP2fQsuM5leQ/CDZ6V2S+5D+pm0HLjufkP0kBCT+5EOU/6mbQsuM55T+LzJcmDmPlPywyX5o4jOU/zZcmDmO15T9t/e2Bjd7lPw5jtfW3B+Y/r8h8aeIw5j9QLkTdDFrmP/GTC1E3g+Y/kfnSxGGs5j8yX5o4jNXmP9PEYay2/uY/dCopIOEn5z8VkPCTC1HnP7X1twc2euc/Vlt/e2Cj5z/3wEbvisznP5gmDmO19ec/OYzV1t8e6D/a8ZxKCkjoP3pXZL40ceg/G70rMl+a6D+8IvOlicPoP12Iuhm07Og//u2Bjd4V6T+eU0kBCT/pPz+5EHUzaOk/4B7Y6F2R6T+BhJ9ciLrpPyLqZtCy4+k/wk8uRN0M6j9jtfW3BzbqPwQbvSsyX+o/pYCEn1yI6j9G5ksTh7HqP+ZLE4ex2uo/h7Ha+tsD6z8oF6JuBi3rP8l8aeIwVus/auIwVlt/6z8KSPjJhajrP6utvz2w0es/TBOHsdr66z/teE4lBSTsP47eFZkvTew/LkTdDFp27D/PqaSAhJ/sP3APbPSuyOw/EXUzaNnx7D+y2vrbAxvtP1JAwk8uRO0/86WJw1ht7T+UC1E3g5btPzVxGKutv+0/1tbfHtjo7T92PKeSAhLuPxeibgYtO+4/uAc2eldk7j9Zbf3tgY3uP/rSxGGstu4/mjiM1dbf7j87nlNJAQnvP9wDG70rMu8/fWniMFZb7z8ez6mkgITvP780cRirre8/X5o4jNXW7z8AAAAAAADwPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[200]},\"y\":{\"__ndarray__\":\"AAAAADVF9T9LCki4Axz1P5UUkHDS8vQ/4B7YKKHJ9D8qKSDhb6D0P3UzaJk+d/Q/wD2wUQ1O9D8KSPgJ3CT0P1VSQMKq+/M/oFyIennS8z/qZtAySKnzPzVxGOsWgPM/f3tgo+VW8z/KhahbtC3zPxWQ8BODBPM/X5o4zFHb8j+qpICEILLyP/SuyDzviPI/P7kQ9b1f8j+Kw1itjDbyP9TNoGVbDfI/H9joHSrk8T9p4jDW+LrxP7TseI7HkfE///bARpZo8T9KAQn/ZD/xP5QLUbczFvE/3hWZbwLt8D8pIOEn0cPwP3QqKeCfmvA/vjRxmG5x8D8JP7lQPUjwP1RJAQkMH/A/PKeSgrXr7z/SuyLzUpnvP2bQsmPwRu8//ORC1I307j+S+dJEK6LuPyYOY7XIT+4/vCLzJWb97T9RN4OWA6vtP+ZLEwehWO0/fGCjdz4G7T8RdTPo27PsP6aJw1h5Yew/O55TyRYP7D/QsuM5tLzrP2bHc6pRaus/+9sDG+8X6z+Q8JOLjMXqPyYFJPwpc+o/uxm0bMcg6j9PLkTdZM7pP+VC1E0CfOk/eldkvp8p6T8PbPQuPdfoP6WAhJ/ahOg/OpUUEHgy6D/PqaSAFeDnP2S+NPGyjec/+dLEYVA75z+P51TS7ejmPyT85EKLluY/uhB1syhE5j9OJQUkxvHlP+M5lZRjn+U/eU4lBQFN5T8OY7V1nvrkP6R3ReY7qOQ/OYzVVtlV5D/NoGXHdgPkP2O19TcUseM/+MmFqLFe4z+O3hUZTwzjPyPzpYnsueI/uAc2+oln4j9NHMZqJxXiP+IwVtvEwuE/d0XmS2Jw4T8NWna8/x3hP6JuBi2dy+A/OIOWnTp54D/MlyYO2CbgP8JYbf3qqN8/7oGN3iUE3z8Yq62/YF/eP0TUzaCbut0/bv3tgdYV3T+WJg5jEXHcP8JPLkRMzNs/7HhOJYcn2z8Yom4GwoLaP0LLjuf83dk/bPSuyDc52T+WHc+pcpTYP8BG74qt79c/7G8PbOhK1z8WmS9NI6bWP0DCTy5eAdY/bOtvD5lc1T+YFJDw07fUP8A9sNEOE9Q/7GbQsklu0z8UkPCThMnSPzy5EHW/JNI/bOIwVvp/0T+UC1E3NdvQP8A0cRhwNtA/0Lsi81Ujzz8gDmO1y9nNP3hgo3dBkMw/0LLjObdGyz8oBST8LP3JP3hXZL6is8g/0KmkgBhqxz8g/ORCjiDGP3hOJQUE18Q/0KBlx3mNwz8g86WJ70PCP3hF5ktl+sA/kC9NHLZhvz9A1M2goc68P/B4TiWNO7o/kB3PqXiotz9Awk8uZBW1P+Bm0LJPgrI/QBeibnberz+AYKN3TbiqP8CppIAkkqU/IPOliftroD/AeE4lpYuWP4AXom6mfog/AOQ5lRQwXz+AnlNJobKAv4A8p5KipZK/AKqkgPTxnL+AC1E3I5+jv0DCTy5Mxai/4HhOJXXrrb/QlyYOz4ixvzDzpYnjG7S/cE4lBfiutr/QqaSADEK5vyAFJPwg1bu/gGCjdzVovr/wXZH5pH3Av5ALUTcvx8G/QLkQdbkQw7/oZtCyQ1rEv5gUkPDNo8W/QMJPLljtxr/wbw9s4jbIv5gdz6lsgMm/QMuO5/bJyr/weE4lgRPMv5gmDmMLXc2/SNTNoJWmzr/wgY3eH/DPv8yXJg7VnNC/pG4GLZpB0b94ReZLX+bRv1Acxmoki9K/IPOliekv07/4yYWortTTv9CgZcdzedS/pHdF5jge1b98TiUF/sLVv0wlBSTDZ9a/JPzkQogM17/80sRhTbHXv9CppIASVti/qICEn9f62L94V2S+nJ/Zv1AuRN1hRNq/JAUk/Cbp2r/82wMb7I3bv9Sy4zmxMty/pInDWHbX3L98YKN3O3zdv1A3g5YAId6/KA5jtcXF3r8A5ULUimrfv+hdkfmnB+C/VEkBiQpa4L++NHEYbazgvyog4afP/uC/lAtRNzJR4b/+9sDGlKPhv2riMFb39eG/1M2g5VlI4r9AuRB1vJriv6qkgAQf7eK/FJDwk4E/47+Ae2Aj5JHjv+pm0LJG5OO/VlJAQqk25L/APbDRC4nkvywpIGFu2+S/lBSQ8NAt5b8AAACAM4Dlvw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[200]}},\"selected\":{\"id\":\"2909\"},\"selection_policy\":{\"id\":\"2908\"}},\"id\":\"2878\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"children\":[{\"id\":\"2820\"}],\"margin\":[0,0,0,0],\"name\":\"Row00198\"},\"id\":\"2898\",\"type\":\"Row\"},{\"attributes\":{\"code\":\"def polynomial(\\n        coefficients: jnp.ndarray,\\n        n_observations: int = 100) -> [jnp.ndarray, jnp.ndarray]:\\n    x = jnp.linspace(0, 1, n_observations)\\n    y = jnp.polyval(coefficients, x)\\n    return x, \",\"filename\":\"\",\"height\":150,\"js_property_callbacks\":{\"change:disabled\":[{\"id\":\"2941\"}],\"change:readonly\":[{\"id\":\"2940\"}]},\"language\":\"python\",\"margin\":[5,10,5,10],\"sizing_mode\":\"stretch_both\",\"theme\":\"chaos\"},\"id\":\"2925\",\"type\":\"panel.models.ace.AcePlot\"},{\"attributes\":{},\"id\":\"2868\",\"type\":\"Selection\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00205\",\"sizing_mode\":\"stretch_width\"},\"id\":\"2926\",\"type\":\"Spacer\"}],\"root_ids\":[\"2884\",\"2968\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"f44407df-218c-4fbb-bbec-25ee426feea1\",\"root_ids\":[\"2884\"],\"roots\":{\"2884\":\"32d05313-4071-411c-9f70-f81ca1383eec\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Column\n",
       "        [0] Row\n",
       "            [0] Param(RegressionPolynomialSolver, name='Problem parameters', widgets={})\n",
       "            [1] ParamMethod(method, _pane=Bokeh)\n",
       "        [1] Ace(height=150, language='python', sizing_mode='stretch_both', theme='chaos', value='def polynomial(\\n        ...)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "2884"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def solve(x, y, degree):\n",
    "    X = x[:, np.newaxis]\n",
    "    # Create our polynomial model for regression\n",
    "    model = make_pipeline(PolynomialFeatures(degree), LinearRegression())\n",
    "    # Fit the parameters of this model\n",
    "    model.fit(X, y)\n",
    "    # Predict the values\n",
    "    x_predict = np.linspace(np.min(x), np.max(x), 200)[:, np.newaxis]\n",
    "    y_model = model.predict(x_predict)\n",
    "    return x_predict[:, 0], y_model #np.array(jnp.zeros(y_model.shape))\n",
    "\n",
    "explorer.solve = solve\n",
    "explorer.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"linear\"></a>\n",
    "# Simple linear regression \n",
    "\n",
    "As discussed previously, regression allows to model the relationships that exist between inputs $\\mathbf{x}\\in\\mathbb{R}^{n}$ and a continuous output $y\\in\\mathbb{R}$. In the case of **linear** regression, we assume that the data that we observe comes from a linear relationship in the input, so that\n",
    "$$y = w_0 + w_1x_1 + w_2x_2 + ... + w_nx_n + \\epsilon$$\n",
    "with $\\epsilon$ exhibiting the *observation noise* (also called *residual error*) that is always present in measurements.\n",
    "\n",
    "To understand how we could learn a model approximating this relationship, we start with the case of *simple* linear regression, where $\\mathbf{x}\\in\\mathbb{R}$. This implies that our model will follow\n",
    "$$\\bar{y} = w_0 + w_1 x$$\n",
    "\n",
    "We will measure the errors made by our model by using the Mean Squared Error (MSE) loss, defined as \n",
    "$$\\mathcal{L}_{MSE}\\left( \\bar{\\mathbf{y}},\\theta \\right) = \\sum_{i=1}^{n} \\left| y_{i} - \\bar{y}_{i} \\right|^{2} = \\sum_{i=1}^{n} \\left| y_{i} - (w_{0} + w_{1} x_{i}) \\right|^{2}$$\n",
    "\n",
    "Then our goal is to find the most adequate set of parameters $\\theta = \\{w_{0}, w_{1}\\}$, which are those that minimize the MSE loss defined previously. Therefore, we aim to obtain\n",
    "$$\\theta^{*}= \\underset{\\theta}{\\text{argmin }} \\mathcal{L}\\left( \\bar{\\mathbf{y}}, \\mathbf{y} \\mid f_{\\theta}, \\theta \\right)$$\n",
    "\n",
    "To do so, we will implement the **gradient descent** algorithm discussed in the course.\n",
    "\n",
    "## Manual implementation - `NumPy`\n",
    "\n",
    "We start by performing a *full manual implementation*, in the sense that we need to manually derive the gradient in order to apply the gradient descent updates. To do so, we will rely on [NumPy](https://numpy.org/), which is a fundamental library for numerical computing offering support for N-dimensional arrays and scientific computing tasks, such as linear algebra, statistical analysis, and matrix manipulation. We strongly encourage you to learn NumPy through the set of [tutorials](https://numpy.org/learn/). For the sake of this introductory tutorial, we will provide the explanation for all of the functions that we will use in this first exercise. After that exercise, we will assume for the rest of the course that knowledge of Numpy should be found online. \n",
    "\n",
    "We start by import libraries (`NumPy`) and set a random seed to ensure that the random number generator produces always a reproducible series of random numbers.\n",
    "\n",
    "**Used functions**\n",
    "- `np.random.seed`: sets the seed for the NumPy random number generator. [Documentation](https://numpy.org/doc/stable/reference/random/generated/numpy.random.seed.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate a synthetic dataset.\n",
    "\n",
    "For the sake of this exercise, we will generate the data ourselves, so that we know the true values that we are looking for in advance. Hence, we define a linear relationship following\n",
    "$$y = w^{t}_0 + w^{t}_1 x + \\epsilon$$\n",
    "\n",
    "**Used functions**\n",
    "- `np.random.rand`: generates an array of random numbers uniformly distributed over [0, 1). [Documentation](https://numpy.org/doc/stable/reference/random/generated/numpy.random.rand.html)\n",
    "- `np.random.randn`: generates an array of random numbers from the standard normal distribution (mean 0, variance 1). [Documentation](https://numpy.org/doc/stable/reference/random/generated/numpy.random.randn.html)\n",
    "- `np.ones`: generates an array of ones with a specified shape. [Documentation](https://numpy.org/doc/stable/reference/generated/numpy.ones.html)\n",
    "- `np.c_`: concatenates arrays along the second axis. [Documentation](https://numpy.org/doc/stable/reference/generated/numpy.c_.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "true_w0 = 2\n",
    "true_w1 = 3\n",
    "x = np.random.rand(100)\n",
    "y = true_w0 + true_w1 * x + np.random.randn(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computing the MSE loss\n",
    "\n",
    "We will measure the errors made by our model by using the Mean Squared Error (MSE) loss, defined as \n",
    "$$\\mathcal{L}_{MSE}\\left( \\bar{\\mathbf{y}},\\theta \\right) = \\sum_{i=1}^{n} \\left| y_{i} - \\bar{y}_{i} \\right|^{2} = \\sum_{i=1}^{n} \\left| y_{i} - (w_{0} + w_{1} x_{i}) \\right|^{2}$$\n",
    "\n",
    "This will allow us to evaluate the performances of our model, but is also the basis for the following gradient descent algorithm.\n",
    "\n",
    "**Used functions**\n",
    "- `np.sum`: computes the sum of the provided array (optionally across a provided `axis`). [Documentation](https://numpy.org/doc/stable/reference/generated/numpy.mean.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse_loss(y, y_bar):\n",
    "    return np.sum((y - y_bar) ** 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implement the gradient descent algorithm.\n",
    "\n",
    "As seen in the course Initialize the weight $w_1$ and bias $w_0$ to small random values\n",
    "1. Evaluate the predictions made by our model \n",
    "$$\\hat{y} = w_{1}x + w_0$$\n",
    "2. Compute the mean squared error (MSE) loss between the predicted values and the ground truth labels: \n",
    "$$\\mathcal{L}_{MSE} = \\frac{1}{n} \\sum_{i=1}^n (\\bar{y_i} - y_i)^2$$\n",
    "3. Compute the gradients of the loss with respect to the weight and bias (see derivation in the course)\n",
    "$$\\frac{\\partial \\mathcal{L}}{\\partial w_1} = \\sum_{i} 2 * (\\bar{y}_{i} - y_{i})*x_{i}$$\n",
    "$$\\frac{\\partial \\mathcal{L}}{\\partial w_0} = \\sum_{i} 2 * (\\bar{y}_{i} - y_{i})$$\n",
    "4. Update the weights and bias using the gradients and a learning rate $\\eta$: \n",
    "$$w_1 \\leftarrow w_1 - \\eta \\frac{\\partial \\mathcal{L}}{\\partial w_1}$$ \n",
    "$$w_0 \\leftarrow w_0 - \\eta \\frac{\\partial \\mathcal{L}}{\\partial w_0}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters\n",
    "n_iter = 1000\n",
    "lr = 0.001\n",
    "def gradient_descent(x, y, n_iter, lr):\n",
    "    # Initialize the parameters\n",
    "    w_1 = np.random.randn()\n",
    "    w_0 = np.random.randn()\n",
    "    # Perform gradient descent\n",
    "    for i in range(n_iter):\n",
    "        # 1. Calculate the predictions\n",
    "        y_bar = w_1 * x + w_0\n",
    "        # 2. Compute the loss\n",
    "        loss = mse_loss(y, y_bar)\n",
    "        # 3. Calculate the gradients\n",
    "        dw_1 = 2 * np.sum((y_bar - y) * x)\n",
    "        dw_0 = 2 * np.sum((y_bar - y))\n",
    "        # 4. Update the parameters\n",
    "        w_1 -= lr * dw_1\n",
    "        w_0 -= lr * dw_0\n",
    "    return w_1, w_0\n",
    "w_1, w_0 = gradient_descent(x, y, n_iter, lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the results.\n",
    "\n",
    "We can see how well our model fits to the observed data by plotting it against the observed samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='5034'>\n",
       "  <div class=\"bk-root\" id=\"e413e117-7a9b-46ee-92e0-cfa4f96b72d9\" data-root-id=\"5034\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"5ed52e6b-f512-45e8-9815-427d01f8f61c\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"4967\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"4970\"},{\"id\":\"4974\"},{\"id\":\"5006\"}],\"height\":450,\"left\":[{\"id\":\"4971\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"4994\"},{\"id\":\"5018\"}],\"title\":{\"id\":\"4957\"},\"toolbar\":{\"id\":\"4982\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"4959\"},\"x_scale\":{\"id\":\"4963\"},\"y_range\":{\"id\":\"4961\"},\"y_scale\":{\"id\":\"4965\"}},\"id\":\"4956\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"5010\",\"type\":\"Circle\"},{\"attributes\":{\"line_alpha\":0.6,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5015\",\"type\":\"Line\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"4994\"}],\"tooltips\":null},\"id\":\"5008\",\"type\":\"HoverTool\"},{\"attributes\":{\"source\":{\"id\":\"5014\"}},\"id\":\"5019\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"4989\"}},\"id\":\"4995\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"4998\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"4989\"},\"glyph\":{\"id\":\"4990\"},\"group\":null,\"hover_glyph\":{\"id\":\"4992\"},\"muted_glyph\":{\"id\":\"4993\"},\"nonselection_glyph\":{\"id\":\"5011\"},\"selection_glyph\":{\"id\":\"5010\"},\"view\":{\"id\":\"4995\"}},\"id\":\"4994\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"5032\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"children\":[{\"id\":\"5035\"},{\"id\":\"4956\"},{\"id\":\"5062\"}],\"margin\":[0,0,0,0],\"name\":\"Row00302\"},\"id\":\"5034\",\"type\":\"Row\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"4994\"}]},\"id\":\"5007\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"5011\",\"type\":\"Circle\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00299\",\"sizing_mode\":\"stretch_width\"},\"id\":\"5062\",\"type\":\"Spacer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4993\",\"type\":\"Circle\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00298\",\"sizing_mode\":\"stretch_width\"},\"id\":\"5035\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"4999\",\"type\":\"AllLabels\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"5014\"},\"glyph\":{\"id\":\"5015\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"5017\"},\"nonselection_glyph\":{\"id\":\"5016\"},\"view\":{\"id\":\"5019\"}},\"id\":\"5018\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4990\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"4981\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"client_comm_id\":\"ac0f2c88422540b888f8fc0e268182cf\",\"comm_id\":\"b5d155bf269b496e8331b82353026905\",\"plot_id\":\"5034\"},\"id\":\"5102\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{},\"id\":\"5001\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5017\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4979\",\"type\":\"ResetTool\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5016\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"4978\",\"type\":\"SaveTool\"},{\"attributes\":{\"overlay\":{\"id\":\"4981\"}},\"id\":\"4977\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"5002\",\"type\":\"AllLabels\"},{\"attributes\":{\"label\":{\"value\":\"Learned model\"},\"renderers\":[{\"id\":\"5018\"}]},\"id\":\"5033\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"4975\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"4976\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"vNau47j9yD+MLP/lUx7nP/B2k6Qs+NE/QPpf4kjmmD93piyEtafkP8it6Q+Qq8Y/IjnhmTwY7j8QRR45lYbuP6iyiK+RRu0/+lP7Ha6w1z8AhPzzt6ePPzWwUiHJtO0/CBBsfF5n2z8JB0oW1u7uPwrc64/51e4/kSdNfNpL6z9GEXIqQNjSP1z8hvFwpdg/EusS+YI86z9q2uE7c0jUP4CG9jXwscU//ymU4VDR4T9oeWDb+vTtP1VWuEfgReY/KhtI7PA94j9Idq8Gj+C4Pz5Ar6IjruM/GkFvaYWu7z/kDRHkRe7BP6dgLhEoluA/v0pKsXAT7D+xCWVjYLTnP3bmi/TzTeY/w+zV5r965j/gLGws5wHXP7RAX3I1ytI/CNutXUnm6T8cWwbtcuzpP6YwcnIOv+s/bI1TQEQ57T8uTwW86lzgP8D2guZrDOA/wO/GVKKL6T+8pEgogczkP39Olj2DduY/qvwzMCJ36T9d5Yt77HrsPySPOHC2odU/6oOhFI0J2D/g+gdOMw+4P54nsFpFgeI/AHQAlQZnoj90hj2hW8zdP7Jn7UdYXeE/xCK2HrFW0j/oDXEnG+jiP4A8CCB1O58/oER8Qk0foz88izZpvlLqP97xSwxdDdc/pPY51YRDwD/kvE9/N7bgP1gPFoXJo+g/5Ktx/wWgyz9RPdQBuO7jP2Da8dpU2bU/8LhwzAZ2qj/1OaVt2wDhP+1MzAbiTOE/bqyNZNNl5D9q7ofkIzznP0XO4CMuOu8/0b7bToiF4D80G1SgUavUP4FkmFEqcuk/uNl3y1BV0T+gzPOVGxjcP6gMB6i3FbQ/4D6Yl4v1mT9c0FIMBM7uPxGYtmFZwOo/HgbBsmtF5j80Lnf4SCzaPyS3pB6CLsY/7N8TByEGxD+iFevJ+gPQP1EgYMxDk+E/OUG7RPjd5j9zC70/ViDlP5wtG91v6tE/0hgRokGO7j/K4aD+2ZznPwkhvrVEveE/bY/rYjeT4z/+uGE4utraPwh9Dimmtc8/bOhEoUHI1j9xDHt8RkDoPwBthVdVeo0/mATwwu+2vT8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"poX63YM8CEDyE5chmcQTQFAZnW9spf4/ObRBR30bAEBg4p9Dqj7mP9tPWLpxHPg/wHr7f3VGEkBKZh4reukMQEJKA3YNghlAE4qMqabi+j83ikU5grP5P+XSHUisqRNAn8943D7nEkA/exBFe7YLQEYKBvxRNxhA8pm2d+xGEkCerYW17W3+P90//aht8AxAetpwa4cCE0D1deW07M0CQH4FYG9RoARA51GlXd1HCkCRtVn2G7ATQJocSLA5ABNAhwpsI08vFUCozmExA9zwP3prp0OH6RdAlHvE2QMlCEDvnCkA0iUCQDB+19m+khBAsFCFlwanE0AS3cRn/ssMQP0W0vc6EA9A/WLGc2XqDECJpATSsekDQC9EDjrQ1w1A0LzS+vAjE0Ae7y4CQuYNQAu1nCDWABZAVUzgUSYwFED8ReTzNmMRQCDVw+JliRBA7oqxnPGGDEDEmbqrIB4LQCFsAsCraRNABnI1ubH9E0B6axp0sZgSQPsZ58DtDAlAEuuYox6eEUCQtRwPxgv7P+B3wU20IBFA3uscQXp9/j+hjfC30m4JQPySTN0m6BJADTOITvZ6DUBPz1DaElgSQAn8Mp4DLQtAsQwFr3sQAUAI8ahIWZkUQOSjd8GVKQZAcxROb42kBUA+UWkuIX4LQCU57nq+oBFA8DOnz+LwCUDH/UO4UmcIQBThYb/KZBFAWpae9Gpi8j/jIEMN/AkDQG3pp5O8HhNAi0PF5djQEkCBEp+5pjUTQPzQx04+ORZAze8dZRFLDEAOvvHoqpIAQGN3YpRv2BFAiI/kiSoVAUD4kLD1CisRQAwqH8zctABAMoWo/kUC9D/23eElKEQSQAnjcgX5rhNAOM5ewY4xDEBXMsQTszwDQK1fna3KGwZApO3vOde2BUAn9n7SRfMBQDu+GR7DaQlAHS52mduAEUBOI4lxk0IEQOZZwCnb6vY/NEs9xaiVEEBMOZzbPwAQQBAtYaOwyg9AXFTPjPg9FUAZVZU9QXcQQPA1lk2BqgRAdY1BpCZkCEAUGMm/BisKQG5++ciEMgBA8YqXN/p5AEA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"5004\"},\"selection_policy\":{\"id\":\"5003\"}},\"id\":\"4989\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"5007\"},{\"id\":\"5033\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"5006\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"4972\",\"type\":\"BasicTicker\"},{\"attributes\":{\"axis\":{\"id\":\"4971\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4974\",\"type\":\"Grid\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"vNau47j9yD+MLP/lUx7nP/B2k6Qs+NE/QPpf4kjmmD93piyEtafkP8it6Q+Qq8Y/IjnhmTwY7j8QRR45lYbuP6iyiK+RRu0/+lP7Ha6w1z8AhPzzt6ePPzWwUiHJtO0/CBBsfF5n2z8JB0oW1u7uPwrc64/51e4/kSdNfNpL6z9GEXIqQNjSP1z8hvFwpdg/EusS+YI86z9q2uE7c0jUP4CG9jXwscU//ymU4VDR4T9oeWDb+vTtP1VWuEfgReY/KhtI7PA94j9Idq8Gj+C4Pz5Ar6IjruM/GkFvaYWu7z/kDRHkRe7BP6dgLhEoluA/v0pKsXAT7D+xCWVjYLTnP3bmi/TzTeY/w+zV5r965j/gLGws5wHXP7RAX3I1ytI/CNutXUnm6T8cWwbtcuzpP6YwcnIOv+s/bI1TQEQ57T8uTwW86lzgP8D2guZrDOA/wO/GVKKL6T+8pEgogczkP39Olj2DduY/qvwzMCJ36T9d5Yt77HrsPySPOHC2odU/6oOhFI0J2D/g+gdOMw+4P54nsFpFgeI/AHQAlQZnoj90hj2hW8zdP7Jn7UdYXeE/xCK2HrFW0j/oDXEnG+jiP4A8CCB1O58/oER8Qk0foz88izZpvlLqP97xSwxdDdc/pPY51YRDwD/kvE9/N7bgP1gPFoXJo+g/5Ktx/wWgyz9RPdQBuO7jP2Da8dpU2bU/8LhwzAZ2qj/1OaVt2wDhP+1MzAbiTOE/bqyNZNNl5D9q7ofkIzznP0XO4CMuOu8/0b7bToiF4D80G1SgUavUP4FkmFEqcuk/uNl3y1BV0T+gzPOVGxjcP6gMB6i3FbQ/4D6Yl4v1mT9c0FIMBM7uPxGYtmFZwOo/HgbBsmtF5j80Lnf4SCzaPyS3pB6CLsY/7N8TByEGxD+iFevJ+gPQP1EgYMxDk+E/OUG7RPjd5j9zC70/ViDlP5wtG91v6tE/0hgRokGO7j/K4aD+2ZznPwkhvrVEveE/bY/rYjeT4z/+uGE4utraPwh9Dimmtc8/bOhEoUHI1j9xDHt8RkDoPwBthVdVeo0/mATwwu+2vT8=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"WN2CusR4BEAnZeoHicIQQAMppcyqlgZAqhXgPs49AEDew78cV50PQAlQk6viBQRA8rrv/ih1E0BZbwb11J8TQBYjWn0UJBNA2wWWvwDNCEBA5ZKlrAUAQHw/9Z6zThNA3MM+R6Q8CkDDFHS/JcgTQMeTZomIvhNAbveRESFgEkA2Yv63Ue0GQFGCEmenKwlA88TfRDJaEkCmun1utHsHQL2B2Jye1QNAaNOOXYlrDUAl9amrhmcTQAG9ou3UbhBAAvhUiYy/DUCTj9fTbwsCQCR9669R3A5AcjKFAkYSFEBImx2JShsDQHC1cpLJdwxA4VJImE+tEkDK+lBwj/wQQLUHp4P0cRBA8E8UQkeDEEDo1hNOaokIQBqx7Jzj5wZA4GRQ9NrVEUDLKk8CPdgRQKGzxNatjBJAmc/tlO8eE0AlDA11hEsMQGL4489CDQxA8hstncyyEUDo3alozLkPQJIVrs+jgRBA0vfMHd+qEUAxY/MsVNUSQN9hAnM4AQhAbtLyuF7vCEA2xHFbMvcBQNivt3Wf8w1AwjQixHeHAEC8I+VysCkLQHhVzr3XEQ1As8XJwze7BkBrbOs9KEMOQHWsn/f8ZABAKBItHmCQAEB1tqTpy/8RQKjaZdvYjQhA9751zsbIAkDyem9TlZAMQLKf4GskWRFAGyaAsCX7BECounYURA4PQBaWun99wAFAt0ELrTDrAECncDuyT8oMQLmBJl8cBQ1A5gg3l2JqD0BgTd9hEM4QQGmNbpZI5RNAs18aGu5qDEAYqxwo8KEHQI8IS0bzqBFANLEBPbBXBkCeVczo/IAKQL/oY17UlAFASslqn1xEAEAIMD6ddLsTQHfzyIguKhJA3hRb2KduEEBidzLYy8IJQMard6a07QNASLVfp+aCA0DVU2vtPNUFQMRZj5iLOw1ABvDpxqWpEECAXgXAovoPQKEUZ9VakQZAPDUVnMyiE0CWkUuIdvMQQI+RiRAIXA1AIkpkHX/HDkByuT8mQQYKQLd9669SxQVA/sbKch9zCEAdhzwHqTIRQAkuFpLg/f8/9scDzyiDAkA=\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"5030\"},\"selection_policy\":{\"id\":\"5029\"}},\"id\":\"5014\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"axis\":{\"id\":\"4967\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"4970\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"4965\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"4998\"},\"group\":null,\"major_label_policy\":{\"id\":\"4999\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4972\"}},\"id\":\"4971\",\"type\":\"LinearAxis\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"5001\"},\"group\":null,\"major_label_policy\":{\"id\":\"5002\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"4968\"}},\"id\":\"4967\",\"type\":\"LinearAxis\"},{\"attributes\":{\"tools\":[{\"id\":\"4975\"},{\"id\":\"4976\"},{\"id\":\"4977\"},{\"id\":\"4978\"},{\"id\":\"4979\"},{\"id\":\"4980\"},{\"id\":\"5008\"},{\"id\":\"5012\"}]},\"id\":\"4982\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"5003\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Simple linear regression\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"4957\",\"type\":\"Title\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"5032\"},\"renderers\":[{\"id\":\"4994\"}]},\"id\":\"5012\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"5029\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"4968\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"4963\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"5004\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"4961\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"5030\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"4959\",\"type\":\"DataRange1d\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4992\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"4980\",\"type\":\"HelpTool\"}],\"root_ids\":[\"5034\",\"5102\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"5ed52e6b-f512-45e8-9815-427d01f8f61c\",\"root_ids\":[\"5034\"],\"roots\":{\"5034\":\"e413e117-7a9b-46ee-92e0-cfa4f96b72d9\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 124,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "5034"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = scatter(x, y, title=\"Simple linear regression\")\n",
    "p.line(x, w_1 * x + w_0, line_width=6, line_alpha=0.6, color=\"red\", legend_label=r\"Learned model\")\n",
    "plot = (center_plot(p))\n",
    "plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observing our solution interactively\n",
    "\n",
    "In the following code, you can observe the behavior of the model by playing interactively with the properties of the original problem. \n",
    "\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> Note that each change in the properties of the original problem requires to run the gradient descent algorithm entirely each time.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='3324'>\n",
       "  <div class=\"bk-root\" id=\"16ed7572-0023-4bc8-85e2-380038ae8259\" data-root-id=\"3324\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"d3cfb4ea-28ae-4585-abff-fcd42b594dc4\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"3304\"},\"glyph\":{\"id\":\"3305\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"3307\"},\"nonselection_glyph\":{\"id\":\"3306\"},\"view\":{\"id\":\"3309\"}},\"id\":\"3308\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"3271\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAALV+JTy1fqU8ED74PLV+JT1i3k49ED54Pd7OkD21fqU9jC66PWLezj05juM9ED74PfN2Bj7ezhA+yiYbPrV+JT6g1i8+jC46PneGRD5i3k4+TjZZPjmOYz4k5m0+ED54Pv1KgT7zdoY+6aKLPt7OkD7U+pU+yiabPr9SoD61fqU+q6qqPqDWrz6WArU+jC66PoFavz53hsQ+bbLJPmLezj5YCtQ+TjbZPkNi3j45juM+L7roPiTm7T4aEvM+ED74PgVq/T79SgE/+OADP/N2Bj/uDAk/6aILP+Q4Dj/ezhA/2WQTP9T6FT/PkBg/yiYbP8W8HT+/UiA/uugiP7V+JT+wFCg/q6oqP6VALT+g1i8/m2wyP5YCNT+RmDc/jC46P4bEPD+BWj8/fPBBP3eGRD9yHEc/bbJJP2dITD9i3k4/XXRRP1gKVD9ToFY/TjZZP0jMWz9DYl4/PvhgPzmOYz80JGY/L7poPylQaz8k5m0/H3xwPxoScz8VqHU/ED54PwrUej8Fan0/AACAPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"UNy+vW3jfD3NFZ69MtBkvW6jfz0roPw9kOwWvHyk0byyUbE93J23Pd11YT2YWMw8MFHSPfPbVz4cv989qqvLPS0beT6Dm28+76MmPoc0TD7qdhg+1VXxPf/HjD5B55M+y699PvrXmT7RFLQ+QaK2PhX9hz5rG5o+gBHDPp5eij4VUcI+ul6YPhCkgD5HLso+CdOnPt1f8D5T+NU+pzLYPhPFtD4eZ7M+qNDYPu8Q1T7YTd8+TKrMPj4i3j589g4/qu/dPjxKBj9JW+4+d88EP5h0Cj9WIA8/wEznPvoDBz8c6w4/deENPxZ5Hj9H/SQ/MtoTP0Z/ND8TWx0/KcIhPyJEFD8yySk/J9USP5X5Kz8T1Ds/sz4gP02FIz8n4D4/j0RAPyWIKj/vBio/PIk5Pz0xRz8zwFk/y3ddP0dDWj8sTjo/5RNWP3sAQD//ekA/IClKP3vwSj9ti2w/P4poPy1QYz9wBX0/zSFiP1q7XT8A1Xc/8bNaP8cohj8gxGM/Wx6IP6Blcz9HQII/f/9pPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"3294\"},\"selection_policy\":{\"id\":\"3293\"}},\"id\":\"3279\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3283\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"3268\",\"type\":\"SaveTool\"},{\"attributes\":{\"tools\":[{\"id\":\"3265\"},{\"id\":\"3266\"},{\"id\":\"3267\"},{\"id\":\"3268\"},{\"id\":\"3269\"},{\"id\":\"3270\"},{\"id\":\"3298\"},{\"id\":\"3302\"}]},\"id\":\"3272\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"3258\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3282\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"3297\"},{\"id\":\"3323\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"3296\",\"type\":\"Legend\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"3279\"},\"glyph\":{\"id\":\"3280\"},\"group\":null,\"hover_glyph\":{\"id\":\"3282\"},\"muted_glyph\":{\"id\":\"3283\"},\"nonselection_glyph\":{\"id\":\"3301\"},\"selection_glyph\":{\"id\":\"3300\"},\"view\":{\"id\":\"3285\"}},\"id\":\"3284\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"3279\"}},\"id\":\"3285\",\"type\":\"CDSView\"},{\"attributes\":{\"overlay\":{\"id\":\"3271\"}},\"id\":\"3267\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"3284\"}]},\"id\":\"3297\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"3266\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"3363\"},\"target\":{\"id\":\"3363\"}},\"code\":\"\\n    var value = source['readonly'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['disabled'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set disabled on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['disabled'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11527165888,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"3378\",\"type\":\"CustomJS\"},{\"attributes\":{\"children\":[{\"id\":\"3325\"},{\"id\":\"3326\"},{\"id\":\"3364\"}],\"margin\":[0,0,0,0],\"name\":\"Row00246\"},\"id\":\"3324\",\"type\":\"Row\"},{\"attributes\":{\"client_comm_id\":\"2a856433f3d64fd691e6509d72618be2\",\"comm_id\":\"9bfd010d8f1e4dbab67434cc3fd65983\",\"plot_id\":\"3324\"},\"id\":\"3406\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00217\",\"sizing_mode\":\"stretch_width\"},\"id\":\"3325\",\"type\":\"Spacer\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} 1.0x^1 + 0.0$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"3288\"},\"group\":null,\"major_label_policy\":{\"id\":\"3289\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"3262\"}},\"id\":\"3261\",\"type\":\"LinearAxis\"},{\"attributes\":{\"axis\":{\"id\":\"3257\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"3260\",\"type\":\"Grid\"},{\"attributes\":{\"source\":{\"id\":\"3304\"}},\"id\":\"3309\",\"type\":\"CDSView\"},{\"attributes\":{\"children\":[{\"id\":\"3327\"},{\"id\":\"3363\"}],\"margin\":[0,0,0,0],\"name\":\"Column00244\"},\"id\":\"3326\",\"type\":\"Column\"},{\"attributes\":{},\"id\":\"3293\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Linear regression problem\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"3247\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"3319\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"3363\"},\"target\":{\"id\":\"3363\"}},\"code\":\"\\n    var value = source['disabled'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['readonly'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set readonly on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['readonly'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11527165888,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"3379\",\"type\":\"CustomJS\"},{\"attributes\":{\"children\":[{\"id\":\"3329\"},{\"id\":\"3330\"},{\"id\":\"3331\"},{\"id\":\"3332\"},{\"id\":\"3333\"},{\"id\":\"3334\"},{\"id\":\"3335\"}],\"margin\":[5,5,5,5],\"min_width\":300,\"name\":\"Problem parameters\",\"width\":300},\"id\":\"3328\",\"type\":\"Column\"},{\"attributes\":{\"children\":[{\"id\":\"3328\"},{\"id\":\"3336\"}],\"margin\":[0,0,0,0],\"name\":\"Row00241\"},\"id\":\"3327\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"3265\",\"type\":\"PanTool\"},{\"attributes\":{\"margin\":[5,10,5,10],\"name\":\"\",\"text\":\"<b>Problem parameters</b>\"},\"id\":\"3329\",\"type\":\"Div\"},{\"attributes\":{\"labels\":[\"Ground truth\"],\"margin\":[5,10,5,10]},\"id\":\"3332\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"labels\":[\"Error\"],\"margin\":[5,10,5,10]},\"id\":\"3333\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"3300\",\"type\":\"Circle\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"3257\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"3260\"},{\"id\":\"3264\"},{\"id\":\"3296\"}],\"height\":450,\"left\":[{\"id\":\"3261\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"3284\"},{\"id\":\"3308\"}],\"title\":{\"id\":\"3247\"},\"toolbar\":{\"id\":\"3272\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"3249\"},\"x_scale\":{\"id\":\"3253\"},\"y_range\":{\"id\":\"3251\"},\"y_scale\":{\"id\":\"3255\"}},\"id\":\"3246\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"3322\"},\"renderers\":[{\"id\":\"3284\"}]},\"id\":\"3302\",\"type\":\"BoxSelectTool\"},{\"attributes\":{},\"id\":\"3291\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"end\":1000,\"margin\":[5,10,5,10],\"start\":10,\"title\":\"Number observations\",\"value\":100},\"id\":\"3330\",\"type\":\"Slider\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":0.0,\"step\":0.1,\"title\":\"Noise level\",\"value\":0.1},\"id\":\"3331\",\"type\":\"Slider\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"3284\"}],\"tooltips\":null},\"id\":\"3298\",\"type\":\"HoverTool\"},{\"attributes\":{\"label\":{\"value\":\"Learned model\"},\"renderers\":[{\"id\":\"3308\"}]},\"id\":\"3323\",\"type\":\"LegendItem\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 1\",\"value\":1.0},\"id\":\"3334\",\"type\":\"Slider\"},{\"attributes\":{},\"id\":\"3262\",\"type\":\"BasicTicker\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABn0LLjOZV0P2fQsuM5lYQ/mjiM1dbfjj9n0LLjOZWUP4GEn1yIupk/mjiM1dbfnj9adjynkgKiP2fQsuM5laQ/dCopIOEnpz+BhJ9ciLqpP47eFZkvTaw/mjiM1dbfrj9USQEJP7mwP1p2PKeSArI/YaN3ReZLsz9n0LLjOZW0P2397YGN3rU/dCopIOEntz96V2S+NHG4P4GEn1yIurk/h7Ha+tsDuz+O3hWZL028P5QLUTeDlr0/mjiM1dbfvj/QsuM5lRTAP1RJAQk/ucA/198e2OhdwT9adjynkgLCP90MWnY8p8I/YaN3ReZLwz/kOZUUkPDDP2fQsuM5lcQ/6mbQsuM5xT9t/e2Bjd7FP/GTC1E3g8Y/dCopIOEnxz/3wEbviszHP3pXZL40ccg//u2Bjd4VyT+BhJ9ciLrJPwQbvSsyX8o/h7Ha+tsDyz8KSPjJhajLP47eFZkvTcw/EXUzaNnxzD+UC1E3g5bNPxeibgYtO84/mjiM1dbfzj8ez6mkgITPP9Cy4zmVFNA/En5yIepm0D9USQEJP7nQP5UUkPCTC9E/198e2Ohd0T8Zq62/PbDRP1p2PKeSAtI/nEHLjudU0j/dDFp2PKfSPx/Y6F2R+dI/YaN3ReZL0z+ibgYtO57TP+Q5lRSQ8NM/JQUk/ORC1D9n0LLjOZXUP6mbQcuO59Q/6mbQsuM51T8sMl+aOIzVP2397YGN3tU/r8h8aeIw1j/xkwtRN4PWPzJfmjiM1dY/dCopIOEn1z+19bcHNnrXP/fARu+KzNc/OYzV1t8e2D96V2S+NHHYP7wi86WJw9g//u2Bjd4V2T8/uRB1M2jZP4GEn1yIutk/wk8uRN0M2j8EG70rMl/aP0bmSxOHsdo/h7Ha+tsD2z/JfGniMFbbPwpI+MmFqNs/TBOHsdr62z+O3hWZL03cP8+ppICEn9w/EXUzaNnx3D9SQMJPLkTdP5QLUTeDlt0/1tbfHtjo3T8Xom4GLTveP1lt/e2Bjd4/mjiM1dbf3j/cAxu9KzLfPx7PqaSAhN8/X5o4jNXW3z/QsuM5lRTgP3EYq62/PeA/En5yIepm4D+z4zmVFJDgP1RJAQk/ueA/9K7IfGni4D+VFJDwkwvhPzZ6V2S+NOE/198e2Ohd4T94ReZLE4fhPxmrrb89sOE/uRB1M2jZ4T9adjynkgLiP/vbAxu9K+I/nEHLjudU4j89p5ICEn7iP90MWnY8p+I/fnIh6mbQ4j8f2OhdkfniP8A9sNG7IuM/YaN3ReZL4z8BCT+5EHXjP6JuBi07nuM/Q9TNoGXH4z/kOZUUkPDjP4WfXIi6GeQ/JQUk/ORC5D/GautvD2zkP2fQsuM5leQ/CDZ6V2S+5D+pm0HLjufkP0kBCT+5EOU/6mbQsuM55T+LzJcmDmPlPywyX5o4jOU/zZcmDmO15T9t/e2Bjd7lPw5jtfW3B+Y/r8h8aeIw5j9QLkTdDFrmP/GTC1E3g+Y/kfnSxGGs5j8yX5o4jNXmP9PEYay2/uY/dCopIOEn5z8VkPCTC1HnP7X1twc2euc/Vlt/e2Cj5z/3wEbvisznP5gmDmO19ec/OYzV1t8e6D/a8ZxKCkjoP3pXZL40ceg/G70rMl+a6D+8IvOlicPoP12Iuhm07Og//u2Bjd4V6T+eU0kBCT/pPz+5EHUzaOk/4B7Y6F2R6T+BhJ9ciLrpPyLqZtCy4+k/wk8uRN0M6j9jtfW3BzbqPwQbvSsyX+o/pYCEn1yI6j9G5ksTh7HqP+ZLE4ex2uo/h7Ha+tsD6z8oF6JuBi3rP8l8aeIwVus/auIwVlt/6z8KSPjJhajrP6utvz2w0es/TBOHsdr66z/teE4lBSTsP47eFZkvTew/LkTdDFp27D/PqaSAhJ/sP3APbPSuyOw/EXUzaNnx7D+y2vrbAxvtP1JAwk8uRO0/86WJw1ht7T+UC1E3g5btPzVxGKutv+0/1tbfHtjo7T92PKeSAhLuPxeibgYtO+4/uAc2eldk7j9Zbf3tgY3uP/rSxGGstu4/mjiM1dbf7j87nlNJAQnvP9wDG70rMu8/fWniMFZb7z8ez6mkgITvP780cRirre8/X5o4jNXW7z8AAAAAAADwPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[200]},\"y\":{\"__ndarray__\":\"uOvbu9iO3LqYSFs7KDYJPCiaWzwV/5Y8FjHAPBZj6TyLSgk9jOMdPYx8Mj2NFUc9ja5bPY1HcD1GcII9xryMPUYJlz3IVaE9SKKrPcjutT1IO8A9yIfKPUjU1D3IIN89SG3pPci58z1IBv49ZCkEPqRPCT7ldQ4+JJwTPmXCGD6k6B0+5Q4jPiU1KD5lWy0+pYEyPuSnNz4lzjw+ZfRBPqUaRz7lQEw+JmdRPmWNVj6ms1s+5dlgPiYAZj5mJms+pkxwPuZydT4mmXo+Zr9/PtNygj70BYU+E5mHPjMsij5Tv4w+c1KPPpTlkT6zeJQ+0wuXPvOemT4UMpw+NMWePlNYoT5z66M+lH6mPrQRqT7UpKs+8zeuPhTLsD40XrM+VPG1PnSEuD6TF7s+tKq9PtQ9wD700MI+FGTFPjX3xz5Uiso+dB3NPpSwzz61Q9I+1dbUPvRp1z4U/dk+NZDcPlUj3z51tuE+lEnkPrTc5j7Vb+k+9QLsPhWW7j40KfE+VbzzPnVP9j6V4vg+tXX7PtUI/j77TQA/i5cBPxvhAj+rKgQ/O3QFP8y9Bj9bBwg/61AJP3uaCj8L5As/nC0NPyt3Dj+7wA8/SwoRP9tTEj9snRM/++YUP4swFj8cehc/q8MYPzwNGj/LVhs/W6AcP+zpHT97Mx8/DH0gP5vGIT8rECM/vFkkP0ujJT/c7CY/bDYoP/t/KT+MySo/GxMsP6xcLT88pi4/y+8vP1w5MT/sgjI/fMwzPwwWNT+bXzY/LKk3P7zyOD9MPDo/3IU7P2vPPD/8GD4/jGI/PxysQD+s9UE/PT9DP8yIRD9c0kU/7BtHP3xlSD8Nr0k/nPhKPyxCTD+9i00/TNVOP90eUD9saFE//LFSP437Uz8cRVU/rY5WPzzYVz/MIVk/XWtaP+y0Wz99/lw/DUheP5yRXz8t22A/vCRiP01uYz/dt2Q/bAFmP/1KZz+MlGg/Hd5pP60naz88cWw/zbptP10Ebz/tTXA/fZdxPwzhcj+dKnQ/LXR1P729dj9NB3g/3VB5P22aej/943s/jS19Px13fj8=\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[200]}},\"selected\":{\"id\":\"3320\"},\"selection_policy\":{\"id\":\"3319\"}},\"id\":\"3304\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"axis\":{\"id\":\"3261\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"3264\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"3301\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"3292\",\"type\":\"AllLabels\"},{\"attributes\":{\"code\":\"def linear(\\n        coefficients: jnp.ndarray,\\n        n_observations: int = 100) -> [jnp.ndarray, jnp.ndarray]:\\n    x = jnp.linspace(0, 1, n_observations)\\n    y = coefficients[0] * x + coefficients[1]\\n    return x, \",\"filename\":\"\",\"height\":150,\"js_property_callbacks\":{\"change:disabled\":[{\"id\":\"3379\"}],\"change:readonly\":[{\"id\":\"3378\"}]},\"language\":\"python\",\"margin\":[5,10,5,10],\"sizing_mode\":\"stretch_both\",\"theme\":\"chaos\"},\"id\":\"3363\",\"type\":\"panel.models.ace.AcePlot\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 0\",\"value\":0.0},\"id\":\"3335\",\"type\":\"Slider\"},{\"attributes\":{\"children\":[{\"id\":\"3246\"}],\"margin\":[0,0,0,0],\"name\":\"Row00238\"},\"id\":\"3336\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"3320\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"3249\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"3255\",\"type\":\"LinearScale\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00245\",\"sizing_mode\":\"stretch_width\"},\"id\":\"3364\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"3289\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"3294\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"3288\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3306\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"3269\",\"type\":\"ResetTool\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3307\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"3253\",\"type\":\"LinearScale\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"3322\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"3270\",\"type\":\"HelpTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3280\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"3251\",\"type\":\"DataRange1d\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} x$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"3291\"},\"group\":null,\"major_label_policy\":{\"id\":\"3292\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"3258\"}},\"id\":\"3257\",\"type\":\"LinearAxis\"},{\"attributes\":{\"line_alpha\":0.7,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3305\",\"type\":\"Line\"}],\"root_ids\":[\"3324\",\"3406\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"d3cfb4ea-28ae-4585-abff-fcd42b594dc4\",\"root_ids\":[\"3324\"],\"roots\":{\"3324\":\"16ed7572-0023-4bc8-85e2-380038ae8259\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Column\n",
       "        [0] Row\n",
       "            [0] Param(RegressionLinearSolver, name='Problem parameters', widgets={})\n",
       "            [1] ParamMethod(method, _pane=Bokeh)\n",
       "        [1] Ace(height=150, language='python', sizing_mode='stretch_both', theme='chaos', value='def linear(\\n        coef...)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "3324"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from cml.tasks import RegressionLinearSolver\n",
    "explorer = RegressionLinearSolver()\n",
    "def solve(x, y):\n",
    "    w_1, w_0 = gradient_descent(x, y, n_iter, lr)\n",
    "    x_predict = np.linspace(np.min(x), np.max(x), 200)\n",
    "    y_model = w_1 * x_predict + w_0\n",
    "    return np.array(x_predict), np.array(y_model) #np.array(jnp.zeros(y_model.shape))\n",
    "\n",
    "explorer.solve = solve\n",
    "explorer.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further\n",
    "> **Exercise 1 \\[Learning rate\\]**: Experiment with different learning rates (e.g., 0.001, 0.01, 0.1, 1) and observe how they affect the convergence of the gradient descent algorithm. Plot the convergence history (cost function value vs. iteration) for each learning rate.\n",
    "\n",
    "> **Exercise 2 \\[Initialization\\]**: Experiment with different initializations of the coefficients (zeros, random values, etc.) and observe their impact on the convergence of the gradient descent algorithm.\n",
    "\n",
    "> **Exercise 3 \\[Variants\\]**: Implement and compare different variants of gradient descent, such as stochastic gradient descent (SGD) and mini-batch gradient descent. Analyze their convergence properties and computational efficiency.\n",
    "\n",
    "> **Exercise 4 \\[Regularization\\]**: Implement Lasso and Ridge regression with gradient descent by incorporating the regularization terms into the cost function and gradient calculations. Compare their performance with the standard linear regression implementation and analyze their impact on the learned coefficients.\n",
    "\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Discovering automatic differentiation - `JAX`\n",
    "\n",
    "The previous implementation required us to perform manual differentiation of our loss function to understand how to update the parameters. However, large developments have been made in the field of **automatic differentiation**. \n",
    "\n",
    "The recent library [JAX](https://github.com/google/jax) extends NumPy with this automatic differentiation feature (*autograd*), while providing a functional approach to numerical computing, allowing for easy gradient computation and just-in-time (JIT) compilation. Its ability to handle complex and custom gradients makes JAX particularly well-suited for advanced research projects. Similar to NumPy, we strongly encourage you to learn JAX through the set of [tutorials](https://jax.readthedocs.io/en/latest/), but we will also provide here the explanation for all of the functions that we will use in this first exercise. After that, we will assume that knowledge of JAX should be found online. \n",
    "\n",
    "Note that `JAX` has been thought as an extension of `NumPy`, therefore an extremely large portion of its API simply mirrors the `NumPy` functions by adding automatic differentiation features to it.\n",
    "\n",
    "**Used functions**\n",
    "- `random.PRNGKey`: function to generate a key for the pseudorandom number generator. [Documentation](https://jax.readthedocs.io/en/latest/_autosummary/jax.random.PRNGKey.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax.numpy as jnp\n",
    "from jax import grad, jit, random\n",
    "key = random.PRNGKey(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataset.\n",
    "\n",
    "We can simply keep our previous approach to generating the dataset but we will rely on `jnp.ndarray` instead of `np.ndarray`. We also provide the code for splitting the dataset between a `training` and `validation` dataset, as discussed in the course\n",
    "\n",
    "**Used functions**\n",
    "- `random.split`: function to split a PRNGKey into a list of subkeys [Documentation](https://jax.readthedocs.io/en/latest/_autosummary/jax.random.split.html)\n",
    "- `random.uniform`: function to generate an array of uniformly distributed random numbers [Documentation](https://jax.readthedocs.io/en/latest/_autosummary/jax.random.uniform.html)\n",
    "- `random.normal`: function to generate an array of normally distributed random numbers [Documentation](https://jax.readthedocs.io/en/latest/_autosummary/jax.random.normal.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "true_w0 = 2\n",
    "true_w1 = 3\n",
    "key_0, key_1, key = random.split(key, 3)\n",
    "x = random.uniform(key_0, (100,))\n",
    "y = true_w0 + true_w1 * x + (random.normal(key_1, (100,)) * 0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Splitting our dataset\n",
    "train_size = int(0.8 * len(x))\n",
    "x_train, x_valid = x[:train_size], x[train_size:]\n",
    "y_train, y_valid = y[:train_size], y[train_size:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loss function\n",
    "\n",
    "Similarly Define the cost function and its gradient.\n",
    "\n",
    "**Used functions**\n",
    "- `jnp.mean`: function to compute the mean of an array [Documentation](https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.mean.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_function(x, y, w_1, w_0):\n",
    "    y_pred = x * w_1 + w_0\n",
    "    residuals = y_pred - y\n",
    "    return jnp.mean(residuals**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**Used functions**\n",
    "- `jit`: function to compile a function for faster execution [Documentation](https://jax.readthedocs.io/en/latest/jit.html)\n",
    "- `grad`: function to compute the gradient of a function [Documentation](https://jax.readthedocs.io/en/latest/jax.html#jax.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "grad_loss_function = jit(grad(loss_function, argnums=[2, 3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implement the gradient descent algorithm.\n",
    "\n",
    "Note that nowhere in the code do we need to explicitly define the gradients of the different variables. Yet, the optimization will be performed adequately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gradient Descent coefficients: [3.0639799] [1.9661584]\n"
     ]
    }
   ],
   "source": [
    "def gradient_descent(key, x, y, lr=0.05, n_iter=1000):\n",
    "    m = x.shape\n",
    "    k_0, k_1 = random.split(key, 2)\n",
    "    w_0 = random.normal(k_0, (1,))\n",
    "    w_1 = random.normal(k_1, (1,))\n",
    "    loss_history = []\n",
    "    for _ in range(n_iter):\n",
    "        gradients = grad_loss_function(x, y, w_1, w_0)\n",
    "        #print(gradients[0])\n",
    "        w_1 -= lr * gradients[0]\n",
    "        w_0 -= lr * gradients[1]\n",
    "        loss_history.append(loss_function(x, y, w_1, w_0))\n",
    "    return w_1, w_0, loss_history\n",
    "\n",
    "key_gd, key = random.split(key, 2)\n",
    "w_1, w_0, loss_history = gradient_descent(key_gd, x_train, y_train)\n",
    "print(\"Gradient Descent coefficients:\", w_1, w_0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate our performances\n",
    "\n",
    "Thanks to our previous split of the dataset, we are now able to evaluate the performances of our model on *unseen data*, which is the overarching goal of building such machine learning models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE for Gradient Descent: 0.03851323\n"
     ]
    }
   ],
   "source": [
    "def mean_squared_error(y_true, y_pred):\n",
    "    return np.mean((y_true - y_pred)**2)\n",
    "\n",
    "y_valid_gd = w_1 * x_valid + w_0\n",
    "mse_gd = mean_squared_error(y_valid, y_valid_gd)\n",
    "print(\"MSE for Gradient Descent:\", mse_gd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the results.\n",
    "\n",
    "As previously we can witness our results, and also evaluate our solution with our interactive solver."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='5458'>\n",
       "  <div class=\"bk-root\" id=\"effa124b-1c03-4a25-b9e9-2781b393a17f\" data-root-id=\"5458\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"0061f68f-28c7-4c39-9b31-d7239c9ebb50\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{\"source\":{\"id\":\"5438\"}},\"id\":\"5443\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"5402\",\"type\":\"SaveTool\"},{\"attributes\":{\"label\":{\"value\":\"Learned model\"},\"renderers\":[{\"id\":\"5442\"}]},\"id\":\"5457\",\"type\":\"LegendItem\"},{\"attributes\":{\"line_alpha\":0.6,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5439\",\"type\":\"Line\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"5431\"},{\"id\":\"5457\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"5430\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"5399\",\"type\":\"PanTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"5405\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5416\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"5403\",\"type\":\"ResetTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"7N0EP9Q2/z5g7bU++IQEPtBIGD7odLA+gJ3xPYCBGz7wBCQ+wCzBPFqjWz/kFmw/YGWPPUT3XT/EQYo+BPgjP/wkxj4gLeU+3D7nPiA43T3w0Qg/JmI8P+iTiD66Eis/rAqlPjgdYz6e7UI/hHlZP+gBXT5c2ME+DvAoP8DoaD7WWTI/rNGfPtJJAz/wCi0+cl4uP+DEBj9AQes+NKKAPrI1GD9OHUQ/cHTMPuwfEj+cmpw+mqxfP7iW9T50PCQ/EphXP5jmIj6WUR8/VO8xP7JYEz/8XpU+WGMPPqSE7z7exh4/CNE6PxIaTD8atGs/6oMGPyxhaD+cBjg/MI6tPvCl0T0A6A47RtlMPyYjIj+SbAc/IHYnPyg2cz9MZK8+mI4SP3RGNT/Oa2M/YLLhPlAHHD/iDwI/2BIpP7Aqbz4MGXw/VgtwP5agUD84HFc+2K94PtSOZD9YSTk/YJp7PdxJBT9klL4+qD1zPuRAzj78Cw4/cGG5PgAfxz0wOc4+6HeePtA+Uz4gIrw95GT5Pg==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"jTtpQEHUQ0D0K0ZAA2QdQOYtDUAko1ZAgjIVQNlw+j99hSxAxcYDQK1ujED+wJRA+l7+P+r5kkA2QS9AHpR8QAn8RkBm9ldAKBVZQInyCECdtFRAqsmIQP70HkDJNYNA+fI+QG8hL0CgXIdAk1CHQHbqLUAw3ktAozOBQDyzI0DOxoZAkl06QOI3ZECT6zZACK5xQJ2VWkCbVVtARDMuQJlhgECBY4ZA8ftiQNhSZECXMzxAoIGSQChOakABt3JAAiWLQMNcKUDCI3tAQjWOQFMdd0CcdCdAOQEXQGiuSkA50IhAeIyRQEJIjEB6nJRApXdfQOofmEClpolA3jhGQAeWEkB0K/Q/69uNQH0GiUDOpGFAX79yQDM3pUBhiUpAQzloQMdniUDZ/JNAr+xMQOzockASOWZA/aODQGtvIUCw56JAx0ObQH4xkkCP6zhA9r4rQDbBnUBhFIVAsNYfQF6sX0CpQDlAojoqQOaFOUDbe29AFv5YQCCVLkCxf0hAqSk/QHnbKkACNxlADmheQA==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"5428\"},\"selection_policy\":{\"id\":\"5427\"}},\"id\":\"5413\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"5422\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"5456\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"tools\":[{\"id\":\"5399\"},{\"id\":\"5400\"},{\"id\":\"5401\"},{\"id\":\"5402\"},{\"id\":\"5403\"},{\"id\":\"5404\"},{\"id\":\"5432\"},{\"id\":\"5436\"}]},\"id\":\"5406\",\"type\":\"Toolbar\"},{\"attributes\":{\"axis\":{\"id\":\"5391\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"5394\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"5426\",\"type\":\"AllLabels\"},{\"attributes\":{\"source\":{\"id\":\"5413\"}},\"id\":\"5419\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label\":\"x\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"5425\"},\"group\":null,\"major_label_policy\":{\"id\":\"5426\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"5392\"}},\"id\":\"5391\",\"type\":\"LinearAxis\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5441\",\"type\":\"Line\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00312\",\"sizing_mode\":\"stretch_width\"},\"id\":\"5459\",\"type\":\"Spacer\"},{\"attributes\":{},\"id\":\"5425\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"overlay\":{\"id\":\"5405\"}},\"id\":\"5401\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"5434\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"5423\",\"type\":\"AllLabels\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"5456\"},\"renderers\":[{\"id\":\"5418\"}]},\"id\":\"5436\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"7N0EP9Q2/z5g7bU++IQEPtBIGD7odLA+gJ3xPYCBGz7wBCQ+wCzBPFqjWz/kFmw/YGWPPUT3XT/EQYo+BPgjP/wkxj4gLeU+3D7nPiA43T3w0Qg/JmI8P+iTiD66Eis/rAqlPjgdYz6e7UI/hHlZP+gBXT5c2ME+DvAoP8DoaD7WWTI/rNGfPtJJAz/wCi0+cl4uP+DEBj9AQes+NKKAPrI1GD9OHUQ/cHTMPuwfEj+cmpw+mqxfP7iW9T50PCQ/EphXP5jmIj6WUR8/VO8xP7JYEz/8XpU+WGMPPqSE7z7exh4/CNE6PxIaTD8atGs/6oMGPyxhaD+cBjg/MI6tPvCl0T0A6A47RtlMPyYjIj+SbAc/IHYnPyg2cz9MZK8+mI4SP3RGNT/Oa2M/YLLhPlAHHD/iDwI/2BIpP7Aqbz4MGXw/VgtwP5agUD84HFc+2K94PtSOZD9YSTk/YJp7PdxJBT9klL4+qD1zPuRAzj78Cw4/cGG5PgAfxz0wOc4+6HeePtA+Uz4gIrw95GT5Pg==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"CJxjQJ2UX0AGg0NAITYXQBX/GkCmakFA+fcUQAKdG0BePh1ATnUCQLQJk0C2VplAcJALQPDtk0BDyTJA9G57QBS5SUCqm1VAjmZWQAgEE0A7o2ZARBGHQKAkMkAKcIBAcQs9QIJTKUD2kolAljWSQCMoKECLE0hAVT1/QJpvKkCbOYNAXws7QH5mYkC7+B5ALrOBQAkRZUCo71dAuRkvQBFtckBGB4pAzCNMQNLDbUAk0DlAcJWUQN7kW0Bho3tAMn2RQIgHHUAY33dA0BCDQGezbkD5CjdA80oZQKiRWUDWdHdAo3eGQGoWjUDgMJlARt9kQPnql0AEZoVAME5AQGfoEUAChvw/pV+NQM4HekB+kWVA1xt+QAURnEA+AkFAmBhuQFNYhEDPBJZAg0ZUQOpZdUAEdmFA+1d/QGCiK0BMeJ9AetqaQCDSjkAIBydAFnUtQEV0lkCf4YVAKuEJQLbuY0Be00ZAGmosQCbUTEArpGpAntVEQF3mEEAy0UxA8oY6QJBJJkAI2Q9A+FldQA==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"5454\"},\"selection_policy\":{\"id\":\"5453\"}},\"id\":\"5438\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"client_comm_id\":\"3f8082156a314625950b480c5be14186\",\"comm_id\":\"0a4bcbbc602f45a38a653a4d1cefe59d\",\"plot_id\":\"5458\"},\"id\":\"5526\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{\"axis\":{\"id\":\"5395\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"5398\",\"type\":\"Grid\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"5418\"}],\"tooltips\":null},\"id\":\"5432\",\"type\":\"HoverTool\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"5418\"}]},\"id\":\"5431\",\"type\":\"LegendItem\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":6,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5440\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"5404\",\"type\":\"HelpTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"5435\",\"type\":\"Circle\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00313\",\"sizing_mode\":\"stretch_width\"},\"id\":\"5486\",\"type\":\"Spacer\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Linear regression\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"5381\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"5389\",\"type\":\"LinearScale\"},{\"attributes\":{\"children\":[{\"id\":\"5459\"},{\"id\":\"5380\"},{\"id\":\"5486\"}],\"margin\":[0,0,0,0],\"name\":\"Row00316\"},\"id\":\"5458\",\"type\":\"Row\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5414\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"5385\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"5427\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"5383\",\"type\":\"DataRange1d\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"5391\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"5394\"},{\"id\":\"5398\"},{\"id\":\"5430\"}],\"height\":450,\"left\":[{\"id\":\"5395\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"5418\"},{\"id\":\"5442\"}],\"title\":{\"id\":\"5381\"},\"toolbar\":{\"id\":\"5406\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"5383\"},\"x_scale\":{\"id\":\"5387\"},\"y_range\":{\"id\":\"5385\"},\"y_scale\":{\"id\":\"5389\"}},\"id\":\"5380\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5417\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"5453\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"5400\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"axis_label\":\"y\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"5422\"},\"group\":null,\"major_label_policy\":{\"id\":\"5423\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"5396\"}},\"id\":\"5395\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"5428\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"5392\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"5454\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"5413\"},\"glyph\":{\"id\":\"5414\"},\"group\":null,\"hover_glyph\":{\"id\":\"5416\"},\"muted_glyph\":{\"id\":\"5417\"},\"nonselection_glyph\":{\"id\":\"5435\"},\"selection_glyph\":{\"id\":\"5434\"},\"view\":{\"id\":\"5419\"}},\"id\":\"5418\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"5396\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"5387\",\"type\":\"LinearScale\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"5438\"},\"glyph\":{\"id\":\"5439\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"5441\"},\"nonselection_glyph\":{\"id\":\"5440\"},\"view\":{\"id\":\"5443\"}},\"id\":\"5442\",\"type\":\"GlyphRenderer\"}],\"root_ids\":[\"5458\",\"5526\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"0061f68f-28c7-4c39-9b31-d7239c9ebb50\",\"root_ids\":[\"5458\"],\"roots\":{\"5458\":\"effa124b-1c03-4a25-b9e9-2781b393a17f\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Bokeh(Figure)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 135,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "5458"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = scatter(np.array(x), np.array(y), title=\"Linear regression\")\n",
    "p.line(np.array(x), np.array(w_1 * x + w_0), line_width=6, line_alpha=0.6, color=\"red\", legend_label=r\"Learned model\")\n",
    "plot = (center_plot(p))\n",
    "plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {},
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='8182'>\n",
       "  <div class=\"bk-root\" id=\"56efbbaf-82df-4cdf-9502-751c995961b6\" data-root-id=\"8182\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  function embed_document(root) {\n",
       "    var docs_json = {\"0db575ed-8e30-47d4-bbf7-de70b3cf18e2\":{\"defs\":[{\"extends\":null,\"module\":null,\"name\":\"ReactiveHTML1\",\"overrides\":[],\"properties\":[]},{\"extends\":null,\"module\":null,\"name\":\"FlexBox1\",\"overrides\":[],\"properties\":[{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_content\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"align_items\"},{\"default\":\"row\",\"kind\":null,\"name\":\"flex_direction\"},{\"default\":\"wrap\",\"kind\":null,\"name\":\"flex_wrap\"},{\"default\":\"flex-start\",\"kind\":null,\"name\":\"justify_content\"}]},{\"extends\":null,\"module\":null,\"name\":\"GridStack1\",\"overrides\":[],\"properties\":[{\"default\":\"warn\",\"kind\":null,\"name\":\"mode\"},{\"default\":null,\"kind\":null,\"name\":\"ncols\"},{\"default\":null,\"kind\":null,\"name\":\"nrows\"},{\"default\":true,\"kind\":null,\"name\":\"allow_resize\"},{\"default\":true,\"kind\":null,\"name\":\"allow_drag\"},{\"default\":[],\"kind\":null,\"name\":\"state\"}]},{\"extends\":null,\"module\":null,\"name\":\"click1\",\"overrides\":[],\"properties\":[{\"default\":\"\",\"kind\":null,\"name\":\"terminal_output\"},{\"default\":\"\",\"kind\":null,\"name\":\"debug_name\"},{\"default\":0,\"kind\":null,\"name\":\"clears\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationAreaBase1\",\"overrides\":[],\"properties\":[{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"}]},{\"extends\":null,\"module\":null,\"name\":\"NotificationArea1\",\"overrides\":[],\"properties\":[{\"default\":[],\"kind\":null,\"name\":\"notifications\"},{\"default\":\"bottom-right\",\"kind\":null,\"name\":\"position\"},{\"default\":0,\"kind\":null,\"name\":\"_clear\"},{\"default\":[{\"background\":\"#ffc107\",\"icon\":{\"className\":\"fas fa-exclamation-triangle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"warning\"},{\"background\":\"#007bff\",\"icon\":{\"className\":\"fas fa-info-circle\",\"color\":\"white\",\"tagName\":\"i\"},\"type\":\"info\"}],\"kind\":null,\"name\":\"types\"}]},{\"extends\":null,\"module\":null,\"name\":\"Notification\",\"overrides\":[],\"properties\":[{\"default\":null,\"kind\":null,\"name\":\"background\"},{\"default\":3000,\"kind\":null,\"name\":\"duration\"},{\"default\":null,\"kind\":null,\"name\":\"icon\"},{\"default\":\"\",\"kind\":null,\"name\":\"message\"},{\"default\":null,\"kind\":null,\"name\":\"notification_type\"},{\"default\":false,\"kind\":null,\"name\":\"_destroyed\"}]},{\"extends\":null,\"module\":null,\"name\":\"TemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]},{\"extends\":null,\"module\":null,\"name\":\"MaterialTemplateActions1\",\"overrides\":[],\"properties\":[{\"default\":0,\"kind\":null,\"name\":\"open_modal\"},{\"default\":0,\"kind\":null,\"name\":\"close_modal\"}]}],\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"8128\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"8116\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"8149\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis\":{\"id\":\"8115\"},\"coordinates\":null,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"8118\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"8150\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis\":{\"id\":\"8119\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.15,\"grid_line_color\":\"#E0E0E0\",\"group\":null,\"ticker\":null},\"id\":\"8122\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"8120\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"red\"},\"hatch_alpha\":{\"value\":0.5},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8140\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"8109\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"8124\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"8123\",\"type\":\"PanTool\"},{\"attributes\":{\"overlay\":{\"id\":\"8129\"}},\"id\":\"8125\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"8126\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"8127\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"8151\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"8129\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8141\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"8152\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAAAAAAABn0LLjOZV0P2fQsuM5lYQ/mjiM1dbfjj9n0LLjOZWUP4GEn1yIupk/mjiM1dbfnj9adjynkgKiP2fQsuM5laQ/dCopIOEnpz+BhJ9ciLqpP47eFZkvTaw/mjiM1dbfrj9USQEJP7mwP1p2PKeSArI/YaN3ReZLsz9n0LLjOZW0P2397YGN3rU/dCopIOEntz96V2S+NHG4P4GEn1yIurk/h7Ha+tsDuz+O3hWZL028P5QLUTeDlr0/mjiM1dbfvj/QsuM5lRTAP1RJAQk/ucA/198e2OhdwT9adjynkgLCP90MWnY8p8I/YaN3ReZLwz/kOZUUkPDDP2fQsuM5lcQ/6mbQsuM5xT9t/e2Bjd7FP/GTC1E3g8Y/dCopIOEnxz/3wEbviszHP3pXZL40ccg//u2Bjd4VyT+BhJ9ciLrJPwQbvSsyX8o/h7Ha+tsDyz8KSPjJhajLP47eFZkvTcw/EXUzaNnxzD+UC1E3g5bNPxeibgYtO84/mjiM1dbfzj8ez6mkgITPP9Cy4zmVFNA/En5yIepm0D9USQEJP7nQP5UUkPCTC9E/198e2Ohd0T8Zq62/PbDRP1p2PKeSAtI/nEHLjudU0j/dDFp2PKfSPx/Y6F2R+dI/YaN3ReZL0z+ibgYtO57TP+Q5lRSQ8NM/JQUk/ORC1D9n0LLjOZXUP6mbQcuO59Q/6mbQsuM51T8sMl+aOIzVP2397YGN3tU/r8h8aeIw1j/xkwtRN4PWPzJfmjiM1dY/dCopIOEn1z+19bcHNnrXP/fARu+KzNc/OYzV1t8e2D96V2S+NHHYP7wi86WJw9g//u2Bjd4V2T8/uRB1M2jZP4GEn1yIutk/wk8uRN0M2j8EG70rMl/aP0bmSxOHsdo/h7Ha+tsD2z/JfGniMFbbPwpI+MmFqNs/TBOHsdr62z+O3hWZL03cP8+ppICEn9w/EXUzaNnx3D9SQMJPLkTdP5QLUTeDlt0/1tbfHtjo3T8Xom4GLTveP1lt/e2Bjd4/mjiM1dbf3j/cAxu9KzLfPx7PqaSAhN8/X5o4jNXW3z/QsuM5lRTgP3EYq62/PeA/En5yIepm4D+z4zmVFJDgP1RJAQk/ueA/9K7IfGni4D+VFJDwkwvhPzZ6V2S+NOE/198e2Ohd4T94ReZLE4fhPxmrrb89sOE/uRB1M2jZ4T9adjynkgLiP/vbAxu9K+I/nEHLjudU4j89p5ICEn7iP90MWnY8p+I/fnIh6mbQ4j8f2OhdkfniP8A9sNG7IuM/YaN3ReZL4z8BCT+5EHXjP6JuBi07nuM/Q9TNoGXH4z/kOZUUkPDjP4WfXIi6GeQ/JQUk/ORC5D/GautvD2zkP2fQsuM5leQ/CDZ6V2S+5D+pm0HLjufkP0kBCT+5EOU/6mbQsuM55T+LzJcmDmPlPywyX5o4jOU/zZcmDmO15T9t/e2Bjd7lPw5jtfW3B+Y/r8h8aeIw5j9QLkTdDFrmP/GTC1E3g+Y/kfnSxGGs5j8yX5o4jNXmP9PEYay2/uY/dCopIOEn5z8VkPCTC1HnP7X1twc2euc/Vlt/e2Cj5z/3wEbvisznP5gmDmO19ec/OYzV1t8e6D/a8ZxKCkjoP3pXZL40ceg/G70rMl+a6D+8IvOlicPoP12Iuhm07Og//u2Bjd4V6T+eU0kBCT/pPz+5EHUzaOk/4B7Y6F2R6T+BhJ9ciLrpPyLqZtCy4+k/wk8uRN0M6j9jtfW3BzbqPwQbvSsyX+o/pYCEn1yI6j9G5ksTh7HqP+ZLE4ex2uo/h7Ha+tsD6z8oF6JuBi3rP8l8aeIwVus/auIwVlt/6z8KSPjJhajrP6utvz2w0es/TBOHsdr66z/teE4lBSTsP47eFZkvTew/LkTdDFp27D/PqaSAhJ/sP3APbPSuyOw/EXUzaNnx7D+y2vrbAxvtP1JAwk8uRO0/86WJw1ht7T+UC1E3g5btPzVxGKutv+0/1tbfHtjo7T92PKeSAhLuPxeibgYtO+4/uAc2eldk7j9Zbf3tgY3uP/rSxGGstu4/mjiM1dbf7j87nlNJAQnvP9wDG70rMu8/fWniMFZb7z8ez6mkgITvP780cRirre8/X5o4jNXW7z8AAAAAAADwPw==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[200]},\"y\":{\"__ndarray__\":\"mYUxvJOpu7ugPyK6rhmTO6U9HTx37nA8pE+iPAoozDxyAPY8bewPPaLYJD3WxDk9CrFOPTydYz1wiXg90rqGPewwkT0Gp5s9IB2mPTqTsD1VCbs9b3/FPYn1zz2ja9o9veHkPddX7z3vzfk9BSICPhFdBz4fmAw+K9MRPjkOFz5FSRw+U4QhPl+/Jj5t+is+eTUxPoZwNj6Tqzs+oOZAPq4hRj66XEs+yJdQPtTSVT7iDVs+7khgPvyDZT4Iv2o+FvpvPiI1dT4wcHo+PKt/PiRzgj6sEIU+Mq6HPrhLij4+6Yw+xYaPPkwkkj7SwZQ+WF+XPt/8mT5mmpw+7DefPnLVoT75cqQ+gBCnPgauqT6MS6w+E+muPpqGsT4gJLQ+psG2Pi1fuT6z/Ls+Opq+PsA3wT5H1cM+zXLGPlQQyT7brcs+YUvOPufo0D5uhtM+9SPWPnvB2D4BX9s+iPzdPg+a4D6VN+M+G9XlPqFy6D4pEOs+r63tPjVL8D676PI+Q4b1Pskj+D5Pwfo+1V79Pl38/z7xTAE/tZsCP3nqAz87OQU//4cGP8PWBz+FJQk/SXQKPwvDCz/PEQ0/k2AOP1WvDz8Z/hA/3EwSP5+bEz9j6hQ/JTkWP+mHFz+t1hg/byUaPzN0Gz/2whw/uREeP31gHz9AryA/A/4hP8ZMIz+JmyQ/TeolPxA5Jz/Thyg/l9YpP1olKz8ddCw/4MItP6MRLz9nYDA/Kq8xP+39Mj+xTDQ/dJs1PzfqNj/6ODg/vYc5P4HWOj9EJTw/B3Q9P8rCPj+OEUA/UWBBPxSvQj/X/UM/m0xFP16bRj8h6kc/5DhJP6iHSj9r1ks/LiVNP/JzTj+1wk8/eBFRPztgUj/+rlM/wv1UP4VMVj9Im1c/DOpYP844Wj+Sh1s/VdZcPxglXj/cc18/n8JgP2IRYj8mYGM/6K5kP6z9ZT9vTGc/MptoP/bpaT+4OGs/fIdsP0DWbT8CJW8/xnNwP4nCcT9MEXM/EGB0P9KudT+W/XY/Wkx4PxybeT/g6Xo/ozh8P2aHfT8q1n4/dhKAP9i5gD8=\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[200]}},\"selected\":{\"id\":\"8178\"},\"selection_policy\":{\"id\":\"8177\"}},\"id\":\"8162\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_alpha\":0.2,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8165\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"8111\",\"type\":\"LinearScale\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"8221\"},\"target\":{\"id\":\"8221\"}},\"code\":\"\\n    var value = source['readonly'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['disabled'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set disabled on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['disabled'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11756854912,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"8236\",\"type\":\"CustomJS\"},{\"attributes\":{\"tools\":[{\"id\":\"8123\"},{\"id\":\"8124\"},{\"id\":\"8125\"},{\"id\":\"8126\"},{\"id\":\"8127\"},{\"id\":\"8128\"},{\"id\":\"8156\"},{\"id\":\"8160\"}]},\"id\":\"8130\",\"type\":\"Toolbar\"},{\"attributes\":{\"children\":[{\"id\":\"8183\"},{\"id\":\"8184\"},{\"id\":\"8222\"}],\"margin\":[0,0,0,0],\"name\":\"Row00474\"},\"id\":\"8182\",\"type\":\"Row\"},{\"attributes\":{\"client_comm_id\":\"f3cf0af23cc34b10bc1f646d8684fe5b\",\"comm_id\":\"8ab2e6f921694d108c131641336b7907\",\"plot_id\":\"8182\"},\"id\":\"8264\",\"type\":\"panel.models.comm_manager.CommManager\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00445\",\"sizing_mode\":\"stretch_width\"},\"id\":\"8183\",\"type\":\"Spacer\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"8162\"},\"glyph\":{\"id\":\"8163\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"8165\"},\"nonselection_glyph\":{\"id\":\"8164\"},\"view\":{\"id\":\"8167\"}},\"id\":\"8166\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"background_fill_alpha\":0.25,\"background_fill_color\":\"#505050\",\"border_line_alpha\":0.25,\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"8155\"},{\"id\":\"8181\"}],\"label_standoff\":8,\"label_text_color\":\"#E0E0E0\",\"label_text_font\":\"Josefin Sans\",\"label_text_font_size\":\"1.15em\",\"spacing\":8},\"id\":\"8154\",\"type\":\"Legend\"},{\"attributes\":{\"children\":[{\"id\":\"8185\"},{\"id\":\"8221\"}],\"margin\":[0,0,0,0],\"name\":\"Column00472\"},\"id\":\"8184\",\"type\":\"Column\"},{\"attributes\":{\"args\":{\"bidirectional\":true,\"properties\":{\"readonly\":\"disabled\"},\"source\":{\"id\":\"8221\"},\"target\":{\"id\":\"8221\"}},\"code\":\"\\n    var value = source['disabled'];\\n    value = value;\\n    value = value;\\n    try {\\n      var property = target.properties['readonly'];\\n      if (property !== undefined) { property.validate(value); }\\n    } catch(err) {\\n      console.log('WARNING: Could not set readonly on target, raised error: ' + err);\\n      return;\\n    }\\n    try {\\n      target['readonly'] = value;\\n    } catch(err) {\\n      console.log(err)\\n    }\\n    \",\"tags\":[[11756854912,[null,\"readonly\"],[null,\"disabled\"]]]},\"id\":\"8237\",\"type\":\"CustomJS\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} 1.0x^1 + 0.0$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"8146\"},\"group\":null,\"major_label_policy\":{\"id\":\"8147\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"8120\"}},\"id\":\"8119\",\"type\":\"LinearAxis\"},{\"attributes\":{\"source\":{\"id\":\"8162\"}},\"id\":\"8167\",\"type\":\"CDSView\"},{\"attributes\":{\"children\":[{\"id\":\"8187\"},{\"id\":\"8188\"},{\"id\":\"8189\"},{\"id\":\"8190\"},{\"id\":\"8191\"},{\"id\":\"8192\"},{\"id\":\"8193\"}],\"margin\":[5,5,5,5],\"min_width\":300,\"name\":\"Problem parameters\",\"width\":300},\"id\":\"8186\",\"type\":\"Column\"},{\"attributes\":{\"children\":[{\"id\":\"8186\"},{\"id\":\"8194\"}],\"margin\":[0,0,0,0],\"name\":\"Row00469\"},\"id\":\"8185\",\"type\":\"Row\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"8137\"},\"glyph\":{\"id\":\"8138\"},\"group\":null,\"hover_glyph\":{\"id\":\"8140\"},\"muted_glyph\":{\"id\":\"8141\"},\"nonselection_glyph\":{\"id\":\"8159\"},\"selection_glyph\":{\"id\":\"8158\"},\"view\":{\"id\":\"8143\"}},\"id\":\"8142\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"margin\":[5,10,5,10],\"name\":\"\",\"text\":\"<b>Problem parameters</b>\"},\"id\":\"8187\",\"type\":\"Div\"},{\"attributes\":{\"label\":{\"value\":\"Learned model\"},\"renderers\":[{\"id\":\"8166\"}]},\"id\":\"8181\",\"type\":\"LegendItem\"},{\"attributes\":{\"labels\":[\"Ground truth\"],\"margin\":[5,10,5,10]},\"id\":\"8190\",\"type\":\"CheckboxGroup\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.25},\"fill_color\":{\"value\":\"midnightblue\"},\"hatch_alpha\":{\"value\":0.25},\"line_alpha\":{\"value\":0.25},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8138\",\"type\":\"Circle\"},{\"attributes\":{\"labels\":[\"Error\"],\"margin\":[5,10,5,10]},\"id\":\"8191\",\"type\":\"CheckboxGroup\"},{\"attributes\":{},\"id\":\"8113\",\"type\":\"LinearScale\"},{\"attributes\":{\"source\":{\"id\":\"8137\"}},\"id\":\"8143\",\"type\":\"CDSView\"},{\"attributes\":{\"end\":1000,\"margin\":[5,10,5,10],\"start\":10,\"title\":\"Number observations\",\"value\":100},\"id\":\"8188\",\"type\":\"Slider\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":0.0,\"step\":0.1,\"title\":\"Noise level\",\"value\":0.1},\"id\":\"8189\",\"type\":\"Slider\"},{\"attributes\":{\"label\":{\"value\":\"Observations\"},\"renderers\":[{\"id\":\"8142\"}]},\"id\":\"8155\",\"type\":\"LegendItem\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 1\",\"value\":1.0},\"id\":\"8192\",\"type\":\"Slider\"},{\"attributes\":{},\"id\":\"8146\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"code\":\"def linear(\\n        coefficients: jnp.ndarray,\\n        n_observations: int = 100) -> [jnp.ndarray, jnp.ndarray]:\\n    x = jnp.linspace(0, 1, n_observations)\\n    y = coefficients[0] * x + coefficients[1]\\n    return x, \",\"filename\":\"\",\"height\":150,\"js_property_callbacks\":{\"change:disabled\":[{\"id\":\"8237\"}],\"change:readonly\":[{\"id\":\"8236\"}]},\"language\":\"python\",\"margin\":[5,10,5,10],\"sizing_mode\":\"stretch_both\",\"theme\":\"chaos\"},\"id\":\"8221\",\"type\":\"panel.models.ace.AcePlot\"},{\"attributes\":{\"axis_label\":\"$$\\\\color{white} x$$\",\"axis_label_standoff\":10,\"axis_label_text_color\":\"#FFFFFF\",\"axis_label_text_font\":\"Josefin Sans\",\"axis_label_text_font_size\":\"1.9em\",\"axis_label_text_font_style\":\"bold\",\"axis_line_alpha\":0.4,\"axis_line_color\":\"#E0E0E0\",\"coordinates\":null,\"formatter\":{\"id\":\"8149\"},\"group\":null,\"major_label_policy\":{\"id\":\"8150\"},\"major_label_text_color\":\"#E0E0E0\",\"major_label_text_font\":\"Josefin Sans\",\"major_label_text_font_size\":\"1.15em\",\"major_tick_line_alpha\":0.4,\"major_tick_line_color\":\"#E0E0E0\",\"minor_tick_line_alpha\":0.4,\"minor_tick_line_color\":\"#E0E0E0\",\"ticker\":{\"id\":\"8116\"}},\"id\":\"8115\",\"type\":\"LinearAxis\"},{\"attributes\":{\"line_alpha\":0.7,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8163\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"8177\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"8147\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"8107\",\"type\":\"DataRange1d\"},{\"attributes\":{\"end\":5.0,\"margin\":[5,10,5,10],\"start\":-5.0,\"step\":0.1,\"title\":\"X 0\",\"value\":0.0},\"id\":\"8193\",\"type\":\"Slider\"},{\"attributes\":{\"children\":[{\"id\":\"8104\"}],\"margin\":[0,0,0,0],\"name\":\"Row00466\"},\"id\":\"8194\",\"type\":\"Row\"},{\"attributes\":{},\"id\":\"8178\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Linear regression problem\",\"text_color\":\"#E0E0E0\",\"text_font\":\"Josefin Sans\",\"text_font_size\":\"2.1em\"},\"id\":\"8105\",\"type\":\"Title\"},{\"attributes\":{\"margin\":[5,5,5,5],\"name\":\"HSpacer00473\",\"sizing_mode\":\"stretch_width\"},\"id\":\"8222\",\"type\":\"Spacer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.8},\"fill_color\":{\"value\":\"red\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"8158\",\"type\":\"Circle\"},{\"attributes\":{\"mode\":\"append\",\"overlay\":{\"id\":\"8180\"},\"renderers\":[{\"id\":\"8142\"}]},\"id\":\"8160\",\"type\":\"BoxSelectTool\"},{\"attributes\":{\"data\":{\"x\":{\"__ndarray__\":\"AAAAALV+JTy1fqU8ED74PLV+JT1i3k49ED54Pd7OkD21fqU9jC66PWLezj05juM9ED74PfN2Bj7ezhA+yiYbPrV+JT6g1i8+jC46PneGRD5i3k4+TjZZPjmOYz4k5m0+ED54Pv1KgT7zdoY+6aKLPt7OkD7U+pU+yiabPr9SoD61fqU+q6qqPqDWrz6WArU+jC66PoFavz53hsQ+bbLJPmLezj5YCtQ+TjbZPkNi3j45juM+L7roPiTm7T4aEvM+ED74PgVq/T79SgE/+OADP/N2Bj/uDAk/6aILP+Q4Dj/ezhA/2WQTP9T6FT/PkBg/yiYbP8W8HT+/UiA/uugiP7V+JT+wFCg/q6oqP6VALT+g1i8/m2wyP5YCNT+RmDc/jC46P4bEPD+BWj8/fPBBP3eGRD9yHEc/bbJJP2dITD9i3k4/XXRRP1gKVD9ToFY/TjZZP0jMWz9DYl4/PvhgPzmOYz80JGY/L7poPylQaz8k5m0/H3xwPxoScz8VqHU/ED54PwrUej8Fan0/AACAPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]},\"y\":{\"__ndarray__\":\"AwGgvRPyXb06TxY97utUPcSr4jz++go+vG3NvG8nHT54Z+M9zlMzPlTuNT5iS7o9bbL7PbbstD2kzDY94hlwPaKvYD4gqCQ+S3OyPbF4kD6kbVU+M9RIPvpNPT6MNUc+rgcePjqgPz6Dz7A+Z5uWPhtSoT4sxYg+h3OMPruaoT6T9oA+u0OpPnb9gD5Ea9M+SKLEPmwvrz4V69Q+yKCePoqa3T7eacY+RBDlPmZzyj4WAuM+fnzmPpbb5j7HYw0/j3IAPxLm0j70rhI/PdQIP4CD4j50KQs/PdMGPxFP/D6l2Qk/b/MmPyK6Ez/39yk/qTkyP6XYEj+Y+iY/gP4iP9VVFT9HaRY/2CInP0WRMj/b+CM/6bM4PzQCNj9RcTg/oPMnP7HpNj9tW0s/l2ZLPwRhWT/8ITo/lQc3P53qPj9Bg2U/bwJgPxeOUj+U9mI/PRtbPyZAVj+K318/BuBOP01DUT/vRWU/apqAPxcLfD9nW38/EaF8P2kIZT+iJXM/55JvP4Zghz/6l4M/vXlwPw==\",\"dtype\":\"float32\",\"order\":\"little\",\"shape\":[100]}},\"selected\":{\"id\":\"8152\"},\"selection_policy\":{\"id\":\"8151\"}},\"id\":\"8137\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"mode\":\"vline\",\"renderers\":[{\"id\":\"8142\"}],\"tooltips\":null},\"id\":\"8156\",\"type\":\"HoverTool\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"8180\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"background_fill_alpha\":0.9,\"background_fill_color\":\"#0f0f0f\",\"below\":[{\"id\":\"8115\"}],\"border_fill_alpha\":0.8,\"border_fill_color\":\"#111111\",\"center\":[{\"id\":\"8118\"},{\"id\":\"8122\"},{\"id\":\"8154\"}],\"height\":450,\"left\":[{\"id\":\"8119\"}],\"outline_line_alpha\":0.25,\"outline_line_color\":\"#E0E0E0\",\"outline_line_width\":3,\"renderers\":[{\"id\":\"8142\"},{\"id\":\"8166\"}],\"title\":{\"id\":\"8105\"},\"toolbar\":{\"id\":\"8130\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"8107\"},\"x_scale\":{\"id\":\"8111\"},\"y_range\":{\"id\":\"8109\"},\"y_scale\":{\"id\":\"8113\"}},\"id\":\"8104\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"midnightblue\"},\"line_color\":{\"value\":\"white\"}},\"id\":\"8159\",\"type\":\"Circle\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"red\",\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8164\",\"type\":\"Line\"}],\"root_ids\":[\"8182\",\"8264\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.3\"}};\n",
       "    var render_items = [{\"docid\":\"0db575ed-8e30-47d4-bbf7-de70b3cf18e2\",\"root_ids\":[\"8182\"],\"roots\":{\"8182\":\"56efbbaf-82df-4cdf-9502-751c995961b6\"}}];\n",
       "    root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined && root.Bokeh.Panel !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ],
      "text/plain": [
       "Row\n",
       "    [0] HSpacer()\n",
       "    [1] Column\n",
       "        [0] Row\n",
       "            [0] Param(RegressionLinearSolver, name='Problem parameters', widgets={})\n",
       "            [1] ParamMethod(method, _pane=Bokeh)\n",
       "        [1] Ace(height=150, language='python', sizing_mode='stretch_both', theme='chaos', value='def linear(\\n        coef...)\n",
       "    [2] HSpacer()"
      ]
     },
     "execution_count": 142,
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "8182"
      }
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from cml.tasks import RegressionLinearSolver\n",
    "explorer = RegressionLinearSolver()\n",
    "global key\n",
    "key = random.PRNGKey(23)\n",
    "def solve(x, y):\n",
    "    global key\n",
    "    key_gd, key = random.split(key, 2)\n",
    "    w_1, w_0, loss_history = gradient_descent(key_gd, x, y)\n",
    "    x_predict = np.linspace(np.min(x), np.max(x), 200)\n",
    "    y_model = w_1 * x_predict + w_0\n",
    "    return np.array(x_predict), np.array(y_model) #np.array(jnp.zeros(y_model.shape))\n",
    "\n",
    "explorer.solve = solve\n",
    "explorer.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further\n",
    "\n",
    "> You are encouraged to experiment in a similar way as outlined for NumPy with different learning rates, maximum number of iterations, and regularization techniques, as well as compare the performance (both accuracy and speed) of the implemented algorithms with `scikit-learn` built-in linear regression functions.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"capacity\"></a>\n",
    "## Understanding model capacity and selection\n",
    "\n",
    "\n",
    "In real-life problem, we are aiming to find the parameters of a model, but we do not really know what is the _real_ function underlying this process. So what we can decide to select _any_ function of _any_ **capacity** (complexity of the function). One of the problem with that, is that if we have a too simple function, it will _underfit_ (it is not complex enough for our observations). On the opposite end, if we have a function which is too complex, it might be able to _fit through all training points exactly_ ... even though there is noise in our observations ! This is examplified in the following\n",
    "\n",
    "<img src=\"images/01_soa_function_families.png\" align=\"center\"/>\n",
    "\n",
    "We can observe this idea and play with it directly by trying to find a function approximating our previous observations with a polynomial function chosen to have a degree inside \\([1,2,8]\\).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Depending on the _capacity_ of the model, what we can observe is that\n",
    "\n",
    "- `capacity too low   -> underfitting   : prediction variance >  noise variance`\n",
    "- `adequate capacity  -> good fit       : prediction variance == noise variance`\n",
    "- `capacity too high  -> overfitting    : prediction variance <  noise variance`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A similar example can be given for a classification problem in two dimensions as follows\n",
    "\n",
    "<img src=\"images/01_underfit.png\" align=\"center\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "In the following, we define the exercises that you should fill for this session. These go further than what we have seen together in the course, but they are based on the exact same principles, simply with slightly more complex definitions. We provide an overall guideline for successfully implementing each of the exercise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1 - Linear classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> In this exercise, we will implement linear classification using NumPy and JAX. The goal of this exercise is to gain a better understanding of how linear classification works and how to implement it ourselves using our two first libraries of choice, namely NumPy and JAX. To complete this exercise, you will need to have a basic understanding of NumPy and JAX. You can use the resources provided in the previous exercises to learn more about these libraries.\n",
    "\n",
    "We help you out by first defining a simple linear classification problem to solve\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1  1 -1  1 -1  1  1 -1 -1  1 -1 -1 -1 -1  1  1  1 -1 -1  1 -1  1 -1  1\n",
      "  1  1 -1 -1 -1 -1 -1  1  1 -1  1 -1 -1 -1 -1  1  1 -1 -1  1 -1 -1  1 -1\n",
      "  1 -1 -1 -1  1  1  1  1 -1  1 -1 -1 -1  1  1 -1  1 -1 -1  1  1 -1  1  1\n",
      "  1 -1  1  1 -1 -1  1  1 -1  1 -1  1 -1 -1  1 -1  1 -1 -1 -1  1 -1  1  1\n",
      "  1 -1 -1 -1  1  1  1 -1 -1  1  1 -1  1  1  1 -1  1 -1 -1  1 -1 -1 -1 -1\n",
      "  1  1 -1 -1  1 -1 -1 -1  1 -1  1  1  1  1  1  1 -1 -1 -1 -1 -1  1  1 -1\n",
      " -1 -1  1 -1 -1  1  1 -1 -1 -1 -1 -1 -1 -1  1 -1  1  1  1 -1 -1 -1  1  1\n",
      "  1 -1  1 -1  1  1 -1  1  1 -1 -1  1 -1  1 -1  1 -1  1  1  1  1  1  1  1\n",
      "  1 -1  1  1  1  1 -1  1]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import make_blobs\n",
    "# Properties of the problem\n",
    "n_observations = 200\n",
    "noise = 0.2\n",
    "c1_center = [-2, -1]\n",
    "c2_center = [2, 1]\n",
    "# Create points\n",
    "x_coords, y_class = make_blobs(n_samples=n_observations, centers=[c1_center, c2_center], n_features=2, cluster_std=0.55)\n",
    "x_data = x_coords + (noise * np.random.randn(n_observations, 2))\n",
    "#x_data = x_coords\n",
    "y_classes = y_class\n",
    "y_classes[y_classes == 0.0] = -1.0\n",
    "y_clases = y_classes.astype(float)\n",
    "print(y_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.1 - Implement linear classification with NumPy\n",
    "\n",
    "> 1. Check the training data with two classes that are linearly separable.\n",
    "> 2. Initialize weights and bias.\n",
    "> 3. Implement the forward pass of the linear classifier.\n",
    "> 4. Implement the hinge loss for classification.\n",
    "> 5. Derive the gradients (backward pass) of the linear classifier.\n",
    "> 6. Implement gradient descent to optimize the weights and bias.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAu4ElEQVR4nO3df4xU5bnA8WcGy6rArlIq1cwqxKWtramxgBWumsUmrCRttUmJ3ORamq4KBsm1XLiCJoJ/GFCoP6LVQp2CaYs/chu0SVOVpCvaC0QgEltbDajr7rgS5druwqZd4u65f5wOO3v2nJnz6z3ve875fpLNuuPunHcGlvc5z/u8z1uwLMsSAAAADYq6BwAAAPKLQAQAAGhDIAIAALQhEAEAANoQiAAAAG0IRAAAgDYEIgAAQBsCEQAAoM0ZugdQz8jIiPT19cmUKVOkUCjoHg4AAPDBsiw5ceKEXHDBBVIs1s95GB2I9PX1SWtrq+5hAACAEHp7e6VUKtX9HqMDkSlTpoiI/UKam5s1jwYAAPgxMDAgra2tp+fxeowORKrLMc3NzQQiAACkjJ+yCopVAQCANgQiAABAGwIRAACgDYEIAADQhkAEAABoQyACAAC0IRABAADaEIgAAABtCEQAAIA2BCIAAEAbAhEAQGpUKiJdXfZnZAOBCAAgFcplkYsuErn2Wvtzuax7RIgDgQgAwHiVisitt4qMjNhfj4yILFtGZiQLCEQAAMY7cmQ0CKkaHhY5elTPeBAfAhEAgPFmzRIpOmasCRNE2tr0jAfxIRABABivVBLZts0OPkTsz1u32o8j3c7QPQAAAPzo7BTp6LCXY9raCEKygkAEAJAapRIBSNawNAMAALQhEAEAANoQiAAAAG0IRAAAgDYEIgAAQBsCEQAAoA2BCAAA0IZABAAAaEMgAgAAtCEQAQD4VqmIdHXZn4E4EIgAAHwpl0Uuukjk2mvtz+Wy7hEhCwhEAAANVSoit94qMjJifz0yIrJsGZkRREcgAgBo6MiR0SCkanjYPgkXiIJABADQ0KxZIkXHjDFhgkhbm57xIDsIRAAADZVKItu22cGHiP1561b7cSCKM3QPAACQDp2dIh0d9nJMWxtBCOJBIAIA8K1UIgBBvFiaAQAA2hCIAAAAbQhEAACpoqO7Kx1l1SEQAYCMyfKkqaO7Kx1l1VIaiGzcuFHmzp0rU6ZMkfPOO09uuOEGeeedd1ReEgByLa5J08RgRkd3VzrKqqc0ENmzZ4+sWLFC9u/fL7t375bPPvtMFi5cKIODgyovCwC5FNekaWoGQEd3VzrKqqd0++6LL7445uvt27fLeeedJ4cOHZJrrrlG5aUBIHfqTZp+t9x6BTMdHfq37Va7u9a+RtXdXXVcM28SrRHp7+8XEZGpU6e6/v+hoSEZGBgY8wEA8CeONuwmZwB0dHelo6x6BcuyrCQuZFmWXH/99fK3v/1NXnvtNdfv2bBhg9x7773jHu/v75fm5mbVQwSA1CuX7QzG8PDopNnZ6f/nKxV7OcaZAejuNmfyrVSS7+5aveakSSInT9pBnynvh4kGBgakpaXF1/ydWCCyYsUK+d3vfid//OMfpeTxpzc0NCRDQ0Onvx4YGJDW1lYCEQAIIOpEHTWYyapyeXTZqli0MyW8L+6MC0RWrlwpzz//vLz66qsyc+ZM3z8X5IUAAOKjI+tgsjRkikwSZP5WWqxqWZasXLlSdu3aJa+88kqgIAQAEJ9Kxa7/8LukwJkyY8VRCAx3SotVV6xYIb/61a9k586dMmXKFDl27JgcO3ZM/vGPf6i8LACghqnbcdMkjkJguFO6NFMoFFwf3759u/zwhz9s+PMszQBANCwpxIfaGf+MWpoBAOjDkkJ8OjvtfirUzsRLaSACANArbQ25gtayJI3amfhx6B0AZFiaGnJRy5JPifURCYMaEQCIh+nbcallyRZjakQAAGYIs6SQ5DIJtSz5xdIMAGCcpJdJ2B6bXwQiAIAxvE7grVTUXTNNtSyIF0szAIAxdC2TsD02nwhEAABj6Nzyy/bY/GFpBgAwBsskSBIZEQDAOCyTICkEIgAAVyyTIAkszQAAAG0IRADAoVIR6epSu10VgI1ABABqpPm8EwIopBGBCAD8i45GXnFJcwCFfCMQAYB/qdfIy2RpDqCqyObkF4EIAPxLWs87SWsAVaUqm0Nwkw4EIgDwL2lt5JXWAEpEXTaHpar0IBABgBqdnSLd3faddHe3/bXp4gig4soeBH0eFdmcLCxVNZKlbA+BCAA4lEoi7e3mZ0JqRQmg4soehHkeFdmctC9VNZK1bE/BsixL9yC8DAwMSEtLi/T390tzc7Pu4QBA5lQq9mTmPOCuu9v+7yNH7GChUVBW73ka/Wy5bGcshodHszlRMlFRxmK6tLy2IPM3GREAyDGv7MEjjwS7646ShYh7OSyttT5+ZDHbQ0YEAHLM7Q67ulQS5K7bxDv1SiV7h/aZ+D67ISMCAPDFLXuwalXwu24TsxBx1/qYUCBq4vscFRkRAMCY7IFI+LoR5/P4rTExXbk8uhOnWLSDAZ07quLK9lQqav6MgszfBCIAgHHcCkhF/E/Gpk3cUaRlOSQolX9GBCIAgMj8Zkmck3HWJu6uLrto1+3x9vbEhxML1X9G1IgAACKrrbEIsltD986OuGs50ty51ovuP6NaBCIAgIaCTMY6J24Vzb6yWCBqUnBFIAIAaCjIZKxr4lbZ2j2Nrf/rMSm4okYEAOBbkN0aSffxyGIth2qq/oyCzN9nxHdZAEDWlUr+J6wg31sr7JbS6nKDswAzzbUcqoX9M4oTSzMAgMQ0KiSNUuNh0nID/GNpBgA0UtVQykSN+lbEtaU0i63d04btuwCQAlk7zr0eP4WkcW0pjdLa3YQ27nlDIAIAGqjc4WEiP0FGkltK3QIOnYFhngMgAhEA0MCkhlJJ8BNkJFXj4RZwJBUYmhYAmYBABAA0MKmhVBL8Bhmq+3V4BRx796oPDHUGQCYjEAEADfK4w8NvkBGlxqMRr0xUoaA2MNQZAJmOPiIAoElnp0hHR752eOjuW+HVa2TePDswdJ44HNdYGwVAee59QkYEADRSefeP8eplolQuC3ktxVUDoDxlxpzoIwIAyB0dvUbK5fEZl2qwk7XeJ0HmbwIRAMA4QRut5akxWxRZCzi80NAMABCac3fHli3q2rLnDUtx45ERAQCc5tZmvUplW/Y8yFPWiIwIACAUt90dVSrbsmeNs3EZWSNvBCIAkEFhW4a77e6opbMte1q4LW3lvWlZPQQiAJAxW7aEv/t2bm910tWWPS3cGpfdeSdZo3oIRAAgQzZvFlmzJtrdd20/jQce0N+WPU3clqpGRuzGZbXynjWqRWdVAMiISsW++3aq3n0HyVJUO6C2t4v8+7833nKqu2OqKbw6t27cKLJunZqurWlHIAIAGXHkiIjbPshiMdrdN0GGf9WlKrfGZX4CujwiEAGAjHC7GxcRuf9+e+LL0/ZRnbzOECKgc0eNCABkhLNwtFi0azxWr07H9tGwO31MROMy/2hoBgAZ42wjnoamY+Xy6G4Tt8ZpSBcamgFATlWXX2qXBExvOua25TVrfTaylO2JG4EIAGSE1/JL1KZjqifROAIlFWOM6znTsCymE4EIAMRE511vvaxClKZjSUyiUQMlFWOM6znzkO2JikAEAGKg+663UVYhTNOxpCbRKIGSijHG+ZymL4uZgEAEACIy4a7XT1Yh6E6OJCfRsN1ZVSzrxPm6OYunMQIRAIjIhLteFWe+JD2Jhtny6jbGIA3c3DJZcb5uzuJpjEAEACJSPWE779i9alHiPvMlqUk0Sm1NdYy1Z7lYlshLL/m7rlsmSyTe181ZPA1YCu3Zs8f69re/bZ1//vmWiFi7du0K9PP9/f2WiFj9/f1qBggAMXnyScuaMMGyROzPTz4Z3/MWi/bzFouWtXTp2K8bXae317L+8Af7c1i9vZbV1RXtObw4X1+Y9623d/Q5qh8TJjQe7x/+MPZnqh9dXaPPq+p1Z12Q+VtpRmRwcFAuu+wyeeyxx1ReBgC0U3HX63bH/tRT/mtR4iqgVdUlNK7amrBLY40yWXRHTYbSs2YWLVokixYtUnkJADBG3GeJPPLI+AnWyetkXa9JvqPDnIm1XgARpk7E2Tm20dKY2wF1P/6x/+siHtSIAICBKhWRn/yk8fd5TbhhsgSq+6A4nz+u2pootSzVTNbq1fb7tWULTceSZlQgMjQ0JAMDA2M+ACCPjhyxKxacOjr8TbhBJ3m/yzhhgxW354+zGLazU2TfPpEHH7Q/N1oac76OBx8cfb9pOpawBGpWLMuyfBWrrl+/3hKRcR8UqwLIG7cCzGLRftxvEaXfAlq/xZ5hC0sbPX8cRaFBxub83tWr6xetqhZHQbFpghSrGhWI/POf/7T6+/tPf/T29hKIAMitOHbi+JnkG+0eqT5PmJ0p9Z7/wQfjmXx7ey2rUHAP2ty+1+11OH/e72uLKo5dQyYyZtdMUE1NTdLc3DzmAwDyKo6dOH52fvhZxonStM3t+UVEVq2Kpx5j797xy1gjI/YSjZPX6/iv/0q+6ZgJHXlNoDQQOXnypBw+fFgOHz4sIiLvv/++HD58WHp6elReFgAyI8gW0rD1G35qNaIUljqfv1bSk6/X6/jP/0y+6ZgJHXmNoDI109XV5VrzsXTpUl8/T0MzAPAnrsZg9ZZxoi4V9fbayzFx12O4Lc0UCupeR1yiLHeZLsj8XbAst7psMwwMDEhLS4v09/ezTAMAHioVe4nD2Ueju1tNE7KjR+1MiFvvkiNH7KyD13VVjbVcHl3mKBbtDEy9rEa915GkcnlsH5OtW7PRAj7I/K20oRkAQL24GoP54dW0zW8g4NZELI56jM5Oe2tzveDCGSiZ0NjNz7izjowIAKRc0hkRZ9YjzPVrMxIijTMpUQXNmCCaIPO3UbtmAADBJXVKrlfTszBFl9Ui3Jdeiuc8nHrYnWI2MiIAkBEq6x7qZT1EwmVkksrkdHXZgY7b4+3t8V0Ho8iIAEAOqTwttlEdyk03jf1///EfjceR1PbVuM60SZrqs39MQSACAGio3mReqYj88pdj/9+vftV4Ak0qQEhq6SpOfs/+yQICEQAwmCl3xfUm87CZjSQDhHpdak15j6vyVtNCIAIAhjLtrthrMvfKbEya1HiCj6ONvV9uS1emvcci+eu4SrEqABgoyS25cXA25vre90R+8xu7X6jq7bJ+G6mF2Xbs57njlrY/ezcUqwJAyqXtrrg2s7F2rcj//M/oQXQqlxb8ZDTCbjvWlS1JY01LFGREAMBAab0rdht3VdzbZf1mNMJsO673/5LMjKS14yoZEQBIubTeFbtlGUTs5Zm4d8P4yRo12nYcdwFunFRuxzYJZ80AgKHSeA5JtXDVOYlv2hT/+N2u5dz+2+h7vN7jyZMbPzfiQUYEAAyWtrtiZ5ahWBTZvFlkzRr113LLGvn9ntr3uFwWufLK8UFIvYxU3FuATdtSrBI1IgCAwBrtJqlURPbtswtW588PHkgF2a3ip5bCb72FW01JsSiyf7/I3LnuPxP3gXpZOKCPGhEAQCh+7sT97CZ56SWRJUtEbrwx+I6ToLtV6mWNqq9HxF9mya02ZGREZHDQ/fvjbj6Wt2ZmIgQiAIB/8RMA+JkoDxwQueWWcJNpnBNxmO23QdvOx13UakKRbNIIRAAADQOAamZh797GvTeuvHK0h4jb99TjNRHv2xesZiJsQBN0t1Lc5+Wk9YC+KAhEAAB178RrMwtLlogUCmO/r/bwu9rJ3+17GnGbiItFe4mnNrPRaAkpSmYhSNv5uLdZp3XbdhQUqwIAPBt/7ds3fgdJoWB/jIyMTpSdnfbEfe214587aMGls138yMjYDEvt9b2eO+mGcHE3H0tzMzMRilUBAAF53YmfPDk+s2BZ9mOFgsjGjfUPv6vuOAmy66M2I7Fz5/hlnur1RbyXXJLOLMS9zTpt27ajICMCADjNeSder2W7yPgsgzObUc2WRBlPvetXebWPT3tmIa2CzN8EIgCAumqDCzfOICDuyb/2+sWinRGpnbnScAZP3rA0AwCITXWp5Lnn/O3oiHtZoXap5oMPRH7+83wVc2YdZ80AABoqlUQWLxYZGBi/9JJEEFAqjV4njWfwwBtLMwCAQKi7QCNB5m8yIgCAQGqzE0BU1IgAAABtCEQAAIA2BCIAAEAbAhEAAKANgQgAANCGQAQAAGhDIAIAALQhEAEAANoQiAAAAG0IRAAAgDYEIgCQM5WKfZJtpaJ7JACBCNKCfzmBWJTLIhddJHLttfbncln3iJB3BCIwH/9yArGoVERuvVVkZMT+emREZNky4nvoRSACs/EvJxCbI0dGf5WqhodFjh7VMx5AhEAEpuNfTiA2s2aJFB3/6k+YINLWpmc8gAiBCEzHv5xAbEolkW3b7F8hEfvz1q3244AuBCIwG/9yArHq7BTp7rZrv7u77a8BnQqWZVm6B+FlYGBAWlpapL+/X5qbm3UPBzpVKvZyTFsbQQgAGC7I/H1GQmMCoimVCEAAjFOp2KVks2bxT0RasTQDAEgldvZnA4EIACB14tzZn0S/RHoyeiMQAQCkTlw7+5PIqpC5qY9iVQBA6lQq9qReG4xMmGDvBPJbKxLHc5hwDRMFmb/JiAAAUqVaoLppU7Sd/Un0S6QnY2PsmkF6US4P5E65PFobUiyK3H+/yJw54Xb2V/slOrMVcfZLTOIaaUdGJA2ochqPRVcgd9wKVNeuDd9eKIl+ifRkbIwaEdM5w/9t22iFmNdFVyDnurrsew+3x9vbwz9vEv0S89aTkYZmWeG1P62jIx9/k73UW3TN8/sCZFyQZY4gK7dJ9EukJ6M3lmZMRpWTOw7CA3LJ7zIHK7fpQiBiMiZcdyy6AqkTpdSt9mcbHdoXZ6MzJINAxGRMuN44QhRIjSgZCrefLZXsmhC3fwpJJKcPxappkLcqJwCZEaW2PMzPUstuBhqaZU298B8ADBYlQxHmZ0kkpw+7ZkBjMCAF0vprGqWhV9if7ey0NxeSSE4HMiJ5R3k5YLw0/5pGyVBE/VkSyemQSI3I448/Lps3b5aPPvpIvva1r8nDDz8sV199dcOfo0ZEMRZTAeNl5dc0SqkbZXLpY1SNyLPPPit33HGH3H333fLGG2/I1VdfLYsWLZKenh7Vl0YjlJcDxsvKr2mUDAXZjWxTHog8+OCD0tnZKTfffLNccskl8vDDD0tra6s88cQTqi+NRuhTAhiPX1NkndJA5NSpU3Lo0CFZuHDhmMcXLlwoe/fuHff9Q0NDMjAwMOYDClFeDhiPX1OzcAZp/JQGIsePH5fh4WGZPn36mMenT58ux44dG/f9GzdulJaWltMfra2tKoeXT87fIhqDAcbj19QMaS4aNlkiu2YKhcKYry3LGveYiMi6deukv7//9Edvb28Sw8sPr98iFmAB4/Frqhet49VRGohMmzZNJkyYMC778fHHH4/LkoiINDU1SXNz85gPxITfIgAILStFwyZSGohMnDhRZs+eLbt37x7z+O7du2X+/PkqLw0nU3+Lkl5wZYEXQAgUDaujfGlm1apV8uSTT8ovfvEL+etf/yo//vGPpaenR5YvX6760qhl4m9R0guuLPACCImiYXUSa2j2wAMPyEcffSSXXnqpPPTQQ3LNNdc0/DkamsWsXLaXY4aHR3+LdFW9Jd2lKStdoQBoRXM1f4LM35y+mzdev0VJH2TR1WVnJtweb29P//UAIMeM6qwKw7iV3utYskh6qcjEpSkAAIFI7unaTZP0gisLvABgpDN0DwCa1dtNo3qSTvqsbs4GBwDjEIjkXXXJwlnEmdSSRamUbECQ9PVEkq+/AYAUYWkm77KwZGFybxC2DANAXeyagS2te9LK5dEal2LRDqpMOYiDLcMAcopdMwgujQdZmN623tRutgBgkPwGIian8+GP6RM9W4YBoKF8BiKs22eD6RN9FupvAECx/NWIsG6fLSa1rfeS1vobIKPYyKZekPk7f9t3dfbNQPzS0BtEx5ZhAK5Mrm/PKzIiImREoB+3aIBy/POfHHbN1MO6/SgKds1AzRKQCNPr2/MqfxmRqryv25OfdJd0ZoJbNCAx/Lolh4yIH2nsmxEX0/tv6KIjM8EtGpAYEuJmym8gkmdMfuPpCs5M34IMZExnp50B6eqyP5MI1o9AJI+yNvnFUeuiKzjjFg1IXJ4T4iYiEMmjLE1+tcspF14osmZNuIBEZ3DGLRqAHMtvsSrSX7DrVnkmEr74Ng3N0QDkXhp2+weZvwlEkF5dXXYmxE3YUvi0B2cAMi0tGx7ZNYP081P34bacUhW2voPFY7XoXQOEltUNjwQiCEflhOJ3G2211sUtGElz8W1W0bgNiCSrGx4JRBCcygklaMjf2SnywQciq1dno/g2q7J6KwckKGsbHqsIRBCMigmlNrsSJuQvlUQ2b2bnicmyeisHJChLGx5r5e/0XUQT9+nFzsqrTZvsz84ezH5C/rCn3KahBD3tqrdyYf5cAZyWhgPHgyIjgmDizA26ZVfWrRO5//7kQn5nH5LNm9VcJ++yeisHaJC1mnoyIgimOqE4+22E+Y3wyq7MmWMvr6gO+Z2BkGWJ/Pd/ixQKds0J4pXFWzkAkdFHBOHE0W8jylGYcSynePUhKRbtAlgmSgAIhT4i8CfKFtw4coNh0/Vx7dqZNcvOfjiNjFBECQAJIRDJK1N6OgQ9ZyXOXTulkl2P4kQRJQAkhkAkj5Lq6eA34xIkuxL3NtA1a+wC1WoBLkWUAJAoApE8SqKng6qMi4qOPqtX2zUh9CABgMQRiOSR6vZ8KjMuqraBZm0/HACkBIFIHqnu6aA64xK0rgQAYCz6iOhS3X46ebLIyZPJdfWsXrejQ12vjiS6aIbtogoAMAoZER1q6yeuuCK5nSvOuo2nn7abeMWNLpoAAJ9oaJY0tyZeVX6becV93WLRDhziXuKoVET27bODnfnzCUQAICdoaGYyt/qJKpWnkda7btzbd6vbdp95RmTJEpEbb9TbqyQOUZq/AQA8EYgkzW3HSpXKRlr1risSXxBUu/yzZs3YnTO33hrPRJ50UGBK8zcAyCACkaQ56yeqVNdReF23qlgUmTQp2jWc23adRkZEHnkk2jWSDgqSav4GADlFjYgu1UPjJk0SGRxM7jTS6nUPHhRZu9bOhFRFrRXxOkSuVpQ6mCiH5Pl5brdD9LxeU1eX3XcEADAONSJpUG2gNXduso20qtddvdouJK099C3q3X6j5R+RaEtAqvqT1MuyqG7+BgA5RyCSZydPjt++G2Vidy7/FIvjT7eNMomrCAoaLb2wFRkAlCIQyTMVE3tt19MPPhD5+c/jm8RVBAV+sizOTq4dHcGKZdlxAwCeqBHJu3LZzgAMD49O7Cr6icTZwdXr+bzqPBo9V5C6k3J5NIPip6Ym6PcDQAYEmb8JRBB/oKBDlAnfbzAWNGhRWVwLAAajWBXBpP3k2ahbbP0eohe0WFb14X8AkAEcege1wiyXBH3uTz7xnvD9XtPPIXpBD/NL4vA/AEg5MiJQR2XzsdrnXrIk3t05XoIWy7LjBgAaokYEaqhuPuZ87kLBzj6oLLqtvX6Qmpos1OAAQABB5m+WZtJM5bJHVI3qI6rjrv1vv6/B7bktS+Tpp0W+8AX1E76fZZwo3w8AOUIgklZJbAt1C3T8Bj9e9REHDoh861v249XlFMsK9hq8nnvePCZ8AEgZakTSKImD2NzqO4LUfLjVR2zaZJ9vUx23ZY12dg3yGqi9AIDMoEYkjVQfxOZWg1HtwBq05qO2PuLIkcaH4gV5DdReAICRqBHJOtXbQt1qMJxfi/jfIluNdd3GXSvoa6D2AgBSj6WZNFK9NOF1im7QLbLOpZyXXho77upOFxWvAQCQCizNpJnKpYktW0TWrBn7WJAtsvW274qMjrv2vwlCACATWJrJC5VLE7Nnj38syBbZvXu9t+8628n77cURZauyyVudASDHWJqBO7flmeoW2Ubn0pTLdrdTp7B1LFE7tKrs8AoAiIRABO7C1qFUtxa7rfiNjNh1IkFE3aqcxFZnP2Po6kr2mgCQEgQi8OZ2Km2jSdVtx02VZQUPAqKeYKv7BFyyMQBQF4FIUHm7uy2VRpdi/EyqXjtuqoIGAV5LRH6XeKL+fBQmZGMAwHBKA5H77rtP5s+fL2effbacc845Ki+VjDzf3fqdVJ1LOk5heoVE2aqsswur7mwMAKSA0u2769evl3POOUcqlYqUy2X5+9//Hujnjdq+q/I02TQI2s31wAGRP/5R5P/+z27tHvVU3KhblXV0Yc373xkAuWXM9t17771XRER27Nih8jLJqHd3m4dJJUg3V+eBfPffLzJnTrQgIOpWZR1dWKvZmGXLxgZiefj7AgA+GdVHZGhoSIaGhk5/PTAwoHE0DqrbqpvO76TqtoSzdm1+swCdnSIdHTRtAwAPRhWrbty4UVpaWk5/tLa26h7SKE58dd9F40RdxHi1Bb8AgDECByIbNmyQQqFQ9+PgwYOhBrNu3Trp7+8//dHb2xvqeZTxMxFXZXV3jdukWvtaw+xSyep7BQBoKPDSzO233y5L3Lpm1pgxY0aowTQ1NUlTU1Oon02Mn1oDZ43Etm3hCjTTwO21BqmLyNN7BQAYJ5FD73bs2CF33HFHunfN+JWnnRJ+D7bzet15eq8AIEeM2TXT09Mjn376qfT09Mjw8LAcPnxYRETa2tpk8uTJKi+dHOdhannaXVPvtfqpiVD1XnHAHQCkhtJi1XvuuUcuv/xyWb9+vZw8eVIuv/xyufzyy0PXkBjHrcGZzk6eSTOx62ncTeeoXwEApZQGIjt27BDLssZ9tLs1wEobr06jIvnYXVPNOmzaZE7X07hbque5ky4AJMSoPiKpUm9ZwaTeEc5liijLFtWfPXRI5M4742lYFva9OnBA5LXXRK6+WmTuXPuxOJd6vIKajo7sBZUAoBGBSFiNGpzp6OTp5NyRctNNIr/8ZbgdKrXPVSuOhmVB36sf/lDkqadGv166VGTHjnibzuWp1gcANDKqoVmqmN7gzO2O/qmn/C1bOOsinM/llGTDsgMHxgYhIvbXBw74/zPxU/eRp1ofANCIQCSKaoOz554T2bnTTttX6S5ydLujd3ILINzqIho9V5IT9GuvuT/+v/9rf27UdM5v3YfpgSYAZEQifUTCSkUfEbeGXCL6m3S59ehwcvbs8OrrsW+fyJVXuj9XlBN1wzhwQOSKK8Y//vrro7UiXsL0LdFxai8ApFyQ+ZuMSBReBY233BLfzo2w3O7oly6tf4fvVRcxODj+uR54wF+r+7jNnWu/jlpLlzYOQkTCnYPDOTEAoBQZkSi6uuwUv9/vjXPbst/dL847+np3+I0yBiZlBw4csJdj/u3f/AUhInRyBYCEGNNZNfO8dmmMjIjUxndx11AEOZ/FuSOl3g6VahbF65wYE3YCVc2d6z8AqWr0+gAAiSMjElW5PH5iExn/WEdHPG3Hk7irNynzoULWXx8AaEZGJEleDblqH3vppdHgIWrxahL9LUzKfKiQ9dcHAClCRkS1uDMY1DkAAAzHrhmThNmpUQ/9LeKhu88LAEBECETUU9Ghs1HTLtTHYXYAYAwCEdVUZTDy1N8izuxFlBN6VWdRyNIAyCECkSSkPYOhc4KMO3sRdqlMdRaFLA2AnKJYFfUF6VkSNz8N1oJuiQ7b5l1lgTAFyAAyhmJVxCPKMkYc6mUvwmYQwiyVxV1wnPTzA4DBCEQwVu0yTBwTZJRlHa9C30mTogVIQZfKVBQcJ/n8AGAwAhGMcmYZDh2KNkFGrXvwyl6cPBk9QApS7Kt6yzRbsgHkGDUisHnVKWzcKLJu3dh29X5qROKse3A7uE9HTYXq1vC0ngeQEbR4R3BeyzBz59oTfNAJMs5W9G4H98V1eF2QglfVreFpPQ8ghwhEGgmzMyONvE4SrgYfQV97veeLg9cZP0Ho3BEEABARakTqy1NvhzjrFKrB26ZNauseojR1070jCAAgItSIeIujDiGN2ZSodQrOLMP994vMmWNe3UNXlx1guj3e3p74cAAgS+gjEoeoW1fTmk2JO8uwdq15QYgIW2YBwBAEIl6iTFR5TfunqTEXW2YBwAgEIl6iTFRpmpDjlLYsQ9rPAAKADGDXTD1hd2ao3jFiqji31SaFLbMAoBWBSCNhJqo0TshxiWNbLQAgN9g1oxKdMgEAOURnVVOQ9gcAoC6KVaFXlNN5AQCpRyACfdLaawUAEBsCEeiR114rAIAxCESgh6peKyz1AECqEIhAD7fmZ8WiyKRJ4Z+TpR4ASB0CEejh7FwrYmdIrrwyXADBUg8ApBKBCPTp7BTZt0+kUBh9LGwA4bXUs28fSzUAYDACEeh18qSIs6demFoRr6WeG29kqQYADEYgAvXqFZDGdVCe2yGFljUa5LBUAwBGIhCBWo0KSKOccuxUe5ruzp3xZFoAAEpx1gzUqVTs4MN5CnF39/hAI+5zeYJcGwAQqyDzNxkRqBOkV0ipJNLeHl+QEGemJQj6mABAIAQiUCeu+o+wapdqurvtr1WijwkABEYggnD83Pnryko4xxBnpsULfUwAIBQCEQQX5M4/6ayELqpa1gNAxlGsimAoAnXH+wIAp1GsCnW483dnwjIUAKTQGboHgJSpFqA67/yTKkA1WWenSEdHvNuQASDjyIggmDTc+evcQptUcSwAZASBCIIzuQCVLbQAkCoUqyI7KBgFACNQrIp8opAWAFKHQATm81vzobuTKwAgMAIRmC1IzUcaCmkBAGNQIwJzha35iPskXwBAIEHmb/qIwFz1aj7qBRilEgEIAKQESzMwFzUfAJB5BCIwV5prPnQ2VQOAFCEQgdlMbp7mhaZqAOAbxapAnGiqBgBmNDTr7u6Wzs5OmTlzppx11lly8cUXy/r16+XUqVOqLgnoR1M1AAhE2a6Zt99+W0ZGRmTr1q3S1tYmf/7zn+WWW26RwcFB2bJli6rLAnpxOjEABJLo0szmzZvliSeekPfee8/X97M0g1Qql0WWLbMzIdUC2zTUtgBATIztI9Lf3y9Tp05N8pLIi0rFXhaZNUt/LUZnp0hHB03VAMCHxAKRd999Vx599FH5yU9+4vk9Q0NDMjQ0dPrrgYGBJIaGtCuXRW691V4OKRbtLb+6MxA0VQMAXwIXq27YsEEKhULdj4MHD475mb6+Prnuuutk8eLFcvPNN3s+98aNG6WlpeX0R2tra/BXhHypVEaDEBH787Jl9O8AgJQIXCNy/PhxOX78eN3vmTFjhpx55pkiYgchCxYskG9+85uyY8cOKTo7ZdZwy4i0trZSIwJvXV12vw63x9vbEx8OAEBxjci0adNk2rRpvr73ww8/lAULFsjs2bNl+/btdYMQEZGmpiZpamoKOiTkGbtUACDVlPUR6evrk/b2dmltbZUtW7bIJ598IseOHZNjx46puiTyKM1t4AEA6opVX375ZTl69KgcPXpUSo5JweBmrulk0o4RHdilAgCpRYv3tDNxx4ibvAdLAJAjRrR4RwLSsmOEQ+AAAB4IRNIsDeeapCVYAgBoQSCSZtUdI7VM2zGShmAJAKANgUiapWHHSBqCJQCANgQiadfZKdLdbTfw6u42r1A1DcESAEAbds0gGZUK22sBICfYNQOzVLfuEoQAABwIRKAWW3cBAHUQiEAdtu4CABogEIE6bN0FADRAIAJ12LoLAGiAQATqsHUXANCAstN3ARHhZFwAQF0EIlCvVCIAAQC4YmkGAABoQyACAAC0IRABAADaEIgAAABtCEQAAIA2BCIAAEAbAhEAAKANgQgAANCGQAQAAGhDIAIAALQhEAEAANoYfdaMZVkiIjIwMKB5JAAAwK/qvF2dx+sxOhA5ceKEiIi0trZqHgkAAAjqxIkT0tLSUvd7CpafcEWTkZER6evrkylTpkihUNA9nEAGBgaktbVVent7pbm5WfdwtOK9GIv3YxTvxSjei7F4P0al8b2wLEtOnDghF1xwgRSL9atAjM6IFItFKaX8+Pjm5ubU/MVRjfdiLN6PUbwXo3gvxuL9GJW296JRJqSKYlUAAKANgQgAANCGQESRpqYmWb9+vTQ1Nekeina8F2PxfozivRjFezEW78eorL8XRherAgCAbCMjAgAAtCEQAQAA2hCIAAAAbQhEAACANgQiCfjud78rF154oZx55ply/vnny0033SR9fX26h6VFd3e3dHZ2ysyZM+Wss86Siy++WNavXy+nTp3SPTQt7rvvPpk/f76cffbZcs455+geTqIef/xxmTlzppx55pkye/Zsee2113QPSZtXX31VvvOd78gFF1wghUJBnn/+ed1D0mLjxo0yd+5cmTJlipx33nlyww03yDvvvKN7WNo88cQT8vWvf/10I7N58+bJ73//e93Dih2BSAIWLFggzz33nLzzzjvym9/8Rt599135/ve/r3tYWrz99tsyMjIiW7dulbfeekseeugh+dnPfiZ33XWX7qFpcerUKVm8eLHcdtttuoeSqGeffVbuuOMOufvuu+WNN96Qq6++WhYtWiQ9PT26h6bF4OCgXHbZZfLYY4/pHopWe/bskRUrVsj+/ftl9+7d8tlnn8nChQtlcHBQ99C0KJVKsmnTJjl48KAcPHhQrr32Wrn++uvlrbfe0j20eFlI3AsvvGAVCgXr1KlTuodihAceeMCaOXOm7mFotX37dqulpUX3MBJzxRVXWMuXLx/z2Fe+8hVr7dq1mkZkDhGxdu3apXsYRvj4448tEbH27NmjeyjGOPfcc60nn3xS9zBiRUYkYZ9++qn8+te/lvnz58vnPvc53cMxQn9/v0ydOlX3MJCQU6dOyaFDh2ThwoVjHl+4cKHs3btX06hgov7+fhER/n0QkeHhYXnmmWdkcHBQ5s2bp3s4sSIQScidd94pkyZNks9//vPS09MjL7zwgu4hGeHdd9+VRx99VJYvX657KEjI8ePHZXh4WKZPnz7m8enTp8uxY8c0jQqmsSxLVq1aJVdddZVceumluoejzZ/+9CeZPHmyNDU1yfLly2XXrl3y1a9+VfewYkUgEtKGDRukUCjU/Th48ODp71+zZo288cYb8vLLL8uECRPkBz/4gVgZamob9P0QEenr65PrrrtOFi9eLDfffLOmkccvzHuRR4VCYczXlmWNewz5dfvtt8ubb74pTz/9tO6haPXlL39ZDh8+LPv375fbbrtNli5dKn/5y190DytWZ+geQFrdfvvtsmTJkrrfM2PGjNP/PW3aNJk2bZp86UtfkksuuURaW1tl//79mUmxBX0/+vr6ZMGCBTJv3jzZtm2b4tElK+h7kTfTpk2TCRMmjMt+fPzxx+OyJMinlStXym9/+1t59dVXpVQq6R6OVhMnTpS2tjYREZkzZ44cOHBAHnnkEdm6davmkcWHQCSkamARRjUTMjQ0FOeQtAryfnz44YeyYMECmT17tmzfvl2KxWwl5qL83ciDiRMnyuzZs2X37t3yve997/Tju3fvluuvv17jyKCbZVmycuVK2bVrl7zyyisyc+ZM3UMyjmVZmZo7RAhElHv99dfl9ddfl6uuukrOPfdcee+99+See+6Riy++ODPZkCD6+vqkvb1dLrzwQtmyZYt88sknp//fF7/4RY0j06Onp0c+/fRT6enpkeHhYTl8+LCIiLS1tcnkyZP1Dk6hVatWyU033SRz5sw5nRXr6enJba3QyZMn5ejRo6e/fv/99+Xw4cMydepUufDCCzWOLFkrVqyQnTt3ygsvvCBTpkw5nTVraWmRs846S/PoknfXXXfJokWLpLW1VU6cOCHPPPOMvPLKK/Liiy/qHlq8dG7ZyYM333zTWrBggTV16lSrqanJmjFjhrV8+XKrUqnoHpoW27dvt0TE9SOPli5d6vpedHV16R6acj/96U+tiy66yJo4caL1jW98I9dbNLu6ulz/HixdulT30BLl9W/D9u3bdQ9Nix/96Eenf0e+8IUvWN/61resl19+WfewYlewrAxVTAIAgFTJ1uI8AABIFQIRAACgDYEIAADQhkAEAABoQyACAAC0IRABAADaEIgAAABtCEQAAIA2BCIAAEAbAhEAAKANgQgAANCGQAQAAGjz/w/R6yy510WCAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize the training data\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "for x, y in zip(x_data, y_classes):\n",
    "    if y == 1:\n",
    "        ax.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "    else:\n",
    "        ax.plot(x[0], x[1], marker=\".\", color=\"red\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [],
   "source": [
    "#forward pass\n",
    "def linear_classification(x, w, b):\n",
    "    return np.dot(x, w) + b\n",
    "\n",
    "#hinge loss\n",
    "def hinge_loss(x, y, w, b):\n",
    "    return max(0, 1 - y * linear_classification(x, w, b))\n",
    "\n",
    "#gradient\n",
    "def gradient(x, y, w, b):\n",
    "    loss = hinge_loss(x, y, w, b)\n",
    "    if loss <= 0:\n",
    "        return np.array([0, 0]), 0\n",
    "    else: \n",
    "        return -y * x, -y\n",
    "\n",
    "def gradient_descent(X, Y, w, b, epoch=1000, lr=1):\n",
    "    for i in range(epoch):\n",
    "        grad_w = np.array([0.0, 0.0])\n",
    "        grad_b = 0.0\n",
    "        for x, y in zip(X, Y):\n",
    "            d_w, d_b = gradient(x, y, w, b)\n",
    "            grad_w += d_w\n",
    "            grad_b += d_b\n",
    "        grad_w /= len(X)\n",
    "        grad_b /= len(X)\n",
    "        w -= lr * grad_w\n",
    "        b -= lr * grad_b\n",
    "    return w, b\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "w_old = np.random.randn(2)\n",
    "b_old = 10 * np.random.randn()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [],
   "source": [
    "w_new, b_new =gradient_descent(x_data, y_classes, w_old, b_old)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-4.0, 4.0)"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABk+ElEQVR4nO3deXxU1d0/8M/MZCUbkLAHEpZAZAlLRMBaQa2gLT7an6JYF1RErdAHRRTQWrC1xQJqfbQuSFzqBliXPq1W5JEU9RHLDsomIIGENWwTQAgkc35/zHOTWe6duXdm7tzt83695mUzmblzEnI/Pfec7z3HJYQQICIiIjKA2+gGEBERkXOxI0JERESGYUeEiIiIDMOOCBERERmGHREiIiIyDDsiREREZBh2RIiIiMgw7IgQERGRYdgRISIiIsOwIxLFrFmz4HK5cPjw4YQe99lnn0WPHj2QlpYGl8uF48ePJ/T4ZvH888/jtddeC3u+qqoKLpdL9nux0uOYgTZv3oxZs2ahqqoq7Hu33XYbiouLdflccrZ169Zh+PDhyMvLg8vlwp/+9Cd8/PHHmDVrltFNixvzgQAAgiKaOXOmACBqa2sTdsx169YJAOLOO+8UX3zxhVixYoVoaGhI2PHNpE+fPmL48OFhz585c0asWLFCHDp0KGGfpccxA7377rsCgKisrAz73o4dO8TatWt1+VxytgEDBoiSkhLx8ccfixUrVoj9+/eLiRMnCjvEN/OBhBAixdBekENt2rQJADBhwgRccMEFCTnmDz/8gBYtWiTkWMmQnp6OoUOHmv6YanXv3t2QzyX7+/bbbzFhwgRceeWVun+WWXKE+eAwRveEzE4aEVm7dq34+c9/LnJyckRubq646aabZHvWCxcuFEOHDhUtWrQQWVlZYuTIkUE94eHDhwsAQY9x48Y1fb+iokKUlZWJ9PR00apVK3HNNdeIzZs3B33GuHHjRFZWlti4caO4/PLLRXZ2thg6dKgQQoj6+nrxu9/9TvTq1UukpaWJgoICcdttt6m+Cvjb3/4mhg4dKjIzM0V2drb4yU9+Ir766quYfidFRUVhP2tRUZEQQohdu3YJAOLVV18NO+6GDRvEddddJ3Jzc0WrVq3E/fffL86dOye2bt0qRo0aJbKzs0VRUZH44x//GNQuuWOGfn7gY9euXUIIIVatWiVuuOEGUVRUJDIyMkRRUZEYO3asqKqqajrOq6++KnsM6bPGjRvX9LNJTp8+LaZPny6Ki4tFamqq6Nixo7j33nvFsWPHgl5XVFQkfvazn4l//vOfYuDAgSIjI0P06tVLVFRUqPo3I+vZvn27uO2220SPHj1EZmam6Nixoxg9erTYuHFj02uU/ubGjRsX8e/Z5/OJP//5z6J///4iIyNDtGzZUlx77bVi586dQW0YPny46NOnj1i+fLkYNmyYyMzMFDfccEPEdjMfmA96YEckCumPv6ioSDz44INiyZIl4qmnnhJZWVli4MCB4uzZs02v/f3vfy9cLpe44447xD/+8Q/x/vvvi2HDhomsrCyxadMmIYQQmzZtEr/+9a+b/khXrFghduzYIYQQ4g9/+IMAIG688Ubx0Ucfib/85S+iW7duIi8vT3z33XdNnzNu3DiRmpoqiouLxezZs8Vnn30mlixZIhobG8UVV1whsrKyxGOPPSaWLl0qFixYIDp16iR69+4tfvjhh4g/61tvvSUAiJEjR4oPP/xQLFq0SJSXl4u0tDTxxRdfaP6drF27VnTr1k0MHDhQrFixQqxYsaKpUxYpaHr16iV+97vfiaVLl4qHHnpIABCTJk0SpaWl4r/+67/E0qVLxe233y4AiPfee6/p/XLHlD5Xeixbtkx06tRJtG/fXni9XiGEf0j1N7/5jfjggw/E8uXLxcKFC8Xw4cNFmzZtmqbkDh061PTv8+c//7npeFKwhgaNz+cTo0aNEikpKeLRRx8Vn376qZg3b17T7+jMmTNNry0qKhKFhYWid+/e4i9/+YtYsmSJGDNmjAAgli9fHvHfjKxp+fLl4oEHHhB//etfxfLly8UHH3wgrrnmGpGZmSm2bt0qhPD/za1YsUIAENddd13T39yOHTvEddddJwAE/W1Lf1MTJkwQqamp4oEHHhCffPKJePvtt0Vpaalo166dOHDgQFMbhg8fLlq3bi06d+4snn32WVFZWRnx7435wHzQCzsiUUh//Pfff3/Q89JJ+eabbwohhNizZ49ISUkRv/rVr4Jed+LECdG+fXtx/fXXNz0n9Z5XrVrV9NyxY8dEZmam+OlPfxr0/j179oj09HTxi1/8ouk56YrolVdeCXrtO++8E3byCeHv0QMQzz//vOLP2djYKDp27Cj69esnGhsbg9rftm1bceGFF2r+nQihPAccKWiefPLJoNcOGDBAABDvv/9+03Pnzp0Tbdq0Ef/v//2/iMcM1NDQIK6++mqRnZ0t1qxZo/i7aGhoECdPnhRZWVnimWeeaXo+0hxwaNB88sknAoCYM2dO0OsWLVokAIj58+c3PSddae3evbvpudOnT4vWrVuLu+++W7GdZB8NDQ3i7NmzoqSkJOy8AiAmTpwY9JxSjYjUcQk9h6qrq0VmZqZ46KGHmp6TRmc/++yzqO1jPgS/jvmQWLxrRqWbbrop6Ovrr78eKSkpqKysBAAsWbIEDQ0NuPXWW9HQ0ND0yMjIwPDhw/Gvf/0r4vFXrFiB06dP47bbbgt6vnPnzrj00kvx2Wefhb3n2muvDfr6H//4B1q2bImrrroqqA0DBgxA+/btI7Zh27Zt2LdvH2655Ra43c1/FtnZ2bj22mvx9ddf44cfftD0O4nV6NGjg74+77zz4HK5gubIU1JS0KNHD+zevVv1cSdNmoSPPvoI7777LgYNGtT0/MmTJzFt2jT06NEDKSkpSElJQXZ2Nk6dOoUtW7bE9DMsW7YMAML+PceMGYOsrKywf88BAwagS5cuTV9nZGSgZ8+emn4+so6Ghgb84Q9/QO/evZGWloaUlBSkpaVh+/btMf/NAf4McLlcuPnmm4MyoH379ujfv39YBrRq1QqXXnpp1OMyH5gPemKxqkrt27cP+jolJQX5+fk4cuQIAODgwYMAgMGDB8u+P/DklSMdp0OHDmHf69ixI5YuXRr0XIsWLZCbmxv03MGDB3H8+HGkpaXJfkakW5Cjfb7P58OxY8eCCtmi/U5i1bp166Cv09LS0KJFC2RkZIQ9X1dXp+qYjz/+OF588UVUVFTgiiuuCPreL37xC3z22Wd49NFHMXjwYOTm5sLlcuGnP/0pTp8+HdPPcOTIEaSkpKBNmzZBz7tcLrRv3z7sd5Sfnx92jPT09Jg/n8xtypQp+POf/4xp06Zh+PDhaNWqFdxuN+688864/s0PHjwIIQTatWsn+/1u3boFfS13vsthPjAf9MSOiEoHDhxAp06dmr5uaGjAkSNHmv5ACgoKAAB//etfUVRUpPn40nH2798f9r19+/Y1HV/icrnCXldQUID8/Hx88sknsp+Rk5MT8+e73W60atUq6PlovxOzeO211/Doo49i1qxZuOOOO4K+5/V68Y9//AMzZ87E9OnTm56vr6/H0aNHY/7M/Px8NDQ0oLa2NihshBA4cOCAYoeVnOHNN9/Erbfeij/84Q9Bzx8+fBgtW7aM+bgFBQVwuVz44osvkJ6eHvb90OfkckQO84H5oCdOzaj01ltvBX29ePFiNDQ0YMSIEQCAUaNGISUlBTt37sT5558v+4hk2LBhyMzMxJtvvhn0fE1NDZYtW4bLLrssahtHjx6NI0eOoLGxUfbze/XqpfjeXr16oVOnTnj77bchhGh6/tSpU3jvvfcwbNiwsNv6ov1OAON77Z988gkmTJiAO+64AzNnzgz7vsvlghAiLKAXLFiAxsbGoOek16j5eaR/r9B/z/feew+nTp1S9e9J9uVyucL+5j766CPs3btX1fuV/hZHjx4NIQT27t0rmwH9+vWLqb3MB+aDnjgiotL777+PlJQUXH755di0aRMeffRR9O/fH9dffz0AoLi4GL/97W/xyCOP4Pvvv8cVV1yBVq1a4eDBg1i5ciWysrLw2GOPKR6/ZcuWePTRR/Hwww/j1ltvxY033ogjR47gscceQ0ZGhuxJEmrs2LF466238NOf/hSTJ0/GBRdcgNTUVNTU1KCyshJXX301fv7zn8u+1+12Y86cObjpppswevRo3H333aivr8fcuXNx/PhxPPHEE5p/JwDQr18/LFy4EIsWLUK3bt2QkZERcxhqtWvXLowZMwbdunXD7bffjq+//jro+wMHDkRubi4uvvhizJ07FwUFBSguLsby5ctRUVERdmXat29fAMD8+fORk5ODjIwMdO3aVfYK7/LLL8eoUaMwbdo01NXV4Uc/+hE2btyImTNnYuDAgbjlllt0+7nJ/EaPHo3XXnsNpaWlKCsrw5o1azB37lwUFhaqer90Dv3xj3/ElVdeCY/Hg7KyMvzoRz/CXXfdhdtvvx2rV6/GxRdfjKysLOzfvx9ffvkl+vXrh1/+8pea28t8YD7oyrg6WWuQKrXXrFkjrrrqKpGdnS1ycnLEjTfeKA4ePBj2+g8//FBccsklIjc3V6Snp4uioiJx3XXXif/5n/9peo3cXTOSBQsWiLKyMpGWliby8vLE1Vdf3XTrr0RaR0TOuXPnxLx585rWEMjOzhalpaXi7rvvFtu3b4/683744YdiyJAhIiMjQ2RlZYnLLrtM/O///m/Mv5OqqioxcuRIkZOTo3qdgNBVbJV+XmkdBEnoMSsrK1WtE1BTUyOuvfZa0apVK5GTkyOuuOIK8e2334qioqKgNV6EEOJPf/qT6Nq1q/B4PKrWCZg2bZooKioSqampokOHDuKXv/yl4joBcj+f3B0FZH3Hjh0T48ePF23bthUtWrQQF110kfjiiy9k/80hc9dMfX29uPPOO0WbNm2Ey+UK+nsWQohXXnlFDBkyRGRlZYnMzEzRvXt3ceutt4rVq1c3vSb0/FGD+cB80INLiIBxNiIVZs2ahcceewy1tbVhtStE5GzMB9KKNSJERERkGHZEiIiIyDCcmiEiIiLDJG1EZPbs2XC5XLjvvvuS9ZFEZAPMDiJ7S0pHZNWqVZg/fz7KysqS8XFEZBPMDiL7070jcvLkSdx00014+eWXw1beIyJSwuwgcgbdFzSbOHEifvazn+EnP/kJHn/88Yivra+vR319fdPXPp8PR48eRX5+vuqliIkosYQQOHHiBDp27Bh1z6REYnYQWZeW3NC1I7Jw4UKsXbsWq1atUvX62bNnR1x9lIiMU11drXrlz3gxO4jsQU1u6NYRqa6uxuTJk/Hpp5+G7YqoZMaMGZgyZUrT116vF126dEF1dXXYTrNkLj/5yU+watUqvPzyy0FLOJP11dXVoXPnzhE3TUwkZoc1/exnP8OXX36J5557znlLlFMYLbmhW0dkzZo1OHToEMrLy5uea2xsxOeff47nnnsO9fX18Hg8Qe9JT0+X3TEyNzeXYWJyQ4YMwapVq7Blyxb+W9lUsqY4mB3WNGTIEHz55ZfYvHkzf+fURE1u6NYRueyyy/DNN98EPXf77bejtLQU06ZNCwsSsjbp/zTWrFljcEvI6pgd1sQMoFjp1hHJyclp2o1QkpWVhfz8/LDnyfqkEFq7di18Pl9SixrJXpgd1iRlwIYNG9DQ0ICUFG7uTurw/y0oIc477zxkZmbixIkT2LFjh9HNIaIk69GjB3JycnDmzBls3rzZ6OaQhSS1y/qvf/0rmR9HSZSSkoL+/fvj66+/xpo1a9CzZ0+jm0Q2wuwwP7fbjUGDBmH58uVYs2YNF6Ej1TgiQgnDOWIiZ2MGUCzYEaGEYQgRORszgGLBjgglTGjBKhE5S2jBKpEa7IhQwvTu3RsZGRmoq6vDzp07jW4OESVZSUkJcnJycPr0aWzZssXo5pBFsCNCCSMVrAIcmiVyIrfbjYEDBwJgBpB67IhQQnGOmMjZmAGkFTsilFAMISJnYwaQVuyIUEIFFqwKIQxuDRElm5QB69evZ8EqqcKOCCVU7969kZ6eDq/Xy4JVIgfq2bMnsrOzcfr0aWzdutXo5pAFsCNCCZWamsqCVSIHY8EqacWOCCUc54iJnI0ZQFqwI0IJxxAicjZmAGnBjgglHAtWiZwtsGC1sbHR4NaQ2bEjQgnXp08fpKen4/jx4/j++++Nbg4RJVnPnj2RlZWFH374gQWrFBU7IpRwqampTVuAc2iWyHk8Hg8LVkk1dkRIF5wjJnI2ZgCpxY4I6YIhRORszABSix0R0gULVomcTcqAdevWsWCVImJHhHTRp08fpKWl4dixY9i1a5fRzSGiJOvVq1dTwep3331ndHPIxNgRIV2kpaWxYJXIwTweDwYMGACAGUCRsSNCuuEcMZGzMQNIDXZESDcMISJnGzRoEABmAEXGjgjpJrAjwoJVIucJLFj1+XwGt4bMih0R0k3fvn2RmpqKY8eOoaqqyujmEFGSlZaWIjMzEydPnmTBKiliR4R0k5aWhn79+gHg0CyRE6WkpLBglaJiR4R0xToRImdjBlA07IiQrhhCRM7GDKBo2BEhXbFglcjZWLBK0bAjQrrq168fUlNTcfToUezevdvo5hBRkp133nnIzMzEiRMnsH37dqObQybEjgjpKj09HX379gXAoVkiJ0pJSUH//v0BMANIHjsipDvOERM5GzOAImFHhHTHECJyNmYARcKOCOmOBatEziZlwNq1a1mwSmHYESHd9evXDykpKThy5Aj27NljdHOIKMl69+6NjIwMnDhxAjt27DC6OWQy7IiQ7jIyMliwSuRgLFilSHTtiLzwwgsoKytDbm4ucnNzMWzYMPzzn//U8yPJpDhHTGoxN+yJGUBKdO2IFBYW4oknnsDq1auxevVqXHrppbj66quxadMmPT+WTIghRGoxN+yJGUBKXCLJ1YOtW7fG3LlzMX78+KivraurQ15eHrxeL3Jzc5PQOtLLypUrMWTIEBQUFODQoUNwuVxGN4lUMsN5qCU3AHO0mYJt2LABAwYMQG5uLo4dOwa3m5UBdqblHExJUpvQ2NiId999F6dOncKwYcNkX1NfX4/6+vqmr+vq6pLVPNJZWVkZUlJScPjwYVRXV6NLly5GN4ksQE1uAMwOK+jduzfS09NRV1eHnTt3oqSkxOgmkUno3iX95ptvkJ2djfT0dNxzzz344IMP0Lt3b9nXzp49G3l5eU2Pzp076908SpKMjAz06dMHAIdmKTotuQEwO6wgNTWVBaskS/eOSK9evbB+/Xp8/fXX+OUvf4lx48Zh8+bNsq+dMWMGvF5v06O6ulrv5lEScY6Y1NKSGwCzwyqYASRH96mZtLQ09OjRAwBw/vnnY9WqVXjmmWfw0ksvhb02PT0d6enpejeJDFJeXo5XXnmFIURRackNgNlhFeyIkJykVwsJIYLmcsk5uMIqxYq5YQ+BK6wyA0ii64jIww8/jCuvvBKdO3fGiRMnsHDhQvzrX//CJ598oufHkkmVlZXB4/GgtrYWe/fuRWFhodFNIhNibthXnz59kJ6eDq/Xi++//x7du3c3uklkArqOiBw8eBC33HILevXqhcsuuwz//ve/8cknn+Dyyy/X82PJpDIzM1mwSlExN+wrNTUVZWVlAJgB1EzXEZGKigo9D08WVF5ejo0bN2LNmjW4+uqrjW4OmRBzw97Ky8uxatUqrFmzBtdff73RzSET4IoylFQsViNyNmYAhWJHhJKKBatEzsaCVQrFjgglVf/+/eHxeHDw4EHs27fP6OYQUZL16dMHaWlpOHbsGHbt2mV0c8gE2BGhpMrMzGxaIZNDs0TOk5aWhn79+gFgBpAfOyKUdIMGDQLAECJyKtaJUCB2RCjpGEJEzsYMoEDsiFDSMYSInI1F6xSIHRFKugEDBsDtduPAgQMsWCVyoL59+yI1NRXHjh1DVVWV0c0hg7EjQknXokULnHfeeQA4KkLkROnp6SxYpSbsiJAhOD1D5GzMAJKwI0KGYAgRORszgCTsiJAhGEJEzsaCVZKwI0KGkApW9+/fj/379xvdHCJKsn79+iE1NRVHjx7F7t27jW4OGYgdETJEVlYWSktLAXBUhMiJ0tPT0bdvXwDMAKdjR4QMw+kZImdjBhDAjggZiCFE5GzMAALYESEDMYSInI0FqwSwI0IGGjBgAFwuF/bt24cDBw4Y3RwiSrJ+/fohJSUFR44cwZ49e4xuDhmEHREyTHZ2NgtWiRwsIyODBavEjggZy+nTMzU1QGWl/79ETuT0DIiVnbKDHREylJNDqKICKCoCLr3U/9+KCqNbRJR8Ts6AWNktO9gRIUM5NYRqaoC77gJ8Pv/XPh9w9932uLoh0oIFq9rYMTvYESFDDRw4EC6XC3v37sXBgweNbk7SPPNMc5BIGhuBHTuMaQ+RUcrKypCSkoLDhw+jxsr/b5okdswOdkTIUNnZ2ejVqxcAYO3atQa3JjlqaoAnnwx/3uMBevRIfnuIjJSRkYE+ffoAcN7IqFZ2zQ52RMhwTpue2b4dkBuBvv9+oLAw+e0hMprTMiBWds0OdkTIcE4LoZISwB1y5rndwOTJxrSHyGhOy4BY2TU72BEhwzkthAoLgfnz/cOpgD9InnjC2lc0RPFgwao6UnZInRG32/+11bODHREynFSwWl1djdraWtnX2OmeeQAYP97f+XC7/YVn06db/xY8oliVlZXB4/Hg0KFD2Lt3b0KPbbfssCN2RMhwOTk56NmzJwD5URG73TMP+ENx2jR73YJHFKvMzExdClbtlh12vHUXYEeETGLQoEEAwkPIrife9u32uwWPKB5KGRArO2aHXXODHREyBaU6kUSeeGYaopUrOrP6LXhE8Uh0rZgds8OuucGOCJmCUggl6sQz2xBtaMGqxwO89JL1i86IYpXoglU7Zoddc8MlTFyiXFdXh7y8PHi9XuTm5hrdHNKR1+tFy5YtAQC1tbUoKCho+l5FhX9ItbGx+cQbP179sWtq/AESeHXk8QBVVYk9gWtq/FdhJSXqj1tT479C69HDvGFixfPQim12uh9++AE5OTnw+XyoqalBp06d4j6mXbPDbrnBEREyhby8PJSUlAAIHxUZP95/4ldW+v+rJUiA5MyrxnrVVFgIjBhh3jAhSpYWLVqgd+/eABI3PWPX7LBbbrAjQqYRaY44nhNP73lVOxbFERlBjzWFmB3mp2tHZPbs2Rg8eDBycnLQtm1bXHPNNdi2bZueH0kWptfCZvHMq6opUtN61WSWwjezYm44l9kWNzRTdtg5N3TtiCxfvhwTJ07E119/jaVLl6KhoQEjR47EqVOn9PxYsig9QyiWIVq1Q6ZarprMVPhmVswN5zJbRwQwR3bYPjdEEh06dEgAEMuXL1f1eq/XKwAIr9erc8vIDI4fPy4ACADi8OHDhraluloIt1sI/xZT/ofH439ezoIF/u9Lr1uwIP5jmoXR56HW3BDC+DZTbE6dOiXcbrcAIPbu3Wt0c2KS6OxwQm4ktUbE6/UCAFq3bi37/fr6etTV1QU9yDny8vLQ4/8uBYy+ItI63aLmqsmuixHpLVpuAMwOu2jRogXOO+88AMZnQKwSnR1OyI2kdUSEEJgyZQouuugi9O3bV/Y1s2fPRl5eXtOjc+fOyWoemYQZhmZraoDaWu1FatGK4uy6GJGe1OQGwOywEzNkQKz0yA4n5EbSOiKTJk3Cxo0b8c477yi+ZsaMGfB6vU2P6urqZDWPTMLoEJLmYm+4wT8IKgVALAsHhRaX2XUxIj2pyQ2A2WEnRmdArBKVHY7MDd0nioQQkyZNEoWFheL777/X9D7O8zrPZ599JgCI4uLipH+23Fys2y3E4sXa52MXLGg+ltsdPO9bXS1EZaX553glRp2HseaGEMwOK/vyyy8FANGhQwejm6JaorLDqbmh64iIEAKTJk3C+++/j2XLlqFr1656fhzZgLTxVVVVFY4cOZLUz5abi/X5gDZttI+EhK4NcNddwVc4dlqMKNGYG842YMAAuN1u7N+/H/v37ze6OaokIjuUcmPVKv/Xds4NXTsiEydOxJtvvom3334bOTk5OHDgAA4cOIDTp0/r+bFkYS1btkT37t0BAGvXrk3qZydqLlYplJ55Jr72OQVzw9mysrJQWloKwDrTM4nIDqXcGDrUhrfrhtC1I/LCCy/A6/VixIgR6NChQ9Nj0aJFen4sWZxRc8SJmov9v5Xqwzz9tD0XI0o05gZZrU4kEdlRUgK4XOHPO2G11RQ9Dy7Mu58emVh5eTkWL15sSAiNHw+MGhW8oZSaDakCX7NkifxrpFvu7Di0mkjMDSovL8cbb7xhmY4IEH92RPpR7Z4dunZEiGJh9NVQYWHzCV9R0Txv63b7r3pC7/MPfI3cFY3EbrfcEenF6AyIVTzZEYnds8MlTHz5wa28nenYsWNNi1cdOXIk4kJWelKzBbjca+QoBZEVWPE8tGKbqdnJkyeRm5sLIQT279+P9u3bG90kTRKZHdI0j9WyQ8s5yN13yXRatWqFbt26AYi/YDWejaLUrGgo95pQbjfw9dfWCxIio2RnZzcVrCa7aF1idHa43cDixer3t7EydkTIlKSh2XhCKN6NotRUwsu9xuUKLlqbPx8YPFh7+4mczMjpGaOyI/C18+cDY8bYty4kEDsiZErxhpDcPflaK8/VVMLLvebll7Xv1klEwYzqiBiZHXPmODM3WKxKphRvCEUaGtVyhSFXCa/2NU64kiHSi1EdETNkh9OwI0KmJK2wunPnThw/fhwtW7bU9H5p2DO0WCyWyvPASvhQgbfnjRih/dhEJG/gwIFwuVyoqanBoUOH0LZt26R8bjKyI/S2Xqd2QCScmiFTat26ddPS3rHUiSRjo6h455GJSFl2djZ69eoFILmjInpnB3MjHDsiZFrSqEisITR+vH61GtHmkeOpuCciP6OmZ/TKDjX1J07MDnZEyLQSEUJ6bRQVaR6ZVzxEiWHknTN6ZEe023qdmh3siJBpmXl1RaXb87Ky4q+4JyI/M2dALCLd1puIu3Wsih0RMi0phHbs2AGv12twa4IpzSOfPBl9ISMiUmfgwIEAgOrqatTW1hrcmvhFqj9RswiaXbEjQqaVn5+PoqIiAMatrhiJ3DxyIrYDJyK/nJwc9OzZE4B9RkWU6k+cnB3siJCpJXNoVioSW7VKfbFY6DxyMu7WIXISK0zPaM0OufoTJ2cHOyJkaolYYVVNMAQWiV1wgXyxWLRjSd8fNUp9xb0TK+SJtDByhdVkZEfg97TcrWOr7BAm5vV6BQDh9XqNbgoZ5JNPPhEARElJieb3LlgghNstBOD/74IF8q+rrm5+XejD4/F/P9qxtHzWsmXqjmkWVjwPrdhmkldZWSkAiC5duiTtM5OVHbHkhpb3GUnLOciOCJlabW2tACAAiOPHj6t+n1xASMEQatEi+SCRHosXRz6W2s8KDQ+XS137jGbF89CKbSZ5x48fb8qA2tpa3T9P7flcXS3Ek0/Gnh2x5sbcueqzzUhazkFOzZCpFRQUoEuXLgCAdevWqX6f2gr0igpg7Fjl43g8/lNd7ljvvtu8VHO0z5K7NU+I6O0jcrq8vDyUlJQASM70jJrzWZqOeeAB5eNEyo4VK2LPjWnT7Hd3DTsiZHqxzBFHq0CvqQEWL/af5KEdgsDXv/QScOGF8tt1T5niD6PVq6NXu8uFjtznOaFCnkirZNaJRMuOVauCOwdyomXH2LGx54bPB7hckd9nNeyIkOnFEkKRKtClq5kbbpAPkxdeCC4WCz1WIJ8PmDED+OMflavda2qA2trw0AkNk5tvdkaFPJFWyeyIRMuOIUPkc+Ppp4GVK+WzI/Tcjyc3PB7g2muDn7N6driEULoeNF5dXR3y8vLg9XqRm5trdHPIIJ988gmuvPJK9OzZE9u2bdP03pqa4C22a2r8nRClqxmPxx8iSjtmvvuufyQk1OLF/pEVlwsYNqz5/RUVzVdPLpf/4fP5Pyd0eibSZxvJiuehFdtMyiorK3HppZeiqKgIVVVVSflMLdkR7dxdvNh/4SP3vNbceOKJ8OkZM2aHlnOQIyJketLV0HfffYe6ujpN7w29Xz/SFInLBdx/f+RjjRkjP7Ixdqw/aMaOBZYs8T8fOr8rdToWLwbefps1IkRqSRtg7t69G0eOHEnKZ2rJjki5AchP0cSSG1VVQHk5a0SIkq5Nmzbo3LkzAG0Fq3Lk5n8lQgDz5kXebCp02FY6ltz+EErzu23ayAeT1ed5ifSSl5eHHv93chi1sFmk7EhWbhQW2nMFVnZEyBISNUccqd5DorTZlNyCZe+8ozyyESkwnLyKIlEsjF5hVaneQxJpk7qaGqBbN//dMvHkRmA77JQd7IiQJSQyhKTVC596Svk1gbfnAuHbcy9Z4h+2jTSyES0wtKyiSOR0RndEAP85+s47yt8PzQ0gODuGDgV27owvN6R22Co7dF/VJA5clIgkH3/8sQAgevXqlbBjRloVUXqoWUBowQL/19LzoascVlcLUVlpvgWH1LLieWjFNlNkn332mQAgiouLDW2H2txYsCDyomXMjWa8a4Ys4dChQ2jXrh1cLhe8Xi9ycnISctyKCv9wamOj8mvcbvkitcpK/6gIEF5hbydWPA+t2GaK7Pjx42jVqhUA4PDhw8jPzzesLWpyw+PxF6XL3S0jZQdzw49TM2QJbdu2Rfv250OI4ViyZFPCjhs4xLlypfx0jZoFhOR20ySixGnZsiW6d+8OAFi7dq2m9yZ6gzg1udHY6M+NaPUezA12RMgiKiqAAwf+DaAS119/gWJ1eiykMBg8WP72XI8n8sJDRJQcsdSJhNZ3JSo71OTGsGH2KyzVAzsiZHrSffXSn6sQbsXq9MD3xHIFJFcoNns2cP75zRXvtigOI7IgrR0Rub1a9MgOpQJTIPhuGWaHPHZEyPTUbmAnifcKKHDY9YkngOnTgyveeTVDZAytHZFkZkfonSxA+N0yzA55LFYl05NbWtnjEaiqcoWd2PKvjb78sbSQUElJ8F4PiTqWlVnxPLRimym6Y8eOoXXr1gCAI0eONP1vJVrOYaOzw8m5wRERMr3wRcgaMHXqDtmTVesVEBB+FTRvXuKOFet8dKKL64jsoFWrVujWrRsAdQWrWhb/MjI7HJ8bOt9KHBeuBUCBqquFuPDChwXQSfzpT39SfE2kNT/UvB4QYs6cxBwr0uuVLFjQfBxpPQIjWfE8tGKbSZ0xY8YIAOKJJ55Q/R41a3IYlR3MDSE4IkKWUVgIjByZBmCv4hyx1uWPlTaymj7d/99bbgl+Xm67bekq5Kuv4t+MKpbiOiInkepEtNzCq+Y2WaOyY/Fi5oauHZHPP/8cV111FTp27AiXy4UPP/xQz48jB1BTrKZl+WOljax8Pn+l+xtvBD//5pvKyzffeGP09UaiiWV42I6YHaREz6XejciOBx4IP4bTckPXjsipU6fQv39/PPfcc3p+DDmIFEJbt27FqVOnFF+ndqGgwkL/GiGhPB7/IGmkk1vuKiRwASOPJ/r24KHsuLNmLJgdpGTQoEEAgJ07d+L48eMJP74R2RH6/iee8Hcu1I5oWD43kjBVJIQQAoD44IMPNL2H87wkp0OHDgKA+PLLLxN2zOuuC56jHTcu+tzt1Kny+0wsXuz/nsslP19bXS3EsmXKc8CBe1C43f45ZyMZfR4yOyhU165dBQDx2WefGdqOBQuCz32XK/oeM0IoZ8fTT/vPd6Vaj0jZYeXcMFVH5MyZM8Lr9TY9qqurGSYUZvTo0QKAeOaZZxJyvL//PTwQom1MVV0tHyQejxArV0be6Er6nsvlDyS5UAncaM/owjOj/0+d2UGhrrvuOgFAzDHw/20j5YYQxmSHVXPDVMWqs2fPRl5eXtOjc+fORjeJTCiRc8S33QZcdVX489IwqtKc8VdfyR9vwgTg5En5YdkVK4KHY4Xw3+4XerteTQ0wbZp1C8+MwOxwFj3rRNSIlhtA8rPDyrlhqo7IjBkz4PV6mx7V1dVGN4lMSCmEtN5Dv2oV8Prr8t+LdWOqSy+Vn691u+XnjYHwwLB64ZkRmB3OksiOiF65ASQmO5RqToDg7LBybpiqI5Keno7c3NygB1EoKYS2bNnSVLCqZkGg0MD54gv547tc0TemuvDC8Dtk3G7/JlfSbYCB3xcC2LhR+XiBgWH5wjMDMDucRSpY3bFjB7xeb8zHSXZuANGzQ+62365dlY8nZYeVc8NUHREiNTp27Ij27dvD5/Nhw4YNqu6hlwucH/9Y/vj//d/RN6YqLATmzAm+Q2b+/OYQGjUqvCMye7by8TweICvLH3gAd+wkiiQ/Px/FxcUAtK0nEsiI3AAiZ0dNjfxtv7t2KR/P7QYOHfL/b6vmhq4dkZMnT2L9+vVYv349AGDXrl1Yv3499uzZo+fHkgMEDs1GG5JUCpwOHYBx44LfN24cMHp09M+vqGiej3W5/J2MwBCSa5P02lBut/+qZ+jQ5sAD1K9nYEfMDoom3ukZI3IDiJwdSm0KXBYglBDADTdYPDf0rJqtrKwUAMIe48aNU/V+o6v1ybx+85vfNP0tRbtVbtky+Sr1ykr/91eu9N82t3Klus9WsySz0mvmzAm+xW7q1MiV8mZgxHnI7KBofv/73wsAYuzYsTG9P9m5oeYzI30/8I4Yj0eIe+6xT27oOiIyYsQICP8twkGP1157Tc+PJQcIvBqKtjRztLnTwYOB++7z/1cNpauWd99tHtZValPr1sGV76WlypXyVigy0wuzg6KJd0Qk2bkBKGfHihWR27RkSfgoyvXX2yg3dOsOJQCvakhJTU2NACDcbrc4deqUECLyxlZK9/THQmmzK6UFiKQ2KV3tcEQk8azYZtKmtra2aaTs+PHjMR8nWbkhfZZcdjg9N1isSpbUsWNHtGvXrqlgFYh8q5yWPSSiCb1qCRRa8Ca1CVDe3OrUKesWmREZpaCgAF26dAEArFu3LubjJCs3pM+aPz98pEUpNwoLlTfTtFNusCNCluRyuTQPzRYW+odVI+3hoHZNASmgnnoq/Huhw6NqNrdKdOAROUEyFjZLZG4A/nP7nXfCn5ebVqmoAMaODX+t3XKDHRGyLK0hFG3NADVrCgQqLATGjIk8jxxpcyu32z/XK13BaFn8iIiS0xFJdG4A/rVEoq35IWWHEMGvs2NusCNClqUlhKKtGaBmTQE50Qre5IrTJD4fMH26uuAionB6d0SMyg1AOTvsmBvsiJBlSSG0efNmnD59OuJro60Z8Mwzyt8PHXYN/TrS8Khc5X0gK+0HQWQ2UgZ89913qKurS/jx48kNIDgrtOQG4M8OuXWHAPvlBjsiZFmdOnVCfn5/NDb+GEuXbon42ki34tXUAE8+Gf4ejwdYvTp42PW22+SHYZWGR5WK0wJZ9pY7IoO1adOmaYNDLQWrams6Ys2NHj2Cp2y6dPE/1OaG9D25ujKJnXKDHRGyrFdeceHIkbUAKnHNNQMiDlVGGgrdvj18HhYA7rwzfDfL11+PPEwrF27jxwN/+5ty26yyHwSRGSW6VixQLLlx//3+/4buliu9Vm40Qyk7Jk9WvoixU26wI0KWJM3NSn/CQrijDlXKDYXW1AC1tfKbUF16qXJ9h0S6KokWbllZ8u93u617yx2RGSSyVkyO0hRKdrZ8bkyeHLk2DAgezYiUHYWFwJQp8se4/3775AY7ImRJsW55HTgUKgXADTf4vyeFirQJlVxleyi3G9iyJXq4Kc33fv21dW+5IzIDLR2RROQG4M+OoUODR0QCN6+LVhsmbVS3alX07Jg8Wf4YSs9bETsiZEnyc7dC9VBl6JWREP7jvfAC8Pbb/t1z5YZlx41r/trl8r/v3ntjCze327+BFhHFLrBg9cSJExFfG23ZdjXkbsl3u4Fnn/XnBhCeHW5384WIlBs33AAMGaIuO+RGXuzEZj8OOUXziS5dkjRg+vRdqocqla6M7r23eSfLiorwYdnXXvPvC/Gb3/jfIzdHDISHm9x8ss9nn2IzIqO0bdsWhYWFEEJELVhVc9tsNEo7a997b/DUSmB27N4N/Pvf4bkhlx9OzA52RMgy5G9/c2Hw4IcAFKNTpyWqj6U0dCpXUBY6nTN0KPDb30buhMyeHbwSYyKuxIhIXrTpmcDsiHc10kjTLkpLtS9ZEj03AOdmBzsiZAlKBV2FhcDll6cC2KtpUSO5odNQoUOkkVZJlbjd/sWGpk8PbmthIXDLLcGvvflm+xSbERkpUkdELjviWY000l5TQGy5ATg8O/Tfgy923EGThFDefVLaZfK9994TAMSAAQNiOnZlpbqdLJctk99xV24nTavtlBmJFc9DK7aZYvfRRx8JAKK0tDTo+WjZEY/qaiEWLxbC5UpMbtgtO7j7LtlKtEp36Wro22+/xZkzZwCoX7BIujIaPDj63HG0SnjA/325tn75ZWwFrUQUnZQB27ZtCypYjeUuGS3ZMWYM8PLL2nMjsHg18DmnZgc7ImR60eZIu3Tpgvz8fDQ0NOCbb74JG4qdN0/d50SbOy4sBP74x+a2hN5F4/EATzwh39aLLrL/PC+RUdq1a4dOnTpBCIH169c3Pa+1viKWDezU5EboRc78+eEdGCdnBzsiZHrRKt1dLlfTFdH//M/WsPvyH3wQmDtX/WcpzR1XVDSvtOpy+YvKXnvNHz6LF/tv+73xRvm2qhlxIaLYydWJaLlLpqYGmDAhODvuukvdfi5qak4Cjws0d2CYHWCNCFmHVM8hNzc6Y8YMAUD89KdzFedeY5lTra72z/FGmqedO7f5e263EAsWKLc10s9gVlY8D63YZorPY489JgCIm2++Oex7as67RYvk6zYWL46tPU7PDi3nYIqRnSAiLQoLla8CpKuh3bv/B273VNn7/Hfs0HYVUVHRXO2uNH/7+OP+q5PAz7n7bv+VzogR2n4GIoqdlAFr164N+16yzztmhzacmjErtRVTWl9rU82rKy7D44+fC/t+vKsnyt1653b7h0xD2a2QjCxEaxbYKDukDNi6dStOnTql+f0XXhheQOpyAcOGaTsOs0M7dkTMSEvFVCzVVTZUVFSE1q1b49y5c7j88g2YMye4qDQRqycCwcecMEF+cSK3216FZGQRWrPAZtnRvn17dOzYET6fL6hgVa3CQn8BqXSOu93+r7WOQjA7YpCEqaKYOXKeV8uN73reJG9Bl19+uQAgXnzxRSFEfHOqSr/alSv9xwyc2w19zJ2b0B/LcFY8D63Y5rhozQKbZsdVV10lAIhnnnkm5mPEW4vB7PDjOiJWpuXG91i3krSp0Kr5RK6eGFjB3qNH890zgVwuYM4cYOrU6Me30Yg4mYHWLLBpdmjZiVdJPLkhvZ/ZoQ2LVc0mO1v++ays8Oekm+QD/6rtdoO5BokIoUDjx/t309yxw/8rlYJJaeh10SL/AkfRhBayzZ+vfb8LoiBacgOwbXYkOgNixezQhiMiZnPypPzzcsVXidhK0kakEPrmm29QX1+fkGPKXR0pLZIUWtQmd+UiV8gWuEkWUUy05AZg2+yQMmDLli0xFawmUqzZoTTiYefsYEfEbLQuBRjvVpI2UlxcjFatWuHcuXP49ttvE3rswHCQMjywqC00w5XqAG06Ik5Gi2WLVhtmR4cOHdChQwf4fD5s2LDB6OaEdSqiZUek+mE7Zwc7ImYTy5VKvJOaNhG4wmoih2a13lwgt0KjdOXihC29yQCxjnDYMDvMMj0TS25EGvGwc3awI2JGNrxSSZZEh5BcONx1V+TAeOaZ8FvzpCsXm46IkxkwNwCYoyOi1KlYtUo5O6KNeNg5O1isagbSX2FJSfNflZ2X0dNRokNILhzkis0CA+Opp8K/H7g+gFIhG5EmzA1ZZuiIKHUqIu2kq6be2K7ZwRERo9lsUSGjBRasnj17Nu7jZWfLb+GtNESqVBU/ZUpwaNhwRJySibmhaNCgQQCAzZs344cffjCkDUrTKJF20lVbb2zH7GBHxEh2LoM2SNeuXdGqVSucPXs27oLVigpg6NDwOxznz1ceIlUKoMmT42oKUTPmRkQdO3ZEu3btDC1YXbIkeHpWKkqNtJOunWtAomFHxEh2LoM2iMvlaroiimdoNjTrAX9IrFjhHx4NnY4fNcr/vwF187h2XJSIkoS5EZFeRetqSdkRWic2apT/v4HZsWIF0K1b8N14TswOdkSM5OQusI4SEUJKtSGBw6TSEOmSJcGj5EDkmkGOqlNcmBtRGdkRUcqOwH5iYSGwc6d/xDUwB6LVG9s1O9gRMZKdy6ANlIgQUpv1SqPkgPw8brJH1e149eR4zI2ojOyIqMmOSDmgVANi5+xgR8RovOUu4aQQ2rhxY8wFq2qz3sxbfNj16onA3IhCyoDNmzfj9OnTSf1sNdkRSw7YOjuSsAmf+POf/yyKi4tFenq6GDRokPj8889Vvc9xO2hSQvh8PtGyZUsBQKxduzauY61cKcRTT/n/K8esm54m8nOMOg9jzQ0hmB1O5/P5RNu2bQUAsWLFCkPaECk7Yjk/rZYdptp9d9GiRbjvvvvwyCOPYN26dfjxj3+MK6+8Env27NH7o62L4+lxSVTBqnTXzJQp/v/KXRVoHSVP1qi61esZmRsxYnYAML5gNVp2xLqAtm2zI8ZOk2oXXHCBuOeee4KeKy0tFdOnTw977ZkzZ4TX6216VFdXO++qZsGC5u6o2+3/Opmqq4VYtizx3ewke/DBBwWAsL89tWIZ6aisVP9r0/p6raw+IqIlN4RgdgghmB0hfv3rXwsA4vbbb0/q52o592LJAatkh5bc0LUjUl9fLzwej3j//feDnv/P//xPcfHFF4e9fubMmQJA2MMxYZKssTclRgdZAi1cuFAAEIMHD47p/cuWBf8zSI/KysS2U08LFvj/fKQ/o1j/OZPdEdGaG0IwO5gd4T744AMBQJSVlSX1c5kdfqaZmjl8+DAaGxvRrl27oOfbtWuHAwcOhL1+xowZ8Hq9TY/q6mo9m2c+Ro6n22yRpMCC1XPnzml+vx3ukLRqPaPW3ACYHcyOcFIGbNq0KakFq8wO7ZJy14zL5Qr6WggR9hwApKenIzc3N+hhGYmYmzXyL9jqRQUhunfvjry8PNTX12Pz5s2a32+XOyStvBy02twAmB3MjnCFhYVo06YNGhsbsXHjxiR+LrNDK107IgUFBfB4PGFXMYcOHQq72rG0RN3rZORfsB268QESUbCqtAIi6csxuQEwO3RkZMEqs0MbXTsiaWlpKC8vx9KlS4OeX7p0KS688EI9Pzp5Ej0sadR4ul268QESEUJKKyCSfhyRGwCzIwmMvHOG2aGB9hIUbRYuXChSU1NFRUWF2Lx5s7jvvvtEVlaWqKqqivpeS6wFYIfKpEB6l2Qn0TvvvCMAiCFDhsR8DKNrAM3AiPMwntwQgtlhCBNmx/vvvy8AiP79+yf9s52eHVrOwRS9Ozo33HADjhw5gt/+9rfYv38/+vbti48//hhF0qYcVicNS4Zu0Sq3Fvj27f7Xm+BKQVFhobnbp4F0NbRhwwY0NDQgJUX7n3uk6W+b/JpMyfa5AajLDqvkBmDK7AgsWD1z5gwyMjKS9tnMDg2S0DGKmSWuaoSIfq9T6K1tc+aou9/eqPvyTbYeQKwaGxtFTk6OACA2bNgQ0zGcflUjhIXOwwCWaXOk7LBabhj92TJ8Pp8oKCgQAMS///3vpH6207PDNOuIxMsyYSKE8rCk3F+j9Ih0v71R9+WbcD2AeAwfPlwAEK+88krMx0jUehxWZanz8P9Yqs1y2WG13DD6syMYNWqUACCef/75pH+2k7ODHREzUZoHjtRFNqorbcMu/JQpUwQAMXHixLiOY8Lp76Sx4nloxTYHsVJuGP3ZUTz88MMCgBg/frwhn+/U7DDNgmYE+VvbAsndb2/UffkmXQ8gHomqmrfyehxkQVbKDaM/Owoj75wBmB1qsCOit9Bb20LJFbZGuy9fr42tTLoeQDxCC1aJLEGP3AAcmR1SBnz77bc4c+aMwa0hOeyIJMP48cD06UDoqpBK99svWeIf3JS43c2vS9QCSHJMvB5ArEpKSpCTk4PTp09jy5YtRjeHSD2tuVFYCNxyS/BzN9/c/DqHZkeXLl2Qn5+PhoYGfPvtt0Y3h+QkYaooZpaf55XMnStfcLZyZfhr5eZa3W7/88mah7XZpObFF18sAIhXX33V6KZYkhXPQyu2OYyW3BAicj44PDtGjhwpAIgXX3zR6KY4BmtEzKSmBpg2Lfx5nw84dSr8ebm5Vp/PP9eqNA/77ruJHWq12aSm0XPERJppzQ0gcp2Gw7ODGWBu7IjoTS4AAP90i9z8aaS5VqUCtilTuH5wBAwhshytuQEwOyJgBpgbOyJ6UwqAJ56Qv2qQm2u9/3757wUyydbbZiSF0Pr161mwStagNTcAZkcEUgZ88803qK+vN7g1FIodEb2FBoDbDcydCzz4oPJ7pM2rpk71h8S8ec1XLdL3nnoq/H0muV3ObHr27Ins7GycPn0aW7duNbo5RNHFkhsAs0NBUVERWrdujXPnzrFg1YTYEUmUSLfFBe6KuXu3PyTUeOqp5rtnAq9aCguBMWNMe7uc2bjdbgwcOBAAh2bJhJSyI9bcAJgdIVwuF6dnTIwdkURQui0uMGC0FnFFWyDIxLfLmRFDiExJLjviyQ2A2aGAGWBeuu++a3s1NcBddzWf+NLVx7Fj/qp3n89/9TF/vv8KRy01O3OOHw+MGuUPmB49bB8k8WAIkenIZceECf51Q2LNDYDZoYAZYF4cEYmX0tWH1AkBtBeDSVt///GP0a9aTHq7nNkEFqw2NjYa3BoiyGeHELHnBsDsiCCwYPXs2bMGt4YCcUQkXnJXH6FfA81Do9FO+oqK5qskt9tfJT94sParFimQSkocEzSR9OzZE1lZWTh16hS2bt2KPn36qHoff42kG7nsCKU2NwBmRxTFxcVo1aoVjh07hm+//RaDBg3S9fNs+mvUBUdE4iU33yoNrwaKdP+/RG6odsYM7UGi51LOFuXxeDQXrPLXSLqSuzMmNDcAYNWq6MdidkTlcrmaOh96T8/Y+NeoC3ZEEiH0lrmXXgreKwYI/1pOInawVKpZsfEaAWppmSPW+9eo195jZDGB2SEtvh5qxozofyjMDlWSUSei56/RrrnBjkgiBd4yF0qI6KGQiB0sTbwdt9G0hJCev0ZeLVGYSNmh5g+P2aFKMjoiev0a7Zwb7IjES+qifvVV5LleNaGQiNvqTLwdt9GkEFq3bl3UglW9fo0OuOgkNQIvbZWWc5cwOxJGyoCNGzfqVrCqx6/R7rnBjkg8AruoN94oP78r8fmAJUvCnw8dawtcxKiqSvutew5dI0CNXr16ISsrCz/88AO2bdsW8bV6/RodcNFJ0YRe2q5ZI7+cu0QuO+TG6JkdUXXr1g0tW7bE2bNnsWnTJl0+Q49fo+1zIwm7AcfM1Ft5y22r7XY3P+fxhG/hHbrt9oIFza93uYSYM0f9Zy9bFnmrbZNux220H/3oRwKA+Mtf/qLq9Yn+NSZrN/ZEMvV5qMC0bVb6A5gzpzkzomVHaG5MnaruD0hNbkivs3F2XHrppQKAePnll3X9nET+Gu2eGxwRiZVcF9XnAxYu9F+RvP12+HsCu7ChY21CAA895N8bQklNjX+viS5dIq/iCjhujQC1tM4RJ/rX6ICLTopE6dJ28ODm0YxI2SGXG4H7yciJlBvS9x2UHcla2CyRv0bb50YSOkYxM+1VjRDRu6jRvr9sWfhVj3SFI9fNDbwKCh2Fuesu//ukrxcsSN7vwWJef/11AUBcdNFFhrbDShedpj4PFZi2zWoubSO9Rik3pHM/9A9KKTc8HiFWrvSPpjgsOxYuXCgAiMGDBxvdFM3smhscEYmFmtULI3Vha2qA2lr5YwsBPPNM+OcFXgUF8vn8nyO3wRWF0VKwqiebX3RSJFOmRL60VcoOwJ8bSrVoPl9wdkTKjcZGYMgQ/2iKw7IjsGD13LlzBrdGG9vmRhI6RjEz5VVN4BWG2+2f243URQ3twobO78pd2YReIUW6ClJ6VFbq/IuwpoaGBtGiRQsBQGzevNno5liCKc/DKEzZZq21HYHZEfpeNdkRS244IDt8Pp/Iy8sTAMS6deuMbo5tcUREL7GsXhjYhZWb35W7ugkth9Y6l2mzW+4SyePxYMCAAQCAtWvXGtsYcg65c//ppyO/R8oOIPy9bjcwenT4ewKzQyk3It3d54DsSOYKq6QOOyJaqL2HSmn5O6VNrkKDITAMamr8G+iFuukm+Vv+3O7IVUx2XZpPA+7CSUmnJju05EZjI3DrrcoLVijlxt13A//+t/bssFluMAPMhR0RLdSsVBNp+Tul90eqNVFa7OjOO8PnkadOBXbvVl4/wM5L82nAEKKki5YdseTGsGHKdWhKuTF2rP8OHS3ZYcPcYAaYTBKmimJm2nnewPv9A6vM1VTEK71fqRxazd050cqoq6uFWLQofF7Z7Dei6+Sbb74RAEROTo5obGw0ujmmZ8rzMApTtjnSuR9rbkjvD80AtXfnRMuOlSttmRvfffedACDS09PF2bNnjW6OLWk5B9kRicXKlUI89ZT/v4GUisMCi7+kTsHixepP5kghpOa9crfvKbVNzYJHFnfu3DmRmZkpAIitW7ca3RzTM+15GIFp2yyXHWbMDen9StkRWtBqsexobGwUubm5AoBYv3690c2xJXZE9BR614yWEZFI740mlhvI5dqjR9ssaNiwYQKAeOutt4xuiumZ8jyMwpRtVjrHzJYbSm2KNFpjwewYMWKEACAqKiqMbootsSOil3inXpK9Rm+k2/eMbpvBJk2aJACIKVOmGN0U0zPdeaiC6dqsprNhltwQQjk7tF58mdgDDzwgAIh7773X6KbYkpZzMCV51SgWJS1eVlISufJdKi4dPx4YNcr/XOBtvc88E/29sbZL6f1SkVvg53o8wDvv+AvdIhXExtM2C2CxGuku8ByV25078BxLVm6EtktLdrjdwNdf+4tdJRbODmaAiSShYxQzw69qQoccx4yJPkwpp7pafgGiSO+NNOeqZShUzTyxha9qYrVhwwYWrKpk+HkYA8PbrGYBsmjnWKJzI7RdDs+Obdu2CQAiIyNDnDt3zujm2A6nZhIhWn2F9FDaMVcKhJUrhXjySfn3Tp0q/9546lCU2hJtnjjewjaLOXfunMjIyBAAxLZt24xujqkZ/n/qMTB9dih1AvTKDaV2OTg7AgtWN2zYYHRzbMc0HZHHH39cDBs2TGRmZoq8vDzN7zc0TNQujyxXPR64kVSkIJI7uWPdLC8RyzJbaUelBBgyZIgAIN5++22jm2JqyT4P480NISyQHYsXB79H79yI1C4HZ8fw4cMFAPHKK68Y3RTbMc0S72fPnsWYMWPwy1/+Us+P0YfcIkKhlBYzC9xISul98+eHz6HW1ACLF0degVHNomqxsu2OSvI4R2xOls4NIHp2SIuRSZKRG0rtcnh2SBnA7R4MloSOkXj11VetNyIiRPiQ47hx2hYzk3s8/XTwYmTPPy/Eo48K8cgj2m6Xs+BQqNlUVFQIAGLEiBFGN8XUjDoPY80NIUyWHW5380hHonJj2TIh/v53Ia6/Xn1uhLaL2SHeeustAUAMGzbM6KbYjmXvmqmvr0d9fX3T13V1dQa2BvKV7I8/Hl7ZDigvqRzI4wGuu87/vnnzgAcfjN4GuW3ClSrsSZPAqyGfzwd3tBEwMi3TZweQmNyoqAjeAC/Se+T2jWF2BJEyYP369WhoaEBKiqn+L9ExTPVbnz17Nh577DGjmxGssDD4ZA39WlJS4t+8TmloNTAY5s4FHnoo+mc//XRzAEkCb72TduakmPTu3Rvp6emoq6vDzp07UVJSYnSTKEaWyY5QcrfJBgrciK6mBpgwIfL0DRA9N5QyzIFKSkqQk5ODEydOYOvWrejbt6/RTXIkzZeAs2bNgsvlivhYvXp1TI2ZMWMGvF5v06O6ujqm4xiisNAfEnKefhqoqvJfjSjtihkq8CpIYsPNp4yUmpqK/v37A2CdiN70zA3AwtlRWOiv+wjdgVuycGHzRnRffRW9E8Lc0MTtdmPgwIEAmAFG0jwiMmnSJIwdOzbia4qLi2NqTHp6OtLT02N6r+EqKoCXXw5/PjQYtm9XFyahw6o1NcFDsj6ff0vvUaMiX/lQROXl5Vi5ciXWrFkT9e+aYqdnbgAWzw4loUWt0QSOnkjU5ob0WgdmR3l5OT7//HOsWbMG48aNM7o5jqS5I1JQUICCggI92mJd0ske2sGQ61BkZytP4fz0p/66Ebm5WzUrGAbOH7vd/istuW29qQnvnEkO5oYCLdnRtavycS6+GHjrrdhyA3B0djADjKdrdd6ePXuwfv167NmzB42NjVi/fj3Wr1+PkydP6vmxyadUcPbcc8Enc0UFMHSofCfkkUeAjz5SvgUu2q13clc+d93lf54UBRasimgjVZQUjskNQDk73nlHPjvkPPIIsHx5bLkBOD47AgtWGxsbDW6NQ+l5+864ceMEgLBHpcoFdAy/BU8tpVvwXK7Im1e53UK88ILyAkXSUs3S/3744eZjhN56p7RYkdIqjCSEEOLs2bMiPT1dABDbt283ujmmlOzzMN7cEMIG2RG4YrOW7NCaG0IIsWiRo7OjsbFRZGdnCwDi22+/Nbo5tmGalVXjZZkwEcK/xkike/m1rGoYbZ8Kt1uIuXP9r5WO/cgj6tcSkETbl8IhBg8eLACIhQsXGt0UU7LUefh/LNPmBQuin7dqs0NLbgjRvJqrlnVIAt9ro+z48Y9/LACI119/3eim2IZpVlZ1jJoa4I035L8nzceqXdUwdJhUioVAPh8wfbp/LRKpGv73v4/8+aFYSd+Ec8RkCOlcl6N1NWUtuVFT4z/fu3TxZ4jSrcMOyo5BgwYBYAYYhR2RRIi0KJEUGNJteh5P8/NyCw6pWeAI8IfEtGnqFjaK1tmRKukdMiccih0RMoSa3ADUZYeW3FixQr5ANlIbJDbNDmaAsUy1oJlpaL2NTWlRIikwAKCy0n/LXFWV/KqG0mdmZ0de4Eii9jVqOztylfQOEVqw6lJa04EokkTnhrSI2fbt0bOjtlZdJng8/g5IrCuz2jQ7pAxYt24dGhsb4ZE6fZQc+s8Uxc6Qed5oW2lHel/g3hJTp/rnT9VszR2466bLJcR11zUfy+WS31Vzzhzlud3Az5c+I3A+N5btwG2svr5epKWlCQBix44dRjfHdCxTbxEg6W1OdG6oOabcjr2BOSJ3ji9YEHl/m0ceCd5F1yHZ0dDQILKysgQAsWnTJqObYwssVo1VvCdZ6FbY0Y4XGDSRAkE67sqVwcefOlX+vYFbjCuFGTe/CnL++ecLAGLRokVGN8V02BGJItG5oeaYkbLjnnsi54YQytkRWADrsOz40Y9+JACIv/zlL0Y3xRbYEYmVljtb4j1etF033e7oQVZdHbnSPVqYyQWgQ919990CgHjooYeMborpsCMSRaJzI9oxmR26+M///E8BQEyePNnoptiCZXffNZzcnK1cwVYijhetuMzniz7vumRJ8CqtLlfwvG60+VxuftWExWoUM7nz3O0GsrISe0xmh66YAcbhXTOBpOr0wFvlfD7/KoeVldorwwsLgSeeaD5eYAGY3C15gaJ1gOSWhna7/UVtErW3DBNXWKXYhd7VAvhzY+hQ/07bzA5LCC1YpeRhRyTUqFHBJ6gQwEMPxXbPfEWF/759af+G6dOBbt38QRAaXi5X8w6cShXrgSJdsUjU3jJM6Nu3L9LS0nDs2DHs2rXL6OaQ1Ywf778tNvCOK5+P2WEhpaWlaNGiBU6dOoXvvvvO6OY4i/4zRbEzZG5aaW5WaxFapHncwMKvwLlWLfOuWgrkbDifq4fy8nIBQCwOLPYl1oioxeywvAsvvFAAEG+88YbRTbE8rqwaj2jDnkqrDYaKNI8buAhQYWHzRneB/zsaLVcsWo7rYJwjprgwOyyPGWAMdkRCyc33BlI7T5qoUIpk/Hj/IkeVlf7/OmTbbr0E1okQacbssDx2RIzBjoicwJN07tzY5kkTFUpqPif0iqWmJrYCOYcLDCHBglWKBbPD0gILVn1qlsynhHAJEyduXV0d8vLy4PV6kZuba1xDamrkl1bW8t7Vq/0FZ42NzaGkx1VIRUXzXhButz/QeLWjSn19PXJycnDu3Dns2rULxcXFRjfJFExzHmpgmjYzOyyloaEBubm5OH36NLZs2YLS0lKjm2RZWs5BdkSSKZ5QUnv8oqLwtQeqqhwzxxuv8vJyrF27Fn/9619x7bXXGt0cU7DieWjFNkfE7EiaCy+8ECtWrMCbb76Jm266yejmWJaWc5BTM8mkpvArnqFRNbflUUScIyZTYnYkDTMg+dgRMZOKCv9VibTuwIMPagsVBy5ClGgMIbKk0OyYN0/b+5kdTZgByceOiFlIqx1KVyU+nz9MunRRvxCSAxchSjQWrJLlyGXHgw/6i2XVYnY0YcFq8rFGJJlqavxDoCUl4ZXqixcDDzwg/z6tc7V6zyfbWGDBalVVFYqKioxukuGseB5asc0RyWWH9FxtLXDDDeHvcbuB3bu1ZQCzI6hgdevWrejVq5fRTbIk1oiYUejQqTTKIT2v1AkBtM/VOmwRokRKT09H3759AXBolkxCLjsCnxs7NnhpeYm0+Z0WzA6kpKSgf//+AJgBycKOSDLU1AATJgQPnd51F7BqVfCQqhK3Gzh0SHsRmsPXBIjVoEGDADCEyATksmPChODnlAa1PR7/DsBaM4C5wTqRJGNHRC+BJ/NXX4WHhc8H/OMf8p2Q669vLhyTtuq+4QZtG2cpjcBQVAwhMlS07JB2iAl97u67g3frvflm/w7AWjKAuQGAqywnGzsiegg9mZctk39dhw7ylepPPumf2128uLkjAgTvMxGJXPGamvcRABaskoHUZkfoVIzHA/z61/7cqKz07wT8xhvaMoC50SSwI8KCVf2xI6JVtGFLuZN5wYLw17lcwOjRypXqhYVAQUFs9/bHuyaAw4dmy8rKkJKSgsOHD6O6utro5pAdqDmntGTHnDnKuTFiBHDypPYMSMRaIjbJjt69eyMjIwN1dXXYuXOn0c2xPXZEtFAzbKl0Mk+d2jz64XYDL7/sD41Im0/Fem9/PGsCcGgWGRkZ6NOnDwBOz1ACqD2ntGTH1KmRN62LJQPiXUvERtnBgtUkEybm9XoFAOH1eo1uihDV1UK43dLsrP/h8fifV/u66mohKivD3xPJggX+90vHWbBAv/ep/Rkd4I477hAAxCOPPGJ0UwxnqvNQJdO0Wcs5ZYbsiDVvbJgd9957rwAgpk6danRTLEnLOcgREbXUDltGWhgollvjQkdMRo1SN/QZyzbfXOa5CQtWKSG0nFNmyI5YcgOwZXYwA5InxegGWIY0bBm6KZTcsOX48f6TPlELA0lBpHV3TOl9amn5GW0utGDVJbdOA1E0Ws8pM2SH1twAbJkdgQWrzAB9cURELa1LICd6YaBkVLRzmecmZWVl8Hg8qK2tRY3FC+/IQLGcU8wOU+jduzfS09Ph9XpZsKozdkS0iHXYMhGSNfRp5M9oIpmZmSxYpcQw+pxidsQkNTWVBatJwo6IVkYtgZzM3TG5zDMAzhFTAhl5TjE7YsYMSA52RPSgx730Nhz6NDuGECWVXmtwMDtixgxIDnZEEk3Pe+ltNvRpdlxhlZJG7zU4mB0xkfadkgpWSR8uYeLfruW28q6p8YdIaOV4VZX2rbhDt/ympDt9+jRycnLQ2NiI6upqFDr038Jy5yEs1mbmhmmdPXsWOTk5OHv2LHbs2IHu3bsb3STL0HIOckQkkRJRFGaj1QmtLjMzE7179wbAoVnSEXPDtNLS0lBWVgaAGaAn3ToiVVVVGD9+PLp27YrMzEx0794dM2fOxNmzZ/X6SOPFWxTGTadMh3PEycXc+D/MDdNgBuhPt47I1q1b4fP58NJLL2HTpk14+umn8eKLL+Lhhx/W6yONF29RmA1XJ7Q6hlByMTfA3DAZZoD+klojMnfuXLzwwgv4/vvvVb3eUvO8gWpqYlsZMVFzxZQwK1aswIUXXoh27dph//79jlxd0ejzUGtuAMa3OSbMDVNau3YtysvL0apVKxw5csSRGRAL09aIeL1etG7dWvH79fX1qKurC3pYUqz30vM2O9Pp378/3G43Dh48iH379hndHEeKlhuATbKDuWFKffv2RVpaGo4dO4Zdu3YZ3RxbSlpHZOfOnXj22Wdxzz33KL5m9uzZyMvLa3p07tw5Wc0zD95mZyotWrRgwaqB1OQGwOxgbugnLS0N/fr1A8AM0IvmjsisWbPgcrkiPlavXh30nn379uGKK67AmDFjcOeddyoee8aMGfB6vU2P6upq7T+RHdhsdUKr4xxx/PTMDYDZAYC5oSNmgL407747adIkjB07NuJriouLm/73vn37cMkll2DYsGGYP39+xPelp6cjPT1da5OIdFVeXo7XX3+dIRQHPXMDYHaQvtgR0ZfmjkhBQQEKCgpUvXbv3r245JJLUF5ejldffRXu0FvUSF9c4CghGELxY25YCHMjTOgqyyxYTSzdzvB9+/ZhxIgR6Ny5M+bNm4fa2locOHAABw4c0OsjKRAXOEqYAQMGwO1248CBA9i/f7/RzbE15obBmBuy+vbti9TUVBw7dgxVVVVGN8d2dOuIfPrpp9ixYweWLVuGwsJCdOjQoelBOuMCRwnVokULnHfeeQA4KqI35oaBmBuK0tPTWbCqI906IrfddhuEELIPUiHSTpzRdunkAkcJx+mZ5GBuJECs2cHciIgZoB9OvpqJFBJz5yoPj6oZOo13yWgKwxAiU0tEdjA3ImIG6EiYmNfrFQCE1+s1uin6W7BACLdbCCD84fEIUV3tf4S+Rvqe3PE8nubXLFiQ/J/JRr788ksBQHTs2NHopiSdFc9DK7Y5ZonMDuaGolWrVgkAonXr1sLn8xndHNPTcg5qvmuGdBA6NxtKGh4VQnnoNLS6ffx4YNSo2JaMpjBSweq+fftw4MABtG/f3ugmESU+O5gbivr164fU1FQcPXoUu3fvDrrdnOLDqRkzkJubDSQNj2odOuUCRwmTlZWF0tJSAByaJRPRIzuYG7LS09PRt29fAMyARGNHxAzkQkISuG8E95QwFOeIyXSYHUnFDNAHOyJmEBoSbjfwyCPy+0ZwTwnDMITIdJgdScUM0AdrRMxi/Hjg2DFg2jT/UOvs2UDXrv4h0lDSFQ4lFUOITInZkTRcYVUfLiHMe4N+XV0d8vLy4PV6kZuba3Rz9FVT47+lLnC+1+PxX7kwOEzh5MmTyM3NhRACBw4cQLt27YxuUlJY8Ty0YptjxuxImjNnziAnJwcNDQ3YvXs3unTpYnSTTEvLOcipGbPgYkKml52djV69egHgqAiZCLMjaTIyMliwqgN2RMyCiwlZAqdnyHSYHUnFDEg8dkTMglXtlsAQItNhdiQVMyDxWKxqJlxMyPQYQmRKzI6kYcFq4rEjYjasaje1gQMHwuVyoaamBocOHULbtm2NbhKRH7MjKcrKypCSkoLa2lrU1NSgc+fORjfJ8jg1Q6RBTk4OevbsCYCjIkROlJGRgT59+gBgBiQKOyJEGnF6hsjZmAGJxY4IkUYMISJnYwYkFjsiRBoxhIicLbRgleLDjgiRRgMHDgQAVFdXo7a21uDWEFGylZWVwePx4NChQ9i7d6/RzbE8dkSINMrNzWXBKpGDZWZmonfv3gCYAYnAjghRDDg9Q+RszIDEYUeEKAYMISJnYwYkDjsiRDFgCBE5GwtWE4cdEaIYSAWre/bsweHDhw1uDRElW//+/eF2u3Hw4EHs27fP6OZYGjsiRDHIy8tDSUkJAI6KEDlRixYtWLCaIOyIEMWI0zNEzsYMSAx2RIhixBAicjZmQGKwI0IUIymE1q5da3BLiMgI7IgkBjsiRDEaNGgQAKCqqgpHjhwxuDVElGwDBgyA2+3GgQMHWLAaB3ZEiGKUl5eHHj16AOCoCJETtWjRAueddx4AjorEgx0RojhwaJbI2ZgB8WNHhCgODCEiZ2MGxI8dEaI4MISInI0ZED92RIjiIBWs7tq1C0ePHjW4NUSUbFLB6v79+7F//36jm2NJ7IgQxaFly5bo1q0bABasEjlRVlYWSktLAXBUJFbsiBDFiUOzRM7GDIiPrh2R//iP/0CXLl2QkZGBDh064JZbbuG91mQ7DKHEYm6Q1TAD4qNrR+SSSy7B4sWLsW3bNrz33nvYuXMnrrvuOj0/kijpGEKJxdwgq2EGxMclhBDJ+rD//u//xjXXXIP6+nqkpqZGfX1dXR3y8vLg9XqRm5ubhBYSaXf06FHk5+c3/e9WrVoZ3KLEMvo81JobgPFtJmc5efIkcnNzIYTA/v370b59e6ObZDgt52DSakSOHj2Kt956CxdeeKFimNTX16Ouri7oQWR2rVu3RteuXQGwYDXR1OQGwOwgY2VnZ7NgNQ66d0SmTZuGrKws5OfnY8+ePfjb3/6m+NrZs2cjLy+v6dG5c2e9m0eUEByaTSwtuQEwO8h43AQzdpo7IrNmzYLL5Yr4WL16ddPrH3zwQaxbtw6ffvopPB4Pbr31VijNBs2YMQNer7fpUV1dHftPRpRE7IhEpmduAMwOMh4zIHaaa0QOHz6Mw4cPR3xNcXExMjIywp6vqalB586d8dVXX2HYsGFRP4vzvGQVS5cuxciRI9G9e3fs2LHD6OYkVCLOw2TmRqLaTKTFF198gYsvvhiFhYXsCEPbOZii9eAFBQUoKCiIqWFSn6e+vj6m9xOZlbTC6s6dO3H8+HG0bNnS2AaZDHOD7G7gwIFwuVyoqanBoUOH0LZtW6ObZBm61YisXLkSzz33HNavX4/du3ejsrISv/jFL9C9e3fVVzVEVpGfn4/i4mIAnCOOB3ODrCo7Oxu9evUCwOkZrXTriGRmZuL999/HZZddhl69euGOO+5A3759sXz5cqSnp+v1sUSG4Rxx/JgbZGXMgNhonppRq1+/fli2bJlehycynfLycrz33nsMoTgwN8jKysvL8dZbbzEDNOJeM0QJwqshImdjBsSGHRGiBJFCaMeOHfB6vQa3hoiSTSpYra6uRm1trdHNsQx2RIgSJD8/H0VFRQBYsErkRDk5OejZsycAjopowY4IUQJxaJbI2aRb+ZkB6rEjQpRA7IgQORszQDt2RIgSiCFE5GzMAO3YESFKICmEtm/fzoJVIgcaOHAgAGDPnj1RtzUgP3ZEiBKooKAAXbp0AQCsW7fO4NYQUbLl5eWhpKQEAEdF1GJHhCjBODRL5GzMAG3YESFKMCmEeAsvkTOxI6INOyJECcYQInI2ZoA27IgQJZgUQt999x1OnDhhcGuIKNmktUR2796NI0eOGNwa82NHhCjB2rRpg86dO0MIwYJVIgfKy8tDjx49AHBURA12RIh0wNUViZyN0zPqsSNCpAOGEJGzMQPUY0eESAcMISJnYwaox44IkQ6kENq2bRsLVokcSJqeraqqYsFqFOyIEOmgXbt26NSpE4QQWL9+vdHNIaIka9myJbp37w6AawpFw44IkU44NEvkbMwAddgRIdIJQ4jI2ZgB6rAjQqQThhCRszED1GFHhEgnUght3boVJ0+eNLg1RJRsUsHqrl27cPToUYNbY17siBDppH379ujYsSMLVokcqlWrVujWrRsAFqxGwo4IkY44NEvkbMyA6NgRIdIRQ4jI2ZgB0bEjQqQjhhCRs0kZwKkZZeyIEOkosGD11KlTBreGiJJNKljduXMnjh8/bmxjTIodESIddejQAR06dIDP52PBKpEDtW7dGl27dgXAUREl7IgQ6YzTM0TOxgyIjB0RIp0xhIicTZqeYQbIY0eESGfsiBA5GzMgMnZEiHQmhdCWLVtYsErkQFIG7NixA16v1+DWmA87IkQ669ixI9q3bw+fz4cNGzYY3RwiSrL8/HwUFRUBYMGqHHZEiJKAQ7NEzsYMUMaOCFESMISInI0ZoCwpHZH6+noMGDAALpeLaymQIzGEtGNukJ0wA5QlpSPy0EMPoWPHjsn4KCJTkkJo8+bN+OGHHwxujTUwN8hOpAzYvn07C1ZD6N4R+ec//4lPP/0U8+bN0/ujiEyrY8eOaNeuHQtWVWJukN0UFBSgS5cuAIB169YZ3BpzSdHz4AcPHsSECRPw4YcfokWLFlFfX19fj/r6+qavpV5jXV2dbm0kSpaysjIsXboUX375Jfr06WN0c1STzj8hRFI+T2tuAMwOsoaysjLs2bMH//u//9u0yJldacoNoROfzyeuuOIK8bvf/U4IIcSuXbsEALFu3TrF98ycOVMA4IMPPkz4qK6u1isu4soNZgcffJj3oSY3XEJou8yZNWsWHnvssYivWbVqFb766issWrQIn3/+OTweD6qqqtC1a1esW7cOAwYMkH1f6FWNz+fD0aNHkZ+fD5fLpfh5dXV16Ny5M6qrq5Gbm6vlxzElO/08/FnMScvPIoTAiRMn0LFjR7jdsc3m6pkbALMD4M9iZnb6edT+LFpyQ3NH5PDhwzh8+HDE1xQXF2Ps2LH4+9//HhQCjY2N8Hg8uOmmm/D6669r+diI6urqkJeXB6/Xa/l/ZMBePw9/FnNK9s9ixtwA+G9qVnb6WQB7/Tx6/Cyaa0QKCgpQUFAQ9XX/9V//hccff7zp63379mHUqFFYtGgRhgwZovVjicjCmBtEpES3YlWpOliSnZ0NAOjevTsKCwv1+lgisjDmBpHz2GJl1fT0dMycORPp6elGNyUh7PTz8GcxJzv9LPGw0++BP4t52enn0eNn0VwjQkRERJQothgRISIiImtiR4SIiIgMw44IERERGYYdESIiIjIMOyJERERkGFt3ROrr6zFgwAC4XC6sX7/e6OZoVlVVhfHjx6Nr167IzMxE9+7dMXPmTJw9e9bopqny/PPPo2vXrsjIyEB5eTm++OILo5sUk9mzZ2Pw4MHIyclB27Ztcc0112Dbtm1GNyshZs+eDZfLhfvuu8/oppiG1XMDYHaYAXNDPVt3RB566CF07NjR6GbEbOvWrfD5fHjppZewadMmPP3003jxxRfx8MMPG920qBYtWoT77rsPjzzyCNatW4cf//jHuPLKK7Fnzx6jm6bZ8uXLMXHiRHz99ddYunQpGhoaMHLkSJw6dcropsVl1apVmD9/PsrKyoxuiqlYPTcAZocZMDc0iGenTDP7+OOPRWlpqdi0aZMAou/eaRVz5swRXbt2NboZUV1wwQXinnvuCXqutLRUTJ8+3aAWJc6hQ4cEALF8+XKjmxKzEydOiJKSErF06VIxfPhwMXnyZKObZAp2zQ0hmB1GY24os+WIyMGDBzFhwgS88cYbaNGihdHNSSiv14vWrVsb3YyIzp49izVr1mDkyJFBz48cORJfffWVQa1KHK/XCwCm/3eIZOLEifjZz36Gn/zkJ0Y3xTTsnBsAs8NozA1luu01YxQhBG677Tbcc889OP/881FVVWV0kxJm586dePbZZ/Hkk08a3ZSIDh8+jMbGRrRr1y7o+Xbt2uHAgQMGtSoxhBCYMmUKLrroIvTt29fo5sRk4cKFWLt2LVatWmV0U0zDzrkBMDuMxtyIzDIjIrNmzYLL5Yr4WL16NZ599lnU1dVhxowZRjdZkdqfJdC+fftwxRVXYMyYMbjzzjsNark2gVu5A/6TMfQ5q5k0aRI2btyId955x+imxKS6uhqTJ0/Gm2++iYyMDKObozs75QbA7LAq5kZkltlr5vDhwzh8+HDE1xQXF2Ps2LH4+9//HvRH29jYCI/Hg5tuugmvv/663k2NSu3PIv2D79u3D5dccgmGDBmC1157DW63ufuPZ8+eRYsWLfDuu+/i5z//edPzkydPxvr167F8+XIDWxe7X/3qV/jwww/x+eefo2vXrkY3JyYffvghfv7zn8Pj8TQ919jYCJfLBbfbjfr6+qDvWZ2dcgNgdlgxO5gb0VmmI6LWnj17UFdX1/T1vn37MGrUKPz1r3/FkCFDLLeV+N69e3HJJZegvLwcb775pmX+T2LIkCEoLy/H888/3/Rc7969cfXVV2P27NkGtkw7IQR+9atf4YMPPsC//vUvlJSUGN2kmJ04cQK7d+8Oeu72229HaWkppk2bZtlh43jZLTcAZofRmBvq2a5GpEuXLkFfZ2dnAwC6d+9uuTDZt28fRowYgS5dumDevHmora1t+l779u0NbFl0U6ZMwS233ILzzz8fw4YNw/z587Fnzx7cc889RjdNs4kTJ+Ltt9/G3/72N+Tk5DTNVefl5SEzM9Pg1mmTk5MTFhpZWVnIz893bCcEsFduAMwOM2BuqGe7joidfPrpp9ixYwd27NgRFoZmH8i64YYbcOTIEfz2t7/F/v370bdvX3z88ccoKioyummavfDCCwCAESNGBD3/6quv4rbbbkt+g4iiYHYYj7mhnu2mZoiIiMg6zF25RERERLbGjggREREZhh0RIiIiMgw7IkRERGQYdkSIiIjIMOyIEBERkWHYESEiIiLDsCNCREREhmFHhIiIiAzDjggREREZhh0RIiIiMsz/B4yxbcK6LeT/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize optimization\n",
    "\n",
    "fig = plt.figure()\n",
    "ax0 = fig.add_subplot(121)\n",
    "ax1 = fig.add_subplot(122)\n",
    "x0_old = np.arange(-4, 5)\n",
    "x1_old = (- b_old - x0_old * w_old[0]) / w_old[1]\n",
    "ax0.plot(x0_old, x1_old, color = \"black\")\n",
    "x0_new = np.arange(-4, 5)\n",
    "x1_new = (- b_new - x0_new * w_new[0]) / w_new[1]\n",
    "ax1.plot(x0_new, x1_new, color = \"black\")\n",
    "for x, y in zip(x_data, y_classes):\n",
    "    if y == 1:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "    else:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "\n",
    "ax0.set_title(\"before optimization\")\n",
    "ax1.set_title(\"after optimization\")\n",
    "ax0.set_ylim(-4, 4)\n",
    "ax1.set_ylim(-4, 4)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further (optional)\n",
    "> Try to replace the Hinge loss by the cross-entropy loss in your implementation\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.2 - Implement linear classification with JAX\n",
    "\n",
    "> 1. Initialize weights and bias.\n",
    "> 2. Define the loss function using the jax.nn.softmax_cross_entropy_with_logits function.\n",
    "> 3. Implement gradient descent to optimize the weights and bias.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use w_old and b_old defined before for initial weights and bias\n",
    "#define loss fanction\n",
    "def loss_function(x, y, w, b):\n",
    "    y_pred = jax.numpy.dot(x, w) + b\n",
    "    return jax.numpy.maximum(0.0, 1.0 - y * y_pred)\n",
    "\n",
    "grad_loss_function = jax.jit((jax.grad(loss_function, argnums=[2, 3])))\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [],
   "source": [
    "#optimize by jax\n",
    "def gradient_descent_by_jax(X, Y, w, b, epoch=1000, lr=1):\n",
    "    for i in range(epoch):\n",
    "        grad_w = np.array([0.0, 0.0])\n",
    "        grad_b = 0.0\n",
    "        for x, y in zip(X, Y):\n",
    "            d_w, d_b = grad_loss_function(x, y, w,  b)\n",
    "            grad_w += d_w\n",
    "            grad_b += d_b\n",
    "        grad_w /= len(X)\n",
    "        grad_b /= len(X)\n",
    "        w -= lr * grad_w\n",
    "        b -= lr * grad_b\n",
    "    return w, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [],
   "source": [
    "w_jax, b_jax =gradient_descent_by_jax(x_data, y_classes, w_old, b_old)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-4.0, 4.0)"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABoF0lEQVR4nO3deXwU9f0/8Nfu5iQXgXCFQLhB7jsEUPACD1rtVxGsVVRErdiqVAX0Z0FriwXUWqu2CPWoB2A9+q1VkUpEjiD3IcopR0K4wrHhDCT7+f2x30n2mNmd2d3ZOfb1fDz2YbPZnf0kZF59z2fe8xmHEEKAiIiIyABOowdAREREiYuFCBERERmGhQgREREZhoUIERERGYaFCBERERmGhQgREREZhoUIERERGYaFCBERERmGhQgREREZhoVIGNOnT4fD4UBlZWVMt/vyyy+jQ4cOSElJgcPhwMmTJ2O6fbN49dVX8eabbwY9v3fvXjgcDtnvRUqPbfr6/vvvMX36dOzduzfoe3feeSfatGmjy+dSYtuwYQOGDRuGnJwcOBwO/OlPf8Jnn32G6dOnGz20qDEfCAAgKKRp06YJAOLo0aMx2+aGDRsEAHHPPfeIZcuWidLSUlFTUxOz7ZtJt27dxLBhw4KeP3/+vCgtLRVHjhyJ2WfpsU1fH3zwgQAgSkpKgr63a9cusX79el0+lxJb7969RceOHcVnn30mSktLxcGDB8XEiROFHeKb+UBCCJFkaBWUoLZu3QoAmDBhAgYOHBiTbZ49exYNGjSIybbiITU1FYMGDTL9NtVq3769IZ9L9vfdd99hwoQJuPbaa3X/LLPkCPMhwRhdCZmdNCOyfv168bOf/UxkZWWJ7Oxscdttt8lW1vPnzxeDBg0SDRo0EBkZGWLEiBF+lfCwYcMEAL/HuHHj6r4/b9480bNnT5Gamipyc3PFjTfeKL7//nu/zxg3bpzIyMgQmzdvFldffbXIzMwUgwYNEkIIUV1dLX73u9+Jzp07i5SUFJGXlyfuvPNO1UcB//rXv8SgQYNEenq6yMzMFFdddZVYuXJlRL+TwsLCoJ+1sLBQCCHEnj17BADxxhtvBG1306ZN4uabbxbZ2dkiNzdXPPLII+LixYti27ZtYuTIkSIzM1MUFhaKP/7xj37jkttm4Of7Pvbs2SOEEGLNmjVizJgxorCwUKSlpYnCwkIxduxYsXfv3rrtvPHGG7LbkD5r3LhxdT+b5Ny5c2LKlCmiTZs2Ijk5WeTn54sHHnhAnDhxwu91hYWF4vrrrxeff/656NOnj0hLSxOdO3cW8+bNU/VvRtazc+dOceedd4oOHTqI9PR0kZ+fL0aNGiU2b95c9xqlv7lx48aF/Hv2eDzilVdeEb169RJpaWmiYcOG4qabbhK7d+/2G8OwYcNEt27dxNKlS0VxcbFIT08XY8aMCTlu5gPzQQ8sRMKQ/vgLCwvFY489JhYtWiReeOEFkZGRIfr06SMuXLhQ99rf//73wuFwiLvvvlt8+umn4qOPPhLFxcUiIyNDbN26VQghxNatW8X/+3//r+6PtLS0VOzatUsIIcQf/vAHAUDceuut4j//+Y94++23Rbt27UROTo7YsWNH3eeMGzdOJCcnizZt2ogZM2aIr776SixatEjU1taKa665RmRkZIinn35aLF68WMydO1e0bNlSdO3aVZw9ezbkz/ruu+8KAGLEiBHik08+EQsWLBD9+vUTKSkpYtmyZZp/J+vXrxft2rUTffr0EaWlpaK0tLSuKAsVNJ07dxa/+93vxOLFi8Xjjz8uAIgHH3xQdOnSRfz5z38WixcvFnfddZcAID788MO698ttU/pc6bFkyRLRsmVL0bx5c+F2u4UQ3inV3/72t+Ljjz8WS5cuFfPnzxfDhg0TTZo0qTsld+TIkbp/n1deeaVue1KwBgaNx+MRI0eOFElJSeKpp54SX375pZg9e3bd7+j8+fN1ry0sLBQFBQWia9eu4u233xaLFi0So0ePFgDE0qVLQ/6bkTUtXbpU/OY3vxH//Oc/xdKlS8XHH38sbrzxRpGeni62bdsmhPD+zZWWlgoA4uabb677m9u1a5e4+eabBQC/v23pb2rChAkiOTlZ/OY3vxFffPGFeO+990SXLl1Es2bNxKFDh+rGMGzYMNGoUSPRqlUr8fLLL4uSkpKQf2/MB+aDXliIhCH98T/yyCN+z0s75TvvvCOEEGL//v0iKSlJ/OpXv/J73alTp0Tz5s3FLbfcUvecVD2vWbOm7rkTJ06I9PR0cd111/m9f//+/SI1NVX8/Oc/r3tOOiL6+9//7vfa999/P2jnE8Jb0QMQr776quLPWVtbK/Lz80WPHj1EbW2t3/ibNm0qBg8erPl3IoTyOeBQQfP888/7vbZ3794CgPjoo4/qnrt48aJo0qSJ+J//+Z+Q2/RVU1MjbrjhBpGZmSnWrVun+LuoqakRp0+fFhkZGeKll16qez7UOeDAoPniiy8EADFz5ky/1y1YsEAAEHPmzKl7TjrS2rdvX91z586dE40aNRL33Xef4jjJPmpqasSFCxdEx44dg/YrAGLixIl+zyn1iEiFS+A+VFZWJtLT08Xjjz9e95w0O/vVV1+FHR/zwf91zIfY4lUzKt12221+X99yyy1ISkpCSUkJAGDRokWoqanBHXfcgZqamrpHWloahg0bhq+//jrk9ktLS3Hu3Dnceeedfs+3atUKV1xxBb766qug99x0001+X3/66ado2LAhfvKTn/iNoXfv3mjevHnIMWzfvh0VFRW4/fbb4XTW/1lkZmbipptuwqpVq3D27FlNv5NIjRo1yu/rSy65BA6Hw+8ceVJSEjp06IB9+/ap3u6DDz6I//znP/jggw/Qt2/fuudPnz6NyZMno0OHDkhKSkJSUhIyMzNx5swZ/PDDDxH9DEuWLAGAoH/P0aNHIyMjI+jfs3fv3mjdunXd12lpaejUqZOmn4+so6amBn/4wx/QtWtXpKSkICkpCSkpKdi5c2fEf3OANwMcDgd+8Ytf+GVA8+bN0atXr6AMyM3NxRVXXBF2u8wH5oOe2KyqUvPmzf2+TkpKQuPGjXHs2DEAwOHDhwEAAwYMkH2/784rR9pOixYtgr6Xn5+PxYsX+z3XoEEDZGdn+z13+PBhnDx5EikpKbKfEeoS5HCf7/F4cOLECb9GtnC/k0g1atTI7+uUlBQ0aNAAaWlpQc9XVVWp2uazzz6Lv/71r5g3bx6uueYav+/9/Oc/x1dffYWnnnoKAwYMQHZ2NhwOB6677jqcO3cuop/h2LFjSEpKQpMmTfyedzgcaN68edDvqHHjxkHbSE1NjfjzydwmTZqEV155BZMnT8awYcOQm5sLp9OJe+65J6p/88OHD0MIgWbNmsl+v127dn5fy+3vcpgPzAc9sRBR6dChQ2jZsmXd1zU1NTh27FjdH0heXh4A4J///CcKCws1b1/azsGDB4O+V1FRUbd9icPhCHpdXl4eGjdujC+++EL2M7KysiL+fKfTidzcXL/nw/1OzOLNN9/EU089henTp+Puu+/2+57b7cann36KadOmYcqUKXXPV1dX4/jx4xF/ZuPGjVFTU4OjR4/6hY0QAocOHVIsWCkxvPPOO7jjjjvwhz/8we/5yspKNGzYMOLt5uXlweFwYNmyZUhNTQ36fuBzcjkih/nAfNATT82o9O677/p9vXDhQtTU1GD48OEAgJEjRyIpKQm7d+9G//79ZR+hFBcXIz09He+8847f8+Xl5ViyZAmuvPLKsGMcNWoUjh07htraWtnP79y5s+J7O3fujJYtW+K9996DEKLu+TNnzuDDDz9EcXFx0GV94X4ngPFV+xdffIEJEybg7rvvxrRp04K+73A4IIQICui5c+eitrbW7znpNWp+HunfK/Df88MPP8SZM2dU/XuSfTkcjqC/uf/85z84cOCAqvcr/S2OGjUKQggcOHBANgN69OgR0XiZD8wHPXFGRKWPPvoISUlJuPrqq7F161Y89dRT6NWrF2655RYAQJs2bfDMM8/gySefxI8//ohrrrkGubm5OHz4MFavXo2MjAw8/fTTittv2LAhnnrqKTzxxBO44447cOutt+LYsWN4+umnkZaWJruTBBo7dizeffddXHfddXjooYcwcOBAJCcno7y8HCUlJbjhhhvws5/9TPa9TqcTM2fOxG233YZRo0bhvvvuQ3V1NWbNmoWTJ0/iueee0/w7AYAePXpg/vz5WLBgAdq1a4e0tLSIw1CrPXv2YPTo0WjXrh3uuusurFq1yu/7ffr0QXZ2Ni677DLMmjULeXl5aNOmDZYuXYp58+YFHZl2794dADBnzhxkZWUhLS0Nbdu2lT3Cu/rqqzFy5EhMnjwZVVVVGDJkCDZv3oxp06ahT58+uP3223X7ucn8Ro0ahTfffBNdunRBz549sW7dOsyaNQsFBQWq3i/tQ3/84x9x7bXXwuVyoWfPnhgyZAjuvfde3HXXXVi7di0uu+wyZGRk4ODBg1i+fDl69OiBX/7yl5rHy3xgPujKuD5Za5A6tdetWyd+8pOfiMzMTJGVlSVuvfVWcfjw4aDXf/LJJ+Lyyy8X2dnZIjU1VRQWFoqbb75Z/Pe//617jdxVM5K5c+eKnj17ipSUFJGTkyNuuOGGukt/JdI6InIuXrwoZs+eXbeGQGZmpujSpYu47777xM6dO8P+vJ988okoKioSaWlpIiMjQ1x55ZVixYoVEf9O9u7dK0aMGCGysrJUrxMQuIqt0s8rrYMgCdxmSUmJqnUCysvLxU033SRyc3NFVlaWuOaaa8R3330nCgsL/dZ4EUKIP/3pT6Jt27bC5XKpWidg8uTJorCwUCQnJ4sWLVqIX/7yl4rrBMj9fHJXFJD1nThxQowfP140bdpUNGjQQAwdOlQsW7ZM9t8cMlfNVFdXi3vuuUc0adJEOBwOv79nIYT4+9//LoqKikRGRoZIT08X7du3F3fccYdYu3Zt3WsC9x81mA/MBz04hPCZZyNSYfr06Xj66adx9OjRoN4VIkpszAfSij0iREREZBgWIkRERGQYnpohIiIiw8RtRmTGjBlwOBx4+OGH4/WRRGQDzA4ie4tLIbJmzRrMmTMHPXv2jMfHEZFNMDuI7E/3QuT06dO47bbb8PrrrwetvEdEpITZQZQYdF/QbOLEibj++utx1VVX4dlnnw352urqalRXV9d97fF4cPz4cTRu3Fj1UsREFFtCCJw6dQr5+flh75kUS8wOIuvSkhu6FiLz58/H+vXrsWbNGlWvnzFjRsjVR4nIOGVlZapX/owWs4PIHtTkhm6FSFlZGR566CF8+eWXQXdFVDJ16lRMmjSp7mu3243WrVujrKws6E6zZC5XXXUV1qxZg9dff91vCWeyvqqqKrRq1SrkTRNjidlhbidPnqy7seeOHTsU7/RLiU1LbuhWiKxbtw5HjhxBv3796p6rra3FN998g7/85S+orq6Gy+Xye09qaqrsHSOzs7MZJiZXVFSENWvW4IcffuC/lU3F6xQHs8PcsrOz0atXL2zatAkbN27E6NGjjR4SmZia3NCtELnyyiuxZcsWv+fuuusudOnSBZMnTw4KErI26f801q1bZ/BIyOqYHeZ32WWXYdOmTVi2bBkLEYqaboVIVlZW3d0IJRkZGWjcuHHQ82R9UiGyfv16eDyeuDY1kr0wO8zv0ksvxcsvv4xvvvnG6KGQDfD/LSgmLrnkEqSnp+PUqVPYtWuX0cMhIh1deumlAIDNmzfj5MmTxg6GLE/3y3d9ff311/H8OIqjpKQk9OrVC6tWrcK6devQqVMno4dENsLsMJfmzZujY8eO2LlzJ1asWIHrr7/e6CGRhXFGhGKGfSJEieOyyy4DACxbtszgkZDVsRChmGEhQpQ4pNMz7BOhaLEQoZgJbFglIvuSZkTWrl2Ls2fPGjwasjIWIhQzXbt2RVpaGqqqqrB7926jh0NEOmrTpg1atmyJixcv4ttvvzV6OGRhLEQoZqSGVYCnZ4jszuFwsE+EYoKFCMUU+0SIEgf7RCgWWIhQTLEQIUoc0oxIaWkpLl68aPBoyKpYiFBM+TasCiEMHg0R6emSSy5Bo0aNcPbsWaxfv97o4ZBFsRChmOratStSU1PhdrvZsEpkc06nk6dnKGosRCimkpOT2bBKlECkQoQNqxQpFiIUc+wTIUocUp/I8uXLuX4QRYSFCMUcCxGixNGnTx9kZGTgxIkT2Lp1q9HDIQtiIUIxx4ZVosSRlJSEwYMHA2CfCEWGhQjFXLdu3ZCamoqTJ0/ixx9/NHo4RKQzLmxG0WAhQjGXnJyMnj17AuDpGaJE4HvlDGdBSSsWIqQL9okQJY6BAwciJSUFBw8e5CwoacZChHTBQoQocaSnp2PAgAEA2CdC2rEQIV2wYZUosbBPhCLFQoR00a1bN6SkpODEiRPYs2eP0cMhIp1xhVWKFAsR0kVKSgobVokSyODBg+F0OrF7925UVFQYPRyyEBYipBv2iRAljpycnLrbO/D0DGnBQoR0w0KEKLFIfSI8PUNasBAh3fgWImxYJbI/3gCPIsFChHTTvXt3JCcn48SJE9i7d6/RwyEinUmFyJYtW3D8+HGDR0NWwUKEdJOSkoIePXoA4OkZokTQtGlTdO7cGQCwYsUKg0dDVsFChHTFPhGixMI+EdKKhQjpioUIUWJhnwhpxUKEdMWGVaLEIs2IrFu3DmfOnDF4NGQFLERIVz169EBycjKOHz+Offv2GT0cItJZYWEhWrdujZqaGqxatcro4ZAFsBAhXaWmpqJ79+4AeHqGKFFwuXfSgoUI6Y59IkSJhTfAIy1YiJDuWIgQJRZpRqS0tBQXLlwweDRkdixESHdsWCVKLF26dEFeXh7Onz/PAxAKi4UI6a5Hjx5ISkrCsWPHsH//fqOHQ0Q6czgc7BMh1ViIkO7S0tLYsEqUYLiwGamlayHy2muvoWfPnsjOzkZ2djaKi4vx+eef6/mRZFLsEyG1mBv2IM2IrFixArW1tQaPhsxM10KkoKAAzz33HNauXYu1a9fiiiuuwA033ICtW7fq+bFkQixESC3mhj306tULWVlZcLvd2LJli9HDIRPTtRD5yU9+guuuuw6dOnVCp06d8Pvf/x6ZmZlc5CYBsWGV1GJu2ENSUhIGDx4MgJfxUmhx6xGpra3F/PnzcebMGRQXF8u+prq6GlVVVX4PsoeePXsiKSkJlZWVKCsrM3o4ZBFqcgNgdpgV+0RIDd0LkS1btiAzMxOpqam4//778fHHH6Nr166yr50xYwZycnLqHq1atdJ7eBQnaWlp6NatGwCenqHwtOQGwOwwK98b4HEmlJToXoh07twZGzduxKpVq/DLX/4S48aNw/fffy/72qlTp8Ltdtc9eORsL+wTIbW05AbA7DCrAQMGIDU1FYcPH8bOnTuNHg6ZlO6FSEpKCjp06ID+/ftjxowZ6NWrF1566SXZ16amptZ1yksPsg8WIqSWltwAmB1mlZaWhoEDBwJgnwgpi/s6IkIIVFdXx/tjyQTYsEqRYm5YF/tEKJwkPTf+xBNP4Nprr0WrVq1w6tQpzJ8/H19//TW++OILPT+WTKpnz55wuVw4evQoDhw4gIKCAqOHRCbE3LAX3z4RIjm6FiKHDx/G7bffjoMHDyInJwc9e/bEF198gauvvlrPjyWTSk9PR7du3bB582asW7eOhQjJYm7Yy+DBg+F0OrFnzx6Ul5dzv6cguhYi8+bN03PzZEH9+vWrK0RuuOEGo4dDJsTcsJesrCz06dMH69atw7Jly3DrrbcaPSQyGd5rhuKKDatEiYd9IhQKCxGKKzasEiUeFiIUCgsRiqtevXrB5XLh8OHDqKioMHo4RBQHQ4cOBQB8//33qKysNHg0ZDYsRCiu0tPT61bI5OkZosSQl5dXt98vX77c4NGQ2bAQobjr27cvABYiRImEl/GSEhYiFHdsWCVKPOwTISUsRCjuWIgQJR5pRmTDhg04deqUwaMhM2EhQnHXu3dvOJ1OHDp0iA2rRAmiVatWaNOmDWpra1FaWmr0cMhEWIhQ3DVo0ACXXHIJAM6KECUS9omQHBYiZAieniFKPOwTITksRMgQLESIEo80I/Ltt9/ybspUh4UIGYKFCFHi6dSpE5o2bYrq6mqsWbPG6OGQSbAQIUNIDasHDx7EwYMHjR4OEcWBw+FgnwgFYSFChsjIyECXLl0AcFaEKJGwT4QCsRAhw/D0DFHikWZEVqxYgdraWoNHQ2bAQoQMw0KEKPH07NkT2dnZOHXqFDZt2mT0cMgEWIiQYViIECUel8uFIUOGAODpGfJiIUKG6d27NxwOByoqKnDo0CGjh0NEcSL1ibBhlQAWImSgzMxMNqwSJSDfQkQIYfBoyGgsRMhQiX56prwcKCnx/pcoUfTv3x9paWk4evQotm/fbvRwLMlO2cFChAyVyIXIvHlAYSFwxRXe/86bZ/SIiOIjJSUFgwYNAsA+kUjYLTtYiJChErUQKS8H7r0X8Hi8X3s8wH332ePohkgNLmwWGTtmBwsRMlSfPn3gcDhw4MABHD582OjhxM1LL9UHiaS2Fti1y5jxEMUbFzaLjB2zg4UIGSozMxOdO3cGAKxfv97g0cRHeTnw/PPBz7tcQIcO8R8PkREGDRoEl8uF/fv3Y9++fUYPxxLsmh0sRMhwiXZ6ZudOQO5CgUceAQoK4j8eIiNkZmbW7fs8PaOOXbODhQgZLtEKkY4dAWfAnud0Ag89ZMx4iIzCPhFt7JodLETIcIlWiBQUAHPmeKdTAW+QPPectY9oiCLBPhFtpOyQihGn0/u11bODhQgZTmpYLSsrw9GjR2VfY6dr5gFg/Hhv8eF0ehvPpkyx/iV4RFpJS71v27YNR44c0eUz7JYddsRChAyXlZWFTp06AZCfFbHbNfOANxQnT7bXJXhEWjVu3Bjdu3cHACxfvjzm27dbdtjx0l2AhQiZRN++fQEEFyJ23fF27rTfJXhEkZD6RGJ9esaO2WHX3GAhQqag1CcSyx3PTFO0ck1nVr8EjygSet0Az47ZYdfcYCFCpqBUiMRqxzPbFG1gw6rLBfztb9ZvOiPSSpoR2bhxI6qqqmK2XTtmh11zg4UImUKfPn0AAPv370dlZWXd87HY8eI1Rav1qGn8eGDvXu979u71fk2UaFq2bIl27drB4/Fg5cqVMduuXbPDjrnBQoRMIScnBx07dgQQPCsS7Y4Xj/OqkR41FRQAw4db/4iGKBp69YnYNTvslhssRMg0Qq0nEs2Op/d5VTs2xRHFk159IgCzwwp0LURmzJiBAQMGICsrC02bNsWNN96I7du36/mRZGF6LWwWzRStmilTrUdNZml8MyvmRuKRZkRWr16N8+fPGzyaembKDjvnhq6FyNKlSzFx4kSsWrUKixcvRk1NDUaMGIEzZ87o+bFkUXqusBrJFK3aKVMtR01manwzK+ZG4unQoQOaN2+OCxcuYPXq1UYPx48ZssP2uSHi6MiRIwKAWLp0qarXu91uAUC43W6dR0ZmcPLkSQFAABCVlZWGjqWsTAinUwjvLaa8D5fL+7ycuXO935deN3du9Ns0C6P3Q625IYTxYybtbrnlFgFA/O53vzN6KFGJdXYkQm7EtUfE7XYDABo1aiT7/erqalRVVfk9KHHk5OSgw/8dChh93xmtp1vUHDXZdTEivYXLDYDZYQd2uQFerLMjEXIjboWIEAKTJk3C0KFD65b0DTRjxgzk5OTUPVq1ahWv4ZFJmOEGeOXlwNGj2pvUwjXF2XUxIj2pyQ2A2WEHUsPqihUrUFNTY/BoIqNHdiRCbsStEHnwwQexefNmvP/++4qvmTp1Ktxud92jrKwsXsMjkzC6EJHOxY4Z450ElQIg0jUIfJvL7LoYkZ7U5AbA7LCD7t27o2HDhjhz5gw2bNhg9HA0i1V2JGRu6H6iSAjx4IMPioKCAvHjjz9qeh/P8yaer776SgAQbdq0iftny52LdTqFWLhQ+/nYuXPrt+V0+p/3LSsToqTE/Od4JUbth5HmhhDMDqsaNWqUACCef/55o4eiSayyI1FzQ9cZESEEHnzwQXz00UdYsmQJ2rZtq+fHkQ1IN7/bu3cvjh07FtfPljsX6/EATZpEvxrjvff6H+HYaTGiWGNuJC69FjbTWyyyQyk31qzxfm3n3NC1EJk4cSLeeecdvPfee8jKysKhQ4dw6NAhnDt3Ts+PJQtr2LAh2rdvDwBYv359XD87VudilULppZeiG1+iYG4kLqlPZPny5fAE7kQmFovsUMqNQYNseLluAF0Lkddeew1utxvDhw9HixYt6h4LFizQ82PJ4ozqE4nVudj/W6k+yIsv2nMxolhjbiSuvn37Ij09HceOHcMPP/xg9HBUi0V2dOwIOBzBzyfCaqtJem5cCKHn5smm+vXrh4ULFxrSsDp+PDBypPfSuA4dvEFSXu49WunYUTlYfF+zaJH8a6RL7uw4tRpLzI3ElZKSguLiYixZsgTLli1Dt27djB6SatFmR6i4s3t28F4zZDpGXznjey5WzYqGvq9p3RqYMEF+u3a75I5ID1btEwGiy47HHvO2ucqxe3awECHTkRpW9+zZg+PHjxs2DjU3pAp8jdQzH8jptOEld0Q6kPpEvvnmG8vOjkWSHUpsebluABYiZDq5ublo164dgOgbVqO5UZSaFQ3lXhPI6QRWrdJ+C3KiRDRo0CAkJSXhwIED2Lt3r2HjMDo7nE5g4UL197exMhYiZErS6ZloCpFobxSlphNe7jUOh3/T2pw5wIAB2sdPlIgaNGiA/v37AzBuuXejssP3tXPmAKNH23smRMJChEwp2j4RNVOj4ajphJd7zeuva79bJxHVM7JPxMjsmDkzMXND16tmiCIVbSESampUyxGGXCe82tckwpEMkR4uu+wyzJo1y5AZETNkR6JhIUKmJDWs7t69GydPnkTDhg01vV+a9vQNlEg7zwsK1F22O3y49m0TUbAhQ4bA4XBgx44dOHToEJo3bx63z45HdgRe1puoBYiEp2bIlBo1alS3tHckfSLxuFFUtOeRiUhebm4uevToASD+fSJ6ZwdzIxgLETItaVYk0tMz48fr16sR7jxyNB33RFR/Ga8Rp2f0yg61l/UmWnawECHTisXCZnrdKCrUeWQe8RBFz+iFzfTIjnCX9SZqdrAQIdMyeoXVUJQuz8vIiL7jnojqC5HNmzfj5MmTxg4mRkJd1huLq3WsioUImZZUiOzatQtut9vg0fhTOo98+nT4hYyIKLwWLVqgQ4cOEEJgxYoVRg8nJkL1n6hZBM2uWIiQaTVu3BiFhYUAol9hVQ9y55FjcTtwIvIysk9EL0r9J4mcHSxEyNTieXpGahJbs0Z9s1jgeeR4XK1DlCiM7hNRS2t2yPWfJHJ2cB0RMrV+/frho48+imqF1XC34Qa8TWGBN6ByOr3BIB2xhNuW9P2RI71HOmoWKVI7PqJEJM2IrF27FmfPnkWDBg3i9tnxyg7f72lZ4MxW2SFMzO12CwDC7XYbPRQyyBdffCEAiI4dO2p+79y5Qjid3vvhOp3er+WUldW/LvDhcnm/H25bWj5ryRJ12zQLK+6HVhwzBfN4PCI/P18AEEuWLInb58YrOyLJDS3vM5KWfZCFCJna0aNHBQABQJw8eVL1++QCQgqGQAsWyAeJ9Fi4MPS21H5WYHg4HOrGZzQr7odWHDPJGzt2rAAgnn766bh8ntr9uaxMiOefjzw7Is2NWbPUZ5uRtOyD7BEhU8vLy0Pr1q0BABs2bFD9PrUd6PPmAWPHKm/H5fLu6nLb+uCD+unRcJ8ld2meEOHHR5To4t0nomZ/ltb7+M1vlLcTKjtKSyPPjcmT7Xd1DQsRMr1IGlbDdaCXlwMLF3p38sCCwPf1f/sbMHiw/O26J03yhtHateG73eVCR+7zEqFDnkgLqU+ktLQUFy9e1P3zwmXHmjXBPSGBwmXH2LGR54bHAzgcod9nNSxEyPQiKURCdaBLRzNjxsiHyWuv+V9aF7gtXx4PMHUq8Mc/Kne7l5cDR48Gh05gmPziFzZoOiOKsa5duyI3Nxdnz56Ny2X84bKjqEg+N158EVi9Wj47Avf9aHLD5QJuusn/OctnRxxOFUWM53lJCCE+//xzAUB06tRJ83vLyoQoKQndz6H2XGtZmRAvvKB8LnjBAu9/fd/ve37X4aj/3y4Xe0T0ZMUxk7Kf/vSnAoCYOXNm3D5TS3aE23eV+tAiyQ32iBAZQJoR2bFjB6qqqjS9N/B6/VCnSBwO4JFHQm9r9Gj5mY2xY70zLGPHAosWeZ8PPL8rnQJauBB47z32iBCpZcTCZlqyI1RuAPKnaCLJjb17gX792CNCFHdNmjRBq1atAGhrWJUjd/5XIgQwe3bom00FTttK25K7P4TS+d0mTeSDyerneYn0IjWsLl++HJ5wzVY6CZUd8cqNggJ7rsDKQoQsIVYrrIbq95Ao3WxKWj1RWrCspAR4/33lmY1QgZHIqygSadWnTx9kZGTgxIkT2Lp1qyFjUOr3kIS6SV15OdCunfdqmWhyw3ccdsoOFiJkCbFc6l2618MLLyi/xvfyXCD49tyLFnmnbUPNbIQLDKV7ThCRv+TkZBQXFwMwdrn38eO9RYSSwNwA/LNj0CBg9+7ockMah62yIw49KxFjwxlJPvvsMwFAdO7cOWbbDNe4qnYBoblzvV9LzweuchjY9GY1VtwPrThmCu2ZZ54RAMSYMWMMHYfa3Jg7N/SiZcyNeg4hlFZRMF5VVRVycnLgdruRnZ1t9HDIQEeOHEGzZs3gcDjgdruRlZUVk+3Om+edTq2tVX6N0ynfpFZS4p0VAbxHQGruD2FFVtwPrThmCu3rr7/G5ZdfjhYtWuDAgQNwBF7/HkdqcsPl8jaljxkT/D0pO5gbXjw1Q5bQtGlTNG/eH0IMw6JFsTtH7DvFuXq1/OkaNQsIyd1Nk4hip6ioCMnJyTh48CB+/PFHTe+V+rvU3FFbDTW5UVvrzY1w/R7MDRYiZBHz5gGHDn0LoAS33DJQsTs9ElIYDBggf3muyxV64SEi0l96ejoGDBgAQFufSGB/V6yyQ01uFBfbr7FUDyxEyPSk6+qlP1chnIrd6b7vieQISK5RbMYMoH//+o53WzSHEVmQ1vVE5O7Vokd2KDWYAv5XyzA75LEQIdNTewM7SbRHQL7Trs89B0yZ4t/xzqMZImNovQFePLMj8EoWIPhqGWaHPDarkumVl3t3aN9AcbkE9u51BO3Y8q/1BkOoEJAWEurY0f9eD7HalpVZcT+04pgpPLfbjdzcXAghcODAAeTn54d8vZZ92OjsSOTc4IwImV7wImQ1ePTRXbI7q9YjICD4KGj27NhtK9Lz0bFuriOyg5ycHPTq1QuAutMzWhb/MjI7Ej439L2SODpcC4B8lZUJMXjwEwJoKf70pz8pvkbLDaGU1gSYOTM224rkZlS+N7yS1iMwkhX3QyuOmdT59a9/LQCIBx54QPV71KzJYVR2MDd40zuykIICYMSIFAAHFFdY1br8sdKNrKZM8f739tv9n5e73bZ0FLJyZfQ3o4qkuY4okURyAzw1l8kalR0LFzI3dC1EvvnmG/zkJz9Bfn4+HA4HPvnkEz0/jhKAmqXetSx/rHQjK4/H2+n+j3/4P//OO8rLN996a/j1RsKJZHrYjpgdpGTo0KEAgC1btuD48eMx3bYR2fGb3wRvI9FyQ9dC5MyZM+jVqxf+8pe/6PkxlECkQmTbtm04c+aM4uvULhRUUOBdIySQy+WdJA21c8sdhfguYORyhb89eCA73lkzEswOUtKsWTN07twZALBixYqYb9+I7Ah8/3PPeYsLtTMaVs8NXQuRa6+9Fs8++yz+53/+R8+PoQTSokULtGjRAh6PBxs3bozJNh99FLj5Zv/nfvGL0DemAoCXXpK/Xff8+d5tejzytwcP1VAmd7vwGTPs0UWvBbODQtF6Ga9ecnP9v3Y4vKdzIskOAHjxRe/+PnmyfOOqUnZYPjfi0LMihBACgPj4449Dvub8+fPC7XbXPcrKythwRkFGjRolAIiXXnopJtv797+DG87C3ZiqrEz+ZlculxCrV4e+0ZX0PYdDiEcflW9K873RntGNZ0Y3fjI7KNDbb78tAIiioiLDxhAqN4QwJjusmhumKkSmTZsmAAQ9GCbkS/o7ueOOO6Le1rhxynfQLCnxvkau437BAvn33H+/EEuWyH9v4UL5LvvAwIhVF32sWKEQYXYklj179ggAIikpSZw+fTrun68mN4SIb3ZYOTdMddXM1KlT4Xa76x5lZWVGD4lMSKlhVes19GvWAG+9Jf+9SG9MdcUV8udrnU7588ZAcIe71RvPjMDsSCyFhYVo1aoVampqsGrVqqi2pVduALHJDqWeE8A/O6ycG6YqRFJTU5Gdne33IAokFSI//PBDXcOqmgWBAgNH6eo/6TxvqPAYPDj4Chmn03uTK+l8re/3hQA2b1benm9gWL3xzAjMjsTicDhi0icS79wAwmeH3GW/bdsqb0/KDivnhqkKESI18vPz0bx5c3g8HmzatEnVNfRygfN/ORbkf/83/I2pCgqAmTP9r5CZM6c+hEaODC5EZsxQ3p7LBWRkeAMP4B07icKJZD0RX0bkBhA6O8rL5S/73bNHeXtOJ3DkiPd/WzU3dC1ETp8+jY0bN9Zd3bBnzx5s3LgR+/fv1/NjKQH4np4JNyWpFDgtWgDjxvm/b9w4YNSo8J8/b563s126ZHfGDP8QkhuT9NpATqf3qGfQoPrAA9SvZ2BHzA4KR5oRKS0txYULFzS/34jcAEJnh9KYfJcFCCQEMGaMxXNDz2aVkpIS2QaycePGqXq/0U1yZF6//e1v6/6WwjVpKTWASU1lq1cL8eKL3v+qoaYpTOk1M2fWd9I7nd7O91Cd8mZgxH7I7KBwPB6PyMvLEwDEypUrNb8/3rmh5jNDfd/3ihiXy9vcapfc0HVGZPjw4RDeK3P8Hm+++aaeH0sJwHdGJNzSzOHOnQ4YADz8sPe/aigdtXzwQf20rtKYGjWqf68QQJcuwOnT1m0y0wuzg8JxOBx1q6xG0icS79wAlLOjtDT0mBYtCp5FueUWG+WGbuVQDPCohpSUl5cLAMLpdIozZ84IIULf2Erpmv5IKN3sSulSXGlMSkc7nBGJPSuOmbR74YUXBABx/fXXR7yNeOWG9FlqL+FPpNxgsypZUn5+Ppo1a1bXsAqEvlROyz0kwgk8avEV2PAmjQlQvrnVmTPWbTIjMpLUsLp8+XLU1tZGtI145Yb0WXPmBM+0KOVGQYHyzTTtlBssRMiSHA6Hqhvg+Soo8E6rhrqHg9o1BaSAeuGF4O8FTo+qublVrAOPKBH06tULmZmZcLvd2LJliy6fEcvcALz79vvvBz8vd1pl3jxg7Njg19otN1iIkGVpLUTCrRmgZk0BXwUFwOjRoc8jh7q5VeD9ILQsfkREQFJSEoYMGQIg8st4w4l1bgDh70UD1GeHEP6vs2NusBAhy9JSiIRbM0DNmgJywjW8yTWnSTweYMoUdcFFRPL0vAGeUbkBKGeHHXODhQhZllSIfP/99zh37lzI14ZbM0DubpjS9wOnXQO/DjU9Ktd570ttcBGRPN+FzUTg9EGUoskNwD8rtOQG4M0OuXWHAPvlBgsRsqyWLVuiceNeqK29FIsX/xDytaEuxSsvB55/Pvg9Lhewdq3/tOudd8pPwypNjyo1p/my7CV3RCYwYMAApKSk4PDhw9i5c6eq96jt6Yg0Nzp08D9l07q196E2N6TvyfWVSeyUGyxEyLL+/ncHjh1bD6AEN97YO+RUZaip0J07g8/DAsA999Rfuw94//vWW6GnaeXCbfx44F//Uh6bVe4HQWRGaWlpKCoqAqCuT0RLT0ckufHII97/+p6ykS6wBeRnM5Sy46GHlA9i7JQbLETIkqRzs9KfsBDOsFOVclOh5eXA0aPyN6G64grl/g6JdFQSLtwyMuTf73Ra95I7IrNQ2ycSSU+H0imUzEz53HjoodC9YYD/bEao7CgoACZNkt/GI4/YJzdYiJAlRXrLa9+pUCkAxozxfk8KFekmVHKd7YGcTuCHH8KHm9L53lWrrHvJHZFZqL0BXixyA/Bmx6BB/jMivjevC9cbJt2obs2a8Nnx0EPy21B63opYiJAlyZ+7FaqnKgOPjITwbu+114D33vPePVduWnbcuPqvHQ7v+x54ILJwczq9N9AiougUFxfD6XRiz549KA8xvRFu2XY15C7JdzqBl1/25gYQnB1OZ/2BiJQbY8YARUXqskNu5sVObPbjUKKo39GlQ5IaTJmyR/VUpdKR0QMP1N/Jct684GnZN9/03hfit7/1vkepST8w3OTOJ3s89mk2IzJSdnY2+vTpAyD0rIiay2bDUbqz9gMP+J9a8c2OffuAb78Nzg25/EjE7GAhQpYhf/mbAwMGPA6gDVq2XKR6W0pTp3INZYGncwYNAp55JnQRMmOG/0qMsTgSIyJlofpEfLMj2tVIQ512UVqqfdGi8LkBJG52sBAhS1Bq6CooAK6+OhnAAdUrrErvC5w6DRQ4RRpqlVSJ0+ldbGjKFP+xFhQAt9/u/9pf/MI+zWZERlPqE5HLjmhWIw11rykgstwAEjs7HCLWK8DEUFVVFXJycuB2u5GdnW30cMgg5eXendJ3R3a5vEczBQXARx99hJtuugm9e/fGhg0bNG971y7vVS2DBil/BuA9grriivDbdDqDt1NaGn77ZmXF/dCKY6boHD16FE2bNq3733l5eWGzIxrl5d79esyY4KbVSHIDsFd2aNkHOSNCpheu011aYfW7777D+fPnAahfsEg6MhowIPy543Cd8EBwkEhjXb48soZWIlKnSZMmuOSSSwB478YLRHaVjJbsGD0aeP117bnh27zq+1yiZgcLETK9cOdIW7dujcaNG6OmpgZbtmwJmoqdPVvd54Q7d1xQAPzxj/VjCbyKxuUCnntOfqxDh9r/PC+R0QJPz2jtr4jkBnZqciPwIGfOnOACJpGzg4UImV64TneHw1E3K/Lf/24Lui7/sceAWbPUf5bSueN58+pXWnU4vE1lb77pDZ+FC72X/d56q/xY1cy4EFF0AhtWtVwlU14OTJjgnx333qvufi5qek58twvUFzDMDgDCxNxutwAg3G630UMhEygrE6KkxPvfQFOnThUAxHXXzRL1CyrXP5xO+fep+cwlS4RYvdq7Dd9tulze78+aVf89p1OIuXOVxxrqZzArK+6HVhwzRW///v0CgHC5XKKqqqrueTX73YIFwbkBCLFwYWRjSfTs0LIPJhlZBBFpUVCgfBQgzYjs2/dfOJ2Pyl7nv2uXtqOIefPqu92Vzt8++6z36MT3c+67z3ukM3y4tp+BiKLTqlUrFBYWYt++fSgtLcWIESMAxH+/Y3Zow1MzZqW2Y0rra21KKkR27FiCZ5+9GPT9aFdPlLv0zun0TpkGslsjGVmI1iywYXaoXe490ODBwQ2kDgdQXKzt85kd2rEQMSMtHVORdFfZUGFhIRo1aoSLFy/i6qs3YeZM/6bSWKyeCPhvc8IE+cWJnE57NZKRRWjNAptmh9ob4AUqKPA2kEr7uNPp/VrrLASzIwJxOFUUsYQ8z1tWpnxCMZrXJoCrr75aABB//etfhRDRnVNV+tWuXu3dpu+53cDHrFkx/bEMZ8X90IpjjorWLLBxdmzbtk0AEKmpqeL8+fOa3x9tLwazw0vLPsgZEbPRcuF7pLeStCnp9Iy0wmosV0/07WDv0KH+6hlfDgcwcybw6KPht2/DGXEyktYssHF2dOrUCU2bNkV1dTXWrFmj+f3R5Ib0fmaHNixEzCYzU/75jIzg5xLhJgQaBBYi0VJaH0Bp6nXBAu+lwuHYdEacjKQlNwBbZ4fD4ag7PaO1TyRWmB3asBAxm9On5Z8/cyb4uVjcStJGpEJky5YtqK6ujsk25Y6OlDI8sKlN7shFrpHN9yZZRBHRkhuA7bMj0j6RWIo0O5RmPOycHSxEzEbrkUq0t5K0kTZt2iA3NxcXL17Ed999F9Nt+4aDlOG+TW2BGa505GLjGXEyUiQzHDbODunKmRUrVqC2ttawcQQWFeGyI9SMh52zg4WI2URypBLtSU2b8F1hNVanZwDt06FyKzRKRy42nhEnI0U6w2HT7OjZsyeys7Nx6tQpbNq0yZAxRJIboWY87JwdLETMyMZHKnqLdSEiFw733hs6MF56KfjSPOnIxeYz4mQk5kYdl8uFIUOGADDm9IxSUbFmjXJ2hJvxsHN2sBAxA7mTgjY9UtFbrAsRuXDweJQDo7wceOGF4O34rg/A/7+gmGBuhBTpwmaxoFRUhLqTrpp+Y7tmBwsRo9m1Ddogvg2rFy5ciHp7mZnyt/BWmiJV6oqfNMn//xv4/xcUFeZGWL5Xzgi51cN0pHQaJdSddNX2G9sxO1iIGMnObdAGadu2LXJzc3HhwoWoG1bnzQMGDfIvLKRbeCtNkSoF0EMPRTUUonrMDVX69++PtLQ0HD16FNu3b4/rZy9a5H96VmpKDXUnXTv3gITDQsRIdm6DNojD4UDfvn0BRHd6JjDrAW9IlJZ6p0MDp0hHjvT+b0DdeVw7LkpEccLcUCU1NRVFRUUA4tsnImVH4CTMyJHe//pmR2kp0K6d/9V4iZgdLESMlMglsI5i0Sei1BviO00qTZEuWuQ/Sw6EPo/LWXWKCnNDNSP6RJSyw7dOLCgAdu/2zrj65kC4HhC7ZgcLESPZuQ3aQLEoRNRmvdIsOSB/Hjfes+p2PHpKeMwN1YxY2ExNdoTKAaUeEDtnBwsRo9m1DdpAUiGyefPmiBtW1Wa9mW/xYdejJwJzQ6Xi4mK4XC7s378f+/bti8tnqsmOSHLAztkRl0Lk1VdfRdu2bZGWloZ+/foZtv6/admxDdpA7dq1Q8OGDXHhwgVs3bo14u2MH+89h/vCC/W9IYG0zpLHa1bdDv2MzI0wmBthZWZm1vWMxfPvJ1x2RJIDds4O3QuRBQsW4OGHH8aTTz6JDRs24NJLL8W1116L/fv36/3R1sX59KjEqmFVumpm0iTvf+WOCrTOksdrVt3q/YzMjQgxO4IY0ScSLjsiXUDbttkhdDZw4EBx//33+z3XpUsXMWXKlKDXnj9/Xrjd7rpHWVmZACDcbrfewzSPuXOFcDqFALz/nTs3vp9fVibEkiXe/1rYY489JgAE/e2pVVZW/88gPVwu5V9LWZkQJSXqf21aX6+V1vGH4na7474faskNIZgdQghmh4JPPvlEABBdunSJy+dp2fciyQGrZIeW3NC1EKmurhYul0t89NFHfs//+te/FpdddlnQ66dNmyYABD0SJkxi+f8ekTA6yGJo/vz5AoAYMGBARO9fssT/n0F6lJTEdpx6mjvX++cj/RlF+s8Z70JEa24IwexgdiirrKys+3s4fPiw7p/H7PDSkhu6npqprKxEbW0tmjVr5vd8s2bNcOjQoaDXT506FW63u+5RVlam5/DMx8j5dDs0FfjwbVi9ePGi5vfb4QpJq/Yzas0NgNnB7FDWuHFjdOvWDQCwfPly3T+P2aFdXJpVHQ6H39dCiKDnAO8CNNnZ2X4Py4jFuVkj/4Kt3lQQoH379sjJyUF1dTW+//57ze+3yxWSVu5nVJsbALOD2RGa1CcSj8t4mR3a6VqI5OXlweVyBR3FHDlyJOhox9Jida2TkX/BdijjfcSiYVVpBUTSV8LkBsDsiJN4N6wyO7TRtRBJSUlBv379sHjxYr/nFy9ejMGDB+v50fET62lJo+bT7VLG+4jFwmZKKyCSfhIiNwBmRxxJC5tt3LgRVVVVcflMZocG2ltQtJk/f75ITk4W8+bNE99//714+OGHRUZGhti7d2/Y9xrRra+ZHTqTfOndkh1H77//vgAgioqKIt6G0T2AZmDEfhhNbgjB7DCEybOjXbt2AoD4/PPP4/J5iZ4dWvbBJL0LnTFjxuDYsWN45plncPDgQXTv3h2fffYZCqWbclidNC0ZeItWubXAd+70vt5ERwpBCgrMPT4NpBmRTZs2oaamBklJ2v/cQ53+tsmvyZRsnxuAuuywSm4Aps+OSy+9FD/++CO++eYbXHPNNbp/HrNDvbg0qz7wwAPYu3cvqqursW7durrzdbagZloy8DzwrFnqmtOMWpzIJositW/fHllZWTh//nxEDauAJU5/25atcwMInx1Wyw2jPzuMePeJMDs0iMMMTcQsMb0qUZqWlJufkx6hrrc36rp8E68HEIlhw4YJAOLvf/97xNuI1XocVmWp/fD/WGrMctlhtdww+rNV2LFjhwAgUlJSxLlz5+LymYmcHaZZ0CxalgoTJUrngUOdNDTq5KINT2pOmjRJABATJ06MajsmP/2tKyvuh1Ycsx8r5YbRn62Sx+MRzZs3FwDE0qVL4/a5iZodplnQjCA/P+dL7np7o67Lt8B6AFrF4soZwNrrcZAFWSk3jP5slRwOR93VM/FYT0TC7AiPhYjeAs8DB5I7aRju5KJe52FteFIzsGGVyBL0yA0g4bPDiBvgUXgsROJh/HhgyhQgcFVIpevtFy3yTm5KnM7618VqASQ5FlgPQKuOHTsiKysL586dww8//GD0cIjU05obBQXA7bf7P/eLXyg3vyZgdkgzIitXruSBiYk4hPD9fzxzqaqqQk5ODtxut7WWbA40ezbw2GP+zzmdwKpVwIAB/s+Xl3tDwnea0+kE9u3z/u/A77lc3sWLYrnDl5d7p1Q7dDBdkERi2LBh+Oabb/DGG2/gzjvvNHo4lmPF/dCKYw6iJTcA+eyQ8gFgdgCora1FXl4eTp48iTVr1qB///5GD8m2tOyDnBHRW3k5MHly8PMeD3DmTPDzcudaPR7vzq10HvaDD2I71Wqzk5qx6hMhihutuQGE7tNgdgAAXC4XhgwZAiC+fSIUGgsRvckFAOA9spE7fxrqXKtSA9ukSVw/OAQWImQ5WnMDYHaoFM8b4JE6LET0phQAzz0nf9Qgd671kUfkv+fLZLfeNhOpENm4cSPPC5M1aM0NgNmhklSILF++HB65Yo/ijoWI3gIDwOn0rpAYeO7Xl3Tzqkcf9YbE7Nn1Ry3S9154Ifh9Jrtcziw6deqEzMxMnDt3Dtu2bTN6OEThRZIbALNDhb59+yI9PR3Hjh1jA7tJsBCJlVCXxfneFXPfPm9IqPHCC/VXz/getRQUAKNHW+JyOTNwOp3o06cPAJ6eIRNSyo5IcwNgdoSQkpKC4uJiALyM1yxYiMSC0mVxvgGjtYkr3AJBFrlczizYJ0KmJJcd0eQGwOxQwYiFzUiZ7nfftb3ycuDee+t3fOno48QJb9e7x+M9+pgzx3uEo5aaO3OOHw+MHGnqy+XMgoUImY5cdkyY4F03JNLcAJgdKvg2rAoh4Ahcq4XiijMi0VI6+pCKEEB7M5h06+8//jH8UYvJL5czC9+G1draWoNHQwT57BAi8twAmB0qDRo0CElJSThw4AD2SuuskGE4IxItuaOPwK+B+qnRcDv9vHn1R0lOp7dLfsAA7UctUiB17JiQQROoU6dOyMjIwJkzZ7Bt2zZ069ZN1fv4ayTdyGVHILW5ATA7NGjQoAH69++PVatWYdmyZWjbtm3MPyMBfo0xwxmRaMmdb5WmV32Fuv5fIjdVO3Wq9iDRcylni3K5XJobVvlrJF3JXRkjd4pgzZrw22J2aKZnn0gC/RpjgoVILAReMve3v/nfKwYI/lpOLO5gqdSzkiBrBISipU9E71+jXvceI4vxzQ4h5HNi6tTwfyjMDs30ugGenr9Gu+YGC5FY8r1kLpAQ4UMhFnewtMDtuI2ipRDR89fIoyUKEio71PzhMTs0GzJkCBwOB3bs2IFDhw7FbLt6/RrtnBssRKIllagrV4Y+16smFGJxWZ1FbsdtBKkQ2bBhQ9iGVb1+jQl20ElKfA9tlZZzlzA7dJGbm4sePXoA8K6yGit6/BrtnhssRKLhW6Leeqv8+V2JxwMsWhT8fOBcm+8iRnv3ar90j2sEKOrcuTMyMjJw9uxZbN++PeRr9fo1JthBJ8kJPLRdt05+OXeJXHbIzdEzOzTTo09Ej1+j7XNDmJjb7RYAhNvtNnoowcrKhHA6pbO63ofTWf+cy+X/Pem5srL6bcydW/96h0OImTPVf/aSJf7bkntNSUno1ySgIUOGCADi7bffVvX6WP8a5f5sAv8szMbU+6EC045Z6Q9g5sz6zAiXHYG58eij6v6A1OSG9LoEyY4FCxYIAKJ3794x33Ysf412zw3OiERKrkT1eID5871HJO+9F/we3xI2cK5NCODxx733hlBSXu6910Tr1qFXcQUSeo2AULQubBbrX2MCHnSSL6VD2wED6mczQmWHXG743k9GTqjckL6foNkhzYhs2rQJJ0+ejOm2Y/lrtH1uxKEwiphpj2qECF+ihvv+kiXBRz3SEY5cmet7FBQ4C3Pvvd73SV/PnRu/34PFvPXWWwKAGDp0qKHjsNJBp6n3QwWmHbOaQ9tQr1HKDWnfD/yDUsoNl0uI1au9sykJnh0dOnQQAMSnn35q9FDCsmtucEYkEmpWLwxVwpaXA0ePym9bCOCll4I/z/coyJfH4/0cuRtcURAtDat6SqCDTgo0aVLoQ1ul7AC8uaHUi+bx+GdHqNyorQWKiryzKQmeHXpdxqsHu+YGCxGtfBvNJk8GZsxQbg6Tax6T3j9mjHKgvPiifxiE66oPZKsuptjq0qULGjRogDNnzmDHjh1GD4cSibTvz57t3Z8ffVS5qTQwO4D63ADUZUe43JC7XDgBs4M3wDMeCxEtIlm90LeElTu/KxcogWGg9UZtNr7kLloulwu9e/cGAKxfv97YwVDikNv3X3wx9Huk7ACC3+t0AqNGBb/HNzuUciPU1X0JmB3SjMjatWtx9uxZg0eTmFiIaKH2Giql5e+UbnIVGAy+YVBe7p15CXTbbfKX/DmdobuY7Lo0nwa8Ey/FnZrs0JIbtbXAHXcoL1ihlBv33Qd8+6327LBxbrRt2xb5+fm4ePEivv32W6OHk5BYiGihZqWaUMvfKb0/VK+J0vTqPfcEn0d+9FFg3z7l9QPsvDSfBixEKO7CZUckuVFcrNyHppQbY8d6r9DRkh02zw2Hw2GpPhFbikPzbMRM2fk+d67/9f6+XeZqOuKV3q/UDq3m6pxwbdRlZUIsWFDfHW+VC9F1smXLFgFAZGVlidraWqOHY3qm3A/DMOWYQ+37keaG9P7ADFB7dU647Fi9OiFy45VXXhEAxJVXXmn0UGxDyz6YZGQRZEnjxwM9ewLLlwNDh3qPLiShpl+lGY6RI73rBDgc3iMa36ts5KZEpQ76++7zbkvu6pxQLdS+twYPFDi2BLlvdZcuXZCeno5Tp05h586d6Ny5s9FDokSglB3R5AYgnwHhckPpfb6k7Ahsag0cG2D57JBmREpLS3Hx4kUkJycbPKIEE4fCKGKmPaqRjjQCr7sPdxQS6r3hRHIBudx49BibBRUXFwsA4t133zV6KKZnyv0wDFOOWWkfM1tuKI1JzQrRFs2O2tpakZubKwCIVatWGT0cW9CyD7IQ0SLaUy/xXqM31OJHRo/NYA8++KAAICZNmmT0UEzPdPuhCqYbs5piwyy5IYRydmg9+LKQn/70pwKAmDVrltFDsQUuaBZL4e6SGdj5rnTjqZdeiu1di9R0sSs1uS1c6D82299RKRgbVkl3vvuo3N25ffexeOVG4LiUyGWH0wmsWuXf0Gqj7OB6IgaKQ2EUMcOPagKnHEePDj9NKaesLLjhK9x7Q92gSstUaKgmN9/PsslRjVqbNm1iw6pKhu+HETB8zIE3ptO6/wsR+9wIHBezw8+3334rAIjc3FxmQgzw1EwshOuvkB5Kd8yVAmH1aiGef17+vY8+Kv/eaPpQlMYS7jyxmtCxkYsXL4q0tDQBQGzfvt3o4Zia4f+nHgHTZ4dSEaBXbiiNi9lR58KFCyIjI0MAEJs3bzZ6OJZnmkLk2WefFcXFxSI9PV3k5ORofr+hYRKqv8L3UVLi/76yMv8bSYUKIrmdO9Kb5QWOIxJWuqNSDBQVFQkA4r333jN6KKYW7/0w2twQwgLZsXCh/3v0zo1Q42J21LnqqqsEAPHKK68YPRTLM02PyIULFzB69Gj88pe/1PNj9CF3jjSQ0mJmvjeSUnrfnDnBl7mVl3v7N0Kdc1WzqFqk7HpHJQXsEzEnS+cGED47pMXIJPHIDaVxMTv8SJfxsk8kzuJQGIk33njDejMiQgRPOY4bp20xM7nHiy/6L0b26qtCPPWUEE8+qe1yORtMhRpt3rx5AoAYPny40UMxNaP2w0hzQwiTZYfTWT/TEavcWLJEiH//W4hbblGfG4HjYnYEKSkpEQBEfn6+8Hg8Rg/H0iy7oFl1dTWqq6vrvq6qqjJwNPB2h48c6T2qkG5s9+yz/l9L1Nwh1+UCbr7Z+77Zs4HHHgs/BrmFiOTGRZpJMyLr16+Hx+OBM9wMGJmW6bMDiE1uhFqgMPA9cveNYXaEVFRUhOTkZFRUVODHH39E+/btjR5SQjBVITJjxgw8/fTTRg/DX+Dqg0qrEXbs6F31UGlq1TcYZs0CHn88/Ge/+GJ9AEl8VzCU7sxJEenatStSU1NRVVWF3bt3o2PHjkYPiSJkmewIJJ0uUSosfG9EV14OTJgQ+vQNED43wq2omsDS09MxYMAArFy5EsuWLWMhEieaDwGnT58Oh8MR8rF27dqIBjN16lS43e66R1lZWUTbMURBgTck5Lz4Yv3aAEp3xQzkexQksfnNp+ItOTkZvXr1AsA+Eb3pmRuAhbNDWoo98A7ckvnz69ftWLkyfBHC3Iga+0TiT/OMyIMPPoixY8eGfE2bNm0iGkxqaipSU1Mjeq/h5s0DXn89+PnAYNi5U12YBE6rlpf7T8l6PN77SIwcaat7PsRbv379sHr1aqxbty7s3zVFTs/cACyeHUoCm1rD8Z09kajNDem1zA5ceumleO6551iIxJHmQiQvLw95eXl6jMW6pJ09sMCQKygyM5VP4Vx3nbdvRO7crZobY/meP3Y6vUdacrf1pjq8ciY+mBsKtGRH27bK27nsMuDddyPLDYDZ4WPIkCFwOBzYvXs3KioqkJ+fb/SQbE/X7rz9+/dj48aN2L9/P2pra7Fx40Zs3LgRp0+f1vNj40+p4ewvf/HfmefNAwYNki9CnnwS+M9/lC+BC3fpndyRz733hl7GmfwaVkW4mSqKi4TJDUA5O95/Xz475Dz5JLB0aWS5ATA7AuTk5NSdsl22bJnBo0kQel6+M27cOAEg6FGicgEdwy/BU0vpEjyHI/TNq5xOIV57TXmBImmpZul/P/FE/TYCL71TWqxIaRVGEkJ4V1NMTU0VAMTOnTuNHo4pxXs/jDY3hLBBdviu2KwlO7TmhhBCLFjA7Ajw61//WgAQEydONHoolmWalVWjZZkwEcK7xkioa/m1rGoY7j4VTqcQ0h0ipW0/+aT6tQQk4e5LkSAGDBggAIj58+cbPRRTstR++H8sM+a5c8Pvt2qzQ0tuCFG/mquWdUh832vj7Pjggw8EANGjRw+jh2JZpllZNWGUlwP/+If896TzsWpXNQycJpViwZfHA0yZ4l2LROqG//3vQ39+IHbS12GfCBlC2tflaF1NWUtulJd79/fWrb0ZonTpcAJnh3Qn3u+++w7Hjx83eDT2x0IkFkItSiQFhnSZnstV/7zcgkNqFjgCvCExebK6hY3CFTtSJ32CnhNmIUKGUJMbgLrs0JIbpaXyDbKhxiBJkOxo1qwZOnfuDCEEVqxYYfRwbI+FiJzycqCkRP3OpXRvCSkwAO/2Ro70ridSUlK/rkjgZ2Zmhr/HDRB6ESTf16gtdpSOfhIAG1YpJmKdG9IiZmqy4+hRdbnhcnkLkEhXZk2g7JBmRdiwGge6nyiKgiHnecPdSjvU+3zvLfHoo97zp2puze17102HQ4ibb67flsMhf1fNmTOVz+36fr70Gb7ncyO5HbiNVVdXi5SUFAFA7Nq1y+jhmI5l+i18xH3Msc4NNduUu2Ovb47I7eNz54a+v82TT/rfRTeBs+Ptt98WAERRUZHRQ7EkNqtGKtqdLPBW2OG25xs0oQJB2u7q1f7bf/RR+ff63mJcKcx48ys//fv3FwDEggULjB6K6bAQCSPWuaFmm6Gy4/77Q+eGEMrZ4dsAm+DZsWfPHgFAJCUlidOnTxs9HMthIRIpLVe2RLu9cHfddDrDB1lZWehO93BhJheACeq+++4TAMTjjz9u9FBMh4VIGLHOjXDbZHbEhcfjEa1atRIAxH//+1+jh2M5vGomUmqvbInF9sI1l3k84c+7Llrkf48Kh8P/vG6487kFBcoLqCUYNqxSxOT2c6cTyMiI7TaZHXHlcDjYJxInLER8Sd3pvgHg8XhXOdTShOa7veeeq9+ebwOYUqOaJFwBJLc0tNPpbWqTxLqwsjE2rFLEAq9qAby5MWiQ907bzA7L4g3w4oOFSKCRI/13UCGAxx+P7Jr5efO81+1L92+YMgVo184bBIHh5XDUH6Eodaz7UtO9rvaSYUL37t2RkpKCEydOYM+ePUYPh6xm/HjvZbG+swweD7PD4qQZkVWrVuHChQsGj8a+HMLEh39VVVXIycmB2+1GdnZ2fD60pMQbHEpcLu/lc+F2yPJyb/jITaH63lSqvNwbANKRhvS/I9m+0th8PyPBgkSL/v37Y926dVi4cCFGjx5t9HBMw5D9MErMDo3bZ3bIEkKgadOmqKysxMqVK1Gs5W7ICU7LPsgZkUDhpj3VXjMf6jyu7yJAvudatZx31XLEkgDnc2OBfSIUFWaH7TgcDgwdOhQA+0T0xEIkkNz5Xl9qz5PGKpRCGT9eeZEj0sy3T4RIM2aHLbFPRH8sROT47qSzZkV2njRWoaTmcwKPWLSu8EgA/GdETHzGksyM2WE7Up/I8uXLUVtba/Bo7Ik9ImpEc55Ueu/atd6Gs9ra+lDS4yhk3rz6e0H4nk+msKqrq5GVlYWLFy9iz549aNOmjdFDMgXT7IcamGbMzA7Lq6mpQW5uLk6fPo2NGzeiV69eRg/JErTsgyxE4knvxi8tTWgkq1+/fli/fj3++c9/4qabbjJ6OKZgxf3QimMOidlhqGuuuQaLFi3Cn//8Z/zqV78yejiWwGZVs1LT+BXN1GgC3ZBKL2xYJVNidhiKC5vpi4WImcyb5z0qkdYdeOwxbaHCRYiixkKELCkwO2bP1vZ+ZkdIvg2rJj6JYFksRMxCWu1QOirxeLxh0rq1+oWQuAhR1NiwSpYjlx2PPeZtllWL2RHSgAEDkJKSgsOHD2MXZ4lijoVIPClNnZaXAwsXyq8dIET9ugFq8LK8qPTo0QPJyck4duwY9u/fb/RwiLzkskN6buVK+eyYMkXbjCqzQ1FaWhqKiooA8DJePbAQiZfAqVNplkN6/je/UX6v1nO1Cb4IUTRSU1PRvXt3ADw9QyYhlx2+z40d67+0vETNze8CMTsUsU9EPyxE4qG8HJgwwX/q9N57gTVr/KdUlTidwJEj2pvQuCZARPr27QuAhQiZgFx2TJjg/5zSKUSXy3sHYK0ZwNyQxYXN9MNCRC++O/PKlcFh4fEAn34qX4Tcckt945jD4X3vmDHabpylNANDYbFhlQwVLjuEkH/uvvv879b7i1947wCsJQOYG4qKi4vhdDqxZ88elLNIiykWInoI3JmXLJF/XYsW8p3qzz8P7Nvn7RuRChHA/z4Tocg1r2npM0lwbFglw6jNjsBTMS4X8P/+nzc3Skq8dwL+xz+0ZQBzI6Ts7Gz06dMHAE/PxBoLEa3CTVvK7cxz5wa/zuEARo1S7lQvKADy8iK7tj/aNQESfGq2Z8+eSEpKQmVlJcrKyoweDtmBmn1KS3bMnKmcG8OHA6dPa8+AWKwlYvPsYJ+IPliIaKFm2lJpZ3700frZD6cTeP11b2iE6lSP9Nr+aNYE4NQs0tLS0K1bNwA8PUMxoHaf0pIdjz4a+gqXSDIg2rVEEiA72CeiE2FibrdbABBut9vooQhRViaE0ymdnfU+XC7v82pfV1YmRElJ8HtCmTvX+35pO3Pn6vc+tT9jArj77rsFAPHkk08aPRTDmWo/VMk0Y9ayT5khOyLNmwTJjiNHjggAAoCorKw0ejimpmUf5IyIWmqnLUMtDBTJpXGBMyYjR6qb+oxkTQAu81yHDasUE1r2KTNkR6RriSRIdjRp0gSXXHIJAO/deCk2WIiopWXaMtYLA0khtGiRtqlPreHFZZ7rsGGVYkLrPmWG7Iik6Emg7JBOz7BPJHZYiKildQnkWC8MFI+Odi7zXKdnz55wuVw4evQoL9WjyEWyTzE7TE1qWGWfSOwkGT0ASxk/3ju9qeftuJWEmvqM5TiM/BlNJD09Hd26dcPmzZuxbt06tGrVyughkVUZvU8xO2JKmhFZv349Tp8+jczMTINHZH2cEdHKqCWQ4zn1yWWeAbBPhGLIyH2K2RFTrVq1QmFhIWpra1FaWmr0cGyBhYge9LiWPoGmPs2ChQjFlV5rcDA7Yo6X8cYWC5FY0/Naet4dM67YsEpxo/caHMyOmOLCZrHlECZO2KqqKuTk5MDtdiM7O9vo4YRXXu4NEd/zsS6Xd8fXcvRRXu49r9uxI49aDHTu3DlkZWWhtrYWZWVlKEjQfwvL7Yew2JiZG5azfft2dOnSBampqXC73UhNTTV6SKajZR/kjEgsxeJa+gRYndAq0tPT0bVrVwA8PUM6Ym5YTqdOndC0aVNUV1dj7dq1Rg/H8nQrRPbu3Yvx48ejbdu2SE9PR/v27TFt2jRcuHBBr480XrRNYbzplOmwTyS+mBv/h7lhag6Hg5fxxpBuhci2bdvg8Xjwt7/9DVu3bsWLL76Iv/71r3jiiSf0+kjjRdsUliCrE1oJC5H4Ym6AuWER7BOJnbj2iMyaNQuvvfYafvzxR1Wvt9R5Xl/l5ZFdSx+rc8UUM6WlpRg8eDCaNWuGgwcPwhF4+/UEYPR+qDU3AOPHHBHmhqVs2LABffv2RXZ2No4fPw6XVEgSABP3iLjdbjRq1Ejx+9XV1aiqqvJ7WFKk19LzMjvT6dWrF5xOJw4fPoyKigqjh5OQwuUGYJPsYG5YSs+ePZGdnY2qqips3rzZ6OFYWtwKkd27d+Pll1/G/fffr/iaGTNmICcnp+6RkKtZ8jI7U2nQoAEbVg2kJjcAZgdzI/5cLheGDBkCgH0i0dJciEyfPh0OhyPkI7CLuKKiAtdccw1Gjx6Ne+65R3HbU6dOhdvtrnuUlZVp/4nsIAFWJ7QS9olET8/cAJgdAJgbBuAN8GJD871mHnzwQYwdOzbka9q0aVP3vysqKnD55ZejuLgYc+bMCfm+1NRUXo9NptOvXz+89dZbLESioGduAMwOMobvlTNCiITsIYsFzYVIXl4e8vLyVL32wIEDuPzyy9GvXz+88cYbcAZeokb64gJHMcEZkegxNyyEuaFa//79kZaWhqNHj2LHjh3o3Lmz0UOyJN328IqKCgwfPhytWrXC7NmzcfToURw6dAiHDh3S6yPJFxc4ipnevXvD6XTi0KFDOHjwoNHDsTXmhsGYG5qkpqaiqKgIAPtEoqFbIfLll19i165dWLJkCQoKCtCiRYu6B+mMCxzFVIMGDXDJJZcA4KyI3pgbBmJuRIR9ItHTrRC58847IYSQfZAKoe7EGe4unVzgKOZ4eiY+mBsxEGl2MDciwhVWo8eTr2YihcSsWcrTo2qmTqNdMpqCsBAhU4tFdjA3IlJcXAyXy4V9+/Zh//79Rg/HkliImIVvSDz+uPz0qNqpUy5wFHMsRMi0YpUdzI2IZGZmom/fvgB4eiZSLETMIDAkAknTo1qmTrnAUUxJDasVFRVsnCTziHV2MDciIvWJ8PRMZFiImIFcSPiSpke1Tp1ygaOYycjIQJcuXQBwVoRMRI/sYG5oxhvgRYeFiBnIhYTEd3qUU6eG4ukZMh1mhykMHToUAPDDDz/g6NGjBo/GeliImEFgSDidwJNPyk+PcurUMCxEyHSYHabQuHFjdO/eHQCwfPlyg0djPSxEzGL8eOC557xB4vEAM2YAu3fLH7Fw6tQQLETIlJgdpsDLeCPHQsQsysuByZO5mJCJ9e7dGw6HAwcOHMDhw4eNHg6RF7PDFLiwWeRYiJgFFxMyvczMzLp7SXBWhEyD2WEK0ozIhg0bUFVVZfBorIWFiFlwMSFL4OkZMh1mhym0bNkS7dq1g8fjQWlpqdHDsRQWImbBrnZLYCFCpsPsMA32iUSGhYiZsKvd9FiIkCkxO0yBfSKRSTJ6ABRAuuafTKlPnz5wOBwoLy/HkSNH0LRpU6OHROTF7DCcNCPy7bff4vz580hLSzN4RNbAGREiDbKystCpUycAnBUhIn8dOnRA8+bNceHCBaxZs8bo4VgGCxEijXh6hojkOBwO9olEgIUIkUYsRIhICftEtGMhQqQRCxEiUiLNiKxYsQI1NTUGj8YaWIgQadSnTx8AQFlZGW9wRUR+unfvjoYNG+L06dPYtGmT0cOxBBYiRBplZ2ezYZWIZLlcrrq78bJPRB0WIkQR4OkZIlIinZ5hn4g6LESIIsBChIiUSA2r33zzDYQQBo/G/FiIEEWAhQgRKenbty/S09Nx7Ngx/PDDD0YPx/RYiBBFQGpY3b9/PyorKw0eDRGZSUpKCoqLiwHw9IwaLESIIpCTk4OOHTsC4KwIEQXjwmbqsRAhihBPzxCREvaJqMdChChCLESISMmgQYOQlJSE8vJy7Nu3z+jhmBoLEaIISYXI+vXrDR4JEZlNgwYN0L9/fwDsEwmHhQhRhPr27QsA2Lt3L44dO2bwaIjIbNgnog4LEaII5eTkoEOHDgA4K0JEwXgDPHVYiBBFgX0iRKRkyJAhcDgc2L59Ow4fPmz0cEyLhQhRFFiIEJGS3Nxc9OjRAwCwfPlyg0djXixEiKLAQoSIQvG9jJfksRAhioLUsLpnzx4cP37c4NEQkdmwYTU8FiJEUWjYsCHatWsHgA2rRBRMKkQ2bdoEt9tt8GjMiYUIUZR4eoaIlLRo0QIdOnSAEAIrV640ejimpGsh8tOf/hStW7dGWloaWrRogdtvvx0VFRV6fiRR3LEQiS3mBtkN+0RC07UQufzyy7Fw4UJs374dH374IXbv3o2bb75Zz48kijsWIrHF3CC7YZ9IaA4Rx7vx/O///i9uvPFGVFdXIzk5Oezrq6qqkJOTA7fbjezs7DiMkEi748ePo3HjxnX/Ozc31+ARxZbR+6HW3ACMHzORrx9//BHt27dHcnIy3G430tPTjR6S7rTsg3HrETl+/DjeffddDB48WDFMqqurUVVV5fcgMrtGjRqhbdu2ANiwGmtqcgNgdpC5tW3bFvn5+bh48SK+/fZbo4djOroXIpMnT0ZGRgYaN26M/fv341//+pfia2fMmIGcnJy6R6tWrfQeHlFM8PRMbGnJDYDZQebmcDi43HsImguR6dOnw+FwhHysXbu27vWPPfYYNmzYgC+//BIulwt33HEHlM4GTZ06FW63u+5RVlYW+U9GFEcsRELTMzcAZgeZH/tElGnuEamsrERlZWXI17Rp0wZpaWlBz5eXl6NVq1ZYuXIliouLw34Wz/OSVSxevBgjRoxA+/btsWvXLqOHE1Ox2A/jmRuxGjNRLH333Xfo0aMHMjIycOLECdX9TlalZR9M0rrxvLw85OXlRTQwqeaprq6O6P1EZiWtsLp7926cPHkSDRs2NHZAJsPcoETXtWtX5Obm4sSJE9iwYQMGDhxo9JBMQ7cekdWrV+Mvf/kLNm7ciH379qGkpAQ///nP0b59e9VHNURW0bhxY7Rp0wYAG1ajwdwgu3I6nXWnZ9gn4k+3QiQ9PR0fffQRrrzySnTu3Bl33303unfvjqVLlyI1NVWvjyUyDPtEosfcIDtjn4g8zadm1OrRoweWLFmi1+aJTKdfv3748MMPWYhEgblBduZ75YzH44HTybusALzXDFHMcEaEiELp06cPGjRogBMnTuD77783ejimwUKEKEakQmTXrl28yyYRBUlOTsbgwYMB8PSMLxYiRDHSuHFjFBYWAmDDKhHJ48JmwViIEMUQT88QUSi+DatxvNWbqbEQIYohFiJEFEpRURGSk5NRUVGBPXv2GD0cU2AhQhRDLESIKJT09HQMGDAAAPtEJCxEiGJIKkR27tzJhlUiksU+EX8sRIhiKC8vD61btwYAbNiwweDREJEZcWEzfyxEiGKMp2eIKJQhQ4bA4XBg165dOHjwoNHDMRwLEaIYkwoRXsJLRHJycnLQq1cvADw9A7AQIYo5zogQUThSnwhPz7AQIYo5qRDZsWMHTp06ZfBoiMiMeCfeeixEiGKsSZMmaNWqFYQQbFglIllSIbJlyxacOHHC4NEYi4UIkQ769u0LgKdniEhes2bN0KlTJwghsGLFCqOHYygWIkQ6YJ8IEYXDPhEvFiJEOmAhQkThsE/Ei4UIkQ6kQmT79u1sWCUiWdKMyNq1a3HmzBmDR2McFiJEOmjWrBlatmwJIQQ2btxo9HCIyIQKCwtRUFCAmpoafPvtt0YPxzAsRIh0wtMzRBSKw+FgnwhYiBDphoUIEYXD+86wECHSDQsRIgpHmhFZtWoVLly4YPBojMFChEgnUiGybds2nD592uDREJEZXXLJJcjLy8O5c+cS9qCFhQiRTpo3b478/Hw2rBKRIofDgaFDhwJI3Mt4WYgQ6YinZ4gonERvWGUhQqQjFiJEFI7UsLp8+XLU1tYaPJr4YyFCpCMWIkQUTu/evZGZmQm3243vvvvO6OHEHQsRIh35Nqwm8sqJRKQsKSkJgwcPBpCYfSIsRIh01KJFC7Ro0QIej4cNq0SkKJH7RFiIEOmMp2eIKBzfG+AJIQweTXyxECHSGQsRIgpn4MCBSElJwaFDh7Br1y6jhxNXLESIdMZChIjCSUtLw8CBAwEk3ukZFiJEOpMKkR9++IENq0SkSOoTSbSGVRYiRDrLz89H8+bN4fF4sGnTJqOHQ0Qmlag3wGMhQhQHPD1DROEMHjwYTqcTe/bsQXl5udHDiRsWIkRxwEKEiMLJzs5G7969ASTW6Zm4FCLV1dXo3bs3HA4H11KghMRCRDvmBiWiROwTiUsh8vjjjyM/Pz8eH0VkSlIh8v333+Ps2bMGj8YamBuUiBKxT0T3QuTzzz/Hl19+idmzZ+v9UUSmlZ+fj2bNmrFhVSXmBiUqqRDZunUrjh07ZvBo4iNJz40fPnwYEyZMwCeffIIGDRqEfX11dTWqq6vrvna73QCAqqoq3cZIFC89e/bE4sWLsXz5cnTr1s3o4agm7X/xWu1Ra24AzA6yj9TUVHTq1Ak7duzAl19+ieuvv97oIUVEU24InXg8HnHNNdeI3/3ud0IIIfbs2SMAiA0bNii+Z9q0aQIAH3zwYcJHWVmZXnERVW4wO/jgw7wPNbnhEELbYc706dPx9NNPh3zNmjVrsHLlSixYsADffPMNXC4X9u7di7Zt22LDhg11XcGBAo9qPB4Pjh8/jsaNG8PhcCh+XlVVFVq1aoWysjJkZ2dr+XFMyU4/D38Wc9LyswghcOrUKeTn58PpjOxsrp65ATA7AP4sZmann0ftz6IlNzQXIpWVlaisrAz5mjZt2mDs2LH497//7RcCtbW1cLlcuO222/DWW29p+diQqqqqkJOTA7fbbfl/ZMBePw9/FnOK989ixtwA+G9qVnb6WQB7/Tx6/Cyae0Ty8vKQl5cX9nV//vOf8eyzz9Z9XVFRgZEjR2LBggUoKirS+rFEZGHMDSJSoluzauvWrf2+zszMBAC0b98eBQUFen0sEVkYc4Mo8dhiZdXU1FRMmzYNqampRg8lJuz08/BnMSc7/SzRsNPvgT+Lednp59HjZ9HcI0JEREQUK7aYESEiIiJrYiFCREREhmEhQkRERIZhIUJERESGYSFCREREhrF1IVJdXY3evXvD4XBg48aNRg9Hs71792L8+PFo27Yt0tPT0b59e0ybNg0XLlwwemiqvPrqq2jbti3S0tLQr18/LFu2zOghRWTGjBkYMGAAsrKy0LRpU9x4443Yvn270cOKiRkzZsDhcODhhx82eiimYfXcAJgdZsDcUM/Whcjjjz+O/Px8o4cRsW3btsHj8eBvf/sbtm7dihdffBF//etf8cQTTxg9tLAWLFiAhx9+GE8++SQ2bNiASy+9FNdeey32799v9NA0W7p0KSZOnIhVq1Zh8eLFqKmpwYgRI3DmzBmjhxaVNWvWYM6cOejZs6fRQzEVq+cGwOwwA+aGBtHcKdPMPvvsM9GlSxexdetWAYS/e6dVzJw5U7Rt29boYYQ1cOBAcf/99/s916VLFzFlyhSDRhQ7R44cEQDE0qVLjR5KxE6dOiU6duwoFi9eLIYNGyYeeugho4dkCnbNDSGYHUZjbiiz5YzI4cOHMWHCBPzjH/9AgwYNjB5OTLndbjRq1MjoYYR04cIFrFu3DiNGjPB7fsSIEVi5cqVBo4odt9sNAKb/dwhl4sSJuP7663HVVVcZPRTTsHNuAMwOozE3lOl2rxmjCCFw55134v7770f//v2xd+9eo4cUM7t378bLL7+M559/3uihhFRZWYna2lo0a9bM7/lmzZrh0KFDBo0qNoQQmDRpEoYOHYru3bsbPZyIzJ8/H+vXr8eaNWuMHopp2Dk3AGaH0ZgboVlmRmT69OlwOBwhH2vXrsXLL7+MqqoqTJ061eghK1L7s/iqqKjANddcg9GjR+Oee+4xaOTa+N7KHfDujIHPWc2DDz6IzZs34/333zd6KBEpKyvDQw89hHfeeQdpaWlGD0d3dsoNgNlhVcyN0Cxzr5nKykpUVlaGfE2bNm0wduxY/Pvf//b7o62trYXL5cJtt92Gt956S++hhqX2Z5H+wSsqKnD55ZejqKgIb775JpxOc9ePFy5cQIMGDfDBBx/gZz/7Wd3zDz30EDZu3IilS5caOLrI/epXv8Inn3yCb775Bm3btjV6OBH55JNP8LOf/Qwul6vuudraWjgcDjidTlRXV/t9z+rslBsAs8OK2cHcCM8yhYha+/fvR1VVVd3XFRUVGDlyJP75z3+iqKjIcrcSP3DgAC6//HL069cP77zzjmX+T6KoqAj9+vXDq6++Wvdc165dccMNN2DGjBkGjkw7IQR+9atf4eOPP8bXX3+Njh07Gj2kiJ06dQr79u3ze+6uu+5Cly5dMHnyZMtOG0fLbrkBMDuMxtxQz3Y9Iq1bt/b7OjMzEwDQvn17y4VJRUUFhg8fjtatW2P27Nk4evRo3feaN29u4MjCmzRpEm6//Xb0798fxcXFmDNnDvbv34/777/f6KFpNnHiRLz33nv417/+haysrLpz1Tk5OUhPTzd4dNpkZWUFhUZGRgYaN26csEUIYK/cAJgdZsDcUM92hYidfPnll9i1axd27doVFIZmn8gaM2YMjh07hmeeeQYHDx5E9+7d8dlnn6GwsNDooWn22muvAQCGDx/u9/wbb7yBO++8M/4DIgqD2WE85oZ6tjs1Q0RERNZh7s4lIiIisjUWIkRERGQYFiJERERkGBYiREREZBgWIkRERGQYFiJERERkGBYiREREZBgWIkRERGQYFiJERERkGBYiREREZBgWIkRERGSY/w8/QjbBKDEeFwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize optimization\n",
    "\n",
    "fig = plt.figure()\n",
    "ax0 = fig.add_subplot(121)\n",
    "ax1 = fig.add_subplot(122)\n",
    "x0_old = np.arange(-4, 5)\n",
    "x1_old = (- b_old - x0_old * w_old[0]) / w_old[1]\n",
    "ax0.plot(x0_old, x1_old, color = \"black\")\n",
    "x0_new = np.arange(-4, 5)\n",
    "x1_new = (- b_jax - x0_new * w_jax[0]) / w_jax[1]\n",
    "ax1.plot(x0_new, x1_new, color = \"black\")\n",
    "for x, y in zip(x_data, y_classes):\n",
    "    if y == 1:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "    else:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "\n",
    "ax0.set_title(\"before optimization\")\n",
    "ax1.set_title(\"after optimization\")\n",
    "ax0.set_ylim(-4, 4)\n",
    "ax1.set_ylim(-4, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 1.3 - Compare the results of the NumPy and JAX implementations.\n",
    "> 1. Plot the decision boundary for each implementation.\n",
    "> 2. Compare the training time and accuracy of each implementation.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-4.0, 4.0)"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABhFklEQVR4nO3deZgU1b038G91D7PALALD6rDJvu+yGzRGXBKj3iAY9eoNEo2aixIUCLnBeL0XRQ0xmnhFiPpqjOAbl6s3ucj7gOzLsAwgKCowMOOwDUsPi8zAzHn/6NRMd09Vd1V3VZ2q6u/neebR6enuOoPU11+f86tTihBCgIiIiEiCgOwBEBERUfpiIUJERETSsBAhIiIiaViIEBERkTQsRIiIiEgaFiJEREQkDQsRIiIikoaFCBEREUnDQoSIiIikYSGShl5//XUoioLs7GwcPHiw0c/Hjx+Pfv36SRgZEfmZmj1btmyRPRRyERYiaay6uhq/+tWvZA+DiIjSGAuRNHb99dfj7bffxo4dO2QPhYiI0hQLkTT2+OOPo2XLlpg5c6buc0pLS6EoCl5//fVGP1MUBU888UT990888QQURcHOnTsxceJEFBQUoEWLFpg+fTouXbqEvXv34vrrr0deXh46d+6M+fPnR73fp59+CkVR8NZbb2H69Olo27YtcnJy8J3vfAfbt2+vf96bb74JRVGwYcOGRmN68skn0aRJE1RUVJj/AyEiR23ZsgWTJ09G586dkZOTg86dO+OOO+6IWjIWQuDGG29Ey5YtcejQofrHz58/j759+6J37944d+6cjOGTRViIpLG8vDz86le/wrJly7BixQrL3vf222/HwIED8de//hVTp07FggUL8Oijj+KWW27BTTfdhPfffx/XXHMNZs6ciffee6/R63/5y19i//79WLRoERYtWoSKigqMHz8e+/fvBwBMmjQJbdu2xR/+8Ieo1126dAmvvPIKbr31VrRv396y34eI7FFaWoqePXvid7/7HZYtW4ZnnnkGhw8fxvDhw1FZWQkg/IHnzTffRNOmTXH77bfj4sWLAIAHH3wQBw4cwNKlS9GsWTOZvwalSlDaee211wQAUVxcLKqrq8UVV1whhg0bJurq6oQQQnznO98Rffv2FUIIceDAAQFAvPbaa43eB4CYO3du/fdz584VAMTzzz8f9bxBgwYJAOK9996rf+zixYuiVatW4rbbbqt/bOXKlQKAGDJkSP1YhBCitLRUNGnSRNx3331Rx8rMzBRHjx6tf2zJkiUCgFi1alVyfzBEZKvI7NFy6dIlcfbsWdGsWTPxwgsvRP1s7dq1IiMjQzzyyCPiT3/6kwAgFi1a5MSwyWacEUlzmZmZeOqpp7BlyxYsXbrUkvf8/ve/H/V97969oSgKbrjhhvrHMjIy0K1bN82rdn784x9DUZT67zt16oTRo0dj5cqV9Y/97Gc/AwC8+uqr9Y+99NJL6N+/P6666ipLfg8istfZs2cxc+ZMdOvWDRkZGcjIyEBubi7OnTuHzz//POq5Y8aMwX/8x3/gd7/7HX72s5/hrrvuwpQpUySNnKzEQoQwefJkDBkyBHPmzKmf9kxFixYtor7PzMxE06ZNkZ2d3ejxCxcuNHp927ZtNR87ceJE/fdt2rTBpEmT8Morr6C2thY7d+7EmjVr8PDDD6c8fiJyxo9//GO89NJLuO+++7Bs2TJs3rwZxcXFaNWqFb799ttGz7/zzjuRmZmJ6upqPPbYYxJGTHbIkD0Akk9RFDzzzDP43ve+h4ULF0b9TC0eqqurox6PLAqsduTIEc3HWrZsGfXYtGnT8Oabb+LDDz/E//7v/+Kyyy7DnXfeadu4iMg6oVAIH3/8MebOnYtZs2bVP15dXY2TJ082en5tbS3uvPNONG/eHFlZWZgyZQrWrVuHzMxMJ4dNNuCMCAEArr32Wnzve9/Dk08+ibNnz9Y/3qZNG2RnZ2Pnzp1Rz//www9tG8tf/vIXCCHqvz948CDWr1+P8ePHRz1v6NChGD16NJ555hn8+c9/xr333sumNSKPUBQFQghkZWVFPb5o0SLU1tY2ev7cuXOxZs0a/PnPf8aSJUuwY8cOzor4BGdEqN4zzzyDoUOH4tixY+jbty+AcFjcdddd+NOf/oSuXbti4MCB2Lx5M95++23bxnHs2DHceuutmDp1KkKhEObOnYvs7GzMnj270XOnTZuGSZMmQVEUPPjgg7aNiYisoygK8vPzcdVVV+HZZ59FYWEhOnfujFWrVmHx4sW47LLLop6/fPlyzJs3D//2b/+G7373uwCAefPmYcaMGRg/fjxuvfVWCb8FWYUzIlRv8ODBuOOOOxo9/vzzz+Ouu+7C/Pnz8cMf/hAbNmzAxx9/bNs4/vM//xOdOnXCv/zLv+AnP/kJ2rVrh5UrV6Jr166NnnvLLbcgKysLEyZMQPfu3W0bExGl7vz58wBQPwvy9ttv4+qrr8bjjz+O2267DVu2bMHy5ctRUFBQ/5rDhw/jrrvuwvjx4/HrX/+6/vHp06fjBz/4AX7yk5+gtLTU0d+DrKWIyDlwIok+/fRTXH311Xj33Xfxox/9yNBrPvroI9x88834n//5H9x44402j5CIUjFt2jS89NJLOH36NPLy8mQPh1yCSzPkSXv27MHBgwfxi1/8AoMGDYq6NJiI3GXr1q0oLi7Gn/70J9x8880sQigKCxHypAcffBDr1q3DkCFD8MYbb0TtO0JE7vKjH/0IoVAIN998M37/+9/LHg65DJdmiIiISBrHmlXnzZsHRVHwyCOPOHVIIvIBZgeRvzlSiBQXF2PhwoUYMGCAE4cjIp9gdhD5n+2FyNmzZ3HnnXfi1VdfRfPmze0+HBH5BLODKD3Y3qz60EMP4aabbsK1116Lp556Ku5zq6uro7YSr6urw8mTJ9GyZUs2IxJJIoTAmTNn0L59ewQCzm09xOwg8i4zuWFrIfLOO+9g27ZtKC4uNvT8efPm4Te/+Y2dQyKiJJWVlaGoqMiRYzE7iPzBSG7YVoiUlZVh2rRp+OSTTxrddVXP7NmzMX369PrvQ6EQOnbsiLKyMuTn59s1VIqwbNky3H777ejRo4fh/wmQv1VVVaFDhw6O7f3A7HC306dPo1OnTgCAL7/8Em3atJE8InIjM7lh2+W7H3zwAW699VYEg8H6x2pra6EoCgKBAKqrq6N+pqWqqgoFBQUIhUIME4ccOXIE7dq1g6IoqKqqQm5uruwhkWROn4fMDvcbNGgQduzYgaVLl2LixImyh0MuZOYctG3B97vf/S527dqFkpKS+q9hw4bhzjvvRElJScIgITnatm2L9u3bQwiBkpIS2cOhNMTscL+rrroKALBmzRrJIyE/sG1pJi8vD/369Yt6rFmzZmjZsmWjx8ldhg4dioqKCmzduhVjx46VPRxKM8wO9xs3bhxefPFFrF69WvZQyAd4911qZOjQoQDC94cgIoo1btw4AMDOnTtx+vRpuYMhz3P0XjOffvqpk4ejJLEQIbdhdrhL27Zt0b17d3z11VdYt24dbrrpJtlDIg/jjAg1ohYiX3zxBc6dOyd5NETkRuwTIauwEKFG2rVrh3bt2qGuro4Nq0SkSV2eYZ8IpYqFCGni8gwRxaPOiGzZsgXnz5+XPBryMhYipImFCBHF07lzZ1x++eW4ePEiNm3aJHs45GEsREgTCxEiikdRFPaJkCVYiJAmtRD5/PPP2bBKRJrYJ0JWYCFCmtq3b4+2bduirq4OO3bskD0cInIhdUZkw4YNuHjxouTRkFexECFdXJ4honh69+6NFi1a4Pz589i2bZvs4ZBHsRAhXSxEiCieQCDA5RlKGQsR0sVChIgSUQsRNqxSsliIkC61ENmzZw/3CSAiTWqfyNq1a1FXVyd5NORFLERIV/v27dGmTRs2rBKRrsGDB6NZs2Y4deoUdu/eLXs45EEsREiXoihcniGiuDIyMjB69GgA7BOh5LAQobhYiBBRItzYjFLBQoTiYiFCRIlEXjkjhJA8GvIaFiIUV2TD6rfffit5NETkRldeeSUyMzNx+PBh7N+/X/ZwyGNYiFBcl19+OVq3bo3a2lo2rBKRppycHAwfPhwA+0TIPBYiFBcbVonICPaJULJYiFBCLESIKBHusErJYiFCCbEQIaJERo8ejUAggH379qGiokL2cMhDWIhQQmohsnv3bjasEpGmgoICDBw4EACXZ8gcFiKUUFFREVq1aoXa2lrs3LlT9nCIyKXUPhEuz5AZLEQoITasEpERvAEeJYOFCBnCQoSIElELkV27duHkyZOSR0NewUKEDGEhQkSJtG7dGj179gQArFu3TvJoyCtYiJAhkQ2rFy5ckDwaInIr9omQWSxEyJAOHTqgsLAQly5dwq5du2QPh4hcin0iZBYLETKEDatEZIQ6I7J161acO3dO8mjIC1iIkGEsRIgokU6dOqFjx464dOkSNm7cKHs45AEsRMgwFiJEZAS3eyczWIiQYWoh8tlnn6G6ulryaIjIrXgDPDKDhQgZ1rFjR7Ro0QIXL15kwyoR6VJnRDZs2ICamhrJoyG3YyFChrFhlYiM6NWrFwoLC3HhwgVmBSXEQoRMYSFCRIkoisI+ETKMhQiZwkKEiIzgxmZklK2FyMsvv4wBAwYgPz8f+fn5GDVqFP7+97/beUiymVqI7Nq1iw2rZAvmhj+oMyLr1q1DbW2t5NGQm9laiBQVFeHpp5/Gli1bsGXLFlxzzTX44Q9/iN27d9t5WLJR586d0bx5c1y8eBGfffaZ7OGQDzE3/GHgwIHIy8tDKBRiczvFZWsh8oMf/AA33ngjevTogR49euA//uM/kJuby01uPIwNq2Q35oY/ZGRkYPTo0QB4GS/F51iPSG1tLd555x2cO3cOo0aN0nxOdXU1qqqqor7IfViIkFOM5AbA7HAr9omQEbYXIrt27UJubi6ysrLwwAMP4P3330efPn00nztv3jwUFBTUf3Xo0MHu4VESWIiQ3czkBsDscKvIG+AJISSPhtxKETb/7aipqcGhQ4dw+vRp/PWvf8WiRYuwatUqzVCprq6OaoCsqqpChw4dEAqFkJ+fb+cwyYT9+/eja9euyMzMxJkzZ5CZmSl7SGSjqqoqFBQUOHoemskNgNnhVhcuXMBll12G6upq7N27Fz169JA9JHKImdywfUYkMzMT3bp1w7BhwzBv3jwMHDgQL7zwguZzs7Ky6jvl1S9yny5duqB58+aoqalhwyrZwkxuAMwOt8rOzsaVV14JgH0ipM/xfUSEELzs0+MURcGQIUMAcHmGnMHc8C72iVAithYiv/zlL7FmzRqUlpZi165dmDNnDj799FPceeeddh6WHMA+EbILc8NfIvtEiLRk2PnmR48exd13343Dhw+joKAAAwYMwP/+7//ie9/7np2HJQewECG7MDf8ZfTo0QgEAjhw4ADKy8tRVFQke0jkMrYWIosXL7bz7UkitRDZuXMnampq2LBKlmFu+EteXh4GDx6MrVu3Ys2aNbjjjjtkD4lchveaoaRcccUVuOyyy1BTU8MdL4koLvaJUDwsRCgpbFglIqNYiFA8LEQoaewTISIjxo4dCwDYs2cPKisrJY+G3IaFCCWNhQgRGVFYWFi/Gd3atWslj4bchoUIJS2yYfXixYuSR0NEbsbLeEkPCxFKWteuXVFQUIDq6mo2rBJRXOwTIT0sRChpbFglIqPUGZHt27fjzJkzkkdDbsJChFLCPhEiMqJDhw7o3LkzamtrsWHDBtnDIRdhIUIpYSFCREaxT4S0sBChlKiFyI4dO9iwSkRxsU+EtLAQoZR07doV+fn5qK6uxp49e2QPh4hcTJ0R2bRpE++mTPVYiFBKAoEAG1aJyJAePXqgdevWqK6uRnFxsezhkEuwEKGUsU+EiIxQFIV9ItQICxFKGQsRIjKKfSIUi4UIpSyyYfXSpUuSR0NEbqbOiKxbtw61tbWSR0NuwEKEUtatWzfk5eXhwoUL+Pzzz2UPh4hcbMCAAcjPz8eZM2ewY8cO2cMhF2AhQiljwyoRGRUMBjFmzBgAXJ6hMBYiZAn2iRCRUWqfCBtWCWAhQhZhIUJERkUWIkIIyaMh2ViIkCXUQqSkpIQNqyaUlwMrV4b/SZQuhg0bhuzsbBw/fhx79+6VPRxP8lN2sBAhS3Tv3h15eXn49ttv8cUXX8gejicsXgx06gRcc034n4sXyx4RkTMyMzMxcuRIAOwTSYbfsoOFCFkiEAhg8ODBALg8Y0R5OfDTnwJ1deHv6+qA++/3x6cbIiO4sVly/JgdLETIMuwTMe6FFxqCRFVbC3z9tZzxEDmNG5slx4/ZwUKELMNLeI0pLweef77x48Eg0K2b8+MhkmHkyJEIBoM4dOgQDh48KHs4nuDX7GAhQpaJbFjljon6vvoK0LpQ4NFHgaIi58dDJENubm59ZnB5xhi/ZgcLEbJMjx490KxZM5w/f54Nq3F07w4EYs68QACYNk3OeIhkYZ+IOX7NDhYiZJlgMMiGVQOKioCFC8PTqUA4SJ5+2tufaIiSwT4Rc9TsUIuRQCD8vdezg4UIWcquhlU/XTMPAFOmhIuPQCDceDZrlvcvwSMyS93q/YsvvsCxY8dsOYbfssOPWIiQpewoRPx2zTwQDsWZM/11CR6RWS1btkS/fv0AAGvXrrX8/f2WHX68dBdgIUIWUwuR7du3W9Kw6tcT76uv/HcJHlEy1D4Rq5dn/Jgdfs0NFiJkqZ49e9Y3rFqxdbOVJ56bpmi1ms68fgkeUTLsugGeH7PDr7nBQoQsFQwGMWjQIADWLM9YdeK5bYo2tmE1GAReecX7TWdEZqkzIiUlJaiqqrLsff2YHX7NDRYiZDkr+0SsOPGcmqI1+6lpyhSgtDT8mtLS8PdE6ebyyy/HFVdcgbq6Oqxfv96y9/VrdvgxN1iIkOWsblhN9cRzYl012U9NRUXA+PHe/0RDlAq7+kT8mh1+yw0WImQ5qxtWgdROPLvXVf3YFEfkJLv6RABmhxfYWojMmzcPw4cPR15eHlq3bo1bbrnFkgZGcrdevXqhadOmOHfuHL788kvZw0lpitbIlKnZT01uaXxzK+ZG+lFnRDZv3owLFy5IHk0DN2WHn3PD1kJk1apVeOihh7Bx40YsX74cly5dwnXXXYdz587ZeViSzOqGVSskM0VrdMrUzKcmNzW+uRVzI/1069YNbdu2RU1NDTZv3ix7OFHckB2+zw3hoGPHjgkAYtWqVYaeHwqFBAARCoVsHhlZ7ec//7kAIB555BHZQ0lKWZkQgYAQ4VtMhb+CwfDjWhYtCv9cfd6iRam/p1vIPg/N5oYQ8sdM5t1+++0CgPj3f/932UNJidXZkQ654WiPSCgUAgC0aNFC8+fV1dWoqqqK+iJvsmurd6eYXW4x8qnJr5sR2S1RbgDMDj/wyw3wrM6OdMgNxwoRIQSmT5+OsWPH1m/pG2vevHkoKCio/+rQoYNTwyOLRTas1sWeRS5XXg4cP26+SS1RU5xfNyOyk5HcAJgdfqA2rK5btw6XLl2SPJrk2JEd6ZAbjhUiDz/8MHbu3Im//OUvus+ZPXs2QqFQ/VdZWZlTwyOL9erVCzk5OTh79qwrGlaNUtdiJ00KT4KqAZDsHgSRzWV+3YzITkZyA2B2+EG/fv1w2WWX4dy5c9i+fbvs4ZhmVXakZW7YvlAkhHj44YdFUVGR2L9/v6nXcZ3X20aNGiUAiLfeekv2UAzRWosNBIRYutT8euyiRQ3vFQhEr/uWlQmxcqX713hVss7DZHNDCGaHV33/+98XAMTzzz8veyimWJUd6Zobts6ICCHw8MMP47333sOKFSvQpUsXOw9HLuO1PhGttdi6OqBVq9R3Y/zpT6M/4fhpMyKrMTfSl10bm9nNiuzQy43i4vD3fs4NWwuRhx56CG+99Rbefvtt5OXl4ciRIzhy5Ai+/fZbOw9LLuG1QsSqtVi9UHrhhdTGly6YG+lL7RNZu3atp3rLrMgOvdwYOdKHl+vGsnNqBoDm12uvvWbo9Zxe9badO3cKACIvL0/U1tbKHo4hRi7DTaSsLHqK1kuX3Glx+jxMNTeEYHZ4VXV1tcjJyREAxGeffSZ7OKakmh1lZUIoin+yw8w5mGFzkWPn25PL9e7dGzk5OThz5gy++uor9OzZU/aQEpoyBZgwIXxpXLdu4WnQ8vLwp5Xu3fWnRSOfs2yZ9nPUS+78OLVqJeZG+srMzMSoUaOwYsUKrFmzBn379pU9JMNSzY54E8d+zw7ea4Zsk5GRgYEDBwLwzvIMEL0Wa2RHw8jndOwITJ2q/b5+u+SOyA5e7RMBUsuOxx4Lz39o8Xt2sBAhW6l9Itu2bZM8EvOM3JAq9jnqZGqsQMCHl9wR2UDtE1m9erVnZ8eSyQ49vrxcNwYLEbKV7IbVVG4UZWRHQ63nxAoEgI0bzd+CnCgdjRw5EhkZGfjmm29QWloqbRyysyMQAJYuNX5/Gy9jIUK2ipwRcboLPtUbRRnphNd6jqJEbz60cCEwfLj58ROlo6ZNm2LYsGEA5G33Lis7Ip+7cCEwcaK/Z0JULETIVn369EF2djaqqqqwb98+x45rZGo0ESM7Gmo959VXzd+tk4gayOwTkZkd8+enZ27YetUMkdqwumnTJmzduhXdu3d35LjxpkbNfMLQ6oQ3+px0+CRDZIerrroKzz77rJQZETdkR7rhjAjZTkafiJU3ioq3o6G6jgz4d9dDIqeNGTMGiqLgyy+/xJEjRxw9thPZEdl/4ucdU41iIUK2k1GIOHGjqFTXkYlIW/PmzdG/f38AzveJ2J0dzI3GWIiQ7SIbVp28HG/KFPt6NRKtI6fScU9EDZfxyliesSs7jF7Wm27ZwUKEbNenTx9kZWUhFAo52rAK2DftGW8dmZ94iFIne2MzO7Ij0WW96ZodLETIdk2aNMGAAQMAeGuH1Xj01pGbNUu9456IGgqRnTt34vTp03IHY5F4/SdWXK3jVSxEyBGyNzazmt468tmziTcyIqLE2rVrh27dukEIgXXr1skejiXi9Z8Y2QTNr1iIkCP8VogA2uvIVnbcE6U7mX0idtHrP0nn7GAhQo6Q1bBqhtokVlxsvFksdh3Ziat1iNKF7D4Ro8xmh1b/STpnBzc0I0f07dsXmZmZOH36NPbv34+uXbs6clwjt+EGwk1hsTegCgTCwaB+Ykn0XurPJ0wIf9IxskmR0fERpSN1RmTLli04f/48mjZt6tixncqOyJ+Z2eDMV9khXCwUCgkAIhQKyR4KWWDYsGECgFiyZIkjx1u0SIhAIHw/3EAg/L2WsrKG58V+BYPhnyd6LzPHWrHC2Hu6hRfPQy+OmRqrq6sT7du3FwDEihUrHDuuU9mRTG6YeZ1MZs5BFiLkmPvvv18AEI8//rjtx9IKCDUYYi1Zoh0k6tfSpfHfy+ixYsNDUYyNTzYvnodeHDNpmzx5sgAgfvOb3zhyPKPnc1mZEM8/n3x2JJsbzz5rPNtkMnMOskeEHONkw6rRDvTFi4HJk/XfJxgMn+pa7/Xuuw3To4mOpXVpXmyrTLp0yBOZ4XSfiJHzWd3v4xe/0H+feNmxYUPyuTFzpv+urmEhQo5xsmE1UQd6eTmwdGn4JNcbitosNnq09u26p08Ph9GWLYm73bVCR+t46dAhT2SG2ieyYcMGXLx40fbjJcqO4uLGPSGxEmXH5MnJ50ZdHaAo8V/nNSxEyDH9+vVDZmYmTp06hQMHDth6rHgd6OqnmUmTtMPk5ZejL62Lfa9IdXXA7NnAM8/od7uXlwPHjzcOndgwuesuHzSdEVmsT58+aN68Oc6fP49t27bZfrxE2TFihHZuLFgAbN6snR2x534quREMAv/0T9GPeT477F8pSh7Xef1n6NChAoBYunSpI8crKxNi5cr4/RxG11rLyoT47W/114KXLAn/M/L1keu7itLw78Ege0Ts5MUxk76bb75ZABDz58937JhmsiPRuavXh5ZMbrBHhChFTm9sFnu9frwlEkUBHn00/ntNnKg9szF5cniGZfJkYNmy8OOx67vqEtDSpcDbb7NHhMgoGRubmcmOeLkBaC/RJJMbpaXA0KHsESFKiewdVrXWf1VCAM89F/9mU7HTtup7ad0fQm99t1Ur7WDy+jovkV3UhtW1a9eiLlGzlU3iZYdTuVFU5M8dWFmIkKMiCxEhYYfVeP0eKr2bTam7J6oblq1cCfzlL/ozG/ECI513USQya/DgwWjWrBlOnTqF3bt3SxmDXr+HKt5N6srLgSuuCF8tk0puRI7DT9nBQoQc1a9fPzRp0gSnTp1CaWmplDGo93r47W/1nxN5eS7Q+Pbcy5aFp23jzWwkCgy9e04QUbQmTZpg1KhRAORu9z5lSriI0BObG0B0dowcCezbl1puqOPwVXbY37KSPDac+dOQIUMEAPHuu+9KHUeixlWjGwgtWhT+Xn08dpfD2KY3r/HieejFMVN8Tz75pAAgJk2aJHUcRnNj0aL4m5YxNxpwRoQcJ7tPRGV0mSbRBkKJPp1o3eCKiMyJ3NhMSLxxppnl3fXr9bODudGAhQg5LnJjMzPUHg0jd8U1KjIMNm/WXq4xsoFQOoUGkQwjRoxAkyZNcPjwYezfv9/Ua63ODiO5UVsbzo1E/R7MDRYiJEEyDauxPRp63enJUMNg+HDty3ODwfgbDxGR/XJycjB8+HAA5vpE7MoOI7kxapT/GkvtwEKEHNe/f380adIEJ06cwKFDhxI+X+t+C3rd6ZGvSeYTkFaj2Lx5wLBhDR3vvmgOI/Igs/uJOJUdeg2mQPTVMswObSxEyHFZWVno168fAGN9IkZvYKdK9RNQ5LTr008Ds2ZFd7zz0wyRHGZvgOdkdsT2fACNr5ZhdmhjIUJSmGlYNbOBTzKfgNTXRX4KKioKv39ko2qy70VE1hgzZgwURcG+fftQUVGR8PlOZ4e6XAM4N4vrByxESAozhYiZDXzMfgICGn8Keu45694r2fXodA4lIj0FBQUYOHAgAGPLM17JjrTPDdsvJk4B9wLwr82bNwsAorCwUNTV1Rl6jZHr6uNdt2/0+YAQ8+db817J3Iwq8oZX6n4EMnnxPPTimMmYf/3XfxUAxIMPPmj4NW7ODuYG9xEhSfr374+MjAxUVlairKzM0GuMXOpmdvtjvRtZzZoV/ufdd0c/rnW7bfVTSLw9A4xKdnqYKF0kcwM8N2fH0qXMDVsLkdWrV+MHP/gB2rdvD0VR8MEHH9h5OPKQ7Oxs9O3bF4D1G5uZ2f5Y70ZWdXXhTvc334x+/K239LdvvuOOxPuNJJLM9LAfMTtIz9ixYwEAu3btwsmTJy19bxnZ8YtfNH6PdMsNWwuRc+fOYeDAgXjppZfsPAx5lJ07rBrdKKioKLxHSKxgMDxJGu/k1voUErmBUTCY+Pbgsfx4Z81kMDtIT5s2bdCzZ08AwLp16yx/fxnZEfv6p58OFxdGZzS8nhu2FiI33HADnnrqKdx22212HoY8yi1bvc+YAfzoR9GP3XVX/BtTAcALL2jfrvudd8LvWVenfXvweA1lWrcLnzcv/S77Y3ZQPGYv47VL8+bR3ytKeDknmewAgAULwuf7zJnajat62eH53HCgZ0UIIQQA8f7778d9zoULF0QoFKr/KisrY8OZj23cuFEAEK1atTLcsGqHjz5q3HCW6MZUZWXaN7sKBoXYvDn+ja7UnymKEDNmaDelRd5oT3bjmezGT2YHxfo//+f/CABixIgR0sYQLzeEkJMdXs0NVxUic+fOFQAafTFM/On8+fMiGAwKAOLQoUNSxnDPPfp30Fy5MvwcrY77JUu0X/PAA0KsWKH9s6VLtbvsYwPDqi56q3ihEGF2pJcDBw4IACIjI0OcPXvW8eMbyQ0hnM0OL+eGq66amT17NkKhUP2X0aspyJtycnIsbVg1ew19cTHwxhvaP0v2xlTXXKO9XhsIaK8bA4073L3eeCYDsyO9dOrUCR06dMClS5ewcePGlN7LrtwArMkOvZ4TIDo7vJwbripEsrKykJ+fH/VF/mZVn4iRDYFiA0fv6j91nTdeeIwe3fgKmUAgfJMrdb028udCADt36r9fZGB4vfFMBmZHelEUxZI+EadzA0icHVqX/Xbpov9+anZ4OTdcVYhQ+rGiEDFyDb1W4Pwjxxr57/9OfGOqoiJg/vzoK2QWLmwIoQkTGhci8+bpv18wCDRrFg48gHfsJEokmf1EIsnIDSB+dpSXa1/2e+CA/vsFAsCxY+F/92pu2FqInD17FiUlJSgpKQEAHDhwACUlJYbuuErpIbIQEUIk9R6JpiT1AqddO+Cee6Jfd889wPe/n/iYixc33IdGUcJFRmQIaY1JfW6sQCD8qWfkyIbAA4zvZ+BHzA5KRJ0R2bBhA2pqaky/XkZuAPGzQ29MkdsCxBICmDTJ47lhZ7PKypUrNRvI7rnnHkOvl90kR/aLbFgtS7KrKlGTll4DmNpUtnmzEAsWhP9pxfHiPWf+/IZO+kAg3Pker1PeDWSch8wOSqSurk4UFhYKAGL9+vWmX+90bhg5ZryfR14REwyGm1v9khu2zoiMHz8eInxlTtTX66+/budhyUNycnLQp08fAMkvzyTamjnR2unw4cAjj4T/aYTep5Z3342+e6/WmFq0aHitEECvXsDZs95tMrMLs4MSURSlfpfVZPpEnM4NQD87NmyIP6ZlyxrPotx+u39ygz0iJJ0VfSLxtmY2ew+JRPS2dp4+PbrhLXZMEyaEp3rVFSghwlO9ubnebTIjkinVPhEncwPQz47JkxPnRuQHmNmz/ZUbLERIOquunIl3qZyZe0gYOU5kQEWKbXhTxwTo39zq3DnvNpkRyaQWImvXrkVtbW1S7+FUbqjHWriwcQGhlxtFRfo30/RTbmTIHgBRbMOqotXRaQH1BP3qq+jvI6nX43fvHv+EnjIl/Enl3XfDMyGR1OlR9fWLF8e/r0S3buHQmTAh/Lpu3bwZJkROGzhwIHJzcxEKhbBr1y4MGjTI8mNYmRtAODvy8sINppFicwMIZ8fUqY3fw2+5wRkRkm7gwIEIBAI4evQoKioqbDtOoj0DjOwpEKmoCJg4Mf70aLybW8XeD8LM5kdEBGRkZGDMmDEAkl+eScTq3AAS34sGaMiO2IsJ/ZgbLERIuqZNm6bcsJpIoj0DjOwpoCXROrJWc5qqrg6YNctYcBGRNjtvgCcrNwD97PBjbrAQIVew+068ifYM0Lobpvrz2J0VY7+Pt46s15ymMhpcRKQtsmFVJLkXkZ5UcgOIzgozuQGEs0NvldpvucFChFxBLUS2bdtm6nVG7xMR71K88nLg+ecbvyYYBLZsiZ52vfde7WlYvelRvea0SF695I7IDYYPH47MzEwcPXoUX6mNHAnYnRvdukUv2XTsGP4ymhvqz37xC/2x+Sk3WIiQKyQzI2JmbTbeVOhXXzVehwWA++5ruHYfCP/zjTfiT9NqhduUKcCHH+qPzauX3BG5QXZ2NkaMGAHAWJ+I3bnx6KPhf8Zecqs+V2s2Qy87pk3T/xDjp9xgIUKuMGjQIAQCARw+fBiHDx9O+Pxk1ma1pkLLy4Hjx7VvQnXNNfr9HSr1U0micGvWTPv1gYB3L7kjcgujfSJW5QYQ3sdDKzemTYvfGwZEz2bEy46iosZX5akefdQ/ucFChFyhadOm6N27NwBjsyLJ3vI6cipUDQD1Mjo1VNSbUGl1tscKBIDPP08cbnrrvRs3euh+EEQuZXRjMytyAwhnx8iR0TMikTevS9Qbpt6orrg4cXZMm6b9HnqPexELEXINM8szqd7yOvaTkRDh93v5ZeDtt8PX5mtNy95zT8P3ihJ+3YMPJhdugUD4BlpElJpRo0YhEAjgwIEDKI8zvZFqbgDal+QHAsCLL4ZzA2icHYFAwwcRNTcmTQJGjDCWHVozL37is1+HvMxMIZLq9st6n4wefLDhTpaLFzeeln399fB9IX796/Br9Jr0Y8NNaz25rs4/zWZEMuXn52Pw4MEA4s+KWLFtu96dtR98UP8WDwcPAps2Nc4NrfxIx+zgzqrkGokKkdjdC9XdTZPZVVD9ZBQbKLENZerMiJFdUlXBYHjDocidGLWO56dmMyLZxo0bh61bt2L16tW44447on4WmR2p5Aagnx2Adm6oy8CJcgNI4+yw6Q7AluCtvNPL2bNnRSAQEADE4cOHo362aFHDLa8DgfD3qVq0KHzbbPU9493yWwjtW3THfgUCQsyZoz3We+6Jfq7BO9pL58Xz0ItjptS89957AoDo27dv1ON2Z4cVueHH7DBzDipCWLwDjIWqqqpQUFCAUCiE/Px82cMhB/Tt2xd79uzBxx9/jJtuuglA+NNMp06NPxGUlqbeNV5eHv5k1KxZuPks3jFWrgx3tiei9ellw4bE7+9WXjwPvThmSs3x48fRunXr+n8vLCy0PTs2bAgv5cY2rSaTG4C/ssPMOcgeEXKVIUOGAIhenkmm093ohkVqN/zw4YnXjhN1wgPaU7a1tcDatck1tBKRMa1ataq/8m7t2rUA7M+OiROBV181nxuRzauRj6VrdrAQIVfR6hMx2+kee13+c88ZO3aiLZeLioBnnmkYS+xVNMEg8PTT2mMdOzb1bn0iii/2Mt5Us8PI/VyM5Ebsh5yFCxsXMGmdHbYvFKWA67zpZ/Xq1QKAuPzyy6Mej1yTDQb113n11mPnz099bJFrzYrS8J5lZUIsXSrEkiXhf9cbq9HfwW28eB56ccyUurfeeksAEMOGDat/zEx2KErjvo2ystTHtWhRw3srSsMY/Jwd7BEhzzp79izy8/MhhMCRI0fQpk2b+p+p/RzxOt311mMDgfAldGbXVNVu+9xc/XXad95p2Ao+EAh/2tHryjfyO7iNF89DL46ZUldWVoaOHTsiGAzi1KlTyMvLA2DsvFu6tGFzw9jHJ040P5Z0zw4z5yAv3yVXyc3NRc+ePfHFF19g69atuPHGG+t/FnkZrR69S+vU6+7NnMCRl9zprd8+9VR4TTjyOPffHw6Z8eMbv6eR34GIktOhQwd06tQJBw8exIYNG3DdddcBcP68Y3aYwx4RtzLaMWX2uR6QzA3wVGofR6xUd0/Uuv5f/QQTy2+NZOQhZrPAZ9kBGN/uPdbo0Y0bSBUFGDXK3PGZHeaxEHEjMx1TyXRXuVwqhQgAzJgBzJ8f3VRqxe6JQPR7Tp2qvTNiIOCzRjLyBrNZ4MPsAIzfAC9WUVG4gVQ9xwOB8PdmZyGYHeaxR8RtzFz4budF8hKtXr0a3/nOd1BUVISysrKk3yeVNVW9P9oNG4Bz54AtWxrWdmM9+2y4GPILL56HXhxzSsxmgU+zAwD27t2LXr16ISsrC6FQCFlZWaZen2ovBrMjjPuIeJmZC9+TvZWkyw0ePBiKoqC8vBzHjh1L+n1i75hp9rVa+4oMHx4OKK0gUZTwTIyRIPHhjDjJZDYLfJodANCjRw+0bt0a1dXVKC4uNv36VHJDfT2zwxwWIm6Tm6v9eLNmjR+z4laSLpSXl4cePXoASH55xgp6+wPoTb0uWQI89lji9/XpjDjJZCY3AN9mBwAoilK/PGO2T8QqzA5zWIi4zdmz2o+fO9f4MStuJelSqfaJWEXr05Fehsc2tWl9ctFqZLv/fn99uiEJzOQG4OvsAJLvE7FSstmhN+Ph5+xgIeI2Zj+pJNrWz6PcUoioIsNBzfDIprbYDNf75OLjGXGSKZkZDp9mB9Bw5cy6detQW1srbRyxRUWi7Ig34+Hn7GAh4jbJfFJJdVHThdxUiJidDi0vD3fFa31y8fGMOMmU7AyHD7MDAAYMGID8/HycOXMGO3bskDKGZHIj3oyHr7PD3k1eU5PW2zSXlYXvJW3F/sIepP63ByCOHTsmbRxaW8YHAo0fCwYb/lPNmBH/1uBe267Zi+ehF8dsiTTPjUg33HCDACAWLFjg+LG1ciMYFGLzZv3sWLEifm4I4a3sMHMOckbEDbQWBX36ScWo/Px8VzSsak2H1tXpT5GWlwO//W3j94ncH8DHM+LkJOZGXMlubGYFvWWUeHfSNdJv7NfsYCEim1/boC3ghuWZ3FztW3jrTZHqdcVPnx79/wb+/4JSwtxIKPLKGeHwdll6yyjx7qRrtN/Yj9nBQkQmP7dBW0AtRLZt2ybl+IsXa9+sauFC/eV4vQCaNs25cZPPMTcMGTZsGLKzs3H8+HHs3bvX0WMvWxa9c6ralDp8uPns8EUPSAIsRGTycxu0BWTOiMRmPRAOiQ0bwtOhsVOkEyaE/x0w1jPox02JyCHMDUOysrIwYsQIAM5exqtmR+wkzIQJ4X9GZseGDcAVV0RfjZeO2cFCRKZ0LoENGDx4MADg4MGDOHHihKPH1usNiZwmVadIly2LniUH4q/jcladUsLcMExGn4hedkTWiUVFwL594RnXyBxI1APi1+xgISKTzzcVSlVBQQG6d+8OwPlZEaNZrzdLDmiv4zo9q+7HT09pj7lhmIyNzYxkR7wc0OsB8XN2sBCRza9t0BaRtTxjNOvdfIsPv356IjA3DBo1ahSCwSAOHTqEgwcPOnJMI9mRTA74OTscKUT++Mc/okuXLsjOzsbQoUOl7f/vWn5sg7aIzD6RKVPCa7i//W1Db0gss7PkTs2q+6GfkbmRAHMjodzcXAwZMgSAs8szibIjmRzwc3bYXogsWbIEjzzyCObMmYPt27dj3LhxuOGGG3Do0CG7D+1dnE+vJ7MQUa+amT49/E+tTwVmZ8mdmlX3ej8jcyNJzI5GZPSJJMqOZDfQ9m122L272pVXXikeeOCBqMd69eolZs2a1ei5Fy5cEKFQqP6rrKws/XZHXLSoYeu9QMD5rfPULf5csjPj6dOn63dYraysdOy4ejsj6v2xmN3Q0u4NMM2OPx4Zu5SayQ0hmB1CCGaHjg8++EAAEL169XLkeGbOvWRywCvZYSY3bC1EqqurRTAYFO+9917U4//6r/8qrrrqqkbPnzt3bv3/dCK/0iZMrPy/RzJkB5mObt26CQDik08+ceyYRrZbdjurtoN2uhAxmxtCMDuYHfoqKyvr/z4cPXrU9uMxO8Jcs8V7ZWUlamtr0aZNm6jH27RpgyNHjjR6/uzZsxEKheq/ysrK7Bye+8icT3dxU4G6xuvk8owfrpD0aj+j2dwAmB3MDn0tW7ZE3759AQBr1661/XjMDvMcaVZVFCXqeyFEo8eA8AY0+fn5UV+eYcXarMy/wS5uKpDRJ+KXKyS93M9oNDcAZgezIz61T8SJy3iZHebZWogUFhYiGAw2+hRz7NixRp92PM2qa51k/g12cRkvq2FVbwdEslfa5AbA7HCI0w2rzA5zbC1EMjMzMXToUCxfvjzq8eXLl2P06NF2Hto5Vk9LyppPd3EZry7NHDhwACdPnnT02Ho7IJJ90iI3AGaHg9SNzUpKSlBVVeXIMZkdJphvQTHnnXfeEU2aNBGLFy8We/bsEY888oho1qyZKC0tTfhaGd36pvmhMymS3S3ZSbriiisEALF8+XJHjyu7B9ANZJyHqeSGEMwOKVyaHSo1Q/7+9787crx0zw4z52CG3YXOpEmTcOLECTz55JM4fPgw+vXrh7/97W/opN6Uw+vUacnYW7Rq7QX+1Vfh57vok0IjRUWuHN/QoUOxf/9+bN26Fddee61jx423/O3CPybf8H1uAMaywyu5Abg2O1Tjxo3D/v37sXr1alx//fW2H4/ZYZwjzaoPPvggSktLUV1dja1bt9av1/mCkWnJ2HXgZ5811pwma3MiF26KJKtPxAPL377l69wAEmeH13JD9rETcLpPhNlhggMzNEnzxPSqSm9aUmt+Tv2Kd729rOvyXbofwPLlywUAccUVVzh+bKv24/AqT52H/+CpMWtlh9dyQ/axDfjyyy8FAJGZmSm+/fZbR46Zztlh5hxUhBBCZiEUT1VVFQoKChAKhbx1OV6klSvDn2j0BIPhxrLIGZTy8vAnoNgp29jnWU3WcQ04efIkWrZsWf/vzZs3d/T45eXhKdVu3aT/UTjOi+ehF8ccxUu5IfvYBgkh0L59exw5cgSrVq1ybIYtXbPDzDnIu+/aTWt+LpLW9fayrst38X4ALVq0QJcuXQAA27Ztc/z4Xt6PgzzIS7kh+9gGKYpSf/WME/uJqJgdibEQsVvsOnAsrUXDRIuLdq3DunxRU+YN8IgcZUduAGmbHSoZN8CjxFiIOGHKFGDWLCB2V0i96+2XLQuvBqsCgYbnWbUBkhaX7wfAQoTSitncKCoC7r47+rG77tJvfk2j7FCpMyLr16/HpUuXJI+GVOwRccJzzwGPPRb9WCAAbNwIDB8e/bjWWmsgABw8GP53J9ZhXbqouXz5clx33XXo2rUrvnbRlK/fefE89OKYGzGTG0D8Pg0grbNDVVtbi8LCQpw+fRrFxcUYNmyY7CH5FntE3KS8HJg5s/HjdXXAuXONH9daa62rC5/ceuuw775r7VSrSxc11R1W9+3bh9OnT8sdDJGdzOYGEL9PI82zQxUMBjFmzBgAzvaJUHwsROymFQBA+JON1vppvLVWvQa26dPTYv/gli1bonPnzgDkNKwSOcZsbgDMDoOcvAEeGcNCxG56AfD009qfGrTWWh99VPtnkVx26227sE+E0oLZ3ACYHQaphcjatWtRp1XskeNYiNgtNgACgfAOibFrv5HUm1fNmBEOieeea/jUov7st79t/DqXXS5nBxYilBaSyQ2A2WHAkCFDkJOTgxMnTuDzzz+XPRwCCxHrxLssLvKumAcPhkPCiN/+tuHqmchPLUVFwMSJnrhczmpqIcKlGfINvexINjcAZkccmZmZGDVqFABexusWLESsoHdZXGTAmG3iSrRBkEcul7OaWoh89dVXCIVCkkdDlCKt7EglNwBmhwEyNjYjfSxEUlVeDvz0pw0nvvrpQ50STfaafSMbBEV+YiotDX/vcy1btqy/A+v27dslj4YoBVrZMXVq6nt9MDsSimxYdfEOFmmDhUiq9D59zJzZuDgx2gym3vr7mWcSf2px+eVydmCfCPmCVnYIkXxuAMwOg0aOHImMjAx88803KFX3WSFpMmQPwPPUTx+xG5DpTY0mOukXL274lBQIhLvkhw83v0GQGkjdu/suaIYOHYr33nvPkULEx3+MJJtWdsQymhsAs8OEpk2bYtiwYdi4cSPWrFlTfx8rK6XBH6NlOCOSKq311qlTG2/LHO/6f5XWVO3s2eaDxM6tnF3AqRkRn/8xkmxaV8bE5gYAFBcnfi9mh2l29omk0R+jNYSLhUIhAUCEQiHZQ0msrEyIGTOEUBQhwhOs0V+KEn5OPCtWaL925Upz4wgEol8fDCY+toccP35cALD174bdf4xlZeH/3F74z+Kp8/AfPDXmRNlh5C8es8O0jz76SAAQPXr0sPR97fxj9GtucEbESpGXzMUSIvF1+lbcwdIDt+NOVWFhITp27AjAvoZVO/8Y+WmJGomXHUb+4jE7TBszZgwURcGXX36JI0eOWPa+dv0x+jk3WIikSr3Ubv36+Gu9RkLBisvqPHI77lSp952xa3nGrj9GvYus0mRTS1JFXqKrt527itlhi+bNm6N///4AwrusWsWOP0a/5wYLkVRElqh33KG9vquqqwOWLWv8eOxmRqleVpcmewTY3Sdi1x9jmn3oJC2xH223btXezl2llR1am6AxO0yzo0/Ejj9G3+eGA0tFSXP1Oq/WQmAg0PBYMJh4rXfRoobnK4oQ8+cbP3aihcKysvD6sBcWE5Pwt7/9TQAQPXv2tPU4Vv8xenEZ3tXnoQ7XjlnvL8D8+Q2ZkSg7YnNjxgxjf4GMNhj4PDsiLVmyRAAQgwYNsvy9rfxj9HtusBBJll5z2NKl4b99S5bEbx7T+psFCPHss/rHjG1qCwTCoaT+zCtdTBY4evSoACAURRFVVVWyh2PKokXR/89R/xO6lavPQx2uHXO8plL1/1zxskMvNyKzIFa83FB/nkbZEamioqI+R06dOiV7OHH5OTdYiCQrUYma6Od6gaR3dU3kp6DYAPrpT/VDxseKiooEALFq1SrZQzHNSx86XX0e6nDtmI18tI33HL3cUM/92L9QerkRDAqxeXP8AiVNdOvWTQAQH3/8seyhJOTX3GCPSDKM7F4Yb6GwvBw4flz7vYUAXnih8fEiO5Ui1dWFj6N23PutiykOL++wmsabWtL06fEbCPSyAwjnhl4vWl1ddHbEy43aWmDEiPCtKNIwOyKp27174QZ4fs0NFiJmRTaazZwJzJun3xym1Tymvn7SJP1AWbAgOgwSddXH8lUXkz4vFyKUhtRz/7nnwufzjBn6TaWx2QE05AZgLDsS5YbW5cJpkh2ReAM8+ViImJHM7oWRJWzs64XQDpTYMDD7P1ofX3IXiYUIeYbWub9gQfzXqNkBNH5tIAB8//uNXxOZHXrnRbyr+9IkOyKpMyJbtmzB+fPnJY8mPbEQMcPoNVRal9bpvV6rGIkMg/Ly8MxLrDvv1L7kLxCIf62Y3tg8SC1E9u7dizNnzkgeDVEcRrLDTG7U1gL//M/6G1bo5cb99wObNpnPDh/lRqwuXbqgffv2uHjxIjZt2iR7OGmJhYgZRnaqibf9nd7r4/Wa6E2v3ndf43XkGTOAgwf19w/w2dZ8bdq0weWXXw4hBEpKSmQPh0hfouxIJjdGjdLvQ9PLjcmTwzfCM5MdPsuNWIqieKpPxJfs751Nnis73+NdQ2WkI17v9Xrt0EauzknURl1WFr4kMPZeFm6/EN2Am2++WQAQCxYskD0U33LleZiAK8cc79xPNjfU18dmgNGrcxJlx+bNvsyNWH/4wx8EAPHd735X9lB8w8w5mCGzCPKkKVOAAQOAtWuBsWPDny5U8aZf1RmOCROAt98OL8eMGhV9lY3WlKjaQX///eH30ro6J14LdeStwWPFjs2D960eOnQo/vu//5t9IuR+etmRSm4A2hmQKDf0XhdJzY7YptbYsQGezI5I6ozIhg0bcPHiRTRp0kTyiNKMA4VR0lz7qUb9pKG1MVC8TyHxXptIMheQ621+ZPXYJPr4448FANG7d2/ZQ/EtV56HCbhyzHrnmNtyQ29M8WZrPJgdkWpra0Xz5s0FALFx40bZw/EFbmhml1SXXpzeozfe5keyx2aRw4cPC/xjZ8QzZ87IHo4vue48NMB1YzZSbLglN4TQzw6zH748RF3mfTbe7tZkGDc0s1Kiu2TGdr7r3XjqhResvWuRkS52vSa3pUujx+bhOyq1bdsW7du3Z8MquU/kOap1d+7Ic8yp3Igdlx6t7AgEgI0boxtaPZwdsbifiET210XJk/6pJnbKceLExNOUWsrKGjd8JXptvPs/mJkKNXKDAo9/qvnBD34gAIjf/e53sofiS9LPwyRIH3PsjenMnv9CWJ8bseNidkTZtGmTACCaN28uamtrZQ/H87g0Y4VE/RXql94dc9VA2LxZiOef137tjBnar02lD0VvLInWib12R6UITzzxhAAg7r77btlD8SXp/1NPguuzQ68IsCs39MaV5tkRqaamRjRr1kwAEDt37pQ9HM9zTSHy1FNPiVGjRomcnBxRUFBg+vVSwyRef0Xkl3o3XVXsnS7jBZHWyZ3szfJix5EML91RKcJHH30kAIg+ffrIHoovOX0eppobQnggO5YujX6N3bkRb1xpnB2xrr32WgFA/OEPf5A9FM9zTY9ITU0NJk6ciJ/97Gd2HsYeWmuksfQ2M4u8kZTe6xYubHyZW3l5uH8j3pqrkU3VkuXROyqpO6x+8cUXOHfunOTRUKo8nRtA4uxQNyNTOZEbeuNK8+yIpV7Gyz4RhzlQGInXXnvNezMiQjSecrznHnObmWl9LVgQvRnZH/8oxL/9mxBz5pi7XM4HU6FWateunQAg1q1bJ3soviPrPEw2N4RwWXYEAg0zHVblxooVQnz0kRC33248N2LHxexoZOXKlQKAaN++vairq5M9HE/z7IZm1dXVqK6urv++qqpK4mgQ7g6fMCH8qUK9sd1TT0V/rzJyh9xgEPjRj8Kve+454LHHEo9BayMirXGluaFDh+Ljjz/G1q1bMXr0aNnDIYe5PjsAa3Ij3gaFsa/Rum8MsyOuESNGoEmTJqioqMD+/fvRtWtX2UNKC64qRObNm4ff/OY3socRLXb3Qb3dCLt3D+96qDe1GhkMzz4LPP544mMvWNAQQKrIHQzVO3NSVCFC6ccz2RFLXS7RKywib0RXXg5MnRp/+QZInBuJdlRNYzk5ORg+fDjWr1+PNWvWsBBxiOkekSeeeAKKosT92rJlS1KDmT17NkKhUP1XWVlZUu8jRVFROCS0LFjQsDeA3l0xY0V+ClL5/OZTqVD7RFiIuJOduQF4ODvUrdhj78Cteuedhn071q9PXIQwN1LGPhHnmZ4RefjhhzF58uS4z+ncuXNSg8nKykJWVlZSr5Vu8WLg1VcbPx4bDF99ZSxMYqdVy8ujp2Tr6sL3kZgwwVf3fEiWWojs2bMH58+fR9OmTSWPiCLZmRuAx7NDT2xTayKRsycqo7mhPjcNsyPWuHHj8PTTT7MQcZDpQqSwsBCFhYV2jMW71JM9tsDQKihyc/WXcG68Mdw3orV2a+TGWJHrx4FA+JOW1m29fah9+/Zo27Ytjhw5gh07dmCUmQAn2zE3dJjJji5d9N/nqquAP/85udwA0jo7Yo0ZMwaKomDfvn2oqKhA+/btZQ/J92y9fPfQoUMoKSnBoUOHUFtbi5KSEpSUlODs2bN2HtZ5eg1nL70UfTIvXgyMHKldhMyZA/zP/+hfApfo0jutTz4//Wn8bZx9hssz/pA2uQHoZ8df/qKdHVrmzAFWrUouNwBmR4yCggIMHDgQALBmzRrJo0kPthYiv/71rzF48GDMnTsXZ8+exeDBgzF48OCU1oJdSW/fgAcfbFiPjT3ZgfBrXn4ZKCsLX40TKfJ+EOqU6axZDceJ/cSkFWh1deF7VaQJFiL+kDa5AehnR2lpw7+byQ6zuQFo3wcnzbIjltonwkLEIQ5cTpw06XsBmHHPPfGv5Tezq2Gi+1QEAkKod4hU33vOHON7CagS3ZfCYz788EMBQPTv31/2UHzFU+fhP3hmzIsWJT5vjWaHmdwQomE3VzP7kES+1kfZEevdd99llqTINVu8p8ozYZJoUyJ162Mj93kwusFRMBgOFSPPTVTsJLr5lUeUlZUJACIYDIrz58/LHo5veOY8jOCJMRvJDb3nxWaHmdwoKwuf74m2k0+j7Ih15MgRAUAoiiJOnDghezie5Jot3tNGvE2J1PVY9TK9YLDhca0Nh4xscASEG85mzjS2sVHsFs56nfQeXxO+/PLL0bp1a9TW1mLHjh2yh0MUn5HcAIxlh5nc2LBBu0E23hhUPs2OWG3atEHPnj0hhMC6detkD8f3WIhoiVxnNUJvnVcNDCD8fhMmhNd+V65s2Fck9pi5uYnvcQPE3wQp8jlGi53Y+1J4kKIo7BMheazODXUTMyPZcfy4sdwIBsMFSLI7s/o0O7SMGzcOAPtEnMBCJFYym//EfmIJBIAZMxoaziLfb9my6CtjysvDl+x27Bh+zogRwG23NbyXomh3vT/9tH7wqMc/eLBhE7XIgLTz5leSsRAhKazOjSlTGr9nvOyYNClcIKgbo+nlxiuvAKNH62fHnDnRxU4aZUcsbmzmIPtXipLn+Dqv0T6OeK+PvBV2oveLXGuN/Zozp+G91PfdvDn6/WfM0H5t5C3G9dZzfXrzq/fff18AEAMGDJA9FN/wRL9FDEfHbHVuGHnPeNnxwAPxc0MI/eyI7AlJs+yIdeDAAQFAZGRkiLNnz8oejuewWTVZZq5sSfX9EjWXBQKJg6ysLH6ne6Iw0wpAjzt06BAbVi3GQiQBq3Mj0XsyOxxRV1cnOnToIACI//f//p/s4XgOm1WTZfW0Y7z3S9RcVleXeN112bLoe1QoSuK9RSLXc4uK9DdQ86iioiK0atUKtbW12Llzp+zhUDrQOs8DAaBZM2vfk9nhKEVR2CfiEBYikdQ128gAqKsL73Jopgkt8v0iezkiG8D0GtVUiQogra2hA4FwU5sqjdZzVWxYJcfF9noA4dwYOTJ8p21mh2exT8QZLERiTZgQfYIKATz+eHJ3rly8OLyroXr/hlmzgCuuCAdBbHgpSsMnFL2O9UhGuteNXjLsMyxEyHFTpoQvi42cZairY3Z4nDojsnHjRtTU1EgejX8pQiS6mFyeqqoqFBQUIBQKIT8/35mDrlwZDg49wWC4ozzRCVleHg4frSnUyJtKlZeHA0D9pKH+ezLvrze2yGOkQZC8//77uO222zBw4ECUlJTIHo7nSTkPU8TsMPn+zA5NQgi0bt0alZWVWL9+PW+maYKZc5AzIrESTXsavWY+3jpu5CZAkWutZtZdzXxiSYP13EjqjMju3btx4cIFyaOhtMHs8B1FUTB27FgA7BOxEwuRWFrrvZGMrpNaFUrxTJmiv8lRGuvQoQMKCwtx6dIlNqySc5gdvsQ+EfuxENESeZI++2xy66RWhZKR48R+YjG7w6PPsGGVpGF2+I7aJ7J27VrU1tZKHo0/sRDRo56k6k6HyXxysCKUzEpmh0cfYiFC0jA7fGXQoEHIzc1FKBTCZ599Jns4vsRmVSfZ3fhlpgnN59577z380z/9EwYNGoTt27fLHo6nefE89OKY42J2SHX99ddj2bJl+P3vf4+f//znsofjCWxWdSsjjV+pTI2m0Q2pElFnRD777DM2rJL3MTuk4sZm9mIh4iaxU6OPPWYuVLgJUb2OHTuiZcuWuHTpEnbt2iV7OET2is2O554z93pmR1yRDasuXkTwLBYibqHudqh+KqmrC4dJx47G12q5CVE9NqxS2tDKjsceC/eWGMXsiGv48OHIzMzE0aNH8TVniSzHQsRJelOn5eXA0qXaewcI0bBvgBG8LK8eCxHyDa3sUB9bv147O2bNMjejyuzQlZ2djREjRgDgZbx2YCHiFL2OdPXxX/xC/7Vm12rTfBMiFQsR8gWt7Ih8bPLk6K3lVUZufheL2aGLfSL2YSHihPJyYOrU6KnTn/4UKC6OnlLVEwgAx46Zb0JL8z0BIhtWq6urJY+GKAla2TF1avRjej0LwWD4DsBmMyDNc0MPNzazDwsRu0SezOvXNw6Lujrg44+1i5Dbb29oHFOU8GsnTTJ3bT/3BECnTp3QokULXLx4kdf/k3ckyg4htB+7//7ou/XedVf4DsBmMoC5oWvUqFEIBAI4cOAAylmkWYqFiB1iT+YVK7Sf166ddqf6888DBw+G+0bUQgSIvs9EPFrNa2b6THyCDavkOUazI3YpJhgEfvWrcG6sXBm+E/Cbb5rLAOZGXPn5+Rg8eDAALs9YjYWIWYmmLbVO5kWLGj9PUYDvf1+/U72oCCgsTO7a/lT3BPDR1CwLEXIFI+eUmeyYP18/N8aPB86eNZ8BVuwl4qPs0MI+EXuwEDHDyLSl3sk8Y0bD7EcgALz6ajg04nWqJ3ttfyp7AvhsapaFCEln9Jwykx2Jto9PJgNS3UvEZ9mhhX0iNhEuFgqFBAARCoVkD0WIsjIhAgF1dTb8FQyGHzf6vLIyIVaubPyaeBYtCr9efZ9Fi+x7ndHf0UP2798vAIjMzExRXV0tezie5Krz0CDXjNnMOeWG7Eg2b3yYHVqOHTsmAAgAorKyUvZwXM3MOcgZEaOMTlvG2xgomUvjYmdMJkwwNvWZzJ4APtzmuXPnzmjevDlqamrYsErOM3NOuSE7kt1LxIfZoaVVq1bo3bs3gPDdeMkaLESMMjNtafXGQGoILVtmburTbHj5cJtnNqySVGbPKTdkRzJFjw+zQ4+6PMM+EeuwEDHK7BbIVm8M5ERHu0+3eWYhQtIkc04xO1xNbVhln4h1MmQPwFOmTAlPb9p5O2498aY+rRyHzN/RJixESCrZ5xSzw1LqjMi2bdtw9uxZ5ObmSh6R97EQMUtdr3WaOvUZGSh2TX3K+h1tohYiO3fuRE1NDTIzMyWPiNKOzHOK2WGpDh06oFOnTjh48CA2bNiA733ve7KH5HlcmrGDHdfSp9HUp9W6dOmCyy67DDU1Ndi9e7fs4RBps2sPDmaH5XgZr7VYiFjNzmvpeXfMpCiKgiFDhgDg8gy5lN17cDA7LMWNzazFQsRKVjWFxftkxLtjJoV9IuRaTuQGwOywkDojsnHjRt5Q0wIsRKxkxbX0abA7oQwsRMi1mBue06NHD7Ru3RrV1dXYsmWL7OF4nm2FSGlpKaZMmYIuXbogJycHXbt2xdy5c1FTU2PXIeVL9Vp63nTKNpENqxcvXpQ8GtLD3PgH5oarKYrCy3gtZFsh8sUXX6Curg6vvPIKdu/ejQULFuC//uu/8Mtf/tKuQ8qXalNYmuxOKEPXrl1RUFCA6upqNqy6GHMDzA2PYJ+IdRQh1HvM2+/ZZ5/Fyy+/jP379xt6flVVFQoKChAKhZCfn2/z6CxUXp7ctfTl5eFp1djL7EpLua5rgWuuuQYrV67EokWLMIXNeobJPg/N5gYgf8xJYW54yvbt2zFkyBDk5+fj5MmTCKqFJAEwdw462iMSCoXQokUL3Z9XV1ejqqoq6suTkm0K42V2tmKfiDclyg3AJ9nB3PCUAQMGID8/H1VVVdi5c6fs4XiaY4XIvn378OKLL+KBBx7Qfc68efNQUFBQ/9WhQwenhucevMzONixEvMdIbgDMDuaG84LBIMaMGQOAfSKpMl2IPPHEE1AUJe5XbBdxRUUFrr/+ekycOBH33Xef7nvPnj0boVCo/qusrMz8b+QHvMzOFmohsmPHDjasOszO3ACYHQCYGxLwBnjWMN0jUllZicrKyrjP6dy5M7KzswGEw+Tqq6/GiBEj8PrrryMQ2x0ehyfXecm16urq0Lx5c1RVVaGkpAQDBw6UPSRPsOI8dDI3rBozUSLr1q3D2LFj0apVKxw9ehSKosgekmuYOQdN32umsLAQhYWFhp77zTff4Oqrr8bQoUPx2muvmQ4TSlF5ebijvnt3fkoCEAgEMGTIEHz66afYunUrCxEHMTc8hLlh2LBhw5CdnY3jx4/jyy+/RM+ePWUPyZNsO8MrKiowfvx4dOjQAc899xyOHz+OI0eO4MiRI3YdkiJxgyNN7BNxN+aGZMwNU7KysjBixAgA7BNJhW2FyCeffIKvv/4aK1asQFFREdq1a1f/RTbjBke6WIi4G3NDIuZGUtgnkjrbCpF7770XQgjNLzIg3n0jEt1Tghsc6YpsWL106ZLk0VAs5oYFks0O5kZSuMNq6rj46iZqSDz7rP70qJGp01S3jPaxbt26IS8vDxcuXMCePXtkD4fIGlZkB3MjKaNGjUIwGMTBgwdx6NAh2cPxJBYibhEZEo8/rj09anTqlBsc6VIbVgEuz5BPWJUdzI2k5Obm1mcKl2eSw0LEDWJDIpY6PWpm6pQbHOlinwj5htXZwdxIitonwuWZ5Ji+fJdsoBUSkSKnRwOBxveU0Js6LSripxkNLETIN+zIDuaGaePGjcPzzz/PGZEkcUbEDbTWZlWR06OcOrUEG1bJN5gdrjB27FgAwOeff47jx49LHo33sBBxg9iQCASAOXO0p0c5dZqy7t27Iy8vD99++y0+//xz2cMhSh6zwxVatmyJfv36AQDWrl0reTTew0LELaZMAZ5+umH6dN48YN8+7U8svKdESgKBAAYPHgyAyzPkA8wOV+BlvMljIeIW5eXAzJncTMgh6vLMtm3bJI+EKEXMDlfgxmbJYyHiFtxMyFFsWCXfYHa4gjojsn37dlRVVUkejbewEHELbibkKLUQKSkpQW1treTREKWA2eEKl19+Oa644grU1dVhw4YNsofjKSxE3IJd7Y7q0aMHcnNzcf78eXzxxReyh0OUPGaHa7BPJDksRNyEXe2OYcMq+QqzwxXYJ5IcFiJuw652x7BPhHyF2SGdOiOyadMmXLhwQfJovIOFCKUtFiJEZKVu3bqhbdu2qKmpQXFxsezheAYLEUpbaiGyfft2NqwSUcoURWGfSBJYiFDa6tGjB5o1a4bz589j7969sodDRD7APhHzWIhQ2goGg2xYJSJLqTMi69at472sDGIhQmltyJAhAFiIEJE1+vXrh8suuwxnz57Fjh07ZA/HE1iIUFpjwyoRWSkYDNbfjZd9IsawEKG0xoZVIrKaujzDPhFjWIhQWuvVqxeaNm2Kc+fO4csvv5Q9HCLyAbVhdfXq1RBCSB6N+7EQobQWDAYxaNAgAFyeISJrDBkyBDk5OThx4gQ+//xz2cNxPRYilPbYJ0JEVsrMzMSoUaMAcHnGCBYilPZYiBCR1bixmXEsRCjtRTas1tXVSR4NEfkB+0SMYyFCaa9Xr17IycnB2bNn2bBKRJYYOXIkMjIyUF5ejoMHD8oejquxEKG0l5GRwYZVIrJU06ZNMWzYMADsE0mEhQgR2CdCRNZjn4gxLESIwEKEiKzHG+AZw0KECGxYJSLrjRkzBoqiYO/evTh69Kjs4bgWCxEiAL1790ZOTg7OnDmDr776SvZwiMgHmjdvjv79+wMA1q5dK3k07sVChAjhhtWBAwcC4PIMEVkn8jJe0sZChOgf2CdCRFZjw2piLESI/oGFCBFZTS1EduzYgVAoJHk07sRChOgf1EJk27ZtbFglIku0a9cO3bp1gxAC69evlz0cV7K1ELn55pvRsWNHZGdno127drj77rtRUVFh5yGJktanTx9kZ2fjzJkz+Prrr2UPJ20xN8hv2CcSn62FyNVXX42lS5di7969+Otf/4p9+/bhRz/6kZ2HJEoaG1bdgblBfsM+kfgy7HzzRx99tP7fO3XqhFmzZuGWW27BxYsX0aRJEzsPTZSUoUOHYtOmTdi6dSvuuOMO2cNJS8wN8ht1RqS4uBjffvstcnJyJI/IXRzrETl58iT+/Oc/Y/To0bphUl1djaqqqqgvIiexYdVdjOQGwOwgd+vSpQvat2+PixcvYtOmTbKH4zq2FyIzZ85Es2bN0LJlSxw6dAgffvih7nPnzZuHgoKC+q8OHTrYPTyiKGxYdQczuQEwO8jdFEXhdu9xmC5EnnjiCSiKEvdry5Yt9c9/7LHHsH37dnzyyScIBoP453/+ZwghNN979uzZCIVC9V9lZWXJ/2ZESejTpw+ysrJQVVWF/fv3yx6Ob9iZGwCzg9yPfSL6FBHv7NZQWVmJysrKuM/p3LkzsrOzGz1eXl6ODh06YP369Rg1alTCY1VVVaGgoAChUAj5+flmhkmUtBEjRmDz5s145513MGnSJNnDkc6K89DJ3LBqzERW+uyzz9C/f380a9YMp06d8n2/k5lz0HSzamFhIQoLC5MamFrzVFdXJ/V6IicMHToUmzdvxtatW1mIWIS5QemuT58+aN68OU6dOoXt27fjyiuvlD0k17CtR2Tz5s146aWXUFJSgoMHD2LlypX48Y9/jK5duxr+VEMkAxtW5WFukF8FAoH65Rn2iUSzrRDJycnBe++9h+9+97vo2bMnfvKTn6Bfv35YtWoVsrKy7DosUcoiG1ZNrlxSipgb5GfsE9FmukfESVznJRkuXryIvLw8VFdX4+uvv0bXrl1lD0kqL56HXhwz+d/mzZsxYsQING/eHJWVlQgE/HuXFTPnoH//FIiS1KRJEwwYMAAAl2eIyDqDBw9G06ZNcerUKezZs0f2cFyDhQiRBvaJEJHVmjRpgtGjRwPg8kwkFiJEGliIEJEduLFZYyxEiDQMGTIEABtWichakQ2rzJYwFiJEGvr164fMzEycOnUKBw4ckD0cIvKJESNGoEmTJqioqGC2/AMLESINmZmZ6N+/PwAuzxCRdXJycjB8+HAA7BNRsRAh0sE+ESKyA/tEorEQIdLBQoSI7MCNzaKxECHSEVmIsKmMiKwyZswYKIqCr7/+GocPH5Y9HOlYiBDp6NevH5o0aYJTp06htLRU9nCIyCcKCgowcOBAAFyeAViIEOnKyspiwyoR2ULtE+HyDAsRorjYJ0JEduCdeBuwECGKg4UIEdlBLUR27dqFU6dOSR6NXCxEiOJgwyoR2aFNmzbo0aMHhBBYt26d7OFIxUKEKI7+/fujSZMmOHnyJA4ePCh7OETkI+wTCWMhQhRHVlYW+vXrB4DLM0RkLfaJhLEQIUqAfSJEZAd1RmTLli04d+6c5NHIw0KEKAEWIkRkh06dOqGoqAiXLl3Cpk2bZA9HGhYiRAmwYZWI7KAoCvtEwEKEKKH+/fsjIyMDJ06cwKFDh2QPh4h8hPedYSFClFB2djYbVonIFuqMyMaNG1FTUyN5NHKwECEygH0iRGSH3r17o7CwEN9++23a5gsLESIDWIgQkR0URcHYsWMBpO9lvCxEiAxgwyoR2SXdG1ZZiBAZMGDAAGRkZKCyshJlZWWyh0NEPqI2rK5duxa1tbWSR+M8FiJEBmRnZ6Nv374AgG3btkkeDRH5yaBBg5Cbm4tQKITPPvtM9nAcx0KEyCD2iRCRHTIyMjB69GgA6dknwkKEyCAWIkRkl3TuE2EhQmQQG1aJyC6RN8BLt3xhIUJk0IABAxAMBnHs2DF88803sodDRD5y5ZVXIjMzE0eOHMHXX38teziOYiFCZFBOTk59wyqXZ4jIStnZ2bjyyisBpN/yDAsRIhPYJ0JEdlH7RNKtYZWFCJEJLESIyC7pegM8FiJEJrBhlYjsMnr0aAQCARw4cADl5eWyh+MYFiJEJgwcOBDBYBBHjx5FRUWF7OEQkY/k5+dj0KBBANJrecaRQqS6uhqDBg2CoigoKSlx4pBEtsjJyUGfPn0AcHnGbswNSkfp2CfiSCHy+OOPo3379k4cish27BNxBnOD0lE69onYXoj8/e9/xyeffILnnnvO7kMROYKFiP2YG5Su1EJk9+7dOHHihOTROCPDzjc/evQopk6dig8++ABNmzZN+Pzq6mpUV1fXfx8KhQAAVVVVto2RyKxevXoBAIqLixEKhaAoiuQR2Us9/5xqzjWbGwCzg/wjKysLPXr0wJdffolPPvkEN910k+whJcVUbgib1NXVieuvv178+7//uxBCiAMHDggAYvv27bqvmTt3rgDAL37xy4VfZWVldsVFSrnB7OAXv9z7ZSQ3FCHMfcx54okn8Jvf/Cbuc4qLi7F+/XosWbIEq1evRjAYRGlpKbp06YLt27fXdwXHiv1UU1dXh5MnT6Jly5ZxP3VWVVWhQ4cOKCsrQ35+vplfx5X89Pvwd3EnM7+LEAJnzpxB+/btEQgkt5prZ24AzA6Av4ub+en3Mfq7mMkN04VIZWUlKisr4z6nc+fOmDx5Mj766KOoEKitrUUwGMSdd96JN954w8xh46qqqkJBQQFCoZDn/yMD/vp9+Lu4k9O/ixtzA+B/U7fy0+8C+Ov3seN3Md0jUlhYiMLCwoTP+/3vf4+nnnqq/vuKigpMmDABS5YswYgRI8welog8jLlBRHpsa1bt2LFj1Pe5ubkAgK5du6KoqMiuwxKRhzE3iNKPL3ZWzcrKwty5c5GVlSV7KJbw0+/D38Wd/PS7pMJPfw78XdzLT7+PHb+L6R4RIiIiIqv4YkaEiIiIvImFCBEREUnDQoSIiIikYSFCRERE0rAQISIiIml8XYhUV1dj0KBBUBQFJSUlsodjWmlpKaZMmYIuXbogJycHXbt2xdy5c1FTUyN7aIb88Y9/RJcuXZCdnY2hQ4dizZo1soeUlHnz5mH48OHIy8tD69atccstt2Dv3r2yh2WJefPmQVEUPPLII7KH4hpezw2A2eEGzA3jfF2IPP7442jfvr3sYSTtiy++QF1dHV555RXs3r0bCxYswH/913/hl7/8peyhJbRkyRI88sgjmDNnDrZv345x48bhhhtuwKFDh2QPzbRVq1bhoYcewsaNG7F8+XJcunQJ1113Hc6dOyd7aCkpLi7GwoULMWDAANlDcRWv5wbA7HAD5oYJqdwp083+9re/iV69eondu3cLIPHdO71i/vz5okuXLrKHkdCVV14pHnjggajHevXqJWbNmiVpRNY5duyYACBWrVoleyhJO3PmjOjevbtYvny5+M53viOmTZsme0iu4NfcEILZIRtzQ58vZ0SOHj2KqVOn4s0330TTpk1lD8dSoVAILVq0kD2MuGpqarB161Zcd911UY9fd911WL9+vaRRWScUCgGA6/87xPPQQw/hpptuwrXXXit7KK7h59wAmB2yMTf02XavGVmEELj33nvxwAMPYNiwYSgtLZU9JMvs27cPL774Ip5//nnZQ4mrsrIStbW1aNOmTdTjbdq0wZEjRySNyhpCCEyfPh1jx45Fv379ZA8nKe+88w62bduG4uJi2UNxDT/nBsDskI25EZ9nZkSeeOIJKIoS92vLli148cUXUVVVhdmzZ8sesi6jv0ukiooKXH/99Zg4cSLuu+8+SSM3J/JW7kD4ZIx9zGsefvhh7Ny5E3/5y19kDyUpZWVlmDZtGt566y1kZ2fLHo7t/JQbALPDq5gb8XnmXjOVlZWorKyM+5zOnTtj8uTJ+Oijj6L+0tbW1iIYDOLOO+/EG2+8YfdQEzL6u6j/wSsqKnD11VdjxIgReP311xEIuLt+rKmpQdOmTfHuu+/i1ltvrX982rRpKCkpwapVqySOLnk///nP8cEHH2D16tXo0qWL7OEk5YMPPsCtt96KYDBY/1htbS0URUEgEEB1dXXUz7zOT7kBMDu8mB3MjcQ8U4gYdejQIVRVVdV/X1FRgQkTJuD//t//ixEjRnjuVuLffPMNrr76agwdOhRvvfWWZ/4nMWLECAwdOhR//OMf6x/r06cPfvjDH2LevHkSR2aeEAI///nP8f777+PTTz9F9+7dZQ8paWfOnMHBgwejHvuXf/kX9OrVCzNnzvTstHGq/JYbALNDNuaGcb7rEenYsWPU97m5uQCArl27ei5MKioqMH78eHTs2BHPPfccjh8/Xv+ztm3bShxZYtOnT8fdd9+NYcOGYdSoUVi4cCEOHTqEBx54QPbQTHvooYfw9ttv48MPP0ReXl79WnVBQQFycnIkj86cvLy8RqHRrFkztGzZMm2LEMBfuQEwO9yAuWGc7woRP/nkk0/w9ddf4+uvv24Uhm6fyJo0aRJOnDiBJ598EocPH0a/fv3wt7/9DZ06dZI9NNNefvllAMD48eOjHn/ttddw7733Oj8gogSYHfIxN4zz3dIMEREReYe7O5eIiIjI11iIEBERkTQsRIiIiEgaFiJEREQkDQsRIiIikoaFCBEREUnDQoSIiIikYSFCRERE0rAQISIiImlYiBAREZE0LESIiIhImv8PqcTJe7WT5UMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize optimization\n",
    "fig = plt.figure()\n",
    "ax0 = fig.add_subplot(121)\n",
    "ax1 = fig.add_subplot(122)\n",
    "x0_numpy = np.arange(-4, 5)\n",
    "x1_numpy = (- b_new - x0_numpy * w_new[0]) / w_new[1]\n",
    "ax0.plot(x0_numpy, x1_numpy, color = \"black\")\n",
    "x0_jax = np.arange(-4, 5)\n",
    "x1_jax = (- b_jax - x0_jax * w_jax[0]) / w_jax[1]\n",
    "ax1.plot(x0_jax, x1_jax, color = \"black\")\n",
    "for x, y in zip(x_data, y_classes):\n",
    "    if y == 1:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"blue\")\n",
    "    else:\n",
    "        ax0.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "        ax1.plot(x[0], x[1], marker=\".\", color=\"red\")\n",
    "\n",
    "ax0.set_title(\"Numpy\")\n",
    "ax1.set_title(\"Jax\")\n",
    "ax0.set_ylim(-4, 4)\n",
    "ax1.set_ylim(-4, 4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to experiments above, running time is as follows in my settings.\n",
    "Numpy: 1.8 s\n",
    "JaX: 8.6 s\n",
    "\n",
    "According to the figure above, the accuracy of classification is 100% for both implementations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "> ### Going further (optional)\n",
    "> Implement a more complex dataset and compare the results of the NumPy and JAX implementations.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2 - Polynomial regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> In this exercise, you will implement polynomial regression using NumPy and JAX. This should be a quite straightforward extension of what we have seen until now. We deliberately removed details of the implementation (list of points to adress) so that you can start defining your first entire machine learning setup by yourself.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use dataset similar to the above one\n",
    "import numpy as np\n",
    "true_w0 = 2\n",
    "true_w1 = -3\n",
    "x = np.random.rand(100) * 5\n",
    "y = true_w0 + true_w1 * x + np.random.randn(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA6C0lEQVR4nO3de3TU9Z3/8deQknAxiYQICSVCFrUSouXmJUgroCDUomh/KlZd+VXZiuBKKUeXbRXwspRqW9y1olgWpWjRPZYKPymVLoJLDeVmrEjLQQoFISkSZAIoCSbz+4OdmMtcvt+Z732ej3PmnM7kOzOfmbbO28/nfQlFIpGIAAAAAqSD2wsAAACwGgEOAAAIHAIcAAAQOAQ4AAAgcAhwAABA4BDgAACAwCHAAQAAgUOAAwAAAudLbi/ADU1NTTp06JByc3MVCoXcXg4AADAgEono+PHj6tWrlzp0SLxHk5EBzqFDh1RSUuL2MgAAQAoOHDig3r17J7wmIwOc3NxcSWe+oLy8PJdXAwAAjKirq1NJSUnz73giGRngRI+l8vLyCHAAAPAZI+klJBkDAIDAIcABAACBQ4ADAAAChwAHAAAEDgEOAAAIHAIcAAAQOAQ4AAAgcAhwAABA4GRkoz+3NTZFtHnvUR0+fko9cjvp0tICZXVgJhYAAFYhwHHYmh3Vmrtqp6rDp5ofK87vpNnjyzS2vNjFlQEAEBwcUTlozY5qTVm2vVVwI0k14VOasmy71uyodmllAAAECwGOQxqbIpq7aqciMf4WfWzuqp1qbIp1BQAAMIMAxyGb9x5tt3PTUkRSdfiUNu896tyiAAAIKAIchxw+Hj+4SeU6AAAQHwGOQ3rkdrL0OgAAEB8BjkMuLS1QcX4nxSsGD+lMNdWlpQVOLgsAgEAiwHFIVoeQZo8vk6R2QU70/uzxZfTDAQDAAgQ4DhpbXqyFtw9WUX7rY6ii/E5aePtg+uAAAGARGv05bGx5sUaXFdHJGAAAGxHguCCrQ0gV/bq7vQwAAAKLIyoAABA4rgc48+bN0yWXXKLc3Fz16NFDEyZM0K5duxI+Z/369QqFQu1uf/nLXxxaNQAA8DLXA5wNGzZo6tSp2rRpk9auXavPP/9cY8aM0cmTJ5M+d9euXaqurm6+nX/++Q6sGAAAeJ3rOThr1qxpdX/JkiXq0aOHtm3bpq9//esJn9ujRw+dffbZNq4OAAD4kes7OG2Fw2FJUkFB8oZ3gwYNUnFxsa666iq99dZbca+rr69XXV1dqxsAAAguTwU4kUhEM2bM0PDhw1VeXh73uuLiYi1atEivvfaafv3rX+srX/mKrrrqKr399tsxr583b57y8/ObbyUlJXZ9BAAA4AGhSCQScXsRUVOnTtUbb7yhjRs3qnfv3qaeO378eIVCIa1cubLd3+rr61VfX998v66uTiUlJQqHw8rLy0t73QAAwH51dXXKz8839PvtmR2c++67TytXrtRbb71lOriRpMsvv1y7d++O+becnBzl5eW1ugEAgOByPck4Eonovvvu04oVK7R+/XqVlpam9DrvvvuuiosZdQAAADwQ4EydOlUvv/yyXn/9deXm5qqmpkaSlJ+fr86dO0uSZs2apYMHD2rp0qWSpAULFqhv374aMGCAGhoatGzZMr322mt67bXXXPscAADAO1wPcBYuXChJGjFiRKvHlyxZokmTJkmSqqurtX///ua/NTQ0aObMmTp48KA6d+6sAQMG6I033tA3vvENp5adtsamCPOoAACwiaeSjJ1iJknJDmt2VGvuqp2qDp9qfqw4v5Nmjy9jojgAAHH4Msk4U6zZUa0py7a3Cm4kqSZ8SlOWbdeaHdUurQwAgOAgwHFQY1NEc1ftVKwts+hjc1ftVGNTxm2qAQBgKQIcB23ee7Tdzk1LEUnV4VPavPeoc4sCACCAXE8yDpJkicOHj8cPbloyeh0AAIiNAMciRhKHe+R2MvRaRq8DAACxcURlAaOJw5eWFqg4v5PiFYOHdCYourQ0+aBRAAAQHwFOmswkDmd1CGn2+DJJahfkRO/PHl9GPxwAANJEgJMms4nDY8uLtfD2wSrKb30MVZTfSQtvH0wfHAAALEAOTppSSRweW16s0WVFdDIGAMAmBDhpSjVxOKtDSBX9urd6jPENAABYgwAnTdHE4ZrwqZh5OCGdOX5KljjM+AYAAKxDDk6arEgcZnwDAADWIsCxQDqJw0aqsH6wYodWbP9IlXtqGeMAAIABHFFZJFHicKLcGiNVWLUnG/S9V9+TxLEVAABGEOBYKFbi8Oo/VeuHr+/Q0ZMNzY+1DFLMjmWIHltRUg4AQHwcUdlo3uqduvfl7a2CG+lMX5xobo3ZsQxMHQcAIDkCHJus/tMhPff23rh/j+hMkDKkT7eE4xviPZep4wAAxEeAY4PGpoh++PqOpNdVh09p298+iVuFlQxTxwEAiI0Axwab9x7V0ZOnDV17+PipuFVYyTB1HACA2EgytoGZnZVokNKyCqum7pQe/X8fxA2SjDYPBAAgUxHg2MDozkr3rtmtgpSWVVidO3bQlGXbJalVjxymjgMAkBxHVDaIjm9I5tHry2MGKY1NEeV3ztZ3ruirbl07tvobU8cBAEiOHRwbRMc3TFm2PWaHYkn67tdL9Y2L2wcpsWZSFXTN1oSBvTS6rIgBnAAAGMAOjk2iicNtd3IKunbUM98epFnfKGv3nHgzqT452aAlf9in8GcNBDcAABjADo6NEo1vaCvZTKqQzvTNGV1WRJADAEASBDg2izW+IRYjM6mizf2MvB4AAJmMIyqPMFpaTnM/AACSI8DxCKOl5TT3AwAgOY6oPCJaWl4TPhUzD4fmfu01NkUM5TcBADIPAY5HtCwtD8md5n5+ChhildMX53fS7PFlMXsE+emzAQDSF4pEIvFatQRWXV2d8vPzFQ6HlZeX5/ZyWjH7w+33901FtJy+7f9wo+FK20aIfvpsAID4zPx+E+B4LMCRnN9tMBswuKmxKaLh89fFrTiLHuVtfHCUsjqEfPXZAACJmfn9JsnYg6Kl5dcP/LIq+nW3/VgqUf8d6Uz/ncYmb8TBZsrp/fbZAADWIcDJcGYCBi8wU07vt88GALAOAU6G81v/HTPl9H77bAAA6xDgZDi/9d+JltPHO7QL6UwC8aWlBb77bAAA6xDg+FBjU0SVe2r1etVBVe6pTSuHxEzA4AXRcnpJ7dbctpzeb58NAGAdAhyfWbOjWsPnr9Otz2/S/curdOvzmzR8/jqt2VGd0uuZCRi8IjqpvajNpPai/E6tqqL8+NkAANagTNyDZeLx2Fny7MdeMUbL6f342QAA7fmuD84zzzyjJ554QtXV1RowYIAWLFigr33ta3Gv37Bhg2bMmKEPPvhAvXr10gMPPKB77rnH8Pv5McAx2/8l1fcIarffIH82AMgUZn6/XR/V8Morr2j69Ol65plndMUVV+i5557TuHHjtHPnTp177rntrt+7d6++8Y1vaPLkyVq2bJn+8Ic/6N5779U555yjb33rWy58AmeYKXmu6Nc9pfeI9t8JoiB/NgBAe67n4Pz0pz/VXXfdpbvvvlv9+/fXggULVFJSooULF8a8/tlnn9W5556rBQsWqH///rr77rv1ne98R08++aTDK3dW0EqerUyUBgCgLVd3cBoaGrRt2zb9y7/8S6vHx4wZo3feeSfmcyorKzVmzJhWj11zzTVavHixTp8+rY4dO7Z7Tn19verr65vv19XVWbB6ZwWp5NmtnBiOqQAgc7ga4Bw5ckSNjY3q2bNnq8d79uypmpqamM+pqamJef3nn3+uI0eOqLi4/Q/kvHnzNHfuXOsW7oJoyXNN+FTM0QPRHByvlzzHS5SuCZ/SlGXbbZsNRaIxAGQW14+oJCkUav1v0ZFIpN1jya6P9XjUrFmzFA6Hm28HDhxIc8XOC0LJs1uzoaJBVdscpmhQlWqJPQDAu1wNcAoLC5WVldVut+bw4cPtdmmiioqKYl7/pS99Sd27x04izcnJUV5eXqubHxnt/+JVbsyGYuAmAGQmV4+osrOzNWTIEK1du1Y33HBD8+Nr167V9ddfH/M5FRUVWrVqVavH3nzzTQ0dOjRm/k3QjC0v1qgLe+qXlfv0t6Ofqk9BF91R0VfZX/LEZlxCbiRKp1t9ZmXeDjlAAOAc18vEZ8yYoTvuuENDhw5VRUWFFi1apP379zf3tZk1a5YOHjyopUuXSpLuuecePf3005oxY4YmT56syspKLV68WL/61a/c/BiOiZVL8ouNe32RS+JGonQ6QZWVeTvkAAGAs1z/1/5bbrlFCxYs0COPPKKBAwfq7bff1urVq9WnTx9JUnV1tfbv3998fWlpqVavXq3169dr4MCBevTRR/Xv//7vge6BE+X3XJJoonQyn5xssOw9Uw2qrPyu471WdfiU7lm2XY+u+sBwqTzl9QBgjCc6GTuNTsbuWf2nQ7r35XcTXlNs4eeIfm/Jqs9avp+V33Wy12op2Y4Ou0AAMp2Z32/Xd3BgjBsJunbo1jUn6TVWfo5Uqs+s/K6TvVZLiXaH/L57BwBOI8DxiaB0Mnbjc5itPrNyjWY+R7yqLirBAMA815OMYUxQOhm79TnGlhdrdFmRoSomK9do9nPEqupyYg4ZAAQNAY5PBKWTsZufw+jATSvXmOy14mm58xOU3TsAcBJHVD4RhE7Gkj8+h5VrTPRaibTc+QnK7h0AOIkAx0f83sk4yg+fw8o1xnutWEI6UxnVcncougsUL0CK9RwAyHSUifukTLyloHTE9cPnsKOT8dqdNfrPP+xTSGp1bBV91VgBVLSKSiaeAwBBY+b3mwDHhwEO/C+Vnjb0wQGQ6QhwkiDAgV3M7Piksjvkh10vALCLmd9vqqiQUaIBQk3dKR09Ua+Crtkqyu9sSaBgdofFaFVXus8BgExEgIOMESsAiUr3qCeaI9N2OzTaaZgcGQBwFlVUyAjxRh1EVacx8oBOwwDgPQQ4CLxEAUhLEaUWiARlThgABAkBDgLPzMDLVAIROg0DgPcQ4CDwzAYWZq+n0zAAeA8BDgLPbGBh9no6DQOA9xDgIPCiAYgRqQQifpivBQCZhgAHgZfVIaSHri0zdO1D1/ZPKRDxw3wtAMgk9MFBRujWNdvgdTkpv8fY8mKNLiui0zAAeAABDjKCU5VOdBoGAG8gwIHvpDKPqfAsYzszRq8DAHgbAQ48wWjQkvJEbaO9+2g2DACBQIAD1xkNWtKZ93TkZL2htfxuZ406dAiROwMAPkcVFVwVb0ZUTZvZUOnOezLa22Zp5d906/ObNHz+upTmUgEAvIEAJ8M0NkVUuadWr1cdVOWeWlcHQJoJWtKd95SsGV9bbQMsAIC/cESVQVLOX7GJmaAl3SqoaDO+Kcu2K6TkqTYRnWnSN3fVTo0uK+K4CgB8hh2cDGH0KMhJZoIWK+Y9xWvGFw9TwAHAvwhwMkC6+St2MRO0WDXvaWx5sTY+OEq/mny5/rGij6H3Zwo4APgPAU4GSDd/xS5mghYr5z1Fm/GNM3gsxxRwAPAfApwM4FQXX7PMBi1Wz3tiCjgABBdJxhnAivwVu0SDlrbJz0Vxkp+tnPeUKPGYKeAA4G+hSCSScb1b6+rqlJ+fr3A4rLy8PLeXY7vGpoiGz1+nmvCpmHk4IZ0JKDY+OMq1H/NUxi9YxWvVZQCA2Mz8fhPgZECAI31RRSXF3qlI5YgnSNwMsAAAxhDgJJGJAY7ETgUAwN/M/H6Tg5NBrMxfQXzsBgGA+whwMky0RBr2YJcMALyBMnHAIl7sFg0AmYoAB7CAV7tFA0Cmci3A2bdvn+666y6Vlpaqc+fO6tevn2bPnq2GhoaEz5s0aZJCoVCr2+WXX+7QqoHYvNotGgAylWs5OH/5y1/U1NSk5557Tuedd5527NihyZMn6+TJk3ryyScTPnfs2LFasmRJ8/3s7Gy7lwsk5NVu0QCQqVwLcMaOHauxY8c23/+Hf/gH7dq1SwsXLkwa4OTk5KioqMjuJQKGeblbNABkIk/l4ITDYRUUJJ/7s379evXo0UMXXHCBJk+erMOHDye8vr6+XnV1da1ugJW8PNeqsSmiyj21er3qoCr31JIHBCAjeKZMfM+ePfqP//gP/eQnP0l43bhx43TTTTepT58+2rt3rx566CGNGjVK27ZtU05OTsznzJs3T3PnzrVj2YAk7861omwdQKayvJPxnDlzkgYTW7Zs0dChQ5vvHzp0SFdeeaWuvPJK/eIXvzD1ftXV1erTp4+WL1+uG2+8MeY19fX1qq+vb75fV1enkpKSjOtkDPt5KaCIlq23/T844zkA+JWrnYynTZumiRMnJrymb9++zf/50KFDGjlypCoqKrRo0SLT71dcXKw+ffpo9+7dca/JycmJu7sDWCndbtFWdUFOVrYe0pmy9dFlRXRZBhBIlgc4hYWFKiwsNHTtwYMHNXLkSA0ZMkRLlixRhw7mU4Jqa2t14MABFRfzb6LwhlS7RVu5+2OmbJ3O1gCCyLUk40OHDmnEiBEqKSnRk08+qY8//lg1NTWqqalpdd2FF16oFStWSJJOnDihmTNnqrKyUvv27dP69es1fvx4FRYW6oYbbnDjYwCWsLoLMmXrADKda0nGb775pj788EN9+OGH6t27d6u/tUwL2rVrl8LhsCQpKytL77//vpYuXapjx46puLhYI0eO1CuvvKLc3FxH1w9v8uOgSzuOkyhbB5DpXAtwJk2apEmTJiW9rmWw07lzZ/3ud7+zcVXwMy8l+Jphx3FStGy9JnwqZuAUklTkUtk6ADjBU31wgFT5edClHcdJ0bJ1Se1687hZtg4ATiHAge/5fdClXcdJY8uLtfD2wSrKb/28ovxOlIgDCDzPNPoDUuX3iiE7j5PSLVsHAL8iwIHv+b1iyO4uyKmWrQOAn3FEBd8LQsUQx0kAYC12cOB7fqkYSlbCznESAFiHAAe+59VBly0ZLWHnOAkArMERFQLBy0c8fi5hBwC/YgcHgZHKEY/dnY/T7VLsx87MAOAFBDgIFDNHPE50Pk6nhN2vnZkBwAs4okJGcurYKNUSdo61ACA9BDjIOE52Pk6lhN3vnZkBwAsIcJBxzBwbpStawh4vayakM8dOLUvYnVwfAAQVAQ4yjpOdj1MZeun3zswA4AUEOMg4Tnc+NlvCHoTOzADgNqqokHHc6HxspoTdL52ZAcDL2MFBxknl2Miq963o113XD/yyKvp1j/v6bq0vqrEposo9tXq96qAq99SSzAzAl0KRSCTj/ulVV1en/Px8hcNh5eXlub0cuMTrfWbcWJ/XvxMAmc3M7zcBDgFORvN6p2An1xftvdP2HwjRd2ubL2TV2rz+3wEA7zDz+00ODjKa14dbOrU+syMlku30GA1a2DECYBcCHACmeu+EP2uIudMT7bL8T18v1cr3qpMGLfF2jKKv4/aQVAD+RpIxAMM9dWrCnyXc6YlIeu7tvUlHTNCtGYDdCHAAGO6pc/RkQ8KdnnjaBi12dGum+gtASxxRATDce6fgrJyU36Nl0GJ1t2ZyeQC0xQ4OAMO9d4ry0u+eHE08NsLIdUxeBxALAQ4AScZGSiQbHmpEtKrK7BDSWMjlARAPR1QAmiUbKRHd6ZmybLtCUszAIp6WIyYSvY6Zbs1mcnm83A4AgPXYwQHQSsuREpeWFmjz3qOtEnfj7fQU53fSd79eqpCMjZgwO4Q0FiavA4iHHRwAMSVL3I230zPo3G7tnlcUJ+HXzBDSWJi8DiAeRjUwqgEpCvKIAbNjG9py6rtpbIpo+Px1Sau/Nj44KjD/3QCZjFENgM2CXJZsdmxDLE6NmLAqlwdA8JCDA5gU9LJkO5rw2cmKXB4AwcMODmCCFbsbXufHxN10c3kABA8BDmBCJpQl+zVx1+uT4QE4iyMqwAQ/7m6YZVUTPgBwEwEOYIJfdzfMMDq2wc/HPwzmBIKPIyrABKNDKf2+uxFN3DXaz8YsN0vsg1wBB+AL9MGhDw5MilZRSbHLkoNUuWNHIOJmgJFufx8A7jLz+02AQ4CDFLALkBqrAgyjgVfL6wrPytH3X61STV19zNekKSDgfb5p9Ne3b1/97W9/a/XYgw8+qB/96EdxnxOJRDR37lwtWrRIn3zyiS677DL9/Oc/14ABA+xeLtCMsmTzrCqxNxpcxroukSBUwAH4gutJxo888oiqq6ubbz/84Q8TXv/jH/9YP/3pT/X0009ry5YtKioq0ujRo3X8+HGHVgyc0XIoZUW/7gQ3SVjRQNBok8V41xnh5wo4AF9wPcDJzc1VUVFR8+2ss86Ke20kEtGCBQv0gx/8QDfeeKPKy8v14osv6tNPP9XLL7/s4KoBmJVuiX2yHSDpzA5Qw+dNca8zws8VcAC+4HqAM3/+fHXv3l0DBw7U448/roaGhrjX7t27VzU1NRozZkzzYzk5Obryyiv1zjvvxH1efX296urqWt0AOCvdEnujO0C/rNyX0s4N/X2AYHE1B+f+++/X4MGD1a1bN23evFmzZs3S3r179Ytf/CLm9TU1NZKknj17tnq8Z8+e7XJ5Wpo3b57mzp1r3cIBmJZuib3RHaC/Hf3U9NqC0t8HwBcs38GZM2eOQqFQwtvWrVslSd/73vd05ZVX6uKLL9bdd9+tZ599VosXL1ZtbW3C9wiFWv8DKBKJtHuspVmzZikcDjffDhw4kP4HBWBKug0Eje4A9SnoYnptDOYEgsfyHZxp06Zp4sSJCa/p27dvzMcvv/xySdKHH36o7t3bVzEUFRVJOrOTU1z8xT+IDh8+3G5Xp6WcnBzl5OQkWzoAm6XTQNDoDtAdFX31i417k1735P/5qo6crKcCDggoywOcwsJCFRYWpvTcd999V5JaBS8tlZaWqqioSGvXrtWgQYMkSQ0NDdqwYYPmz5+f2oIBOCrVEvvoDtCUZdsVUuwmi7PHlyn7Sx0MXXfF+an9cwqAP7iWZFxZWamf/exnqqqq0t69e/Xqq6/qu9/9rq677jqde+65zdddeOGFWrFihaQzR1PTp0/Xv/3bv2nFihXasWOHJk2apC5duujb3/62Wx8FgEmplthHd4CK8lsfV7U9YjJ6HYDgci3JOCcnR6+88ormzp2r+vp69enTR5MnT9YDDzzQ6rpdu3YpHA4333/ggQf02Wef6d57721u9Pfmm28qNzfX6Y8AwAVGd4CMXufmXKxU+XHNgNMY1cCoBiBj+XHkhh/XDFjFzO+3631wAMBOjU0RVe6p1etVB1W5p1aNTWf+nc5oV2Qv8eOaAbe42gcHAOwUb7fjoWv769E3/pz2XCwnWTXLC8gU7OAACKREux33vvyu6blY8XaCnGLFLC8gk7CDAyBwjMytMiLaPdkLeS/pzvICMg07OAAs5fZOh5R8t8OoHrmdPJP3ku4sLyDTsIMDwDJe2OmQ0t/FiHY7HtKnm6584i1P5L2kO8sLyDTs4ACwhFd2OiRzuxiJ5mJt+9snnsl7SXeWF5BpCHAApM1IzsvcVTsdO66K7nbE+6kP6czO0jPfTtzt2Gt5L3RoBozjiApA2sxU+FT0az9I12pG51aNLS/WNeXxux17Me8l1VleQKYhwAGQNq/tdEjGJ5dH52LF4tW8l0RrBnAGAQ6AtHlxp0NKf7fD6E4QuyeA95CDAyBtRnNe3KjwSXVyeRR5L4A/sYMDIG1B3+kg7wXwH6aJM00csIxX+uAACCYzv9/s4ACwjB07HY1NEXZOAJhGgAPAUlZW+Li9IxQNrmrqTunoiXoVdM1WUX5ngizABwhwAHhStDNy2zP0aGdkuxN8YwVXURy7Ad5HFRUAz3G7M3K8sRNR1S6MnwBgDgEOAM8x0xnZaomCq7ZrcHL8BABzCHAAeI6bnZGTBVctOTVoE4B5BDgAPMfNzshmgyYnx08AMI4AB4DnuNkZ2WzQ5PT4CQDGEOAA8JxoZ2RJ7YIcuzsjR4MrI9waPwEgOQIcAJ7k1gyorA4hXffV5K8dkrvjJxqbIqrcU6vXqw6qck8tyc5AG/TBAeBZbsyAamyKaOV7icu/O4Skp28d5FofHLcbIAJ+QIADwNOs7IxshJEqqqaI1K1rjkMras3tBoiAX3BEBQAtuFminozbDRABPyHAAYAW3CxRT8bNBoiA33BEBQAtRKuoasKnYu6UhHQm0dmN6ikv7S4x5R1eR4ADAC1ES9SnLNuukNQqyLG7RD0Zr+wukeQMP+CICgDacKtEPRk3GyBGxRtEWuPAAFJK42EGOzgAEIMbJerJuL27lCzJOaQzSc6jy4osXwO7RjCLHRwAiCNaon79wC+rol93T+SYuLm75FaSs5u7RvAvdnAAwGOSJfC6tbvkRpKzm7tG8DcCHADwEKNHMU43QJTcSXI2s2vk9PcBb+OICgA8wutHMW4kOXupNB7+QoADAB7ghy7Fbkx590ppPPyHAAcAPMCqBF67S6mdTnL2Qmk8/Mm1HJz169dr5MiRMf+2efNmXXLJJTH/NmnSJL344outHrvsssu0adMmy9cIAE6x4ijGqVJqJ5Oc3S6Nh3+5FuAMGzZM1dWtz5Mfeugh/f73v9fQoUMTPnfs2LFasmRJ8/3s7Gxb1ggATkn3KMbpKeNOJjlHd43aBm9F9MFBAq4FONnZ2SoqKmq+f/r0aa1cuVLTpk1TKJQ4Es/JyWn1XADwu3RmYGVCKbUXGy/C2zyTg7Ny5UodOXJEkyZNSnrt+vXr1aNHD11wwQWaPHmyDh8+nPD6+vp61dXVtboBgJekk8CbKVPGvdh4Ed7lmQBn8eLFuuaaa1RSUpLwunHjxumll17SunXr9JOf/ERbtmzRqFGjVF9fH/c58+bNU35+fvMt2XsAgBtSTeCllBpoLxSJRCxNsZ8zZ47mzp2b8JotW7a0yrP56KOP1KdPH7366qv61re+Zer9qqur1adPHy1fvlw33nhjzGvq6+tbBUB1dXUqKSlROBxWXl6eqfcDgHQl61Sc7O9tVe6p1a3PJy+0+NXky2mGB1+rq6tTfn6+od9vy3Nwpk2bpokTJya8pm/fvq3uL1myRN27d9d1111n+v2Ki4vVp08f7d69O+41OTk5ysnJMf3aAGA1I5VOZhN408nfAYLK8gCnsLBQhYWFhq+PRCJasmSJ/vEf/1EdO3Y0/X61tbU6cOCAiovJogfgbXZVOlFKDbTneg7OunXrtHfvXt11110x/37hhRdqxYoVkqQTJ05o5syZqqys1L59+7R+/XqNHz9ehYWFuuGGG5xcNgCYYnenYjenjANe5PqwzcWLF2vYsGHq379/zL/v2rVL4XBYkpSVlaX3339fS5cu1bFjx1RcXKyRI0fqlVdeUW5urpPLBgBTnBgaSSk18AXXA5yXX3454d9b5kB37txZv/vd7+xeEgBYzqlKJzemjANe5PoRFQBkAoZGAs4iwAEABzA0EnAWAQ4AOCCdTsV+Zvd0cyAe13NwACBTxBsaWdA1W9cP7KX8ztlqbIoEJshxaro5EIvlnYz9wEwnRACwWrRT8e931mhF1UEdPXm6+W9BCQDi9fyJhm6UriMVZn6/OaICAIdldQgp/FmD/vMP+1oFN9IXTf/W7Kh2dE1WHiXZ3fMHMIIjKgBwWLIAIKQzAcDosiJHjqusPkpyoucPkAw7OADgMDMBgN2iR0lt15POThLTzeEF7OAAgMPsDADMTCK3ayfJ6Z4/ZqevIzMQ4ACAw+wKAMweNdl1lOTkdHMqtRAPR1QA4DA7mv6lctRk106SUz1/7DheQ3AQ4ACAw6wOAFKtWrLzKMnu6eZOVGrRpNDfOKICABfEa/pXlMLxSqpHTXYfJdk53dzuSi2OvvyPAAcAXGJVAJDqUVN0J2nKsu0KSa2CHKuOkuyabm7F8Vq85OR4TQqjR180KfQHAhwAcJEVAUA6R01W7iQ5Kd3jtXg7NA9dW6ZH3/BOjyKkjgAHAHwu2VGTJJ3dpaOamiIxZ13ZeZRkl3SO1xLt0Nz78vaE70uTQv8gyRgAfC5R0nLUsU9P67bFf9Tw+etiVhdFd5KuH/hlVfTr7ungRko9UdtIcrIRNCn0PgIcAAiAeFVLbQWphDqVSq1kyclGWdWkEPbhiAoAAiJ61LTpr7Wa+tJ2HfvsdLtrvJpHkmo3YrPHa+nuvFjZpBD2IsABgADJ6hBSh1AoZnAT5bU8knRLss0kapvZebGrsgzO4IgKAALGT8Mune5GbLSL9DPfHmRbk0I4gx0cAAgYp4ddpsquYZ+JGO39M7a8WNeUF/uqsgytsYMDAAFjx6wrO5jpRmyleMnJBV2z9fNvf7FD47fKMrRGgAMAAePUsMt0uXmUNra8WA9d218FXTs2P1Z7skGPvrEzEBVmIMABgECye9ilFdw8Sluzo1pTX35XR0+2TsYOUhl9piMHBwACyusdiu0e9hmPG7k/cB4BDgAEmF3DLo1K1N/GiWGfsdZx5Hi9rZPI4Q0EOAAAWxjpb+PEsM9Y6zDCC2X0SB0BDgDAcokGWk5Ztr1VHpCdR2nx1mGE22X0SA8BDgDAUqnkuNhxlJZoHYkwjiEYqKICAFjKrf42ZtcRi5fK6JEednAAAJbyyqiIVF7fytwfO6Q6lDQTEeAAACzllVERRl//oWv7qzA3x/MBQ7pDSTMNR1QAAEt5YVREY1NETU0Rnd25Y9xrouuYdEWp58cxOD2UNAgIcAAAlnJ7VMSaHdUaPn+dblv8Rx377HTMa/yUa5MsaVs6k7Td2JRKrVhwEeAAACzn1qiIeDsdbVm5jsamiCr31Or1qoOq3FNreaDhlaRtvyEHBwBgC6dHRRgpCz+7c0f9/LbBuvwfrDmOciIvxitJ235DgAMAsI2ToyKMlIUf++y0OoRCtjYRjNXMMB1eSdr2G46oAACB4OROR8PnTfrXFTscyYvxQtK2H9ka4Dz++OMaNmyYunTporPPPjvmNfv379f48ePVtWtXFRYW6p//+Z/V0NCQ8HXr6+t13333qbCwUF27dtV1112njz76yIZPAADwC6d2OtbsqNZl/7ZWR0/G/62yMi/G7aRtv7I1wGloaNBNN92kKVOmxPx7Y2Ojrr32Wp08eVIbN27U8uXL9dprr+n73/9+wtedPn26VqxYoeXLl2vjxo06ceKEvvnNb6qxsdGOjwEA8IhECb1O7HSs2VGte5Zt1yeffm7oeqvyYtxK2vazUCQSsb2u7IUXXtD06dN17NixVo//9re/1Te/+U0dOHBAvXr1kiQtX75ckyZN0uHDh5WXl9futcLhsM455xz98pe/1C233CJJOnTokEpKSrR69Wpdc801SddTV1en/Px8hcPhmO8BAPAeIwm90bwYSa2Oj6JBTzrBQGNTREMeW6tjn8YuPY/lV5MvtzQHKdM7GZv5/XY1B6eyslLl5eXNwY0kXXPNNaqvr9e2bdtiPmfbtm06ffq0xowZ0/xYr169VF5ernfeeSfmc+rr61VXV9fqBgDwD6ON7uzc6di0p9ZwcGNXXkw0advrjQm9wNUqqpqaGvXs2bPVY926dVN2drZqamriPic7O1vdunVr9XjPnj3jPmfevHmaO3euNYsGADjK7HRyu8rTK/96xNT15MW4y/QOzpw5cxQKhRLetm7davj1QqH2/+VHIpGYjyeS6DmzZs1SOBxuvh04cMDUawMA3JNKozt7djqMvcZZOVnkxXiA6R2cadOmaeLEiQmv6du3r6HXKioq0h//+MdWj33yySc6ffp0u52dls9paGjQJ5980moX5/Dhwxo2bFjM5+Tk5CgnJ8fQmgAA3uJ0o7t4eS4V/brr6bc+TPr8hbcN0dcuOMeStSB1pgOcwsJCFRYWWvLmFRUVevzxx1VdXa3i4jOR7ptvvqmcnBwNGTIk5nOGDBmijh07au3atbr55pslSdXV1dqxY4d+/OMfW7IuAIB3ONnoLlEi8+iyIp3dpWPCPJxuXTpq2HnxfyMzPUnYSbbm4Ozfv19Hjx7V/v371djYqKqqKknSeeedp7POOktjxoxRWVmZ7rjjDj3xxBM6evSoZs6cqcmTJzdnRx88eFBXXXWVli5dqksvvVT5+fm666679P3vf1/du3dXQUGBZs6cqYsuukhXX321nR8HAOCCaPl3TfhUzDyckM4kEaeb0GukM/GPbrxI9/xvlVYs8268KG7A4sRYB3zB1iqqhx9+WIMGDdLs2bN14sQJDRo0SIMGDWrO0cnKytIbb7yhTp066YorrtDNN9+sCRMm6Mknn2x+jdOnT2vXrl369NNPmx/72c9+pgkTJujmm2/WFVdcoS5dumjVqlXKysqy8+MAAFzgRKM7oxO7R5cV6dnbB6sor/VuUXF+Jz2bIO/GaBUYrONIHxyvoQ8OAPiPnTsglXtqdevzm5JeF+1rY+aoqbEpouHz18VNlI7uQG18cBTHVUmY+f1m2CYAwBfsnE5uNpHZzBBRM1VgTg0mTSYIuUIEOAAA37BrOrmdicxOV4GlKyi5QkwTBwD4XqIZVUbYOcfKySqwdAUpV4gdHACAr1mx4xBNZJ6ybLtCij3HKtVEZjurwKw8SjLbMdrr2MEBAPiWlTsOds2xsqsKbM2Oag2fv063Pr9J9y+v0q3Pb9Lw+etS3mVJpWO0l7GDAwDwJTt2HOxKZI4GT213mopSzG0x0rPH7Gv6LVcoGQIcAIAv2VWdZFcis9ngKd7xk11HSX7KFTKCAAcA4Et+3HEwGjwlyivK75xtS2DnVMdop5CDAwDwpaDtOEQlyytau7PG0OuYDeyc6BjtJAIcAIAv2Vna7RYjIyNerzpk6LVSCezsSrR2A0dUAABfsrO02y1G8opqTzaooGtHfXLytC1HSXZ2jHYSOzgAAN8K0o6DZPxY6YaBX5Zkz1FSEMY0SOzgAEDGCMoPV1tB2XGQjB8rXV1WpEtKCywrO48KypgGiQAHADJCkH64YrGrtNtpZiqZsjqELA3s7Oit4yaOqAAg4II0XyjozFYyRQO76wd+WRX9uts2pkE601vH7IwvNxHgAECABfGHK+jcyCsK2pgGiSMqAAg0u7r9wl5O5xX5sWliMgQ4ABBgQfzhyhRO5hUFsWkiAQ4ABFgQf7jcFNRKtKCNaZAIcAAg0IL4w+WWIFeiBbFpIknGABBgQZsv5JZMqEQLWtPEUCQSybjU+bq6OuXn5yscDisvL8/t5QCA7YK8+2CVeMdPjU0RDZ+/Lm6ydnQXbOODowIRKHr5GM7M7zdHVACQAYLU7dcOiQLA/M7ZtlWieTGYCErTRAIcAMgQQfnhslqyDr7fuaKvodcxW4nGrpq9yMEBAGQsI40QV1QdNPRaZirRvJbT09gUUeWeWr1edVCVe2oD0fiRHRwAQMYy0gjx6MnTKuiarU9ONlhSiZYsqArpTHfp0WVFjhxXBXUniR0cAEDGMnqsNGFgL0nWVKJ5aSyC13aSrESAAwDIWEaPlUaXFVlWQu2V7tJBn1PGERUAIGOZaYSY1SFkSSWaV7pLm9lJurS0wHPVXskQ4AAAMpbZDr5WVKJ5pbu00R2i3++s0YxXq1oFQ2d37qj/e0VfTRt1vmcDHY6oAAAZzWwH33QrjrzSXdroDtHiP+xrt9Nz7LPT+tnvd2vIY2s9m6dDJ2M6GQMAZKzpnpUVR25XL0U7NCfaSQqFpGTxW0hybJSDmd9vAhwCHACAAfEaAkZDoFR+5N3uZBz9TFL74zkzwUGxQ6MqzPx+c0QFAEASdlUcRXN6rh/4ZVX06+54Pkui4zmjHZwl58razSDJGACAJMxUHPltHEa8OWWb9x7Vf/5hn+HXsbus3SwCHAAAkvBK7xq7xKoOi1Z7JQrsWrK7rN0sjqgAAEjCK71rnNSy2iuRkM7k4Nhd1m4WAQ4AAElEdzPiZch49UdeSq+sfWx5sZ69fbDO7tIx5t+dLGs3y9YA5/HHH9ewYcPUpUsXnX322e3+/t577+nWW29VSUmJOnfurP79++upp55K+rojRoxQKBRqdZs4caINnwAAAO/0rjFrzY5qDZ+/Trc+v0n3L6/Src9v0vD560z1rhlbXqxtPxyt7119gc7u3DrQSWVUhVNszcFpaGjQTTfdpIqKCi1evLjd37dt26ZzzjlHy5YtU0lJid555x390z/9k7KysjRt2rSErz158mQ98sgjzfc7d+5s+foBAIiKVhy17V1T5NHJ2/HK2qODNM0EJlkdQrr/6vM1bdR5vhnZ4EgfnBdeeEHTp0/XsWPHkl47depU/fnPf9a6deviXjNixAgNHDhQCxYsSGk99MEBAKTK7d41RkSb+MVLEI6Og3Cid42VfN0HJxwOq6Ag+RnmSy+9pMLCQg0YMEAzZ87U8ePHHVgdACDTud27xggzZe1B5aky8crKSr366qt64403El532223qbS0VEVFRdqxY4dmzZql9957T2vXro15fX19verr65vv19XVWbpuAAC8xM2ydq/scJkOcObMmaO5c+cmvGbLli0aOnSoqdf94IMPdP311+vhhx/W6NGjE147efLk5v9cXl6u888/X0OHDtX27ds1ePDgdtfPmzcv6ZoBAAgKt8ra3Z6v1ZLpHJwjR47oyJEjCa/p27evOnX64ktLloOzc+dOjRw5Unfffbcef/xxM8uRJEUiEeXk5OiXv/ylbrnllnZ/j7WDU1JSQg4OACCQjAzStDoHx45ZXW2ZycExvYNTWFiowsLClBfX1gcffKBRo0bpzjvvTCm4ib7G6dOnVVwc+4vLyclRTk5OOssEAMA3omXtU5Ztbzc4046y9samiOasjD+rK6Qzs7pGlxU5dlxla5Lx/v37VVVVpf3796uxsVFVVVWqqqrSiRMnJJ0JTEaOHKnRo0drxowZqqmpUU1NjT7++OPm1zh48KAuvPBCbd68WZK0Z88ePfLII9q6dav27dun1atX66abbtKgQYN0xRVX2PlxAADwhcamiPI7Z+s7V/RVt6729655et1u1dR5K6nZ1iTjhx9+WC+++GLz/UGDBkmS3nrrLY0YMUL/9V//pY8//lgvvfSSXnrppebr+vTpo3379kmSTp8+rV27dunTTz+VJGVnZ+u///u/9dRTT+nEiRMqKSnRtddeq9mzZysrK8vOjwMAgOfFyoMp6JqtCQN7aXRZkeVJv2t2VOtnv99t6FonZ3U50gfHa+iDAwAIIifyYFpK1m+nrV9Nvjytaeu+7oMDAADMa2yKaO6q+Hkw0pk8GDOzqJJJ1m+nJadndRHgAAAQAG409zNz5OT0rC4CHAAAAsCN5n5G++h87+oLHO+DQ4ADAEAAuNHc79LSAhXnd2o3Yb2lorwcTRt1nmXvaRQBDgAAAZAs2AjJ+jyYaL+d6Ou3fb+QpDnXDXBlVAMBDgAAAZAs2JDsyYMZW16shbcPVlF+650hO/rtmEGZOGXiAIAAcWselBNDNs38fhPgEOAAAALGKxO9rWbrLCoAAOBtWR1CaTXUCwJycAAAQOAQ4AAAgMAhwAEAAIFDgAMAAAKHAAcAAAQOAQ4AAAgcysQBAEDavNZ7hwAHAACkxa3uyYlwRAUAAFK2Zke1pizb3iq4kaSa8ClNWbZda3ZUu7IuAhwAAJCSxqaI5q7aqVgzn6KPzV21U41Nzk+FIsABAAAp2bz3aLudm5YikqrDp7R571HnFvW/CHAAAEBKDh+PH9ykcp2VCHAAAEBKeuR2svQ6KxHgAACAlFxaWqDi/E6KVwwe0plqqktLC5xcliQCHAAAkKKsDiHNHl8mSe2CnOj92ePLXOmHQ4ADAABSNra8WAtvH6yi/NbHUEX5nbTw9sGu9cGh0R8AAEjL2PJijS4ropMxAAAIlqwOIVX06+72MppxRAUAAAKHAAcAAAQOAQ4AAAgcAhwAABA4BDgAACBwCHAAAEDgEOAAAIDAIcABAACBQ4ADAAACJyM7GUciEUlSXV2dyysBAABGRX+3o7/jiWRkgHP8+HFJUklJicsrAQAAZh0/flz5+fkJrwlFjIRBAdPU1KRDhw4pNzdXoVDqg8Dq6upUUlKiAwcOKC8vz8IVoi2+a2fxfTuL79s5fNfOsvr7jkQiOn78uHr16qUOHRJn2WTkDk6HDh3Uu3dvy14vLy+P/6M4hO/aWXzfzuL7dg7ftbOs/L6T7dxEkWQMAAAChwAHAAAEDgFOGnJycjR79mzl5OS4vZTA47t2Ft+3s/i+ncN37Sw3v++MTDIGAADBxg4OAAAIHAIcAAAQOAQ4AAAgcAhwAABA4BDgpOiZZ55RaWmpOnXqpCFDhuh//ud/3F5SYL399tsaP368evXqpVAopN/85jduLymw5s2bp0suuUS5ubnq0aOHJkyYoF27drm9rEBauHChLr744uYGaBUVFfrtb3/r9rIyxrx58xQKhTR9+nS3lxJIc+bMUSgUanUrKipydA0EOCl45ZVXNH36dP3gBz/Qu+++q6997WsaN26c9u/f7/bSAunkyZP66le/qqefftrtpQTehg0bNHXqVG3atElr167V559/rjFjxujkyZNuLy1wevfurR/96EfaunWrtm7dqlGjRun666/XBx984PbSAm/Lli1atGiRLr74YreXEmgDBgxQdXV18+3999939P0pE0/BZZddpsGDB2vhwoXNj/Xv318TJkzQvHnzXFxZ8IVCIa1YsUITJkxweykZ4eOPP1aPHj20YcMGff3rX3d7OYFXUFCgJ554QnfddZfbSwmsEydOaPDgwXrmmWf02GOPaeDAgVqwYIHbywqcOXPm6De/+Y2qqqpcWwM7OCY1NDRo27ZtGjNmTKvHx4wZo3feecelVQH2CIfDks788MI+jY2NWr58uU6ePKmKigq3lxNoU6dO1bXXXqurr77a7aUE3u7du9WrVy+VlpZq4sSJ+utf/+ro+2fksM10HDlyRI2NjerZs2erx3v27KmamhqXVgVYLxKJaMaMGRo+fLjKy8vdXk4gvf/++6qoqNCpU6d01llnacWKFSorK3N7WYG1fPlybd++XVu2bHF7KYF32WWXaenSpbrgggv097//XY899piGDRumDz74QN27d3dkDQQ4KQqFQq3uRyKRdo8BfjZt2jT96U9/0saNG91eSmB95StfUVVVlY4dO6bXXntNd955pzZs2ECQY4MDBw7o/vvv15tvvqlOnTq5vZzAGzduXPN/vuiii1RRUaF+/frpxRdf1IwZMxxZAwGOSYWFhcrKymq3W3P48OF2uzqAX913331auXKl3n77bfXu3dvt5QRWdna2zjvvPEnS0KFDtWXLFj311FN67rnnXF5Z8Gzbtk2HDx/WkCFDmh9rbGzU22+/raefflr19fXKyspycYXB1rVrV1100UXavXu3Y+9JDo5J2dnZGjJkiNauXdvq8bVr12rYsGEurQqwRiQS0bRp0/TrX/9a69atU2lpqdtLyiiRSET19fVuLyOQrrrqKr3//vuqqqpqvg0dOlS33XabqqqqCG5sVl9frz//+c8qLi527D3ZwUnBjBkzdMcdd2jo0KGqqKjQokWLtH//ft1zzz1uLy2QTpw4oQ8//LD5/t69e1VVVaWCggKde+65Lq4seKZOnaqXX35Zr7/+unJzc5t3KvPz89W5c2eXVxcs//qv/6px48appKREx48f1/Lly7V+/XqtWbPG7aUFUm5ubrtcsq5du6p79+7kmNlg5syZGj9+vM4991wdPnxYjz32mOrq6nTnnXc6tgYCnBTccsstqq2t1SOPPKLq6mqVl5dr9erV6tOnj9tLC6StW7dq5MiRzfej57d33nmnXnjhBZdWFUzR1gcjRoxo9fiSJUs0adIk5xcUYH//+991xx13qLq6Wvn5+br44ou1Zs0ajR492u2lAWn76KOPdOutt+rIkSM655xzdPnll2vTpk2O/k7SBwcAAAQOOTgAACBwCHAAAEDgEOAAAIDAIcABAACBQ4ADAAAChwAHAAAEDgEOAAAIHAIcAAAQOAQ4AAAgcAhwAABA4BDgAACAwCHAAQAAgfP/AWu8cmcZoMiAAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt \n",
    "\n",
    "plt.scatter(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 2.1 - Implement polynomial regression using NumPy with gradient descent.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model(w0, w1, x):\n",
    "    return w0 + w1 * x \n",
    "\n",
    "def grad(w0, w1, x, y):\n",
    "    y_pred = model(w0, w1, x)\n",
    "    return 2 * (y_pred - y), 2 * (y_pred - y) * x \n",
    "\n",
    "def gradient_descent(X, Y, grad_func, epoch=1000, lr=0.1):\n",
    "    w0 = 0.0\n",
    "    w1 = 0.0\n",
    "    for i in range(epoch):\n",
    "        dw0 = 0\n",
    "        dw1 = 0\n",
    "        for x, y in zip(X, Y):\n",
    "            tmp0, tmp1 = grad_func(w0, w1, x, y)\n",
    "            dw0 += tmp0 \n",
    "            dw1 += tmp1 \n",
    "        dw0 /= len(X)\n",
    "        dw1 /= len(X)\n",
    "        w0 -= lr * dw0 \n",
    "        w1 -= lr * dw1 \n",
    "    return w0, w1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0177232162357135 -2.9400863075081993\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABFCklEQVR4nO3dfXhU5b3v/88kQkAMUYiQoJGkqNUYlacKQVRQQSRF0O4qVN1ytuZUBCtSjgatQqxKqHZXe6z4UDeKKOD+WYr8glQqorWG8hgromxNE0FIREASQEkwmfMHTkwys2ZmzaxZa2bN+3VduXazsmbmnunW+fa+vw8er9frFQAAgIukOL0AAAAAqxHgAAAA1yHAAQAArkOAAwAAXIcABwAAuA4BDgAAcB0CHAAA4DoEOAAAwHWOc3oBTmhpadHu3buVnp4uj8fj9HIAAEAYvF6vDh48qD59+iglJfgeTVIGOLt371ZOTo7TywAAABHYuXOnTj311KD3JGWAk56eLunYB9S9e3eHVwMAAMLR0NCgnJyc1u/xYJIywPEdS3Xv3p0ABwCABBNOeglJxgAAwHUIcAAAgOsQ4AAAANchwAEAAK5DgAMAAFyHAAcAALgOAQ4AAHAdAhwAAOA6Sdnoz2nNLV6tr96vPQePqFd6F12Q10OpKczEAgDAKgQ4Nlu1tValK7aptv5I67XsjC6aPS5fYwqyHVwZAADuwRGVjVZtrdWURZvbBTeSVFd/RFMWbdaqrbUOrQwAAHchwLFJc4tXpSu2yRvgb75rpSu2qbkl0B0AAMAMAhybrK/e77dz05ZXUm39Ea2v3m/fogAAcCkCHJvsOWgc3ERyHwAAMEaAY5Ne6V0svQ8AABgjwLHJBXk9lJ3RRUbF4B4dq6a6IK+HncsCAMCVCHBskpri0exx+ZLkF+T4fp89Lp9+OAAAWIAAx0ZjCrI1/4aByspofwyVldFF828YSB8cAAAsQqM/m40pyNao/Cw6GQMAEEMEOA5ITfGosF9Pp5cBAIBrcUQFAABcx/EAZ+7cufrRj36k9PR09erVSxMmTND27duDPmbt2rXyeDx+Px9//LFNqwYAAPHM8QDn7bff1tSpU7Vu3TqtXr1a3377rUaPHq3Dhw+HfOz27dtVW1vb+nPGGWfYsGIAABDvHM/BWbVqVbvfFyxYoF69emnTpk26+OKLgz62V69eOvHEE2O4OgAAkIgc38HpqL6+XpLUo0fohncDBgxQdna2LrvsMr311luG9zU2NqqhoaHdDwAAcK+4CnC8Xq9mzJih4cOHq6CgwPC+7OxsPfPMM3r11Vf1pz/9ST/84Q912WWX6Z133gl4/9y5c5WRkdH6k5OTE6u3AAAA4oDH6/V6nV6Ez9SpU1VeXq53331Xp556qqnHjhs3Th6PR6+99prf3xobG9XY2Nj6e0NDg3JyclRfX6/u3btHvW4AABB7DQ0NysjICOv7O252cG6//Xa99tpreuutt0wHN5I0dOhQffLJJwH/lpaWpu7du7f7AQAA7uV4krHX69Xtt9+uZcuWae3atcrLy4voebZs2aLsbEYdAACAOAhwpk6dqpdfflnLly9Xenq66urqJEkZGRnq2rWrJGnWrFnatWuXFi5cKEl67LHHlJubq3POOUdNTU1atGiRXn31Vb366quOvQ8AABA/HA9w5s+fL0kaMWJEu+sLFizQ5MmTJUm1tbXasWNH69+ampo0c+ZM7dq1S127dtU555yj8vJyjR071q5lR625xcs8KgAAYiSukoztYiZJKRZWba1V6Yptqq0/0notO6OLZo/LZ6I4AAAGEjLJOFms2lqrKYs2twtuJKmu/oimLNqsVVtrHVoZAADuQYBjo+YWr0pXbFOgLTPftdIV29TcknSbagAAWIoAx0brq/f77dy05ZVUW39E66v327coAABcyPEkYzcJlTi856BxcNNWuPcBAIDACHAsEk7icK/0LmE9V7j3AQCAwDiiskC4icMX5PVQdkYXGRWDe3QsKLogL/SgUQAAYIwAJ0qhEoe9+j5xODXFo9nj8iXJL8jx/T57XD79cAAAiBIBTpRCJQ5L7ROHxxRka/4NA5WV0f4YKiuji+bfMJA+OAAAWIAcnCiFmxA86dl1+v2kAbrq/D4aU5CtUflZdDIGACBGCHCiZCYh+BeLt+gXi7fok4euVKfUFBX269nu74xvAADAGgQ4UfIlDtfVHwmYh+ORdFyqR0ebv//rGfe+rn4nd9ObvxzReo3xDQAAWIccnCiFkzj8fycN0IZ7L2/3t6ovDyu3pFzVew8zvgEAAIsxbNOiYZvh7sD8+P/+TVt3NYT9vB5JPbp11q+KzlZWRleOrQAAScvM9zcBjoXTxI1yaDpeH9T3JJ35q9cjeg2OrQAAyYoAJ4RYBTiBrPxnrX61fKv2H25qveYLUl7Z+LnWfLzH1PP59m4oKQcAJBsz39/k4MTQ3JXbdNvLm9sFN9KxvjhTFm3WtYNP1eLioaaek6njAACERoATIyv/uVtPv1Nt+Hdfh+NBfU9Sdoa52VNMHQcAIDgCnBhobvHqV8u3hryvtv6INn32lWaPy5dH/lVYoTB1HACAwAhwYmB99X7tP3w0rHv3HDxiOL4hFKaOAwAQGI3+YsDMzoovSGk7vqGu4Yh+/f9/aBgkeXRsdhVTxwEACIwAJwbC3Vnp2a1zuyAlNcXTOr6ha6cUTVm02XBKeXZGF/rhAABggCOqGPCNbwjl1+MLAgYpzS1eZXTtrP+4MFc9unUK+NjNOw4ot6Q86rUCAOBG7ODEgG98g9EOjCT9/OI8jT3Pv49NoI7IPbp11oT+fTQqP0uTnl3X7n5fkFNTVmTZ+gEASHTs4MSIL3G4405Oj26d9OTPBmjW2Hy/xxjNpPrqcJMW/L1G9d80qaasSDcPz/N7bG5Jueq/CS+xGQAAt6OTcYw7GRuNbwh03/B5a/yCGx9fYvG7d1/a+nijIyp2cwAAbkQn4zjiSxwe3/8UFfbraZgYvL56v2FwIwVu7ldTVqQl/9u/E3JuSbmWV+6Keu0AACQqApw4EW5pecf7hv6gZ8AdmzuWVJKEDABIWgQ4cSLc0nKj+2rKivQ/D17pdz23pFwXlq2Jam0AACQaApw44SstN+ps49Gx3jfBmvt1Pi4l4G7OrgPfuHI3p7nFq4qqfVpeuUsVVfsYPgoAaEWScYyTjM3wVVFJalde7gt65t8wUGMK/EvLjZhNQg43IToeBCqnz87ootnj8gN+Ron03gAAgZn5/ibAiaMARzL/xR1K8cKNWr3tC7/rlfeP0onHd47Z68aSLxDs+P+4RoFgIr03AIAxApwQ4jnAkWKz2xBsN8dswOAks+X0ifTeAADBEeCEEO8BTqysr96va5+u8LvevctxajjybcDHBOq/46SKqn1+3ZwDWVw8VBfk9TDdWwgAEL/og4OALsjrETD/xii4kQL333GSmXL6SHoLAQDcgQAnCRmVlAcTbmARa2bK6SPtLQQASHwEOEnKqKTcSLiBRayZKaePtrcQACBxEeAkICv7v1Q9PNZvIGhb4fTfsZNvUrskvyDH9/vscflKTfFY0lsIAJCYCHASzKqttRo+b40mPbtOdyyp1KRn12n4vDVatbU2oufzBQxGQYBX0p2XnxFXSbi+Se1ZHQKzrIwu7aqizARDAAB3oYoqgaqoYlnyHKhXTEfxNqU83HJ6+uAAgDskXJn4k08+qUceeUS1tbU655xz9Nhjj+miiy4yvP/tt9/WjBkz9OGHH6pPnz666667dOutt4b9eokY4Jjt/xLpa/gChrr6I5r7+sd+99w95ixNGdEvoud3Ep2MASDxJVSZ+NKlSzV9+nTde++92rJliy666CJdeeWV2rFjR8D7q6urNXbsWF100UXasmWL7rnnHv3iF7/Qq6++avPK7WVHyXNqikeF/XpqfP9T9PNL+gXcsZm36uOEnGvV9r0V9utJcAMALuf4Ds6QIUM0cOBAzZ8/v/Xa2WefrQkTJmju3Ll+999999167bXX9NFHH7Veu/XWW/X++++rosK/iV0gibiDs7xyl+5YUhnyvscn9tf4/qdY+tpHm1t0xr2vB/xbpMdW7KgAAMwy8/19nE1rCqipqUmbNm1SSUlJu+ujR4/We++9F/AxFRUVGj16dLtrV1xxhZ577jkdPXpUnTp18ntMY2OjGhsbW39vaGiwYPX2crLkuVPqsZLyQDs3uSXlpoMcp3JiCKoAIHk4GuDs3btXzc3N6t27d7vrvXv3Vl1dXcDH1NXVBbz/22+/1d69e5Wd7f8FOXfuXJWWllq3cAf4Sp7r6o/4JRlL3+fgxLLk2RfIdAx0fL+HE+gYJUrX1R/RlEWbYzYbikRjAEgujufgSJLH0/5/RXu9Xr9roe4PdN1n1qxZqq+vb/3ZuXNnlCu2XzyVPNeUFem8UzP8rueWlOuLBuM8oeYWr0pXbAsYoPmula7YFlVfn0B8QVXHHCZfUBVpiT0AIH45GuBkZmYqNTXVb7dmz549frs0PllZWQHvP+6449SzZ8+Aj0lLS1P37t3b/SSicPu/2OG1acMD7tgMefhNwyRkJ2ZDORVUAQCc5egRVefOnTVo0CCtXr1aV199dev11atXa/z48QEfU1hYqBUrVrS79sYbb2jw4MEB82/cZkxBti49q7derKjRZ/u/Vt8ex+vGwlx1Ps6ZWLWmrEjvfbpXP/vjP9pdzy0p1y9HnanbLzuj9ZoTs6HMBFWF/fwDZCvzdsgBAgD7OBrgSNKMGTN04403avDgwSosLNQzzzyjHTt2tPa1mTVrlnbt2qWFCxdKOlYx9cQTT2jGjBkqLi5WRUWFnnvuOS1evNjJt2GbQLkkf3y32tFckmGnZwZMQv7t6v/Rb1f/T+tOjxOJ0tEEVVbm7ZADBAD2cjzAue6667Rv3z498MADqq2tVUFBgVauXKm+fftKkmpra9v1xMnLy9PKlSt155136g9/+IP69Omj3//+9/rJT37i1FuwjVMJuuGqKSsKWFLuC3x8c6+C7ahI0leHmyxbU6RBlZWftdFz1dYf0a2LNuvmC3N1eX5WWDs67AIBQHgc74PjhETsg2NHJ2MrGeXhPPmzAbrt5S1BH5tt4fvwfW6hqs/avp6Vn3Wo52or1I4Ou0AAkl1CdTJGeJxI0I1GTVlRwCTkUMGNZO37iKT6zMrPOtRztRWsqotKMAAwhwAnQTiRoGuFmrIiDep7kunHWfk+zFafWflZm3kfRlVdVIIBgHmO5+AgPE52Mo7Wq1OGSTI+tgrE6vcxpiBbo/KzwspfsfKzNvs+AlV1RVsJBgDJiAAnQcRDJ+NoGZWUtxXL9+EbuBmKlZ91qOcy0nbnJ1F37wDASRxRJYh46mQcjWGnZ+qpGwYGvcfp92HlZx3suYJpu/OTyLt3AOAUApwEEk+djKMxpiBbT90wUFnd0/z+5pV066LN9i+qAys/a6PnCsSjY5VRbXeHfLtARgFSoMcAQLKjTDxBysTbcksvFN/7mPTsuoB/NzulPBZi0cl49bY6/dffa+SR2h1b+Z41UADlq6KSiccAgNuY+f4mwEnAAMetjJKQ4yHQsVokPW3ogwMg2RHghECAE78uLFujXQe+8bv+9v8Zob49uzmwInPM7PhEsjvklt07AIgEAU4IBDjxL1a7Ob4Aoa7hiPYfalSPbp2VldHVkkCBHRYAiC0CnBAIcBLDO//zpf79v9b7Xb928Kn6zb+db/r5AgUgPtEGIkbzpsiRAQDrMKoBrnDxmScH3LF5ZePnppoGSsajDnxqoxh5QKdhAIg/BDiIezVlRap6eKzf9dyS8rACnWABSFteRRaIJNqcMABIBgQ4SAipKR7D/JtQQY6ZgZeRBCJ0GgaA+EOAg4RiNKU82G6O2cDC7P10GgaA+EOAg4RUU1akvEz/svHcknL968tD7a6ZDSzM3k+nYQCIPwQ4SFhvzRwRcDfn0t++3W43xxeAhCOSQMQtc8IAwE0IcJDwasqK9NItQ/yu55aUq3jhRqWmeHRfUX5Yz3Vf0dkRBSJumRMGAG5BHxz64LiKUR7O4uKhhjOvOt5X2K9nxK9Pp2EAiB0z39/H2bQmwBY1ZUVqbvGq3z0r210PJ7iRoq90Sk3xRBUgAQCswREVEk5zi1cVVfu0vHKXKqr2+fWtCVZSHkrmCWlWLBEA4DB2cBAXwj3aMTPvyRfkmOp6nHQHtgDgTgQ4cFy4QYvRvKe678YsGCXz1pQV6bw5f1HDkW9DruUv2+qUkuIhdwYAEhxHVHCU0Yyoug6zoaKd9/T0jYPDWs/Cis806dl1Gj5vTURzqQAA8YEAJ8mEyl+xey3hBi3RznsK1Yyvo44BFgAgsXBElUTM5K/YwUzQEu28J18zvimLNsuj0Kk2Xh1r0le6YptG5WdxXAUACYYdnCQR7lGQncwELVbMezJqxmeEKeAAkLgIcJJAtPkrsWImaLFq3tOYgmy9e/elWlw8VP9e2Des12cKOAAkHgKcJBBt/kqsmAlarJz35GvGd2WYx3J3LKkM6z4AQPwgwEkC0eavxIrZoMXqeU9mEo9zS8q17l/7TD0/AMA5JBknASvyV2LFF7R0TH7OMkh+HlOQrVH5WZbMewqWeBwoEXniM8fGPUTaJRkAYB+GbSbBsM3mFq+Gz1ujuvojAfNwPDoWULx796WOVQs5OaQyWHXZ0Wavbl+8JeDjCHQAwF5mvr8JcJIgwJG+r6KS/HcqJEV0xOMmoQIso3EPBDkAYB8CnBCSMcCR4q8PTqJpafHqBx2mlPsQ6ABA7BHghJCsAY7k7FGQWxjt5lTPHSuPx8NnDAAxQoATQjIHOLCOUaCTndGFXTIAiAEz39+UiQMRMjqWiqdu0QCQrAhwgCjUlBWFzL9xsls0ACQrxwKcmpoa3XzzzcrLy1PXrl3Vr18/zZ49W01NTUEfN3nyZHk8nnY/Q4cOtWnVQGCLi4P//yBzrQDAXo41+vv444/V0tKip59+Wqeffrq2bt2q4uJiHT58WI8++mjQx44ZM0YLFixo/b1z586xXi4QVLx2iwaAZOVYgDNmzBiNGTOm9fcf/OAH2r59u+bPnx8ywElLS1NWVlaslwiELdwu0HcsqdT4/qfEeDUAgLjKwamvr1ePHsGnQUvS2rVr1atXL5155pkqLi7Wnj17gt7f2NiohoaGdj+AlczOtbKzeLG5xauKqn1aXrlLFVX7yAMCkBTipky8qqpKAwcO1G9/+1vdcssthvctXbpUJ5xwgvr27avq6mrdd999+vbbb7Vp0yalpaUFfMycOXNUWlrqd50ycVgpWLdoo3/IYt0gkOaOANzE0T44RsFEWxs2bNDgwYNbf9+9e7cuueQSXXLJJfrjH/9o6vVqa2vVt29fLVmyRNdcc03AexobG9XY2Nj6e0NDg3JycghwYLlgAcWt3wU/HT15/UCNPdf6YMMXcHX8B5zxHAASlaMBzt69e7V3796g9+Tm5qpLl2M5C7t379bIkSM1ZMgQPf/880pJMX9qdsYZZ+iWW27R3XffHdb9NPpDLEUz18qqLsi+Aasde/L4xMOAVQAwy8z3t+VJxpmZmcrMzAzr3l27dmnkyJEaNGiQFixYEFFws2/fPu3cuVPZ2fwvUcSH1BSPCvv1NPx7TVmRllfu0h1LKttdDxT4RHqctL56v2FwI7UvWw+2VgBIVI4lGe/evVsjRoxQTk6OHn30UX355Zeqq6tTXV1du/vOOussLVu2TJJ06NAhzZw5UxUVFaqpqdHatWs1btw4ZWZm6uqrr3bibQARGd//lLDybyLtgkzZOoBk51iZ+BtvvKFPP/1Un376qU499dR2f2t7arZ9+3bV19dLklJTU/XBBx9o4cKFOnDggLKzszVy5EgtXbpU6enptq4f8SnRBl3WlBXp2+YWnX7v6wH/7tWx46TSFds0Kj8r7PcSbtl6uPcBQKJxLMCZPHmyJk+eHPK+tsFO165d9Ze//CWGq0IiS9SKoQ01XwX9eyTHSb6y9br6IwEruHw5OBfkhW7LAACJKK764ACR8lUMJeKgy1gcJ6WmeDR7XL4k+fXm8f0+e1x+XO9uAUA0CHCQ8JpbvCpdsS3gTkUiDLo00wX51U2fh/28YwqyNf+GgcrKaP/8WRldKBEH4Hpx0+jPTpSJu0tF1T5NenZdyPsWFw+Ny4ohX0m30XFSIGYaBCZaXhIAGHG0TBywW6JXDPmOk6Ys2uzX9dioC7KvpDycQCdU2ToAuBFHVEh4bqgYCnac9NQNAw0DGaOmgQCQ7Dii4ogq4YU64omXrr3hHBWFusfr9Spv1sqAzx/ruVYA4DRHRzUkAgIc9wk26FJyfu6S1SXsRjs3/3p4rFLIrwHgUgQ4IRDguFO89sGJ5dDLYHOtAMBtCHBCIMBxL7MVQ7GuMIp26GU46zMKcsquOVcTLzgt6vcAAPGCACcEAhxI9uz4RFPCbnZ97OYAcDsz399UUSEp2dX5ONIS9kjWV1NWpKduGOh3PbeknGorAEmHAAdJx87Ox5GUsEezvjEF2ZSUA4AIcJCE1lfvN8yJkdoPt4yWb+ilUVaPR8eOndoOvbRifTVlRaqeO9bvOrs5AJIFAQ6Sjp2djyMZemnV+jweT9DdnHidzQUAViDAQdKxu/Ox2aGXVq+vpqwoYKDT756V7OYAcC1mUSHp+I6NQnU+bntsFK0xBdkalZ8VVkl6rNZXU1YUMKDJLSnXrycU6MahfU09HwDEM8rEKRNPSonQ+TiW6wtWUs70cQDxij44IRDgQIrfzsc+sV7fmx99oZtf2Bjyvnj6TAAkNwKcEAhw4BPvuxV2rC9UHo7RrpFVa4v3/w4AxA8CnBAIcID2mlu86ndP4Cnlkv9IiVC7S+EGLfG+iwYgvpj5/ibJGEDInj9te+/Uf9MUcHior8vy/744T6+9XxsyaDEaQup7HqfzoAAkNsrEAYTde6eu/pugXZa9kp5+pzrkiAk7u0kDSE4EOADC7qlz5yvvB+2ybKRj0BKLbtLNLV5VVO3T8spdqqjaR3AEJDmOqACE7L1jhbZBi9XdpMnlAdAROzgAwhopYRVf4nE4wrnPrsnwABILAQ4AScFHSjx1w0DDuVZm+aqqzA4hDYRcHgBGOKIC0CrUSImasiKt2lqrW7/rsmxG2xETvh2jKYs2y6PA3Zo7DiENxEwuT2G/nqbXDCBxsYMDoJ3UFI8K+/XU+P6n6IK8Hlpfvb9d4u6Ygmw9dcPAoM8RzuR0s0NIA7FzMjyAxMIODoCAQiXuVj08Vuur92vSs+v8HtvxQCjLIOHXzBDSQOyeDA8gcdDJmE7GiJCbRwwYNeEzGttgNO5h7LlZunFobsw+m+YWr4bPWxNy8rqvAzOAxMaohhAIcBAtN5cl+4IGo9yWYEFDsCnlsRLvk+EBWMfM9zc5OIBJbi9LjqYJX01ZkV74jwv8rueWlIcc6hkpK3J5ALgPOTiACaHKkj06VpY8Kj8rYY9Eok3cveTMk1VTVhQwoMktKY/Jbk60uTwA3IcABzAhGcqSrUrc9QUyHQMd3+9WBzq+6i8AkDiiAkxJhrJkq5rw+RgFMrkl5Wr6tiWyRQJACAQ4gAnJUJYcztiGcJrwtVVTVhQw0DnzV6/HLDcnGAZzAu5HFRVVVDAhmcqSY1UpZhTQ3HpJP5VceVbEzxsuN1fAAW5HmXgIBDiIRjKVJcei10+gAKMtO0rKw+3vAyC+EOCEQICDaLELEBmjACOQYIFOuIFX2/syT0jTL1+pVF1DY8DndNPuG+BWCRPg5Obm6rPPPmt37e6771ZZWZnhY7xer0pLS/XMM8/oq6++0pAhQ/SHP/xB55xzTtivS4ADK7i5k3EshGogGEigICfc4DLUTpGRxcVDqcYC4pSZ72/Hy8QfeOABFRcXt/5+wgknBL3/N7/5jf7zP/9Tzz//vM4880w9+OCDGjVqlLZv36709PRYLxdoRVmyOaFK7APpWFJutAPka7LoO2Iys1PUUSJXwAH4nuNVVOnp6crKymr9CRbgeL1ePfbYY7r33nt1zTXXqKCgQC+88IK+/vprvfzyyzauGoBZ0QQOuSXl+rrx26BNFqVjTRabvm0xvC8ciVwBB+B7jgc48+bNU8+ePdW/f3899NBDampqMry3urpadXV1Gj16dOu1tLQ0XXLJJXrvvfcMH9fY2KiGhoZ2PwDsFW7gsLh4aMCjqfzZfwmryeKLFTWmd4ok8/19AMQ3RwOcO+64Q0uWLNFbb72ladOm6bHHHtNtt91meH9dXZ0kqXfv3u2u9+7du/VvgcydO1cZGRmtPzk5Oda8AQBhM9tAMNJqqs/2f236MZH29wEQvywPcObMmSOPxxP0Z+PGjZKkO++8U5dcconOO+883XLLLXrqqaf03HPPad++fUFfw+Np/y8gr9frd62tWbNmqb6+vvVn586d0b9RAKZE0kDQqEFgMH17HG96bQzmBNzH8iTjadOmaeLEiUHvyc3NDXh96NChkqRPP/1UPXv6J29mZWVJOraTk539/b+I9uzZ47er01ZaWprS0tJCLR1AjPkmf3esbsoKUWJfU1akv3+6V9f/8R+Gz+0r876xMFd/fLc6ZDPGR//tfO093EgFHOBSlgc4mZmZyszMjOixW7ZskaR2wUtbeXl5ysrK0urVqzVgwABJUlNTk95++23NmzcvsgUDsFWkk78vPD1TT90wULd+12QxkNnj8tX5uBTNHpevKYs2y6PAzRhnj8vXhWdE9u8pAInBsRyciooK/e53v1NlZaWqq6v1yiuv6Oc//7muuuoqnXbaaa33nXXWWVq2bJmkY0dT06dP18MPP6xly5Zp69atmjx5so4//nj97Gc/c+qtADDJV2I/vv8pKuzXM+zdkzEF2XrqhoHKzvBPWPZKrcGPb6coq8N9HEUBycOxPjhpaWlaunSpSktL1djYqL59+6q4uFh33XVXu/u2b9+u+vr61t/vuusuffPNN7rttttaG/298cYb9MABkkTbHaBJz67z+3tuSbk+emBM2DtFidiwMRHXDNiNUQ10MgYSntEAz1AJyok4ciMR1wxYJWFGNTiFAAdwH6MgZ9IFObrq/FP8djsScfBmIq4ZsBIBTggEOIB7GQU6PtkZXXRf0dn6dflHhg0B43HwZqhZXvG4ZsBqZr6/He9kDABWqikr0qtThhn+va7+iG57eUtYXZHXV+9vvdbc4lVF1T4tr9yliqp9am6x938bhprlFWjNQDJzfNgmAFitf86Jys7oEjAgMBOW+OZnxUPeS7izvBgWChzDDg4ASzm90yFFNrk8kF7pXVrzXjo+n2+C+aqttVG/TrhrsfI+wO3YwQFgmXjY6ZCi38Xw5bMM6nuSLnnkLcMJ5h4dm2A+Kj8r5nkvvlleoTo0MywUOIYdHACWiJedDim6XYy23Y43ffZV3OS9RDLLC0hmBDgAotbc4lXpim2GOx3SsZ0Ou46rQk0uD6Ztt+N4y3uhQzMQPo6oAETNTIVPYT//QbpW8+12BJtHNd9grlVt/ZHWQCEe814ineUFJBt2cABELd52OqTwdjtqyor0p9v8S8pzS8qVW1IecifIo2M5RnbnvUQ6ywtIJuzgAIhaPO50SOHtdgw87STVlBUFbBDY756VeuqGgSEnkxNgAPGHAAdA1OK5wse32xGKb25Vx0DHd4zVsa9OFvOfgLhGgAMgauHkvCTKTofRbk5t/RE9d9NgHWr8lrwXIAEwi4pZVIBl4qUPjlUinVIOIDYYthkCAQ4QO80tXksrfKx+PrOMgpxh/Xrq5eKhtq0DAAFOSAQ4QGJwekfIF1zVNRzRnUsrA97Dbg5gHwKcEAhwgPjn64zc8V9QbfvYxDLICRRcBUOgA8Seme9v+uAAiDtOd0Y2GjsRjNFRFgBnEOAAiDtmOiNbLVhwFYqvQSAA5xHgAIg7TnZGDhVchSO3pFxfHW6yaEUAIkGAAyDuONkZ2WzQ9PjE/gHzbwb8ejW7OYCDCHAAxB0nZ0CZDZp89xslGeeWlOsn89+Lel0AzCHAARB3fJ2RJfkFObHujOwLrsLRMciqKSsKGOhs+uwrdnMAmxHgAIhL4UwDj4XUFI+uOj/0c3tkHGTVlBWp/BfD/a5bmYTc3OJVRdU+La/cpYqqfTGrKAMSFX1w6IMDxDW7Oxk3t3g1fN6aoInGKR7piUkDNPa8PiGfLxbjHpxugAg4xcz3N8M2AcS1cKeBWyWcKqoWr3RSt7Swns9oSrnvd7OBjlEDxLr6I5qyaHPMGyACiYIjKgBoI1Yl6sGSkPcdagzrOZxugAgkEgIcAGgjliXqRknIgx78a1i5OU42QAQSDQEOALRhR4l6sN2cq5/8u+HjnGyA2BFJzoh35OAAQBu+EvUpizbLI7U7DrKyRN0oN2fLjgPKLSkPGAQ52QCxLZKckQjYwQGADuwsUa8pK9JfZ1zsdz1QSbmTDRB9jAaR+pKcV22tjdlrs2sEMygTp0wcgAG7S9TDKSn3BRhS4N2lWFZRhSqh9+hYEPju3Zda/jmxawTJ3Pc3AQ4BDoA4EyrQcerLvqJqnyY9uy7kfYuLh1pa2m9UGm9HUIf4Qh8cAEhgVQ+PVb97Vvpdzy0p1/p7LtOYgmyNys+ydXdJcibJOVRpvEfHSuNH5WfF/P0jsRDgAEAc8e3OGLng4TclHdvNsbMBouRMkrOZ0ni7Pw/EN5KMASBOGCXwBpJbUq7Lfrs29otqw4kk53gqjUdiIcABgDgQ7ChG8p+qLklVXx62dUq5E1Pe46U0HomHAAcA4kA4RzGS9OhPz/f7W9uS8liXUts95T0eSuORmBzLwVm7dq1GjhwZ8G/r16/Xj370o4B/mzx5sl544YV214YMGaJ160Jn9gNAvAr3iKVTqkc1ZUUBd25yS8qVndEl5tVVdiY529V4Ee7j2A7OsGHDVFtb2+7nlltuUW5urgYPHhz0sWPGjGn3uJUr/asNACCRmD2KMZprZVcDPt+U9/H9T1Fhv54xDTDs3jWCOzi2g9O5c2dlZWW1/n706FG99tprmjZtmjye4P+gpKWltXssACQ631FMXf2RgHk4viZ6HY9ijHZzfNxSSu1UaTwSV9zk4Lz22mvau3evJk+eHPLetWvXqlevXjrzzDNVXFysPXv2BL2/sbFRDQ0N7X4AIJ5Ek8C7uHho0Od2y5RxO3eNkPjiJsB57rnndMUVVygnJyfofVdeeaVeeuklrVmzRr/97W+1YcMGXXrppWpsbDR8zNy5c5WRkdH6E+o1AMAJkR7FUEoN+LN8VMOcOXNUWloa9J4NGza0y7P5/PPP1bdvX73yyiv6yU9+Yur1amtr1bdvXy1ZskTXXHNNwHsaGxvbBUANDQ3KyclhVAMAR4SacWV2Bla4IxQkBczbARKFo6Mapk2bpokTJwa9Jzc3t93vCxYsUM+ePXXVVVeZfr3s7Gz17dtXn3zyieE9aWlpSktLM/3cAGC1cOZI+Y5iwhUqf6ctX74OgQ7czvIAJzMzU5mZmWHf7/V6tWDBAv37v/+7OnXqZPr19u3bp507dyo7myx6APHNaGikr9Ip0oqgUKXUgYKe3JJyghy4muM5OGvWrFF1dbVuvvnmgH8/66yztGzZMknSoUOHNHPmTFVUVKimpkZr167VuHHjlJmZqauvvtrOZQOAKaGGRkrHKp0ibcwXLH/nqRsGBgxm2jYIBNzG8WGbzz33nIYNG6azzz474N+3b9+u+vp6SVJqaqo++OADLVy4UAcOHFB2drZGjhyppUuXKj093c5lA4ApdgyNDFVKHaxB4N/uGqmcHsdH9LpAPLI8yTgRmElSAgArLK/cpTuWVIa87/GJ/TW+/ykxX4/Rzg3HVohnZr6/HT+iAoBkEG9DI2vKipR5Qme/67kl5frRQ3+1ZQ1ALBHgAIAN4nFo5MZfjQq4Y/PlwUZyc5DwCHAAwAbRdCqOtZqyIr17t//wYyuSkGM93RwwQg4OOTgAbBSoD07Pbp01vn8fjcrPcny+kpW5OeH0/AHMMPP9TYBDgAPAZr5OxX/dVqdllbu0//DR1r/FSwAQbaBj1PPHF7oxBRyRIMkYAOJYaopH9d806b/+XtMuuJG+b/q3amutrWvqeJRU9fDYgPfllpSrZu/hkM8Vy54/QDgc74MDAMkmVADg0bEAYFR+li3HVUZHSU99t8vScTdnxKNrJRnv5tjR8wcIhR0cALCZmQAg1nxHSR3X03YnqaasSKcFaAKYW1Kuc+5f5Xed6eaIBwQ4AGCzWAYAZqqWzBwlvXPXyIA7Noebmv12eOzu+UOlFgLhiAoAbBarAMBs1VIkR0k1ZUWqqz+ioXPfbHdv2ynloaabe3RsRpYVPX+o1IIRdnAAwGaxaPoXzlFTR5HuJGVldDHMv8ktKbet508k7xnJgwAHAGxmdQAQadVStDtJNWVFhlPKb1202XC6uRUl4nZUanH0ldg4ogIAB4wpyNb8Gwb6Ha9kRXC8EmnVklVHSUZTym9dtFl/nXGxvjzYFHC6eTRiXanF0VfiI8ABAIeMKcjWqPwsra/eH1UAEOlRk28nacqizfJI7YIcsztJvp2cjoHO5f/5Tru/W8WKRG1fw8WOn71Rk0Lf0RdNChMDAQ4AOCg1xRN1L5hojpqs3EmSjgUyVz3xrv75eX2767kl5Tq+c6q2PTDG1PMZifZ4zWiH5r6ifP26PH56FCFyBDgAkOBCHTVJ0onHd1JLi1fNLV6/L2ardpJ8Xps2XJL/bs7X35WUW7GbE83xWrAdmtte3hz0dWlSmDhIMgaABBcsadnnwNdHdf1z/9DweWsCVhf5dpLG9z9Fhf16WrI7UVNWpPX3XuZ33Yop5ZEmaoeTnBwOmhTGPwIcAHAB31FTx6qljuwuoe6VHrykPBpG7zlYpVao5ORwWdWkELHDNHGmiQNwkeYWr9b9a5+mvrRZB745GvAe3/HNu3dfanseiVFQU/Xw2IiPyIyShQNZXrlLdyypjHT5jn52MPf9TQ4OALhIaopHKR6PYXAjOZtHYlRS3u+ele1+N1OSbSZR28zOS7SVZXAWR1QA4DLxPuzSqEFgW7E6Sgu3i/STPxsQsyaFsAc7OADgMnYPu4xU1cNjdfZ9r6up2T9TIlYl2eH2/hlTkK0rCrItqyyD/djBAQCXicWsq1hYX70/YHDj0/YozUpGyck9unXWH372/Q5NLCrLYB8CHABwGbuGXUYr3COySc+us/y1xxRk676is9WjW6fWa/sON+nX5dsY0ukSBDgA4EKRlFDbzcwRWbQl5R2t2lqrqS9v0f7D7ZOxmUTuHpSJUyYOwMXMlFDbrbnFq+Hz1gTtwBxItJ2Qfa9r1A+HUvD4RZk4AECSNbOuohEswAon4TdQ4JNbUq437rxYZ/ZOj2gdew82xnQSOeIDAQ4AICaMBlq27W8T7rDPjkdUo38X/pTyQOsIB+MYEhtHVBxRAYDljAZa+nZmOuYBhXOUVrxwo1Zv+8LvtU46vpO23D/a1DrCsbh4KDs4ccbM9zcBDgEOAFgq1jkuRgnHHXdzQq3DCDk48cvM9zdVVAAAS4UaaBltf5uasiL9c47/jk3HKeWRDNaMpzJ6RIcABwBgKTtGRXTv0inklPJInj+eyugDaW7xqqJqn5ZX7lJF1T41tyTdIUzYSDIGAFjKzlERviCn47GVmb459xWdrcz0tLgro+8onKRtfI8dHACApZwYFRFJbxzfOiZfmBf34xh8ydIdj9xoTGiMAAcAYCmnRkWEM6XcjnVYrbnFq9IV2wJWgvmula7YxnFVBwQ4AADLOTUqYtXWWsOdo1itI9Z5MbFO2nYrcnAAADExpiBbo/KzbBsVEWynoyOrSsDtyIuxI2nbjQhwAAAxY+eoCDNl4f3uWSkpurlWRk0EfXkxVu0Q2Zm07SYcUQEAXCGSHYxIp5Q3fduie5ZttSUvxomkbTeIaYDz0EMPadiwYTr++ON14oknBrxnx44dGjdunLp166bMzEz94he/UFNTU9DnbWxs1O23367MzEx169ZNV111lT7//PMYvAMAQKKIdAejY4PAUFZtrdWQh1dr/2Hj7yor82KcStpOdDENcJqamvTTn/5UU6ZMCfj35uZmFRUV6fDhw3r33Xe1ZMkSvfrqq/rlL38Z9HmnT5+uZcuWacmSJXr33Xd16NAh/fjHP1Zzc3Ms3gYAIE4ES+gNd6ej6uGxAf+eW1Kuqi8PBX39VVtrdeuizfrq62/DWq9VeTFOJW0nMltmUT3//POaPn26Dhw40O7666+/rh//+MfauXOn+vTpI0lasmSJJk+erD179gScM1FfX6+TTz5ZL774oq677jpJ0u7du5WTk6OVK1fqiiuuCLkeZlEBQOIJJ6HXlxcjqd3xkdGQz3DnWknHgqtBD67Wga+Phr1mqwd2hjOU1M0SZhZVRUWFCgoKWoMbSbriiivU2NioTZs2BXzMpk2bdPToUY0e/f0ckj59+qigoEDvvfdewMc0NjaqoaGh3Q8AIHGE2+jO7E5HTVmRZl15lt/r5ZaU62fPrmt3bV3VvrCDm1jlxfiStuO9MWE8cLSKqq6uTr1792537aSTTlLnzp1VV1dn+JjOnTvrpJNOane9d+/eho+ZO3euSktLrVk0AMBWoRrdeXQsoXdUfpZSUzymy9N/fkk//fySfn67Oe9V7VNuSXnrbk7Fv/aaWjd5Mc4yvYMzZ84ceTyeoD8bN24M+/k8Hv//8r1eb8DrwQR7zKxZs1RfX9/6s3PnTlPPDQBwTiSN7iLZ6agpK9LHvx7jd/37JOTwvpdOSEslLyYOmN7BmTZtmiZOnBj0ntzc3LCeKysrS//4xz/aXfvqq6909OhRv52dto9pamrSV1991W4XZ8+ePRo2bFjAx6SlpSktLS2sNQEA4oudje66dEpV1cNjW/vktPXEW5+G9Rzzrx+ki848Oeq1IDqmA5zMzExlZmZa8uKFhYV66KGHVFtbq+zsY5HuG2+8obS0NA0aNCjgYwYNGqROnTpp9erVuvbaayVJtbW12rp1q37zm99Ysi4AQPyws9GdL5E5Uicd30nDTjf+jkz2JGE7xTQHZ8eOHdq/f7927Nih5uZmVVZWSpJOP/10nXDCCRo9erTy8/N144036pFHHtH+/fs1c+ZMFRcXt2ZH79q1S5dddpkWLlyoCy64QBkZGbr55pv1y1/+Uj179lSPHj00c+ZMnXvuubr88stj+XYAAA7wlX/X1R8JmIfj0bEk4mgTeo06E5sx95pzDQMWO8Y64HsxraK6//77NWDAAM2ePVuHDh3SgAEDNGDAgNYcndTUVJWXl6tLly668MILde2112rChAl69NFHW5/j6NGj2r59u77++uvWa7/73e80YcIEXXvttbrwwgt1/PHHa8WKFUpNTY3l2wEAOMCORneh5liFeubsjC56KkjeTbhVYLCOLX1w4g19cAAg8cRyB6Siap8mdSgLD+TWS36gp97+l9/1G4f21a8nFAR8THOLV8PnrTFMlPbtQFk1ANTNzHx/M2wTAJAQYjmdPNwE5bOzu6umrMivpPzFdZ/pxXWfBWwQaKYKzK7BpKG4IVeIAAcAkDBiNZ3cbCJzTVmRjja36Ix7X2/3d1/g0zbQsbMKzApuyRVimjgAIOEFm1EVjkgmdndKTQm4YyO1HwFhZxVYtNyUK8QODgAgoVmx4+BLZJ6yaLM8CjzHyiiR2RfkdDy28v1e9fDYmFWBWXmUZLZjdLxjBwcAkLCs3HGIdmJ3TVmRTk73byrb756VmjqynyRrq8BWba3V8HlrNOnZdbpjSaUmPbtOw+etiXiXJZKO0fGMKiqqqAAgIcWqOsmKXRGjKeXZGV0syW0x6tljNDU9HMsrd+mOJZUh73t8Yn+N73+Kqee2ClVUAADXi1V1khWJzDVlRXpr+x79rwUb2l2vrT+iy87qpav69wkZPBkFWrE6SkqkXKFwEOAAABJSvFcnjfxhr4Al5W9+vEdvfrzHMEFZCp5XlNG1c0wCO7s6RtuFHBwAQEJKlB2HmrIiffrQlX7Xv59S3l6ovKLV2+rCel2zgZ0dHaPtRIADAEhIkZR2O+W4MEvKQx0/SdLyyt1hvWYkgV20idbxhCRjkowBIGH5djukwKXd8fqlbJSEvLh4aFgjI3p066SvDh8NepQUzeiHeO1kbOb7mx0cAEDCStQdh5qyIuVn+39BhxPcSNLV31UxxeIoKV6DG7PYwWEHB0CScMsXVyCJ/N6MdnOCWVw8VPXfNFk+UiHexzSY+f4mwCHAAZAE4v2LK9mFO8284/GTlYFdLHrrWI0AJwQCHADJJBG+uHBMsN2cWP73FaumiVYjBwcAICm8qpzSFdtMD6dEbNSUFanq4bEB/+ZV7IJRt41pkAhwAMDV3PjF5XapKR7DkvJbv6sYs1q8N02MBAEOALiYG7+4kkVNWVHAQMeoQWA0EqVpohkEOADgYm784nJSc4tXFVX7tLxylyqq9tlytFdTVqSLzsj0u55bUq7dB76x5DUSqWliuJhFBQAu5rb5Qk5yshLtxZuHSPJPQh5WtkaSgs61CodvTMOURZvlUeCmiYk0pkFiBwcAXM1t84WcEmo+1Kqttbaso6asSCumDfe7nltSrj/+7V9RPXeiNk00Qpk4ZeIAkgB9cEIz6ikTryXURnk40e7mxHPTRPrghECAAyAZxfMXl9OCBYAZXTuH1YRvcfFQFfbraep1o/3vpKXFqx/cszLg36INdOKRme9vcnAAIEmkpnhMfwEnA6NGiL7jp/+4MDes5zFbiWbFrlrKdyXlgXZzckvKXRnkhIsdHHZwACBphXP8dFK3Ttp/+GjI5zKzgxOr7tKRHlslyu4enYwBAAhDOI0Q9x8+qh7dOltWQh3L7tI1ZUW6qbCv3/XcknLtaQj8PldtrdXweWs06dl1umNJpSY9u07D562xLXE6VghwAABJK9xjpQn9+0iyphIt1t2lS8cXBNyxueDhN/12eOKlOiwWCHAAAEkr3AaHo/KzLCuhtqu7dE1Zkd785SV+13NLyvXnLbtcP6eMJGMAQNIy0wgxNcWjUflZUeeq2Nldut/JJwRMQp6+tFLTl1YGfWzbnaQL8nokRI5OWwQ4AICkZbaDrxWVaE50l64pK5LX61XerMAl5cH8dVudZrxS2e4Y68SunfS/LszVtEvPiNtAhyMqAEBSM9vBN9p5VE51l/Z4jpWUn3R8J1OPe+7vNX45Oge+Oarf/fUTDXpwddzm6VAmTpk4AEDhlUpb2RHa6e7SoSaSeyR5PFKo+M2jyMvazaKTcQgEOAAAs2LRu8bp/jP/5/97X/+98fOonyfbplEV9MEBAMBCsao48uX0jO9/igr79bQ9n+WRfztfT90wMOrniaasPVYIcAAACCHWvWucNKYgW1UPj9XvJ/aP6nmiLWu3GgEOAAAh2NW7ximpKR5d1f+UqGZXWVHWbiUCHAAAQrCzd43TasqKVD13bNj3mx1VYRcCHAAAQvD1rrFqHpWdIilr95WUX3V+n+D3ffd/Y1HWHq2YBjgPPfSQhg0bpuOPP14nnnii39/ff/99TZo0STk5OeratavOPvtsPf744yGfd8SIEfJ4PO1+Jk6cGIN3AACAc71rohXtIM3fTxoQ9NgqklEVdolpgNPU1KSf/vSnmjJlSsC/b9q0SSeffLIWLVqkDz/8UPfee69mzZqlJ554IuRzFxcXq7a2tvXn6aeftnr5AAC0MtsQ0GlWDtKsKSvS0v891O96bf0RXXzmyVGvNRZs6YPz/PPPa/r06Tpw4EDIe6dOnaqPPvpIa9asMbxnxIgR6t+/vx577LGI1kMfHABApJzuXROO5havhs9bY1j55RsHEUnvmkANAlNTPKp6OPy8nUgldB+c+vp69egR+gzzpZdeUmZmps455xzNnDlTBw8etGF1AIBk53TvmnDEsqy9pqxIW0uvaHetucWr3JJyffJF/HwXx9WwzYqKCr3yyisqLw/ePvr6669XXl6esrKytHXrVs2aNUvvv/++Vq9eHfD+xsZGNTY2tv7e0NBg6boBAIgnsS5rPyHtONWUFem2lzZp5Qd1rddH/e4dSdLi4qGO73CZDnDmzJmj0tLSoPds2LBBgwcPNvW8H374ocaPH6/7779fo0aNCnpvcXFx638uKCjQGWecocGDB2vz5s0aONC/I+PcuXNDrhkAALewq6z9yesHBZxSPunZda3/2c75Wm2ZzsHZu3ev9u7dG/Se3Nxcdeny/YcWKgdn27ZtGjlypG655RY99NBDZpYjSfJ6vUpLS9OLL76o6667zu/vgXZwcnJyyMEBALiSLwenrv5IwPES0eTgGLnvzx/oxXU7Ar6WZM1ATjM5OKZ3cDIzM5WZmRnx4jr68MMPdemll+qmm26KKLjxPcfRo0eVnR34g0tLS1NaWlo0ywQAIGH4ytqnLNosj9QuyIlFWXtzi1ert+0J+Dfvd69ZumKbRuVn2XZcFdMk4x07dqiyslI7duxQc3OzKisrVVlZqUOHDkk6FpiMHDlSo0aN0owZM1RXV6e6ujp9+eWXrc+xa9cunXXWWVq/fr0kqaqqSg888IA2btyompoarVy5Uj/96U81YMAAXXjhhbF8OwAAJITmFq8yunbWf1yYq5O6dWr3t1iUtT+x5hPVNcTXrK6YJhnff//9euGFF1p/HzBggCTprbfe0ogRI/Tf//3f+vLLL/XSSy/ppZdear2vb9++qqmpkSQdPXpU27dv19dffy1J6ty5s9588009/vjjOnTokHJyclRUVKTZs2crNTU1lm8HAIC4t2prrUpXbGtXRdWjW2dN6N9Ho/KzLE/6XbW1Vr/76ydh3WvnrC5b+uDEG/rgAADcyNfcr+MXu5V5MG2F6rfT0eLioSrs1zPi10voPjgAAMC85havSldsC5hU7LtWumJbWLOowhWq305bds/qIsABAMAFYtncz4iZIye7Z3UR4AAA4AKxbu4XSLh9dO68/Ezb++AQ4AAA4AJ2Nfdr64K8HsrO6OI3Yb2trO5pmnbp6Za9ZrgIcAAAcIFQwYZH1ufB+Prt+J6/4+t5JM256hxHRjUQ4AAA4AKhgg0pNnkwYwqyNf+GgcrKaL8zFIt+O2ZQJk6ZOADARQL1wbFjHlRzi1frq/fHdMimme9vAhwCHACAy9gRbDghprOoAABAfEtN8UTVUM8NyMEBAACuQ4ADAABchwAHAAC4DgEOAABwHQIcAADgOgQ4AADAdSgTBwAAUYu33jsEOAAAICpOdU8OhiMqAAAQsVVbazVl0eZ2wY0k1dUf0ZRFm7Vqa60j6yLAAQAAEWlu8ap0xTYFmvnku1a6YpuaW+yfCkWAAwAAIrK+er/fzk1bXkm19Ue0vnq/fYv6DgEOAACIyJ6DxsFNJPdZiQAHAABEpFd6F0vvsxIBDgAAiMgFeT2UndFFRsXgHh2rprogr4edy5JEgAMAACKUmuLR7HH5kuQX5Ph+nz0u35F+OAQ4AAAgYmMKsjX/hoHKymh/DJWV0UXzbxjoWB8cGv0BAICojCnI1qj8LDoZAwAAd0lN8aiwX0+nl9GKIyoAAOA6BDgAAMB1CHAAAIDrEOAAAADXIcABAACuQ4ADAABchwAHAAC4DgEOAABwHQIcAADgOknZydjr9UqSGhoaHF4JAAAIl+972/c9HkxSBjgHDx6UJOXk5Di8EgAAYNbBgweVkZER9B6PN5wwyGVaWlq0e/dupaeny+OJfBBYQ0ODcnJytHPnTnXv3t3CFaIjPmt78Xnbi8/bPnzW9rL68/Z6vTp48KD69OmjlJTgWTZJuYOTkpKiU0891bLn6969O/+g2ITP2l583vbi87YPn7W9rPy8Q+3c+JBkDAAAXIcABwAAuA4BThTS0tI0e/ZspaWlOb0U1+Oztheft734vO3DZ20vJz/vpEwyBgAA7sYODgAAcB0CHAAA4DoEOAAAwHUIcAAAgOsQ4EToySefVF5enrp06aJBgwbpb3/7m9NLcq133nlH48aNU58+feTxePTnP//Z6SW51ty5c/WjH/1I6enp6tWrlyZMmKDt27c7vSxXmj9/vs4777zWBmiFhYV6/fXXnV5W0pg7d648Ho+mT5/u9FJcac6cOfJ4PO1+srKybF0DAU4Eli5dqunTp+vee+/Vli1bdNFFF+nKK6/Ujh07nF6aKx0+fFjnn3++nnjiCaeX4npvv/22pk6dqnXr1mn16tX69ttvNXr0aB0+fNjppbnOqaeeqrKyMm3cuFEbN27UpZdeqvHjx+vDDz90emmut2HDBj3zzDM677zznF6Kq51zzjmqra1t/fnggw9sfX3KxCMwZMgQDRw4UPPnz2+9dvbZZ2vChAmaO3eugytzP4/Ho2XLlmnChAlOLyUpfPnll+rVq5fefvttXXzxxU4vx/V69OihRx55RDfffLPTS3GtQ4cOaeDAgXryySf14IMPqn///nrsscecXpbrzJkzR3/+859VWVnp2BrYwTGpqalJmzZt0ujRo9tdHz16tN577z2HVgXERn19vaRjX7yInebmZi1ZskSHDx9WYWGh08txtalTp6qoqEiXX36500txvU8++UR9+vRRXl6eJk6cqH/961+2vn5SDtuMxt69e9Xc3KzevXu3u967d2/V1dU5tCrAel6vVzNmzNDw4cNVUFDg9HJc6YMPPlBhYaGOHDmiE044QcuWLVN+fr7Ty3KtJUuWaPPmzdqwYYPTS3G9IUOGaOHChTrzzDP1xRdf6MEHH9SwYcP04YcfqmfPnrasgQAnQh6Pp93vXq/X7xqQyKZNm6Z//vOfevfdd51eimv98Ic/VGVlpQ4cOKBXX31VN910k95++22CnBjYuXOn7rjjDr3xxhvq0qWL08txvSuvvLL1P5977rkqLCxUv3799MILL2jGjBm2rIEAx6TMzEylpqb67dbs2bPHb1cHSFS33367XnvtNb3zzjs69dRTnV6Oa3Xu3Fmnn366JGnw4MHasGGDHn/8cT399NMOr8x9Nm3apD179mjQoEGt15qbm/XOO+/oiSeeUGNjo1JTUx1cobt169ZN5557rj755BPbXpMcHJM6d+6sQYMGafXq1e2ur169WsOGDXNoVYA1vF6vpk2bpj/96U9as2aN8vLynF5SUvF6vWpsbHR6Ga502WWX6YMPPlBlZWXrz+DBg3X99dersrKS4CbGGhsb9dFHHyk7O9u212QHJwIzZszQjTfeqMGDB6uwsFDPPPOMduzYoVtvvdXppbnSoUOH9Omnn7b+Xl1drcrKSvXo0UOnnXaagytzn6lTp+rll1/W8uXLlZ6e3rpTmZGRoa5duzq8One55557dOWVVyonJ0cHDx7UkiVLtHbtWq1atcrppblSenq6Xy5Zt27d1LNnT3LMYmDmzJkaN26cTjvtNO3Zs0cPPvigGhoadNNNN9m2BgKcCFx33XXat2+fHnjgAdXW1qqgoEArV65U3759nV6aK23cuFEjR45s/d13fnvTTTfp+eefd2hV7uRrfTBixIh21xcsWKDJkyfbvyAX++KLL3TjjTeqtrZWGRkZOu+887Rq1SqNGjXK6aUBUfv88881adIk7d27VyeffLKGDh2qdevW2fo9SR8cAADgOuTgAAAA1yHAAQAArkOAAwAAXIcABwAAuA4BDgAAcB0CHAAA4DoEOAAAwHUIcAAAgOsQ4AAAANchwAEAAK5DgAMAAFyHAAcAALjO/wN9AmZT5QMLNwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w0, w1 = gradient_descent(x, y, grad)\n",
    "print(w0, w1)\n",
    "plt.scatter(x, y)\n",
    "pred_y = w0 + w1 * x \n",
    "plt.plot(x, pred_y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 2.2. Implement polynomial regression using JAX with gradient descent.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0177236 -2.9400856\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABFCElEQVR4nO3dfXhU5bnv8d8kQkAMUYiQoJGkqNUYlbcKQVRQQSRF0O4qVK2crdkVwYrUo0GrEKsSqt3VHiu+1I1SFHAfS5ETpFIRrDWU11gRZSslgpCIgCSAkmAy5w+cmMnMmpk1s2atmTXfz3Xl2s3Kmplnplvn7vPcLx6v1+sVAACAi6Q5vQAAAACrEeAAAADXIcABAACuQ4ADAABchwAHAAC4DgEOAABwHQIcAADgOgQ4AADAdY5zegFOaGlp0e7du5WZmSmPx+P0cgAAQAS8Xq8OHjyoXr16KS0t9B5NSgY4u3fvVl5entPLAAAAUdi5c6dOPfXUkPekZICTmZkp6dgH1LVrV4dXAwAAItHQ0KC8vLzW7/FQUjLA8R1Lde3alQAHAIAkE0l6CUnGAADAdQhwAACA6xDgAAAA1yHAAQAArkOAAwAAXIcABwAAuA4BDgAAcB0CHAAA4Dop2ejPac0tXq3dvl97Dh5Rj8xOuqCgm9LTmIkFAIBVCHBstnxzrcqXblFt/ZHWa7lZnTRjTKFGFeU6uDIAANyDIyobLd9cq0nzN/oFN5JUV39Ek+Zv1PLNtQ6tDAAAdyHAsUlzi1flS7fIG+RvvmvlS7eouSXYHQAAwAwCHJus3b4/YOemLa+k2vojWrt9v32LAgDApQhwbLLnoHFwE819AADAGAGOTXpkdrL0PgAAYIwAxyYXFHRTblYnGRWDe3SsmuqCgm52LgsAAFciwLFJeppHM8YUSlJAkOP7fcaYQvrhAABgAQIcG40qytWcG/orJ8v/GConq5Pm3NCfPjgAAFiERn82G1WUqxGFOXQyBgAgjghwHJCe5lFxn+5OLwMAANfiiAoAALiO4wHOrFmz9IMf/ECZmZnq0aOHxo0bp61bt4Z8zKpVq+TxeAJ+PvroI5tWDQAAEpnjAc7q1as1efJkrVmzRitWrNA333yjkSNH6vDhw2Efu3XrVtXW1rb+nHHGGTasGAAAJDrHc3CWL1/u9/vcuXPVo0cPbdiwQRdffHHIx/bo0UMnnnhiHFcHAACSkeM7OO3V19dLkrp1C9/wrl+/fsrNzdVll12mt956y/C+xsZGNTQ0+P0AAAD3SqgAx+v1atq0aRo6dKiKiooM78vNzdWzzz6rV199VX/605/0/e9/X5dddpnefvvtoPfPmjVLWVlZrT95eXnxegsAACABeLxer9fpRfhMnjxZlZWVeuedd3TqqaeaeuyYMWPk8Xj02muvBfytsbFRjY2Nrb83NDQoLy9P9fX16tq1a8zrBgAA8dfQ0KCsrKyIvr8TZgfn9ttv12uvvaa33nrLdHAjSYMHD9bHH38c9G8ZGRnq2rWr3w8AAHAvx5OMvV6vbr/9di1evFirVq1SQUFBVM+zadMm5eYy6gAAACRAgDN58mS9/PLLWrJkiTIzM1VXVydJysrKUufOnSVJ06dP165duzRv3jxJ0uOPP678/Hydc845ampq0vz58/Xqq6/q1Vdfdex9AACAxOF4gDNnzhxJ0rBhw/yuz507VxMnTpQk1dbWaseOHa1/a2pq0l133aVdu3apc+fOOuecc1RZWanRo0fbteyYNbd4mUcFAECcJFSSsV3MJCnFw/LNtSpfukW19Udar+VmddKMMYVMFAcAwEBSJhmniuWbazVp/ka/4EaS6uqPaNL8jVq+udahlQEA4B4EODZqbvGqfOkWBdsy810rX7pFzS0pt6kGAIClCHBstHb7/oCdm7a8kmrrj2jt9v32LQoAABdyPMnYTcIlDu85aBzctBXpfQAAIDgCHItEkjjcI7NTRM8V6X0AACA4jqgsEGni8AUF3ZSb1UlGxeAeHQuKLigIP2gUAAAYI8CJUbjEYa++SxxOT/NoxphCSQoIcny/zxhTSD8cAABiRIATo3CJw5J/4vCoolzNuaG/crL8j6Fysjppzg396YMDAIAFyMGJUaQJwROeW6PfTeinq87vpVFFuRpRmEMnYwAA4oQAJ0ZmEoJ/vmCTfr5gkz5++Ep1SE9TcZ/ufn9nfAMAANYgwImRL3G4rv5I0Dwcj6Tj0j062vzdX8+473X1ObmL3vzFsNZrjG8AAMA65ODEKJLE4f8zoZ/W3Xe539+2fXFY+WWV2r73MOMbAACwGMM2LRq2GekOzA//z9+0eVdDxM/rkdStS0f9suRs5WR15tgKAJCyzHx/E+BYOE3cKIem/fUBvU/Smb98ParX4NgKAJCqCHDCiFeAE8yyf9bql0s2a//hptZrviDllfWfaeVHe0w9n2/vhpJyAECqMfP9TQ5OHM1atkW3vbzRL7iRjvXFmTR/o64deKoWlA429ZxMHQcAIDwCnDhZ9s/deubt7YZ/93U4HtD7JOVmmZs9xdRxAABCI8CJg+YWr365ZHPY+2rrj2jDp19qxphCeRRYhRUOU8cBAAiOACcO1m7fr/2Hj0Z0756DRwzHN4TD1HEAAIKj0V8cmNlZ8QUpbcc31DUc0a/+3weGQZJHx2ZXMXUcAIDgCHDiINKdle5dOvoFKelpntbxDZ07pGnS/I2GU8pzszrRDwcAAAMcUcWBb3xDOL8aWxQ0SGlu8Sqrc0f9+4X56talQ9DHbtxxQPlllTGvFQAAN2IHJw584xuMdmAk6WcXF2j0eYF9bIJ1RO7WpaPG9e2lEYU5mvDcGr/7fUFOTUWJZesHACDZsYMTJ77E4fY7Od26dNBTP+mn6aMLAx5jNJPqy8NNmvv3GtV/3aSaihLdPLQg4LH5ZZWq/zqyxGYAANyOTsZx7mRsNL4h2H1DZ68MCG58fInF79xzaevjjY6o2M0BALgRnYwTiC9xeGzfU1Tcp7thYvDa7fsNgxspeHO/mooSLfyPwE7I+WWVWlK9K+a1AwCQrAhwEkSkpeXt7xv8ve5Bd2zuWFhNEjIAIGUR4CSISEvLje6rqSjR/zx0ZcD1/LJKXVixMqa1AQCQbAhwEoSvtNyos41Hx3rfhGru1/G4tKC7ObsOfO3K3ZzmFq+qtu3Tkupdqtq2j+GjAIBWJBnHOcnYDF8VlSS/8nJf0DPnhv4aVRRYWm7EbBJypAnRiSBYOX1uVifNGFMY9DNKpvcGAAjOzPc3AU4CBTiS+S/ucErnrdeKLZ8HXK9+YIROPL5j3F43nnyBYPv/xzUKBJPpvQEAjBHghJHIAY4Un92GULs5ZgMGJ5ktp0+m9wYACI0AJ4xED3DiZe32/br2maqA6107HaeGI98EfUyw/jtOqtq2L6CbczALSgfrgoJupnsLAQASF31wENQFBd2C5t8YBTdS8P47TjJTTh9NbyEAgDsQ4KQgo5LyUCINLOLNTDl9tL2FAADJjwAnRRmVlBuJNLCINzPl9LH2FgIAJC8CnCRkZf+XbY+MDhgI2lYk/Xfs5JvULikgyPH9PmNModLTPJb0FgIAJCcCnCSzfHOths5eqQnPrdEdC6s14bk1Gjp7pZZvro3q+XwBg1EQ4JV05+VnJFQSrm9Se067wCwnq5NfVZSZYAgA4C5UUSVRFVU8S56D9YppL9GmlEdaTk8fHABwh6QrE3/qqaf06KOPqra2Vuecc44ef/xxXXTRRYb3r169WtOmTdMHH3ygXr166e6779att94a8eslY4Bjtv9LtK/hCxjq6o9o1usfBdxzz6izNGlYn6ie30l0MgaA5JdUZeKLFi3S1KlTdd9992nTpk266KKLdOWVV2rHjh1B79++fbtGjx6tiy66SJs2bdK9996rn//853r11VdtXrm97Ch5Tk/zqLhPd43te4p+dkmfoDs2s5d/lJRzrdq+t+I+3QluAMDlHN/BGTRokPr37685c+a0Xjv77LM1btw4zZo1K+D+e+65R6+99po+/PDD1mu33nqr3nvvPVVVBTaxCyYZd3CWVO/SHQurw973xPi+Gtv3FEtf+2hzi8647/Wgf4v22IodFQCAWWa+v4+zaU1BNTU1acOGDSorK/O7PnLkSL377rtBH1NVVaWRI0f6Xbviiiv0/PPP6+jRo+rQoUPAYxobG9XY2Nj6e0NDgwWrt5eTJc8d0o+VlAfbuckvqzQd5DiVE0NQBQCpw9EAZ+/evWpublbPnj39rvfs2VN1dXVBH1NXVxf0/m+++UZ79+5Vbm7gF+SsWbNUXl5u3cId4Ct5rqs/EpBkLH2XgxPPkmdfINM+0PH9HkmgY5QoXVd/RJPmb4zbbCgSjQEgtTiegyNJHo///4r2er0B18LdH+y6z/Tp01VfX9/6s3PnzhhXbL9EKnmuqSjReadmBVzPL6vU5w3GeULNLV6VL90SNEDzXStfuiWmvj7B+IKq9jlMvqAq2hJ7AEDicjTAyc7OVnp6esBuzZ49ewJ2aXxycnKC3n/cccepe/fuQR+TkZGhrl27+v0ko0j7v9jhtSlDg+7YDHrkTcMkZCdmQzkVVAEAnOXoEVXHjh01YMAArVixQldffXXr9RUrVmjs2LFBH1NcXKylS5f6XXvjjTc0cODAoPk3bjOqKFeXntVTf6yq0af7v1LvbsfrxuJ8dTzOmVi1pqJE736yVz/5wz/8rueXVeoXI87U7Zed0XrNidlQZoKq4j6BAbKVeTvkAAGAfRwNcCRp2rRpuvHGGzVw4EAVFxfr2Wef1Y4dO1r72kyfPl27du3SvHnzJB2rmHryySc1bdo0lZaWqqqqSs8//7wWLFjg5NuwTbBckj+8s93RXJIhp2cHTUL+zYr/0W9W/E/rTo8TidKxBFVW5u2QAwQA9nI8wLnuuuu0b98+Pfjgg6qtrVVRUZGWLVum3r17S5Jqa2v9euIUFBRo2bJluvPOO/X73/9evXr10u9+9zv96Ec/cuot2MapBN1I1VSUBC0p9wU+vrlXoXZUJOnLw02WrSnaoMrKz9rouWrrj+jW+Rt184X5urwwJ6IdHXaBACAyjvfBcUIy9sGxo5OxlYzycJ76ST/d9vKmkI/NtfB9+D63cNVnbV/Pys863HO1FW5Hh10gAKkuqToZIzJOJOjGoqaiJGgScrjgRrL2fURTfWblZx3uudoKVdVFJRgAmEOAkyScSNC1Qk1FiQb0Psn046x8H2arz6z8rM28D6OqLirBAMA8x3NwEBknOxnH6tVJQyQZH1sFY/X7GFWUqxGFORHlr1j5WZt9H8GqumKtBAOAVESAkyQSoZNxrIxKytuK5/vwDdwMx8rPOtxzGWm785Osu3cA4CSOqJJEInUyjsWQ07P19A39Q97j9Puw8rMO9VyhtN35SebdOwBwCgFOEkmkTsaxGFWUq6dv6K+crhkBf/NKunX+RvsX1Y6Vn7XRcwXj0bHKqLa7Q75dIKMAKdhjACDVUSaeJGXibbmlF4rvfUx4bk3Qv5udUh4P8ehkvGJLnf7r7zXySH7HVr5nDRZA+aqoZOIxAOA2Zr6/CXCSMMBxK6Mk5EQIdKwWTU8b+uAASHUEOGEQ4CSuCytWateBrwOur/7fw9S7excHVmSOmR2faHaH3LJ7BwDRIMAJgwAn8cVrN8cXINQ1HNH+Q43q1qWjcrI6WxIosMMCAPFFgBMGAU5yePt/vtBP/2ttwPVrB56qX//b+aafL1gA4hNrIGI0b4ocGQCwDqMa4AoXn3ly0B2bV9Z/ZqppoGQ86sCnNoaRB3QaBoDEQ4CDhFdTUaJtj4wOuJ5fVhlRoBMqAGnLq+gCkWSbEwYAqYAAB0khPc1jmH8TLsgxM/AymkCETsMAkHgIcJBUjKaUh9rNMRtYmL2fTsMAkHgIcJCUaipKVJAdWDaeX1apf31xyO+a2cDC7P10GgaAxEOAg6T11l3Dgu7mXPqb1X67Ob4AJBLRBCJumRMGAG5CgIOkV1NRopduGRRwPb+sUqXz1is9zaP7Swojeq77S86OKhBxy5wwAHAL+uDQB8dVjPJwFpQONpx51f6+4j7do359Og0DQPyY+f4+zqY1AbaoqShRc4tXfe5d5nc9kuBGir3SKT3NE1OABACwBkdUSDrNLV5VbdunJdW7VLVtX0DfmlAl5eFkn5BhxRIBAA5jBwcJIdKjHTPznnxBjqmuxyl3YAsA7kSAA8dFGrQYzXuq+3bMglEyb01Fic6b+Rc1HPkm7Fr+sqVOaWkecmcAIMlxRAVHGc2Iqms3GyrWeU/P3DgwovXMq/pUE55bo6GzV0Y1lwoAkBgIcFJMuPwVu9cSadAS67yncM342msfYAEAkgtHVCnETP6KHcwELbHOe/I145s0f6M8Cp9q49WxJn3lS7doRGEOx1UAkGTYwUkRkR4F2clM0GLFvCejZnxGmAIOAMmLACcFxJq/Ei9mghar5j2NKsrVO/dcqgWlg/XT4t4RvT5TwAEg+RDgpIBY81fixUzQYuW8J18zvisjPJa7Y2F1RPcBABIHAU4KiDV/JV7MBi1Wz3syk3icX1apNf/aZ+r5AQDOIck4BViRvxIvvqClffJzjkHy86iiXI0ozLFk3lOoxONgicjjnz027iHaLskAAPswbDMFhm02t3g1dPZK1dUfCZqH49GxgOKdey51rFrIySGVoarLjjZ7dfuCTUEfR6ADAPYy8/1NgJMCAY70XRWVFLhTISmqIx43CRdgGY17IMgBAPsQ4ISRigGOlHh9cJJNS4tX32s3pdyHQAcA4o8AJ4xUDXAkZ4+C3MJoN2f7rNHyeDx8xgAQJwQ4YaRygAPrGAU6uVmd2CUDgDgw8/1NmTgQJaNjqUTqFg0AqYoAB4hBTUVJ2PwbJ7tFA0CqcizAqamp0c0336yCggJ17txZffr00YwZM9TU1BTycRMnTpTH4/H7GTx4sE2rBoJbUBr6/weZawUA9nKs0d9HH32klpYWPfPMMzr99NO1efNmlZaW6vDhw3rsscdCPnbUqFGaO3du6+8dO3aM93KBkBK1WzQApCrHApxRo0Zp1KhRrb9/73vf09atWzVnzpywAU5GRoZycnLivUQgYpF2gb5jYbXG9j0lzqsBACRUDk59fb26dQs9DVqSVq1apR49eujMM89UaWmp9uzZE/L+xsZGNTQ0+P0AVjI718rO4sXmFq+qtu3Tkupdqtq2jzwgACkhYcrEt23bpv79++s3v/mNbrnlFsP7Fi1apBNOOEG9e/fW9u3bdf/99+ubb77Rhg0blJGREfQxM2fOVHl5ecB1ysRhpVDdoo3+IYt3g0CaOwJwE0f74BgFE22tW7dOAwcObP199+7duuSSS3TJJZfoD3/4g6nXq62tVe/evbVw4UJdc801Qe9pbGxUY2Nj6+8NDQ3Ky8sjwIHlQgUUt34b/LT31PX9Nfpc64MNX8DV/h9wxnMASFaOBjh79+7V3r17Q96Tn5+vTp2O5Szs3r1bw4cP16BBg/TCCy8oLc38qdkZZ5yhW265Rffcc09E99PoD/EUy1wrq7og+wastu/J45MIA1YBwCwz39+WJxlnZ2crOzs7ont37dql4cOHa8CAAZo7d25Uwc2+ffu0c+dO5ebyv0SRGNLTPCru093w7zUVJVpSvUt3LKz2ux4s8In2OGnt9v2GwY3kX7Yeaq0AkKwcSzLevXu3hg0bpry8PD322GP64osvVFdXp7q6Or/7zjrrLC1evFiSdOjQId11112qqqpSTU2NVq1apTFjxig7O1tXX321E28DiMrYvqdElH8TbRdkytYBpDrHysTfeOMNffLJJ/rkk0906qmn+v2t7anZ1q1bVV9fL0lKT0/X+++/r3nz5unAgQPKzc3V8OHDtWjRImVmZtq6fiSmZBt0WVNRom+aW3T6fa8H/btXx46Typdu0YjCnIjfS6Rl65HeBwDJxrEAZ+LEiZo4cWLY+9oGO507d9Zf/vKXOK4KySxZK4bW1XwZ8u/RHCf5ytbr6o8EreDy5eBcUBC+LQMAJKOE6oMDRMtXMZSMgy7jcZyUnubRjDGFkhTQm8f3+4wxhQm9uwUAsSDAQdJrbvGqfOmWoDsVyTDo0kwX5Fc3fBbx844qytWcG/orJ8v/+XOyOlEiDsD1EqbRn50oE3eXqm37NOG5NWHvW1A6OCErhnwl3UbHScGYaRCYbHlJAGDE0TJxwG7JXjHkO06aNH9jQNdjoy7IvpLySAKdcGXrAOBGHFEh6bmhYijUcdLTN/Q3DGSMmgYCQKrjiIojqqQX7ognUbr2RnJUFO4er9ergunLgj5/vOdaAYDTHB3VkAwIcNwn1KBLyfm5S1aXsBvt3PzrkdFKI78GgEsR4IRBgONOidoHJ55DL0PNtQIAtyHACYMAx73MVgzFu8Io1qGXkazPKMipuOZcjb/gtJjfAwAkCgKcMAhwINmz4xNLCbvZ9bGbA8DtzHx/U0WFlGRX5+NoS9ijWV9NRYmevqF/wPX8skqqrQCkHAIcpBw7Ox9HU8Iey/pGFeVSUg4AIsBBClq7fb9hTozkP9wyVr6hl0ZZPR4dO3ZqO/TSivXVVJRo+6zRAdfZzQGQKghwkHLs7HwczdBLq9bn8XhC7uYk6mwuALACAQ5Sjt2dj80OvbR6fTUVJUEDnT73LmM3B4BrMYsKKcd3bBSu83HbY6NYjSrK1YjCnIhK0uO1vpqKkqABTX5ZpX41rkg3Du5t6vkAIJFRJk6ZeEpKhs7H8VxfqJJypo8DSFT0wQmDAAdS4nY+9on3+t788HPd/OL6sPcl0mcCILUR4IRBgAOfRN+tsGN94fJwjHaNrFpbov93ACBxEOCEQYAD+Gtu8arPvcGnlEuBIyXC7S5FGrQk+i4agMRi5vubJGMAYXv+tO29U/91U9Dhob4uy/9xcYFee682bNBiNITU9zxO50EBSG6UiQOIuPdOXf3XIbsseyU98/b2sCMm7OwmDSA1EeAAiLinzp2vvBeyy7KR9kFLPLpJN7d4VbVtn5ZU71LVtn0ER0CK44gKQNjeO1ZoG7RY3U2aXB4A7bGDAyCikRJW8SUeRyKS++yaDA8guRDgAJAUeqTE0zf0N5xrZZavqsrsENJgyOUBYIQjKgCtwo2UqKko0fLNtbr12y7LZrQdMeHbMZo0f6M8Ct6tuf0Q0mDM5PIU9+lues0Akhc7OAD8pKd5VNynu8b2PUUXFHTT2u37/RJ3RxXl6ukb+od8jkgmp5sdQhqMnZPhASQXdnAABBUucXfbI6O1dvt+TXhuTcBj2x8I5Rgk/JoZQhqM3ZPhASQPOhnTyRhRcvOIAaMmfEZjG4zGPYw+N0c3Ds6P22fT3OLV0Nkrw05e93VgBpDcGNUQBgEOYuXmsmRf0GCU2xIqaAg1pTxeEn0yPADrmPn+JgcHMMntZcmxNOGrqSjRi/9+QcD1/LLKsEM9o2VFLg8A9yEHBzAhXFmyR8fKkkcU5iTtkUisibuXnHmyaipKggY0+WWVcdnNiTWXB4D7EOAAJqRCWbJVibu+QKZ9oOP73epAx1f9BQASR1SAKalQlmxVEz4fo0Amv6xSTd+0RLdIAAiDAAcwIRXKkiMZ2xBJE762aipKggY6Z/7y9bjl5oTCYE7A/aiioooKJqRSWXK8KsWMAppbL+mjsivPivp5I+XmCjjA7SgTD4MAB7FIpbLkePT6CRZgtGVHSXmk/X0AJBYCnDAIcBArdgGiYxRgBBMq0Ik08Gp7X/YJGfrFK9Wqa2gM+pxu2n0D3CppApz8/Hx9+umnftfuueceVVRUGD7G6/WqvLxczz77rL788ksNGjRIv//973XOOedE/LoEOLCCmzsZx0O4BoLBBAtyIg0uw+0UGVlQOphqLCBBmfn+drxM/MEHH1RpaWnr7yeccELI+3/961/rP//zP/XCCy/ozDPP1EMPPaQRI0Zo69atyszMjPdygVaUJZsTrsQ+mPYl5UY7QL4mi74jJjM7Re0lcwUcgO84XkWVmZmpnJyc1p9QAY7X69Xjjz+u++67T9dcc42Kior04osv6quvvtLLL79s46oBmBVL4JBfVqmvGr8J2WRROtZksembFsP7IpHMFXAAvuN4gDN79mx1795dffv21cMPP6ympibDe7dv3666ujqNHDmy9VpGRoYuueQSvfvuu4aPa2xsVENDg98PAHtFGjgsKB0c9GiqcMZfImqy+MeqGtM7RZL5/j4AEpujAc4dd9yhhQsX6q233tKUKVP0+OOP67bbbjO8v66uTpLUs2dPv+s9e/Zs/Vsws2bNUlZWVutPXl6eNW8AQMTMNhCMtprq0/1fmX5MtP19ACQuywOcmTNnyuPxhPxZv369JOnOO+/UJZdcovPOO0+33HKLnn76aT3//PPat29fyNfwePz/BeT1egOutTV9+nTV19e3/uzcuTP2NwrAlGgaCBo1CAyld7fjTa+NwZyA+1ieZDxlyhSNHz8+5D35+flBrw8ePFiS9Mknn6h798DkzZycHEnHdnJyc7/7F9GePXsCdnXaysjIUEZGRrilA4gz3+Tv9tVNOWFK7GsqSvT3T/bq+j/8w/C5fWXeNxbn6w/vbA/bjPGxfztfew83UgEHuJTlAU52drays7OjeuymTZskyS94aaugoEA5OTlasWKF+vXrJ0lqamrS6tWrNXv27OgWDMBW0U7+vvD0bD19Q3/d+m2TxWBmjClUx+PSNGNMoSbN3yiPgjdjnDGmUBeeEd2/pwAkB8dycKqqqvTb3/5W1dXV2r59u1555RX97Gc/01VXXaXTTjut9b6zzjpLixcvlnTsaGrq1Kl65JFHtHjxYm3evFkTJ07U8ccfr5/85CdOvRUAJvlK7Mf2PUXFfbpHvHsyqihXT9/QX7lZgQnLXqk1+PHtFOW0u4+jKCB1ONYHJyMjQ4sWLVJ5ebkaGxvVu3dvlZaW6u677/a7b+vWraqvr2/9/e6779bXX3+t2267rbXR3xtvvEEPHCBFtN0BmvDcmoC/55dV6sMHR0W8U5SMDRuTcc2A3RjVQCdjIOkZDfAMl6CcjCM3knHNgFWSZlSDUwhwAPcxCnImXJCnq84/JWC3IxkHbybjmgErEeCEQYADuJdRoOOTm9VJ95ecrV9VfmjYEDARB2+Gm+WViGsGrGbm+9vxTsYAYKWaihK9OmmI4d/r6o/otpc3RdQVee32/a3Xmlu8qtq2T0uqd6lq2z41t9j7vw3DzfIKtmYglTk+bBMArNY370TlZnUKGhCYCUt887MSIe8l0lleDAsFjmEHB4ClnN7pkKKbXB5Mj8xOrXkv7Z/PN8F8+ebamF8n0rVYeR/gduzgALBMIux0SLHvYvjyWQb0PkmXPPqW4QRzj45NMB9RmBP3vBffLK9wHZoZFgocww4OAEskyk6HFNsuRttuxxs+/TJh8l6imeUFpDICHAAxa27xqnzpFsOdDunYToddx1XhJpeH0rbbcaLlvdChGYgcR1QAYmamwqe4T+AgXav5djtCzaOaYzDXqrb+SGugkIh5L9HO8gJSDTs4AGKWaDsdUmS7HTUVJfrTbYEl5flllcovqwy7E+TRsRwju/Neop3lBaQSdnAAxCwRdzqkyHY7+p92kmoqSoI2COxz7zI9fUP/sJPJCTCAxEOAAyBmiVzh49vtCMc3t6p9oOM7xmrfVyeH+U9AQiPAARCzSHJekmWnw2g3p7b+iJ6/aaAONX5D3guQBJhFxSwqwDKJ0gfHKtFOKQcQHwzbDIMAB4if5havpRU+Vj+fWUZBzpA+3fVy6WDb1gGAACcsAhwgOTi9I+QLruoajujORdVB72E3B7APAU4YBDhA4vN1Rm7/L6i2fWziGeQEC65CIdAB4s/M9zd9cAAkHKc7IxuNnQjF6CgLgDMIcAAkHDOdka0WKrgKx9cgEIDzCHAAJBwnOyOHC64ikV9WqS8PN1m0IgDRIMABkHCc7IxsNmh6YnzfoPk3/X61gt0cwEEEOAASjpMzoMwGTb77jZKM88sq9aM578a8LgDmEOAASDi+zsiSAoKceHdG9gVXkWgfZNVUlAQNdDZ8+iW7OYDNCHAAJKRIpoHHQ3qaR1edH/65PTIOsmoqSlT586EB161MQm5u8apq2z4tqd6lqm374lZRBiQr+uDQBwdIaHZ3Mm5u8Wro7JUhE43TPNKTE/pp9Hm9wj5fPMY9ON0AEXCKme9vhm0CSGiRTgO3SiRVVC1e6aQuGRE9n9GUct/vZgMdowaIdfVHNGn+xrg3QASSBUdUANBGvErUQyUh7zvUGNFzON0AEUgmBDgA0EY8S9SNkpAHPPTXiHJznGyACCQbAhwAaMOOEvVQuzlXP/V3w8c52QCxPZKckejIwQGANnwl6pPmb5RH8jsOsrJE3Sg3Z9OOA8ovqwwaBDnZALEtkpyRDNjBAYB27CxRr6ko0V+nXRxwPVhJuZMNEH2MBpH6kpyXb66N22uzawQzKBOnTByAAbtL1CMpKfcFGFLw3aV4VlGFK6H36FgQ+M49l1r+ObFrBMnc9zcBDgEOgAQTLtBx6su+ats+TXhuTdj7FpQOtrS036g03o6gDomFPjgAkMS2PTJafe5dFnA9v6xSa++9TKOKcjWiMMfW3SXJmSTncKXxHh0rjR9RmBP394/kQoADAAnEtztj5IJH3pR0bDfHzgaIkjNJzmZK4+3+PJDYSDIGgARhlMAbTH5ZpUb852obVvUdJ5KcE6k0HsmFAAcAEkCooxgpcKq6JH2855CtU8qdmPKeKKXxSD4EOACQACI5ipGkx358fsDf2paUx7uU2u4p74lQGo/k5FgOzqpVqzR8+PCgf1u7dq1+8IMfBP3bxIkT9eKLL/pdGzRokNasCZ/ZDwCJKtIjlg7pHtVUlATduckvq1RuVqe4V1fZmeRsV+NFuI9jOzhDhgxRbW2t388tt9yi/Px8DRw4MORjR40a5fe4ZcsCqw0AIJmYPYoxmmtlVwM+35T3sX1PUXGf7nENMOzeNYI7OLaD07FjR+Xk5LT+fvToUb322muaMmWKPJ7Q/6BkZGT4PRYAkp3vKKau/kjQPBxfE732RzFGuzk+bimldqo0HskrYXJwXnvtNe3du1cTJ04Me++qVavUo0cPnXnmmSotLdWePXtC3t/Y2KiGhga/HwBIJLEk8C4oHRzyud0yZdzOXSMkv4QJcJ5//nldccUVysvLC3nflVdeqZdeekkrV67Ub37zG61bt06XXnqpGhsbDR8za9YsZWVltf6Eew0AcEK0RzGUUgOBLB/VMHPmTJWXl4e8Z926dX55Np999pl69+6tV155RT/60Y9MvV5tba169+6thQsX6pprrgl6T2Njo18A1NDQoLy8PEY1AHBEuBlXZmdgRTpCQVLQvB0gWTg6qmHKlCkaP358yHvy8/P9fp87d666d++uq666yvTr5ebmqnfv3vr4448N78nIyFBGRobp5wYAq0UyR8p3FBOpcPk7bfnydQh04HaWBzjZ2dnKzs6O+H6v16u5c+fqpz/9qTp06GD69fbt26edO3cqN5csegCJzWhopK/SKdqKoHCl1MGCnvyySoIcuJrjOTgrV67U9u3bdfPNNwf9+1lnnaXFixdLkg4dOqS77rpLVVVVqqmp0apVqzRmzBhlZ2fr6quvtnPZAGBKuKGR0rFKp2gb84XK33n6hv5Bg5m2DQIBt3F82Obzzz+vIUOG6Oyzzw76961bt6q+vl6SlJ6ervfff1/z5s3TgQMHlJubq+HDh2vRokXKzMy0c9kAYIodQyPDlVKHahD4t7uHK6/b8VG9LpCILE8yTgZmkpQAwApLqnfpjoXVYe97Ynxfje17StzXY7Rzw7EVEpmZ72/Hj6gAIBUk2tDImooSZZ/QMeB6flmlfvDwX21ZAxBPBDgAYINEHBq5/pcjgu7YfHGwkdwcJD0CHACwQSydiuOtpqJE79wTOPzYiiTkeE83B4yQg0MODgAbBeuD071LR43t20sjCnMcn69kZW5OJD1/ADPMfH8T4BDgALCZr1PxX7fUaXH1Lu0/fLT1b4kSAMQa6Bj1/PGFbkwBRzRIMgaABJae5lH91036r7/X+AU30ndN/5ZvrrV1Te2PkrY9MjrofflllarZezjsc8Wz5w8QCcf74ABAqgkXAHh0LAAYUZhjy3GV0VHS09/usrTfzRn22CpJxrs5dvT8AcJhBwcAbGYmAIg331FS+/W03UmqqSjRaUGaAOaXVeqcB5YHXGe6ORIBAQ4A2CyeAYCZqiUzR0lv3z086I7N4abmgB0eu3v+UKmFYDiiAgCbxSsAMFu1FM1RUk1Fierqj2jwrDf97m07pTzcdHOPjs3IsqLnD5VaMMIODgDYLB5N/yI5amov2p2knKxOhvk3+WWVtvX8ieY9I3UQ4ACAzawOAKKtWop1J6mmosRwSvmt8zcaTje3okTcjkotjr6SG0dUAOCAUUW5mnND/4DjlZwojleirVqy6ijJaEr5rfM36q/TLtYXB5uCTjePRbwrtTj6Sn4EOADgkFFFuRpRmKO12/fHFABEe9Tk20maNH+jPJJfkGN2J8m3k9M+0Ln8P9/2+7tVrEjU9jVcbP/ZGzUp9B190aQwORDgAICD0tM8MfeCieWoycqdJOlYIHPVk+/on5/V+13PL6vU8R3TteXBUaaez0isx2tGOzT3lxTqV5WJ06MI0SPAAYAkF+6oSZJOPL6DWlq8am7xBnwxW7WT5PPalKGSAndzvvq2pNyK3ZxYjtdC7dDc9vLGkK9Lk8LkQZIxACS5UEnLPge+Oqrrn/+Hhs5eGbS6yLeTNLbvKSru092S3YmaihKtve+ygOtWTCmPNlE7kuTkSNCkMPER4ACAC/iOmtpXLbVndwl1j8zQJeWxMHrPoSq1wiUnR8qqJoWIH6aJM00cgIs0t3i15l/7NPmljTrw9dGg9/iOb96551Lb80iMgpptj4yO+ojMKFk4mCXVu3THwupol+/oZwdz39/k4ACAi6SneZTm8RgGN5KzeSRGJeV97l3m97uZkmwzidpmdl5irSyDsziiAgCXSfRhl0YNAtuK11FapF2kn/pJv7g1KYQ92MEBAJexe9hltLY9Mlpn3/+6mpoDMyXiVZIdae+fUUW5uqIo17LKMtiPHRwAcJl4zLqKh7Xb9wcNbnzaHqVZySg5uVuXjvr9T77boYlHZRnsQ4ADAC5j17DLWEV6RDbhuTWWv/aoolzdX3K2unXp0Hpt3+Em/apyC0M6XYIABwBcKJoSaruZOSKLtaS8veWbazX55U3af9g/GZtJ5O5BmThl4gBczEwJtd2aW7waOntlyA7MwcTaCdn3ukb9cCgFT1yUiQMAJFkz6yoWoQKsSBJ+gwU++WWVeuPOi3Vmz8yo1rH3YGNcJ5EjMRDgAADiwmigZdv+NpEO+2x/RDXyt5FPKQ+2jkgwjiG5cUTFERUAWM5ooKVvZ6Z9HlAkR2ml89ZrxZbPA17rpOM7aNMDI02tIxILSgezg5NgzHx/E+AQ4ACApeKd42KUcNx+NyfcOoyQg5O4zHx/U0UFALBUuIGWsfa3qako0T9nBu7YtJ9SHs1gzUQqo0dsCHAAAJayY1RE104dwk4pj+b5E6mMPpjmFq+qtu3Tkupdqtq2T80tKXcIEzGSjAEAlrJzVIQvyGl/bGWmb879JWcrOzMj4cro24skaRvfYQcHAGApJ0ZFRNMbx7eOiRcWJPw4Bl+ydPsjNxoTGiPAAQBYyqlREZFMKbdjHVZrbvGqfOmWoJVgvmvlS7dwXNUOAQ4AwHJOjYpYvrnWcOcoXuuId15MvJO23YocHABAXIwqytWIwhzbRkWE2uloz6oScDvyYuxI2nYjAhwAQNzYOSrCTFl4n3uXSYptrpVRE0FfXoxVO0R2Jm27CUdUAABXiGYHI9op5U3ftOjexZttyYtxImnbDeIa4Dz88MMaMmSIjj/+eJ144olB79mxY4fGjBmjLl26KDs7Wz//+c/V1NQU8nkbGxt1++23Kzs7W126dNFVV12lzz77LA7vAACQLKLdwWjfIDCc5ZtrNeiRFdp/2Pi7ysq8GKeStpNdXAOcpqYm/fjHP9akSZOC/r25uVklJSU6fPiw3nnnHS1cuFCvvvqqfvGLX4R83qlTp2rx4sVauHCh3nnnHR06dEg//OEP1dzcHI+3AQBIEKESeiPd6dj2yOigf88vq9S2Lw6FfP3lm2t16/yN+vKrbyJar1V5MU4lbSczW2ZRvfDCC5o6daoOHDjgd/3111/XD3/4Q+3cuVO9evWSJC1cuFATJ07Unj17gs6ZqK+v18knn6w//vGPuu666yRJu3fvVl5enpYtW6Yrrrgi7HqYRQUAySeShF5fXowkv+MjoyGfkc61ko4FVwMeWqEDXx2NeM1WD+yMZCipmyXNLKqqqioVFRW1BjeSdMUVV6ixsVEbNmwI+pgNGzbo6NGjGjnyuzkkvXr1UlFRkd59992gj2lsbFRDQ4PfDwAgeUTa6M7sTkdNRYmmX3lWwOvll1XqJ8+t8bu2Ztu+iIObeOXF+JK2E70xYSJwtIqqrq5OPXv29Lt20kknqWPHjqqrqzN8TMeOHXXSSSf5Xe/Zs6fhY2bNmqXy8nJrFg0AsFW4RnceHUvoHVGYo/Q0j+ny9J9d0kc/u6RPwG7Ou9v2Kb+ssnU3p+pfe02tm7wYZ5newZk5c6Y8Hk/In/Xr10f8fB5P4H/5Xq836PVQQj1m+vTpqq+vb/3ZuXOnqecGADgnmkZ30ex01FSU6KNfjQq4/l0ScmTfSydkpJMXkwBM7+BMmTJF48ePD3lPfn5+RM+Vk5Ojf/zjH37XvvzySx09ejRgZ6ftY5qamvTll1/67eLs2bNHQ4YMCfqYjIwMZWRkRLQmAEBisbPRXacO6dr2yOjWPjltPfnWJxE9x5zrB+iiM0+OeS2IjekAJzs7W9nZ2Za8eHFxsR5++GHV1tYqN/dYpPvGG28oIyNDAwYMCPqYAQMGqEOHDlqxYoWuvfZaSVJtba02b96sX//615asCwCQOOxsdOdLZI7WScd30JDTjb8jUz1J2E5xzcHZsWOH9u/frx07dqi5uVnV1dWSpNNPP10nnHCCRo4cqcLCQt1444169NFHtX//ft11110qLS1tzY7etWuXLrvsMs2bN08XXHCBsrKydPPNN+sXv/iFunfvrm7duumuu+7Sueeeq8svvzyebwcA4ABf+Xdd/ZGgeTgeHUsijjWh16gzsRmzrjnXMGCxY6wDvhPXKqoHHnhA/fr104wZM3To0CH169dP/fr1a83RSU9PV2VlpTp16qQLL7xQ1157rcaNG6fHHnus9TmOHj2qrVu36quvvmq99tvf/lbjxo3TtddeqwsvvFDHH3+8li5dqvT09Hi+HQCAA+xodBdujlW4Z87N6qSnQ+TdRFoFBuvY0gcn0dAHBwCSTzx3QKq27dOEdmXhwdx6yff09Op/BVz/aXFvPTi2KOhjmlu8Gjp7pWGitG8HyqoBoG5m5vubYZsAgKQQz+nkkSYon53bVTUVJQEl5fOqPtW8qk+DNgg0UwVm12DScNyQK0SAAwBIGvGaTm42kbmmokRHm1t0xn2v+/3dF/i0DXTsrAKzgltyhZgmDgBIeqFmVEUimondHdLTgu7YSP4jIOysAouVm3KF2MEBACQ1K3YcfInMk+ZvlEfB51gZJTL7gpz2x1a+37c9MjpuVWBWHiWZ7Rid6NjBAQAkLSt3HGKd2F1TUaKTMwObyva5d5kmD+8jydoqsOWbazV09kpNeG6N7lhYrQnPrdHQ2Suj3mWJpmN0IqOKiioqAEhK8apOsmJXxGhKeW5WJ0tyW4x69hhNTY/EkupdumNhddj7nhjfV2P7nmLqua1CFRUAwPXiVZ1kRSJzTUWJ3tq6R/9r7jq/67X1R3TZWT10Vd9eYYMno0ArXkdJyZQrFAkCHABAUkr06qTh3+8RtKT8zY/26M2P9hgmKEuh84qyOneMS2BnV8dou5CDAwBISsmy41BTUaJPHr4y4Pp3U8r9hcsrWrGlLqLXNRvY2dEx2k4EOACApBRNabdTjouwpDzc8ZMkLaneHdFrRhPYxZponUhIMibJGACSlm+3Qwpe2p2oX8pGScgLSgdHNDKiW5cO+vLw0ZBHSbGMfkjUTsZmvr/ZwQEAJK1k3XGoqShRYW7gF3QkwY0kXf1tFVM8jpISNbgxix0cdnAApAi3fHEFk8zvzWg3J5QFpYNV/3WT5SMVEn1Mg5nvbwIcAhwAKSDRv7hSXaTTzNsfP1kZ2MWjt47VCHDCIMABkEqS4YsLx4TazYnnf1/xappoNXJwAACSIqvKKV+6xfRwSsRHTUWJtj0yOujfvIpfMOq2MQ0SAQ4AuJobv7jcLj3NY1hSfuu3FWNWS/SmidEgwAEAF3PjF1eqqKkoCRroGDUIjEWyNE00gwAHAFzMjV9cTmpu8apq2z4tqd6lqm37bDnaq6ko0UVnZAdczy+r1O4DX1vyGsnUNDFSzKICABdz23whJzlZifbHmwdJCkxCHlKxUpJCzrWKhG9Mw6T5G+VR8KaJyTSmQWIHBwBczW3zhZwSbj7U8s21tqyjpqJES6cMDbieX1apP/ztXzE9d7I2TTRCmThl4gBSAH1wwjPqKZOoJdRGeTix7uYkctNE+uCEQYADIBUl8heX00IFgFmdO0bUhG9B6WAV9+lu6nVj/e+kpcWr7927LOjfYg10EpGZ729ycAAgRaSneUx/AacCo0aIvuOnf78wP6LnMVuJZsWuWtq3JeXBdnPyyypdGeREih0cdnAAIGVFcvx0UpcO2n/4aNjnMrODE6/u0tEeWyXL7h6djAEAiEAkjRD3Hz6qbl06WlZCHc/u0jUVJbqpuHfA9fyySu1pCP4+l2+u1dDZKzXhuTW6Y2G1Jjy3RkNnr7QtcTpeCHAAACkr0mOlcX17SbKmEi3e3aXLxxYF3bG54JE3A3Z4EqU6LB4IcAAAKSvSBocjCnMsK6G2q7t0TUWJ3vzFJQHX88sq9edNu1w/p4wkYwBAyjLTCDE9zaMRhTkx56rY2V26z8knBE1CnrqoWlMXVYd8bNudpAsKuiVFjk5bBDgAgJRltoOvFZVoTnSXrqkokdfrVcH04CXlofx1S52mvVLtd4x1YucO+l8X5mvKpWckbKDDERUAIKWZ7eAb6zwqp7pLezzHSspPOr6Dqcc9//eagBydA18f1W//+rEGPLQiYfN0KBOnTBwAoMhKpa3sCO10d+lwE8k9kjweKVz85lH0Ze1m0ck4DAIcAIBZ8ehd43T/mf/9f9/Tf6//LObnybVpVAV9cAAAsFC8Ko58OT1j+56i4j7dbc9nefTfztfTN/SP+XliKWuPFwIcAADCiHfvGieNKsrVtkdG63fj+8b0PLGWtVuNAAcAgDDs6l3jlPQ0j67qe0pMs6usKGu3EgEOAABh2Nm7xmk1FSXaPmt0xPebHVVhFwIcAADC8PWusWoelZ2iKWv3lZRfdX6v0Pd9+3/jUdYeq7gGOA8//LCGDBmi448/XieeeGLA39977z1NmDBBeXl56ty5s84++2w98cQTYZ932LBh8ng8fj/jx4+PwzsAAMC53jWxinWQ5u8m9At5bBXNqAq7xDXAaWpq0o9//GNNmjQp6N83bNigk08+WfPnz9cHH3yg++67T9OnT9eTTz4Z9rlLS0tVW1vb+vPMM89YvXwAAFqZbQjoNCsHadZUlGjRfwwOuF5bf0QXn3lyzGuNB1v64LzwwguaOnWqDhw4EPbeyZMn68MPP9TKlSsN7xk2bJj69u2rxx9/PKr10AcHABAtp3vXRKK5xauhs1caVn75xkFE07smWIPA9DSPtj0Sed5OtJK6D059fb26dQt/hvnSSy8pOztb55xzju666y4dPHjQhtUBAFKd071rIhHPsvaaihJtLr/C71pzi1f5ZZX6+PPE+S5OqGGbVVVVeuWVV1RZGbp99PXXX6+CggLl5ORo8+bNmj59ut577z2tWLEi6P2NjY1qbGxs/b2hocHSdQMAkEjiXdZ+QsZxqqko0W0vbdCy9+tar4/47duSpAWlgx3f4TId4MycOVPl5eUh71m3bp0GDhxo6nk/+OADjR07Vg888IBGjBgR8t7S0tLW/1xUVKQzzjhDAwcO1MaNG9W/f2BHxlmzZoVdMwAAbmFXWftT1w8IOqV8wnNrWv+znfO12jKdg7N3717t3bs35D35+fnq1Om7Dy1cDs6WLVs0fPhw3XLLLXr44YfNLEeS5PV6lZGRoT/+8Y+67rrrAv4ebAcnLy+PHBwAgCv5cnDq6o8EHS8RSw6Okfv//L7+uGZH0NeSrBnIaSYHx/QOTnZ2trKzs6NeXHsffPCBLr30Ut10001RBTe+5zh69Khyc4N/cBkZGcrIyIhlmQAAJA1fWfuk+RvlkfyCnHiUtTe3eLViy56gf/N++5rlS7doRGGObcdVcU0y3rFjh6qrq7Vjxw41Nzerurpa1dXVOnTokKRjgcnw4cM1YsQITZs2TXV1daqrq9MXX3zR+hy7du3SWWedpbVr10qStm3bpgcffFDr169XTU2Nli1bph//+Mfq16+fLrzwwni+HQAAkkJzi1dZnTvq3y/M10ldOvj9LR5l7U+u/Fh1DYk1qyuuScYPPPCAXnzxxdbf+/XrJ0l66623NGzYMP33f/+3vvjiC7300kt66aWXWu/r3bu3ampqJElHjx7V1q1b9dVXX0mSOnbsqDfffFNPPPGEDh06pLy8PJWUlGjGjBlKT0+P59sBACDhLd9cq/KlW/yqqLp16ahxfXtpRGGO5Um/yzfX6rd//Tiie+2c1WVLH5xEQx8cAIAb+Zr7tf9itzIPpq1w/XbaW1A6WMV9ukf9ekndBwcAAJjX3OJV+dItQZOKfdfKl26JaBZVpML122nL7lldBDgAALhAPJv7GTFz5GT3rC4CHAAAXCDezf2CibSPzp2Xn2l7HxwCHAAAXMCu5n5tXVDQTblZnQImrLeV0zVDUy493bLXjBQBDgAALhAu2PDI+jwYX78d3/O3fz2PpJlXnePIqAYCHAAAXCBcsCHFJw9mVFGu5tzQXzlZ/jtD8ei3YwZl4pSJAwBcJFgfHDvmQTW3eLV2+/64Dtk08/1NgEOAAwBwGTuCDSfEdRYVAABIbOlpnpga6rkBOTgAAMB1CHAAAIDrEOAAAADXIcABAACuQ4ADAABchwAHAAC4DmXiAAAgZonWe4cABwAAxMSp7smhcEQFAACitnxzrSbN3+gX3EhSXf0RTZq/Ucs31zqyLgIcAAAQleYWr8qXblGwmU++a+VLt6i5xf6pUAQ4AAAgKmu37w/YuWnLK6m2/ojWbt9v36K+RYADAACisuegcXATzX1WIsABAABR6ZHZydL7rESAAwAAonJBQTflZnWSUTG4R8eqqS4o6GbnsiQR4AAAgCilp3k0Y0yhJAUEOb7fZ4wpdKQfDgEOAACI2qiiXM25ob9ysvyPoXKyOmnODf0d64NDoz8AABCTUUW5GlGYQydjAADgLulpHhX36e70MlpxRAUAAFyHAAcAALgOAQ4AAHAdAhwAAOA6BDgAAMB1CHAAAIDrEOAAAADXIcABAACuQ4ADAABcJyU7GXu9XklSQ0ODwysBAACR8n1v+77HQ0nJAOfgwYOSpLy8PIdXAgAAzDp48KCysrJC3uPxRhIGuUxLS4t2796tzMxMeTzRDwJraGhQXl6edu7cqa5du1q4QrTHZ20vPm978Xnbh8/aXlZ/3l6vVwcPHlSvXr2UlhY6yyYld3DS0tJ06qmnWvZ8Xbt25R8Um/BZ24vP21583vbhs7aXlZ93uJ0bH5KMAQCA6xDgAAAA1yHAiUFGRoZmzJihjIwMp5fienzW9uLztheft334rO3l5OedkknGAADA3djBAQAArkOAAwAAXIcABwAAuA4BDgAAcB0CnCg99dRTKigoUKdOnTRgwAD97W9/c3pJrvX2229rzJgx6tWrlzwej/785z87vSTXmjVrln7wgx8oMzNTPXr00Lhx47R161anl+VKc+bM0XnnndfaAK24uFivv/6608tKGbNmzZLH49HUqVOdXoorzZw5Ux6Px+8nJyfH1jUQ4ERh0aJFmjp1qu677z5t2rRJF110ka688krt2LHD6aW50uHDh3X++efrySefdHoprrd69WpNnjxZa9as0YoVK/TNN99o5MiROnz4sNNLc51TTz1VFRUVWr9+vdavX69LL71UY8eO1QcffOD00lxv3bp1evbZZ3Xeeec5vRRXO+ecc1RbW9v68/7779v6+pSJR2HQoEHq37+/5syZ03rt7LPP1rhx4zRr1iwHV+Z+Ho9Hixcv1rhx45xeSkr44osv1KNHD61evVoXX3yx08txvW7duunRRx/VzTff7PRSXOvQoUPq37+/nnrqKT300EPq27evHn/8caeX5TozZ87Un//8Z1VXVzu2BnZwTGpqatKGDRs0cuRIv+sjR47Uu+++69CqgPior6+XdOyLF/HT3NyshQsX6vDhwyouLnZ6Oa42efJklZSU6PLLL3d6Ka738ccfq1evXiooKND48eP1r3/9y9bXT8lhm7HYu3evmpub1bNnT7/rPXv2VF1dnUOrAqzn9Xo1bdo0DR06VEVFRU4vx5Xef/99FRcX68iRIzrhhBO0ePFiFRYWOr0s11q4cKE2btyodevWOb0U1xs0aJDmzZunM888U59//rkeeughDRkyRB988IG6d+9uyxoIcKLk8Xj8fvd6vQHXgGQ2ZcoU/fOf/9Q777zj9FJc6/vf/76qq6t14MABvfrqq7rpppu0evVqgpw42Llzp+644w698cYb6tSpk9PLcb0rr7yy9T+fe+65Ki4uVp8+ffTiiy9q2rRptqyBAMek7OxspaenB+zW7NmzJ2BXB0hWt99+u1577TW9/fbbOvXUU51ejmt17NhRp59+uiRp4MCBWrdunZ544gk988wzDq/MfTZs2KA9e/ZowIABrdeam5v19ttv68knn1RjY6PS09MdXKG7denSReeee64+/vhj216THByTOnbsqAEDBmjFihV+11esWKEhQ4Y4tCrAGl6vV1OmTNGf/vQnrVy5UgUFBU4vKaV4vV41NjY6vQxXuuyyy/T++++rurq69WfgwIG6/vrrVV1dTXATZ42Njfrwww+Vm5tr22uygxOFadOm6cYbb9TAgQNVXFysZ599Vjt27NCtt97q9NJc6dChQ/rkk09af9++fbuqq6vVrVs3nXbaaQ6uzH0mT56sl19+WUuWLFFmZmbrTmVWVpY6d+7s8Orc5d5779WVV16pvLw8HTx4UAsXLtSqVau0fPlyp5fmSpmZmQG5ZF26dFH37t3JMYuDu+66S2PGjNFpp52mPXv26KGHHlJDQ4Nuuukm29ZAgBOF6667Tvv27dODDz6o2tpaFRUVadmyZerdu7fTS3Ol9evXa/jw4a2/+85vb7rpJr3wwgsOrcqdfK0Phg0b5nd97ty5mjhxov0LcrHPP/9cN954o2pra5WVlaXzzjtPy5cv14gRI5xeGhCzzz77TBMmTNDevXt18skna/DgwVqzZo2t35P0wQEAAK5DDg4AAHAdAhwAAOA6BDgAAMB1CHAAAIDrEOAAAADXIcABAACuQ4ADAABchwAHAAC4DgEOAABwHQIcAADgOgQ4AADAdQhwAACA6/x/FvxmTy3b/fIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import jax\n",
    "def loss_with_jax(w0, w1, x, y):\n",
    "    return (y - (w0 + x * w1)) ** 2\n",
    "\n",
    "grad_loss_function = jax.jit((jax.grad(loss_with_jax, argnums=[0, 1])))\n",
    "\n",
    "w0, w1 = gradient_descent(x, y, grad_loss_function)\n",
    "print(w0, w1)\n",
    "plt.scatter(x, y)\n",
    "pred_y = w0 + w1 * x \n",
    "plt.plot(x, pred_y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #013220; border-color: #03C03C\">\n",
    "\n",
    "> ### Question 2.3 - Compare the results of both implementations and the speed of the optimization algorithm.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the results is almost same\n",
    "ground truth: w0 = 2, w1 = -3\n",
    "numpy: w0 = 2.0177232162357135, w1 = -2.9400863075081993\n",
    "jax: w0 = 2.0177236, w1 = -2.9400856\n",
    "the speed is faster in numpy than jax\n",
    "numpy: 0.4 s\n",
    "jax: 4.0 s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"audio\"></a>\n",
    "# Audio applications\n",
    "\n",
    "In order to test our algorithms on audio and music data, we will work with several datasets. We will both be using well-known state-of-art datasets (through the TF dataset system). But first, we will rely on a simple dataset (`Musclefish`) that should be downloaded on your local computer first from this [link](https://nubo.ircam.fr/index.php/s/ByK4QL7nE4Mq5MA)\n",
    "\n",
    "**The following set of instructions will perform that automatically for you :)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl https://nubo.ircam.fr/index.php/s/ByK4QL7nE4Mq5MA/download/musclefish.zip --output musclefish.zip\n",
    "!unzip musclefish.zip\n",
    "!mkdir data\n",
    "!mv musclefish data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset details\n",
    "\n",
    "  |**Type**|*Origin*|\n",
    "  |-------:|:---------|\n",
    "  |**Classification**|[*MuscleFish*](http://knight.cis.temple.edu/~vasilis/Courses/CIS750/Papers/muscle_fish.pdf) dataset|\n",
    "  |**Music-speech**|[*MIREX Recognition*](http://www.music-ir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection) set|\n",
    "  |**Source separation**|[*SMC Mirum*](http://smc.inesctec.pt/research/data-2/) dataset|\n",
    "  |**Speech recognition**|[*CMU Arctic*](http://festvox.org/cmu_arctic/) dataset|\n",
    "\n",
    "**Unzip the file and place the `musclefish` folder inside a `data` folder in the root of this notebook**\n",
    "For the first parts of the tutorial, we will mostly rely solely on the classification dataset. In order to facilitate the interactions, we provide a dataset class called `AudioSupervisedDataset` that will allow to import all audio datasets along the tutorials. This class also contains a set of holders and will allow us to perform batch-wise gradient descent.\n",
    "\n",
    "```Python\n",
    "class AudioSupervisedDataset():\n",
    "    \"\"\"\n",
    "    Helper class to import datasets\n",
    "    % class_path  : Path to the dataset (string)\n",
    "    % type       : Type of dataset (string: 'classify', 'plain', 'metadata')\n",
    "    \"\"\" \n",
    "    audio_files\n",
    "    labels\n",
    "    labels_names\n",
    "    num_examples \n",
    "    num_classes \n",
    "```\n",
    "\n",
    "We also provide a simplified function `import_dataset` that is demonstrated below.\n",
    "\n",
    "***\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "**_Exercise_**  \n",
    "\n",
    "  1. Launch the import procedure and check the corresponding structure\n",
    "  2. Code a count function that prints the name and number of examples for each classes \n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recursive search in /tmp/data//musclefish\n",
      "Metal device set to: Apple M1 Pro\n",
      "(16, 44100)\n",
      "(16,)\n"
     ]
    }
   ],
   "source": [
    "from cml.data import import_dataset, AudioSupervisedDataset\n",
    "data_path = \"data/\"\n",
    "batch_size = 16\n",
    "# Instantiate the dataset class\n",
    "train_dataset = import_dataset(data_path, \"musclefish\", 'train', batch_size=batch_size)\n",
    "# Generate a batch of data from the train dataset\n",
    "batch_x, batch_y = next(train_dataset)\n",
    "# Print the shape of the input and output batch tensors\n",
    "print(batch_x.shape)  # (16, 44100)\n",
    "print(batch_y.shape)  # (16,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "52\n"
     ]
    }
   ],
   "source": [
    "#%% Q-0.1.2 - Count function to print the number of examples in each class along with class label\n",
    "\n",
    "n_batch = 0\n",
    "train_dataset._reset_generator()\n",
    "for batch_x, batch_y in iter(train_dataset):\n",
    "    n_batch += 1\n",
    "    ######################\n",
    "    # YOUR CODE GOES HERE\n",
    "    ######################\n",
    "\n",
    "print(n_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing\n",
    "\n",
    "We will rely on a set of spectral transforms that allow to obtain a more descriptive view over the audio information. As most of these are out of the scope of the machine learning course, we redirect you to a [signal processing course](https://ccrma.stanford.edu/~jos/sasp/) proposed by [Julius O. Smith](https://ccrma.stanford.edu/~jos/).  \n",
    "\n",
    "The following functions to compute various types of transforms are given as part of the basic audio dataset class, in the `cml.data.audio` package\n",
    "\n",
    "  |**Name**|*Transform*|\n",
    "  |-------:|:----------|\n",
    "  |`stft`       |[Short-term Fourier transform](https://en.wikipedia.org/wiki/Short-time_Fourier_transform)|\n",
    "  |`mel`  |[Mel scale](https://en.wikipedia.org/wiki/Mel_scale) transform|\n",
    "  |`chroma` |[Chromas vector](https://en.wikipedia.org/wiki/Harmonic_pitch_class_profiles)|\n",
    "  |`cqt`        |[Constant-Q](https://en.wikipedia.org/wiki/Constant_Q_transform) transform|\n",
    "\n",
    "In order to perform the various computations, we provide the following function, in the `AudioSupervisedDataset` which performs the different transforms on a complete dataset.  \n",
    "\n",
    "``` Python\n",
    "dataset.transform(index, name)\n",
    "    \"\"\" index   : Specific index in our dataset \"\"\"\n",
    "    \"\"\" name    : Name of the transform to apply \"\"\"\n",
    "\n",
    "# The name can be selected in \n",
    "\"stft\"      # Power spectrum (STFT)\n",
    "\"mel\"       # Spectrum in Mel scale\n",
    "\"chroma\"    # Chroma vectors\n",
    "\"cqt\"       # Constant-Q transform\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "**Exercise**  \n",
    "\n",
    "  1. Launch the transform computation procedure and check the corresponding structure\n",
    "  2. For each class, select a random element and plot its various transforms on a single plot. You should obtain plots similar to those shown afterwards.\n",
    "  3. For each transform, try to spot major pros and cons of their representation.\n",
    "  \n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 0.2 - Pre-process the audio to obtain spectral transforms \n",
    "power_spec = train_dataset.transform(0, \"stft\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% Q-0.2.2 - Plot the various transforms \n",
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Features\n",
    "\n",
    "<div markdown = \"1\">\n",
    "\n",
    "As you might have noted from the previous exercice, most spectral transforms have a very high dimensionality, and might not be suited to exhibit the relevant structure of different classes. To that end, we provide a set of functions for computing several spectral features in the `cml.data` package, we redirect interested readers to this [exhaustive article](http://recherche.ircam.fr/anasyn/peeters/ARTICLES/Peeters_2003_cuidadoaudiofeatures.pdf) on spectral features computation.\n",
    "\n",
    "  |**File**|*Transform*|\n",
    "  |-------:|:----------|\n",
    "  |`spectral_centroid`|Spectral centroid|\n",
    "  |`spectral_bandwidth`|Spectral bandwidth|\n",
    "  |`spectral_contrast`|Spectral contrast|\n",
    "  |`spectral_flatness`|Spectral flatness|\n",
    "  |`spectral_rolloff`|Spectral rolloff|\n",
    "\n",
    "Once again, we provide a function to perform the computation of different features on a complete set. Note that for each feature, we obtain the temporal evolution in a vector. Therefore, for further learning tasks, if you wish to obtain simplified spaces, you might need to compute the mean and standard deviation of each feature.\n",
    "\n",
    "``` Python\n",
    "dataset.feature(index, name)\n",
    "    \"\"\" index   : Specific index in our dataset \"\"\"\n",
    "    \"\"\" name    : Name of the feature to obtain \"\"\"\n",
    "\n",
    "# Name can be chosen inside the following\n",
    "\"loudness\"    # Loudness\n",
    "\"centroid\"    # Spectral centroid\n",
    "\"bandwidth\"   # Spectral bandwidth\n",
    "\"contrast\"    # Spectral contrast\n",
    "\"flatness\"    # Spectral flatness\n",
    "\"rolloff\"     # Spectral rolloff\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "<div class=\"alert alert-success\" markdown=\"1\" style=\"color:white; background-color: #192841; border-color: #779ecb\">\n",
    "\n",
    "**Exercise**\n",
    "\n",
    "  1. Launch the feature computation procedure and check the corresponding structure\n",
    "  2. This time for each class, superimpose the plots of various features on a single plot, along with a boxplot of mean and standard deviations. You should obtain plots similar to those shown afterwards.\n",
    "  3. What conclusions can you make on the discriminative power of each feature ?\n",
    "  4. Perform scatter plots of the mean features for all the dataset, while coloring different classes.\n",
    "  5. What conclusions can you make on the discriminative power of mean features ?\n",
    "\n",
    "</div>\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 0.3 - Compute temporal spectral features\n",
    "power_spec = train_dataset.feature(0, \"centroid\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% Q-0.3.2 - Plot the various features \n",
    "\n",
    "# Use these styles for boxplot\n",
    "boxprops=dict(linewidth=3, color='white')\n",
    "whiskerprops=dict(linewidth=3, color='white')\n",
    "medianprops=dict(linewidth=2.5, color='firebrick')\n",
    "flierprops = dict(markeredgecolor='white', markerfacecolor='firebrick')\n",
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% Q-0.3.4 - Observe the distribution of classes for different features\n",
    "\n",
    "# This allows to use 3D rendering in matplotlib\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "plt.figure(figsize=(12,8))\n",
    "# Create a vector of random colors for each class\n",
    "colorVect = np.zeros((3, len(data_struct[\"class_names\"])));\n",
    "for c in range(len(data_struct[\"class_names\"])):\n",
    "    colorVect[:,c] = np.random.rand(3);\n",
    "\n",
    "######################\n",
    "# YOUR CODE GOES HERE\n",
    "######################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it for this tutorial, now remember that we can use any form of description (features) as a basis for learning algorithms. We will see in the next tutorial what we an do with these features."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}