notebooks/plot_scalable_poly_kernels.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Express sklearn pipeline as codeflare pipeline\n",
    "Reference: https://scikit-learn.org/stable/auto_examples/kernel_approximation/plot_scalable_poly_kernels.html#sphx-glr-auto-examples-kernel-approximation-plot-scalable-poly-kernels-py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Scalable learning with polynomial kernel aproximation\n",
    "\n",
    "This example illustrates the use of :class:`PolynomialCountSketch` to\n",
    "efficiently generate polynomial kernel feature-space approximations.\n",
    "This is used to train linear classifiers that approximate the accuracy\n",
    "of kernelized ones.\n",
    "\n",
    ".. currentmodule:: sklearn.kernel_approximation\n",
    "\n",
    "We use the Covtype dataset [2], trying to reproduce the experiments on the\n",
    "original paper of Tensor Sketch [1], i.e. the algorithm implemented by\n",
    ":class:`PolynomialCountSketch`.\n",
    "\n",
    "First, we compute the accuracy of a linear classifier on the original\n",
    "features. Then, we train linear classifiers on different numbers of\n",
    "features (`n_components`) generated by :class:`PolynomialCountSketch`,\n",
    "approximating the accuracy of a kernelized classifier in a scalable manner.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Automatically created module for IPython interactive environment\n"
     ]
    }
   ],
   "source": [
    "print(__doc__)\n",
    "\n",
    "# Author: Daniel Lopez-Sanchez <lope@usal.es>\n",
    "# License: BSD 3 clause\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import fetch_covtype\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import MinMaxScaler, Normalizer\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.kernel_approximation import PolynomialCountSketch\n",
    "from sklearn.pipeline import Pipeline, make_pipeline\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the Covtype dataset, which contains 581,012 samples\n",
    "with 54 features each, distributed among 6 classes. The goal of this dataset\n",
    "is to predict forest cover type from cartographic variables only\n",
    "(no remotely sensed data). After loading, we transform it into a binary\n",
    "classification problem to match the version of the dataset in the\n",
    "LIBSVM webpage [2], which was the one used in [1].\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = fetch_covtype(return_X_y=True)\n",
    "\n",
    "y[y != 2] = 0\n",
    "y[y == 2] = 1  # We will try to separate class 2 from the other 6 classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we select 5,000 samples for training and 10,000 for testing.\n",
    "To actually reproduce the results in the original Tensor Sketch paper,\n",
    "select 100,000 for training.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=5_000,\n",
    "                                                    test_size=10_000,\n",
    "                                                    random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now scale features to the range [0, 1] to match the format of the dataset in\n",
    "the LIBSVM webpage, and then normalize to unit length as done in the\n",
    "original Tensor Sketch paper [1].\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm = make_pipeline(MinMaxScaler(), Normalizer())\n",
    "X_train = mm.fit_transform(X_train)\n",
    "X_test = mm.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a baseline, train a linear SVM on the original features and print the\n",
    "accuracy. We also measure and store accuracies and training times to\n",
    "plot them latter.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVM score on raw features: 75.62%\n"
     ]
    }
   ],
   "source": [
    "results = {}\n",
    "\n",
    "lsvm = LinearSVC()\n",
    "start = time.time()\n",
    "lsvm.fit(X_train, y_train)\n",
    "lsvm_time = time.time() - start\n",
    "lsvm_score = 100 * lsvm.score(X_test, y_test)\n",
    "\n",
    "results[\"LSVM\"] = {\"time\": lsvm_time, \"score\": lsvm_score}\n",
    "print(f\"Linear SVM score on raw features: {lsvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we train linear SVMs on the features generated by\n",
    ":class:`PolynomialCountSketch` with different values for `n_components`,\n",
    "showing that these kernel feature approximations improve the accuracy\n",
    "of linear classification. In typical application scenarios, `n_components`\n",
    "should be larger than the number of features in the input representation\n",
    "in order to achieve an improvement with respect to linear classification.\n",
    "As a rule of thumb, the optimum of evaluation score / run time cost is\n",
    "typically achieved at around `n_components` = 10 * `n_features`, though this\n",
    "might depend on the specific dataset being handled. Note that, since the\n",
    "original samples have 54 features, the explicit feature map of the\n",
    "polynomial kernel of degree four would have approximately 8.5 million\n",
    "features (precisely, 54^4). Thanks to :class:`PolynomialCountSketch`, we can\n",
    "condense most of the discriminative information of that feature space into a\n",
    "much more compact representation. We repeat the experiment 5 times to\n",
    "compensate for the stochastic nature of :class:`PolynomialCountSketch`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVM score on 250 PolynomialCountSketch features: 76.48%\n",
      "Linear SVM score on 500 PolynomialCountSketch features: 77.41%\n",
      "Linear SVM score on 1000 PolynomialCountSketch features: 77.81%\n",
      "Linear SVM score on 2000 PolynomialCountSketch features: 78.02%\n"
     ]
    }
   ],
   "source": [
    "n_runs = 3\n",
    "for n_components in [250, 500, 1000, 2000]:\n",
    "\n",
    "    ps_lsvm_time = 0\n",
    "    ps_lsvm_score = 0\n",
    "    for _ in range(n_runs):\n",
    "\n",
    "        pipeline = Pipeline(steps=[(\"kernel_approximator\",\n",
    "                                    PolynomialCountSketch(\n",
    "                                        n_components=n_components,\n",
    "                                        degree=4)),\n",
    "                                   (\"linear_classifier\", LinearSVC())])\n",
    "\n",
    "        start = time.time()\n",
    "        pipeline.fit(X_train, y_train)\n",
    "        ps_lsvm_time += time.time() - start\n",
    "        ps_lsvm_score += 100 * pipeline.score(X_test, y_test)\n",
    "\n",
    "    ps_lsvm_time /= n_runs\n",
    "    ps_lsvm_score /= n_runs\n",
    "\n",
    "    results[f\"LSVM + PS({n_components})\"] = {\n",
    "        \"time\": ps_lsvm_time, \"score\": ps_lsvm_score\n",
    "    }\n",
    "    print(f\"Linear SVM score on {n_components} PolynomialCountSketch \" +\n",
    "          f\"features: {ps_lsvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train a kernelized SVM to see how well :class:`PolynomialCountSketch`\n",
    "is approximating the performance of the kernel. This, of course, may take\n",
    "some time, as the SVC class has a relatively poor scalability. This is the\n",
    "reason why kernel approximators are so useful:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kernel-SVM score on raw featrues: 79.78%\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "ksvm = SVC(C=500., kernel=\"poly\", degree=4, coef0=0, gamma=1.)\n",
    "\n",
    "start = time.time()\n",
    "ksvm.fit(X_train, y_train)\n",
    "ksvm_time = time.time() - start\n",
    "ksvm_score = 100 * ksvm.score(X_test, y_test)\n",
    "\n",
    "results[\"KSVM\"] = {\"time\": ksvm_time, \"score\": ksvm_score}\n",
    "print(f\"Kernel-SVM score on raw featrues: {ksvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, plot the resuts of the different methods against their training\n",
    "times. As we can see, the kernelized SVM achieves a higher accuracy,\n",
    "but its training time is much larger and, most importantly, will grow\n",
    "much faster if the number of training samples increases.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbgAAAGpCAYAAAD/QCONAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA3XElEQVR4nO3deXhV5dn+/e+VgEQsiANqapQ5yJAQRgELURksiCgVEUpVbAEZrIh9aHnrU2trrdXya0XrI2JVFKkgzgWlFQqComXQREGGEIiagBhRFGVQkuv9Y+9sk5AJyM6wcn6OI0f2mq+92HByr7X2fZu7IyIiEjQx1V2AiIhINCjgREQkkBRwIiISSAo4EREJJAWciIgEUr3qLqAiTj/9dG/evHl1lyEiIjXI+vXrP3X3pqUtrxUB17x5c9atW1fdZYiISA1iZh+UtVyXKEVEJJAUcCIiEkgKOBERCaRacQ+uJN9++y3Z2dkcPHiwuksRqVPi4uJISEigfv361V2KSJlqbcBlZ2fTqFEjmjdvjplVdzkidYK7s2fPHrKzs2nRokV1lyNSplp7ifLgwYOcdtppCjeRKmRmnHbaabpyIrVCrQ04QOEmUg30905qi1odcCIiIqVRwB2H733ve0fMmzVrFk888USV1rFo0SI6d+5Mp06daN++PQ899BArVqygV69eRdY7fPgwZ555Jrt27WLMmDE0bNiQffv2RZZPmTIFM+PTTz+t0vpFRKKh1j5kUlNNmDAhqvt3d9ydmJjQ/02+/fZbxo8fz5o1a0hISODQoUNkZWXRpk0bsrOzycrKoqCbs6VLl9KxY0fi4+MBaN26NS+++CI/+clPyM/PZ/ny5Zx99tlRrV9EpKrUqRbcrn27aHVfKz7+6uOoHeP2229nxowZAFx44YX86le/okePHiQmJrJq1SoA8vLymDZtGt27dyc5OZmHHnoIgK+++op+/frRpUsXkpKSePHFFwHIysqiXbt2TJo0iS5duvDRRx9Fjrdv3z4OHz7MaaedBkCDBg1o27YtMTExXHXVVSxYsCCy7vz58xk1alRketSoUZHlK1as4IILLqBePf2fR0SCoU4F3B0r7yBrbxZ3vHZHlR3z8OHDrFmzhnvvvZff/e53ADzyyCOcfPLJrF27lrVr1/Lwww+zY8cO4uLieP7553n77bdZvnw5v/jFL3B3ALZs2cK1117LO++8Q7NmzSL7P/XUUxk6dCjNmjVj1KhRzJs3j/z8fCAUYPPnzwfg0KFDvPzyy1x55ZWRbdu0aUNubi6ff/45Tz31FCNHjqyq0yIiEnV1JuB27dvFY2mPke/5PJb2WFRbcYX96Ec/AqBr165kZWUB8O9//5snnniClJQUzj//fPbs2UNGRgbuzq9//WuSk5Pp378/OTk57N69G4BmzZrRs2fPEo/x97//nWXLltGjRw9mzJjBT3/6UwC6d+/OV199xZYtW3jllVfo2bMnp5xyyhH1zZ8/n//+97/06dMnSmdBRAQI/4e91OlKVmeuR92x8g7yPdSyyfM87njtDh649IGoH7dBgwYAxMbGcvjwYSB0H+3+++/nkksuKbLunDlzyM3NZf369dSvX5/mzZtHvm900kknlXmcpKQkkpKSuOaaa2jRogVz5swBYOTIkcyfP59NmzYVuTxZYOTIkXTp0oXrrrsucl9PRKTS3X477N0Lf/0rmIXCbepUaNIktCwK6sS/aAWtt2/yvgHgm7xvqrQVV9wll1zCgw8+yLfffgvA1q1b+frrr/niiy8444wzqF+/PsuXL+eDD8ocCQII3bdbsWJFZDotLa3IJcxRo0bx5JNP8p///IehQ4cesf25557LnXfeyaRJk47/jYmIlMQ9FG4zZ4ZCrSDcZs4MzY9SS65OtOAKt94KVEYrbv/+/SQkJESmb7nllgptN3bsWLKysujSpQvuTtOmTXnhhRcYPXo0l112Gd26dSMlJYXzzjuv3H25O/fccw833HADJ554IieddFKk9QbQvn17GjZsSNeuXUttBd5www0VqltE5JiYhVpuEAq1mTNDr6dM+a5FF43DepSvgVaGbt26efEBTzdt2kS7du0qtH3CXxLI2ZdzxPyzG51N9i3ZlVKjSF1yNH//RCLcofCtkPz84wo3M1vv7t1KW14nWnAKMRGRalZwWbKwqVOj2oKrE/fgRESkGhW+5zZlSqjlNmVK0XtyUVAnWnAiIlKNzEJPSxa+51ZwT65Jk6i14BRwIiISfbffHmqpFYRZQchFcXQKXaIUEZGqUTzMojz0kgJOREQCSQF3HDRcTsnmzJlD06ZNSUlJoX379jz88MNlrn/hhRdS/GsgVaV3797lrlP4z3nr1q0MHjyY1q1b065dO0aMGBHpTq2yvPDCC7z//vuR6bfeeovzzz+flJQU2rVrx+3hXh8Kd+xdEX/84x/LXWfMmDE888wzR12zSE2kgKtkEyZM4Nprr43a/t090pkyfDdczj//+U/S09N55513uPDCC+nbt29kuJwCpQ2XA1R4uJysrCwuvPDCcuu8+uqrSUtLY8WKFfz617+u9BCoLKtXr67wugcPHuTSSy9l4sSJbNu2jU2bNjFx4kRyc3MrtabiAXfdddcxe/Zs0tLS2LBhAyNGjDim/VYk4ESCpM4E3Lx50Lx56DuGzZuHpqNBw+UUdcYZZ9CqVSs++OADli1bRufOnUlKSuKnP/0phw4dKrLuI488wtRC35N5+OGHueWWWyLvf9y4cXTo0IGBAwdy4MABINQ1Wc+ePUlOTmbYsGF8/vnnQOjcT506lb59+9KuXTvWrl3Lj370I9q0acP//u//Ro5R0Dor7dwX9o9//INevXpx2WWXReZddNFFdOzYkYMHD3L99deTlJRE586dWb58ORBqzd54442R9YcMGRLpWu173/set956K506daJnz57s3r2b1atX89JLLzFt2jRSUlLIzMzkk08+ifynJDY2lvbt2x9R28MPP8ygQYM4cOAATz75JD169CAlJYUbbriBvLw8pk+fzoEDB0hJSWH06NEAPPHEEyQnJ9OpUyeuueaayL5WrlxJ7969admypVpzUrsVDKBZk3+6du3qxb3//vtHzCvNk0+6N2zoHnqEJ/TTsGFo/vE46aSTjpj329/+1v/85z+7u3tqaqrfcsst7u6+ePFi79evn7u7P/TQQ37HHXe4u/vBgwe9a9euvn37dv/222/9iy++cHf33Nxcb9Wqlefn5/uOHTvczPzNN98ssY6f/exn3rRpUx85cqQ/+eSTnpeX5+7ua9as8ZSUlMhxmjZt6p999pm7u1933XW+cOFCP//88/2zzz7zsWPH+ooVK7xZs2aem5tb6nvesWOHp6amlnleHnvsMZ88ebK7u2dmZnrTpk09JyfHExISfMuWLe7ufs011/hf//rXyHlau3atf/XVV96yZUv/5ptv3N29V69e/u677/qOHTs8NjbW33nnHXd3v+qqq3zu3Lnu7p6UlOQrVqxwd/ff/OY3PmXKlMg+f/nLX7q7+7333uvx8fG+c+dOP3jwoJ999tn+6aefuvt3f4alnfvC60ydOtXvvffeEt/zjBkzfMyYMe7uvmnTJj/nnHP8wIEDRc6Fu/ull17qy5cvd3d3wF966SV3d582bVrkM1HwZ1Pgd7/7nTdp0sSvuOIKnzVrlh84cMDdv/us3X///X7ZZZf5wYMH/f333/chQ4ZEzuHEiRP98ccfL/I+3N03bNjgiYmJkT/rPXv2RI49fPhwz8vL840bN3qrVq1KfL9H8/dPJFqAdV5GdtSJFtytt8L+/UXn7d8fmh9tQRkuZ9iwYaSkpDB48GDWrVtHSkoKKSkpPPbYYyWuv2DBAlJSUhg1ahQPPfQQubm5tGjRgsTERCB02W3lypVFtjnppJO4+OKLWbRoEZs3b+bbb78lKSkJgBYtWpCSklLkXH7xxRfs3buX1NTUEvdZ0Ll0UlISHTp0ID4+ngYNGtCyZcsirWCgzHNfEa+//nqkFXTeeefRrFkztm7dWuY2J5xwAkOGDCnynkpy2223sW7dOgYOHMg//vEPfvjDH0aWzZ07l1deeYVnn32WBg0asGzZMtavX0/37t1JSUlh2bJlbN++/Yh9/uc//2H48OGcfvrpQGhcwQJXXHEFMTExtG/fvsZeWhapiDrxPbgPPzy6+ZUpKMPlPP/880DocumYMWOKjGBQkquvvpq//e1vkem0tLQy1y8wduxY/vjHP3Leeedx/fXXR+YXnEcIncuCS5RlKdgmJiamyPYxMTGRP4sC8+bNK/XcF+jQoQOvvfZaicfyUnpiqFevXpF7poX3Wb9+fSz8mHThz0dJWrVqxcSJExk3bhxNmzZlz549AHTs2JG0tDSys7Np0aIF7s51113HXXfdVeq+Cuq1Uh7RLnyuSntfIrVBnWjBnXvu0c2Ptro4XM55551HVlYW27ZtA0Itj4KWV2Hnn38+H330Ef/4xz9KDOTCTj75ZE455ZTIvc3S9lkRFTn3P/7xj1m9ejWLFy+OzFuyZAnvvfceffv2ZV74xu7WrVv58MMPadu2Lc2bNyctLY38/Hw++ugj1qxZU24tjRo1KvJ06+LFiyNBk5GRQWxsLE2aNAGgc+fOPPTQQwwdOpSdO3fSr18/nnnmGT755BMAPvvss8h7qV+/fuQz169fP55++ulIUH722WdHe8pEarw60YK7804YP77oZcqGDUPzj4eGy6m4uLg4HnvsMa666ioOHz5M9+7dmTBhQonrjhgxgrS0tCMup5bk8ccfZ8KECezfv5+WLVuWesm0PBU59yeeeCKLFi3i5ptv5uabb6Z+/fokJyczc+ZMJk2axIQJE0hKSqJevXrMmTOHBg0acMEFF9CiRQuSkpLo2LEjXbp0KbeWkSNHMm7cOO677z6eeeYZ5s6dy9SpU2nYsCH16tVj3rx5xMbGRtb/wQ9+wIwZM7j00kt59dVX+cMf/sDAgQPJz8+nfv36PPDAAzRr1ozx48eTnJxMly5dmDdvHrfeeiupqanExsbSuXPnIp8bkSCoE8PlQOipyVtvDV2WPPfcULiFHyaTGmbIkCFMnTqVfv36VXcpUgoNlyM1QXnD5dSJS5QQCrOsrFAn1llZCreaaO/evSQmJnLiiScq3ETkuNWJS5RSOzRp0qTcJw9FRCqqzrTgRESkblHAiYhIICngREQkkBRwIiISSAq441B4GJWXX36ZNm3a8GEUu0cp3nFvgd27dzNkyJDIcDmDBw8GQt1bbdmypci6N998M/fccw8rVqzAzHjkkUciy9555x3M7KiGYBERqanqTsAV/75fJX7/b9myZfz85z9nyZIlnFvB7lHy8vIq7fi33XYbAwYMID09nffff58//elPwHfddBXIz8/nmWee4eqrrwZC3XsVH22gU6dOlVaXiEh1qhsBd/vtMHXqd6HmHpoODxx5PFatWsW4ceNYvHgxrVq1AihxuBIItfhuu+02zj//fN58880Sh0sByM3N5corr6R79+50796dN954o8wadu3aVaRHleTkZCDUTVfhgFu5ciXNmzePdOV17rnncvDgQXbv3o27s2TJEgYNGnTc50REpCYIfsC5w969MHPmdyE3dWpoeu/e42rJHTp0iMsvv5wXXngh0rXTpk2bWLBgAW+88QZpaWnExsZG+ij8+uuv6dixI//973/5wQ9+wNdff03Pnj1JT0+nb9++kZGvp0yZwtSpU1m7di3PPvssY8eOLbOOyZMn87Of/YyLLrqIO++8k507dwKhoIuJiSE9PR04cjw4gOHDh7Nw4UJWr15Nly5dinS0KyJSmwX/i95m8Ne/hl7PnBn6AZgyJTS/lB7VK6J+/fr07t2bRx55hJnh/RYergTgwIEDnHHGGUCox/grr7wysn3x4VJeffVVIDTyduERnb/88ssine8Wd8kll7B9+3aWLFnCK6+8QufOndmwYQNNmzaNtOI6dOjAiy++yO9///si244YMYKrr76azZs3M2rUqKMa4VpEpCYLfgsOioZcgeMMNwgNu/L000+zdu1a/vjHPwJEhitJS0sjLS2NLVu2cHv4UmhcXFyRTnJLGy4lPz+fN998M7KPnJwcGjVqVGYtp556Kj/+8Y+ZO3cu3bt3j4yLNmrUKJ5++mmWLl1KcnJyJGwLnHXWWdSvX59XX31V3WOJSKDUjYAruCxZWOF7csehYcOGLFq0iHnz5vHII4+UOVxJRQ0cOPCoxlL7z3/+w/7wUAn79u0jMzMz8rBLq1atOO2005g+fXqpw8/8/ve/5+677y4SviIitV3wA67wPbcpU0K9LU+ZUvSe3HE69dRTWbJkCX/4wx/IyMiIDFeSnJzMgAED2LVr11Ht77777mPdunUkJyfTvn17Zs2aVeb669evp1u3biQnJ9OrVy/Gjh0buUQKoVbc5s2bGTZsWInb9+7dmyuuuOKoahQRqenqxnA5t98eeqCk4LJkQeg1aVIpT1KK1DUaLkdqgvKGywn+QyYQCjH37+65FdyTO857cCIiUnMF/xJlgeJhpnATEQm0Wh1wteHyqkjQ6O+d1Ba1NuDi4uLYs2eP/rKJVCF3Z8+ePcTFxVV3KSLlqrX34BISEsjOziY3N7e6SxGpU+Li4op0DSdSU9XagKtfvz4tWrSo7jJERKSGqrWXKEVERMqigBMRkUBSwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAIpagFnZm3NLK3Qz5dmdrOZdTKzN83sPTP7p5k1jlYNIiJSd0Ut4Nx9i7unuHsK0BXYDzwP/B2Y7u5J4elp0apBRETqrqq6RNkPyHT3D4C2wMrw/FeBK6uoBhERqUOqKuBGAk+FX28AhoZfXwWcU9IGZjbezNaZ2brc3NwqKFFERIIk6gFnZicQCrSF4Vk/BSab2XqgEfBNSdu5+2x37+bu3Zo2bRrtMkVEJGDqVcExBgFvu/tuAHffDAwEMLNE4NIqqEFEROqYqrhEOYrvLk9iZmeEf8cA/wvMqoIaRESkjolqwJlZQ2AA8Fyh2aPMbCuwGdgJPBbNGkREpG6K6iVKd98PnFZs3kxgZjSPKyIiop5MREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMRkUBSwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMRkUBSwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERKJi2rRpnHfeeSQnJzNs2DD27t0bWXbXXXfRunVr2rZty7/+9a/I/PXr15OUlETr1q256aabcHcADh06xNVXX03r1q05//zzycrKKvf4CjgREYmKAQMGsGHDBt59910SExO56667AHj//feZP38+GzduZMmSJUyaNIm8vDwAJk6cyOzZs8nIyCAjI4MlS5YA8Mgjj3DKKaewbds2pk6dyq9+9atyj6+AExGpYbKysmjXrh3jxo2jQ4cODBw4kAMHDpS47rZt2+jfvz+dOnWiS5cuZGZm4u5MmzaNjh07kpSUxIIFCwBYsWIFqampjBgxgsTERKZPn868efPo0aMHSUlJZGZmAjBmzBgmTJhAnz59SExMZNGiRcf0PgYOHEi9evUA6NmzJ9nZ2QC8+OKLjBw5kgYNGtCiRQtat27NmjVr2LVrF19++SW9evXCzLj22mt54YUXIttcd911AAwfPpxly5aVe3wFnIhIDZSRkcHkyZPZuHEjTZo04dlnny1xvdGjRzN58mTS09NZvXo18fHxPPfcc6SlpZGens7SpUuZNm0au3btAiA9PZ2ZM2fy3nvvMXfuXLZu3cqaNWsYO3Ys999/f2S/WVlZvPbaayxevJgJEyZw8ODBI47dp08fUlJSjvhZunTpEes++uijDBo0CICcnBzOOeecyLKEhARycnLIyckhISHhiPnFt6lXrx4nn3wyQL2yzmGZC0VEpHq0aNGClJQUALp27VriPad9+/aRk5PDsGHDAIiLiwPg9ddfZ9SoUcTGxnLmmWeSmprK2rVrady4Md27dyc+Ph6AVq1aMXDgQACSkpJYvnx5ZN8jRowgJiaGNm3a0LJlSzZv3hypp8CqVasq9F7uvPNO6tWrx+jRowEi99UKM7NS55e2DVDizAIKOBGRGqhBgwaR17GxsSVeoizlH/1S5xffb0xMTGQ6JiaGw4cPR5YVBEtp0xBqwe3bt++I+TNmzKB///4APP744yxatIhly5ZF9pGQkMBHH30UWT87O5vvf//7JCQkRC5jFp5feJuEhAQOHz7MF198AZBX6htFlyhFRGqtxo0bk5CQELlPdejQIfbv30/fvn1ZsGABeXl55ObmsnLlSnr06HFU+164cCH5+flkZmayfft22rZte8Q6q1atIi0t7YifgnBbsmQJd999Ny+99BINGzaMbDd06FDmz5/PoUOH2LFjBxkZGfTo0YP4+HgaNWrEW2+9hbvzxBNPcPnll0e2efzxxwF45plnuPjii8t9Dwo4EZFabO7cudx3330kJyfTu3dvPv74Y4YNG0ZycjKdOnXi4osv5p577uGss846qv22bduW1NRUBg0axKxZs4iLi2Pnzp0MHjy4wvu48cYb2bdvHwMGDCAlJYUJEyYA0KFDB0aMGEH79u354Q9/yAMPPEBsbCwADz74IGPHjqV169a0atUqct/uZz/7GXv27KF169b85S9/4U9/+lO5x7eymrI1Rbdu3XzdunXVXYaISJ0wZswYhgwZwvDhw6u7lDKZ2Xp371bacrXgREQkkPSQiYhILTB58mTeeOONIvOmTJnC9ddfX+nHmjNnTqXvszoo4EREaoEHHnigukuodXSJUkREAqnMgDOzBDP7HzN70czWmtlKM/s/M7vUzBSOIiK13Lx50Lw5xMSEfs+bV90VVZ5SL1Ga2WPA2cAi4G7gEyAOSAR+CNxqZtPdfWVVFCoiIpVr3jwYPx727w9Nf/BBaBog3OlIrVbq1wTMrKO7byh1Q7MTgHPdfVspy9sCCwrNagncBqwAZhEKy8PAJHdfU1aR+pqAiEjla948FGrFNWsGFRiNptqV9zWBUltwJYWbmbUCGrr7e+7+DVBiuIW33wKkhLeLBXKA54GHgd+5+ytmNhi4B7iwQu9GREQqzYcfHt382qbC99HM7NfAH4DpZjb3KI/TD8h09w8IdY7ZODz/ZGDnUe5LJFAWLlxIhw4diImJofiViqoYFFLqrnPPPbr5tU2pAWdmPw+3vAp0cvdR7j4a6HSUxxkJPBV+fTPwZzP7CJgB/H+lHH+8ma0zs3W5ublHeTiR2qNjx44899xz9O3bt8j8qhoUUuquO++EQl1EAqHpO++snnoqW1ktuM+BJWZ2WXj632b2mpmtAv5VxnZFhO/VDQUWhmdNBKa6+znAVOCRkrZz99nu3s3duzVt2rSih5M6JCiDQrZr167Ejmwra1DI2tAdn1SP0aNh9uzQPTez0O/Zs4PxgAmUfQ/uSTN7BphmZmMJPSDyFFDf3b84imMMAt52993h6euAKeHXC4G/H33ZIiEZGRk89dRTPPzww4wYMYJnn32Wn/zkJ0esN3r0aKZPn86wYcM4ePAg+fn5RQaF/PTTT+nevXukFZWens6mTZs49dRTadmyJWPHjmXNmjXMnDmT+++/n3vvvRf4blDIzMxMLrroIrZt2xYZk6tARYYUKUlOTg49e/aMTBcM/li/fv2jGhRyz549nH766RU8o1LXjB4dnEArrryeTFoRehLyYeAOQvfPbgOOJuBG8d3lSQjdc0sl9DTlxUDGUexLpIggDQpZXGUNClnSOF4idUFZ34ObE15+IqEHRMaZWWfgYTNb4+53lLdzM2sIDABuKDR7HDDTzOoBB4Hxx1G/1HFBGRSyJJU1KOSpp55a6jFEgqyse3Cd3f0n7n4loZDC3d9x98uAdyuyc3ff7+6nFb6k6e6vu3tXd+/k7ue7+/rjegci5ajpg0KWprIGhVQLTuqqsgJuSfihkjeBfxRe4O4vRrcskcpVkweFfP7550lISODNN9/k0ksv5ZJLLgGqblBIkaAqc8BTM2sM5Lv7V1VX0pHUk4nURLVlUEiRoDrmAU/N7CfAV6WFm5m1MrMfVEKNIiIila6spyhPA94xs/XAeiCXUP+RrQk9BfkpMD3qFYocBQ0KKSIFyrtEGUvoUf4LgHjgALAJeMXdq6y3Ml2iFBGR4o65s2UAd88DXg3/iIiI1BoatFSkAoI8KKRIUJXXk4lInRf0QSFFgqrcFlyxEQVE6pxbb/0u3Ars3x+aLyI1V0UuUW4zsz+bWfuoVyNSAwV9UEiRoKpIwCUDW4G/m9lb4XHaGpe3kUhQBH1QSJGgKjfg3H2fuz/s7r2BXwK/BXaZ2eNm1jrqFYpUs6APCikSVBW6B2dmQ83seWAm8P+AlsA/gZejXJ9ItQv6oJAiQVWRpygzgOXAn919daH5z5hZ3+iUJVKzBHlQSJGgKjPgwk9QznH335e03N1vikpVIiIix6nMS5ThnkwuqqJaREREKk1FLlGuNrO/AQuArwtmuvvbUatKRETkOFUk4HqHfxe+TOmEOmEWERGpkcoNOHfXJUoREal1KtQXpZldCnQgNB4cAKU9eCIiIlITVOR7cLOAq4GfAwZcBTSLcl0iIiLHpSJddfV292uBz939d0Av4JzoliUiInJ8KhJwB8O/95vZ94FvgRbRK0lEROT4VeQe3D/NrAnwZ+BtQk9QPhzNokRERI5XeT2ZxADL3H0v8KyZLQLi3P2LqihORETkWJXXk0k+oc6VC6YPKdxERKQ2qMg9uH+b2ZVmZlGvRuqM22+/nbPPPpuUlBRSUlJ4+eXvBqa46667aN26NW3btuVf//pXZP769etJSkqidevW3HTTTbh7dZQuIrVERe7B3QKcBBw2s4OEvirg7q5BT+W4TJ06lf/5n/8pMu/9999n/vz5bNy4kZ07d9K/f3+2bt1KbGwsEydOZPbs2fTs2ZPBgwezZMkSBg0aVE3Vi0hNV5EBTxu5e4y7n+DujcPTCrdqkpWVRbt27Rg3bhwdOnRg4MCBHDhwoMR1t23bRv/+/enUqRNdunQhMzMTd2fatGl07NiRpKQkFixYAMCKFStITU1lxIgRJCYmMn36dObNm0ePHj1ISkoiMzMTgDFjxjBhwgT69OlDYmIiixYtqtT39+KLLzJy5EgaNGhAixYtaN26NWvWrGHXrl18+eWX9OrVCzPj2muv5YUXXqjUY4tIsFTki959S/qpiuKkZBkZGUyePJmNGzfSpEkTnn322RLXGz16NJMnTyY9PZ3Vq1cTHx/Pc889R1paGunp6SxdupRp06axa9cuANLT05k5cybvvfcec+fOZevWraxZs4axY8dy//33R/ablZXFa6+9xuLFi5kwYQIHDx484th9+vSJXH4s/LN06dLIOn/7299ITk7mpz/9KZ9//jkAOTk5nHPOd1+zTEhIICcnh5ycHBISEo6YLyJSmopcopxW6HUc0ANYjzpbrjYtWrQgJSUFgK5du5KVlXXEOvv27SMnJ4dhw4YBEBcX6mXt9ddfZ9SoUcTGxnLmmWeSmprK2rVrady4Md27dyc+Ph6AVq1aMXDgQACSkpJYvnx5ZN8jRowgJiaGNm3a0LJlSzZv3hypp8CqVavKfA8TJ07kN7/5DWbGb37zG37xi1/w6KOPlnhfzcxKnS8iUpqKdLZ8WeFpMzsHuCdqFUm5GjRoEHkdGxtb4iXK0h7AKOvBjML7jYmJiUzHxMRw+PDhyLLiwVJS0PTp04d9+/YdMX/GjBn079+fM888MzJv3LhxDBkyBAi1zD766KPIsuzsbL7//e+TkJBAdnb2EfNFREpTkacoi8sGOlZ2IVK5GjduTEJCQuQ+1aFDh9i/fz99+/ZlwYIF5OXlkZuby8qVK+nRo8dR7XvhwoXk5+eTmZnJ9u3badu27RHrrFq1irS0tCN++vfvDxC5LArw/PPP07Fj6CM1dOhQ5s+fz6FDh9ixYwcZGRn06NGD+Ph4GjVqxFtvvYW788QTT3D55Zcf49kRkbqg3Bacmd1PqPcSCAViCpAexZqkksydO5cbbriB2267jfr167Nw4UKGDRvGm2++SadOnTAz7rnnHs466yw2b95c4f22bduW1NRUdu/ezaxZs4iLi2Pnzp2MHTu2yOP+ZfnlL39JWloaZkbz5s156KGHAOjQoQMjRoygffv21KtXjwceeIDY2FgAHnzwQcaMGcOBAwcYNGiQnqAUkTJZed8lMrPrCk0eBrLc/Y2oVlVMt27dfN26dVV5SCnFmDFjGDJkCMOHD6/uUkSkjjOz9e7erbTlFXnI5BngoLvnhXcYa2YN3X1/ZRUpIiJS2SoScMuA/sBX4ekTgX8DvaNVlBydyZMn88YbRRvVU6ZM4frrr6/0Y82ZM6fS9ykiEg0VCbg4dy8IN9z9KzNrGMWa5Cg98MAD1V2CiEiNU5GnKL82sy4FE2bWFSi56wwREZEaoiItuJuBhWa2MzwdD1wdtYpEREQqQUX6olwLnAdMBCYB7dx9fbQLk+o3bx40bw4xMaHf8+ZVd0UiIhVXkb4oJwMnufsGd38P+J6ZTYp+aVKd5s2D8ePhgw/APfR7/HiFnIjUHhW5BzcuPKI3AO7+OTAuahVJjXDrrbC/2BdB9u8PzRcRqQ0qEnAxhQc7NbNY4ITolSQ1wYcfHt18EZGapiIB9y/gaTPrZ2YXA08BS6JbllS3c889uvkiIjVNRQLuV4S+7D0RmBx+Pa3MLaTWu/NOaFjs244NG4bmi4jUBhV5ijLf3We5+3B3vxLYCNxf3nZSu40eDbNnQ7NmYBb6PXt2aL6ISG1Qke/BYWYpwChC33/bATwXxZqkhhg9WoEmIrVXqQFnZonASELBtgdYQGj0gYuqqDYREZFjVlYLbjOwCrjM3bcBmNnUKqlKRETkOJV1D+5K4GNguZk9bGb9ACtjfRERkRqj1IBz9+fd/WpC3XStAKYCZ5rZg2Y2sIrqExEROSYVeYrya3ef5+5DgAQgDZge7cJERESOR0W+Bxfh7p+5+0PufnG0ChIREakMRxVwIiIitYUCTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMRkUCqF60dm1lbYEGhWS2B24BeQNvwvCbAXndPiVYdIiJSN0Ut4Nx9C5ACYGaxQA7wvLvfW7COmf0/4Ito1SAiInVX1AKumH5Aprt/UDDDzAwYAVxcRTWIiEgdUlX34EYCTxWb1wfY7e4ZJW1gZuPNbJ2ZrcvNzY16gSIiEixRDzgzOwEYCiwstmgUR4ZehLvPdvdu7t6tadOm0SxRREQCqCouUQ4C3nb33QUzzKwe8COgaxUcX0RE6qCquERZUkutP7DZ3bOr4PgiIlIHRTXgzKwhMAB4rtiiku7JiYiIVJqoXqJ09/3AaSXMHxPN44qIiKgnExERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMRkUBSwNVR06ZN47zzziM5OZlhw4axd+9eALKysjjxxBNJSUkhJSWFCRMmRLZZv349SUlJtG7dmptuugl3r6bqRUTKp4CrowYMGMCGDRt49913SUxM5K677oosa9WqFWlpaaSlpTFr1qzI/IkTJzJ79mwyMjLIyMhgyZIl1VG6iEiFKOCOUlZWFu3atWPcuHF06NCBgQMHcuDAgRLX3bZtG/3796dTp0506dKFzMxM3J1p06bRsWNHkpKSWLBgAQArVqwgNTWVESNGkJiYyPTp05k3bx49evQgKSmJzMxMAMaMGcOECRPo06cPiYmJLFq06Jjex8CBA6lXrx4APXv2JDs7u8z1d+3axZdffkmvXr0wM6699lpeeOGFYzq2iEhVUMAdg4yMDCZPnszGjRtp0qQJzz77bInrjR49msmTJ5Oens7q1auJj4/nueeeIy0tjfT0dJYuXcq0adPYtWsXAOnp6cycOZP33nuPuXPnsnXrVtasWcPYsWO5//77I/vNysritddeY/HixUyYMIGDBw8ecew+ffpELjMW/lm6dOkR6z766KMMGjQoMr1jxw46d+5Mamoqq1atAiAnJ4eEhITIOgkJCeTk5BzbCRQRqQL1qruA2qhFixakpKQA0LVrV7Kyso5YZ9++feTk5DBs2DAA4uLiAHj99dcZNWoUsbGxnHnmmaSmprJ27VoaN25M9+7diY+PB0KXCQcOHAhAUlISy5cvj+x7xIgRxMTE0KZNG1q2bMnmzZsj9RQoCKby3HnnndSrV4/Ro0cDEB8fz4cffshpp53G+vXrueKKK9i4cWOJ99vMrELHEBGpDgq4Y9CgQYPI69jY2BIvUZb2AEZZD2YU3m9MTExkOiYmhsOHD0eWFQ+WkoKmT58+7Nu374j5M2bMoH///gA8/vjjLFq0iGXLlkX20aBBg8hxu3btSqtWrdi6dSsJCQlFLmNmZ2fz/e9/v9T3IiJS3XSJMkoaN25MQkJC5D7VoUOH2L9/P3379mXBggXk5eWRm5vLypUr6dGjx1Hte+HCheTn55OZmcn27dtp27btEeusWrUq8qBI4Z+CcFuyZAl33303L730Eg0bNoxsl5ubS15eHgDbt28nIyODli1bEh8fT6NGjXjrrbdwd5544gkuv/zyYzw7IiLRp4CLorlz53LfffeRnJxM7969+fjjjxk2bBjJycl06tSJiy++mHvuuYezzjrrqPbbtm1bUlNTGTRoELNmzSIuLo6dO3cyePDgCu/jxhtvZN++fQwYMKDI1wFWrlwZqW/48OHMmjWLU089FYAHH3yQsWPH0rp1a1q1alXkvp2ISE1jteG7TN26dfN169ZVdxk1wpgxYxgyZAjDhw+v7lJERKqVma13926lLVcLTkREAkkPmVSCyZMn88YbbxSZN2XKFK6//vpKP9acOXMqfZ8iIkGkgKsEDzzwQHWXICIixegSpYiIBJICrgabNAnq1QOz0O9Jk6q7IhGR2kOXKGuoSZPgwQe/m87L+276//6vemoSEalN1IKroWbPPrr5IiJSlAKuhgp3JlLh+SIiUpQCroaKjT26+SIiUpQCroYaP/7o5ouISFF6yKSGKniQZPbs0GXJ2NhQuOkBExGRilHA1WD/938KNBGRY6VLlCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMRkUBSwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkKIWcGbW1szSCv18aWY3h5f93My2mNlGM7snWjWIiEjdVS9aO3b3LUAKgJnFAjnA82Z2EXA5kOzuh8zsjGjVICIidVdVXaLsB2S6+wfAROBP7n4IwN0/qaIaRESkDqmqgBsJPBV+nQj0MbP/mtlrZta9pA3MbLyZrTOzdbm5uVVUpoiIBEXUA87MTgCGAgvDs+oBpwA9gWnA02Zmxbdz99nu3s3duzVt2jTaZYqISMBURQtuEPC2u+8OT2cDz3nIGiAfOL0K6hARkTqkKgJuFN9dngR4AbgYwMwSgROAT6ugDhERqUOiGnBm1hAYADxXaPajQEsz2wDMB65zd49mHSIiUvdE7WsCAO6+Hzit2LxvgJ9E87giIiLqyURERAJJASciIoGkgBMRkUBSwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSAo4EREJJAWciIgEkgJOREQCSQEnIiKBpIATEZFAUsCJiEggKeBERCSQFHAiIhJICjgREQmkOhNwu/btotV9rfj4q4+ruxQREakCdSbg7lh5B1l7s7jjtTuquxQREakCdSLgdu3bxWNpj5Hv+TyW9phacSIidUCdCLg7Vt5BvucDkOd5asWJiNQBgQ+4gtbbN3nfAPBN3jdqxYmI1AGBD7jCrbcCasWJiARf4APupS0vRVpvBb7J+4YXt7xYTRWJiEhVqFfdBURb9i3Z1V2CiIhUg8C34EREpG5SwImISCAp4EREJJAUcCIiEkgKOBERCSQFnIiIBJICTkREAkkBJyIigaSAExGRQFLAiYhIICngREQkkBRwIiISSObu1V1DucwsF/jgKDc7Hfg0CuXUJTqHx0/n8PjpHB6/oJ7DZu7etLSFtSLgjoWZrXP3btVdR22mc3j8dA6Pn87h8aur51CXKEVEJJAUcCIiEkhBDrjZ1V1AAOgcHj+dw+Onc3j86uQ5DOw9OBERqduC3IITEZE6TAEnIiKBVOsDzsx+aGZbzGybmU0vYbmZ2X3h5e+aWZfqqLMmq8A5vNDMvjCztPDPbdVRZ01lZo+a2SdmtqGU5foMlqMC51CfwTKY2TlmttzMNpnZRjObUsI6de9z6O619geIBTKBlsAJQDrQvtg6g4FXAAN6Av+t7rpr0k8Fz+GFwKLqrrWm/gB9gS7AhlKW6zN4/OdQn8Gyz1880CX8uhGwVf8Weq1vwfUAtrn7dnf/BpgPXF5sncuBJzzkLaCJmcVXdaE1WEXOoZTB3VcCn5Wxij6D5ajAOZQyuPsud387/HofsAk4u9hqde5zWNsD7mzgo0LT2Rz5h1qRdeqyip6fXmaWbmavmFmHqiktMPQZrBz6DFaAmTUHOgP/Lbaozn0O61V3AcfJSphX/HsPFVmnLqvI+XmbUJ9vX5nZYOAFoE20CwsQfQaPnz6DFWBm3wOeBW529y+LLy5hk0B/Dmt7Cy4bOKfQdAKw8xjWqcvKPT/u/qW7fxV+/TJQ38xOr7oSaz19Bo+TPoPlM7P6hMJtnrs/V8Iqde5zWNsDbi3QxsxamNkJwEjgpWLrvARcG36CqCfwhbvvqupCa7Byz6GZnWVmFn7dg9DnZk+VV1p76TN4nPQZLFv43DwCbHL3v5SyWp37HNbqS5TuftjMbgT+RehpwEfdfaOZTQgvnwW8TOjpoW3AfuD66qq3JqrgORwOTDSzw8ABYKSHH8sSMLOnCD3ld7qZZQO/BeqDPoMVVYFzqM9g2S4ArgHeM7O08LxfA+dC3f0cqqsuEREJpNp+iVJERKRECjgREQkkBZyIiASSAk5ERAJJASciIoGkgBMJM7PTCvVW/7GZ5RSaPqGcbbuZ2X0VOMbqSqo1JdyjR8H00JJGgqikY91rZn3LWH6jmQX+kXOpffQ1AZESmNntwFfuPqPQvHrufrj6qvqOmY0Burn7jVE+zqnAy+7es4x1GgJvuHvnaNYicrTUghMpg5nNMbO/mNly4G4z62Fmq83snfDvtuH1LjSzReHXt4fHN1thZtvN7KZC+/uq0PorzOwZM9tsZvMK9dQxODzv9fD4XYuK1XQC8Hvg6nDr8mozG2NmfytU84MWGh9su5mlhuvZZGZzCu1noJm9aWZvm9nCcD+GxQ0HlhTa5k9m9r6FxhObAeDu+4GscA8jIjVGre7JRKSKJAL93T3PzBoDfcM9wPQH/ghcWcI25wEXERqba4uZPeju3xZbpzPQgVB/gG8AF5jZOuCh8DF2hHv4KMLdv7HQgJ+RFly4RVfYKcDFwFDgn4R6uhgLrDWzFEL9Ev5v+H19bWa/Am4hFJyFXQA8Ez7GqcAw4Dx3dzNrUmi9dUAfYE0J50KkWijgRMq30N3zwq9PBh43szaEemKvX8o2i939EHDIzD4BziQUKoWtcfdsgHD3Ss2Br4Dt7r4jvM5TwPhjqPmf4RB6D9jt7u+Fj7MxfJwEoD3wRrjheALwZgn7iQdyw6+/BA4CfzezxUDhluUnhEJdpMZQwImU7+tCr+8Alrv7MAuNu7WilG0OFXqdR8l/10pap6QhTY5Fwb7zix0nP3ycPOBVdx9Vzn4OAHEQ6be0B9CPUKfcNxJqJRJe50DllC5SOXQPTuTonAzkhF+PicL+NwMtw+EJcHUp6+0jdPnzWL1F6JJoawg9KGJmiSWstwkoWOd7wMnh4WpuBlIKrZcIbDiOekQqnQJO5OjcA9xlZm8QGn2hUrn7AWASsMTMXgd2A1+UsOpyoH3BQybHcJxcQgH9lJm9SyjwSrrEuJhQL/8QCtRF4fVfA6YWWu8CYOnR1iESTfqagEgNY2bfC49cbcADQIa7/7Ua63kdGOLue0tZ3hm4xd2vqdLCRMqhFpxIzTMu/NDJRkKXRB+q3nL4BeFxxUpxOvCbKqpFpMLUghMRkUBSC05ERAJJASciIoGkgBMRkUBSwImISCAp4EREJJD+f/j69iXo1OwcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N_COMPONENTS = [250, 500, 1000, 2000]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 7))\n",
    "ax.scatter([results[\"LSVM\"][\"time\"], ], [results[\"LSVM\"][\"score\"], ],\n",
    "           label=\"Linear SVM\", c=\"green\", marker=\"^\")\n",
    "\n",
    "ax.scatter([results[\"LSVM + PS(250)\"][\"time\"], ],\n",
    "           [results[\"LSVM + PS(250)\"][\"score\"], ],\n",
    "           label=\"Linear SVM + PolynomialCountSketch\", c=\"blue\")\n",
    "for n_components in N_COMPONENTS:\n",
    "    ax.scatter([results[f\"LSVM + PS({n_components})\"][\"time\"], ],\n",
    "               [results[f\"LSVM + PS({n_components})\"][\"score\"], ],\n",
    "               c=\"blue\")\n",
    "    ax.annotate(f\"n_comp.={n_components}\",\n",
    "                (results[f\"LSVM + PS({n_components})\"][\"time\"],\n",
    "                 results[f\"LSVM + PS({n_components})\"][\"score\"]),\n",
    "                xytext=(-30, 10), textcoords=\"offset pixels\")\n",
    "\n",
    "ax.scatter([results[\"KSVM\"][\"time\"], ], [results[\"KSVM\"][\"score\"], ],\n",
    "           label=\"Kernel SVM\", c=\"red\", marker=\"x\")\n",
    "\n",
    "ax.set_xlabel(\"Training time (s)\")\n",
    "ax.set_ylabel(\"Accurary (%)\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "[1] Pham, Ninh and Rasmus Pagh. \"Fast and scalable polynomial kernels via\n",
    "explicit feature maps.\" KDD '13 (2013).\n",
    "https://doi.org/10.1145/2487575.2487591\n",
    "\n",
    "[2] LIBSVM binary datasets repository\n",
    "https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-07-19 11:03:56,500\tINFO services.py:1267 -- View the Ray dashboard at \u001b[1m\u001b[32mhttp://127.0.0.1:8266\u001b[39m\u001b[22m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAGpCAYAAADhiRM+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6oklEQVR4nO3deXxU5d3//9eVBIlYEFHUaJQlIciSELYIWIiyREFEU2mA4oIVkMVbxP6o/OqttbXUpbQVLbcIRRFMBVEUixqrFARRy1ITZQ+BqAloIxYEgSjJ5/vHTMYkJCGBTJaT9/PxyCMz15zlM4eBN9d1zpzLmRkiIiJeEVLbBYiIiFQnBZuIiHiKgk1ERDxFwSYiIp6iYBMREU8Jq+0CKuO8886z1q1b13YZIiJSh2zatOkrM2tZur1eBFvr1q3ZuHFjbZchIiJ1iHPu07LaNRQpIiKeomATERFPUbCJiIin1ItzbGX5/vvvycnJ4dixY7VdikiDEh4eTmRkJI0aNartUkTKVG+DLScnh6ZNm9K6dWucc7VdjkiDYGbs37+fnJwc2rRpU9vliJSp3g5FHjt2jHPPPVehJlKDnHOce+65GimROq3eBhugUBOpBfp7J3VdvQ42ERGR0hRsp+FHP/rRCW1z5sxh4cKFNVrHihUr6Nq1K126dKFjx448/fTTrF69mt69e5dY7vjx41xwwQXs27ePMWPG0KRJEw4dOhR4fcqUKTjn+Oqrr2q0fhGR6lRvLx6pqyZMmBDU7ZsZZkZIiO//JN9//z3jx49n/fr1REZGkp+fT3Z2Nu3atSMnJ4fs7GyKbkf2zjvv0LlzZyIiIgCIjo5m+fLl3HTTTRQWFrJq1SouvvjioNYvIhJsDarHtu/QPqKeiOKLw18EbR8PPvggM2fOBODKK6/k3nvvJSEhgZiYGNauXQtAQUEB06ZNo2fPnsTFxfH0008DcPjwYQYMGEC3bt2IjY1l+fLlAGRnZ9OhQwcmTZpEt27d+PzzzwP7O3ToEMePH+fcc88FoHHjxrRv356QkBB++tOfsmTJksCyixcvZtSoUYHno0aNCry+evVqrrjiCsLC9H8dEanfGlSwPbTmIbIPZPPQuw/V2D6PHz/O+vXrefzxx/nNb34DwPz58zn77LPZsGEDGzZsYN68eezZs4fw8HBeeeUV/v3vf7Nq1Sp+8YtfYGYA7Nixg1tuuYWPPvqIVq1aBbbfokULhg0bRqtWrRg1ahSpqakUFhYCvuBavHgxAPn5+bzxxhvceOONgXXbtWtHXl4e//3vf3nhhRcYOXJkTR0WEZGgaTDBtu/QPp5Nf5ZCK+TZ9GeD2msr7ic/+QkA3bt3Jzs7G4B//OMfLFy4kPj4eC6//HL2799PZmYmZsavfvUr4uLiGDhwILm5uXz55ZcAtGrVil69epW5j7/+9a+sXLmShIQEZs6cyc9//nMAevbsyeHDh9mxYwdvvvkmvXr14pxzzjmhvsWLF/Ovf/2Lvn37BukoiIgA/v+ol/u8mjSYcaeH1jxEofl6MgVWwEPvPsTsa2cHfb+NGzcGIDQ0lOPHjwO+82RPPvkkV199dYllFyxYQF5eHps2baJRo0a0bt068H2hs846q8L9xMbGEhsby80330ybNm1YsGABACNHjmTx4sVs27atxDBkkZEjR9KtWzduvfXWwHk7EZFq9+CDcOAA/PnP4Jwv1KZOhebNfa9VowbxL1lRb+27gu8A+K7guxrttZV29dVX89RTT/H9998DsHPnTr799lsOHjzI+eefT6NGjVi1ahWfflrmjAwlHD58mNWrVweep6enlxiqHDVqFM8//zz//Oc/GTZs2AnrX3rppcyYMYNJkyad/hsTESmLmS/UZs3yhVlRqM2a5Wuv5p5bg+ixFe+tFamOXtuRI0eIjIwMPL/nnnsqtd7YsWPJzs6mW7dumBktW7bk1VdfZfTo0Vx33XX06NGD+Ph4LrvsspNuy8x47LHHuOOOOzjzzDM566yzAr01gI4dO9KkSRO6d+9ebq/vjjvuqFTdIiKnxDlfTw18YTZrlu/xlCk/9OCqc3cWpDHO6tSjRw8rPdHotm3b6NChQ6XWj/xTJLmHck9ov7jpxeTck1MtNYo0JFX5+ycSYAbFT3kUFp5WqDnnNplZj9LtDaLHpvASEallRcOPxU2dGpQeW4M4xyYiIrWo+Dm1KVN8PbUpU0qec6tGDaLHJiIitcg539WPxc+pFZ1za9682ntsCjYREQm+Bx/09cyKQqwo3IIwW4SGIkVEpGaUDrEgTYGkYBMREU9RsJ0GTVtTtgULFtCyZUvi4+Pp2LEj8+bNq3D5K6+8ktJf56gpffr0Oekyxf+cd+7cyZAhQ4iOjqZDhw6kpKQEbntWXV599VW2bt0aeP7hhx9y+eWXEx8fT4cOHXjQf5eG4jfcrozf//73J11mzJgxvPTSS1WuWaQuUbBVswkTJnDLLbcEbftmFrjJMfwwbc3f//53MjIy+Oijj7jyyivp169fYNqaIuVNWwNUetqa7OxsrrzyypPWOWLECNLT01m9ejW/+tWvqv0f/+ry/vvvV3rZY8eOce211zJx4kR27drFtm3bmDhxInl5edVaU+lgu/XWW5k7dy7p6els3ryZlJSUU9puZYJNxAsaTLClpkLr1r7vBrZu7XseDJq2pqTzzz+fqKgoPv30U1auXEnXrl2JjY3l5z//Ofn5+SWWnT9/PlOLfc9l3rx53HPPPYH3P27cODp16kRSUhJHjx4FfLcQ69WrF3FxcSQnJ/Pf//4X8B37qVOn0q9fPzp06MCGDRv4yU9+Qrt27fjf//3fwD6KemPlHfvi/va3v9G7d2+uu+66QNtVV11F586dOXbsGLfddhuxsbF07dqVVatWAb7e65133hlYfujQoYFboP3oRz/ivvvuo0uXLvTq1Ysvv/yS999/n9dee41p06YRHx9PVlYW//nPfwL/GQkNDaVjx44n1DZv3jwGDx7M0aNHef7550lISCA+Pp477riDgoICpk+fztGjR4mPj2f06NEALFy4kLi4OLp06cLNN98c2NaaNWvo06cPbdu2Ve9N6qeiiSvr8k/37t2ttK1bt57QVp7nnzdr0sTMd0mO76dJE1/76TjrrLNOaPv1r39tf/jDH8zMLDEx0e655x4zM3v99ddtwIABZmb29NNP20MPPWRmZseOHbPu3bvb7t277fvvv7eDBw+amVleXp5FRUVZYWGh7dmzx5xz9sEHH5RZx+23324tW7a0kSNH2vPPP28FBQVmZrZ+/XqLj48P7Kdly5b29ddfm5nZrbfeakuXLrXLL7/cvv76axs7dqytXr3aWrVqZXl5eeW+5z179lhiYmKFx+XZZ5+1yZMnm5lZVlaWtWzZ0nJzcy0yMtJ27NhhZmY333yz/fnPfw4cpw0bNtjhw4etbdu29t1335mZWe/eve3jjz+2PXv2WGhoqH300UdmZvbTn/7UFi1aZGZmsbGxtnr1ajMzu//++23KlCmBbf7yl780M7PHH3/cIiIibO/evXbs2DG7+OKL7auvvjKzH/4Myzv2xZeZOnWqPf7442W+55kzZ9qYMWPMzGzbtm12ySWX2NGjR0scCzOza6+91latWmVmZoC99tprZmY2bdq0wGei6M+myG9+8xtr3ry53XDDDTZnzhw7evSomf3wWXvyySftuuuus2PHjtnWrVtt6NChgWM4ceJEe+6550q8DzOzzZs3W0xMTODPev/+/YF9Dx8+3AoKCmzLli0WFRVV5vutyt8/kWABNloZmdEgemz33QdHjpRsO3LE1x5sXpm2Jjk5mfj4eIYMGcLGjRuJj48nPj6eZ599tszllyxZQnx8PKNGjeLpp58mLy+PNm3aEBMTA/iG19asWVNinbPOOov+/fuzYsUKtm/fzvfff09sbCwAbdq0IT4+vsSxPHjwIAcOHCAxMbHMbRbd9Dk2NpZOnToRERFB48aNadu2bYleL1Dhsa+M9957L9Drueyyy2jVqhU7d+6scJ0zzjiDoUOHlnhPZXnggQfYuHEjSUlJ/O1vf+Oaa64JvLZo0SLefPNNXn75ZRo3bszKlSvZtGkTPXv2JD4+npUrV7J79+4TtvnPf/6T4cOHc9555wG+ef2K3HDDDYSEhNCxY8c6O4QsUpEG8T22zz6rWnt18sq0Na+88grgGxYdM2ZMiRkFyjJixAj+8pe/BJ6np6dXuHyRsWPH8vvf/57LLruM2267LdBedBzBdyyLhiIrUrROSEhIifVDQkICfxZFUlNTyz32RTp16sS7775b5r6snDsnhIWFlTgnWnybjRo1wvkvdy7++ShLVFQUEydOZNy4cbRs2ZL9+/cD0LlzZ9LT08nJyaFNmzaYGbfeeisPP/xwudsqqteVc6l18WNV3vsSqcsaRI/t0kur1h5sDXHamssuu4zs7Gx27doF+HoaRT2t4i6//HI+//xz/va3v5UZxMWdffbZnHPOOYFzl+VtszIqc+x/9rOf8f777/P6668H2tLS0vjkk0/o168fqf4Ttzt37uSzzz6jffv2tG7dmvT0dAoLC/n8889Zv379SWtp2rRpiatVX3/99UDAZGZmEhoaSvPmzQHo2rUrTz/9NMOGDWPv3r0MGDCAl156if/85z8AfP3114H30qhRo8BnbsCAAbz44ouBgPz666+reshE6qwG0WObMQPGjy85HNmkia/9dGjamsoLDw/n2Wef5ac//SnHjx+nZ8+eTJgwocxlU1JSSE9PP2HYtCzPPfccEyZM4MiRI7Rt27bcodGTqcyxP/PMM1mxYgV33303d999N40aNSIuLo5Zs2YxadIkJkyYQGxsLGFhYSxYsIDGjRtzxRVX0KZNG2JjY+ncuTPdunU7aS0jR45k3LhxPPHEE7z00kssWrSIqVOn0qRJE8LCwkhNTSU0NDSw/I9//GNmzpzJtddey9tvv83vfvc7kpKSKCwspFGjRsyePZtWrVoxfvx44uLi6NatG6mpqdx3330kJiYSGhpK165dS3xuROqzBjFtDfiugrzvPt/w46WX+kLNf3GY1DFDhw5l6tSpDBgwoLZLkXJo2hqpC8qbtqZBDEWCL8Sys303lc7OVqjVRQcOHCAmJoYzzzxToSYip6xBDEVK/dC8efOTXkkoInIyDabHJiIiDYOCTUREPEXBJiIinqJgExERT1GwnYbi05m88cYbtGvXjs+CeDuT0jfULfLll18ydOjQwLQ1Q4YMAXy3odqxY0eJZe+++24ee+wxVq9ejXOO+fPnB1776KOPcM5VaSoUEZG6puEEW+nv61Xj9/dWrlzJ//zP/5CWlsallbydSUFBQbXt/4EHHmDQoEFkZGSwdetWHnnkEeCH22kVKSws5KWXXmLEiBGA7zZcpe/+36VLl2qrS0SkNjSMYHvwQZg69YcwM/M990/YeDrWrl3LuHHjeP3114mKigIoc9oQ8PXwHnjgAS6//HI++OCDMqctAcjLy+PGG2+kZ8+e9OzZk3Xr1lVYw759+0rcASUuLg7w3U6reLCtWbOG1q1bB265demll3Ls2DG+/PJLzIy0tDQGDx582sdERKQ2eT/YzODAAZg164dwmzrV9/zAgdPqueXn53P99dfz6quvBm7BtG3bNpYsWcK6detIT08nNDQ0cA/Bb7/9ls6dO/Ovf/2LH//4x3z77bf06tWLjIwM+vXrF5hpesqUKUydOpUNGzbw8ssvM3bs2ArrmDx5MrfffjtXXXUVM2bMYO/evYAv4EJCQsjIyABOnI8NYPjw4SxdupT333+fbt26lbgBrohIfeT9L2g7B3/+s+/xrFm+H4ApU3zt5dzhvDIaNWpEnz59mD9/PrP82y0+bQjA0aNHOf/88wHfHdxvvPHGwPqlpy15++23Ad9M18VnUP7mm29K3BS3tKuvvprdu3eTlpbGm2++SdeuXdm8eTMtW7YM9No6derE8uXL+e1vf1ti3ZSUFEaMGMH27dsZNWpUlWaUFhGpi7zfY4OS4VbkNEMNfNOfvPjii2zYsIHf//73AIFpQ9LT00lPT2fHjh086B/yDA8PL3Hz2vKmLSksLOSDDz4IbCM3N5emTZtWWEuLFi342c9+xqJFi+jZs2dgXrJRo0bx4osv8s477xAXFxcI2SIXXnghjRo14u2339ZtrETEExpGsBUNPxZX/JzbaWjSpAkrVqwgNTWV+fPnVzhtSGUlJSVVaS6zf/7znxzxT11w6NAhsrKyAhexREVFce655zJ9+vRyp4H57W9/y6OPPloidEVE6ivvB1vxc2pTpvjugjxlSslzbqepRYsWpKWl8bvf/Y7MzMzAtCFxcXEMGjSIffv2VWl7TzzxBBs3biQuLo6OHTsyZ86cCpfftGkTPXr0IC4ujt69ezN27NjAUCj4em3bt28nOTm5zPX79OnDDTfcUKUaRUTqqoYxbc2DD/ouFCkafiwKu+bNq+XKSJGGRtPWSF1Q3rQ13r94BHzhZfbDObWic26neY5NRETqHu8PRRYpHWIKNRERT6rXwVYfhlFFvEZ/76Suq7fBFh4ezv79+/WXTKQGmRn79+8nPDy8tksRKVe9PccWGRlJTk4OeXl5tV2KSIMSHh5e4hZuInVNvQ22Ro0a0aZNm9ouQ0RE6ph6OxQpIiJSFgWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKUELNudce+dcerGfb5xzdzvnujjnPnDOfeKc+7tzrlmwahARkYYnaMFmZjvMLN7M4oHuwBHgFeCvwHQzi/U/nxasGkREpOGpqaHIAUCWmX0KtAfW+NvfBm6soRpERKQBqKlgGwm84H+8GRjmf/xT4JKyVnDOjXfObXTObczLy6uBEkVExAuCHmzOuTPwBdlSf9PPgcnOuU1AU+C7stYzs7lm1sPMerRs2TLYZYqIiEeE1cA+BgP/NrMvAcxsO5AE4JyLAa6tgRpERKSBqImhyFH8MAyJc+58/+8Q4H+BOTVQg4iINBBBDTbnXBNgELCsWPMo59xOYDuwF3g2mDWIiEjDEtShSDM7Apxbqm0WMCuY+xURkYZLdx4RERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYRqXemTZvGZZddRlxcHMnJyRw4cCDw2sMPP0x0dDTt27fnrbfeCrRv2rSJ2NhYoqOjueuuuzAzAPLz8xkxYgTR0dFcfvnlZGdn1/C7keqmYBORemfQoEFs3ryZjz/+mJiYGB5++GEAtm7dyuLFi9myZQtpaWlMmjSJgoICACZOnMjcuXPJzMwkMzOTtLQ0AObPn88555zDrl27mDp1Kvfee2+tvS+pHgo2kQYkOzubDh06MG7cODp16kRSUhJHjx4tc9ldu3YxcOBAunTpQrdu3cjKysLMmDZtGp07dyY2NpYlS5YAsHr1ahITE0lJSSEmJobp06eTmppKQkICsbGxZGVlATBmzBgmTJhA3759iYmJYcWKFaf0PpKSkggLCwOgV69e5OTkALB8+XJGjhxJ48aNadOmDdHR0axfv559+/bxzTff0Lt3b5xz3HLLLbz66quBdW699VYAhg8fzsqVKwO9OamfFGwiDUxmZiaTJ09my5YtNG/enJdffrnM5UaPHs3kyZPJyMjg/fffJyIigmXLlpGenk5GRgbvvPMO06ZNY9++fQBkZGQwa9YsPvnkExYtWsTOnTtZv349Y8eO5cknnwxsNzs7m3fffZfXX3+dCRMmcOzYsRP23bdvX+Lj40/4eeedd05Y9plnnmHw4MEA5ObmcskllwRei4yMJDc3l9zcXCIjI09oL71OWFgYZ599Nvv376/qYZU6JKy2CxCRmtWmTRvi4+MB6N69e5nnlA4dOkRubi7JyckAhIeHA/Dee+8xatQoQkNDueCCC0hMTGTDhg00a9aMnj17EhERAUBUVBRJSUkAxMbGsmrVqsC2U1JSCAkJoV27drRt25bt27cH6imydu3aSr2XGTNmEBYWxujRowHK7Gk558ptr2gdqb8UbCINTOPGjQOPQ0NDyxyKLG8orqIhuuLbDQkJCTwPCQnh+PHjgddKh0ZZIdK3b18OHTp0QvvMmTMZOHAgAM899xwrVqxg5cqVgW1ERkby+eefB5bPycnhoosuIjIyMjBcWby9+DqRkZEcP36cgwcP0qJFi3Lfp9R9GooUkRM0a9aMyMjIwHmo/Px8jhw5Qr9+/ViyZAkFBQXk5eWxZs0aEhISqrTtpUuXUlhYSFZWFrt376Z9+/YnLLN27VrS09NP+CkKtbS0NB599FFee+01mjRpElhv2LBhLF68mPz8fPbs2UNmZiYJCQlERETQtGlTPvzwQ8yMhQsXcv311wfWee655wB46aWX6N+/v3ps9ZyCTUTKtGjRIp544gni4uLo06cPX3zxBcnJycTFxdGlSxf69+/PY489xoUXXlil7bZv357ExEQGDx7MnDlzCA8PZ+/evQwZMqTS27jzzjs5dOgQgwYNIj4+ngkTJgDQqVMnUlJS6NixI9dccw2zZ88mNDQUgKeeeoqxY8cSHR1NVFRU4Lzc7bffzv79+4mOjuZPf/oTjzzySJXej9Q9rj5c/dOjRw/buHFjbZchIqdpzJgxDB06lOHDh9d2KeIBzrlNZtajdLt6bCIi4im6eESkgZs8eTLr1q0r0TZlyhRuu+22at/XggULqn2bIqUp2EQauNmzZ9d2CSLVSkORIiLiKQo2EQmK1FRo3RpCQny/U1NruyJpKDQUKSLVLjUVxo+HI0d8zz/91PccwH+TEJGgUY9NRKrdfff9EGpFjhzxtYsEm4JNRKrdZ59VrV2kOinYRKTaXXpp1dpFqpOCTUSq3YwZUOwWjoDv+YwZtVOPNCwKNhGpdqNHw9y50KoVOOf7PXeuLhyRmqFgE5GgGD0asrOhsND3uzpDbenSpXTq1ImQkBBK30f24YcfJjo6mvbt2/PWW28F2jdt2kRsbCzR0dHcddddgSl48vPzGTFiBNHR0Vx++eVlzk8n9UuFweaci3TO/X/OueXOuQ3OuTXOuf9zzl3rnFMoikit6Ny5M8uWLaNfv34l2rdu3crixYvZsmULaWlpTJo0iYKCAgAmTpzI3LlzyczMJDMzk7S0NADmz5/POeecw65du5g6dSr33ntvjb8fqV7lhpNz7lngGeA74FFgFDAJeAe4BnjPOdevvPVFpO7Jzs6mQ4cOjBs3jk6dOpGUlFTmRKMAu3btYuDAgXTp0oVu3bqRlZWFmTFt2jQ6d+5MbGwsS5YsAWD16tUkJiaSkpJCTEwM06dPJzU1lYSEBGJjY8nKygJ8d/efMGECffv2JSYmhhUrVpzS++jQoUOZ87gtX76ckSNH0rhxY9q0aUN0dDTr169n3759fPPNN/Tu3RvnHLfccktgrrnly5dz6623AjB8+HBWrlxZ4YSqUvdV9AXtP5rZ5jLaNwPLnHNnAOVe4+Scaw8sKdbUFngAWA3MAcKB48AkM1tfxbpF5BRlZmbywgsvMG/ePFJSUnj55Ze56aabTlhu9OjRTJ8+neTkZI4dO0ZhYSHLli0jPT2djIwMvvrqK3r27BnoNWVkZLBt2zZatGhB27ZtGTt2LOvXr2fWrFk8+eSTPP7444AvXN99912ysrK46qqr2LVrF+Hh4SX2XZkZtMuSm5tLr169As8jIyPJzc2lUaNGREZGntBetM4ll1wCQFhYGGeffTb79+/nvPPOq+QRlbqm3GArK9Scc1FAEzP7xMy+A3ZVsP4OIN6/XiiQC7wCzAN+Y2ZvOueGAI8BV57GexCRKmjTpg3x8fEAdO/evcxzSocOHSI3N5fk5GSAQPC89957jBo1itDQUC644AISExPZsGEDzZo1o2fPnkRERAAQFRVFUlISALGxsaxatSqw7ZSUFEJCQmjXrh1t27Zl+/btgXqKrF279pTeW1k9Ledcue0VrSP1V6VvqeWc+xUQCxQ65wrN7OYq7GcAkGVmnzrnDGjmbz8b2FuF7YjIaWrcuHHgcWhoaJlDkeUNxVU0RFd8uyEhIYHnISEhHD9+PPBa6dAoK0ROtccWGRnJ559/Hniek5PDRRddRGRkJDk5OSe0F18nMjKS48ePc/DgQVq0aFHuPqTuq+gc2//4e1pFupjZKDMbDXSp4n5GAi/4H98N/ME59zkwE/j/y9n/eOfcRufcxry8vCruTkROR7NmzYiMjAych8rPz+fIkSP069ePJUuWUFBQQF5eHmvWrCEhIaFK2166dCmFhYVkZWWxe/fuMs+VrV27lvT09BN+Kgo1gGHDhrF48WLy8/PZs2cPmZmZJCQkEBERQdOmTfnwww8xMxYuXMj1118fWOe5554D4KWXXqJ///7qsdVzFV3Z+F8gzTl3nf/5P5xz7zrn1gJvVbBeCf5zccOApf6micBUM7sEmArML2s9M5trZj3MrEfLli0ruzsRqSaLFi3iiSeeIC4ujj59+vDFF1+QnJxMXFwcXbp0oX///jz22GNceOGFVdpu+/btSUxMZPDgwcyZM4fw8HD27t3LkCFDKr2NV155hcjISD744AOuvfZarr76agA6depESkoKHTt25JprrmH27NmEhvr+f/7UU08xduxYoqOjiYqKYvDgwQDcfvvt7N+/n+joaP70pz/xyCOPVOn9SN3jKhpacM6FA9OAHvgu/MgEGpnZwUrvwLnrgclmluR/fhBobmbmfP8tOmhmzSraRo8ePaz0d1VEpP4ZM2YMQ4cOZfjw4bVdiniAc26TmfUo3X6y76JF4buy8Q7gTuBx4Mwq7nsUPwxDgu+cWqL/cX98YSkiIlItyr14xDm3wP/6mfgu/BjnnOsKzHPOrTezh062cedcE2AQvmAsMg6Y5ZwLA44B40+jfhE5TZMnT2bdunUl2qZMmcJtt91W7ftasGBBtW9TpLRyhyKdcxlm1sX/+CMz61rstevNbHkN1aihSBEROUF5Q5EVXe6f5px7FzgD+FvxF2oy1ERERKqi3HNsZnYvcB0wyMz+UHMliYgXpKZC69YQEuL7nZpa2xVJQ1HR99huAg6b2eFyXo9yzv04aJWJSL2Vmgrjx8Onn4KZ7/f48Qo3qRkVDUWeC3zknNsEbALy8N3fMRrfVY1fAdODXqGI1Dv33QdHjpRsO3LE16452STYKrpX5Czn3F/wXZJ/BRAHHAW2ATeb2Wc1U6KI1DeflfOvQ3ntItWpwntFmlkB8Lb/R0SkUi691Df8WFa7SLBpslARqXYzZkCTJiXbmjTxtYsEm4JNRKrd6NEwdy60agXO+X7Pnavza1IzTjptjXMu1D8kKSJSaaNHK8ikdlSmx7bLOfcH51zHoFcjIiJymioTbHHATuCvzrkP/fOkVXg3fhERkdpy0mAzs0NmNs/M+gC/BH4N7HPOPeeciw56hSIiIlVw0mBzzoU654Y5514BZgF/BNoCfwfeCHJ9IiIiVXLSi0fwzZe2CviDmb1frP0l51y/4JQlIiJyaioMNudcKLDAzH5b1utmdldQqhIRETlFFQ5F+i/zv6qGahERqZQHH3yQiy++mPj4eOLj43njjR/Oijz88MNER0fTvn173nrrrUD7pk2biI2NJTo6mrvuuovy5qKU+q8yV0W+75z7i3Our3OuW9FP0CsTEanA1KlTSU9PJz09nSFDhgCwdetWFi9ezJYtW0hLS2PSpEkUFPi+hjtx4kTmzp1LZmYmmZmZpKWl1Wb5EkSVCbY+QCfgt/guHPkjMDOYRYlIcGRnZ9OhQwfGjRtHp06dSEpK4ujRo2Uuu2vXLgYOHEiXLl3o1q0bWVlZmBnTpk2jc+fOxMbGsmTJEgBWr15NYmIiKSkpxMTEMH36dFJTU0lISCA2NpasrCwAxowZw4QJE+jbty8xMTGsWLGiWt/f8uXLGTlyJI0bN6ZNmzZER0ezfv169u3bxzfffEPv3r1xznHLLbfw6quvVuu+pe446cUjZqahSBEPyczM5IUXXmDevHmkpKTw8ssvc9NNN52w3OjRo5k+fTrJyckcO3aMwsJCli1bRnp6OhkZGXz11Vf07NmTfv1815BlZGSwbds2WrRoQdu2bRk7dizr169n1qxZPPnkkzz++OOAL1zfffddsrKyuOqqq9i1axfh4eEl9t23b18OHTp0Qk0zZ85k4MCBAPzlL39h4cKF9OjRgz/+8Y+cc8455Obm0qtXr8DykZGR5Obm0qhRIyIjI09oF2+qzFWROOeuxddrC3z6yrugRETqtjZt2hAfHw9A9+7dyc7OPmGZQ4cOkZubS3JyMkAgeN577z1GjRpFaGgoF1xwAYmJiWzYsIFmzZrRs2dPIiIiAIiKiiIpKQmA2NhYVq1aFdh2SkoKISEhtGvXjrZt27J9+/ZAPUXWrl1b4XuYOHEi999/P8457r//fn7xi1/wzDPPlHnezDlXbrt4U2XuFTkHaILvIpK/AsOB9UGuS0SCpHHjxoHHoaGhZQ5FlndhRUUXXBTfbkhISOB5SEgIx48fD7xWOlDKCpiT9dguuOCCQNu4ceMYOnQo4OuJff7554HXcnJyuOiii4iMjCQnJ+eEdvGmSp1jM7NbgP+a2W+A3sAlwS1LRGpTs2bNiIyMDJyHys/P58iRI/Tr148lS5ZQUFBAXl4ea9asISEhoUrbXrp0KYWFhWRlZbF7927at29/wjJr164NXBhS/KdoGHLfvn2BZV955RU6d+4MwLBhw1i8eDH5+fns2bOHzMxMEhISiIiIoGnTpnz44YeYGQsXLuT6668/xaMjdV1lhiKP+X8fcc5dBOwH2gSvJBGpCxYtWsQdd9zBAw88QKNGjVi6dCnJycl88MEHdOnSBeccjz32GBdeeCHbt2+v9Hbbt29PYmIiX375JXPmzCE8PJy9e/cyduzYEpftV+SXv/wl6enpOOdo3bo1Tz/9NACdOnUiJSWFjh07EhYWxuzZswkNDQXgqaeeYsyYMRw9epTBgwczePDgqh8UqRfcyb7L4Zy7H3gSGADMBgyYZ2YPBL88nx49etjGjRtranciEiRjxoxh6NChDB8+vLZLEQ9wzm0ysx6l209255EQYKWZHQBeds6tAMLN7GBwyhQRETk9FQabmRU65/6I77waZpYP5NdEYSJSMyZPnsy6detKtE2ZMoXbbrut2ve1YMGCat+mSGmVOcf2D+fcjcAy0z1oRDxn9uzZtV2CSLWqTLDdA5wFHHfOHQMcYGamyUZFRKTOqcxEo03NLMTMzjCzZv7nCjURqVBqKrRuDSEhvt+pqbVdkTQUlfmCdplzrpnZmuovR0S8IDUVxo+HI0d8zz/91PccYPTo2qtLGobKXO7/92JPw4EEYJOZ9Q9mYcXpcn+R+qV1a1+YldaqFZRxBy+RU3JKl/sDmNl1pTZ0CfBYNdYmIh7z2WdVaxepTpW5pVZpOUDn6i5ERLzj0kur1i5SnSpzju1JfHcbAV8QxgMZQaxJROq5GTNKnmMDaNLE1y4SbJW53L/4ya3jwAtmtq68hUVEii4Que8+3/DjpZf6Qk0XjkhNqMzFI2cBx8yswP88FGhsZkcqXLEa6eIREREprbyLRypzjm0lcGax52cC71RXYSIiItWpMsEWbmaHi574HzcJXkkiIiKnrjLB9q1zrlvRE+dcd+DEKXdFRETqgMpcPHI3sNQ5t9f/PAIYEbSKRERETkNlvqC9wTl3GdAe3w2Qt5vZ90GvTERE5BScdCjSOTcZOMvMNpvZJ8CPnHOTgl+aiIhI1VXmHNs4/wzaAJjZf4FxQatIRETkNFQm2EKcc67oif97bGcEryQREZFTV5mLR94CXnTOzcF3a60JQFpQqxIRETlFlQm2e4HxwER8F4/8A5gXzKJEREROVWVm0C40szlmNtzMbgS2AE8GvzQREZGqq0yPDedcPDAK3/fX9gDLgliTiIjIKSs32JxzMcBIfIG2H1iC76bJV9VQbSIiIlVWUY9tO7AWuM7MdgE456bWSFUiIiKnqKJzbDcCXwCrnHPznHMD8F08IiIiUmeVG2xm9oqZjQAuA1YDU4ELnHNPOeeSaqg+ERGRKqnMVZHfmlmqmQ0FIoF0YHqwCxMRETkVlbnzSICZfW1mT5tZ/2AVJCIicjqqFGwiIiJ1nYJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERTwkL1oadc+2BJcWa2gIPAL2B9v625sABM4sPVh0iItKwBC3YzGwHEA/gnAsFcoFXzOzxomWcc38EDgarBhERaXiCFmylDACyzOzTogbnnANSgP41VIOIiDQANXWObSTwQqm2vsCXZpZZ1grOufHOuY3OuY15eXlBL1BERLwh6MHmnDsDGAYsLfXSKE4MuwAzm2tmPcysR8uWLYNZooiIeEhNDEUOBv5tZl8WNTjnwoCfAN1rYP8iItKA1MRQZFk9s4HAdjPLqYH9i4hIAxLUYHPONQEGActKvVTWOTcREZHTFtShSDM7ApxbRvuYYO5XREQaLt15REREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuI1DvTpk3jsssuIy4ujuTkZA4cOABAdnY2Z555JvHx8cTHxzNhwoTAOps2bSI2Npbo6GjuuusuzKyWqpdgU7CJSL0zaNAgNm/ezMcff0xMTAwPP/xw4LWoqCjS09NJT09nzpw5gfaJEycyd+5cMjMzyczMJC0trTZKlxqgYBNpQLKzs+nQoQPjxo2jU6dOJCUlcfTo0TKX3bVrFwMHDqRLly5069aNrKwszIxp06bRuXNnYmNjWbJkCQCrV68mMTGRlJQUYmJimD59OqmpqSQkJBAbG0tWVhYAY8aMYcKECfTt25eYmBhWrFhxSu8jKSmJsLAwAHr16kVOTk6Fy+/bt49vvvmG3r1745zjlltu4dVXXz2lfUvdp2ATaWAyMzOZPHkyW7ZsoXnz5rz88stlLjd69GgmT55MRkYG77//PhERESxbtoz09HQyMjJ45513mDZtGvv27QMgIyODWbNm8cknn7Bo0SJ27tzJ+vXrGTt2LE8++WRgu9nZ2bz77ru8/vrrTJgwgWPHjp2w7759+waGE4v/vPPOOycs+8wzzzB48ODA8z179tC1a1cSExNZu3YtALm5uURGRgaWiYyMJDc399QOoNR5YbVdgIjUrDZt2hAfHw9A9+7dyc7OPmGZQ4cOkZubS3JyMgDh4eEAvPfee4waNYrQ0FAuuOACEhMT2bBhA82aNaNnz55EREQAvuHApKQkAGJjY1m1alVg2ykpKYSEhNCuXTvatm3L9u3bA/UUKQqkk5kxYwZhYWGMHj0agIiICD777DPOPfdcNm3axA033MCWLVvKPJ/mnKvUPqT+UbCJNDCNGzcOPA4NDS1zKLK8CysquuCi+HZDQkICz0NCQjh+/HjgtdKBUlbA9O3bl0OHDp3QPnPmTAYOHAjAc889x4oVK1i5cmVgG40bNw7st3v37kRFRbFz504iIyNLDFfm5ORw0UUXlftepH7TUKSInKBZs2ZERkYGzkPl5+dz5MgR+vXrx5IlSygoKCAvL481a9aQkJBQpW0vXbqUwsJCsrKy2L17N+3btz9hmbVr1wYuACn+UxRqaWlpPProo7z22ms0adIksF5eXh4FBQUA7N69m8zMTNq2bUtERARNmzblww8/xMxYuHAh119//SkeHanrFGwiUqZFixbxxBNPEBcXR58+ffjiiy9ITk4mLi6OLl260L9/fx577DEuvPDCKm23ffv2JCYmMnjwYObMmUN4eDh79+5lyJAhld7GnXfeyaFDhxg0aFCJy/rXrFkTqG/48OHMmTOHFi1aAPDUU08xduxYoqOjiYqKKnFeTrzF1YfvcvTo0cM2btxY22WIyGkaM2YMQ4cOZfjw4bVdiniAc26TmfUo3a4em4iIeIouHhFp4CZPnsy6detKtE2ZMoXbbrut2ve1YMGCat+mSGkKNpEGbvbs2bVdgki10lCkiIh4ioJNRIJi0iQICwPnfL8nTartiqSh0FCkiFS7SZPgqad+eF5Q8MPz//u/2qlJGg712ESk2s2dW7V2keqkYBORaue/+Uel20Wqk4JNRKpdaGjV2kWqk4JNRKrd+PFVaxepTrp4RESqXdEFInPn+oYfQ0N9oaYLR6QmKNhEJCj+7/8UZFI7NBQpIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8ZSgBZtzrr1zLr3YzzfOubv9r/2Pc26Hc26Lc+6xYNUgIiINT1iwNmxmO4B4AOdcKJALvOKcuwq4Hogzs3zn3PnBqkFERBqemhqKHABkmdmnwETgETPLBzCz/9RQDSIi0gDUVLCNBF7wP44B+jrn/uWce9c517OsFZxz451zG51zG/Py8mqoTBERqe+CHmzOuTOAYcBSf1MYcA7QC5gGvOicc6XXM7O5ZtbDzHq0bNky2GWKiIhH1ESPbTDwbzP70v88B1hmPuuBQuC8GqhDREQagJoItlH8MAwJ8CrQH8A5FwOcAXxVA3WIiEgDENRgc841AQYBy4o1PwO0dc5tBhYDt5qZBbMOERFpOIJ2uT+AmR0Bzi3V9h1wUzD3KyIiDZfuPCIiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHhKgwm2fYf2EfVEFF8c/qK2SxERkSBqMMH20JqHyD6QzUPvPlTbpYiISBA1iGDbd2gfz6Y/S6EV8mz6s+q1iYh4WIMItofWPEShFQJQYAXqtYmIeJjng62ot/ZdwXcAfFfwnXptIiIe5vlgK95bK6Jem4iId3k+2F7b8Vqgt1bku4LvWL5jeS1VJCIiwRRW2wUEW849ObVdgoiI1CDP99hERKRhUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4inOzGq7hpNyzuUBn1ZxtfOAr4JQTkOiY3j6dAxPn47h6fPqMWxlZi1LN9aLYDsVzrmNZtajtuuoz3QMT5+O4enTMTx9De0YaihSREQ8RcEmIiKe4uVgm1vbBXiAjuHp0zE8fTqGp69BHUPPnmMTEZGGycs9NhERaYAUbCIi4in1Pticc9c453Y453Y556aX8bpzzj3hf/1j51y32qizLqvEMbzSOXfQOZfu/3mgNuqsq5xzzzjn/uOc21zO6/oMnkQljqE+gxVwzl3inFvlnNvmnNvinJtSxjIN53NoZvX2BwgFsoC2wBlABtCx1DJDgDcBB/QC/lXbddeln0oewyuBFbVda139AfoB3YDN5byuz+DpH0N9Bis+fhFAN//jpsDOhvxvYX3vsSUAu8xst5l9BywGri+1zPXAQvP5EGjunIuo6ULrsMocQ6mAma0Bvq5gEX0GT6ISx1AqYGb7zOzf/seHgG3AxaUWazCfw/oebBcDnxd7nsOJf5iVWaYhq+zx6e2cy3DOvemc61QzpXmGPoPVQ5/BSnDOtQa6Av8q9VKD+RyG1XYBp8mV0Vb6+wuVWaYhq8zx+Te+e7Idds4NAV4F2gW7MA/RZ/D06TNYCc65HwEvA3eb2TelXy5jFU9+Dut7jy0HuKTY80hg7yks05Cd9PiY2Tdmdtj/+A2gkXPuvJorsd7TZ/A06TN4cs65RvhCLdXMlpWxSIP5HNb3YNsAtHPOtXHOnQGMBF4rtcxrwC3+K4J6AQfNbF9NF1qHnfQYOucudM45/+MEfJ+b/TVeaf2lz+Bp0mewYv5jMx/YZmZ/KmexBvM5rNdDkWZ23Dl3J/AWvqv7njGzLc65Cf7X5wBv4LsaaBdwBLittuqtiyp5DIcDE51zx4GjwEjzX2Yl4Jx7Ad9Ve+c553KAXwONQJ/ByqrEMdRnsGJXADcDnzjn0v1tvwIuhYb3OdQttURExFPq+1CkiIhICQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomAT8XPOnVvs7vFfOOdyiz0/4yTr9nDOPVGJfbxfTbXG++/AUfR8WFkzM1TTvh53zvWr4PU7nXOevXRc6h9d7i9SBufcg8BhM5tZrC3MzI7XXlU/cM6NAXqY2Z1B3k8L4A0z61XBMk2AdWbWNZi1iFSWemwiFXDOLXDO/ck5twp41DmX4Jx73zn3kf93e/9yVzrnVvgfP+ifX2y1c263c+6uYts7XGz51c65l5xz251zqcXurDHE3/aef/6sFaVqOgP4LTDC35sc4Zwb45z7S7Gan3K++bl2O+cS/fVsc84tKLadJOfcB865fzvnlvrvM1jacCCt2DqPOOe2Ot98XjMBzOwIkO2/I4hIravXdx4RqSExwEAzK3DONQP6+e/YMhD4PXBjGetcBlyFb26sHc65p8zs+1LLdAU64btf3zrgCufcRuBp/z72+O/IUYKZfed8E20Gemz+Hlxx5wD9gWHA3/HdmWIssME5F4/vvoH/639f3zrn7gXuwReYxV0BvOTfRwsgGbjMzMw517zYchuBvsD6Mo6FSI1SsImc3FIzK/A/Pht4zjnXDt+d0RuVs87rZpYP5Dvn/gNcgC9MiltvZjkA/tsgtQYOA7vNbI9/mReA8adQ89/94fMJ8KWZfeLfzxb/fiKBjsA6f0fxDOCDMrYTAeT5H38DHAP+6px7HSjek/wPvjAXqXUKNpGT+7bY44eAVWaW7HzzXq0uZ538Yo8LKPvvWlnLlDW1yKko2nZhqf0U+vdTALxtZqNOsp2jQDgE7iuaAAzAd7PsO/H1CvEvc7R6Shc5PTrHJlI1ZwO5/sdjgrD97UBbf2gCjChnuUP4hjlP1Yf4hj6jwXcBiHMupozltgFFy/wIONs/bczdQHyx5WKAzadRj0i1UbCJVM1jwMPOuXX4ZkOoVmZ2FJgEpDnn3gO+BA6WsegqoGPRxSOnsJ88fMH8gnPuY3xBV9ZQ4uv47roPviBd4V/+XWBqseWuAN6pah0iwaDL/UXqGOfcj/wzRTtgNpBpZn+uxXreA4aa2YFyXu8K3GNmN9doYSLlUI9NpO4Z57+YZAu+oc+na7ccfoF/Xq9ynAfcX0O1iJyUemwiIuIp6rGJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHjK/wMQPvcNprjniQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ray\n",
    "import codeflare.pipelines.Datamodel as dm\n",
    "import codeflare.pipelines.Runtime as rt\n",
    "from codeflare.pipelines.Datamodel import Xy\n",
    "from codeflare.pipelines.Datamodel import XYRef\n",
    "from codeflare.pipelines.Runtime import ExecutionType\n",
    "\n",
    "ray.shutdown()\n",
    "ray.init()\n",
    "\n",
    "n_runs = 3\n",
    "score_250 = 0\n",
    "score_500 = 0\n",
    "score_1000 = 0\n",
    "score_2000 = 0\n",
    "run_time = 0\n",
    "\n",
    "for _ in range(n_runs):\n",
    "    \n",
    "    start = time.time()\n",
    "    \n",
    "    pipeline = dm.Pipeline()\n",
    "    node_pcs250 = dm.EstimatorNode(\"kernel_approximator_250\", PolynomialCountSketch(n_components=250,\n",
    "                                        degree=4))\n",
    "    node_pcs500 = dm.EstimatorNode(\"kernel_approximator_500\", PolynomialCountSketch(n_components=500,\n",
    "                                        degree=4))\n",
    "    node_pcs1000 = dm.EstimatorNode(\"kernel_approximator_1000\", PolynomialCountSketch(n_components=1000,\n",
    "                                        degree=4))\n",
    "    node_pcs2000 = dm.EstimatorNode(\"kernel_approximator_2000\", PolynomialCountSketch(n_components=2000,\n",
    "                                        degree=4))\n",
    "    node_clf250 = dm.EstimatorNode(\"linear_classifier_250\", LinearSVC())\n",
    "    node_clf500 = dm.EstimatorNode(\"linear_classifier_500\", LinearSVC())\n",
    "    node_clf1000 = dm.EstimatorNode(\"linear_classifier_1000\", LinearSVC())\n",
    "    node_clf2000 = dm.EstimatorNode(\"linear_classifier_2000\", LinearSVC())\n",
    "    \n",
    "    pipeline.add_edge(node_pcs250, node_clf250)\n",
    "    pipeline.add_edge(node_pcs500, node_clf500)\n",
    "    pipeline.add_edge(node_pcs1000, node_clf1000)\n",
    "    pipeline.add_edge(node_pcs2000, node_clf2000)\n",
    "    \n",
    "    # create training input\n",
    "    train_input = dm.PipelineInput()\n",
    "    xy_train = dm.Xy(X_train, y_train)\n",
    "    train_input.add_xy_arg(node_pcs250, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs500, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs1000, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs2000, xy_train)\n",
    "    \n",
    "    # codeflare pipeline is able to execute model FIT once with four n_components \n",
    "    pipeline_fitted = rt.execute_pipeline(pipeline, ExecutionType.FIT, train_input)\n",
    "    \n",
    "    xy_test = dm.Xy(X_test, y_test)\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs250, xy_test)\n",
    "    pipeline_250 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf250)[0])\n",
    "    score_250 += 100 * ray.get(rt.execute_pipeline(pipeline_250, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf250)[0].get_yref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs500, xy_test)\n",
    "    pipeline_500 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf500)[0])\n",
    "    score_500 += 100 * ray.get(rt.execute_pipeline(pipeline_500, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf500)[0].get_yref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs1000, xy_test)\n",
    "    pipeline_1000 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf1000)[0])\n",
    "    score_1000 += 100 * ray.get(rt.execute_pipeline(pipeline_1000, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf1000)[0].get_yref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs2000, xy_test)\n",
    "    pipeline_2000 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf2000)[0])\n",
    "    score_2000 += 100 * ray.get(rt.execute_pipeline(pipeline_2000, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf2000)[0].get_yref())\n",
    "\n",
    "    run_time += ((time.time() - start)/4)\n",
    "\n",
    "# all runs have the the same run time since they are FITTED together\n",
    "results[f\"LSVM + PS(250)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_250/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(500)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_500/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(1000)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_1000/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(2000)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_2000/n_runs\n",
    "}\n",
    "\n",
    "\n",
    "N_COMPONENTS = [250, 500, 1000, 2000]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 7))\n",
    "ax.scatter([results[\"LSVM\"][\"time\"], ], [results[\"LSVM\"][\"score\"], ],\n",
    "           label=\"Linear SVM\", c=\"green\", marker=\"^\")\n",
    "\n",
    "ax.scatter([results[\"LSVM + PS(250)\"][\"time\"], ],\n",
    "           [results[\"LSVM + PS(250)\"][\"score\"], ],\n",
    "           label=\"Linear SVM + PolynomialCountSketch\", c=\"blue\")\n",
    "for n_components in N_COMPONENTS:\n",
    "    ax.scatter([results[f\"LSVM + PS({n_components})\"][\"time\"], ],\n",
    "               [results[f\"LSVM + PS({n_components})\"][\"score\"], ],\n",
    "               c=\"blue\")\n",
    "    ax.annotate(f\"n_comp.={n_components}\",\n",
    "                (results[f\"LSVM + PS({n_components})\"][\"time\"],\n",
    "                 results[f\"LSVM + PS({n_components})\"][\"score\"]),\n",
    "                xytext=(-30, 10), textcoords=\"offset pixels\")\n",
    "\n",
    "ax.scatter([results[\"KSVM\"][\"time\"], ], [results[\"KSVM\"][\"score\"], ],\n",
    "           label=\"Kernel SVM\", c=\"red\", marker=\"x\")\n",
    "\n",
    "ax.set_xlabel(\"Training time (s)\")\n",
    "ax.set_ylabel(\"Accurary (%)\")\n",
    "ax.legend()\n",
    "plt.show()\n",
    "\n",
    "ray.shutdown()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}