8_sgd_vs_gd/gd_and_sgd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style='color:blue' align='center'>Implementation of stochastic and batch grandient descent in python</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We will use very simple home prices data set to implement batch and stochastic gradient descent in python. Batch gradient descent uses *all* training samples in forward pass to calculate cumulitive error and than we adjust weights using derivaties. In stochastic GD, we randomly pick *one* training sample, perform forward pass, compute the error and immidiately adjust weights. So the key difference here is that to adjust weights batch GD will use *all* training samples where as stochastic GD will use one randomly picked training sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 430,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Load the dataset in pandas dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 431,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>area</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1521</td>\n",
       "      <td>3</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2785</td>\n",
       "      <td>4</td>\n",
       "      <td>140.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3300</td>\n",
       "      <td>4</td>\n",
       "      <td>155.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1100</td>\n",
       "      <td>2</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>800</td>\n",
       "      <td>1</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    area  bedrooms  price\n",
       "3   1521         3   75.0\n",
       "11  2785         4  140.0\n",
       "7   3300         4  155.0\n",
       "13  1100         2   40.0\n",
       "19   800         1   32.0"
      ]
     },
     "execution_count": 431,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"homeprices_banglore.csv\")\n",
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Preprocessing/Scaling: Since our columns are on different sacle it is important to perform scaling on them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 432,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.08827586, 0.25      ],\n",
       "       [0.62068966, 0.75      ],\n",
       "       [0.22068966, 0.5       ],\n",
       "       [0.24862069, 0.5       ],\n",
       "       [0.13793103, 0.25      ],\n",
       "       [0.12758621, 0.25      ],\n",
       "       [0.6662069 , 0.75      ],\n",
       "       [0.86206897, 0.75      ],\n",
       "       [0.17586207, 0.5       ],\n",
       "       [1.        , 1.        ],\n",
       "       [0.34482759, 0.5       ],\n",
       "       [0.68448276, 0.75      ],\n",
       "       [0.06896552, 0.25      ],\n",
       "       [0.10344828, 0.25      ],\n",
       "       [0.5       , 0.5       ],\n",
       "       [0.12931034, 0.25      ],\n",
       "       [0.13103448, 0.5       ],\n",
       "       [0.25517241, 0.5       ],\n",
       "       [0.67931034, 0.5       ],\n",
       "       [0.        , 0.        ]])"
      ]
     },
     "execution_count": 432,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "sx = preprocessing.MinMaxScaler()\n",
    "sy = preprocessing.MinMaxScaler()\n",
    "\n",
    "scaled_X = sx.fit_transform(df.drop('price',axis='columns'))\n",
    "scaled_y = sy.fit_transform(df['price'].values.reshape(df.shape[0],1))\n",
    "\n",
    "scaled_X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 433,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.05237037],\n",
       "       [0.65185185],\n",
       "       [0.22222222],\n",
       "       [0.31851852],\n",
       "       [0.14074074],\n",
       "       [0.04444444],\n",
       "       [0.76296296],\n",
       "       [0.91111111],\n",
       "       [0.13333333],\n",
       "       [1.        ],\n",
       "       [0.37037037],\n",
       "       [0.8       ],\n",
       "       [0.04444444],\n",
       "       [0.05925926],\n",
       "       [0.51111111],\n",
       "       [0.07407407],\n",
       "       [0.11851852],\n",
       "       [0.20740741],\n",
       "       [0.51851852],\n",
       "       [0.        ]])"
      ]
     },
     "execution_count": 433,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaled_y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### We should convert target column (i.e. price) into one dimensional array. It has become 2D due to scaling that we did above but now we should change to 1D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 434,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.05237037, 0.65185185, 0.22222222, 0.31851852, 0.14074074,\n",
       "       0.04444444, 0.76296296, 0.91111111, 0.13333333, 1.        ,\n",
       "       0.37037037, 0.8       , 0.04444444, 0.05925926, 0.51111111,\n",
       "       0.07407407, 0.11851852, 0.20740741, 0.51851852, 0.        ])"
      ]
     },
     "execution_count": 434,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaled_y.reshape(20,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Gradient descent allows you to find weights (w1,w2,w3) and bias in following linear equation for housing price prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"hp.jpg\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Now is the time to implement mini batch gradient descent. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 443,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0.70712464, 0.67456527]), -0.23034857438407427, 0.0068641890429808105)"
      ]
     },
     "execution_count": 443,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def batch_gradient_descent(X, y_true, epochs, learning_rate = 0.01):\n",
    "\n",
    "    number_of_features = X.shape[1]\n",
    "    # numpy array with 1 row and columns equal to number of features. In \n",
    "    # our case number_of_features = 2 (area, bedroom)\n",
    "    w = np.ones(shape=(number_of_features)) \n",
    "    b = 0\n",
    "    total_samples = X.shape[0] # number of rows in X\n",
    "    \n",
    "    cost_list = []\n",
    "    epoch_list = []\n",
    "    \n",
    "    for i in range(epochs):        \n",
    "        y_predicted = np.dot(w, X.T) + b\n",
    "\n",
    "        w_grad = -(2/total_samples)*(X.T.dot(y_true-y_predicted))\n",
    "        b_grad = -(2/total_samples)*np.sum(y_true-y_predicted)\n",
    "        \n",
    "        w = w - learning_rate * w_grad\n",
    "        b = b - learning_rate * b_grad\n",
    "        \n",
    "        cost = np.mean(np.square(y_true-y_predicted)) # MSE (Mean Squared Error)\n",
    "        \n",
    "        if i%10==0:\n",
    "            cost_list.append(cost)\n",
    "            epoch_list.append(i)\n",
    "        \n",
    "    return w, b, cost, cost_list, epoch_list\n",
    "\n",
    "w, b, cost, cost_list, epoch_list = batch_gradient_descent(scaled_X,scaled_y.reshape(scaled_y.shape[0],),500)\n",
    "w, b, cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Check price equation above. In that equation we were trying to find values of w1,w2 and bias. Here we got these values for each of them,\n",
    "w1 = 0.66469087\n",
    "w2 = 0.60541671\n",
    "bias = -0.17792104056392882"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Now plot epoch vs cost graph to see how cost reduces as number of epoch increases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 436,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1cefc80ab50>]"
      ]
     },
     "execution_count": 436,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEGCAYAAACQO2mwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcjUlEQVR4nO3deZBdZ33m8e9zN/XV0t1eZFtqeZEzNmDABqKyzTgJWyA2IRZMSMbsMLg8nsLFMKnUYCqZJEMqqVlgKkONg3ARM2yOCYszChHYhGRMMcagFvEOwrK8tSVbLdtaLKmXe+9v/jjndp++3Ve6LfXRlfo+n6pb95z3nPf0+7YtPTrvexZFBGZmZnMpdLsBZmZ24nJImJlZWw4JMzNryyFhZmZtOSTMzKytUrcbsJBOP/30OO+887rdDDOzk8aWLVt2R8TKdtsXVUicd955DA8Pd7sZZmYnDUlPHG67h5vMzKwth4SZmbXlkDAzs7YcEmZm1pZDwszM2nJImJlZWw4JMzNryyEBfOb7j3DXL0a73QwzsxOOQwL43F2P8gOHhJnZLA4JoFopMjZZ73YzzMxOOA4JYEmpyCGHhJnZLA4JfCZhZtaOQwKoloscmnBImJm1ckiQhMTYZKPbzTAzO+E4JIAl5YLnJMzM5uCQoHkm4ZAwM2vlkCCZuPaZhJnZbA4JfCZhZtaOQwLo89VNZmZzckiQhISvbjIzm80hQTLcNFFvUG9Et5tiZnZCcUgAfeXk1+B5CTOzmRwSJFc3Ab7CycyshUOCZE4C8OS1mVkLhwTJnAR4uMnMrFWuISHpSklbJW2TdOMc298j6f70c7ekSzLbHpf0gKR7JQ3n2c6+qZDwFU5mZlmlvA4sqQjcBLwZGAE2S9oYEQ9ndnsMeF1EvCDpKuBm4LLM9jdExO682tjUPJPwnISZ2Ux5nklcCmyLiO0RMQHcBqzP7hARd0fEC+nqPcCaHNvTVrWS/BocEmZmM+UZEkPAU5n1kbSsnQ8D38msB3CnpC2SrmtXSdJ1koYlDY+OHt17qj1xbWY2t9yGmwDNUTbn3WqS3kASEr+SKb4iInZIOgP4nqSfR8QPZh0w4maSYSrWrVt3VHfDNUNivOaQMDPLyvNMYgQ4O7O+BtjRupOki4HPA+sj4rlmeUTsSL93AbeTDF/louozCTOzOeUZEpuBCyStlVQBrgE2ZneQdA7wLeB9EfGLTPkySSuay8BbgAfzaqgnrs3M5pbbcFNE1CTdANwBFIFbIuIhSden2zcAfwScBvylJIBaRKwDzgRuT8tKwK0R8d282tq849qXwJqZzZTnnAQRsQnY1FK2IbN8LXDtHPW2A5e0ludlSclXN5mZzcV3XAOS6CsXfMe1mVkLh0Sq6hcPmZnN4pBI+RWmZmazOSRSfeWi5yTMzFo4JFJ9PpMwM5vFIZGqVnwmYWbWyiGRSuYkfJ+EmVmWQyLVVy746iYzsxYOiZTnJMzMZnNIpKq+usnMbBaHRMpnEmZmszkkUr66ycxsNodEqi+9uqnROKr3FpmZLUoOiVR16u10vgzWzKzJIZHqKye/Cs9LmJlNc0ik/HY6M7PZHBKp5tvpHBJmZtMcEqm+5pmE77o2M5vikEj1TU1cOyTMzJocEqmpOYkJX91kZtbkkEh54trMbDaHRKpaSX4VDgkzs2kOidSSUnIm4fskzMymOSRSzUtgHRJmZtMcEqmqL4E1M5vFIZHq88S1mdksDolUsSAqxYLfc21mlpFrSEi6UtJWSdsk3TjH9vdIuj/93C3pkk7r5qGvXPCchJlZRm4hIakI3ARcBVwEvEvSRS27PQa8LiIuBv4UuHkedRdctVL0nISZWUaeZxKXAtsiYntETAC3AeuzO0TE3RHxQrp6D7Cm07p5qJaLjPmxHGZmU/IMiSHgqcz6SFrWzoeB78y3rqTrJA1LGh4dHT2G5iaT1z6TMDOblmdIaI6yOd8NKukNJCHx8fnWjYibI2JdRKxbuXLlUTW0qa/s91ybmWWVcjz2CHB2Zn0NsKN1J0kXA58HroqI5+ZTd6FVy0VPXJuZZeR5JrEZuEDSWkkV4BpgY3YHSecA3wLeFxG/mE/dPCRXN/kSWDOzptzOJCKiJukG4A6gCNwSEQ9Juj7dvgH4I+A04C8lAdTSoaM56+bV1qZqxcNNZmZZeQ43ERGbgE0tZRsyy9cC13ZaN2+euDYzm8l3XGd4TsLMbCaHREafQ8LMbAaHREY1vQQ2Ys6rbc3Meo5DIqNaKdIImKj7CiczM3BIzNB8XPjYhEPCzAwcEjP0lZNfh5/fZGaWcEhk+O10ZmYzOSQyqn47nZnZDA6JjL6KQ8LMLMshkdFXSieuHRJmZoBDYoZqxSFhZpblkMiYnrj2JbBmZuCQmMET12ZmMzkkMqbuk3BImJkBDokZ+jwnYWY2g0MiwzfTmZnN5JDIKBcLFAvynISZWcoh0SJ58ZCvbjIzA4fELH3lIocma91uhpnZCcEh0WL5kiIvjnu4ycwMHBKzDFTL7D002e1mmJmdEBwSLfodEmZmUxwSLQaqZfY5JMzMAIfELB5uMjOb5pBo0QyJiOh2U8zMus4h0WKgWqbeCA74rmszM4dEq4FqGcBDTmZm5BwSkq6UtFXSNkk3zrH9pZJ+JGlc0u+3bHtc0gOS7pU0nGc7s6ZC4qBDwsyslNeBJRWBm4A3AyPAZkkbI+LhzG7PAx8F3t7mMG+IiN15tXEuPpMwM5vW0ZmEpN/ppKzFpcC2iNgeERPAbcD67A4RsSsiNgMnzN/I/Q4JM7MpnQ43faLDsqwh4KnM+kha1qkA7pS0RdJ17XaSdJ2kYUnDo6Oj8zj83JpnEr5XwszsCMNNkq4C3goMSfpMZlM/cKSn4GmOsvlcV3pFROyQdAbwPUk/j4gfzDpgxM3AzQDr1q075utWB5b6TMLMrOlIZxI7gGFgDNiS+WwEfuMIdUeAszPra9LjdSQidqTfu4DbSYavcre8UqIgh4SZGRzhTCIi7gPuk3RrREwCSDoFODsiXjjCsTcDF0haCzwNXAO8u5NGSVoGFCJif7r8FuCTndQ9VoWC/PwmM7NUp1c3fU/S1en+9wKjku6KiN9rVyEiapJuAO4AisAtEfGQpOvT7RsknUVyptIPNCR9DLgIOB24XVKzjbdGxHePpoNHw4/mMDNLdBoSAxGxT9K1wBci4o8l3X+kShGxCdjUUrYhs/wMyTBUq33AJR22bcE5JMzMEp1e3VSStAr4XeDbObbnhOCQMDNLdBoSnyQZNno0IjZLOh94JL9mdVe/HxduZgZ0ONwUEV8Hvp5Z3w78dl6N6jafSZiZJTq943qNpNsl7ZL0rKRvSpprLmFR8OPCzcwSnQ43fYHk3ojVJHdN/11atigNVMvUGsFBPy7czHpcpyGxMiK+EBG19PO/gZU5tqur/JA/M7NEpyGxW9J7JRXTz3uB5/JsWDc5JMzMEp2GxL8hufz1GWAn8E7gQ3k1qtscEmZmiU5vpvtT4APNR3FIOhX4FEl4LDoOCTOzRKdnEhdnn9UUEc8Dr86nSd3nkDAzS3QaEoX0wX7A1JlEbm+167bm48J9Q52Z9bpO/6L/NHC3pG+QvBPid4E/y61VXebHhZuZJTq94/pLkoaBN5K8TOhftbyrelHx48LNzBIdDxmlobBog6GVH81hZtb5nETPcUiYmTkk2nJImJk5JNrynISZmUOirQG/U8LMzCHRjh8XbmbmkGhroFpmsh4cmvTjws2sdzkk2vCjOczMHBJtOSTMzBwSbU2FxEGHhJn1LodEGz6TMDNzSLTVDIk9Dgkz62EOiTb6q35cuJmZQ6KNFUtKyI8LN7Me55Boo1AQ/X1+NIeZ9bZcQ0LSlZK2Stom6cY5tr9U0o8kjUv6/fnUPR78kD8z63W5hYSkInATcBVwEfAuSRe17PY88FHgU0dRN3enLKvw/IGJ4/1jzcxOGHmeSVwKbIuI7RExAdwGrM/uEBG7ImIz0PrP9SPWPR6GBvvYsefQ8f6xZmYnjDxDYgh4KrM+kpYtaF1J10kaljQ8Ojp6VA1tZ9VAlR17xvyQPzPrWXmGhOYo6/Rv247rRsTNEbEuItatXLmy48Z1YvVglUOTdc9LmFnPyjMkRoCzM+trgB3Hoe6CGRrsA+BpDzmZWY/KMyQ2AxdIWiupAlwDbDwOdRfMqoEqADv2jB3vH21mdkIo5XXgiKhJugG4AygCt0TEQ5KuT7dvkHQWMAz0Aw1JHwMuioh9c9XNq63trB5shoTPJMysN+UWEgARsQnY1FK2IbP8DMlQUkd1j7fTllWolArs2OuQMLPe5DuuD6NQEKsG+jzcZGY9yyFxBKsHqh5uMrOe5ZA4gtWDVXY6JMysRzkkjmD1YB/P7BujVm90uylmZsedQ+IIVg9WaQQ8u3+8200xMzvuHBJH0LwM1kNOZtaLHBJHsHrAd12bWe9ySBzBqkHfdW1mvcshcQTLl5QYqJZ9GayZ9SSHRAdWDfSx03ddm1kPckh0YGiwytMebjKzHuSQ6MDqQd91bWa9ySHRgVWDfew9NMmB8Vq3m2Jmdlw5JDow1LxXwvMSZtZjHBIdaN5Q53kJM+s1DokOrEpvqPNd12bWaxwSHTizv4+C/IY6M+s9DokOlIsFzuzv83CTmfUch0SHVg9WeXrPwW43w8zsuHJIdOjCM5fzs537iYhuN8XM7LhxSHToFUMD7D00yVPPe17CzHqHQ6JDFw8NAvDA03u72xAzs+PIIdGhC89aTrko7n96T7ebYmZ23DgkOrSkVOSlZ/XzoM8kzKyHOCTm4RVDAzwwsteT12bWMxwS83DxmgH2jdV48nlfCmtmvcEhMQ+vHBoA4P4RDzmZWW/INSQkXSlpq6Rtkm6cY7skfSbdfr+k12S2PS7pAUn3ShrOs52duvDMFVSKBc9LmFnPKOV1YElF4CbgzcAIsFnSxoh4OLPbVcAF6ecy4LPpd9MbImJ3Xm2cr0qpwMtWrfCZhJn1jDzPJC4FtkXE9oiYAG4D1rfssx74UiTuAQYlrcqxTcfsFUMDPLhjL42GJ6/NbPHLMySGgKcy6yNpWaf7BHCnpC2Srmv3QyRdJ2lY0vDo6OgCNPvwLl4zwP6xGk948trMekCeIaE5ylr/+X24fa6IiNeQDEl9RNKvzfVDIuLmiFgXEetWrlx59K3t0CumJq/35P6zzMy6Lc+QGAHOzqyvAXZ0uk9ENL93AbeTDF913YVnrqBS8uS1mfWGPENiM3CBpLWSKsA1wMaWfTYC70+vcroc2BsROyUtk7QCQNIy4C3Agzm2tWPlYoGXrernPk9em1kPyO3qpoioSboBuAMoArdExEOSrk+3bwA2AW8FtgEHgQ+l1c8EbpfUbOOtEfHdvNo6X5evPZVb/t9j7Dk4weDSSrebY2aWm9xCAiAiNpEEQbZsQ2Y5gI/MUW87cEmebTsWv3XJaj73g+1858FneNel53S7OWZmufEd10fh5av7Of/0ZWy8t3WKxcxscXFIHAVJ/NYlq7nnsed4dp/fe21mi5dD4ihd/arVRMC379/Z7aaYmeXGIXGUfmnlcl6+up+N93nIycwWL4fEMbj6ktXc99QennjuQLebYmaWC4fEMXjbJasB+DufTZjZIuWQOAZDg1XWnXsKt//z09T9wD8zW4QcEsfog1ecx6OjB7ht85PdboqZ2YJzSByj33zlKi4//1T++x1beeHARLebY2a2oBwSx0gSf3L1y9k/VuNTd27tdnPMzBaUQ2IBvPSsft7/2nO59SdP+umwZraoOCQWyMd+/UJOW1bhD//2QcZr9W43x8xsQTgkFshAtcyfXP1y7n1qD9d/eYuDwswWBYfEAnrbxav583e8kn/aOsq/+8pPHRRmdtJzSCywd192Dn/2jlfwjz/fxb/98hae9xVPZnYSc0jk4D2Xncufv+OV/PCR3bzx0/+Xr21+koZvtjOzk5BDIifvvuwc/v6jv8qFZ6zg4998gN/ecDd/f/9OD0GZ2UlFycvhFod169bF8PBwt5sxQ0TwjS0j/MU/PMLTew5xytIyb3/1EG986RmsO/dUqpVit5toZj1M0paIWNd2u0Pi+Kg3gh9u283XNj/J9x5+lsl6UCkWeNU5g1w8NMBFq/t52ap+1p6+jL6yg8PMjo8jhUSu77i2acWCeN2FK3ndhSs5MF5j8+PPc/ejz/Hj7c/x5XueYLzWmNp31UAf5562lKHBpawe7GPVQJWzBpZwxoo+zlixhFOXVSgVPVJoZvlzSHTBsiUlXv+SM3j9S84AoFZv8NjuA/zsmf08vvsAjz93gCeeO8jdj+7m2X1jtM55S3DK0gqnLatw6rIKpy2vcMrSZHlwaYVTlpY5ZWmFgaVlBqtlBpdW6O8rOVjMbN4cEieAUrHABWeu4IIzV8zaVqs3eHb/OLv2jbFr/zi79o+ze/84u19MPi8cmGTrM/t5/sAEew9NzgqUrOVLSgxUy/RXy/T3ldLvMv3VEv19ZVb0ldJPeep7+ZIS/X0llveVqJaLSMrxN2FmJxqHxAmuVCwwNFhlaLB6xH0bjWDf2CQvHJzkhYNJaOw9OMmegxPsPVRjz6GkbN+hGvsOTfLU8wfZP5Ys7x+vHfH4BSVnQcvTz7Kp7yLLKsn60szyskqRaqXI0srM5aVTy0X6SkUKBQeP2YnKIbGIFApicGky5LSWZfOqW28EL47X2D82yf6x2qzlF8dqU8sHxtOydHl0/zgHJmocnKjz4niNicz8Sif6ygWq5SLVcpG+NDyq5SJ92U+pQLUyvbykXGRJqTC1vbmc/V5SLrCklCxXSoWp70qx4DMisw45JAxIJtYHqmUGquVjPlat3uDgZJ0D40lwHByvc2CixqGJOgcnkuWxyWT54ESdQxM1xiYbHJqsc2iizthknUOTdfaPJQE0Xmsk5bVk29jk/EJoLpVSgSXFNDQy4dFcLheTUCkXk/JyqUC5qKmy5qdSFOVigVIx2d4sLxVFJf0up9tKhen1UkFT+5UKs7c114uFZHuxIAebdYVDwhZcqVigv1igv+/YA2cuEcF4rcH4ZIOxWp3xyQbjtSQ8xmt1xmsNxibrTNQa08v1xtT6eK25nOwzUWtMbc8uvzheYzJdnqzH1LZafeb68dIMi1JBlIoFioWZITL9Pb2tWV7IbC8WRFFJWVGiWEy/C6Kg6f2LBWbsV0i3H668oOayKIjpbVP7MRV4BU3v31xOtmXKCzP3kbLbk2+ps32yP1NMr099I1SY3t6sBzN/TlKXnglth4SddCRNDTMNkE8QdSoiqDeCiTQ4mgEyWW+kn2S51pjeVms0qNVjuiz9rtWDyUaDeiOmjlVrJMev1RvUI5J96kEjkuMk24J6xPRyI9J6DeoB9fTnTdYbHJpMtjc/jcisR9BoMFW/ua3R3NbcnpYbcwbH1DKZ0Mpub6kDaYjNOp6mfkbzeFPrTB9LwGnLlvA31782lz7mGhKSrgT+J1AEPh8R/6Vlu9LtbwUOAh+MiJ92UtfsRCApGTLqwcuL5wqPZDloBFMh1AybmFHG1LZGI7Oc7gNBPQ2sYLpukOwbkZRl929k9on0eM19GtnjpOWR9iGieVwyx02O16wT2TqRKYOp/ZvL0+XTP3/Gceaow9T2lmOR/NBmJMeMbdM/o7+a31/luR1ZUhG4CXgzMAJslrQxIh7O7HYVcEH6uQz4LHBZh3XNrIsKBVGgN4Zcelme//y5FNgWEdsjYgK4DVjfss964EuRuAcYlLSqw7pmZpazPENiCHgqsz6SlnWyTyd1AZB0naRhScOjo6PH3GgzM5uWZ0jMdR7aOtvVbp9O6iaFETdHxLqIWLdy5cp5NtHMzA4nz4nrEeDszPoaYEeH+1Q6qGtmZjnL80xiM3CBpLWSKsA1wMaWfTYC71ficmBvROzssK6ZmeUstzOJiKhJugG4g+Qy1lsi4iFJ16fbNwCbSC5/3UZyCeyHDlc3r7aamdnc/NIhM7MedqSXDvXeHUBmZtaxRXUmIWkUeOIoq58O7F7A5pwMerHP0Jv97sU+Q2/2e759Pjci2l4auqhC4lhIGj7cKddi1It9ht7sdy/2GXqz3wvdZw83mZlZWw4JMzNryyEx7eZuN6ALerHP0Jv97sU+Q2/2e0H77DkJMzNry2cSZmbWlkPCzMza6vmQkHSlpK2Stkm6sdvtWSiSzpb0T5J+JukhSf8+LT9V0vckPZJ+n5Kp84n097BV0m90r/XHTlJR0j9L+na6vqj7LWlQ0jck/Tz9b/7axd5nAEn/If3/+0FJfy2pbzH2W9ItknZJejBTNu9+SvplSQ+k2z6jTl7UHVOvAuy9D8lzoR4Fzid58ux9wEXdbtcC9W0V8Jp0eQXwC+Ai4L8BN6blNwL/NV2+KO3/EmBt+nspdrsfx9D/3wNuBb6dri/qfgNfBK5NlyvAYA/0eQh4DKim638DfHAx9hv4NeA1wIOZsnn3E/gJ8FqS1zF8B7jqSD+7188kFu0b8CJiZ6TvC4+I/cDPSP5QrSf5C4X0++3p8nrgtogYj4jHSB66eOlxbfQCkbQG+E3g85niRdtvSf0kf4n8FUBETETEHhZxnzNKQFVSCVhK8kqBRdfviPgB8HxL8bz6mb71sz8ifhRJYnwpU6etXg+Jjt+AdzKTdB7wauDHwJmRPI6d9PuMdLfF9Lv4C+A/Ao1M2WLu9/nAKPCFdIjt85KWsbj7TEQ8DXwKeBLYSfKqgTtZ5P3OmG8/h9Ll1vLD6vWQ6PgNeCcrScuBbwIfi4h9h9t1jrKT7nch6W3ArojY0mmVOcpOtn6XSIYiPhsRrwYOkAw/tLMY+kw6Br+eZEhlNbBM0nsPV2WOspOu3x045jd+ZvV6SHTy9ryTlqQySUB8NSK+lRY/m552kn7vSssXy+/iCuBqSY+TDB++UdJXWNz9HgFGIuLH6fo3SEJjMfcZ4NeBxyJiNCImgW8B/5LF3++m+fZzJF1uLT+sXg+JRfsGvPSqhb8CfhYR/yOzaSPwgXT5A8D/yZRfI2mJpLXABSSTXCeViPhERKyJiPNI/nv+Y0S8l0Xc74h4BnhK0kvSojcBD7OI+5x6Erhc0tL0//c3kcy9LfZ+N82rn+mQ1H5Jl6e/r/dn6rTX7Vn7bn9I3oz3C5IrAP6g2+1ZwH79Csmp5P3AvennrcBpwPeBR9LvUzN1/iD9PWylg6seTvQP8Hqmr25a1P0GXgUMp/+9/xY4ZbH3Oe3HfwZ+DjwIfJnkip5F12/gr0nmXSZJzgg+fDT9BNalv6tHgf9F+tSNw338WA4zM2ur14ebzMzsMBwSZmbWlkPCzMzackiYmVlbDgkzM2vLIWF2ApD0+uYTa81OJA4JMzNryyFhNg+S3ivpJ5LulfS59L0VL0r6tKSfSvq+pJXpvq+SdI+k+yXd3nzev6R/IekfJN2X1vml9PDLM++E+GpHz/o3y5lDwqxDkl4G/Gvgioh4FVAH3gMsA34aEa8B7gL+OK3yJeDjEXEx8ECm/KvATRFxCcmzhnam5a8GPkbyPoDzSZ5DZdZVpW43wOwk8ibgl4HN6T/yqyQPVWsAX0v3+QrwLUkDwGBE3JWWfxH4uqQVwFBE3A4QEWMA6fF+EhEj6fq9wHnAD3PvldlhOCTMOifgixHxiRmF0n9q2e9wz7o53BDSeGa5jv982gnAw01mnfs+8E5JZ8DUO4bPJflz9M50n3cDP4yIvcALkn41LX8fcFck7/QYkfT29BhLJC09np0wmw//S8WsQxHxsKQ/BO6UVCB5IudHSF7y83JJW4C9JPMWkDy+eUMaAtuBD6Xl7wM+J+mT6TF+5zh2w2xe/BRYs2Mk6cWIWN7tdpjlwcNNZmbWls8kzMysLZ9JmJlZWw4JMzNryyFhZmZtOSTMzKwth4SZmbX1/wESpRPYkA1oYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"cost\")\n",
    "plt.plot(epoch_list,cost_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Lets do some predictions now. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 437,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "124.97561189905038"
      ]
     },
     "execution_count": 437,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def predict(area,bedrooms,w,b):\n",
    "    scaled_X = sx.transform([[area, bedrooms]])[0]\n",
    "    # here w1 = w[0] , w2 = w[1], w3 = w[2] and bias is b\n",
    "    # equation for price is w1*area + w2*bedrooms + w3*age + bias\n",
    "    # scaled_X[0] is area\n",
    "    # scaled_X[1] is bedrooms\n",
    "    # scaled_X[2] is age\n",
    "    scaled_price = w[0] * scaled_X[0] + w[1] * scaled_X[1] + b\n",
    "    # once we get price prediction we need to to rescal it back to original value\n",
    "    # also since it returns 2D array, to get single value we need to do value[0][0]\n",
    "    return sy.inverse_transform([[scaled_price]])[0][0]\n",
    "\n",
    "predict(2600,4,w,b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 438,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34.60197457980031"
      ]
     },
     "execution_count": 438,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict(1000,2,w,b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 439,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "70.50604143757819"
      ]
     },
     "execution_count": 439,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict(1500,3,w,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:purple'>(2) Stochastic Gradient Descent Implementation</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Stochastic GD will use randomly picked single training sample to calculate error and using this error we backpropage to adjust weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 440,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 440,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we will use random libary to pick random training sample.\n",
    "import random\n",
    "random.randint(0,6) # randit gives random number between two numbers specified in the argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 453,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0.70486189, 0.67274269]), -0.2290363484141679, 0.00023657698876079387)"
      ]
     },
     "execution_count": 453,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def stochastic_gradient_descent(X, y_true, epochs, learning_rate = 0.01):\n",
    " \n",
    "    number_of_features = X.shape[1]\n",
    "    # numpy array with 1 row and columns equal to number of features. In \n",
    "    # our case number_of_features = 3 (area, bedroom and age)\n",
    "    w = np.ones(shape=(number_of_features)) \n",
    "    b = 0\n",
    "    total_samples = X.shape[0]\n",
    "    \n",
    "    cost_list = []\n",
    "    epoch_list = []\n",
    "    \n",
    "    for i in range(epochs):    \n",
    "        random_index = random.randint(0,total_samples-1) # random index from total samples\n",
    "        sample_x = X[random_index]\n",
    "        sample_y = y_true[random_index]\n",
    "        \n",
    "        y_predicted = np.dot(w, sample_x.T) + b\n",
    "    \n",
    "        w_grad = -(2/total_samples)*(sample_x.T.dot(sample_y-y_predicted))\n",
    "        b_grad = -(2/total_samples)*(sample_y-y_predicted)\n",
    "        \n",
    "        w = w - learning_rate * w_grad\n",
    "        b = b - learning_rate * b_grad\n",
    "        \n",
    "        cost = np.square(sample_y-y_predicted)\n",
    "        \n",
    "        if i%100==0: # at every 100th iteration record the cost and epoch value\n",
    "            cost_list.append(cost)\n",
    "            epoch_list.append(i)\n",
    "        \n",
    "    return w, b, cost, cost_list, epoch_list\n",
    "\n",
    "w_sgd, b_sgd, cost_sgd, cost_list_sgd, epoch_list_sgd = stochastic_gradient_descent(scaled_X,scaled_y.reshape(scaled_y.shape[0],),10000)\n",
    "w_sgd, b_sgd, cost_sgd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare this with weights and bias that we got using gradient descent. They both of quite similar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 454,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0.70712464, 0.67456527]), -0.23034857438407427)"
      ]
     },
     "execution_count": 454,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w , b "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 455,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1cefc8cc9d0>]"
      ]
     },
     "execution_count": 455,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2q0lEQVR4nO3deXxcdb3/8ddnluxb26Rb0p1SWqCltOwgICIU1CqLgIjg8uPiT9y9inq9/n567wPl4QL+RAsXAVEBFeHK1coi+9o2paUtbdMl3ZI0Tdpsk2SSzPL9/XHOmUwmM8mkySSTzOf5ePSRzJlzpuekzbzn+/kuR4wxKKWUUoNxjfUJKKWUGh80MJRSSiVFA0MppVRSNDCUUkolRQNDKaVUUjxjfQIjqbS01MydO3esT0MppcaNjRs3HjXGlCWz74QKjLlz51JZWTnWp6GUUuOGiBxIdl8tSSmllEqKBoZSSqmkaGAopZRKigaGUkqppGhgKKWUSooGhlJKqaRoYCillEpKSgNDRC4XkSoR2SMid8R5frWIbBGRzSJSKSLnJ3tsqryx5yh7G9tH669TSqlxI2WBISJu4F5gFbAEuEFElsTs9gKwzBhzGvAZ4IEhHJsSX3psE/e/Uj0af5VSSo0rqWxhnAnsMcZUG2N6gMeB1dE7GGPaTe8dnPIBk+yxqdDZE+RYRw/dwVCq/yqllBp3UhkY5cChqMc19rY+RORjIrIT+DtWKyPpY+3jb7XLWZWNjY3DOuG6li4AgmG9C6FSSsVKZWBInG393omNMU8ZY04CPgr8cCjH2sffb4xZaYxZWVaW1PpZCdW1+AEIhjQwlFIqVioDowaYFfW4AqhLtLMx5lVggYiUDvXYkRIJDG1hKKVUP6kMjA3AQhGZJyJZwPXA09E7iMgJIiL296cDWcCxZI5Nhd7ACKf6r1JKqXEnZcubG2OCInI78CzgBh40xrwnIrfZz68BrgY+JSIBwA9cZ3eCxz02VefqqHX6MLQkpZRS/aT0fhjGmLXA2phta6K+/zHw42SPTTVtYSilVGI60ztKXat2eiulVCIaGLZw2HBYh9UqpVRCGhi2ox3d9ISsUpSWpJRSqj8NDJszac/rFi1JKaVUHBoYNqfDu2JSnpaklFIqDg0MmxMYsyfnEQxpSUoppWJpYNhqW/zkZ7mZkp+lLQyllIpDA8NW1+JnZkkuHu3DUEqpuDQwbHUtXXZguHSUlFJKxaGBYYu0MFyiJSmllIpDAwPoCoQ41tFDeUkOHpdLS1JKKRWHBga9I6RmluRa8zC0JKWUUv1oYNA7aW9mSS5ul3Z6K6VUPBoY9C46WB7p9Db03mpcKaUUaGAAVklKBKYV5eBxWXeHDWnHt1JK9aGBgRUYUwuzyfK48LitwNCRUkop1ZcGBr1zMAC8LutHooGhlFJ9aWDQOwcDwG2XpHQ9KaWU6ivjA8MYQ22Ln3KnhWGXpAI6UkoppfpI6T29xwNj4InbzqUo1/pRuO2SlHZ6K6VUXxkfGC6XcGpFceSxJ9LC0JKUUkpFy/iSVCyvjpJSSqm4NDBi9JaktIWhlFLRNDBieF3a6a2UUvGkNDBE5HIRqRKRPSJyR5znbxSRLfafN0VkWdRz+0Vkq4hsFpHKVJ5nNLfO9FZKqbhS1uktIm7gXuBSoAbYICJPG2O2R+22D7jQGNMsIquA+4Gzop6/2BhzNFXnGI/XbWWodnorpVRfqWxhnAnsMcZUG2N6gMeB1dE7GGPeNMY02w/fBipSeD5J0aVBlFIqvlQGRjlwKOpxjb0tkc8C/4h6bIDnRGSjiNya6CARuVVEKkWksrGxcVgnDNEzvTUwlFIqWirnYUicbXHfhUXkYqzAOD9q83nGmDoRmQo8LyI7jTGv9ntBY+7HKmWxcuXKYb/LOyUpvYmSUkr1lcoWRg0wK+pxBVAXu5OILAUeAFYbY445240xdfbXBuAprBJXyjnLm2tJSiml+kplYGwAForIPBHJAq4Hno7eQURmA08CNxljdkVtzxeRQud74IPAthSea4THWa1WS1JKKdVHykpSxpigiNwOPAu4gQeNMe+JyG3282uAfwemAL8SEYCgMWYlMA14yt7mAR41xjyTqnONFun01lFSSinVR0rXkjLGrAXWxmxbE/X954DPxTmuGlgWu300aElKKaXi05neMTza6a2UUnFpYMTw6LBapZSKSwMjhk7cU0qp+DQwYvSOktKSlFJKRdPAiKGd3kopFZ8GRozeYbUaGEopFU0DI0bv0iAaGEopFU0DI0bv4oPah6GUUtE0MGI4fRgBbWEopVQfGhgxRAS3S/Se3kopFUMDIw6PS7TTWymlYmhgxOF1u7TTWymlYmhgxOF2iXZ6K6VUDA2MOLxu0U5vpZSKoYERh9slhLQPQyml+tDAiMPjchHQUVJKKdWHBkYcXrcQ0pKUUkr1oYERh1uH1SqlVD8aGHF43S4COkpKKaX60MCIw5rprS0MpZSKpoERh8ft0mG1SikVQwMjDq+uJaWUUv1oYMThdgkB7fRWSqk+UhoYInK5iFSJyB4RuSPO8zeKyBb7z5sisizZY1PJ63bp0iBKKRUjZYEhIm7gXmAVsAS4QUSWxOy2D7jQGLMU+CFw/xCOTRnt9FZKqf5S2cI4E9hjjKk2xvQAjwOro3cwxrxpjGm2H74NVCR7bCp53VqSUkqpWKkMjHLgUNTjGntbIp8F/jHUY0XkVhGpFJHKxsbGYZxuL4/LpS0MpZSKkcrAkDjb4r4Li8jFWIHxraEea4y53xiz0hizsqys7LhONJbbLbqWlFJKxfCk8LVrgFlRjyuAutidRGQp8ACwyhhzbCjHpopXlwZRSql+UtnC2AAsFJF5IpIFXA88Hb2DiMwGngRuMsbsGsqxqeTWkpRSSvWTshaGMSYoIrcDzwJu4EFjzHsicpv9/Brg34EpwK9EBCBol5fiHpuqc41ldXprSUoppaKlsiSFMWYtsDZm25qo7z8HfC7ZY0eLR5c3V0qpfnSmdxwel65Wq5RSsTQw4vC4hKC2MJRSqg8NjDg8bpcGhlJKxdDAiMPjEl1LSimlYmhgxOFxC2EDYW1lKKVUhAZGHB6XNdFcy1JKKdVLAyMOj9v6sQQTLA/S0NZFdWP7aJ6SUkqNOQ2MOAZrYdz5j53c/uim0TwlpZQacxoYcUQCI8F6UkfaumjrCozmKSml1JjTwIhjsJJUqz9AT1BHUSmlMosGRhyDtTBa/QF6dNitUirDpHQtqfEq0sIYIDB0rSmlVKbRwIjD63Y6vfu3IkJhg68rGGmFKKVUptCSVBzuAUZJ+ezO7mDY6MQ+pVRG0cCIw+NKXJJq9feOjtJ+DKVUJtHAiKN3Hkb/QIgOjG4dKaWUyiAaGHF47D6MQJwWRktnVAtDA0MplUE0MOLw2qOk4o2E0pKUUipTJRUYInJtMtsmikind5xAiA6MgLYwlFIZJNkWxreT3DYh9A6r1RaGUko5BpyHISKrgCuAchH5RdRTRUAwlSc2ltyuxEuDtPm1D0MplZkGm7hXB1QCHwE2Rm33AV9N1UmNNWeUVLxObx0lpZTKVAMGhjHmXeBdEXnUGBMAEJFJwCxjTPNonOBYSLrTWwNDKZVBku3DeF5EikRkMvAu8JCI/CyF5zWm3JEWRvxOb+d57cNQSmWSZAOj2BjTBlwFPGSMWQF8YLCDRORyEakSkT0ickec508SkbdEpFtEvhHz3H4R2Soim0WkMsnzHBFOp3eiFkZpQRagLQylVGZJdvFBj4jMAD4OfDeZA0TEDdwLXArUABtE5GljzPao3ZqALwEfTfAyFxtjjiZ5jiPGPcDy5q3+AGWF2Rxp69bAUEpllGRbGD8AngX2GmM2iMh8YPcgx5wJ7DHGVBtjeoDHgdXROxhjGowxG4C0un2d04cRSLA0SFlBNgA9odConpdSSo2lpALDGPNnY8xSY8zn7cfVxpirBzmsHDgU9bjG3pYsAzwnIhtF5NZEO4nIrSJSKSKVjY2NQ3j5xJxRUrElKWdp87JCOzC0haGUyiDJzvSuEJGnRKRBRI6IyF9EpGKww+JsG8p64OcZY04HVgFfEJH3xdvJGHO/MWalMWZlWVnZEF4+MWe12thhtc7S5hoYSqlMlGxJ6iHgaWAmVivhf+xtA6kBZkU9rsCa15EUY0yd/bUBeAqrxDUqPJFO776B4Cw86JSkdB6GUiqTJBsYZcaYh4wxQfvPw8BgH+c3AAtFZJ6IZAHXY4XOoEQkX0QKne+BDwLbkjzXYXMnmLjnzMEodVoYOqxWKZVBkh0ldVREPgk8Zj++ATg20AHGmKCI3I7VWe4GHjTGvCcit9nPrxGR6VgzyYuAsIh8BVgClAJPiYhzjo8aY54Z0pUNgzfBPb0jgVGgJSmlVOZJNjA+A/wS+DlWP8SbwKcHO8gYsxZYG7NtTdT39VilqlhtwLIkz23EuV2CSP+SlBMYU/KzcLtEA0MplVGSDYwfAjc7y4HYM75/ghUkE5LHJQTC8VsYxblestwuDQylVEZJtg9jafTaUcaYJmB5ak4pPXhcrn7Dap3AKMr1kuVxaR+GUiqjJBsYLnvRQSDSwki2dTIuedzSby2pNn+AbI+LHK/bCgxtYSilMkiyb/o/Bd4UkSew+jA+Dvxnys4qDXhcErfTuzjXC6AlKaVUxkkqMIwxj9gLAL4fa0LeVTFrQk04Hrer3x33ogMj2+OiW0tSSqkMknRZyQ6ICR0S0awWRv9RUpEWhpaklFIZJtk+jIzjcUvcTm8NDKVUptLASMDrcsUdVqt9GEqpTKWBkYA7QUmqKLqFoX0YSqkMooGRQGynt7O0eXRJKt4tXJVSaqLSwEggttO7LWqWN2hJSimVeTQwEvC4pU8LozU2MLTTWymVYTQwEvC6XH0m7sULDL0fhlIqk2hgJOB2CcGo1WojgZHXO3FPO72VUplEAyOBQUtS2oehlMowGhgJxK4lpX0YSqlMp4GRQOyw2riBoSUppVQG0cBIwOvuP6zWWdocIMvtJhQ2/ZYPUUqpiUoDIwG3q38Lw2ldgNXCAL2vt1Iqc2hgJOCNM0oqOjC8bgE0MJRSmUMDIwF3nE7v6MDItlsY3aHQqJ+bUkqNBQ2MBDxuF4GowGjpDFCSpyUppVTm0sBIwOsWQjElqSLtw1BKZTANjATilaRKcrMij7Pc1mgpHVqrlMoUKQ0MEblcRKpEZI+I3BHn+ZNE5C0R6RaRbwzl2FTzRs3DCITCtHcHdZSUUiqjpSwwRMQN3AusApYAN4jIkpjdmoAvAT85jmNTKnotKWdpc+3DUEplslS2MM4E9hhjqo0xPcDjwOroHYwxDcaYDUBgqMemmtclBEIGYwwt8QLDrYGhlMosqQyMcuBQ1OMae9uIHisit4pIpYhUNjY2HteJxuOxAyFsrBFSQNxO727tw1BKZYhUBobE2ZbsOhpJH2uMud8Ys9IYs7KsrCzpkxuM22WdQiAU7i1JxZmHoS0MpVSmSGVg1ACzoh5XAHWjcOyIcGZyh8KGFn8PgHZ6K6UyWioDYwOwUETmiUgWcD3w9CgcOyLcLutHEwyZSEmqJC96WK0GhlIqs3hS9cLGmKCI3A48C7iBB40x74nIbfbza0RkOlAJFAFhEfkKsMQY0xbv2FSdazxOCyMQDkeWNi/K6f1xRVoY2oehlMoQKQsMAGPMWmBtzLY1Ud/XY5Wbkjp2NHnsFkYobLUwCrM9kY5w0JKUUirz6EzvBDwxnd7FUUNqYWIFRk1zJ3eu3UFY7+2hlBqABkYCnj6d3n0XHoSoPowJUJL65/Yj3PdqNTXN/rE+FaVUGtPASMApPwVChpbOnj4jpGBidXr7uoIAtHXFzp9USqleGhgJOCWpoN3pHb3wIIDLJXjdMiFaGO3dVmA4wTGRdAdDnP/jF3nuvfqxPhWlxj0NjAQigREy/ZY2d2S5XROihdHW5QTGxGthNHX0UNPsZ8dh31ifilLjngZGAk4fRtAeJRXbhwFWx/dECAwnKCZiC6PNr+U2pUaKBkYCzrDaNn+AYNj0WRbEMXECY+K2MHrDcOJdm1KjTQMjAaeFcbS9G6BfpzfYgTGB+jCcr8n46+bayITGdOa0LJyWhlLq+GlgJOC0MI61W+tIxStJeSdIH8ZQS1L1rV18+fHN/HVzbSpPa0Q4QeHrTv9wUyrdaWAkEGlhdFgtjESd3t0TIjCcOn9ygdHUYYWoE6bpzKctDKVGjAZGAs4oqUgLI2ZYLVhLnE+IktQQ+zCc1XubO9M/MNp0jolSI0YDI4HekpTVwkg8Sio0quc10sJhQ3vP0OZhOPcHae5M/zdh51wn4ggwpUabBkYC3kind/97YTgmwiip9p4gxl5CKtkWhtPZ3dwxHloYTkkqgDG6VpZSw6GBkYA7UpLqxusW8rLc/fbJco//klR71CfvZD+FRwJjHJWkgmGDPzC+W4NKjTUNjAS89lpRRzusdaRE+t81diK0MJyQyMtyDz0wxkMLI2ror5allBoeDYwEnBZGTzActxwFkOVxT4DAsN5QZ5bkJj0Po3U89WFEhUTbOJg3olQ608BIwBlWC31vzRptIqwl5XzqdgIjlMQ9MVrtIar+QIiuNC/z+LoCkTslJjtsWCkVnwZGAl5X748mcQtj/Pdh+OxWxcziHCC52d4tUX0X6d6P0eYPUj4pz/peh9YqNSwaGAm4o1sYCQIj2zP+J+5Fl6SiHw8kurTTlOb9GG1dAcrta9OSlFLDo4GRQHQLI94sb5hYnd69gTF4C6PVH2BaUTYALWncj9EVCNETDFMxKflrU0olpoGRgNPpDfEn7UHvsNrxPL6/vSuIS2BqoRUAyQbG3Cn5QHq3MJxribQwtCSl1LBoYCTgTaIkleVxYYw1xj9Zr+5q5Lwfvcjh1vS4f7avK0BBtifSihqsJGWMoa0rGAmMljTuw3ACoqwwmyy3S9eTUmqYNDASEJFIK6M4UQvDM7T7eje0dfHVP26mtsXPzjS5A5yvK0hhjpdCeyTRYJ3ezkiqOaVWR3JTR/p+anf6LIpzrevTe2IoNTwpDQwRuVxEqkRkj4jcEed5EZFf2M9vEZHTo57bLyJbRWSziFSm8jwTcQIj3sKDYJWkILnACIcNX/3TZlrsN7FGX/cIneXwtHUFKczxUJid3NBTZw5GaX42hdmetB4l5ZSkCnOsFpQOq1VqeFIWGCLiBu4FVgFLgBtEZEnMbquAhfafW4Ffxzx/sTHmNGPMylSd50C8dmAM1OkNJDW09tev7OWNPcf43pWLAWjwdY3QWQ5Pe3fACoyc5EpSTid3Ua6XknxvWgeGU5IqyvVSlOPRUVJKDVMqWxhnAnuMMdXGmB7gcWB1zD6rgUeM5W2gRERmpPCchiTSwhhmSWp7XRs/e34XVy6dwc3nzqUox0NDmrQwnJJUjteFxyWDdno7b7oleV4m52Wl9Wxvp8+iKMdLYY5XS1JKDVMqA6McOBT1uMbeluw+BnhORDaKyK2J/hIRuVVEKkWksrGxcQROu5ezntRA8zBg8BbGCzuOEAob/mP1KYgIU4ty0qYk5bNLUiKSVJ2/NapfoCQvK63Xk/JFWhgeinI9WpJSaphSGRj9V+uzQiDZfc4zxpyOVbb6goi8L95fYoy53xiz0hizsqys7PjPNg5neZCEJakk+zC21rYyvzSfSflWX0hZQXbatDDau4ORDm/rU3hyfRjFuV4m52elfUnK7RJyvW6KxqiFER7CCDql0l0qA6MGmBX1uAKoS3YfY4zztQF4CqvENao8LhcF2Z5ISyNWsiWpbbWtnFJeHHk8tSg7LfowjDH2sForEK0WRvKBUZLnTesWRps/SFFU62m0h9W+c7CZxf/+DHUt6TGEWqnhSmVgbAAWisg8EckCrgeejtnnaeBT9mips4FWY8xhEckXkUIAEckHPghsS+G5xuVxS8J1pCC5Tu9j7d3UtXZxanRgFGbT6Ose8wl/3cEwgZCJamEkV5LyuKz7g0zOy6KjJ0R3mt51sK0rEGkdFuV48QdCBEZx7a8th1roDobZdSQ9hlArNVyeVL2wMSYoIrcDzwJu4EFjzHsicpv9/BpgLXAFsAfoBD5tHz4NeMq+B4UHeNQY80yqzjURt0vIz0r8I/ImUZLaWtsK0KeFUVaYTVcgjK87SFFO4kBKNac14azmWpDtpaa5c8BjWv2ByP1BSuwSW0tngGlF/W8wNdac/hkgamJikMn58YdJj7Rau2VxpG3sW5NKjYSUBQaAMWYtVihEb1sT9b0BvhDnuGpgWSrPLRlelyvhCClIriS1zQ6Mk8uLItumFlorwza0dY9xYFitiQLnTTWJklSLHRgAk+1l35s7e5hWlJPCMz0+bf5A5OfrBEebPzDqgVHfmh79VUoNl870HsCFi8q4aFHijnSn03ugFWu31rYyrzS/TzA46zaN9UipyMS2Pn0YA5ek2vy9ZZ5J+dbXdF1Pqq2rNzCcr6O5nlRtsx0Y2sJQE0RKWxjj3XeuWDzg8/GG1QZDYTxRneTbats4fc6kPseV2YEx1h3f0TOhra9e2ruDGGPi3pIWrJLUJLtl4XxN1xVro0tSztfRXLFWS1JqotEWxjDElqR+99Z+zvvxi5FP6U0dPdS2+DllZlGf45yS1Fi3MNq7+5akCnM8hA109CTuxG71ByJlOqe0k7YtjKjWkPN1tGZ7dwVCHG23fi71rRoYamLQwBiG2MDYsL+ZI23d/HGDNRfR6fCOHiEF1kSyLI9rzAOjrat3JjSQ1PIgrVF9GE5wpOOKtcFQmI6eUG9JKnd0S1JO6yI/y60lKTVhaGAMQ+/EPesT+d7GdgAefH0fgVA4qsO7b2CISFpM3utfkhq4bBMOG9qiAiPb4yY/y52WK9ZGRoDljk1Jyum/WD57Ek0dPWk79DiVjDF0B0NjPnxcjRztwxiG6HkY4bChurGDhVML2N3Qztqth9la08qcKXlx53JMLcoe8xZGu/3mmZ8d+6YaPwDae4KETd97nJfkZaVlC6M3DK1zLcjyIDJ6JSmnhbFiziRe33OUhrZuZk3OG5W/e6x95fFNrN1WH2l5f2x5OT+/7rSxPSk1IrSFMQzRJan6ti78gRCfOncuJ0wt4L5XqtkaM8M7mtXCGOtO7wC5XndkPslgn8Jbo1aqdUzOz6IpDQMjslKtfU0ul1CYPXrrSdU2+3G7hGWzrH//TClLhcOG57Yf4ZSZRXzpkoUsm1XCG3uOjvVpqRGigTEM0WtJVTd2AHBCWQG3XjCf7YfbqG3x9+u/cFjLgyRuYRw41kEwxbOSo0cRQXQfRoLAiFoWxFGS503LFWudlkR0uBXmeEetD6OmuZPpRTmUl1itikzp+K5p9tPZE+LjK2fxtUtP5EOnzqDB153WS8io5GlgDIOIkOV20R0KR/ovFpTls3r5zMjQ2YSBUZhDS2cgbm27uaOHS3/2Kr9960DqTh5r4cGCPoExSAsjTmBMzk/PFWvbYvpnwAqP0VpPqrbFT3lJLtPtCY2ZMrR2Z30bAIumF/b5urN+/C6PsuNwG3+qPDT4jhlAA2OYsjwuu4XRTmG2h7LCbLI9bm69YD55We7EJSk7UJyhl9F2HG6jJxTmtd0ju1x7rLauQKRVAYOPkooXGJPy0nPF2t6SVO+5Fo3ibVprm/2UT8qlKNdDjteVMS2MKjsYTpxmBcVJdmBU2UEyHt3/ajV3/GXLoLcvzgQaGMPkBMbexg7ml+VHJrx97oJ5vHXHJQkXL3RmezfE+eS5/bD1y1W5vzmlZSlfVzBS4wdrCKhLBm9hRC+XMikvC19XcFQX9UtG4pJU6n/pAyGrT6u8JBcRYXpRTsb0YVQd8TFrcm5kIEVZYTaT8rzjuoWxs95H2MC7h1rG+lTGnAbGMGW5e1sYC8oKIttFhOIB1qEaaPKe88vV3h2MhEcq+LoCfUo2IkJBduJP4XFbGPnOXIz06sdwgqEgO7okNTq3aa1v7SJsoHxSLgDTinIypiRVVe9j0bTeiaoiwqLpheM2MIKhMHsbrHLzOweax/hsxp4GxjBleVy0+APUtXYxvyw/6eN6lweJFxhtkab829XHRuZE42jvDvZ5Q4WBb6LU6g/gdVs3JHL0Lg+SXmUpX1eAwmxP5Da7wKjdRMkZUlthB8b04sxoYXQHQ1Qf7Yj833WcNL2IXUd84/JmUvuPdUSW/nnnoAaGBsYwZXlckbptdAtjMKUFWYj0D4xgKMyuI+2878Qy5pfms666aUTPN5pzP+9ohTl9h55Gl5paOnuXNnc4gZFuy4O0+YP97pRYlOPB1x1M+RuXM2mvvMQOjKIcjrSO/f1PUq26sYNQ2HBiTGAsml5IZ0+ImubxdyMpp2V0ankx7xxsGZehN5I0MIYpy+3iYJN1D4kFU5MPDI/bxZT8rH4lqf3HOugJhlk8o5Cz5k9m/b4mQin4TxoMhensCfUpSYEVGM4aU+8cbObk7z/LRrspHr02k8MpSaXb0Nq2mHIbWP0ZxkBHT2r7MZwWxsyS3pJUTyicdqE60pwPTrEtjN6RUuOv43tXvQ+XwMdXVtDqD1B9tGOsT2lMaWAMkzN5zyUwZ8rQZvKWFmTTGDN5b/th55euiLPmTcHXHWRHCvoxOrqt4bwDlaR+89o+eoJhHnpjH9B3HSnHpKh7YqQTX9TS5o7IPTFS3PFd2+yntCCbHLt0N6PY6q+a6GWpnfU+vG5hXmnf0qwzYqpqHPZj7Kz3Mbc0n3MWlALaj6GBMUxOYMyanEe2Z2h3nZtalNOvJLXzcBsel7CgrICz5k8G+vZjjFRZI96wU+i9r3d9axfPvldPYbaHZ9+rp9HXHTcwnBVra9Os3GCVpGJaGM49MVLc8V3b4o90eANMK86MuRi7jvhYUFYQWTnAUZDtYdbkXHaOw1vV7jri46Tphcwvzac415vx/RgaGMPk3BNjfmnyHd4O597e0XbW+zhhagFZHhczinOZMyWPt+1+jNoWP5ff/Rq/ennPsM87duFBh3MTpUfXHyRkDL/4xHICIcOfKg/FDYwcr5uz5k1m7dbDaVWjb4vTwoi+TWsq1bb4qSjpDQxn8t5Ev/NeVb0vUn6KtWha0bhrYXT2BDnQ1MmiaUW4XMLy2SUaGGN9AuOdszzIUDq8HWV2YER3pO083NanBnz2vCls2N/E4VY/n/ivt6k64uP+V6vpCgxv9dPY27M6nJLUY+sPcvGiqVy8aCrnLpjCo+sO0tzZE3deydWnV1B9tIPNaTROPXbZE+h7m9ZUCYdNvxZGWWE2IhO7JOXrClDb4o+Un2KdNL2QfUc7xtWqvbuPtGMMLJpu/W6vmD2JXUfaI8PLj0dtiz+tPlgNlQbGMDnN7/nHERhTC7MJhk2k/t/aaQ3PXTyjdxz7WfMn0+oPsPqXb3DU182/XraIls4Az2yrT/i6xpi4/yl7guHIgofOrNV4o6SCYUOjr5ubzpkDwCfPnkNtix9fV5CSOIGx6tTpZHtcPPlO7RB/AqkRDhurD6PfKKnU3xPjaEc3PcFwZIQUWP9HSguyOTKBZ3vvOhK/w9uxaHohobBhjz2nYTyosq9p0XTr99G5c+bxfjDaVtvKBT9+kQde2zci5zcWNDCGyenDWDCEORiOyOS9dqtUscMeRXJSn8CYAlifmB/69Jl8/sIFzJ2Sx6PrDiZ83S8+tonr7nu7X2h8/+n3OOfOF/nZ87siI3b6fwq33lTnTMnjwoXW/cwvXTItMjM99k3YOeayk6fzP1vqIktaD4cxhl+9vIebH1x/XC2pDnsZ9tEqSQVDYVr9AYwx/YbUOib6bG9n+GmiklTvEiEDl6XCYRNZFXmsVdX7yPG6mG0vS79sVgkuITJqcKh++lwVYQO/ennPuF1mRO+HMUxOYBxPC2N6sfUmvGF/MydNL2KnPRpqcdQvXXlJLv925WKWzy5hxRyrE/yGM2dz5z92suuIr18JYOOBZv625TAAL+5s4JLF0wA41NTJnysPMaM4h1+8sDsy+a4wdpSU/fims+fgsie9ed0urj9jFr94cU/CpU6uOr2cp9+t46WqBi47efqQfxaOrkCIO/6yhf/eXAfAb17fxxcuPiHyfEtnD/+9qZaVcydz8syiuPced4a1xnZ6O+H427f28+quRjxu4VPnzOW8E0qHfJ4bDzTxixf2cOBYBzXNfoJhQ47XRUG29fOJLkmBNbS2prlzyH/PUPh7Qhxs6mReaX7k/+Vo2VXvoyDb0y8oHXNL88lyuwYMjAZfF196bBObD7Ww9ksXHNfv1EiqqvexcGphZPJnQbaHRdOL2HQc/RgbDzTzUlUjVy6dwd+3HObhN/Zx+/sXjvQpp5wGxjBNyvNSWpBNaUHWkI9dVlHCWfMm8x9/287yWSXsrPcxOT8rMgvc8bkL5vd5fM2KCn763C4eXXeQ//ORkyPbjTHc9cxOSguyyc1ycfc/d/P+k6YiIqx5ZS8uEf582zlsrWnlO09tI2RMvxbDuQumcP0Zs7jujFl9tt949hxe3tXIslklca/l/BNKKSvM5sl3avoFRlcgxAOvVdPWFeQDi6exYs6kPjOwHUfbu7n1kUreOdjCv162iM2HWrj3pT1cu6KCqUU5hMKG2x/dxOv2/RVmFOfwwSXT+NoHF0WCLBgK892ntlGY7eF9J5b1eX2v28XHV1aws95HfVsXR9q6eGvvMZ7/2oVMszumk+HrCvCFP2wiZAxnzpvMFafOoCTPS6Ovm8OtXbjtUW7RphdnU3mgdxKmMSZu2B2vpo4erl3zJnsbO/C6hYVTC1k8o4iF0wpYOLWAUyuKIy3aeHqCYV7ceYRQGPKy3eR53XjcLtwua2b/wqkFkQ8Q8eys93HitIKE1+R1u1gwtYB/7jhCcZ6XyXlZTCvK4cTphcwszqHyQDNf+MM7tHUFcInwn3/fwW9uOWPQ6z7W3s09L+zmylNnRFrjxysQChM2JjLaseqIjwtj/g+tmFPCk+/Usm2Ae93E8/PndzElP4u7rl5KdyDM/a9Wc9M5cxN+AEtXGhjDdPvFC7nxrDnH9cvvcbu498bT+fD/e51/+d1GcrPcLJ5ROOhrTSnI5vJTpvPkOzXcseqkyHj/V3cfZd2+Jn6w+mRyvG6++cQWXtzZwMkzi/lzZQ3XrKxgRnEuM4pzOWveFOpa/ZFjHVOLcvjR1Uv7/Z3TinJ4+vbzB7yWj542k4ff3E9zRw+T7OG266qPcceTW9l31Hoju//VaqbkZ3HNigq+/IGF5GVZ/wWr6n185uENHOvo5lc3ns4Vp85g/9EOLv35K9z1bBU/uXYZP39+F6/vOcq/XbmY4lwvL+xo4NH1B9l4sJnffeYsJuVnce9Le9l4oJl7rj+NGcX9P+3edc2yyPfVje2suuc1vvPkVh64eWXS/4Z3PVPFEV8XT/3v8zgtQYDGml5kLWff3h3kt2/uZ80re/nmZYu46Zy5SR3v6AmG+clzVQjwhfefQFGOl/buILc8tJ6aZj/f//ASGnzdvFfXxmu7G/nLOzWANZrvvptWcNGiqf1eMxgK85U/bmLt1sT9Yh9cMo2fX3daZFFBRyhs+H8v7mbD/iZuOXfegOe+6pTp/PKlPdz1TFWf7YU5Hvw9ISom5fLIZ8/klapG7vzHTl6uaoh7vo6Gti5ufGAduxvaeeStA9xw5izuWLV4yG/C3cEQj607yL0v7yU/y83jt55DlsdFo6+bRTEt+JvPmcs/tzdw1a/e5HsfWswnzx78d//t6mOR/7f52R6+eulCrvzFEX7z+j6+dumJQzrXRHqC4VFpVUoqe+xF5HLgHsANPGCM+VHM82I/fwXQCdxijHknmWPjWblypamsrBzZixgF7x5q4dr73qInGOaz58/jex9aMugxb1cf4/r73+bfP7SET583F2Pgw798nbauAC987SJE4JKfvkJxrpcVcybxu7cP8PI3LkrpbUJ3HG5j1T2vsXhGEVMLswmFDa/vOcqsybn86KqlLK0o5pVdjazdepi1W+uZMyWPu65eij8Q4vZHN5Gb5eaBT63s04q5c+0O7nu1mq9deiI/e34XH19Z0edN/6WqBv7ldxuZX5rP1z+4iNt+v5EPL53B3dcvT+qcH3itmv/4+w5+9vFlXHV6xaD7bzzQxDVr3uKWc+fy/Q+fPOj+jj9XHuJfn9jC7Ml5HGzqpLwkl9oWP3ddvZSPx7TmEmnq6OHzv9/Iun1NiMCU/Gy+efkinnqnlvX7m7j/phWREqSjtTPArgYf3//re+xu8PHLT5zepwUYDhu+8cS7PPlOLd+6/CTef9JUOnqC+HtCBMOGUDjM9ro2fvb8LhZNL+KBm1dGyk4Nvi6+8vhm3tx7jKuWl/PDj57SL1Di8feEaO7sobbFz856H1X1bXjdLr566YkU5XjpCYa57O5XEYFnvvy+uG+EdS1+bnxgHUfaurj3E6fzVvUxHnitmikF2dx8zhw+tHQmc0vz6QqEeG33UV7ceYQTpxVy41lz+twp888bD3Hvi3uoa+3ijLmT2HHYx9TCbL78gYV8+fHNPPKZM/u1VJs6evj6nzbzUlUjl588ne9euTjh75Uxhuvue5v9xzp49ZsXRz6gff73G3lt91Fe/ebFkblMx6O1M8CPn93J9ro2/vL5c+O23AcjIhuNMSuT2jdVgSEibmAXcClQA2wAbjDGbI/a5wrgi1iBcRZwjzHmrGSOjWe8Bgb0vqHcc/1prD6tfND9jTF85JdvsLW2lXml+SyfVcKTm2r5+XXL+Nhy643vT5WH+OYTWxCxhr7+5Nplg7zq8H3/r9vYfriNnmCYnpDh/BOm8NVLT4y0JBxv7T3Gt/6yhYNNnbjEmtn+wM0rI8tpOHxdAS7+ycscbe/h5JlF/OXz5/ZrFb2x5yif/e0GugLW6KR/fOWCfh3eiYTChuvue4tdR3z8+OqlrNvXxCu7Gqlr8eMSQcRaRHD1aeVceeoMbv1dJe1dQZ772oX9ZskP5M29R/nEf61jWlE2P1x9ChcuKuN/PbKR13Y3cvd1p/GRZTM52m69iRbmWH0BznV2dAfZWe/jq3/cTH1bF3ddvZQFZQV876/bIiN27r7uND66PPH/m9bOADc/tJ6tta38x0dPYWlFMR6Xi0fe2s8f1h3k65eeyBcvSVxTf7mqgS8+uolsr4sTpxVS0+ynrsWPxy38YPUpXLuiYkRLbC/sOMJnf1vJHatOYvmsEl6samDTQetas9wudjf46OwO8fBnzoj07W2rbeWHf9vOun1W6W/RtELqWvz4uoPket34AyFmT87jG5ctotUf4NcvWUGxfHYJX790EeedMIXKA83c/OB6AqEwgZBh/XcuYWqccmU4bPiv16r56fO7CIcN16yo4PMXLWDWpDxcLqGzJ8hTm2p5+I397G5o5werT+ZTUa3Jqnofq+55lfwsD5efMp0PLZtJS2cPr+8+ypt7jxEKG2aW5FA+KY8FZfksrSjm1PKSSLnaGMNTm2r5z7/voMUf4JZz5/Kvly3q97uRjHQJjHOA/2OMucx+/G0AY8ydUfvcB7xsjHnMflwFXATMHezYeMZzYID1n2hBWT4ed3JNy47uIGu3HuaJjTWs29fESdML+fuXLoh8ygiEwlzy01eoae7kha9f1G/JhrHW2RPk58/vorkzwP/9yMkJP53+z7t13PPCbh68+QxmJ1h+ZV31MX7wt+38YPXJkTeQZO072sHld79KdzBMtsfFuQumcOK0QgxWoGytaWX9/t7+hwdvWcn7T5qW+AXjCIcN/9hWz/kLSyMlE39PiE8/vJ71+5rI9lhvaNGmFWXTFQhHxv2XFWZz/00rWD57UuQ1/2dLHV63iytOnTHoObR3B/nswxsib6iOz1+0gG9etmjQN/w9DT6+/eRWgmFDxaQ8KiblctXychYmmHsxHMYYbnloA6/ssm4i5nEJSyuKyfK4CIQMXrfwnSsWs7SipN+xdS1+1m49zAs7Gpg1OZcrl87k3AVTeHPvMe5cuyMyouv02SV8+QMn8r6FpX2u/a29x/j0w+vJ8brZ9L1LB/y51Ld28euX9/DY+kORVW3zs9yEjKErEGbJjCI+c/48rlpe3q8PaMP+Jh5ff4hn36uPjJoqzvVy3glTyMvyUNvsp7bFz6HmTpy3aWfel8EQCBlOm1XCf37sFE6emXx/Sqx0CYxrgMuNMZ+zH98EnGWMuT1qn78BPzLGvG4/fgH4FlZgDHhs1GvcCtwKMHv27BUHDqT2tqbpqrbFT67X3a95W7m/if3HOrlmxeDllky28UAzbV0Bzpk/Je6ntENNnfx1cy1et4t/uXDBiP29Hd1B7v7nLsIGZk/Oo7wkF193gIPHrDeKHK+LmSW5zCzO5fyFpZQWZA/+ogPoDoZ4a+8xuoNhQmFDUY71BjWSrYORUtPcyYOv7+eMuZM4f2FpvzlDxyMUNjy//QhFOR7OWZD4ujcdbOZYew8fWJLcB4PDrX7Wbq2n1R+goztIKGy44tQZnDF30qA/265AiDf3HmVyfjanlhf3Kyt1dAd5r66NLTUtHG3vwXm5E8oK+FicIBqqdAmMa4HLYt70zzTGfDFqn78Dd8YExjeB+YMdG894b2EopdRoG0pgpHKUVA0Q3ZtXAdQluU9WEscqpZQaRakch7UBWCgi80QkC7geeDpmn6eBT4nlbKDVGHM4yWOVUkqNopS1MIwxQRG5HXgWa2jsg8aY90TkNvv5NcBarBFSe7CG1X56oGNTda5KKaUGl9J5GKNN+zCUUmpohtKHoYsPKqWUSooGhlJKqaRoYCillEqKBoZSSqmkTKhObxFpBI53qncpcHQET2c8yMRrhsy87ky8ZsjM6x7qNc8xxpQNvtsEC4zhEJHKZEcKTBSZeM2QmdedidcMmXndqbxmLUkppZRKigaGUkqppGhg9Lp/rE9gDGTiNUNmXncmXjNk5nWn7Jq1D0MppVRStIWhlFIqKRoYSimlkpLxgSEil4tIlYjsEZE7xvp8hkNEZonISyKyQ0TeE5Ev29sni8jzIrLb/jop6phv29deJSKXRW1fISJb7ed+Iel4S7YoIuIWkU32XRwz5ZpLROQJEdlp/5ufkyHX/VX7//c2EXlMRHIm2nWLyIMi0iAi26K2jdg1iki2iPzR3r5OROYmdWLGmIz9g7V0+l6sO/xlAe8CS8b6vIZxPTOA0+3vC4FdwBLgLuAOe/sdwI/t75fY15wNzLN/Fm77ufXAOYAA/wBWjfX1DXLtXwMeBf5mP86Ea/4t8Dn7+yygZKJfN1AO7ANy7cd/Am6ZaNcNvA84HdgWtW3ErhH438Aa+/vrgT8mdV5j/YMZ43+Uc4Bnox5/G/j2WJ/XCF7fX4FLgSpghr1tBlAV73qx7j9yjr3PzqjtNwD3jfX1DHCdFcALwPvpDYyJfs1F9hunxGyf6NddDhwCJmPdz+dvwAcn4nUDc2MCY8Su0dnH/t6DNTNcBjunTC9JOf/5HDX2tnHPbmIuB9YB04x1J0Psr1Pt3RJdf7n9fez2dHU31r3gw1HbJvo1zwcagYfsUtwDIpLPBL9uY0wt8BPgIHAY6y6dzzHBr9s2ktcYOcYYEwRagSmDnUCmB0a8muW4H2csIgXAX4CvGGPaBto1zjYzwPa0IyIfAhqMMRuTPSTOtnF1zTYPVsni18aY5UAHVpkikQlx3XbdfjVW6WUmkC8inxzokDjbxt11D+J4rvG4rj/TA6MGmBX1uAKoG6NzGREi4sUKiz8YY560Nx8RkRn28zOABnt7ouuvsb+P3Z6OzgM+IiL7gceB94vI75nY1wzW+dYYY9bZj5/ACpCJft0fAPYZYxqNMQHgSeBcJv51w8heY+QYEfEAxUDTYCeQ6YGxAVgoIvNEJAur8+fpMT6n42aPgPgNsMMY87Oop54Gbra/vxmrb8PZfr09YmIesBBYbzd3fSJytv2an4o6Jq0YY75tjKkwxszF+vd70RjzSSbwNQMYY+qBQyKyyN50CbCdCX7dWKWos0Ukzz7fS4AdTPzrhpG9xujXugbr92bwFtZYd+yM9R/gCqzRRHuB7471+QzzWs7HalZuATbbf67Aqk2+AOy2v06OOua79rVXETVKBFgJbLOf+yVJdIiN9R/gIno7vSf8NQOnAZX2v/d/A5My5Lr/L7DTPuffYY0OmlDXDTyG1UcTwGoNfHYkrxHIAf4M7MEaSTU/mfPSpUGUUkolJdNLUkoppZKkgaGUUiopGhhKKaWSooGhlFIqKRoYSimlkqKBoVQaEJGLxF5pV6l0pYGhlFIqKRoYSg2BiHxSRNaLyGYRuU+s+3C0i8hPReQdEXlBRMrsfU8TkbdFZIuIPOXcv0BEThCRf4rIu/YxC+yXL5De+1v8IZ3uz6AUaGAolTQRWQxcB5xnjDkNCAE3AvnAO8aY04FXgO/bhzwCfMsYsxTYGrX9D8C9xphlWOsgHba3Lwe+gnV/g/lY62QplTY8Y30CSo0jlwArgA32h/9crAXgwsAf7X1+DzwpIsVAiTHmFXv7b4E/i0ghUG6MeQrAGNMFYL/eemNMjf14M9b9EF5P+VUplSQNDKWSJ8BvjTHf7rNR5Hsx+w203s5AZabuqO9D6O+nSjNaklIqeS8A14jIVIjcY3kO1u/RNfY+nwBeN8a0As0icoG9/SbgFWPdn6RGRD5qv0a2iOSN5kUodbz0E4xSSTLGbBeRfwOeExEX1kqiX8C6edHJIrIR685l19mH3AyssQOhGvi0vf0m4D4R+YH9GteO4mUoddx0tVqlhklE2o0xBWN9HkqlmpaklFJKJUVbGEoppZKiLQyllFJJ0cBQSimVFA0MpZRSSdHAUEoplRQNDKWUUkn5/9lpKP1+VQrIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"cost\")\n",
    "plt.plot(epoch_list_sgd,cost_list_sgd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 456,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "128.25785506303845"
      ]
     },
     "execution_count": 456,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict(2600,4,w_sgd, b_sgd) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 457,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30.347665843402435"
      ]
     },
     "execution_count": 457,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict(1000,2,w_sgd, b_sgd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 459,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "69.45899958796899"
      ]
     },
     "execution_count": 459,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict(1500,3,w_sgd, b_sgd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:purple'>Exercise</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Implement mini batch gradient descent in python and plot cost vs epoch graph. Mini batch is intermediate version of batch GD and stochastic GD. In stochastic we used one randomly picked training sample, In mini gradient descent you will use a batch of samples in each iterations. For example if you have total 50 training samples, you can take a batch of 10 samples, calculate cumulitive error for those 10 samples and then adjust weights. In SGD we adjust weights after every one sample. In Batch we adjust weights after going through all samples but in mini batch we do after every m samples (where m is batch size and it is 0 < m < n, where n is total number of samples\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Solution](https://github.com/codebasics/deep-learning-keras-tf-tutorial/blob/main/8_sgd_vs_gd/mini_batch_gd.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}