03a_REINFORCE_Logistic_NumPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "confused-energy",
   "metadata": {},
   "source": [
    "In this notebook, we are going to implement a variation of REINFORCE algorithm. For a CartPole environment, a simple policy with a couple of parameters and logistic function will be enough.\n",
    "\n",
    "## Defining a Policy\n",
    "\n",
    "Logistic function (Sigmoid):\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "Derivative of logistic function:\n",
    "\n",
    "$$\n",
    "\\frac{df}{dx} = f(x)(1 - f(x))\n",
    "$$\n",
    "\n",
    "Lets take four parameters $W = w_0, w_1, w_2, w_3$ since the CartPole state $S$ consists of 4 variables. Now we can define a policy by multiplying weights with the state variables $W * S$ elementwise, then sum up the elements in the resulting vector and apply a sigmoid function. The output of sigmoid lies between 0 and 1, therefore we can treat it as a probability of selecting the first action (Remember, there are only two actions in CartPole). Actually, the element-wise multiplication of $W$ and $S$ with subsequent summation is just a **dot product**: $\\sum_i W_i * S_i = W \\cdot S$.\n",
    "\n",
    "With this, we can already define the first methods.\n",
    "\n",
    "```python\n",
    "class LogisticAgent:\n",
    "    def __init__(self, params, lr, gamma):\n",
    "        # Initialize parameters, learning rate, and discount factor\n",
    "\n",
    "        self.params = params\n",
    "        self.lr = lr\n",
    "        self.gamma = gamma\n",
    "\n",
    "    def logistic(self, y):\n",
    "        # definition of logistic function\n",
    "\n",
    "        return 1 / (1 + np.exp(-y))\n",
    "\n",
    "    def action_probs(self, x):\n",
    "        # returns probabilities of two actions\n",
    "\n",
    "        y = np.dot(x, self.params)\n",
    "        prob0 = self.logistic(y)\n",
    "\n",
    "        return np.array([prob0, 1 - prob0])\n",
    "\n",
    "    def compute_action(self, x):\n",
    "        # sample an action in proportion to probabilities\n",
    "\n",
    "        probs = self.action_probs(x)\n",
    "        action = np.random.choice([0, 1], p=probs)\n",
    "\n",
    "        return action, probs[action]\n",
    "```\n",
    "## Running an episode\n",
    "Next we need to define a method that will run one episode to collect the experiences:\n",
    "\n",
    "```python\n",
    "def run_episode(env, agent):\n",
    "\n",
    "    observation = env.reset()\n",
    "    totalreward = 0\n",
    "\n",
    "    observations = []\n",
    "    actions = []\n",
    "    rewards = []\n",
    "    probs = []\n",
    "\n",
    "    done = False\n",
    "\n",
    "    while not done:\n",
    "        observations.append(observation)\n",
    "\n",
    "        action, prob = agent.compute_action(observation)\n",
    "        observation, reward, done, info = env.step(action)\n",
    "\n",
    "        totalreward += reward\n",
    "        rewards.append(reward)\n",
    "        actions.append(action)\n",
    "        probs.append(prob)\n",
    "\n",
    "    return (\n",
    "        totalreward,\n",
    "        np.array(rewards),\n",
    "        np.array(observations),\n",
    "        np.array(actions),\n",
    "        np.array(probs),\n",
    "    )\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "Let's also define a ```train()``` method that will run several episodes in a loop.\n",
    "\n",
    "```python\n",
    "def train(\n",
    "    env,\n",
    "    params,\n",
    "    lr,\n",
    "    gamma,\n",
    "    agent_cls,\n",
    "    MAX_EPISODES=1000,\n",
    "    plot_period=1,\n",
    "    seed=None,\n",
    "):\n",
    "\n",
    "    # initialize environment and the agent\n",
    "    if seed is not None:\n",
    "        env.seed(seed)\n",
    "    episode_rewards = []\n",
    "    agent = agent_cls(params, lr, gamma)\n",
    "\n",
    "    # train until MAX_EPISODES\n",
    "    for i in range(MAX_EPISODES):\n",
    "        stats = {}\n",
    "        # run a single episode\n",
    "        total_reward, rewards, observations, actions, probs = run_episode(env, agent)\n",
    "        # keep track of episode rewards\n",
    "        episode_rewards.append(total_reward)\n",
    "        # update agent\n",
    "        agent.update(rewards, observations, actions)\n",
    "\n",
    "    return episode_rewards, agent\n",
    "\n",
    "```\n",
    "\n",
    "## Learning\n",
    "\n",
    "Now, we have defined everything, apart from the most important piece, which is `agent.update()` method. Here, the learning will occur. This method will accept the list of rewards, observations and actions collect during one episode.\n",
    "\n",
    "Let's remind ourselves what are the steps of REINFORCE algorithm:\n",
    "\n",
    "<img src=\"https://i.stack.imgur.com/Acbup.png\">\n",
    "\n",
    "After we have collected the experiences we would also need to calculate returns $G_0, G_1, ..., G_{T-1}$. Let's define a method for this. We will simply loop over the rewards in reversed order and discount them with parameter $\\gamma$.\n",
    "```python\n",
    "class LogisticAgent:\n",
    "    def discount_rewards(self, rewards):\n",
    "        # calculate temporally adjusted discounted rewards\n",
    "\n",
    "        discounted_rewards = np.zeros(len(rewards))\n",
    "        cumulative_rewards = 0\n",
    "        for i in reversed(range(0, len(rewards))):\n",
    "            cumulative_rewards = cumulative_rewards * self.gamma + rewards[i]\n",
    "            discounted_rewards[i] = cumulative_rewards\n",
    "\n",
    "        return discounted_rewards\n",
    "\n",
    "```\n",
    "\n",
    "The next step is to take the gradients of the logistics function $\\nabla \\operatorname{ln} \\pi(A_t | S_t, \\theta)$. For this we will define the following methods.\n",
    "\n",
    "```python\n",
    "class LogisticAgent:\n",
    "    def grad_log_p(self, x):\n",
    "        # calculate grad-log-probs\n",
    "\n",
    "        y = np.dot(x, self.params)\n",
    "        grad_log_p0 = x - x * self.logistic(y)\n",
    "        grad_log_p1 = -x * self.logistic(y)\n",
    "\n",
    "        return grad_log_p0, grad_log_p1\n",
    "\n",
    "    def grad_log_p_dot_rewards(self, grad_log_p, discounted_rewards):\n",
    "        # dot grads with future rewards for each action in episode\n",
    "        return np.dot(grad_log_p.T, discounted_rewards)\n",
    "\n",
    "```\n",
    "\n",
    "`grad_log_p()` will calculate $\\nabla \\operatorname{ln} \\pi(A_t | S_t, \\theta)$, while `grad_log_p_dot_rewards()` will calculate the sum of the rewards $\\sum_{t=0}^{T-1} G_t * \\nabla \\operatorname{ln} \\pi(A_t | S_t, \\theta)$. Therefore the final `agent.update()` method will look like the following.\n",
    "\n",
    "```python\n",
    "    def update(self, rewards, obs, actions):\n",
    "        # calculate gradients for each action over all observations\n",
    "        grad_log_p = np.array(\n",
    "            [self.grad_log_p(ob)[action] for ob, action in zip(obs, actions)]\n",
    "        )\n",
    "\n",
    "        assert grad_log_p.shape == (len(obs), 4)\n",
    "\n",
    "        # calculate temporaly adjusted, discounted rewards\n",
    "        discounted_rewards = self.discount_rewards(rewards)\n",
    "\n",
    "        # gradients times rewards\n",
    "        grad_surrogate_objective = self.grad_log_p_dot_rewards(grad_log_p, discounted_rewards)\n",
    "\n",
    "        # gradient ascent on parameters\n",
    "        self.params += self.lr * grad_surrogate_objective\n",
    "```\n",
    "Notice that we use the gradient of the log probability only for already executed action.\n",
    "\n",
    "## Logistic Agent\n",
    "Let's put everything together and run our agent in an environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "supposed-leone",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "loose-atlanta",
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from IPython.core import display\n",
    "\n",
    "\n",
    "class LogisticAgent:\n",
    "    def __init__(self, params, lr, gamma):\n",
    "        # Initialize paramters, learning rate and discount factor\n",
    "\n",
    "        self.params = params\n",
    "        self.lr = lr\n",
    "        self.gamma = gamma\n",
    "\n",
    "    def logistic(self, y):\n",
    "        # definition of logistic function\n",
    "\n",
    "        return 1 / (1 + np.exp(-y))\n",
    "\n",
    "    def action_probs(self, x):\n",
    "        # returns probabilities of two actions\n",
    "\n",
    "        y = np.dot(x, self.params)\n",
    "        prob0 = self.logistic(y)\n",
    "\n",
    "        return np.array([prob0, 1 - prob0])\n",
    "\n",
    "    def compute_action(self, x):\n",
    "        # sample an action in proportion to probabilities\n",
    "\n",
    "        probs = self.action_probs(x)\n",
    "        action = np.random.choice([0, 1], p=probs)\n",
    "\n",
    "        return action, probs[action]\n",
    "\n",
    "    def grad_log_p(self, x):\n",
    "        # calculate grad-log-probs\n",
    "\n",
    "        y = np.dot(x, self.params)\n",
    "        grad_log_p0 = x - x * self.logistic(y)\n",
    "        grad_log_p1 = -x * self.logistic(y)\n",
    "\n",
    "        return grad_log_p0, grad_log_p1\n",
    "\n",
    "    def grad_log_p_dot_rewards(self, grad_log_p, discounted_rewards):\n",
    "        # dot grads with future rewards for each action in episode\n",
    "        return np.dot(grad_log_p.T, discounted_rewards)\n",
    "\n",
    "    def discount_rewards(self, rewards):\n",
    "        # calculate temporally adjusted, discounted rewards\n",
    "\n",
    "        discounted_rewards = np.zeros(len(rewards))\n",
    "        cumulative_rewards = 0\n",
    "        for i in reversed(range(0, len(rewards))):\n",
    "            cumulative_rewards = cumulative_rewards * self.gamma + rewards[i]\n",
    "            discounted_rewards[i] = cumulative_rewards\n",
    "\n",
    "        return discounted_rewards\n",
    "\n",
    "    def update(self, rewards: np.ndarray, obs: np.ndarray, actions: np.ndarray):\n",
    "        # calculate gradients for each action over all observations\n",
    "        grad_log_p = np.array(\n",
    "            [self.grad_log_p(ob)[action] for ob, action in zip(obs, actions)]\n",
    "        )\n",
    "\n",
    "        assert grad_log_p.shape == (len(obs), 4)\n",
    "\n",
    "        # calculate temporaly adjusted, discounted rewards\n",
    "        discounted_rewards = self.discount_rewards(rewards)\n",
    "\n",
    "        # gradients times rewards\n",
    "        grad_surrogate_objective = self.grad_log_p_dot_rewards(grad_log_p, discounted_rewards)\n",
    "\n",
    "        # gradient ascent on parameters\n",
    "        self.params += self.lr * grad_surrogate_objective\n",
    "\n",
    "\n",
    "def run_episode(env, agent, render=False):\n",
    "\n",
    "    observation = env.reset()\n",
    "    totalreward = 0\n",
    "\n",
    "    observations = []\n",
    "    actions = []\n",
    "    rewards = []\n",
    "    probs = []\n",
    "\n",
    "    done = False\n",
    "\n",
    "    while not done:\n",
    "        if render:\n",
    "            env.render()\n",
    "\n",
    "        observations.append(observation)\n",
    "\n",
    "        action, prob = agent.compute_action(observation)\n",
    "        observation, reward, done, info = env.step(action)\n",
    "\n",
    "        totalreward += reward\n",
    "        rewards.append(reward)\n",
    "        actions.append(action)\n",
    "        probs.append(prob)\n",
    "\n",
    "    return (\n",
    "        totalreward,\n",
    "        np.array(rewards),\n",
    "        np.array(observations),\n",
    "        np.array(actions),\n",
    "        np.array(probs),\n",
    "    )\n",
    "\n",
    "\n",
    "def train(\n",
    "    env,\n",
    "    params,\n",
    "    lr,\n",
    "    gamma,\n",
    "    agent_cls,\n",
    "    MAX_EPISODES=1000,\n",
    "    plot_stats=None,\n",
    "    plot_period=1,\n",
    "    seed=None,\n",
    "    evaluate=False,\n",
    "    video_folder=\"\",\n",
    "):\n",
    "    plot_stats = plot_stats or []\n",
    "    if plot_stats:\n",
    "        num_plots = len(plot_stats)\n",
    "        fig, axs = plt.subplots(\n",
    "            num_plots, 1, squeeze=False, figsize=(10, 5 * num_plots)\n",
    "        )\n",
    "        axs = axs.ravel()\n",
    "\n",
    "    history = defaultdict(list)\n",
    "\n",
    "    # initialize environment and the agent\n",
    "    if seed is not None:\n",
    "        env.seed(seed)\n",
    "    episode_rewards = []\n",
    "    agent = agent_cls(params, lr, gamma)\n",
    "\n",
    "    # train until MAX_EPISODES\n",
    "    for i in range(MAX_EPISODES):\n",
    "        stats = {}\n",
    "        # run a single episode\n",
    "        total_reward, rewards, observations, actions, probs = run_episode(env, agent)\n",
    "        # keep track of episode rewards\n",
    "        episode_rewards.append(total_reward)\n",
    "        # update agent\n",
    "        agent.update(rewards, observations, actions)\n",
    "        stats[\"score\"] = total_reward\n",
    "\n",
    "        for k, v in stats.items():\n",
    "            history[k].append(v)\n",
    "\n",
    "        if plot_stats:\n",
    "            if (i + 1) % plot_period == 0:\n",
    "                for ax, stat_name in zip(axs, plot_stats):\n",
    "                    ax.clear()\n",
    "                    # print(stat_name, len(history[stat_name]))\n",
    "\n",
    "                    sns.lineplot(\n",
    "                        x=np.arange(len(history[stat_name])),\n",
    "                        y=history[stat_name],\n",
    "                        ax=ax,\n",
    "                    )\n",
    "\n",
    "                    ax.set_title(stat_name)\n",
    "                display.display(fig)\n",
    "                display.clear_output(wait=True)\n",
    "\n",
    "        else:\n",
    "            print(\n",
    "                f\"episode {i}/{MAX_EPISODES} | {stats}\",\n",
    "            )\n",
    "\n",
    "    return episode_rewards, agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "hazardous-holiday",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAE/CAYAAACEto0QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABhOklEQVR4nO2debwdRZn3f88999x9yXLvzR6yJ4QlQBa2hC0IwaAOjCCo7E7EBcFlFJdR51VH1FedeccZHVSUcUFxUMfRkQFxZZQlQcAACZAQIPu+L3er949z+tw+fXqp7q4+p889vy+fcPtUV1dVd1dVP/XUU0+JUgqEEEIIISQ+dZUuACGEEELIcIGCFSGEEEKIIShYEUIIIYQYgoIVIYQQQoghKFgRQgghhBiCghUhhBBCiCEoWBFCCCGEGIKCFSGEEEKIIShYEUJqFsnBfpAQYgx2KISQ1CMiHxKRTSJyQETWishSEcmIyEdEZF0+fJWITMrHP0tEHheRffm/Z9nS+q2IfEZE/hfAYQDTRGSOiDwoIrvz6V9ZqXslhFQ3wi1tCCFpRkRmA/gVgNOVUptFZAqADIDLAVwL4I0AngdwMoCNABSAdQDeA+AeAFcA+FcAM5RSu0TktwCmAbgEwFoArQBWA/g4gO/k03kAwLlKqWfKc5eEkOECNVaEkLQzAKARwFwRySqlNiil1gF4G4CPKaXWqhxPKaV2AVgO4AWl1HeUUv1KqXsArAHwOlua31ZKPaOU6gewDMAGpdS38vGfAHAfcgIbIYSEgoIVISTVKKVeBHAbgE8C2C4iPxCR8QAmIaeZcjIewMuOsJcBTLD9ftV2fByA00Vkr/UPwFsAjDVyA4SQmoKCFSEk9Silvq+UWoycEKQAfA454Wi6S/TN+Xh2JgPYZE/SdvwqgN8ppUbY/rUppd5h7g4IIbUCBStCSKoRkdkicoGINAI4CuAIctOD3wDwKRGZmV/dd7KIjAbw3wBmicibRaReRN4EYC6An3tk8fN8/GtEJJv/t1BEji/D7RFChhkUrAghaacRwB0AdgLYCqAHwEcAfAnAvcgZmu8H8E0AzXk7q0sBvB/ALgAfBHCpUmqnW+JKqQMALgJwFXLarq3IacQak7slQshwhasCCSGEEEIMQY0VIYQQQoghKFgRQgghhBiCghUhhBBCiCEoWBFCCCGEGIKCFSGEEEKIIeorXQAA6OrqUlOmTKl0MQghhBBCAlm1atVOpVS327lUCFZTpkzBypUrK10MQgghhJBARMS5bVYBTgUSQgghhBiCghUhhBBCiCEoWBFCCCGEGIKCFSGEEEKIIShYEUIIIYQYgoIVIYQQQoghKFgRQgghhBgiULASkUki8hsReU5EnhGRW/Pho0TkQRF5If93pO2aD4vIiyKyVkQuTvIGCCGEEELSgo7Gqh/A+5VSxwM4A8C7RGQugNsBPKSUmgngofxv5M9dBeAEAMsA/KuIZJIoPCGEEEJImgj0vK6U2gJgS/74gIg8B2ACgDcAOC8f7W4AvwXwoXz4D5RSxwC8JCIvAlgE4E+mC1/L7DnUi1f3HMbJE0d4xnl28350tTegp72pELbz4DFs3XcUJ07oxK6Dx/DoS7ux7ISxEAH+8MJO7D/ahw07D6GjOYv9R/pwrH8QAqCrvRHZTB2WzOzCmi0HsOPgMVx8wlj8es129A0MorWxHstPGoeXdx3Coy/txu5DvThz+mgcOtaPZzfvR0dzFgBwrG8AB4/1Y3RbY1FZx3Q04uEXdqG1MYNMnWB0awPqM8Vy/+a9R9CUzaCxvg4N9XW4csEkrN9xCKte3o3dh/owtbsVOw4cw55DvehszmJAKXQ2ZyEAegcGsePAMYwf0YyGTO76g8f6sXnvEcwZ24GejkY88fIezB3fgXXbD2LT3iM4e0YXzpvdAwB4dfdhrHx5N0a0NGB6Vxt+9/x2jGhpQN/AII71DxbKuO9IH+oEaG/K3e9pk0di9th2HO7txzOb92PhlFHYuu8odh48hj+u21mI1zcwiD2H+jCqrQHb9x/F+BHNAICt+46iq70R9XXi+Z437z1SiO/Gtv1HMbqtOI1dB49h35E+dDRlsetQL9qb6jF+RDO27D2CxmwGHU312HWoF2M6mnDgaB+UQuEdAsDOA8fQ0ZzFzoPHfPOuNvoHBrH/aD/qRNBQX4eWhgz2HenDuM4mrN9xCD0djRgcVDh7Rhemdbfh4Rd24szpo5HxeT9evLr7MP64bic27T2K7vZGXDx3DHo6cm117dYDWPXyHiyZ2YXxI5rx86c3o74uV54t+45i/IgmHDzWD6WA48d1YOWG3egdGMT4zmYc7hvAtK5W7Dnci4FBhU17j6CtsR7zJo7A71/YgZ0HjqG9KYumbB0ydXV4ZfdhNGQEIoL9R/swpqMJo1sb0DswiK37jgIAlszsxrb9RzGoFNqb6vHCtoMQAQ4c7QcA9LQ3Yv/RfszsacOarQewYMpIbN9/DId6+7F131GcP6cH+w734UjfAFob67F13xEcP64DT2/ch72HezGytQECwdZ9R9Dd0YSOpnrsPdyHad2tmD2mHT0dTXjspd0Y19mER1/ajZk9bXh2y370DwyiozmLA0f7cehYPzbtPYLWxnqMaM5iSlcrdh/qdX32ew/3IZsRtDbW49Cxfuw4eAxTRrfi4NF+7D/ah5aGejRn6/Dy7sMY09GETlvdD+Lg0X4MKIXe/kF0NmfRUF+qv9i89wjGdTZj35FcOfYf6cOYzibUiWi1eT8OHevH9gPHMGlUC7bvP4pspg6dzVls3nsEE0e1YHRrA0a2NGBGTxt+u3Y7pna1YkRLFis37MFF+T596ZwerN68D53NWUwa2YJNe49gbGcTVr28B0op7Dnch017jmDCyOai+xEp7o+UAg739uOkCZ1Yv/NQSVntabjRnM1ABMhm6nC4dwBtjRnsOdxXyNPO+BHN2Lz3CF7edRiXnTYB5+f77kogSin9yCJTAPwewIkAXlFKjbCd26OUGikiXwHwiFLqu/nwbwL4pVLqPxxprQCwAgAmT548/+WXPb3DExeWfvG3WLfjEDbcsdwzzpTbf4H2xnr85e+HZmMXfPpB7DzYiw13LMd77vkzfvbUZvzXuxdj/c6DuPUHTwbm+5q5Y/Dgs9tcz/34nWfhX379Ih5asz30/UThslMnYP3OQ3jq1b2J5fHCZy5BNlOHq+98BH9avyv09YumjMK9N5+Jm7+zCvc/sxWPfWQpFv3DQwmUlJSbu29chOvuegx/e/FsvOv8GaGvn3L7L4p+nzurG3ffuKjo3AnjO/Dak8bhC/+z1jOd1oYMDvUOhM6/WshmBL9+/3lY8vnfVLoow54Txnfgmc37MXdcB57dsh8AMLWrFS+5CEVpZmxHEx75yNJE8xCRVUqpBW7ntPcKFJE2APcBuE0ptV/EU5p2O1EivSml7gRwJwAsWLBAX7ojAIB1O/Qq+oFj/UW/dx4cGsE9v+0AgJw2Z/Peo1rpuY0AGzJ16B0YRG9/TnvTUF+HXpsWBwDmjG3Ht29YhDM+mxMqfrjiDBw3uhUAcNPdj+OZzftL0n3kw0MN4z+f3ITP/nJN0fnntpRe4+ThD52PxZ8b6pDfdf50/Mtv1gHIfch+9/wO3+utccf2A97P50c3n4lJI1sAoHB/j3x4KT5439PYceAYABQ6qSN9xR/AzuYs/ue2cwrXWbxn6UycOL4DK76zqpCeG796bhs+9tPVmN7diu+97YyS809v3FuSxsu7DuFNdz5SEves6aPxx3XFwuM3rl2At/17bh/Phz90PurzGo4r/21IAb3guJH4yptPcy1fteF8DxfNHYMHPAYS2/fn6sS6HQeN5O3WBp7ZvB/Hj+vwvW44C1UA0DegCpqxsBw3ugU/XHGmI73BgpD2x9svwFl3/BoA8LbFU/GNh19yTefntyxGl0PL7oWzDjnb7kNrtuGjP1mN0a0N2GXrTzN1gq++5bTANh82/zC8vOswAGDDrqHvi59Q9cUr5mFgUOGD9z2NrrYGvGbuGNzz2Kt41/nTcc0ZU4rKcvWiSbh16azC7z+t34n3/vApzOhpw3dvOr0k7Sdf3Yubv7uqJPyN8yfi1Mkj8NGfrC4KXzyjCw+/uBMA0D84WHJdOdESrEQki5xQ9T2l1I/zwdtEZJxSaouIjANgqSk2Aphku3wigM2mCkzMcOO3H8ehXquzUlClsq8rR1w68bo6AAM5IURBIeMidHc0ZTG2c2hKsru9sfDbTVUOoCi+mypeRDA46F/uCY5pKns6I1uC1fvWc/EZSGBMe1NRWYFc2Uc0Z/HKLn8BuK2xvuRaAOhoqkdX+1BH7hYHALraGgAATdmMa5xNe0vTONzr/pFqbSztDrptZRjX2YxMnZQIzS0e9zAcaGvy7iK/mf8Ih1D6RyLahNDwwqf5+VJfJyV10z5LM852zu9dj+1s0has3K61M7p1qM3aaayv02rzSTIYsjL3dDTiWF+uP2isz6At34c4+3sAaG0o7ieswWhLg3vf1b23wTXPtsZ6jO8snT5s93l/5SawJJL7onwTwHNKqS/ZTv0MwHUA7sj//U9b+PdF5EsAxgOYCeAxk4Um8fm1bbouTFs62u8iWOV7PQWFwUHAzTTA2THaBZU4H44ggdApENXZfjc3mFlTUWeTC7905Tw01ufSbWnIFDRUXuWs81k+UqfxNcn4JQD3D5JXum42HXUu76k+UzufevGpnWu25jS+AwHCPakcbgOior5HU2IzWeP98tRp80mi+y24/ZI52LrvKM6YNhoPv5DTEgUVfd+RvqLfQXaJXs9JxP3aRo8BeiXQEfHOBnANgL+IyJP5sI8gJ1DdKyI3AXgFwBUAoJR6RkTuBfAscisK36WUGt666mGA3wfEjjU6Kb42T15j5dYg/BqRTufmFSXsN83ecTlHjG5YHY1fCettws3lp00sSt+p4XM+Zzft3lBZA4tXEHK8knHrqL3iOhcLeMWtKcFK41ZNCVZKKew5lDPkDpM/8Sbs4/Oa+jUp8HilJfDvD8qBpbEKErDGdTbh5nOnAxiqo0HXZB2Cz7TuNgAopOPE+zm5h3vNfFQCnVWBD8O7frpOAiulPgPgMzHKRcqIQrDmx8JpI1SSlnL/GDgFK/E4DkuYxRdAsbCiI1hppenRnpsbMjiaF0S9iunXYet05kErh9zOhtFY2d+bdVk2QEs2nNCpm2GnT7zYdagXp37qQTxmM7rNiFC4QnQBM+x1d167oGRRQZz83bCalLMZKcP5REG3KtsHw1GFzs7mrO/iK6+uTcR9mjTrMjCsFOkpCakK3GyslO2vgpeWxE9jFZyv1ygl7DfNLihkIy5nLknT4waasxn0Dgyif8DbkLLOV5OnkXegOj04DYvAqcD8cS1prHQ+GqanAnccPFY4juLGgZjhQ8vmFI51pwx1sNq8W59W6alA3UGCvZRJldlPszdrTDvecV6xpitNGqv0lIRUDF3hpKG+zldjpfKSldu3wPkttrcZ3WlI1zxDxo9iX5GL633O6+NnhfcPqqEpRUdU/6lAHY2VfxN2S8NLmHMTmNyipmlkmDQ6yjlTGisLe3061j+IP7+y12j61UjUPiJO3zLaNiVrUr71ExgqrQzWrct1RRqroXCTAqhXUlb47DHtReFpEqzSY0ZPKopOe2rMlLpRsF+r8v+5fswdYUUdnk5bdIkjCP9Rs5dj0qiWwPg6yXsJKjqCkZ/GSs94PRcnzGPwStVNSHPrKKM6LqxOomustu0/isb6OoxocV/d5IXzvb+w3Yw7h1rEqwm94ZTxGNsRsOouIYHBaj72JKd3t+J9r5mdAo1V7m+QyYe4PBuRIdMME0MNTwHUlp+dRtuAL+mVukFQsKpylHI3Fg+bhg5eQoBln6VUrmG6r8SJXj4/wk8FDh03ZzP41vULccO3Hw+8zm/k66V1soL9hD8vGeXECZ16xusRpgI9baw0NVacnipmwOP1nv4PD6E5m8Fzn1oWKr1Kf1xrgX+66tTAOMXTXebytvoL+3t+6P3nAQBe3H7AXEYJYn8cSfUHfpo9N9wW31SK9JSERMKEZK5rNBnUgBRyQpp7NIfGKqTCyjVFCa+xKp4KjJixA6/nYgX7FdHt2u/ctAhnTButJTDXeYzenOfdyuUkyMbKwuTofTjg50staOTvBgXXUqIbr0d/liYMtP3SdUuxWtqWvZj2fs7y9aXjIzAIz2bgEe620KZSUGNFAGhOeXlU1sJUoFKexut+6Bmv++etS4ltQEDeP3tqE960cLJ2mnYsLZef8OfWkVpOOU18X12L5iVYuYz4al17Uk53CxaVXnI/nIjzJJN6C37tOo3trU5K3dp42aretHgqRrY04I3zJyIunn6sIL7n0wA1VlWOiS49yhJbr7J4uVvwTTeO8XoMdwsiwTl/6L6/5ON6x/HSMBT8u7iEFa7N//4X25YwVqlMdLJhVh65aaziCL3DAZ3qNWDYoKPSBsxppBJ1zP4eTAo8hf7CdZreWDbGcDXtsB3by1yfqcOVCyf52o7q4pWE1+ObMKIZy04Yi4kjm/Fv18yPnX8cqLGqcnKCRXlao6fGynGs0wkVGz8G5+1ltxX2k1akLvZINyze/lZyJ5TPtlXWs3rN3DG264rPxcEtCa9UrWczeVQLXtmd2zPMr4OcN7ETT23ch9fPGx+3mFVN0LZKpHLEaUL2QYlJ5UiaPa+7ETRlGWSOEDnfANvVkkFqneBrFRaoLChYVTlGNFaaqQQ2+rzKKrTGKkaDjGNjZaoT8+oACrYHEd+STvHsCwf8ylAc5lVefXus9f/w2ryNG22CTLtbqDRXzJ+IH63aWOliGCGWYFVkR2SujvsorIxoepxcMKenaAuzsLgvgLEf65U5vNmGR3k8hoZpkkmpdK5yytmnezYgm7sF3Q9t8V6BGvE9wuM01ro6fV1fFM2WdcWgCp6ydDPm1+lkrXfivSIm2vSeM/2S8DqBiAx7oUrnWT3xyl6jdlaVltPS6AA2svG6IW1+En6s3PqUJJpTg61vuPD4ntDXuz1DcfSjgPl66+1uwb1cadL2UWNV5UTVhjgS0SLI9kNZewXGL5EWAvHcK3BadytuuWBGSXjGIdAl2RYtwcguVHkbZNqOC1OBwXnMHdeBv1kyFdeeOcU9XbepwBDLmFPUV1UE3Y/FK7sPY2pXa+F3dU8PDp+XHk9jJa7HcbEGI64aqwQanH2PvjFBvrs0sQs1SQk0XsmKx/k01VpqrKocU+4WdPBqQPbpKKU0bazsx1o2Vrm/S2Z2leTuxrITxuKyU0tXpiThbsELu8bKIoyxva6D0Y8un+vp7DTMJsxDxvZDZUzTKDDNDAwWG9L1+mxjFESlpxbT+crLX6ikcvR7vkk8+2wmphDkOuIaOvRLM879eKYboMlKAxSsiDaeglVhKlB/VWCcRnDe7O6SvHXTd9oGmJoqcC9DXmNlE1S8y1skaubDDJTBJczrPbY35XzPnDF1dGBcUky/Q0PVF0OwMu2+oZaJU3vLvQdeUnnaN02PMtUY1Ie4eZJ3ixeWIAehLkPGyHmZhlOBVY4RjZXuVKC/iVUMP1Yh49uO42xpUyd6U4FKRZvetNL+3C/XYvO+o/5xXa4r96rA0W0N+NX7zsGkUS0F42X6VNKjf8ApWEVvmJSrSnGrhs3ZTLAD1hD1tzmbiXppKMotWNltNSPZigb0If6rHIHwLnKHrvUrT5q7JmqsqhxdG6t9R/rwjT+sj5VXUKMf8ryuMxVot3UKptg9w5AmKJa7BdHLu38w/EpHYOh53fdEuBVWVlYmOlmdfRuH8hXM6GlHY/3QB0bYQ2jh1DLF0TpVeiqwWjhpQmdgnDAt6PqzpwAAfrjiDHzzugUVmQo0Ybz+3ZtOL5r+K07fzF0Vr5j0ixc9vyAHoWnSUDlht1nl6PbBn/jP1fj0L55zT0NTPPFaAWa3G4rkIDRkfPsWCmGNhIsFNL1rnNoI7bxcwsJMXSa14C6M6QKnAvV4+MWdRXUxzqKS57dVdr+4annjOo5Uw1Rfa0Bx+rTRWHr8mIDY0fHrs00YyS+e2YUlM7tdz5nqU5yaf8D9vuLkF6SxSjMUrKoc3e5735E+7zRMeV7PrwoM7SBUJ+9i5TOAvBsDjWvtlHQIOlOBiKexcqblhtsKJBOdrPuqQI+4LmHD3JuCMb7wP2txp10jHEPp9O7v/zl+gWKQRn2ZWzXUcutiviiJknR7i+Iexe37YE/Fr7+PY0rg7W8vX4YUv1wKVlVO2C1d4hBkY3Xzd1fh+W0HQ2/JEVWAUEqFtjEr8byu0fVGfsQat+Vnv2Cik3VdFRjCwR41Vvo8+crewnEahRNdqmUmshrqptuUnP9UYDL3NL6zKZ+3GcHK3oX42TwlYrxuDTwjp5w8FKyqHDOe1/UENN1GEsfhp2d8L6FOo9z337bENR1dL8dRn7Grxkr5/w66PizhphjDfQSIN9UinLhTHYXX04xXrgJ/64aF+PX7zysJ96sbyTjcVehozq34jZK8m5ZddzV1nMdfzfadVVx0ApTb87peGbQab8ipQK98dUys5oztwNxxHfm87FOBeg0/tyowfCldbax0rstfmNQmzEF7cNmpBq1AWuCjcmdMR2PsNNzaTdqnAs+f3ePpX86LpOtQlPbs1seWw0Qg0PN6ihscBatqx4i7Bb1EdCtyaPcJEduHirAusDgvPXEpssbKpXXpPGurVCb6Dfe9Ar3ydYub3s4rzRjZEaFCmB6sTRndGhwpAmFtOdNC+acCiweTYXHrs8oh1ATtFZjCV1uAglWVU84OXFdNrTeSLDVGD4uuxqo43yFyGis9GytTxusl5fExLjfSybqm72+7YIfG69FwE04eXb8LU27/BZ56dW/Zy1PtuD1Pz+0xbSTpADgJTAlWXgO4KAKRWx9bDoE1WGOVfBmiQsGqyjGxok9XNtFd4WHKfioIVfhfmLzsozfNjA3KrmGSMiHUhPmwuMVMs7o9bdgflfM9f+MP67HiO6sAAH9ctwv7j3qv0q005TQv6GpriHxttW4AbpXarfxpXBXoRsShcAXzLi+BgpWI3CUi20VktS3shyLyZP7fBhF5Mh8+RUSO2M59LcGyE1NoC2d68RJxt+BqCK48nSnq+IvS9rwecWPptBqvm4hLSvETYj/9i+eKXJ78aGU4p7HVjF/3svNgL0a2ZCOlotU+UlinZ/S04W+WTMW/XTO/5FzSAxlTglt5pgKrV2Ols6XNtwF8BcC/WwFKqTdZxyLyRQD7bPHXKaVOMVQ+EoCpwaXOKFV7VWDI/i6Oxiq8H6vifLVsrCI+ZPf70rCxMthx0EaqMgTZ0nk4xk4F5bYP08nN3fmkGeP10yaPQL3OvKIhRHIbp1cCUwKRPZmuttzihHedP8NI2hbBNlbpbUSBgpVS6vciMsXtnOTe0pUALjBcLqKJCT9Wuh2prn+q0HsFRnXPoLy3//AugmjEKckmEnGNa404CI143fzjRmLVy3ti51+rBDXLTBk/5GEpt6uIqPlp2XJqNIAfv/PsaAVIkIVTRiaSrrEtbWzHzQ0ZbLhjuZF07QRprNJM3E2YlwDYppR6wRY2VUT+DGA/gI8ppf4QMw/ig06f9MQre/DrNdt94+hUVqMaKwOtI+d/K9w1kaYCI1qvu12hU17ns2nKRv8IR33M/37jImw/cCxyvtXK/7v6VLznnuQ9n6d5c+s0rmd0K1PYPUmrhd984Dz0tMd3UeFGElvaJIXnDhEFlX7iRYhMXMHqagD32H5vATBZKbVLROYD+KmInKCU2u+8UERWAFgBAJMnT45ZjNrF7UO9/cBRPPDMtsLvL9y/NnQabugKQ6E1VlqCWGlYFG1d6RRkcOYDSkVayeVqF6Zzne34jstPwsKpo0Ln7VcGHVob6zG1MW73UH10NuvY+3hgN14P0lilV2FVNQzXZzi1K757Cq/ql8QmzEnhvQlz8d80Erlqikg9gMsB/NAKU0odU0rtyh+vArAOwCy365VSdyqlFiilFnR3u28YSYJxm8Z7+3dW4WM/Xe0S2ycdjS++rl2IjkdzEzZWfq4W3jh/knu+jlWBOnnbhdQwuKWts6Gz/bqrFk3G9O62SPl7lYF4U67HlQm771MV894LXT8BoXB3t2BmKrCW0N1tIohKagKHu4PQCwGsUUoVlreISLeIZPLH0wDMBLDe43piApcOZ8+h3nBJaCp+dEc7Om23eFVgNJX+gI9k5TXqs6ei28cc6R3Qi+jA7Xn1Dw5GSisq6e16hjd+dosi6da2mLaxOnP6aLMJ5tH5sKb421sRzK0KDH9NdW/zFA4ddwv3APgTgNkislFEbsqfugrF04AAcA6Ap0XkKQD/AeBmpdRukwUmxZjaK1AH3dFOaHVz1FWBUaYCi/LS9bwe7Sm7pd3n0Fj1dDS5XGfua8BVgeEw9biCqmaa30vY+q7TLTz9yYvw7RsWRiyRe5l07NSq0cbKNKNbh3yFmXoalay+1ng6zW9WZ1Xg1R7h17uE3QfgvvjFIrq4deDOkZyp5dO6ox2daPYOL2oDGYhkY2WfCkxWnew22/PyrkNFv++4/KSSOCaLlOLv97DD/qiDamaanVuGbVY97U3Yuv+ob5yOpixaGtw/NzoDJE4FRmPRlFG4ZekM/Oq5nDmDMXcLFRRrBvOSVZrfbYoV0kQHHaEpqN/S1fzorwoMa7werYMciDCr5lwVqEPYbXMKebl0Pu+796nC8bmzutHeFMNYOmIZiDfleF6CdK8K9HJh4kWlbiXNWr+08KaFk9BYnyn8NvXIdNP5+rULzGRoI2z9rAQUrKocV41VQnnFsbFafvLY4gBxPQxF3AZWJ5pTgUYdhNrS9bouWnaRykCKMTcV6F9pTBkRJ0Gc/Td948W45ah7BdYq1vNyas1N1Trdb8Hp00bjHy4r1crHYWgqML1tiFWzyjFlY6WTju7HwE1Ff9mpE4t+mzDDiitYieiVI7KNlUviDXo7xxqDglX5sL/voBpTn2rBKv0aAUBz9XGNN4AS4aOK3C14YdXPNL9aClYpp29gEO/94ZN4cftB1/OuI2NHhTPVTWrbWIUVmiI2EL9Vgbr56ox6on5n3J5Xc8OQWt5zB3qDklWaR3VpJM7TEgD3r96CLz/4fLDxeooFq7CLQnSFlzh3HN14PT7HjW4xkEplcD4iY8brhtKJQsHGqoJlCIKCVcrZuOcIfvLnTbj2m4+6ntfpAx97yX9hpml3C1o2U7ZjHfnI3UGoVnE8SdpGw02oOWdWeX22hfl+p3kEWDZiPoObv/sE/umhFxA0nEm1jZXNdnGUbUVZJamk8frxYzviJ5ISym1jZTJPi5jj6bJAwSrlNNTnXtHmff6rbuKhV1Od7cOa1lpxzrSi8LACy389tTlUfAsjGiudqcCIEpxbvz+oUWazqwLdE2uxac5I+Umz4bV9tW057P2ituKwA7haxPmMTGmwKznFWtBepvjlUrBKOUEf9aTNIV43bzzmjG13PZepE8wa04brz5pSFK7nIDRsqyiNH8Xdgh3dj1vkbFyS/8VftkS5LDJeaT37f5YZzGX4YOrDE1RnUixXFfU5UbebKgdp1vqlBecTMqaxMpNMJGi8ThLHhI+q4I9AvgI76vGRvgE8v+1gSWMt12i8tz+eF3PdYkZ9wmnQSqSgCDWD/VkH1Zk024cXK1XLIeZHQ2sdSI02AOsVOvsgYzZWIZ5rY37WpSlrRkvupfVvqE/Pu669XVZrgCjVy6+jD0rPOXLQGuUGR0kcgd5egVFXSQUl7ZWsyY9BmLTS/LEvF+XwvC4C/PjPG70jVJjBhDRWnvE06p3b8wy7J2ktUmK8buiBhLHdfP288Xh19xG8bclUI3k7VwXOHtOOS08eh/Nm9RhJ3wQUrFJOsHPP4t+b9x7B0b5wmpygfi2oMUbRWJVr5aAfdUmvCoy48qvWPwaVpBzPvm9A4cdPbCpDTtEI78cq+afmpplPop8ZbpRMBZqysQqRTn2mDrdeONNIvkDpljYdzfW4Zam59E3AqcAqx9ndnHXHr7Fp7xGjeRQ6MC8NS0l8o9mXcPYMM5u61omexsqgiZUjXQ93CzX+MRgO+E3R7zhwrIwlCY9KynjdYFqAbj9T242ppC+pwKpA06iCxiq975aCVZUTdcVacRr+5wPrb4mvFB0VfbhGYY/tNJaPirYNVNSpwBQ1/K62dCybTztx3pn9Sr8qs+Ng5QSr9sbgSQr7atsxLpuEOylHNXd315ee9pVekrGxirsiu1rz1oWCVRXR77I5nhnP6/6phLWxctt82CTGBBYfdwvTulsLx1GfcdSNdk1/MO68Zj5+9u7FRtMcrpTDxupYyKl6k7Q3BQtWdhurT7xuLq454zjf+H6P7FfvO3consfDje5uwUyc4UypjZWZB9IXZaNWQ1SBXEXBqpr49ZrtJWFJGBwvntFVHBDQGKM03vA2VuZ7SKeNld2txMUnDO1tGPUZR962xPCtXnTCWIwf0Ww2UVJC8ZY2VdD7e2D/cDU3ZHDFgonekQOY0dNmoETuwpezT3jL6ZNL4xjJvfqwZjKSWhV4LOaK7DhUw5ZLFKxSjr0O7T3S5xbDaB4A8KUr5xX9DjkTWDX+ZfymAu0yUdSGHKSxqoL+oeYoT82t3Iv3yrnFY6slyf/nRzm2tHHD3rxGtmRxq4sBc5V0RYmRlB8rnSnipDBh/pI0FKyqiANH+0vCylHHKrGCb2pXa9HvJPpHp+d1+wfCxHRckMbKeY/2clWCWv8IBfGr953je17XxqqS3wWvvB9479C92TVWursTuKG14bgmbh9Te7FWfuw1rtfVuh1WEu4Wvv+209Hd3hg/oYikX6yiu4XUY59S2O+isTJjY+VPkJG3c8Qadxn0WdNH45+vPjUwjbjUSXG3W3Rs+2Haxqqhvg5fv3YBTp86yvV8bX8KKotfvcwkbTxYQex1NbQfK4/w33/w/OJ4MSp20FRgJjevb5SCX+SQ6f72A+eht4I2SHZKN2E2MGA0KDBHoRqM1ylYVRFujTWR0W9JYwwVPfaoaP5xIzG6LfkRUanGynZsixf1GXsJVnUCnOuzGXOaVhPWHnrTw0H41ZmHXGwly4Wniw/bfds9W+ec6AYtG3YPHtvZ5Ihmtl4n7dZl6fFj8NYzJuM9IX0kTfHQRFeCKM6b004VyFWcCkwr+470ac0lm9nSRvmmc/3ZUwAA0z0MUSM5CA3rbiGBDsGv0zFhiOwtWKWrdzt/treQR4YI897Sarz+7vNn4MQJHSXh9lu7cfFU1/Ck0Orn3FVWxT8NC27ZTB0+/Vcnoae9cvZEUbGel+Wk+G+WTAMATBzZEjvtSndftLEikXhx+wHM+/sHcO/KV0N7XjeBs4O69OTx2HDHcoxudfeF5IxfnzG/KjAJcn2Ou11VUfkirwp0b16BU6vRsiMGSEO9TIJp3a3YcMdyXHPmFPz8liUl5+23/YZTJmB63t2IIFhg0X1k8Z6tv40VKcaa3bDs3P56/kRsuGM5Opurf5KKqwJJJF7YdhAA8Js1O4rC3TqSctYxz47RER7ZzUCUvGPg53nd/jExbWMVdosgUj78Hn3Qe7HXk7T1/YFVqsTIORegoFNfk6+wfQPBW9qw3Qxh+ZlqqHd+4uM/pEo/5pSYr/lCwSrlBPXP9imHqCpS52U6woZffB0j30o3TqC03Hand0XG6xGfq5eAmbapQGIGez1JmVwVWvixqq5S6RBYel38JqWhXGmlz6GxshgOz4xTgSRx7HUs8frmKXAVk01CY5WAKCaOVYH9NqvIYj9W0dLPeEyJBj2eWl8iXkn8BJAg4aRYY5Wuzv/Dl8zxPV+ye0L+XnWmXUzVVr924ebp2xnfVaOfOhG3PPT15+47W++0Q4tPJYWzpmxdweY3zVT/hCspELULcXY+9nZjdwngpWlxfnC8BAq/a8LS3d5oZDNbezn6izRWtqnAqIKVxz0Gu6+Ill9cUiYLEIMsPX6M73mvOjeoFOrFzPhba0rRoxK6aqw0xIT/eWabVtmGG5Ygmi3RWFX3oG3Npy6pdBG0CGwxInKXiGwXkdW2sE+KyCYReTL/77W2cx8WkRdFZK2IXJxUwWuFoJGv/XQcoz6vS+3bROg2SR0bq9DN23HBf7+n1ADXD2vpd1PW0dHYjt3sOADzqwKrvXMbzvjaWAVcW6Q9NlGYMuK8N0v415kKjFudrWfll4zrFioedmFk6HmVTAUaSZ3POQgdjdW3AXwFwL87wr+slPq/9gARmQvgKgAnABgP4FciMkspNWCgrDVDmE652MYqYn4+1xV5I9ecCiyHI8WuNvcVil58+cpT8NCabZjR0+4Zx9vGKnTxAPjZWEVLLyn4QRrC61HouKRIs/F6ECVOfvNNWCkdP3Z69ccrnvWs/Krh4xt2l4QltQ/ecMBbY1WJ0tQegV9ApdTvAZTWanfeAOAHSqljSqmXALwIYFGM8tU8UYWsUHmUGK+HW83mDM+Wwd1CWGGgsyWLy08r3UzWnky/wzFiXKL6sSp355c2e6A0cuHcMcGrAoueY3U9Uy+NVTmXtvu1ue89+opLfOKF16pAE/0ahbNg4qgW3i0iT+enCkfmwyYAeNUWZ2M+rAQRWSEiK0Vk5Y4dO9yiEAduFbqcxuueqwId4UGbD0fLOxnsZTe9KtBL+Eur8To7TP9nH/Reqllj5URsgpWpqcDAeCHrXxL74A0X3rQwZ8LR2pgpCuczKg9RBauvApgO4BQAWwB8MR/uvjDDLVCpO5VSC5RSC7q76fk5KknsFejV9nQ1Vlo2Vils4SNbhqYX7aUzvYVCGu+d5Ij1aqpYmCrdUy5Hru4bsswJHFCEg25LvPnQstl4/tOXoLG+WLDScd4cBJ96MJEEK6XUNqXUgFJqEMDXMTTdtxHAJFvUiQA2xytibWMf+bqNmC1tyr2Pv4q7/7ihTKXyx8vjeBySEkbsyS49vqdwbO+0v/PIy5HTXzqnpyQsSKPH70U6CXYQml4/VkGUulvI/VUaGitjZYirseInv4CIuDgHTaZvJqVEesoiMs728zIA1orBnwG4SkQaRWQqgJkAHotXRGLhZkNlhXzwvqfx2V+uiZauKk7ZWzOlZ3sVd1RUiWmUxvo67y1tYjB/ysiSsOCpQFKNbNh5uHBcLVOBzdm8RsNR6QqrAhF8L7qDniDBhxqo5DGxKwY17sHouFu4B8CfAMwWkY0ichOAz4vIX0TkaQDnA3gvACilngFwL4BnAdwP4F1cERgX/15t0MA8lW4KnlOEHs4FqwGrqOUscrDxevU8v1oi6K08u2V/4djEgoCutsbYaQRhr/8XHt+DCSOaAQxtvD6tqzXxMljDuthTgWw2gXAqsDwEultQSl3tEvxNn/ifAfCZOIUi7riN+Lx8L5nOB/BzEFr8O+6oqBJyhfOeTQk3bs8ybXLTSRM68Zu1O9DT3lTpolQc33eTsvdmAvstfeO6hYXjS08ej0tPHg8A2HukTzsN33iGn1/a2lE1wKnA8kDP61VO/6CBHSk1fNUA+h1ZEqsCk8ISoJz3luQtBGqsksvalVsvnIWLThiLEyd0ljnn9GHKTudNdz5iIJXk5xNNDCBMCTjh9zN0DobMlKOc/PH2C4xokXQxorGqwudcbihYpRifHR6QqRMMDCrXPbSioNOFe08FFpOEYJWUryVx/HWGJ0Ha/Fhl6oRClQa1ahxtqum5PT17XQ/7dIfD2xifn3otFyZsrEgw1AumnIPH+l3DLeHFxFRgiVG8pluFofDiE9WksbJw3oOxqUCXZDjiSy+18m4+99cnYVxnU8U35Y2TP02swmNEQ8knHQgFqxRiHyFe9q9/LBzb20S2IFiZ0Vjp4WV7Vfw7rGB13OiWqAWKTcF41yM8CWi8nl78Hn35PeLHu/4zl53oee5NCyfjTx9eGi+DPNpb2rgOMqI/1KQGQ4TEhYJVlWIJL/0mNFYlW9q4x/PTWN1949DORWFHNF976/xQ8U3i6U0+wU6a9qO1xVV3/inSdVaz/OZ1C8wVJmXE0lgZKwUJA+XXYNjFVynW5pq9BjRWcd0tAEBP+9DS8LAzgY0ujuxK8k7YQWhS0wpu6VSTO4paw39Lm2g8sl53q1V3RrWG23DcolxTNvrVuTRincjQwC5kcUs0VuEuJyQxKFhVKdbqDhMaKyeeRuo+PWiRsBCzg0yaz7/xZFy9aHLFysApi+rgk6+bW/S7Uu8tar5pq2au5RHgG9ctwAVzekq2XwmiCk05SY1AwapKsfyRmLCxUjruleEvLxXLVeF6vHL3j1cumITPXn5SSd72ezClVXJLhh+E9GJ/X9efPbVyBcHQStio1UXnujefnhtg6GiNvThrelfkawFgycxu3HX9wvhb2rBdkZRAwaqKsPcblsYqCeN1rxGyn21Qnbgf6+UXLr5R8nk7y1xJ43VSOXwHD2UrRQ5rqBO1vuhcdvslc1w369Vl7aeX4azpo/XKoxmmi58m74r5E2OkTPxg9xUMBasU88vVWz3PWf5IzNhY6U0n+tqf2FpbNU51ldNegxorEoaozUkg+Pkti/GHD57vk7b7Zr26RBXIhvKPcW3Jb3sfFD3d4c63blgYHInEgg5CqxRrVaCJvQK18ems6iR6p6YzdZicg1CvVYFm0x/b0YSt+4/m02avn1bS5G4hNgJDjl/9257uc3Gr91EM7BsydegdGAxsRx++ZA5OmkjHt05Omzwy1vX0YxUMBasqZP/Rob27TMgbJe4WPOL5NaeqnQq0yuD8bbhQg7aHTI1VmknPy7GqTHSNVXmI86GNdG+a0/dvP3d6pDINd+LY0xE9+ISrjIee24aTP/kAnt92EICZ3cS03S349ILFnav5Lj0pLY81DZqYuwWx8hki7vJ7UhkqNVKPmq85Wz5DCzlCxh/X2YQH33uOZzrUnESjIRPvs5+GgXDaoWBVTYjghe0Hi4JMzZDZk/F0BOpzfdGqwGrSWBVu3GFjZbhQCc1kEsP4vvYy11Nr+juqQ9kk29WFx4/Box9Zmmg+Avd2aAmM/MBHo44q88ThVGCV0d3WWPRb1/DcF82vvl9HZm+sYUfKlbQ5KvgmTEhjVZoTIeGovMaqtO6O62zCmI6mUKmEH3CJ6/ZYXk59SXko53O/+8ZF6B8YxPYDx3Dq5BHlyzgmFKyqnCQ0IZ4G3T4dvL3/EwDTuluxfschzfwqR8GOxRFuetuZM6d34b+e2mw20Zi847zpePwlTkva0dXKlgMvoV+XcpVXN5soAmLG1eDdPT0KWsOPc2d1V7oIkeBUYJXh7DxMrJbTt7HyPudcFfjr95+nnX8lO0RvGyuzhepqa8Dx4zqMphmXDy2bg/94x1mVLkaqcGpPFxyXW0E1a0x72csykF/x67WpeXM2nquDOBQ9pgTbr9/GzSXG67S5Kgt8zsFQY1VF5GwOisNMKazs8lmYTZj/7ZrcBsr2U6GnAivYUIc0VsmMfov8e5lJkpSRy0+bgDOmj8aEEc04YFuNWw4O9w4AALpaGwNi5pg3sRNPbdxX+J2kI9pIi/ki2F66C1bWX7Yokk6osaoiFEoFAFPuFnRstdwEoPamnGwep5NLQ/9YjjKk4T6JP25uNyaMaC57Oe6+cVHhuKNZb/zrtHlKsr4VDxjCZ/TjdwZrSnOCVWnalrsAEeC+d5yJn+TTYvvS57s3nY63nD45OKILfM7BULCqMko1VvElq+e27NcS0FxHjyhVy4d3EBqM25TnTYun4uOXznWJrU/BjsVZJq4KrEn8HYSW54syo6cN587qxlfefCqumD/RM19727/j8pPg9BVcrq2TomQzcWSwsCoeIpvl7X1gUGH+caNwakyHl7XI4pldmDO2/NPbtQIFqyrCrZMx8cH+weOvaolnbvlbnaq9E7eOO5o0Z5oj9v9/d+lc3Lh4arSL83Q2ZwEAb1wwyUSRSigyReFIr6op1+uzBimXnjweX7hintY1y08eVzL4MFXerrbSqci4ddkSmfz6LxF34bC5ISdYHesv3s6Lzas8pOU5Tx7VAiDXTtIGBasqp6w72vit0HHRWN1/W6lzP9d0NZpqUtqCtsZ6rP30Mrz3wpmO/PSuD1MsGn2mnzS8oyiaJhEp8u6fCzNTnhEtDfjV+84tTtu+L59mOmF93bnZlAJDRvtH+wY0cybDkbGdTVjzqWW49szjKl2UEmi8XkW42RwY8WOF6MbrVnncbC7qNR3RVVqT47aRbLmmUUj1UDb3BZoZFbVZlC5kMTkYcW6DYm5xh985dzHXEqxKNFZss2UhTY+5qYIrY/0I1FiJyF0isl1EVtvCviAia0TkaRH5iYiMyIdPEZEjIvJk/t/XEix7zeGqNjfleV1jTtF/KrA0THcom6J2WkB7FB50PobtGSk/aXhHQUWwVuICQEt+WmxQKe09P01QPMWtO4Aq1XL5TgU6M8rT1ECNFUk3OlOB3wawzBH2IIATlVInA3gewIdt59YppU7J/7vZTDFrh8FBhUO9/Z7nnf2MqZlAy2fOO8+b7u0g1NV4PUexjZV1LnyHC+S2y6g0pj+wJvyNkcpSrmlCv7q3aOoonDShs/D7fa+ZBSCndXVOBZrUupb4eYs5YNASxsT9mV8xfyKAIR9jtujEAN3tQzZ1K86ZVsGSVC+BgpVS6vcAdjvCHlBKWV//RwBMTKBsNck//up5fPA/nnY959YXmfpgDyiFpmwdPrhsjmccVxurgrO+ojFs/pxe3vZodQLMmzRC78JEMdNN21NxfvhIdWNa+P7Zu88G4C8Q3fv2M4s29n7bkmnYcMdyNNTXlWqsEpQ0oky7lfYQwfHdsjlz+mhsuGM5pnW3hS4D8SdTJ7j53OlFv0l4TBiv3wjgl7bfU0XkzyLyOxFZYiD9muInT27yPe/8NH/9Dy9h2/6jsfMdHFSF0aGnjZVbmBT/LQrTzDsN0y9OkuhPBgaD45DK4m/z4/htOG9rIUpQe/DSnCVlvA5oTNlpEEmzFSK/NPYj1UadFA/W3exkOT4MJpbxuoh8FEA/gO/lg7YAmKyU2iUi8wH8VEROUErtd7l2BYAVADB5cjRHZbWIm4bqP1ZtjJ3uoNLo0DVXBVaT53UvjPuxAjAwSMkq7YR573UuK/HiYLXtoDJ4nS7VWCXXroptrCJcrzMTKOJ6D179C43X4+N8htRYRSOyxkpErgNwKYC3qHyPoJQ6ppTalT9eBWAdgFlu1yul7lRKLVBKLejurs6NFocT9g+E54jQLcxlKlAc5wLRiFZuGyX9UXjQR3Do/N9e7D3NCuRsy65elL5BxmWnTsDyk8ZVuhipYmRL1thefS0NGSyZ2TWksQqIXzjvaBIlGisThQssRJhLSlcOB2XhpyX345ozpmiXq1Y5ZdLIkjBBsYCuu7KbFBNJYyUiywB8CMC5SqnDtvBuALuVUgMiMg3ATADrjZSUeKpgTQgdg0qFWuFWCMv/dW7CbD8XRBoHmgZlQgC5d/eauf5G+d+4boFmauXly286pdJFKBt+79NeJ/788Ysw7+8fMJLnxy+di6sWTcaql3eX5GOn4PIgFRqrsBZTEfIQd+2UjlB20sTOwDi1zkkTO/GW0yfje4++UggTKXbhU5+hq8soBApWInIPgPMAdInIRgCfQG4VYCOAB/ON95H8CsBzAPwfEekHMADgZqXUbteEiSt+MlKSOpuBQRWsfXHp0NzcLVidoUnhJMxHYlRrg3bcpEmj0EjMYOrdDuQbvdX23YSJpz5+EeoKclXuvNOHnfO3SWWD36rASGloLQqUcEZWJDRuK0mpsYpPoGCllLraJfibHnHvA3Bf3ELVAmu3HsAf1+3EDWdPDXWdm+D1zYdfil2egcHgKTx3jZUlREUbWSbBfe8I3uA1iKQEoruuX4DJo1qTSZzExtd4PaH6bLk68ZsK7GzJFo6tb52zL3DuwmCyvH4+suIIWX4OjnMOkb2vJfEZGHQRrGy/KVhFg57XK8Ql//R7DCqEE6yUcu2I9hzui10epVSkkaCbcWNYB6GmGdfZFOm6JIp76cnj8cPHXy34g7lgTuV9dBFvLAEimylf5e0fsDRWlvG6f3yvgU+p9iF+2bzLYDvWvcbjWPeasNeSYJwrlUWKheiMy1QgFwUGwwnUClHOPf50GNAxXpfivwDQYNvqYmR+VB16KjBlQ9CPvvZ47bhBRR/V2oBfvGcJJuU3DCXpxvJkftuFpWtukqqmJRqrwCn5HM4upKRPSVKwshuiG3owluNPe7phVgWS8Jw/p3jhmKBYi3hqKnwKVh8UrCqM0/A8yMYqqcVxA4PBxutuHZp9DzGraGGN103DfpdEZURLA579PxfjnedNLzmXVLUasrHK/Q3SNHnW75JVgQkarxfZVOpeZL++9KK/vXg21nxqGf79xkWF6FFXBRI9Lj15fNHv2WPbi6rRiRM6seZTy3D8uI4yl6y6oWBVYdLibE2poc7O00GoS3hDfWkV8rO7ck1Xr4hlxe+9BK3uI9VNS0N9WbWolsbKqnJBGhlvB6HFvxOdCizKR7ed27RcbudF0JTNoLkhYwvzT4eY5RvXLiwJa8pmuCVXSChYxeSP63Zix4Fjka8fCFFhlUpYYxU0Us53aJNt01oNtjn4oVVNVvzKYKLj9XvOF8zpMZoXqQ6SEraGpgL1bKysKuf82JV6Xi+PkVUUJ5J6DkL9VyIT83S2ZAOFKMpYwVCwismbv/4orvjaHyNfH8Zzs98KmrgM2PxYBa0KtI9QG7N2wUrTw2EEyj1iYt9BgjBVzQsaq0LzCdBYeZx2NpFyaax0Bauw7le8VgWSZEmb/W81QsHKABt2HQ6O5EFYeSGpOq+Ujh+r4r+AQ2NVOB/OeL0+5gqsOWPbi36b6Iz9BDn29bVJYjZWjqnAYM2xO0nuFVhShiIbqwgaKy3P6/4DPGKWDXcsBwBM7Sp1B0MtVTgoWFWYUBorlZzmxunPxA03NwpFnnlLpgL1esDG+gzuzhuseuftnda9N5+JB997jlZeuvg9jbCODgnxo79kKjBIY5U776yjrY1O7zkJGq/b0tbWWLmE+W7uLO5CG1cFJsulJ5duX5XkbMlwhH6sKoTlL8Qp0FTKSHBAw42V7kyfnxH8T991Nl7YdqAk/HiH1ikMHU1ZdDQNOVA00e36dviUpmoSv+/5vW8/E79dux3/+tt1odMtbM7tGJh4lsMj/F/efBre/t1VeOrVvVrpmCJjWNCxryr220aLmOGrbzkNnc22/lME37phIQYGhjpBaqzCQY1VhbBGXWHmsxWSmwoc1DBeH5qqEHzktXNKVMZDU4HenDJpBK5YMClqMcuGr4DLnr0m8dMkLZo6Cm84ZUKkdC0njfMmjYAI8PZzSl09FJfDPXxsZxPeZXMTkehegfapQM2viOvuDH5FFPchTNr83lU7l5w0DmfN6CoKO392Dy7k6ufIUGNVIeokt5liWpax5qYh9FX6K86ZjhUeHwA3R6JaiRrCRMer+1bYxdceF5/g/sGJul+tpbEa1dqAlz67PDB+Ya/AgEqaNuN1N9zvQQr/d98qi5QbVXScjm9WmqFgFYM4QlGuw1Batk1D+SExldWgCtZYBaEcNiLVPGWmq7Di4Lm2eOwjSwv79jk/+lFtf9pt09g66K+uK4/GSncqMGy7yblb8M+blIe0KACqBQpWMYhT16y+wSlX+SWp4L5XoAkeWb87ME7Q/ZZ4Xq9QB2jExsp3c9ihHKpZeCTh6enw3ocyimD1txfPxk2Lp8YpUhFFdTPRVYFDidcZV43Zttdys7GiZEVSDgWrCmF1wtU0ErCEDd3l4OFmAtPVWfopEqmxIm5EmRK75szj0JTNBEe0ob8HZ+jiREJbY2WL1lhfh8UzunDTklKh0r5IhkIUqUYoWMUgjkhk9RehnLEl6HldK/sAB4ZDntetVYHmOsUwAmjifqxs6bPbJ3GIUn90ByHlGqzoaqycGzd/922nu8azL5Ih6aDIxqp6dAEVg6sCYxBH22QJH84tbbbsOxqrTElSEKw8+jtdjVa1M9zvj0TD/JSYO7r1L0kj46BNmO+/bUlJWCbvCHh8p/d0KqDv1oWUj2ldbZUuQlVBwSoGRjRWYYzXY+YZF92OWmyrenQJnF4MIc3EHekqKO1RGUfVxCKKXBWl/uhekaRmIchBaNZliWRbYz3+5c2nBToDHloEE7OQxBhfftO8wjHfSzAUrCqEVTfDdH4PPLM1HVOBHi3LWbYoDTAtWmZf43X7lEY5CkNSzTvzvqOiTL1Fmgr0aVj2M4kKVkUaq6EfdkeTbiw/eRxmjvF3BuzcGisMty6diR/dfGbo64g/7U1ZzM6/N04FBkMbqxjEqWDWtEGYLW027DqMP63fFT1TQ3h1d6WrAqtX7NB+LdV7i8QQbz3jOADl83SurbFKcirQdmzXWI1ubcC+I33mM9Hkva+ZZSZvQmJAjVUM4nRcXjZWQew4UDkbrCAbqzhEWUmYJEF7mBWOky8KSTlue2iGvjaBa0xqFloailcuRvFjpQttrEi1Q8GqQgxNBYbr/foGKqeHtbaucHayFm0lm8CWl8Z6c9VZW2FVxVo5Eg/nm4/ixyrS9GE+n6asf3032VOMbmvEz29ZjLecPjlXBlu5TRvt18oimGqD70MfTgXGIJaD0Ah7BQJAn7WxWAWYO64D73/NLM+9/n5085n4zZrtof3yAGYElJ/fshgPv7gzdjqAvsDLzoZEWawRl0+8bi4WO/Z3cxLGzECHEyd0orE+17YTrfe04Uk1tLEKhoJVhbAGefYtbV7dfTjwut7+yglWIoJbls70PD+9uw3Tuyu3LHfmmPZAw1hdtFcFGsmNDAciaawiVqAbzg721l6tH8A4xuuEpAFOBVaIIQehuW7kkfW7sOTzvwm8rpIaq0pSDg/19o8cNxolYYm6V6BJiotgvg4PTdOV3utZ00ebySNBW05CykGgYCUid4nIdhFZbQsbJSIPisgL+b8jbec+LCIvishaEbk4qYKXk6N9A5j9sV/il3/ZUhQea1VgYUub3O/ntx3Quq6SNlZJkrY+lH6siC5xjNeTJImxiJdh+WMfWYq7rl9oJg/aWKWS7vZGAEBDPV9MEDoaq28DWOYIux3AQ0qpmQAeyv+GiMwFcBWAE/LX/KuIhDe4SRmb9h7Bsf5BfOF/1haFx9FqWFXT0ljpVtVa1ViVW4ChtwUSlmgOQs2XwyKs/WYYnOXu6WiKZFvpRtDWWaQy/NNVp+Lzf30yZvSYMbcYzgQKVkqp3wPY7Qh+A4C788d3A/grW/gPlFLHlFIvAXgRgL+b3RrFEhQGQvZ+/Un2lqSAvsYq2XKQ9FNQWEWoDKanD+3JnTSh02jagM0ruvGUbXnk/7JtpYtRrQ24cqH7wiVSTFQbqzFKqS0AkP/bkw+fAOBVW7yN+bASRGSFiKwUkZU7duyIWIzy4vzWxnMQmvsbVk4arn1N2jpR/RVVKSs4KRvOOhtFY1WfkFfR82d3o9nDLUocOKwjJBjTxutuvYRrW1RK3amUWqCUWtDd3W24GOUh1l6BcHheT5tkUeP4vVu7ZoKvjVjE8UlVLYzNb6Dclbe3SYKhvQKr69kQYhHV3cI2ERmnlNoiIuMAbM+HbwRg1xVOBLA5TgHThMlmbvUZ1bok2jSps6fQ9WOVcDFIFVDYwqmyxbCTVLeyYsk0TB3dimUnjk0oB7u7BUKqk6gaq58BuC5/fB2A/7SFXyUijSIyFcBMAI/FK2J6KJ0KjNZ9bT9wFC/vOlyUhm4nkqbOezhDeZeEpRbaZn2mDpecNC5ZbZJH4/v4pXPxgYu4FyBJP4EaKxG5B8B5ALpEZCOATwC4A8C9InITgFcAXAEASqlnROReAM8C6AfwLqXUQEJlrzhRP75/9ZX/jZxG6jQ7wxTfvQJtx2nwXUTSAeuCGbzcLZw9owuzx3JFGkk/gYKVUupqj1NLPeJ/BsBn4hRquLN539BGypwKzJOybxK3tCFhSYNgVemB1ymTRmD9zkOx9g3lJsyk2qHndQ28Grj92/vZ/34uUtoKCr97fgc+9tPVwZFBj+Dlgk+ZhIWCAPAPl5+E/3r3YozpaIqcxtB6nuInmgK5lRAtKFjFwfb1/bffr4+cxju/u0o7Ot1YlQfuFUiC4YffSVM2g5MmxvOf5WW8zsdLqgUKVjEwoT0Km8LgMJWs0vZR0vVjxSXhxIJ1wQxD7haKw/l4SbVAwarMHDrWX/Rb3xFltPheLJo6ykg6tQg7eFKEZpNsrGd3Gw7qrEh1wpYegygyjlOwCpuGXWF1xfyJ4QsAYGxHEz7xurmRrk2KtHWZusa3FLKILuXUNVfzohivLW3Y1ki1QMEqBib6rjhTgdmII+DxI5qQzfDV+3HFgkn46GuPD4zHzp548fVrFxQHlEPYGQb1kasCSbXDr2tE1mzdX6J90sHNyeihXn1XX/apwKijUgV2Wn4oBWTqBNefPSUwbqWXt5PKESRUv2bumKLf/3jVKUW/b9CoX2FZOGUU5oxtx99ePNt42uXDvWNjSyPVQnRnIzWIZVTZ2z+IZf/4B5wwviNCGsW/71+9NdT19qnAqJ7fgfRpWtJo+Ju+EpFqZcKIZpwzq3hP1I8tNz8d39ZYj/tvO8d4uuXEa/vUNPYRhLhBjVUELK3RM5v3R77WYuOeI6GuHzCgscpRvk6qoUqnHb06cruWqo59PYnA4x+9EBlWHleG3C043FmUvyiERIIaqxAkMWJqCGknZddSVYOz0B/dfCYmjGiudDEiofO2OYomOogU16eWhkzFypJ2vDRWafBsT4gOFKxCEGfqrZCG43c2E66zsE8FRnVppVTwVOBtF87EoinxXTIs1EwjjV0m+3FiCq5w0+fCuT3469MmltiJ1VWn4pvUIBSsNHBqJY71D0ZOy+ngs6E+3MjVhPE6ECzI3HZhunaRn5PffPXKBZPKlqfnVKAtmN9HotMMBUJhSpPG+gy+eOW8knBOnZJqgYJVSHr7BzHv7x8wll5YjZVdmNKdCmxvqseBo8UrGNM2hRVUnDEdTdhwx/LyFCYM6XqMpIz4vXpnXS3RWLHihCaTsj6LEC+oXA3J0X591whuOLVMsQy7NeSqX73vXPz2A+eVXMYuygx8jkQHAYWpuNRRY0WqBGqsysyAQ7J68NltkdPS2d5mRk+bazgHf2ZIm+aPpBOn4TWrTXiosSLVAjVWIYlrv+4UhnYd6o2cVn1UbZdSqRs9p6E8ukKSeBwT4gflgnhQY0WqBQpWYYkpWJlYWWgxuq0h8rXs5M3A50hIeaDxOqkWKFiFwIRIFNVFghuTR7WYS4wQkhjOZk+BPDycCiTVAgWrMjNgSLK6/ZI5uGrh5MjXp62PSlt5dEnDFCapLAaV0MQH+rEi1QKraghMfEJ1DM51uOHsKZFV4/wOxMP+/KpVICTxCfPulVIO/2esOGGhxopUCxSsQmBCIDE1uo3bMVuG2t3tjZg3sdNEkWqGpiybDSHlhjZWpFrgF8KD99zzZ5z4if8BYHZ/PlNTgXEHb9blue1t2GHp8pdPXoTO5mzhN59d7TKtK+fKRNfJr30wxGoTHrY1Ui3Qj5UHP3tqc+F4aLf1+MbnpqYC41I0LcH+Spv2pmxwJFITfO2t8/HEq3swuq2xEHbfO85CT3tjSdwS4/WEy0YIqRyRBSsRmQ3gh7agaQA+DmAEgL8BsCMf/hGl1H9HzSdNKMR3l2BqVWCcjlmp4tGz5bzw9fPGxyxVdNIg3L3zvOnYuu8o3ny6/qIAzk7ULp0tWZw/u6cobP5xIytUGkJIWogsWCml1gI4BQBEJANgE4CfALgBwJeVUv/XRAHTQPH+fHHTSp/G6tCx3D6CN5w9pTKFSQkjWhrw/64+NdQ1aRAISfpxNntOaxEyfDFlY7UUwDql1MuG0kstceUiczZWMY3XC0cKx43O+cOaNaY9VprDnfGdTZUuAqli4tpnEkKqA1M2VlcBuMf2+90ici2AlQDer5TaYyifCmHOeD0NU4HOBL5wxTzccsFhtDZWzuQu7cvPH3jvOejO29LYhevmbKZCJSLVRpHxegXLQQhJltgaKxFpAPB6AD/KB30VwHTkpgm3APiix3UrRGSliKzcsWOHW5TUUDQVmKItbfz4/ttOxz9ddYp7GVC8V2BHUxYnTqDLBT9mjWnHyNbSLYT++erTKlAaUo001A91t5wJJGT4YmIq8BIATyiltgGAUmqbUmpAKTUI4OsAFrldpJS6Uym1QCm1oLu720AxykP8TZjNlCOoYz5rRhfecMoE13NKpW9aoho/NKdNHoGxnB4kPlx+mnsbJIQMX0wIVlfDNg0oIuNs5y4DsNpAHhVFFR3H9GOVEuN1Eh8aIJMgbl06E0Bpv8G6Q8jwJZZRjYi0AHgNgLfbgj8vIqcgJ49scJyrapQyobFKi/E6O3ZCkobtjJDaI5ZgpZQ6DGC0I+yaWCVKIbruFjbvPYL/emozVpwzzVPwSYu7hbRRTZ8fvkFCysf9ty3Bxt1HKl0MQrThljYaWGp8EWDQx0jqbXevxGd/uQYb93h3Akf7BiOVYeLI5kjXuWHSGL+WqSZhkFQWq5197a2nYdHUUZUtTJUxZ2wHLpw7ptLFIEQbClYhCBJCDuYdbfpN9+0/0hcp7y+8cR5uPHtqpGvdoIkHIcnjbGfLThyHe99+ZmUKQwgpCxSsNDCp4dkXUbACgHrNzV6DSKOSisa8hBBChgMUrEISd1VgHMEqw43pCKlKOOVOSO1AwUoDXY3VK7sPl4St3rQPG/cMhUe1sQKA+gQEK/b3hBBCiDkoWGlg11KFdZdw6T8/jMWf+03k6+3U1/m/rtfNG6+dFnVf0aH2gRBCiBcUrEKi8011813zyq6c1iqOu4UgGytdoU2ptPldHxL05h83sqLlIMQkNB0kpPao3K67VYQJ4/Xntx3A5NEtsba0CbSxSpu0FIK6OsHPb1mM40a3VLoohBBCSGSosQrg3sdfLRzn9DzRpJfvPfoylFKxtrQJsrEy5dW9Upw4oRPtTdlKF0MbaiMIIYQ4oWBlY8u+IyVTdR+872k8vmF34beO7OI20fabtTvw8Is7I08FKqhAjVWYpDubsxABbl82J1J5apnZY9oBADefO73CJSGEEJI2OBWY5+mNe/H6r/wvPnv5Sbh60eSic4d7BwDkbKd0pvK84hzuHcBg9EWBqM/4y8EKCiJ6AlY2U4eXPrs8emFqmM6WLDbcwWdH9OFWVoTUDhSs8ry4/SAA4NH1u0oEKwuV/y8Irym5OpGYqwKDNVZPfvwi9A/4S2/s4wkpD3R8S0jtwanAPH79n320qSOUrN60zzW8ThBoY/U3S7y3rQmaChxUuSm+0W2NwYUkhBBCiHEoWGlgyUIC0RKsbv3Bk67hOtN0Z8/o8jwX7CCUqihC0ghbJiG1AwUrB34doO5UoBeiMRXo5wRUR2NFCEkPnc25Va6vP0XfeS8hpLqhjVUey6mnm9xjDwpjn/T3//WMI494wk+Q53VdA9n0uQclZHjS1liP1X9/MVqymUoXhRBSJihYaRDVQei3/ndD0e+P/XQ15ozt8L3Gz9YraCaQGivzPPjec2Lt70hIWyO7WUJqiZps8bsP9WLdjoNYOGVU6GvjaHs27jmCjXuORL4+aIER5SrzzMz7rCKEEEJ0qEkbq6vvfARXfO1PRWGW0OImnNiFKV2NVW9/NC2Hn+wUtHSbvnIIIYSQylKTgtXabQdCxS+aCtS8ZtbHfhkqj+BC+AtdgL7QR/mLEEIISYaaFKwsomh4EtcK+UhPQRorXeejjdmafu2EEEJIYtSkjZWFUqV2S26Ck7LFr6SBeJDxuo5c9Z6lM3HlgolmCkQIIYSQImpadWGXQ/S3nkhWshKHyuqMaUMG9sHG68Fle99rZmHiyJZIZSOEEEKIPzUtWLlNnbmKJvl4uhscJ4VT6HLiV7bJoyhMEUIIIUkTaypQRDYAOABgAEC/UmqBiIwC8EMAUwBsAHClUmpPvGImg10Q8RNZ7FOBSctVTq2UXZgK1Fj5FO6/b12CQ8f6Y5SMEEIIIUGY0Fidr5Q6RSm1IP/7dgAPKaVmAngo/zuVRPFJVVGNVZC7BZ/7aWusx5iOJtNFIoQQQoiNJKYC3wDg7vzx3QD+KoE8jPCXjftKjdXdtrSxhemuvIuKrx+rgGvpeZ0QQgipLHEFKwXgARFZJSIr8mFjlFJbACD/t8ftQhFZISIrRWTljh07YhYjGm/82p/w0yc35cuTC3PT+kRxEJoEdflCzpvY6XqeDkIJIYSQyhLX3cLZSqnNItID4EERWaN7oVLqTgB3AsCCBQsqJhG8uP1gYJxiB6GVE16CbKzGjWguT0EIIYQQ4kosjZVSanP+73YAPwGwCMA2ERkHAPm/2+MWMlUkLFd5Ja8QPBX4seXHGy4NIYQQQsIQWbASkVYRabeOAVwEYDWAnwG4Lh/tOgD/GbeQ5cBafec2m6Y8jpPAbzbPMl73ijKukxorQgghpJLEmQocA+An+Y99PYDvK6XuF5HHAdwrIjcBeAXAFfGLmRz9eYvv/kHvTZPLabzuN9VYsAOjKRUhhBCSSiILVkqp9QDmuYTvArA0TqHKyb/9bj2WnzQOt/7gSc84dqPwxIUaW/ojWrJFp3R9wxNCCCGkMtS053WLP7yws3DsJjgN2PwYmJCrGuqDH/vo1gb86n3nFoXV1VlTgVRZEUIIIWmEghVQ5JH8/me2lpy35Cql4rs0uOWCGfjlrUs8z1upzx7bjq62xqKVgNRYEUIIIekmrruFYcFBx1Yvz287UPT7cO/Q+aN93rZYOrz/otm+5y25zc21gvY+0YQQQgipCNRYATh4tFiwuujLvy/6/YPHXwWQE2x2HDiaaFmsaT73DZcpWRFCCCFphoIVgB//eZNWPKWAHQeOJVoWp8bKMmBvqK9DHVcFEkIIIamGglVIDvUOGEnne2873TXcKTN99rKT8YnXzcWC40YO+bGiYEUIIYSkEtpYhaR/IJ6NlcXZM7pcwy3jeEuI6mzJ4oazp+bCjORMCCGEkKSgxiokvQNJOwj1pi7A8zohhBBCKgsFq5CY0lgBwJTRLaWBlo2VS3yuCiSEEELSDQWrEOw93IsfrdpoLL17bz6zJKywKpBCFCGEEFJ11JRg9cj6XThwtC/y9aYM1y162ptKwpSGxiquk1JCCCGEJEPNGK/vPdyLq+58BOfM6q50UXwZcrdQKlq5+7YihBBCSFqoGY2V5TF9zZb9xtNeMnNohV9zNhMrLUsXRRsrQgghpPqoGcHKsl3anoCDz6++dX7hOK7wM+RuwS9OvDwIIYQQkgy1I1glKIy0NQ7NqJpTKrlMBVJjRQghhKSamhGsBsuk5nGzjfKPX/xbeYQTQgghJP3UjGBVLdNnfqsCC3HoIpQQQghJJTUjWCWlsfrMZScCAP7xTacACD8VWBrf28aKqwIJIYSQdFMzgtXAYDKCVVdbIwBgwsjmXEBI2cc5dTiksfJOqFq0b4QQQkitQcEqJtb+fZYYFFen5GdjVXAQGjMPQgghhCRD7QhWBtU8N5w9BWdMGwUAqMsLO1L46y1aTe1qDUx7yEFo6TlOBBJCCCHppmYEq/6B+ILVt25YiKc+cRE+tnxuIaygsSr89b7+/tuWlIRNcwhbYQ3T13xqWaj4hBBCCEmOmhGsok4FNmWHHlG2rg6dzVlk6qREs6SjTWqsL/XK/v2/OQN3Xb+g8FvHxqq4fPE8vRNCCCHEHJEFKxGZJCK/EZHnROQZEbk1H/5JEdkkIk/m/73WXHGjE3UqsL0pWzh2W1lYorEKmX53eyMumDOm8LuQg6/ndVpZEUIIIWkkzibM/QDer5R6QkTaAawSkQfz576slPq/8Ytnjqgaq/bGeuzIb4NjT8E6LjFej+nZs7Cljcs5L+P1yaNa0D8wGCtfQgghhMQnsmCllNoCYEv++ICIPAdggqmCmSayYNU09IjsGivLaH3QsbefKQNzdwHNPfXff/B8Q7kSQgghJA5GbKxEZAqAUwE8mg96t4g8LSJ3ichIE3nEJapg1WYTrOxTcNO72wAA2/YfBTCkuYqLlU424zcXaCQrQgghhBgmtmAlIm0A7gNwm1JqP4CvApgO4BTkNFpf9LhuhYisFJGVO3bsiFuMQPwEq5/fstjzXHujzcbKNtv2oUvm4PqzpmD5yeMAAA31uUfZNzCI7950Oj7/xpMjlXPZiWNx0+Kp+DvbykML7h9ICCGEpJtYgpWIZJETqr6nlPoxACiltimlBpRSgwC+DmCR27VKqTuVUguUUgu6u7vjFEOLI30DnudOnNDpeW7epBGFY/tUYEdTFp98/QloachptFoacqvzjvYNYvHMLly5YFKo8jXW1+EDF81CNlOHv7t0Lka2NoS6nhBCCCGVJ7KNleSMgL4J4Dml1Jds4ePy9lcAcBmA1fGKaIaHntsWKv4Fc3pw1/UL8aOVrxbC/Gbg2hpzj7I3ohH52k9foh2XM4GEEEJIOomzKvBsANcA+IuIPJkP+wiAq0XkFOS+/xsAvD1GHkbYd7gP967cGOoaa9bNbjvl5+bA0lwlCWcCCSGEkHQTZ1Xgw3D/1v939OIkQ8bPENwDa1Vepm7oWj/7d8vGys7X3jofE0c249J/fjh0/n7QjxUhhBCSTpJXs6QA3xV2HliKKrvAFCTPnDltNP7q1PGF38tOHAsAeO1JY3H82I7QZXAyYWQzpnW14hOvPyF2WoQQQggxT20IVnXhbfQtRZVllA4A03v8N1G+Z8UZruH/+pb5ofN3o7E+g19/4DwjaRFCCCHEPDWxV2Bdnb7G6upFk4t+t+aN0sd0NGKOAa0TIYQQQoYvNSFYhWFcZxOAoWm/5vwmxyNb6P6AEEIIIf7UxFSgDjctnorFM7rw7Jb9AIZcGtApJyGEEEJ0oWCV5+8uzXk6f27r/qLwCSOaAQBvPn1yyTWEEEIIIXYoWDmw/FZZU4EjWhqw4Y7lFSwRIYQQQqqFmrSx+tuLZ3ueC2HnTgghhBBSRM0JVt+4dgFuWjzV8/yQp3U64SSEEEJIOGpuKvDCuWN8PZeLYyqQEEIIIUSXmtNYATnh6fzZ3QBy3tLtWE7aKVcRQgghJCw1o7F67CNLkc0MyZFffet8PL5hNxYcN6ooXhhnooQQQgghdmpGsOrpaCr63ZTNYMnM7pJ41lTgjgPHylIuQgghhAwfanIq0I9pXbn9AP+yaV+FS0IIIYSQaoOClYOzZ3QBQMEGixBCCCFEl5qZCgzD05+8CE31mUoXgxBCCCFVBgUrFzqaspUuAiGEEEKqEE4FEkIIIYQYgoIVIYQQQoghKFgRQgghhBiCghUhhBBCiCEoWBFCCCGEGIKCFSGEEEKIIShYEUIIIYQYgoIVIYQQQoghKFgRQgghhBiCghUhhBBCiCFEKVXpMkBEdgB4uQxZdQHYWYZ80kgt3ztQ2/fPe69davn+a/negdq+/3Lc+3FKqW63E6kQrMqFiKxUSi2odDkqQS3fO1Db9897r817B2r7/mv53oHavv9K3zunAgkhhBBCDEHBihBCCCHEELUmWN1Z6QJUkFq+d6C275/3XrvU8v3X8r0DtX3/Fb33mrKxIoQQQghJklrTWBFCCCGEJEZNCFYiskxE1orIiyJye6XLYxoRmSQivxGR50TkGRG5NR/+SRHZJCJP5v+91nbNh/PPY62IXFy50ptBRDaIyF/y97kyHzZKRB4UkRfyf0fa4g+L+xeR2bb3+6SI7BeR24bzuxeRu0Rku4istoWFftciMj9fZ14Ukf8nIlLuewmLx71/QUTWiMjTIvITERmRD58iIkdsdeBrtmuq7t4Bz/sPXder8f497v2HtvveICJP5sOH1bv3+cals90rpYb1PwAZAOsATAPQAOApAHMrXS7D9zgOwGn543YAzwOYC+CTAD7gEn9u/jk0Apiafz6ZSt9HzGewAUCXI+zzAG7PH98O4HPD9f7z95UBsBXAccP53QM4B8BpAFbHedcAHgNwJgAB8EsAl1T63iLe+0UA6vPHn7Pd+xR7PEc6VXfvPvcfuq5X4/273bvj/BcBfHw4vnt4f+NS2e5rQWO1CMCLSqn1SqleAD8A8IYKl8koSqktSqkn8scHADwHYILPJW8A8AOl1DGl1EsAXkTuOQ033gDg7vzx3QD+yhY+HO9/KYB1Sik/Z7tVf+9Kqd8D2O0IDvWuRWQcgA6l1J9Urrf9d9s1qcXt3pVSDyil+vM/HwEw0S+Nar13wPPdezHs371FXutyJYB7/NKo4nv3+salst3XgmA1AcCrtt8b4S90VDUiMgXAqQAezQe9Oz9FcJdNTTocn4kC8ICIrBKRFfmwMUqpLUCuYQLoyYcPx/sHgKtQ3LHWyrsHwr/rCfljZ3i1cyNyo3CLqSLyZxH5nYgsyYcNx3sPU9eH4/0vAbBNKfWCLWxYvnvHNy6V7b4WBCu3+dNhuRRSRNoA3AfgNqXUfgBfBTAdwCkAtiCnKgaG5zM5Wyl1GoBLALxLRM7xiTvs7l9EGgC8HsCP8kG19O798LrfYfccROSjAPoBfC8ftAXAZKXUqQDeB+D7ItKB4XfvYev6cLt/ALgaxYOqYfnuXb5xnlFdwsr27mtBsNoIYJLt90QAmytUlsQQkSxyFe57SqkfA4BSaptSakApNQjg6xia8hl2z0QptTn/dzuAnyB3r9vyql9LBb49H33Y3T9yAuUTSqltQG29+zxh3/VGFE+ZVfVzEJHrAFwK4C35KQ7kp0F25Y9XIWdnMgvD7N4j1PVhdf8iUg/gcgA/tMKG47t3+8Yhpe2+FgSrxwHMFJGp+VH9VQB+VuEyGSU/v/5NAM8ppb5kCx9ni3YZAGs1yc8AXCUijSIyFcBM5Az6qhIRaRWRdusYOWPe1cjd53X5aNcB+M/88bC6/zxFI9Zaefc2Qr3r/LTBARE5I99+rrVdU1WIyDIAHwLweqXUYVt4t4hk8sfTkLv39cPp3oHwdX243T+ACwGsUUoVpriG27v3+sYhre3etDV8Gv8BeC1yqwjWAfhopcuTwP0tRk6d+TSAJ/P/XgvgOwD+kg//GYBxtms+mn8ea1EFq0IC7n8acitAngLwjPWOAYwG8BCAF/J/Rw3T+28BsAtApy1s2L575ATILQD6kBuB3hTlXQNYgNxHeB2AryDvMDnN/zzu/UXk7Emstv+1fNy/zreHpwA8AeB11XzvPvcfuq5X4/273Xs+/NsAbnbEHVbvHt7fuFS2e3peJ4QQQggxRC1MBRJCCCGElAUKVoQQQgghhqBgRQghhBBiCApWhBBCCCGGoGBFCCGEEGIIClaEEEIIIYagYEUIIYQQYggKVoQQQgghhvj/+pv62yGQBwsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import gym\n",
    "\n",
    "GLOBAL_SEED = 2021\n",
    "np.random.seed(GLOBAL_SEED)\n",
    "env = gym.make(\"CartPole-v0\")\n",
    "\n",
    "episode_rewards, policy = train(\n",
    "    env,\n",
    "    params=np.random.rand(4),\n",
    "    lr=0.002,\n",
    "    gamma=0.99,\n",
    "    agent_cls=LogisticAgent,\n",
    "    MAX_EPISODES=2000,\n",
    "    seed=GLOBAL_SEED,\n",
    "    plot_stats=['score'],\n",
    "    plot_period=100,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "subtle-holiday",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}