tutorials/pipeline_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial for FedLab users\n",
    "\n",
    "This is a comprehensive tutorial for users who would like to know FedLab. FedLab is built on the top of [torch.distributed](torch.distributed) modules and provides the necessary modules for FL simulation, including communication, compression, model optimization, data partition, and other functional modules. FedLab users can build FL simulation environment with custom modules like playing with LEGO bricks. \n",
    "\n",
    "In this tutorial, we will further describe the architecture of FedLab and its usage. To put it simply, we introduce FedLab by implementing a vanilla federated learning algorithm FedAvg in this file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"../\")\n",
    "\n",
    "# configuration\n",
    "from munch import Munch\n",
    "from fedlab.models.mlp import MLP\n",
    "\n",
    "model = MLP(784, 10)\n",
    "args = Munch\n",
    "\n",
    "args.total_client = 100\n",
    "args.alpha = 0.5\n",
    "args.seed = 42\n",
    "args.preprocess = True\n",
    "args.cuda = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Prepare your dataset\n",
    "\n",
    "FedLab provide necessary module for uses to patition their datasets. Additionally, various implementation of datasets partition for federated learning are also availiable at the [URL](https://github.com/SMILELab-FL/FedLab/tree/master/fedlab/dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We provide a example usage of patitioned MNIST dataset\n",
    "# Download raw MNIST dataset and partition them according to given configuration\n",
    "\n",
    "from torchvision import transforms\n",
    "from fedlab.contrib.dataset.partitioned_mnist import PartitionedMNIST\n",
    "\n",
    "fed_mnist = PartitionedMNIST(root=\"../datasets/mnist/\",\n",
    "                         path=\"../datasets/mnist/fedmnist/\",\n",
    "                         num_clients=args.total_client,\n",
    "                         partition=\"noniid-labeldir\",\n",
    "                         dir_alpha=args.alpha,\n",
    "                         seed=args.seed,\n",
    "                         preprocess=args.preprocess,\n",
    "                         download=True,\n",
    "                         verbose=True,\n",
    "                         transform=transforms.Compose(\n",
    "                             [transforms.ToPILImage(), transforms.ToTensor()]))\n",
    "\n",
    "dataset = fed_mnist.get_dataset(0) # get the 0-th client's dataset\n",
    "dataloader = fed_mnist.get_dataloader(0, batch_size=128) # get the 0-th client's dataset loader with batch size 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Define client local training\n",
    "\n",
    "Client training procedure is implemented by class ClientTrainer in FedLab. We have built-in FedAvg implementation in FedLab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# client\n",
    "from fedlab.contrib.algorithm.basic_client import SGDSerialClientTrainer, SGDClientTrainer\n",
    "\n",
    "# local train configuration\n",
    "args.epochs = 5\n",
    "args.batch_size = 128\n",
    "args.lr = 0.1\n",
    "\n",
    "trainer = SGDSerialClientTrainer(model, args.total_client, cuda=args.cuda) # serial trainer\n",
    "# trainer = SGDClientTrainer(model, cuda=True) # single trainer\n",
    "\n",
    "trainer.setup_dataset(fed_mnist)\n",
    "trainer.setup_optim(args.epochs, args.batch_size, args.lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Define server global aggregation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# server\n",
    "from fedlab.contrib.algorithm.basic_server import SyncServerHandler\n",
    "\n",
    "# global configuration\n",
    "args.com_round = 10\n",
    "args.sample_ratio = 0.1\n",
    "\n",
    "handler = SyncServerHandler(model=model, global_round=args.com_round, sample_ratio=args.sample_ratio, cuda=args.cuda)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Choose simulation mode and run\n",
    "\n",
    "We provide three basic simulation mode in FedLab. Depending on the needs of users.\n",
    "\n",
    "1. Choose Standalone mode to run the simulation with lowest resourch allocation.\n",
    "2. Choose Cross-process mode to run the simulation with multi-machines or multi-gpus with faster calculation.\n",
    "3. Chosse Hierachical mode to run the simulation across computer clusters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standalone\n",
    "\n",
    "We provide an example pipeline implementation. Please see [URL](https://github.com/SMILELab-FL/FedLab/blob/master/fedlab/core/standalone.py).\n",
    "\n",
    "If you change the data partition paramters $\\alpha$, you could get following convergence curves, which reavels the Non-IID challenge in federated learning.\n",
    "\n",
    "![](./examples/imgs/non_iid_impacts_on_fedavg.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Round 0, Loss 21.8655, Test Accuracy 0.2819\n",
      "Round 1, Loss 16.8743, Test Accuracy 0.4256\n",
      "Round 2, Loss 11.4556, Test Accuracy 0.6736\n",
      "Round 3, Loss 9.7989, Test Accuracy 0.6655\n",
      "Round 4, Loss 8.1740, Test Accuracy 0.7111\n",
      "Round 5, Loss 7.3797, Test Accuracy 0.7431\n",
      "Round 6, Loss 6.5602, Test Accuracy 0.7709\n",
      "Round 7, Loss 5.8442, Test Accuracy 0.8077\n",
      "Round 8, Loss 4.9822, Test Accuracy 0.8531\n",
      "Round 9, Loss 4.6129, Test Accuracy 0.8608\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAAGbCAYAAAAm+0NQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABzIklEQVR4nO3deVhU1R8G8HdmgGEfQHZFEBUQVEBURE0xUVxy/VVmlqalZVYW2WKLllqUlVlpWubappZrm6m4r6iIKyKLCMiOwrDIADP39wcyRYKCAndmeD/Pc5+cO/feea/a8cuZc8+RCIIggIiIiIjIQEnFDkBERERE1JRY8BIRERGRQWPBS0REREQGjQUvERERERk0FrxEREREZNBY8BIRERGRQWPBS0REREQGjQUvERERERk0FrxEREREZNCMxA6gizQaDTIyMmBlZQWJRCJ2HCIyQIIgoKioCK6urpBKDa/vge0oETW1BrWjgog+/PBDoXv37oKlpaXg4OAgjBo1Srh06ZL2/fz8fOGFF14QvLy8BFNTU8HNzU148cUXhYKCgjted9KkSQKAGlt4eHi9c6Wlpd12Pjdu3Lg1xZaWlnbPbaguYzvKjRu35trq046K2sO7f/9+zJgxAz169EBlZSXeeustDB48GBcvXoSFhQUyMjKQkZGBTz/9FL6+vrh69Sqee+45ZGRk4Ndff73jtYcMGYLVq1drX8vl8nrnsrKyAgCkpaXB2tr63m6OiOgOlEol3NzctO2NoWE7SkRNrSHtqKgF744dO2q8XrNmDRwdHXHq1Cn069cPnTt3xqZNm7Tvt2/fHh988AGeeOIJVFZWwsio7vhyuRzOzs73lKv66zdra2s21ETUpAz16362o0TUXOrTjurUwLHCwkIAgJ2d3R2Psba2vmOxCwD79u2Do6MjvL29MX36dOTn59d5rEqlglKprLERERERkWHQmYJXo9Hg5ZdfRp8+fdC5c+daj8nLy8P8+fMxbdq0O15ryJAhWLduHaKiovDxxx9j//79GDp0KNRqda3HR0ZGQqFQaDc3N7f7vh8iIiIi0g0SQRAEsUMAwPTp0/HXX3/h0KFDaNOmzW3vK5VKDBo0CHZ2dti+fTuMjY3rfe3k5GS0b98eu3fvxsCBA297X6VSQaVS1fgsNzc3bW8yEVFjUyqVUCgUBtvOGPr9EZH4GtLO6EQP7wsvvIDff/8de/furbXYLSoqwpAhQ2BlZYUtW7Y0qNgFAE9PT9jb2yMxMbHW9+VyuXacGcebERERERkWUQteQRDwwgsvYMuWLdizZw/atWt32zFKpRKDBw+GiYkJtm/fDlNT0wZ/Tnp6OvLz8+Hi4tIYsYmIiIhIj4ha8M6YMQM//PADfvrpJ1hZWSErKwtZWVm4efMmgH+K3ZKSEqxcuRJKpVJ7zL/H4/r4+GDLli0AgOLiYrz22ms4duwYUlJSEBUVhVGjRqFDhw4IDw8X5T6JiIiISDyiTku2bNkyAEBoaGiN/atXr8ZTTz2FmJgYHD9+HADQoUOHGsdcuXIFHh4eAID4+HjtDA8ymQxnz57F2rVrUVBQAFdXVwwePBjz589v0Fy8RERERGQYRC147/a8XGho6F2P+e91zMzM8Pfff993NiIiIiIyDDrx0BoRERERUVNhwUtEREREBo0FLxEREREZNBa8jaRYVYnk3GKxYxARERHpHbVGQNr1UpRXaprk+qI+tGYo9lzKxnM/xMDP1Rpbnu8jdhwiIiIinSMIAnKLVbiSW4IreSW4kl+i/fXVW8Xuby/0RZc2ikb/bBa8jcDPVYHySg1OpxYgR1kGR+uGL45BREREZAgKSyuqitm84qqCNr9U++uScnWd55nIpMhWlqELWPDqJCdrUwS42SA2rQC74rIxIdhd7EhERERETeZmuRop+bd6av+zXS8pr/M8qQRobWuGdvaW8LS3QDt7C3jYW8DT3gKuNmaQSSVNkpcFbyMZ7OeE2LQC7LzAgpeIiIj0X4Vag7TrpbUWtZmFZXc818laDo9WFvB0uFXU3vq1m5055EayZrqDf7DgbSSDfZ2xcEc8jiTloaisAlamxmJHIiIiImqwCxmFeHPTOVzMVEKtqXsBMIWZcVVB26qqqG3nUFXYethbwFKuWyWmbqXRYx0cLeHpYIHk3BLsi8/FCH9XsSMRERERNcje+By88GOMdqytmbFMW8z+u7Bt18oCthYmIqetPxa8jWiwrzOW70/CzovZLHiJiIhIr/x0PBXvbjsPtUZA7/atsPDhrmhtYwaJpGnG1TYnzsPbiAb7OQEA9l7Kgaqy7qcQiYiIiHSFRiPgo78u4a0t56DWCHg4qA3WTO6JNrbmBlHsAix4G1VAGxs4WMlRrKrEseTrYschIiIiuqOyCjVeXH8ay/cnAQAiBnnhk4e7wsTIsEpEw7obkUmlEgzyrerl3XkhS+Q0RERERHW7XlKOJ747jj/OZsJYJsGiR/3x0sCOBtOr+28seBvZ4FsF766L2dDc4clGIiIiIrGk5JXgf8uO4OTVG7AyNcLaKT0xtlsbsWM1GRa8jSykfStYyo2QU6TCmfQCseMQERER1XDq6nWM+fowruSVoLWNGTZP743e7e3FjtWkWPA2MrmRDKHeDgCAnRezRU5DRERE9I8/zmZi/IrjuFFaga5tFNgyozc6OlmJHavJseBtAoP9nAFwHC8RERHpBkEQ8M3+JMz4KQbllRoM8nXC+mm94GhlKna0ZsF5eJtAqLcDjGUSJOWWIDGnGB0cLcWORERERC1UpVqDudsv4MfjqQCAp3p74N2HfCGTGt7DaXVhD28TsDY1RsitsTC7OKyBiIiIRFKiqsTUdSfx4/FUSCTAuw/54r2Rfi2q2AVY8DaZ6tkadl7ksAYiIiJqftnKMjz6zVHsjc+FqbEUyyYE4em+7cSOJQoWvE2kej7e06kFyFGWiZyGiIiIWpJLWUqMXnoYFzKUaGVhgp+n9sKQzs5ixxINC94m4mRtigA3GwDA7rgcccMQERFRi3EwIRcPLzuKzMIyeDpYYMvzfRDY1lbsWKJiwduEBvtxWAMRERE1n40n0jB59QkUqyoR3M4Om6f3RttW5mLHEh0L3iY02Lfqq4MjifkoKqsQOQ0REREZKkEQ8Onf8Xh901lUagSMDnDFuqd7wsbcROxoOoEFbxPq4GgJTwcLlKs12H85V+w4REREZIBUlWq8vCEWS/YmAgBefLADPh8XALmRTORkuoMFbxOr7uXdeYHTkxEREVHjKigtx5PfRWNbbAaMpBIs/F9XvDrYGxJJy5p27G5Y8Dax6nG8ey/loLxSI3IaIiIiMhSp+aUYu+wIolOuw0puhNWTe+DRHm5ix9JJLHibWEAbGzhYyVGkqsSx5Hyx4xAREZEBiEm9gTFfH0ZybglcFab4ZXoIHujoIHYsncWCt4lJpRLtnLycrYGIiIju147zmRj/7THkl5TDz9UaW2b0gY+ztdixdBoL3mZQXfDuupgNjUYQOQ0RERHpI0EQ8N3BZEz/MQaqSg0GeDtg47MhcLI2FTuazmPB2wx6t28FCxMZspUqnL1WKHYcIiIi0jNqjYD3tl/Agj/iIAjAhOC2WDGxOyzkRmJH0wsseJuB3EiGUB9HAMDOCxzWQERERPVXWl6JZ78/ibVHrwIA3hrmgwWjO8NIxjKuvvg71UwGa8fxcnoyIiIiqp+LGUqMXnoYu+NyYGIkxdLHu2Fav/acdqyBWPA2kwE+jjCWSZCYU4yk3GKx4xBRC7N06VJ4eHjA1NQUwcHBiI6OvuPxixcvhre3N8zMzODm5oZXXnkFZWVlzZSWiDQaAd/sT8KopYdwObsY9pZy/Dw1GMO7uogdTS+JWvBGRkaiR48esLKygqOjI0aPHo34+Pgax5SVlWHGjBlo1aoVLC0t8b///Q/Z2XfuJRUEAXPmzIGLiwvMzMwQFhaGhISEpryVu7I2NUYvz1YAqh5eIyJqLhs2bEBERATmzp2LmJgY+Pv7Izw8HDk5ObUe/9NPP+HNN9/E3LlzERcXh5UrV2LDhg146623mjk5UcuUUXATj393DJF/XUKFWsAgXyf8/fIDCHK3Ezua3hK14N2/fz9mzJiBY8eOYdeuXaioqMDgwYNRUlKiPeaVV17Bb7/9hl9++QX79+9HRkYGxo4de8frLly4EF9++SWWL1+O48ePw8LCAuHh4aL3Tgz2q151jeN4iaj5LFq0CFOnTsXkyZPh6+uL5cuXw9zcHKtWrar1+CNHjqBPnz54/PHH4eHhgcGDB2P8+PF37RUmovu3/UwGwhcfwLHk6zAzluGjsV3w7ZNBaGUpFzuaXhO14N2xYweeeuop+Pn5wd/fH2vWrEFqaipOnToFACgsLMTKlSuxaNEiPPjggwgKCsLq1atx5MgRHDt2rNZrCoKAxYsX45133sGoUaPQtWtXrFu3DhkZGdi6dWsz3t3tBnWqGsd7Oq0AOUp+NUhETa+8vBynTp1CWFiYdp9UKkVYWBiOHj1a6zm9e/fGqVOntAVucnIy/vzzTwwbNqzOz1GpVFAqlTU2Iqq/wpsVeHn9abz082kUlVXC380Gf858AI/1bMvxuo1Ap8bwFhZWTdllZ1fVZX/q1ClUVFTUaKh9fHzQtm3bOhvqK1euICsrq8Y5CoUCwcHBdZ7TXA21s8IU/m42EARgd1ztXyUSETWmvLw8qNVqODk51djv5OSErKzav216/PHHMW/ePPTt2xfGxsZo3749QkND7zikITIyEgqFQru5uXF5U6L6Opacj2FfHMTW2AxIJcDMgR3x63MhaGdvIXY0g6EzBa9Go8HLL7+MPn36oHPnzgCArKwsmJiYwMbGpsaxd2qoq/c3pHFvzoZ6MFddIyIdt2/fPnz44Yf4+uuvERMTg82bN+OPP/7A/Pnz6zxn9uzZKCws1G5paWnNmJhIP5VXavDRX5cwfsUxXCu4ibZ25vjlud54ZZAXjDnlWKPSmdmKZ8yYgfPnz+PQoUPN/tmzZ89GRESE9rVSqWyyojfczwmf/B2PI4n5KCqrgJWpcZN8DhERANjb20Mmk932sG92djacnZ1rPefdd9/Fk08+iWeeeQYA0KVLF5SUlGDatGl4++23IZXe/g+xXC6HXM4xhkT1lZhThJnrY3Eho+pb5Ue7t8GcEX6w5EISTUInfnx44YUX8Pvvv2Pv3r1o06aNdr+zszPKy8tRUFBQ4/g7NdTV+xvSuMvlclhbW9fYmkp7B0t42lugXK3B/su5TfY5REQAYGJigqCgIERFRWn3aTQaREVFISQkpNZzSktLbytqZTIZgKrnJIjo3gmCgHVHUzD8y0O4kKGEjbkxlj/RDQsf9mex24RELXgFQcALL7yALVu2YM+ePWjXrl2N94OCgmBsbFyjoY6Pj0dqamqdDXW7du3g7Oxc4xylUonjx4/XeU5zkkgkGOR3a1jDBU5PRkRNLyIiAitWrMDatWsRFxeH6dOno6SkBJMnTwYATJw4EbNnz9YeP2LECCxbtgzr16/HlStXsGvXLrz77rsYMWKEtvAloobLKSrDU6tPYM62C1BVavBAR3v8/XI/DOnMuXWbmqg/SsyYMQM//fQTtm3bBisrK+0YW4VCATMzMygUCjz99NOIiIiAnZ0drK2t8eKLLyIkJAS9evXSXsfHxweRkZEYM2YMJBIJXn75ZSxYsAAdO3ZEu3bt8O6778LV1RWjR48W6U5rGuzrjG/2J2PvpRyUV2pgYqQTHe1EZKDGjRuH3NxczJkzB1lZWQgICMCOHTu0zzqkpqbW6NF95513IJFI8M477+DatWtwcHDAiBEj8MEHH4h1C0R67+8LWZi9+Ryul5TDxEiKt4b6YGKIB6RSzsDQHCSCiN9P1TXNxurVq/HUU08BqFp44tVXX8XPP/8MlUqF8PBwfP311zWGJ0gkkhrnCIKAuXPn4ttvv0VBQQH69u2Lr7/+Gl5eXvXKpVQqoVAoUFhY2CTDGzQaAT0/jEJesQrrpvREPy+HRv8MItJtTd3OiM3Q74+ovkpUlZj/+0WsP1H1IGcnF2t88VgAvJysRE6m/xrSzoha8Oqq5mioZ28+h5+jU/FEr7ZYMLpLk3wGEekuQy8IDf3+iOrjdOoNvLIhFin5pZBIgGn9PBExyAtyIw4NagwNaWf4XbpIBt8ax7vrYjY0Gv7MQUREZCgq1Rp8sTsBDy8/ipT8UrgqTPHTM70we2gnFrsi4eOAIundvhUsTGTIVqpw9lohAtxsxI5ERERE9+lqfgle3hCL06kFAICR/q6YP7ozFGachlRM7OEVidxIhlAfRwDAzgtchIKIiEifCYKAjSfSMPSLgzidWgArUyN88VgAvhwfyGJXB7DgFdE/q65xejIiIiJ9db2kHM/9cAqvbzqL0nI1gtvZ4a+ZD2BUQGuxo9EtHNIgogE+jjCWSZCYU4yk3GK0d7AUOxIRERE1wP7LuZj1yxnkFqlgLJPg1cHemPqAJ2ScbkynsOAVkbWpMXp5tsLBhDzsupiN9v1Z8BIREemDsgo1PvrrEtYcSQEAdHC0xOJxAejcWiFuMKoVhzSIbLBf1XzCHMdLRESkHy5kFOKhrw5pi92nenvg9xf7stjVYezhFdmgTk54d+t5nE4rQE5RGRytTMWORERERLXIVpbhy6gEbDiRhkqNAAcrORY+3BUDvB3FjkZ3wYJXZM4KU/i72eBMWgGi4nIwvmdbsSMRERHRvxSWVmD5gSSsPnwFZRUaAMDQzs5YMLozWlnKRU5H9cGCVwcM9nXCmbQC7LyQxYKXiIhIR9wsV2PNkRQs25cIZVklACDI3RZvDPFBz3Z2IqejhmDBqwPC/Zzwyd/xOJyYj2JVJSzl/GMhIiISS4Vag40n0/DF7gTkFKkAAN5OVngt3BsDOzlCIuEMDPqGlZUOaO9gCU97CyTnlWB/fC6Gd3UROxIREVGLo9EI+ONcJj7bGY+U/FIAQBtbM7w62Asj/VtzqjE9xoJXB0gkEgzyc8I3+5Ox82IWC14iIqJmJAgCDiTkYeGOS7iQoQQA2Fua4MUHO+Kxnm6QG8lETkj3iwWvjhjs64xv9idjz6UclFdqYGLEGeOIiIiaWkzqDSzccQnHkq8DACzlRni2nyem9G0HCw4xNBj8k9QRgW42sLeUI69YheNX8vFARwexIxERERmshOwifPJ3PHZezAYAmBhJMbGXO54f0AF2FiYip6PGxoJXR0ilEgzydcTP0WnYeSGbBS8REVETSL9RisW7E7A5Jh0aAZBKgIeD2mBmmBda25iJHY+aCAteHTLY1xk/R6dh18VsvD/SD1IOjiciImoU+cUqLN2bhB+OXUW5umou3SF+zpgV7oUOjlYip6OmxoJXh4S0bwULExmylGU4d60Q/m42YkciIiLSa8WqSqw8eAUrDiajWFU1l26IZyu8MdQHAfx3tsVgwatDTI1lCPV2xB/nMrHzYhYLXiIionukqlTjp+OpWLInEfkl5QCAzq2t8cYQH/TtYM+5dFsYFrw6ZrCfU1XBeyEbr4X7iB2HiIhIr6g1AracvobPd13GtYKbAIB29haYNdgbQzs7c7hgC8WCV8eEejvCSCpBQk4xknOL4elgKXYkIiIinScIAnbH5eCTvy/hcnYxAMDJWo6Xw7zwcFAbGMs43WdLxoJXxyjMjBHSvhUOJuRh18VsPNufBS8REdGdHE/Ox8c7LiEmtQBA1b+lz4e2x6TeHjA15qIRxIJXJw32dcLBhDzsvJiNZ/u3FzsOERGRTrpWcBPvb7+gnUvX1FiKp/u2w7R+7aEwMxY5HekSFrw6KMzXCe9uu4CY1BvIKSqDo5Wp2JGIiIh0RoVag5WHruCL3Qm4WaGGkVSC8T3b4sUHO8DRmv9m0u1Y8OogF4UZ/NsocCa9EFFxORjfs63YkYiIiHTC8eR8vLP1PBJyqsbp9mxnhwWjO8PLiXPpUt1Y8OqowX7OOJNeiJ0XsljwEhFRi5dXrMKHf8Zhc8w1AEArCxO8NawTxnZrzSnG6K5Y8Oqowb5O+OTveBxOzEexqhKWcv5RERFRy6PRCPj5RCoW7ohH4c0KSCTA4z3b4rVwb9iYm4gdj/QEqygd1cHREu3sLXAlrwT743MxvKuL2JGIiIia1flrhXh763mcSSsAAPi5WmPB6M4IbGsrbjDSOyx4dZREIsFgXyd8cyAZOy9mseAlIqIWQ1lWgUU7L2Pd0RRoBMBKboRXB3vhiV7uMOJ8unQPWPDqsMF+VQXvnks5KK/UwMSI/5MTEZHhEgQB289kYMEfccgtUgEARvq74p3hnTj7At0XFrw6LMDNFvaWcuQVq3D8Sj4e6OggdiQiIqImkZRbjDnbzuNwYj4AwNPeAvNGdUbfjvYiJyNDwIJXh8mkEgzydcTP0WnYeSGbBS8RERmcsgo1lu5NxDf7k1Gu1kBuJMULAzpgWn9PyI24Sho1Dn5HruMG+zoDAHZdzIZGI4ichoiIqPHsvZSDQZ/vx1d7ElGu1iDU2wG7XumPFwd2ZLFLjYo9vDoupH0rWJjIkKUsw7lrhfB3sxE7EhER0X3JKLiJeb9dxI4LWQAAF4Up5o7wRbifM+fUpSYhag/vgQMHMGLECLi6ukIikWDr1q013pdIJLVun3zySZ3XfO+992473sfHp4nvpOmYGssQ6u0IANh5MUvkNERERPeuQq3BtweSELZoP3ZcyIJMKsHUB9phd0R/DOnswmKXmoyoBW9JSQn8/f2xdOnSWt/PzMyssa1atQoSiQT/+9//7nhdPz+/GucdOnSoKeI3m8F+TgCAnReyRU5CRER0b06kXMdDXx7Ch39eQmm5Gt3dbfHHS33x9nBfWHBxJWpiov4NGzp0KIYOHVrn+87OzjVeb9u2DQMGDICnp+cdr2tkZHTbufos1NsRRlIJEnKKkZxbDE8HS7EjERER1Ut+sQof/XUJv5xKBwDYmhtj9rBOeLhbG0il7NGl5qE3D61lZ2fjjz/+wNNPP33XYxMSEuDq6gpPT09MmDABqampzZCw6SjMjBHSvhWAqofXiIiIdJ1GI+Dn6FQ8+Nl+bbE7vqcb9rwaike7u7HYpWalN98hrF27FlZWVhg7duwdjwsODsaaNWvg7e2NzMxMvP/++3jggQdw/vx5WFlZ1XqOSqWCSqXSvlYqlY2avTEM9nXCwYQ87LyYjWf7txc7DhERUZ0uZBTina3ncTq1AADQyaVqSeAgdy4JTOLQm4J31apVmDBhAkxN77zSyr+HSHTt2hXBwcFwd3fHxo0b6+wdjoyMxPvvv9+oeRtbmK8T3t12ATGpN5BbpIKDlVzsSERERDVUqDX4+K9LWHX4CjQCYGEiQ8Rgb0wK4ZLAJC69+Nt38OBBxMfH45lnnmnwuTY2NvDy8kJiYmKdx8yePRuFhYXaLS0t7X7iNgkXhRn82yggCEBUHIc1EBGRblGWVWDy6hP47lBVsTu8iwuiXg3F033bsdgl0enF38CVK1ciKCgI/v7+DT63uLgYSUlJcHFxqfMYuVwOa2vrGpsuGuxX9SDeTo7jJSIiHZJRcBOPLDuKQ4l5MDeR4Zsng7B0Qjc4K+78rSxRcxG14C0uLkZsbCxiY2MBAFeuXEFsbGyNh8yUSiV++eWXOnt3Bw4ciCVLlmhfz5o1C/v370dKSgqOHDmCMWPGQCaTYfz48U16L81hsG/V9GSHEvNQrKoUOQ0R6ZOlS5fCw8MDpqamCA4ORnR0dJ3HhoaG1joH+vDhw5sxMemLCxmFGPP1YcRnF8HBSo6Nz4Yg3M9wZkoiwyBqwXvy5EkEBgYiMDAQABAREYHAwEDMmTNHe8z69eshCEKdBWtSUhLy8vK0r9PT0zF+/Hh4e3vj0UcfRatWrXDs2DE4ODg07c00gw6Olmhnb4HySg0OXM4VOw4R6YkNGzYgIiICc+fORUxMDPz9/REeHo6cnJxaj9+8eXONuczPnz8PmUyGRx55pJmTk67bF5+DR5cfRbZShY6OltjyfG90bq0QOxbRbSSCIAhih9A1SqUSCoUChYWFOje84cM/4/DtgWSMDnDF4scCxY5DRPeoOduZ4OBg9OjRQ/ttmEajgZubG1588UW8+eabdz1/8eLFmDNnDjIzM2FhYVGvz9TldpQax8/RqXhn63moNQJCPFth+ZNBUJgZix2LWpCGtDN6MYaX/lE9rCHqUg4q1BqR0xCRrisvL8epU6cQFham3SeVShEWFoajR4/W6xorV67EY489dsdiV6VSQalU1tjIMAmCgE/+voTZm89BrREwNrA11k7pyWKXdBoLXj0T2NYW9pYmKCqrxPHk62LHISIdl5eXB7VaDScnpxr7nZyckJWVddfzo6Ojcf78+bvOkhMZGQmFQqHd3Nzc7is36SZVpRovb4jF0r1JAICXBnbEZ4/6w8SI5QTpNv4N1TMyqQRhnar+4dp58e7/WBER3Y+VK1eiS5cu6Nmz5x2P04fpHen+FJZWYOLKaGyLzYCRVIKFD3dFxCAvSCRcMY10HwtePTTY71bBeyEbHIJNRHdib28PmUyG7Oya0xlmZ2fD2fnOT9KXlJRg/fr19VrSXV+md6R7k3a9FGOXHcbxK9dhKTfC6sk98Gh39uKT/mDBq4d6t7eHuYkMWcoynE0vFDsOEekwExMTBAUFISoqSrtPo9EgKioKISEhdzz3l19+gUqlwhNPPNHUMUmHnUkrwJivDyMptwQuClP8Oj0ED3TU/5mPqGVhwauHTI1l6O9V1djsuVT7tEJERNUiIiKwYsUKrF27FnFxcZg+fTpKSkowefJkAMDEiRMxe/bs285buXIlRo8ejVatWjV3ZNIRuy5m47FvjyGvuBydXKyx5fk+8HFm7z3pHyOxA9C9GeDtiL/OZ2FffA5eGeQldhwi0mHjxo1Dbm4u5syZg6ysLAQEBGDHjh3aB9lSU1Mhldbs/4iPj8ehQ4ewc+dOMSKTDlh3NAXvbb8AjQD083LA0scDYWXKmRhIP7Hg1VP9vat6eM9eK0ResQr2lnKRExGRLnvhhRfwwgsv1Prevn37btvn7e3NZwRaKI1GQORfcVhx8AoA4LEebpg/ujOMZfxSmPQX//bqKSdrU/i5WkMQwFXXiIioUZRVqPHCzzHaYve1cG9Eju3CYpf0Hv8G67HQW728e+NZ8BIR0f25XlKOCd8dx5/nsmAsk+CLxwIwY0AHTjtGBoEFrx4b4O0IoKqHV63hV49ERHRvUvJKMPbrwzh19QasTY2wbkowRgW0FjsWUaNhwavHAtxsoDAzRuHNCsSm3RA7DhER6aFTV69jzNeHkZJfija2Ztj8fG+EtOfMHGRYWPDqMSOZFA90tAcA7L3EYQ1ERNQwf53LxPgVx3GjtAJd2yiw+fne6OBoJXYsokbHglfPVQ9r2HeZ8/ESEVH9CIKA7w4m4/mfYlBeqUFYJ0esn9YLjlamYkcjahKclkzPVU9Pdv6aEjnKMjhas7EiItJVhaUVKLhZjja25pBJxXkYTK0RMO+3C1h79CoAYGKIO+aO8BMtD1FzYMGr5+wt5ejaRoGz6YXYdzmXa5sTEemoElUlHvxsH/JLymEik8LD3hztHSzh6WCB9g6W2l835eIOpeWVeOnnWOyOywYAvD2sE555oB1nYiCDx4LXAIR6O+JseiH2x7PgJSLSVWfSC5BfUg4AKFdrcDm7GJezi287ztFKflsR3N7BEq1tzCC9j17Y3CIVnll7AmfSC2FiJMXicQEY1sXlnq9HpE9Y8BqAUG8HfBmVgAMJuahQazhBOBGRDjqTVggAGOLnjLeHd0JSbjGSckuQlFuM5Fu/zi1SIefWdiz5eo3z5UZStLO3QHtHS7Sv/q+DJdrZW8BCfud/zhNzivDU6hNIv3ETtubG+G5SdwS52zXZvRLpGha8BsC/jQ1szY1xo7QCMVdvINiT08kQEemaM2kFAIBu7jZwszOHm505Qr1rHqMsq0BybgmScoqRnFeMpJyqgvhqfilUlRpcyirCpayi267tojCttVfYRWGK6CvXMXXdSSjLKuHeyhxrJvdEO3uLZrhjIt3BgtcAyKQS9PdywNbYDOy7nMuCl4hIB51JLwBQ1UlRF2tTYwS42SDAreYxlWoN0m/crFEEJ9/qHc4vKUdmYRkyC8twODG/xnlmxjJUajSoUAvo1tYGKyZ2RytLeSPfGZHuY8FrIEK9HbE1NgN7L+XgjSE+YschIqJ/yVZWFaRSCdC5taLB5xvJpPCwt4CHvQUe/E8TX1Ba/q+hEf8MkbiaX4qbFWoAwNDOzvh8XABMjWWNcTtEeocFr4Ho5+UAiQS4lFWEzMKbcFGYiR2JiIhuqR7O4OVkddfxtg1lY26CIHcTBLnb1thfodYg9XopyirU6ORsfV8PvBHpOz7dZCDsLEy0X4Htj+eqa0REuqQ+wxkam7FMivYOlvBzVbDYpRaPBa8BCfWqWnVtbzxXXSMi0iXVMzT4/2dsLhE1Dxa8BmSAT9Wqa4cS8lBeqRE5DRERAYBGI/zTw+vW8PG7RHT/WPAakM6uCthbmqCkXI2TV6/f/QQiImpyV/JLUFRWCVNjKbycrMSOQ9QiseA1IFKpBP28qnp593EcLxGRTqh+YK2zq4ILAxGJhP/nGZgB3rfG8V7iOF4iIl1QXfBy/C6ReFjwGph+HR0glQAJOcVIv1EqdhwiohYvNp0PrBGJjQWvgVGYG6Nb26q5GDmsgYhIXKpKNeIylACAgGackoyIamLBa4AG+FQNa9jH6cmIiER1KbMI5WoNbM2N4WbHBYGIxMKC1wCFelc9uHY4MR+qSrXIaYiIWq5/piOzgUTCxR+IxMKC1wD5uljD0UqOmxVqRF/h9GRERGKJrX5gjcMZiETFgtcASSQSbS/v3kscx0tEJJbqGRoC+MAakahY8Bqo6unJ9l3mOF4iIjEoyyqQlFsCAOjahiusEYlJ1IL3wIEDGDFiBFxdXSGRSLB169Ya7z/11FOQSCQ1tiFDhtz1ukuXLoWHhwdMTU0RHByM6OjoJroD3dWnoz2MpBIk55bgan6J2HGIiFqcc7emI3OzM0MrS7nIaYhaNlEL3pKSEvj7+2Pp0qV1HjNkyBBkZmZqt59//vmO19ywYQMiIiIwd+5cxMTEwN/fH+Hh4cjJaVk9ndamxghy5/RkRERi4fhdIt0hasE7dOhQLFiwAGPGjKnzGLlcDmdnZ+1ma2t7x2suWrQIU6dOxeTJk+Hr64vly5fD3Nwcq1atauz4Oo/TkxERiYfjd4l0h86P4d23bx8cHR3h7e2N6dOnIz8/v85jy8vLcerUKYSFhWn3SaVShIWF4ejRo3Wep1KpoFQqa2yGoPrBtSNJ+Sir4PRkRETN6d9TkhGRuHS64B0yZAjWrVuHqKgofPzxx9i/fz+GDh0Ktbr24i0vLw9qtRpOTk419js5OSErK6vOz4mMjIRCodBubm5ujXofYvF2soKLwhSqSg2OJtf9gwIRETWurMIyZCtVkEkl8HO1FjsOUYun0wXvY489hpEjR6JLly4YPXo0fv/9d5w4cQL79u1r1M+ZPXs2CgsLtVtaWlqjXl8sVdOTVQ1r2M9xvEREzaZ6/K6XkxXMTYzEDUNEul3w/penpyfs7e2RmJhY6/v29vaQyWTIzs6usT87OxvOzs51Xlcul8Pa2rrGZiiqhzXsuZQDQRBETkNE1DJUD2cIcON0ZES6QK8K3vT0dOTn58PFxaXW901MTBAUFISoqCjtPo1Gg6ioKISEhDRXTJ3Sp4M9jGUSpF4vxZU8Tk9GpC8mTZqEAwcOiB2D7tEZztBApFNELXiLi4sRGxuL2NhYAMCVK1cQGxuL1NRUFBcX47XXXsOxY8eQkpKCqKgojBo1Ch06dEB4eLj2GgMHDsSSJUu0ryMiIrBixQqsXbsWcXFxmD59OkpKSjB58uTmvj2dYCk3Qs92dgA4PRmRPiksLERYWBg6duyIDz/8ENeuXRM7EtWTRiPg7K05ePnAGpFuELXgPXnyJAIDAxEYGAigqlgNDAzEnDlzIJPJcPbsWYwcORJeXl54+umnERQUhIMHD0Iu/2cC76SkJOTl5Wlfjxs3Dp9++inmzJmDgIAAxMbGYseOHbc9yNaShHpVjePdy+nJiPTG1q1bce3aNUyfPh0bNmyAh4cHhg4dil9//RUVFRVix6M7SM4rRrGqEmbGMnR0tBQ7DhEBkAgc2HkbpVIJhUKBwsJCgxjPm5hThLBFB2AikyJ27iA+QEGkAxrazsTExGD16tX47rvvYGlpiSeeeALPP/88Onbs2AxpG87Q2tGG+PVUOmb9cgY9Peyw8bmWOZyOqDk0pJ3RqzG8dG/aO1iija0ZytUaHE3i9GRE+iYzMxO7du3Crl27IJPJMGzYMJw7dw6+vr74/PPPxY5H/6Edv8sH1oh0BgveFqBqerKq2Ro4rIFIP1RUVGDTpk146KGH4O7ujl9++QUvv/wyMjIysHbtWuzevRsbN27EvHnzxI5K/8EFJ4h0D7/bbiEGeDvih2Op2BefC0EQIJFIxI5ERHfg4uICjUaD8ePHIzo6GgEBAbcdM2DAANjY2DR7NqpbWYUacZlVq3VyhgYi3cGCt4UIad8KJkZSpN+4iaTcYnRwtBI7EhHdweeff45HHnkEpqamdR5jY2ODK1euNGMqupu4TCUq1AJaWZigja2Z2HGI6BYOaWghzE2MEHxrerK9lzg9GZGuGzlyJEpLS2/bf/36dSiVShESUX38M37Xht+kEekQFrwtyIBbywzvu8xxvES67rHHHsP69etv279x40Y89thjDb7e0qVL4eHhAVNTUwQHByM6OvqOxxcUFGDGjBlwcXGBXC6Hl5cX/vzzzwZ/bktzpnr+XQ5nINIpLHhbkAE+VQVv9JXrKFZVipyGiO7k+PHjGDBgwG37Q0NDcfz48QZda8OGDYiIiMDcuXMRExMDf39/hIeHIyen9h9+y8vLMWjQIKSkpODXX39FfHw8VqxYgdatW9/TvbQknKGBSDex4G1B2tlbwL2VOSrUAg4n5t39BCISjUqlQmXl7T+YVlRU4ObNmw261qJFizB16lRMnjwZvr6+WL58OczNzbFq1apaj1+1ahWuX7+OrVu3ok+fPvDw8ED//v3h7+9/T/fSUhSWViD51hLu7OEl0i0seFsY7bAGLjNMpNN69uyJb7/99rb9y5cvR1BQUL2vU15ejlOnTiEsLEy7TyqVIiwsDEePHq31nO3btyMkJAQzZsyAk5MTOnfujA8//BBqtbrhN9KCnL1WAABwb2UOWwsTccMQUQ2cpaGFCfV2wJojKdgXn8PpyYh02IIFCxAWFoYzZ85g4MCBAICoqCicOHECO3furPd18vLyoFarb1te3cnJCZcuXar1nOTkZOzZswcTJkzAn3/+icTERDz//POoqKjA3Llzaz1HpVJBpVJpX7fEB+u0wxnYu0ukc9jD28L08mwFuZEUmYVliM8uEjsOEdWhT58+OHbsGNzc3LBx40b89ttv6NChA86ePYsHHnigST9bo9HA0dER3377LYKCgjBu3Di8/fbbWL58eZ3nREZGQqFQaDc3N7cmzaiLYtNuPbDGBSeIdA4L3hbG1FiG3u1bAeCwBiJdVVFRgSlTpsDa2ho//vgjLly4gJMnT2LVqlXo2LFjg65lb28PmUyG7OzsGvuzs7Ph7Oxc6zkuLi7w8vKCTCbT7uvUqROysrJQXl5e6zmzZ89GYWGhdktLS2tQTn0nCAJib/XwBvCBNSKdw4K3BaqerWHvJU5PRqSLjI2NsWnTpka5lomJCYKCghAVFaXdp9FoEBUVhZCQkFrP6dOnDxITE6HRaLT7Ll++DBcXF5iY1D42VS6Xw9rausbWkmQWliGvWAWZVAI/Vxa8RLqGBW8LFOpVVfCevHoDyrIKkdMQUW1Gjx6NrVu3Nsq1IiIisGLFCqxduxZxcXGYPn06SkpKMHnyZADAxIkTMXv2bO3x06dPx/Xr1zFz5kxcvnwZf/zxBz788EPMmDGjUfIYourxuz7OVjA1lt35YCJqdvf00FpaWhokEgnatGkDAIiOjsZPP/0EX19fTJs2rVEDUuNr28ocng4WSM4tweGEPAzt4iJ2JCL6j44dO2LevHk4fPgwgoKCYGFhUeP9l156qd7XGjduHHJzczFnzhxkZWUhICAAO3bs0D7IlpqaCqn0n/4PNzc3/P3333jllVfQtWtXtG7dGjNnzsQbb7zRODdngGLTCwBw/C6RrpIIgiA09KQHHngA06ZNw5NPPomsrCx4e3vDz88PCQkJePHFFzFnzpymyNpslEolFAoFCgsLDfZrufm/X8TKQ1fwaPc2WPgw59Ykam53a2fatWtX57kSiQTJyclNGe++tYR29N8e+/YojiVfx8L/dcWjPVreA3tEYmhIO3NPPbznz59Hz549AVQtc9m5c2ccPnwYO3fuxHPPPaf3BW9LEOrtgJWHrmBffC6nJyPSQVeuXBE7AtWTWiPgXDpnaCDSZfc0hreiogJyuRwAsHv3bowcORIA4OPjg8zMzMZLR02mZzs7mBnLkFOkwsXMljdfJhFRY0nKLUZJuRrmJjJ0cLQUOw4R1eKeenj9/PywfPlyDB8+HLt27cL8+fMBABkZGWjVqlWjBqSmITeSoU8He+yOy8a++Fw+VUykg9LT07F9+3akpqbeNh3YokWLREpF/1U9HVmX1grIpPy2jEgX3VPB+/HHH2PMmDH45JNPMGnSJO366tu3b9cOdSDdF+rtgN1x2dh7KQczBnQQOw4R/UtUVBRGjhwJT09PXLp0CZ07d0ZKSgoEQUC3bt3Ejkf/ckY7/66NqDmIqG73VPCGhoYiLy8PSqUStra22v3Tpk2Dubl5o4WjphXq7QAAiEm9gcLSCijMjUVORETVZs+ejVmzZuH999+HlZUVNm3aBEdHR0yYMAFDhgwROx79yxnO0ECk8+5pDO/NmzehUqm0xe7Vq1exePFixMfHw9HRsVEDUtNpY2sOLydLaATgQAJXXSPSJXFxcZg4cSIAwMjICDdv3oSlpSXmzZuHjz/+WOR0VK2sQo1LmVXLtLPgJdJd91Twjho1CuvWrQMAFBQUIDg4GJ999hlGjx6NZcuWNWpAalqh3rdWXYvnqmtEusTCwkI7btfFxQVJSUna9/Ly8sSKRf9xIUOJSo0Ae0s5XBWmYschojrcU8EbExODBx54AADw66+/wsnJCVevXsW6devw5ZdfNmpAalrVwxoOXM6FRtPgKZmJqIn06tULhw4dAgAMGzYMr776Kj744ANMmTIFvXr1EjkdVftn/K6C0zsS6bB7GsNbWloKKysrAMDOnTsxduxYSKVS9OrVC1evXm3UgNS0urvbwVJuhLzicpzPKETXNjZiRyIiVM3CUFxcDAB4//33UVxcjA0bNqBjx46coUGHaMfvsu0k0mn3VPB26NABW7duxZgxY7TLTwJATk5Oi1hRx5CYGEnRp0Mr/H0hG3sv5bLgJdIRnp6e2l9bWFhg+fLlIqahulT38HL8LpFuu6chDXPmzMGsWbPg4eGBnj17IiQkBEBVb29gYGCjBqSmN+DWON59lzmOl4iovgpKy5GSXwoA6NqGc5kT6bJ76uF9+OGH0bdvX2RmZmrn4AWAgQMHYsyYMY0WjppH9YNrsWkFuF5SDjsLE5ETEZFUKr3jmFC1Wt2Maag2Z24tJ9zO3gI25mw3iXTZPRW8AODs7AxnZ2ekp6cDANq0acNFJ/SUs8IUPs5WuJRVhAOXczE6sLXYkYhavC1bttR4XVFRgdOnT2Pt2rV4//33RUpF/6YdzsDeXSKdd08Fr0ajwYIFC/DZZ59pH6qwsrLCq6++irfffhtS6T2NlCARDfBxxKWsIuyLz2HBS6QDRo0addu+hx9+GH5+ftiwYQOefvppEVLRv3H8LpH+uKfK9O2338aSJUvw0Ucf4fTp0zh9+jQ+/PBDfPXVV3j33XcbOyM1g+pxvPsv50LN6cmIdFavXr0QFRUldowWTxAErrBGpEfuqYd37dq1+O677zBy5Ejtvq5du6J169Z4/vnn8cEHHzRaQGoe3drawMrUCDdKK3AmvQDd2tre/SQialY3b97El19+idat+S2M2K4V3ERecTmMpBL4unB2IiJdd08F7/Xr1+Hj43Pbfh8fH1y/fv2+Q1HzM5JJ0a+jA/44l4l98bkseIlEZmtrW+OhNUEQUFRUBHNzc/zwww8iJiMAOJNW9cBaJxdrmBrLRE5DRHdzTwWvv78/lixZctuqakuWLEHXrl0bJRg1v1Dv6oI3BxGDvMSOQ9Siff755zUKXqlUCgcHBwQHB8PWlj+Qiu2f4Qx8YI1IH9xTwbtw4UIMHz4cu3fv1s7Be/ToUaSlpeHPP/9s1IDUfPrfWmb4bHohcotUcLCSi5yIqOV66qmnxI5AdxCrnaHBRtQcRFQ/9/TQWv/+/XH58mWMGTMGBQUFKCgowNixY3HhwgV8//339b7OgQMHMGLECLi6ukIikWDr1q3a9yoqKvDGG2+gS5cusLCwgKurKyZOnIiMjIw7XvO9996DRCKpsdU2/IJu52hlis6tq8aiHbicK3IaopZt9erV+OWXX27b/8svv2Dt2rUiJKJqlWoNzt2agzeAD6wR6YV7nj/M1dUVH3zwATZt2oRNmzZhwYIFuHHjBlauXFnva5SUlMDf3x9Lly697b3S0lLExMTg3XffRUxMDDZv3oz4+PgaD8rVxc/PD5mZmdrt0KFDDbq3lqx6toa98Vx1jUhMkZGRsLe3v22/o6MjPvzwQxESUbXE3GLcrFDDUm4ETwdLseMQUT3c88ITjWHo0KEYOnRore8pFArs2rWrxr4lS5agZ8+eSE1NRdu2beu8rpGREZydnRs1a0sR6u2Ar/Yk4sDlXFSqNTCScU5lIjGkpqaiXbt2t+13d3dHamqqCImoWvX8u11aKyCT1r0aHhHpDr2qZgoLCyGRSGBjY3PH4xISEuDq6gpPT09MmDDhrv84qFQqKJXKGltLFeBmCxtzYyjLKrVj1Iio+Tk6OuLs2bO37T9z5gxatWolQiKqFntrhgbOv0ukP/Sm4C0rK8Mbb7yB8ePHw9q67jkPg4ODsWbNGuzYsQPLli3DlStX8MADD6CoqKjOcyIjI6FQKLSbm5tbU9yCXpBJJejXserhNQ5rIBLP+PHj8dJLL2Hv3r1Qq9VQq9XYs2cPZs6ciccee0zseC1adQ9vAGdoINIbDRrSMHbs2Du+X1BQcD9Z6lRRUYFHH30UgiBg2bJldzz230MkunbtiuDgYLi7u2Pjxo11LsU5e/ZsREREaF8rlcoWXfSGejtg+5kM7L2Ui9fC+cAfkRjmz5+PlJQUDBw4EEZGVU21RqPBxIkTOYZXRDfL1YjPrupAYQ8vkf5oUMGrUNz5p1mFQoGJEyfeV6D/qi52r169ij179tyxd7c2NjY28PLyQmJiYp3HyOVyyOWcgqtaPy8HSCTAxUwlspVlcLI2FTsSUYtjYmKCDRs2YMGCBYiNjYWZmRm6dOkCd3d3saO1aBcyCqHWCHC0ksOZbSOR3mhQwbt69eqmylGr6mI3ISEBe/fuvadxa8XFxUhKSsKTTz7ZBAkNk72lHF3b2OBMWgH2x+fi0R4tt7ebSGwdO3ZEx44dxY5Bt2jn33WzqbEwCBHpNlHH8BYXFyM2NhaxsbEAgCtXriA2NhapqamoqKjAww8/jJMnT+LHH3+EWq1GVlYWsrKyUF5err3GwIEDsWTJEu3rWbNmYf/+/UhJScGRI0cwZswYyGQyjB8/vrlvT6+FenEcL5GY/ve//+Hjjz++bf/ChQvxyCOPiJCIAOAM598l0kuiFrwnT55EYGAgAgMDAQAREREIDAzEnDlzcO3aNWzfvh3p6ekICAiAi4uLdjty5Ij2GklJScjLy9O+Tk9Px/jx4+Ht7Y1HH30UrVq1wrFjx+Dg4NDs96fPBvhUzcd7KCEPFWqNyGmIWp4DBw5g2LBht+0fOnQoDhw4IEIiAv55YI0rrBHpF1Hn4Q0NDYUgCHW+f6f3qqWkpNR4vX79+vuNRQC6tlaglYUJ8kvKcerqDfTy5DRIRM2puLgYJiYmt+03NjZu0VMniim/WIXU66UAgC5tOEMDkT7Rm2nJqHlJpRL047AGItF06dIFGzZsuG3/+vXr4evrK0IiOntrOIOngwUUZsYipyGihhC1h5d0W6i3A7acvob98bmYPbST2HGIWpR3330XY8eORVJSEh588EEAQFRUFH766Sf8+uuvIqdrmaofWAvgcAYivcOCl+rUr6MDpBLgUlYRMgpuwtXGTOxIRC3GiBEjsHXrVnz44Yf49ddfYWZmBn9/f+zZswd2dnZix2uRzqQXAOD8u0T6iEMaqE62FibaJ5H3xeeKG4aoBRo+fDgOHz6MkpISJCcn49FHH8WsWbPg7+8vdrQWRxCEfx5YY8FLpHdY8NIdDfCumq1hH8fxEoniwIEDmDRpElxdXfHZZ5/hwQcfxLFjx8SO1eKkXb+JG6UVMJZJ0MnFSuw4RNRAHNJAdzTAxxGf7bqMw4l5UFWqITeSiR2JyOBlZWVhzZo1WLlyJZRKJR599FGoVCps3bqVD6yJJPbWcAZfF2u2g0R6iD28dEe+Ltawt5SjpFyNkyk3xI5DZPBGjBgBb29vnD17FosXL0ZGRga++uorsWO1eBzOQKTfWPDSHUmlEoR6V01PxmENRE3vr7/+wtNPP433338fw4cPh0zG3kRdwAUniPQbC166q+pxvHv54BpRkzt06BCKiooQFBSE4OBgLFmypMZqktT8KtQanM+omoOXPbxE+okFL91V3472kEklSMwpRtqtVYaIqGn06tULK1asQGZmJp599lmsX78erq6u0Gg02LVrF4qKisSO2OJczi5CWYUGVnIjeNpbiB2HiO4BC166K4WZMYLa2gIA9l1mLy9Rc7CwsMCUKVNw6NAhnDt3Dq+++io++ugjODo6YuTIkWLHa1HOpFX17nZ1U0AqlYichojuBQteqpdQn6pxvNtOX0OlWiNyGqKWxdvbGwsXLkR6ejp+/vlnseO0OBy/S6T/WPBSvQzv4gITIylOXr2B1389C41GEDsSUYsjk8kwevRobN++XewoLQpXWCPSfyx4qV7cW1lg6ePdIJNKsPn0Nbz32wUIAoteIjJsJapKXM6uGjcdwIKXSG+x4KV6G+TrhEWP+kMiAdYdvYpPd8aLHYmI6mnp0qXw8PCAqakpgoODER0dXeexa9asgUQiqbGZmpo2Y1rdcf5aITQC4GxtCifrlvl7QGQIWPBSg4wKaI35ozoDAJbuTcLy/UkiJyKiu9mwYQMiIiIwd+5cxMTEwN/fH+Hh4cjJqXtubWtra2RmZmq3q1evNmNi3fHPcAaFuEGI6L6w4KUGe6KXO94Y4gMA+OivS/jhWMv8h5BIXyxatAhTp07F5MmT4evri+XLl8Pc3ByrVq2q8xyJRAJnZ2ft5uTk1IyJdUf1DA0cv0uk31jw0j2ZHtoez4e2BwC8u+08tsVeEzkREdWmvLwcp06dQlhYmHafVCpFWFgYjh49Wud5xcXFcHd3h5ubG0aNGoULFy7c8XNUKhWUSmWNzRDE3pqhIYAzNBDpNRa8dM9eC/fGxBB3CAIQsfEMdl/MFjsSEf1HXl4e1Gr1bT20Tk5OyMrKqvUcb29vrFq1Ctu2bcMPP/wAjUaD3r17Iz09vc7PiYyMhEKh0G5ubm6Neh9iyC1S4VrBTUgkQOc2HNJApM9Y8NI9k0gkeG+EH8YGtoZaI+D5n2JwJJFLoBLpu5CQEEycOBEBAQHo378/Nm/eDAcHB3zzzTd1njN79mwUFhZqt7S0tGZM3DTO3hq/297BEtamxuKGIaL7woKX7otUKsHCh7tisK8Tyis1eGbdSZxOvSF2LCK6xd7eHjKZDNnZNb+Byc7OhrOzc72uYWxsjMDAQCQmJtZ5jFwuh7W1dY1N33HBCSLDwYKX7puRTIqvHg9E3w72KC1X46nVJxCXaRjj94j0nYmJCYKCghAVFaXdp9FoEBUVhZCQkHpdQ61W49y5c3BxcWmqmDopNr3qgbUAztBApPdY8FKjkBvJ8O3EIHRra4PCmxV4cmU0ruSViB2LiABERERgxYoVWLt2LeLi4jB9+nSUlJRg8uTJAICJEydi9uzZ2uPnzZuHnTt3Ijk5GTExMXjiiSdw9epVPPPMM2LdQrMTBOGfHl7O0ECk94zEDkCGw9zECKsn98Rj3x5DXKYST3x3HL88FwJXGzOxoxG1aOPGjUNubi7mzJmDrKwsBAQEYMeOHdoH2VJTUyGV/tP/cePGDUydOhVZWVmwtbVFUFAQjhw5Al9fX7FuodldzS9F4c0KmMik8HHW/+EZRC2dROD6sLdRKpVQKBQoLCw0iHFozS2vWIVHlx9Fcl4JPO0tsPG5ENhbysWORaRTDL2d0ff72xZ7DTPXxyLAzQZbZ/QROw4R1aIh7QyHNFCjs7eU4/tnguGqMEVyXgkmroxG4c0KsWMREdWbdv5dDmcgMggseKlJtLYxww/PBMPe0gQXM5WYsuYESssrxY5FRFQv/4zf5QNrRIaABS81GU8HS3z/dDCsTY1w6uoNPPv9Kagq1WLHIiK6owq1Buczqmaa4ZRkRIaBBS81qU4u1lgzpSfMTWQ4mJCHl34+jUq1RuxYRER1is8qQnmlBtamRvBoZSF2HCJqBCx4qcl1a2uLFRO7w0Qmxd8XsvH6prPQaPisJBHppth/TUcmlUrEDUNEjYIFLzWLPh3sseTxQMikEmyOuYb3f7sAThBCRLqIK6wRGR4WvNRsBvs547NH/CGRAGuPXsVnOy+LHYmI6DZn0gsAcMEJIkPCgpea1ejA1pg/qjMAYMneRHyzP0nkRERE/yhWVSIhpxgAZ2ggMiQseKnZPdHLHW8M8QEARP51CT8evypyIiKiKufSCyEIVVMrOlqZih2HiBqJqAXvgQMHMGLECLi6ukIikWDr1q013hcEAXPmzIGLiwvMzMwQFhaGhISEu1536dKl8PDwgKmpKYKDgxEdHd1Ed0D3anpoezwf2h4A8M7W89gWe03kRERE/x7OwN5dIkMiasFbUlICf39/LF26tNb3Fy5ciC+//BLLly/H8ePHYWFhgfDwcJSVldV5zQ0bNiAiIgJz585FTEwM/P39ER4ejpycnKa6DbpHr4V748le7hAEIGLjGey+mC12JCJq4fjAGpFhErXgHTp0KBYsWIAxY8bc9p4gCFi8eDHeeecdjBo1Cl27dsW6deuQkZFxW0/wvy1atAhTp07F5MmT4evri+XLl8Pc3ByrVq1qwjuheyGRSPD+SD+MCWwNtUbA8z/F4EhintixiKgFO/OvKcmIyHDo7BjeK1euICsrC2FhYdp9CoUCwcHBOHr0aK3nlJeX49SpUzXOkUqlCAsLq/McEpdUKsEnD3fFIF8nlFdq8My6kzidekPsWETUAuUoy5BRWAapBOjSmkMaiAyJzha8WVlZAAAnJ6ca+52cnLTv/VdeXh7UanWDzgEAlUoFpVJZY6PmYyST4qvxgejToRVKy9V4avUJXMrinwERNa8z6YUAgI6OVrCQG4mchogak84WvM0pMjISCoVCu7m5uYkdqcUxNZbh2ye7o1tbGxTerMAT30UjJa9E7FhE1IL8M5yBvbtEhkZnC15nZ2cAQHZ2zQeZsrOzte/9l729PWQyWYPOAYDZs2ejsLBQu6Wlpd1neroXFnIjrH6qJzq5WCOvWIUJ3x1HRsFNsWMRUQvBBSeIDJfOFrzt2rWDs7MzoqKitPuUSiWOHz+OkJCQWs8xMTFBUFBQjXM0Gg2ioqLqPAcA5HI5rK2ta2wkDoW5MdZN6QlPewtcK7iJJ1YeR16xSuxYRGTgNBqBMzQQGTBRC97i4mLExsYiNjYWQNWDarGxsUhNTYVEIsHLL7+MBQsWYPv27Th37hwmTpwIV1dXjB49WnuNgQMHYsmSJdrXERERWLFiBdauXYu4uDhMnz4dJSUlmDx5cjPfHd0rBys5vn8mGK4KUyTnluCJ746joLRc7FhEZMBS8kugLKuE3EgKb2crseMQUSMTdVT+yZMnMWDAAO3riIgIAMCkSZOwZs0avP766ygpKcG0adNQUFCAvn37YseOHTA1/Wf1m6SkJOTl/TOV1bhx45Cbm4s5c+YgKysLAQEB2LFjx20PspFua21jhh+n9sKj3xzFpawiTFoVjR+eCYaVqbHY0YjIAFUPZ+jcWgFjmc5++UlE90giCIIgdghdo1QqoVAoUFhYyOENIrucXYRx3xzFjdIK9PCwxdopPWFuwqenSf8Zejujb/f33vYLWHMkBVP6tMOcEb5ixyGiemhIO8MfY0mneTlZ4fung2FlaoQTKTcwdd1JlFWoxY5FRAYmljM0EBk0Fryk8zq3VmDtlJ6wMJHhcGI+nv8xBuWVGrFjEZGBKK/U4GJG1dzfAZyhgcggseAlvdCtrS1WPtUDciMp9lzKwcsbTqNSzaKXiO7fpSwlytUa2Jgbo62dudhxiKgJsOAlvdHLsxW+ndgdJjIp/jyXhdd+PQuNhkPQiej+/Hs6MolEIm4YImoSLHhJr/T3csCSxwMhk0qw5fQ1vL31PPjcJRHdj9i0qiWFueAEkeFiwUt6Z7CfMxaPC4BUAvwcnYp5v19k0UtE96x6SrIAPrBGZLBY8JJeGuHvio//1xUAsPpwCj7dGS9yIiLSR8qyCiTlFgMAunKFNSKDxYKX9NYj3d0wf5QfAGDp3iQs2ZMgciIi0jfn0wshCEAbWzPYW8rFjkNETYQFL+m1J0M88PawTgCAT3dexncHk0VORET6JPbWcAaO3yUybCx4Se9N7eeJV8K8AAAL/ojDj8evipyIiPRF9QwNARzOQGTQWPCSQXhpYAc81789AOCdreex6VS6yImISB+c4QwNRC0CC14yCBKJBG8M8cZTvT0gCMBrv57BH2czxY5FRDosq7AMWcoySCVA59bWYschoibEgpcMhkQiwZyHfDGuuxs0AjBz/Wnsvpgtdiwi0lHV05F5OVnB3MRI3DBE1KRY8JJBkUol+HBsF4wKcEWlRsDzP8bgYEKu2LGISAedunoDQNUKa0Rk2FjwksGRSSX47BF/DPFzRrlag6nrTuJ4cr7YsYhIh2g0An4/kwEA6OflIHIaImpqLHjJIBnJpPhyfCBCvR1QVqHBlDUnEHvraWwiomNX8pFRWAYrUyMM7OQodhwiamIseMlgmRhJsfyJIIR4tkJJuRoTVx7HhYxCsWMRkQ7YEnMNADC8iwtMjWUipyGipsaClwyaqbEM303qjiB3WyjLKvHkymgk5hSJHYuIRHSzXI2/zmcBAMZ2ayNyGiJqDix4yeBZyI2wenIPdGmtwPWScjy+4jhS8krEjkVEItl5MQvFqkq0sTVDd3dbseMQUTNgwUstgrWpMdZN6QlvJyvkFKkw4bvjuFZwU+xYRCSCLaerhjOMCWwNqVQichoiag4seKnFsLUwwQ/PBMPT3gLXCm5iwopjyFGWiR2LiJpRbpEKBxPyAFQVvETUMrDgpRbFwUqOH6cGw83ODCn5pZjw3XHkF6vEjkVEzWT7mQyoNQL83Wzg6WApdhwiaiYseKnFcVGY4adnesHZ2hQJOcV4cmU0CksrxI5FRM1gy+l0AMD/urF3l6glYcFLLZKbnTl+mhoMe0s5LmYqMWl1NIpVlWLHImoyS5cuhYeHB0xNTREcHIzo6Oh6nbd+/XpIJBKMHj26aQM2g8vZRTh/TQkjqQQPdXUVOw4RNSMWvNRieTpY4sdngmFjbozYtAJMWXMCN8vVYscianQbNmxAREQE5s6di5iYGPj7+yM8PBw5OTl3PC8lJQWzZs3CAw880ExJm9bmW3Pvhno7ws7CROQ0RNScWPBSi+btbIXvpwTDSm6E6CvXMe37kyirYNFLhmXRokWYOnUqJk+eDF9fXyxfvhzm5uZYtWpVneeo1WpMmDAB77//Pjw9PZsxbdPQaARsi60qeMdyOANRi8OCl1q8Lm0UWDOlB8xNZDiYkIcxXx/B7ovZEARB7GhE9628vBynTp1CWFiYdp9UKkVYWBiOHj1a53nz5s2Do6Mjnn766eaI2eSOJecj89ZSwg/6cClhopaGBS8RgCB3O3w3qTus5EaIy1TimXUnMXrpYeyLz2HhS3otLy8ParUaTk5ONfY7OTkhKyur1nMOHTqElStXYsWKFfX+HJVKBaVSWWPTJZtvzb37UFdXLiVM1AKx4CW6pXd7e+x/fQCe698eZsYynEkvxFOrT+B/y47gUEIeC19qEYqKivDkk09ixYoVsLe3r/d5kZGRUCgU2s3Nza0JUzbMzXI1/jqXCYDDGYhaKha8RP9iZ2GCN4f64OAbA/BM33aQG0kRk1qAJ1Yex7hvj+FYcr7YEYkaxN7eHjKZDNnZ2TX2Z2dnw9nZ+bbjk5KSkJKSghEjRsDIyAhGRkZYt24dtm/fDiMjIyQlJdX6ObNnz0ZhYaF2S0tLa5L7uRc7L2ahpFwNNzsuJUzUUrHgJaqFvaUc7zzki4OvD8BTvT1gIpMi+sp1PPbtMUz47hhOXb0udkSiejExMUFQUBCioqK0+zQaDaKiohASEnLb8T4+Pjh37hxiY2O128iRIzFgwADExsbW2XMrl8thbW1dY9MV1bMzjAloDYmESwkTtURGYgcg0mWO1qZ4b6Qfnu3viaV7E7HhRBoOJ+bjcOJR9PdywCuDvBDgZiN2TKI7ioiIwKRJk9C9e3f07NkTixcvRklJCSZPngwAmDhxIlq3bo3IyEiYmpqic+fONc63sbEBgNv264OcojIcTMgFAIzp1kbkNEQkFha8RPXgojDDgtFd8Gy/9li6NxG/nErH/su52H85FwN9HPHKIC90bq0QOyZRrcaNG4fc3FzMmTMHWVlZCAgIwI4dO7QPsqWmpkIqNcwv/LbHZkAjAIFtbdDO3kLsOEQkEonAJ3Fuo1QqoVAoUFhYqFNfy5HuuJpfgi+jErHldDo0t/4PCvdzwiuDvODjzL8zdHeG3s7oyv0N++IgLmYqMX+UH54M8RAtBxE1voa0M4b5Iz1RE3NvZYHPHvXH7oj+GBXgCokE+PtCNoYsPogZP8UgIbtI7IhELV58VhEuZiphLONSwkQtnc4XvB4eHpBIJLdtM2bMqPX4NWvW3HasqalpM6emlsLTwRJfPBaInS/3w/AuLgCAP85mYvDiA3h5/Wkk5xaLnJCo5dp8Oh1A1VLCtlxKmKhF0/kxvCdOnIBa/c9Sr+fPn8egQYPwyCOP1HmOtbU14uPjta/5VC41tY5OVlg6oRteyFRi8e7L+PtCNrbGZmD7mQyM7dYGLz3YEW1bmYsdk6jFUGsEbDudAQAYG8i5d4laOp0veB0cHGq8/uijj9C+fXv079+/znMkEkmt80sSNbVOLtb45snuOH+tEJ/vuoyoSzn49VQ6tp6+hke6t8GMAR3QxpaFL1FTO5acjyxlGaxNjfBgJy4lTNTS6fyQhn8rLy/HDz/8gClTptyx17a4uBju7u5wc3PDqFGjcOHChTteV9eXxCT907m1Aiuf6oEtz/dGPy8HVGoE/BydhgGf7sM7W88hq7BM7IhEBm1TTNVwhof8XSE34lLCRC2dXhW8W7duRUFBAZ566qk6j/H29saqVauwbds2/PDDD9BoNOjduzfS09PrPEeXl8Qk/RbY1hbrpvTEr8+FoHf7VqhQC/jhWCr6fbIX722/gJwiFr5Eja20vBI7zmcB4HAGIqqiV9OShYeHw8TEBL/99lu9z6moqECnTp0wfvx4zJ8/v9ZjVCoVVCqV9rVSqYSbm5vo0+mQ4TmalI/Pd11GdErVSm2mxlI82csd00M7wI4P1bQoujJtV1MR8/62nr6GlzfEoq2dOfa/FsrnOIgMVEPaGZ0fw1vt6tWr2L17NzZv3tyg84yNjREYGIjExMQ6j5HL5ZDL5fcbkeiuQtq3Qi/PXjicmI/PdsXjdGoBVhy8gp+j0/DMA+3wdN92sDI1FjsmkV7bfLpqKeHRgVxKmIiq6M2QhtWrV8PR0RHDhw9v0HlqtRrnzp2Di4tLEyUjahiJRIK+He2xeXpvrH6qB/xcrVGsqsTi3Qnot3AvvjuYjLIK9d0vRES3yVGW4dCtpYQ5nIGIqulFwavRaLB69WpMmjQJRkY1O6UnTpyI2bNna1/PmzcPO3fuRHJyMmJiYvDEE0/g6tWreOaZZ5o7NtEdSSQSDPBxxG8v9MXSx7vB094CN0orsOCPOIR+sg8/R6eiQq0ROyaRXtl2aynhbm1t4MGlhInoFr0Y0rB7926kpqZiypQpt7333zXgb9y4galTpyIrKwu2trYICgrCkSNH4Ovr25yRiepNKpVgeFcXhPs5YVNMOr7YnYCMwjLM3nwO3x5IxiuDvPBQFxdIpfxqluhuqoczjOnWRuQkRKRL9OqhteZi6A+TkG4rq1Djp+OpWLo3Efkl5QCq5vd9LdwLA7wdOSbRQBh6OyPG/V3KUmLI4oMwlkkQ/VYYV1cjMnANaWf0YkgDUUtiaizDlL7tsP/1AXh1kBes5EaIy1RiypqTeGT5URxPzhc7IpFO2hJT1bs7gEsJE9F/sOAl0lGWciO8OLAjDrw+AM/284TcSIqTV29g3LfHMGlVNM5fKxQ7IpHOUGsEbI2tKnjHcjgDEf0HC14iHWdrYYLZwzrhwOsDMCG4LYykEuy/nIuHvjqEGT/GICm3WOyIRKI7kpSHbKUKCjNjDPBxuPsJRNSisOAl0hNO1qb4YEwXRL3aH6MDXCGRAH+cy8SgRfvx+q9ncK3gptgRiURTPZzhoa4uXEqYiG7DgpdIz7i3ssDixwLx18wHENbJCRoB2HgyHQM+2Yf3f7uAvGLV3S9CZEBKyyux48KtpYS7ce5dIrodC14iPeXjbI3vJnXHpum90cvTDuVqDVYfTkG/hXvx2c54KMsqxI5I1Cz+vpCF0nI13FuZo1tbW7HjEJEOYsFLpOeC3G3x89Re+OHpYPi3UaC0XI2v9iSi38K9+GZ/Em6Wc9U2Mmybbw1nGMOlhImoDix4iQxA9XLFW2f0wfIngtDR0RIFpRWI/OsS+n+yF98fu4rySq7aRoYnW1mGw4l5AKoKXiKi2rDgJTIgEokEQzo7Y8fL/fDZI/5oY2uGnCIV3t16HmGL9mPL6XSoNVxrhgzHtthr0AhV33S4t+JSwkRUOxa8RAZIJpXgf0FtEPVqf7w/0g/2lnKkXi/FKxvOYNgXB/HH2UwWvmQQ/j2cgYioLix4iQyY3EiGSb09cOD1ULwW7g1rUyPEZxdhxk8xGPDpPnx/NIVjfElvxWUqcSmrCCYyKR7q6iJ2HCLSYSx4iVoAcxMjzBjQAQdffxAzB3aErbkxUq+X4t1tF9Dn4z1YvPsyrpeUix2TqEG2nK7q3X3QxxE25lxKmIjqxoKXqAVRmBvjlUFeOPLmQMwb5Qc3OzNcLynH4t0J6P1RFOZsO4/U/FKxYxLdlVojYOutgncM594lortgwUvUApmZyDAxxAN7Xw3FkscD0aW1AmUVGqw7ehWhn+7FjJ9icDa9QOyYRHU6nJiHnCIVbMyNMcDbUew4RKTjjMQOQETiMZJJ8VBXVwzv4oKjyfn4Zn8y9l/OxR9nM/HH2UyEeLbCs/090d/LgfObkk6pHs7wUFcXmBix74aI7owFLxFBIpGgd3t79G5vj7hMJVYcSMb2Mxk4mpyPo8n58HG2wrR+nhjh7wpjGYsLEleJqhI7zlctJTwmsI3IaYhIH/BfLiKqoZOLNRaNC8D+1wfg6b7tYGEiw6WsIkRsPIP+C/fiu4PJKFZVih2TWrC/L2ThZoUaHq3M0a2tjdhxiEgPsOAlolq1tjHDuw/54sibA/FauDfsLeXIKCzDgj/iEBIZhYU7LiGnqEzsmNQCVQ9nGBPYhkNtiKheWPAS0R0pzI0xY0AHHHpjAD4a2wWeDhYoKqvE1/uS0PejvXhz01kk5hSLHZNaiKzCMhziUsJE1EAseImoXkyNZXisZ1vsfqU/vn0yCN3dbVGu1mD9iTSELdqPqetO4mTKdbFjkoHbFnsNggB0d7dF21bmYschIj3Bh9aIqEGkUgkG+zljsJ8zTqZcxzcHkrHrYrZ2C3K3xbR+nhjUyQlSKb9upsa1hXPvEtE9YMFLRPesu4cdunvYITGnGN8dTMbmmGs4dfUGnv3+FDwdLDDtAU+MDmwNU2OZ2FHJAFzM+NdSwl1cxY5DRHqEQxqI6L51cLTER//rikNvDMDzoe1hZWqE5NwSvLn5HPp+vBfL9iXhZrla7Jik57acTgcADOzkCIW5schpiEifsOAlokbjaG2K14f44OjsgXhneCe4KEyRV6zCxzsuYcCn+/DLyTSoNYLYMUkPVao12BqbAYAPqxFRw7HgJaJGZyk3wjMPeOLA6wPw2SP+aG1jhixlGV779Swe+uoQDibkih2R9MzhpHzkFqlga26MUC4lTEQNxIKXiJqMsUyK/wW1QdSr/fHWMB9YmRohLlOJJ1dGY9KqaFzKUoodkfTElpiq4QwPdXXlUsJE1GBsNYioyZkayzCtX3sceG0ApvRpB2OZBPsv52LYFwfxxq9nka3kAhZUtxJVJf6+kA0AGMvZGYjoHrDgJaJmY2thgjkjfLHrlf4Y1sUZGgHYcDINoZ/sw6Jdl1HCJYupFjvOVy0l3M7eAgFuNmLHISI9xIKXiJqdh70Fvp4QhE3TeyPI3RY3K9T4MioB/T/Zh5+Op6JSrRE7IumQzbdmZxgT2JpLCRPRPWHBS0SiCXK3xa/PhWDZhG7waGWOvGIV3tpyDkO/OIg9l7IhCJzRoaXLLLyJI0n5ADg7AxHdOxa8RCQqiUSCoV1csPOV/pg7whe25sZIyCnGlDUnMeG74zh/rVDsiCSibbEZEASgp4cd3Oy4lDAR3RsWvESkE0yMpJjcpx32vTYAz/b3hImRFEeS8vHQV4cQsSEW1wpuih2RmpkgCNgSw6WEiej+seAlIp2iMDPG7KGdsOfV/hgdULV87ObT1zDg0334eMclKMsqRE6on5YuXQoPDw+YmpoiODgY0dHRdR67efNmdO/eHTY2NrCwsEBAQAC+//77Zkxb5WKmEvHZRTAxkmJYF5dm/3wiMhwseIlIJ7WxNcfixwKx/YU+6OVph/JKDZbtS0LoJ/uw5vAVVPDBtnrbsGEDIiIiMHfuXMTExMDf3x/h4eHIycmp9Xg7Ozu8/fbbOHr0KM6ePYvJkydj8uTJ+Pvvv5s19+ZbvbthnRyhMONSwkR07yQCnwq5jVKphEKhQGFhIaytrcWOQ9TiCYKAqLgcRP4Vh6TcEgBAO3sLvDHEB+F+Tnr55H5ztjPBwcHo0aMHlixZAgDQaDRwc3PDiy++iDfffLNe1+jWrRuGDx+O+fPn1+v4+72/SrUGvSL3IK9YhRUTu2OQr1ODr0FEhq0h7YxO9/C+9957kEgkNTYfH587nvPLL7/Ax8cHpqam6NKlC/78889mSktETUUikSDM1wl/v9wPC0Z3hr2lCa7kleC5H07hkeVHcTr1htgRdVZ5eTlOnTqFsLAw7T6pVIqwsDAcPXr0rucLgoCoqCjEx8ejX79+dR6nUqmgVCprbPfjUGIe8opVsLMwQX8vh/u6FhGRThe8AODn54fMzEztdujQoTqPPXLkCMaPH4+nn34ap0+fxujRozF69GicP3++GRMTUVMxkknxRC937HttAF58sANMjaU4efUGxnx9BDN+ikFqfqnYEXVOXl4e1Go1nJxq9pA6OTkhKyurzvMKCwthaWkJExMTDB8+HF999RUGDRpU5/GRkZFQKBTazc3N7b5ybzldNZxhRFcXLiVMRPdN51sRIyMjODs7azd7e/s6j/3iiy8wZMgQvPbaa+jUqRPmz5+Pbt26ab/GIyLDYCk3wquDvbFv1gA8EtQGEgnwx9lMDFy0D/N/v4iC0nKxI+o9KysrxMbG4sSJE/jggw8QERGBffv21Xn87NmzUVhYqN3S0tLu+bOLVZX4+0JVMT6mW5t7vg4RUTUjsQPcTUJCAlxdXWFqaoqQkBBERkaibdu2tR579OhRRERE1NgXHh6OrVu33vEzVCoVVCqV9vX9fhVHRM3DWWGKTx7xx+Q+7RD5VxwOJuRh5aEr+OVkGgb7OaOHhy16eNihnb2FXo7zbQz29vaQyWTIzs6usT87OxvOzs51nieVStGhQwcAQEBAAOLi4hAZGYnQ0NBaj5fL5ZDL5Y2S+a9zmSir0MDT3gL+bRSNck0iatl0uoc3ODgYa9aswY4dO7Bs2TJcuXIFDzzwAIqKimo9Pisrq8Ff2wGN/1UcETUvX1drfP90MNZO6QkfZysoyyrx66l0vLHpHB78bD96fLAbz31/CisPXcG59MIWtXSxiYkJgoKCEBUVpd2n0WgQFRWFkJCQel9Ho9HU6BhoStXDGbiUMBE1Fp3u4R06dKj21127dkVwcDDc3d2xceNGPP300432ObNnz67RM6xUKln0Eumh/l4O6NvBHocT83D8Sj5OXLmB2PQC5BWXY8eFLOy49TW5hYkM3dxt0d3dDj3a2SLQzRZmJjKR0zediIgITJo0Cd27d0fPnj2xePFilJSUYPLkyQCAiRMnonXr1oiMjARQ1QnQvXt3tG/fHiqVCn/++Se+//57LFu2rMmzZhbexNHkqqWER3MpYSJqJDpd8P6XjY0NvLy8kJiYWOv7zs7ODf7aDmjcr+KISFwyqQT9vBzQ79aT/apKNc6lFyI65TpOptzAyZTrUJZV4mBCHg4m5AEAjKQSdG6tQM92dujhYYfu7rawtTAR8zYa1bhx45Cbm4s5c+YgKysLAQEB2LFjh/YbsdTUVEil/3zhV1JSgueffx7p6ekwMzODj48PfvjhB4wbN67Js249fWsp4XZcSpiIGo9ezcNbXFyMtm3b4r333sNLL7102/vjxo1DaWkpfvvtN+2+3r17o2vXrli+fHm9P4fz8BIZLo1GQHx2EU6mXEd0yg2cuHIdWcqy247r6GiJ7h526NmuahxwG9vGLb4MvZ25l/sTBAGDPz+AhJxifDS2Cx7rWfvzGkREQMPaGZ3u4Z01axZGjBgBd3d3ZGRkYO7cuZDJZBg/fjyA27+GmzlzJvr374/PPvsMw4cPx/r163Hy5El8++23Yt4GEekQqVSCTi7W6ORijSdDPCAIAtJv3MSJlOu3thtIzClGwq3t5+hUAICrwhTdPezQo50denjYwsvRClIpx5c2pgsZSiTkFMPESIqhXEqYiBqRThe86enpGD9+PPLz8+Hg4IC+ffvi2LFjcHCo+qryv1/D9e7dGz/99BPeeecdvPXWW+jYsSO2bt2Kzp07i3ULRKTjJBIJ3OzM4WZnjrG3psC6XlKOk7cK4OiUG7hwrRAZhWXYfiYD289kAAAUZsbo7m6r7QXu3FoBuZHhjgNuDtVLCQ/q5MSlhImoUenVkIbmYuhfNRJRw5SWVyI2tUA7Djgm9QZKy9U1jpEbSfHbi33h5WRVr2saejvT0Pv791LCKyd1x8BOXEqYiO7MYIY0EBHpAnMTI/TuYI/eHaoWvqlQa3AxQ6kdBnEy5QZKyivh0cpC5KT6K+3GTRjLJGhlYaJ94JCIqLGw4CUiaiBjmRT+bjbwd7PBMw94QhAEZBaWcQnc+9DO3gKH33gQqddLYSzj7yMRNS62KkRE90kikcDVxkzsGHpPKpXAw5695ETU+FjwEhEREZFBY8FLRERERAaNBS8RERERGTQWvERERERk0FjwEhEREZFBY8FLRERERAaNBS8RERERGTQWvERERERk0FjwEhEREZFBY8FLRERERAaNBS8RERERGTQWvERERERk0IzEDqCLBEEAACiVSpGTEJGhqm5fqtsbQ8N2lIiaWkPaURa8tSgqKgIAuLm5iZyEiAxdUVERFAqF2DEaHdtRImou9WlHJYKhdi/cB41Gg4yMDFhZWUEikdTrHKVSCTc3N6SlpcHa2rqJE4qD92gYeI+6QRAEFBUVwdXVFVKp4Y0uYztaO96jYeA96oaGtKPs4a2FVCpFmzZt7ulca2trnf2L0Vh4j4aB9yg+Q+zZrcZ29M54j4aB9yi++rajhtetQERERET0Lyx4iYiIiMigseBtJHK5HHPnzoVcLhc7SpPhPRoG3iPpqpbw58Z7NAy8R/3Dh9aIiIiIyKCxh5eIiIiIDBoLXiIiIiIyaCx4iYiIiMigseAlIiIiIoPGgrcRLF26FB4eHjA1NUVwcDCio6PFjtRoIiMj0aNHD1hZWcHR0RGjR49GfHy82LGa1EcffQSJRIKXX35Z7CiN6tq1a3jiiSfQqlUrmJmZoUuXLjh58qTYsRqNWq3Gu+++i3bt2sHMzAzt27fH/Pnz67XGOukGtqWGg+2ofjLkdpQF733asGEDIiIiMHfuXMTExMDf3x/h4eHIyckRO1qj2L9/P2bMmIFjx45h165dqKiowODBg1FSUiJ2tCZx4sQJfPPNN+jatavYURrVjRs30KdPHxgbG+Ovv/7CxYsX8dlnn8HW1lbsaI3m448/xrJly7BkyRLExcXh448/xsKFC/HVV1+JHY3qgW2p4WA7qr8Muh0V6L707NlTmDFjhva1Wq0WXF1dhcjISBFTNZ2cnBwBgLB//36xozS6oqIioWPHjsKuXbuE/v37CzNnzhQ7UqN54403hL59+4odo0kNHz5cmDJlSo19Y8eOFSZMmCBSImoItqWGge2ofjPkdpQ9vPehvLwcp06dQlhYmHafVCpFWFgYjh49KmKyplNYWAgAsLOzEzlJ45sxYwaGDx9e48/TUGzfvh3du3fHI488AkdHRwQGBmLFihVix2pUvXv3RlRUFC5fvgwAOHPmDA4dOoShQ4eKnIzuhm2p4WA7qt8MuR01EjuAPsvLy4NarYaTk1ON/U5OTrh06ZJIqZqORqPByy+/jD59+qBz585ix2lU69evR0xMDE6cOCF2lCaRnJyMZcuWISIiAm+99RZOnDiBl156CSYmJpg0aZLY8RrFm2++CaVSCR8fH8hkMqjVanzwwQeYMGGC2NHoLtiWGga2o/rPkNtRFrxUbzNmzMD58+dx6NAhsaM0qrS0NMycORO7du2Cqamp2HGahEajQffu3fHhhx8CAAIDA3H+/HksX77cYBrqjRs34scff8RPP/0EPz8/xMbG4uWXX4arq6vB3CMZBkNsS9mOGkYbY8jtKAve+2Bvbw+ZTIbs7Owa+7Ozs+Hs7CxSqqbxwgsv4Pfff8eBAwfQpk0bseM0qlOnTiEnJwfdunXT7lOr1Thw4ACWLFkClUoFmUwmYsL75+LiAl9f3xr7OnXqhE2bNomUqPG99tprePPNN/HYY48BALp06YKrV68iMjJS7xtqQ8e2VP+xHTUMhtyOcgzvfTAxMUFQUBCioqK0+zQaDaKiohASEiJissYjCAJeeOEFbNmyBXv27EG7du3EjtToBg4ciHPnziE2Nla7de/eHRMmTEBsbKzeN9IA0KdPn9umQLp8+TLc3d1FStT4SktLIZXWbNJkMhk0Go1Iiai+2JbqP7ajhsGg21Gxn5rTd+vXrxfkcrmwZs0a4eLFi8K0adMEGxsbISsrS+xojWL69OmCQqEQ9u3bJ2RmZmq30tJSsaM1KUN7ujg6OlowMjISPvjgAyEhIUH48ccfBXNzc+GHH34QO1qjmTRpktC6dWvh999/F65cuSJs3rxZsLe3F15//XWxo1E9sC01PGxH9Y8ht6MseBvBV199JbRt21YwMTERevbsKRw7dkzsSI0GQK3b6tWrxY7WpAytoRYEQfjtt9+Ezp07C3K5XPDx8RG+/fZbsSM1KqVSKcycOVNo27atYGpqKnh6egpvv/22oFKpxI5G9cS21LCwHdU/htyOSgTBAJbPICIiIiKqA8fwEhEREZFBY8FLRERERAaNBS8RERERGTQWvERERERk0FjwEhEREZFBY8FLRERERAaNBS8RERERGTQWvET/IpFIsHXr1ib9jDVr1sDGxqZJP0MfeHh4YPHixWLHIKJGxna0+bAdrT8WvFSrrKwsvPjii/D09IRcLoebmxtGjBhRY617Q5SZmYmhQ4c22vVqa4zGjRuHy5cvN9pn1CU0NBQSiQQSiQSmpqbw8vJCZGQkuNYMUfNgO9o42I5SYzASOwDpnpSUFPTp0wc2Njb45JNP0KVLF1RUVODvv//GjBkzcOnSJbEjNhlnZ+cm/wwzMzOYmZk1+ecAwNSpUzFv3jyoVCrs2bMH06ZNg42NDaZPn94sn0/UUrEdbVpsR6nBxF3ZmHTR0KFDhdatWwvFxcW3vXfjxg3tr69evSqMHDlSsLCwEKysrIRHHnlEyMrK0r4/d+5cwd/fX1i5cqXg5uYmWFhYCNOnTxcqKyuFjz/+WHBychIcHByEBQsW1PgMAMLy5cuF4cOHC2ZmZoKPj49w5MgRISEhQejfv79gbm4uhISECImJidpzJk2aJIwaNarGdWbOnCn0799f+7p///7Ciy++KLz22muCra2t4OTkJMydO/e2z96yZYv2dVpamvDYY48Jtra2grm5uRAUFCQcO3ZMEARBSExMFEaOHCk4OjoKFhYWQvfu3YVdu3bV+DwANTZBEITVq1cLCoWixud+/fXXgqenp2BsbCx4eXkJ69atuy3XihUrhNGjRwtmZmZChw4dhG3btt325/Nvta1j361bN2HMmDHa19evXxeefPJJwcbGRjAzMxOGDBkiXL58Wft+9Z/hv33++eeCu7u79nX17/0nn3wiODs7C3Z2dsLzzz8vlJeXa4/Jzs4WHnroIcHU1FTw8PAQfvjhB8Hd3V34/PPP73gPRPqK7egW7Wu2o2xHdQGHNFAN169fx44dOzBjxgxYWFjc9n71mCmNRoNRo0bh+vXr2L9/P3bt2oXk5GSMGzeuxvFJSUn466+/sGPHDvz8889YuXIlhg8fjvT0dOzfvx8ff/wx3nnnHRw/frzGefPnz8fEiRMRGxsLHx8fPP7443j22Wcxe/ZsnDx5EoIg4IUXXmjw/a1duxYWFhY4fvw4Fi5ciHnz5mHXrl21HltcXIz+/fvj2rVr2L59O86cOYPXX38dGo1G+/6wYcMQFRWF06dPY8iQIRgxYgRSU1MBAJs3b0abNm0wb948ZGZmIjMzs9bP2bJlC2bOnIlXX30V58+fx7PPPovJkydj7969NY57//338eijj+Ls2bMYNmwYJkyYgOvXr9frvgVBwMGDB3Hp0iWYmJho9z/11FM4efIktm/fjqNHj0IQBAwbNgwVFRX1um61vXv3IikpCXv37sXatWuxZs0arFmzpsbnpKWlYe/evfj111/x9ddfIycnp0GfQaQv2I7+g+1o/bEdbWJiVtuke44fPy4AEDZv3nzH43bu3CnIZDIhNTVVu+/ChQsCACE6OloQhKqfas3NzQWlUqk9Jjw8XPDw8BDUarV2n7e3txAZGal9DUB45513tK+PHj0qABBWrlyp3ffzzz8Lpqam2tf17Zno27dvjWN69OghvPHGGzU+u7pn4ptvvhGsrKyE/Pz8O/5e/Jufn5/w1VdfaV/X9tP3f3smevfuLUydOrXGMY888ogwbNiwGrn+/XtSXFwsABD++uuvOrP0799fMDY2FiwsLARjY2MBgGBqaiocPnxYEARBuHz5sgBA+1oQBCEvL08wMzMTNm7cKAhC/Xsm3N3dhcrKyhr5x40bJwiCIMTHx9f4eyEIghAXFycAYM8EGSS2o2xH2Y7qHvbwUg1CPQfix8XFwc3NDW5ubtp9vr6+sLGxQVxcnHafh4cHrKystK+dnJzg6+sLqVRaY99/f0rt2rVrjfcBoEuXLjX2lZWVQalU1vPObr8uALi4uNT5E3JsbCwCAwNhZ2dX6/vFxcWYNWsWOnXqBBsbG1haWiIuLk7bM1FfcXFx6NOnT419ffr0qfH7+N/sFhYWsLa2vutP9xMmTEBsbCwOHz6MoUOH4u2330bv3r21n2tkZITg4GDt8a1atYK3t/dtn303fn5+kMlk2tf//n2t/pygoCDt+z4+PnzCmgwW29F/sB2tP7ajTYsPrVENHTt2hEQiabQHKoyNjWu8lkgkte6r/nqrtvMkEkmd+6rPk0qlt/0jU9vXSfX57Gp3eyBi1qxZ2LVrFz799FN06NABZmZmePjhh1FeXn7H8+5VQ7JXUygU6NChAwBg48aN6NChA3r16oWwsLB6fWZT/L4SGTq2o/9gO8p2VFewh5dqsLOzQ3h4OJYuXYqSkpLb3i8oKAAAdOrUCWlpaUhLS9O+d/HiRRQUFMDX17e54mo5ODjcNrYrNjb2vq7ZtWtXxMbG1jm+6/Dhw3jqqacwZswYdOnSBc7OzkhJSalxjImJCdRq9R0/p1OnTjh8+PBt127s30dLS0vMnDkTs2bNgiAI6NSpEyorK2uM+8vPz0d8fLz2sx0cHJCVlVWjsW7o76uPjw8qKytx6tQp7b74+Hjt3yUiQ8N29B9sR9mO6goWvHSbpUuXQq1Wo2fPnti0aRMSEhIQFxeHL7/8EiEhIQCAsLAwdOnSBRMmTEBMTAyio6MxceJE9O/fH927d2/2zA8++CBOnjyJdevWISEhAXPnzsX58+fv65rjx4+Hs7MzRo8ejcOHDyM5ORmbNm3C0aNHAVT14mzevBmxsbE4c+YMHn/88dt+Gvfw8MCBAwdw7do15OXl1fo5r732GtasWYNly5YhISEBixYtwubNmzFr1qz7yl+bZ599FpcvX8amTZvQsWNHjBo1ClOnTsWhQ4dw5swZPPHEE2jdujVGjRoFoGoOytzcXCxcuBBJSUlYunQp/vrrrwZ9pre3N4YMGYJnn30Wx48fx6lTp/DMM88025RCRGJgO1qF7SjbUV3Bgpdu4+npiZiYGAwYMACvvvoqOnfujEGDBiEqKgrLli0DUPVVy7Zt22Bra4t+/fohLCwMnp6e2LBhgyiZw8PD8e677+L1119Hjx49UFRUhIkTJ97XNU1MTLBz5044Ojpi2LBh6NKlCz766CPtGKtFixbB1tYWvXv3xogRIxAeHo5u3brVuMa8efOQkpKC9u3bw8HBodbPGT16NL744gt8+umn8PPzwzfffIPVq1cjNDT0vvLXxs7ODhMnTsR7770HjUaD1atXIygoCA899BBCQkIgCAL+/PNP7VdrnTp1wtdff42lS5fC398f0dHR9/QPyOrVq+Hq6or+/ftj7NixmDZtGhwdHRv79oh0BtvRKmxH2Y7qColQ39H1RERERER6iD28RERERGTQWPASERERkUFjwUtEREREBo0FLxEREREZNBa8RERERGTQWPASERERkUFjwUtEREREBo0FLxEREREZNBa8RERERGTQWPASERERkUFjwUtEREREBo0FLxEREREZtP8DuD38LmzAbcIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x450 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "from fedlab.utils.functional import evaluate\n",
    "from fedlab.core.standalone import StandalonePipeline\n",
    "\n",
    "from torch import nn\n",
    "from torch.utils.data import DataLoader\n",
    "import torchvision\n",
    "\n",
    "class EvalPipeline(StandalonePipeline):\n",
    "    def __init__(self, handler, trainer, test_loader):\n",
    "        super().__init__(handler, trainer)\n",
    "        self.test_loader = test_loader \n",
    "        self.loss = []\n",
    "        self.acc = []\n",
    "        \n",
    "    def main(self):\n",
    "        t=0\n",
    "        while self.handler.if_stop is False:\n",
    "            # server side\n",
    "            sampled_clients = self.handler.sample_clients()\n",
    "            broadcast = self.handler.downlink_package\n",
    "            \n",
    "            # client side\n",
    "            self.trainer.local_process(broadcast, sampled_clients)\n",
    "            uploads = self.trainer.uplink_package\n",
    "\n",
    "            # server side\n",
    "            for pack in uploads:\n",
    "                self.handler.load(pack)\n",
    "\n",
    "            loss, acc = evaluate(self.handler.model, nn.CrossEntropyLoss(), self.test_loader)\n",
    "            print(\"Round {}, Loss {:.4f}, Test Accuracy {:.4f}\".format(t, loss, acc))\n",
    "            t+=1\n",
    "            self.loss.append(loss)\n",
    "            self.acc.append(acc)\n",
    "    \n",
    "    def show(self):\n",
    "        plt.figure(figsize=(8,4.5))\n",
    "        ax = plt.subplot(1,2,1)\n",
    "        ax.plot(np.arange(len(self.loss)), self.loss)\n",
    "        ax.set_xlabel(\"Communication Round\")\n",
    "        ax.set_ylabel(\"Loss\")\n",
    "        \n",
    "        ax2 = plt.subplot(1,2,2)\n",
    "        ax2.plot(np.arange(len(self.acc)), self.acc)\n",
    "        ax2.set_xlabel(\"Communication Round\")\n",
    "        ax2.set_ylabel(\"Accuarcy\")\n",
    "        \n",
    "        \n",
    "test_data = torchvision.datasets.MNIST(root=\"../datasets/mnist/\",\n",
    "                                       train=False,\n",
    "                                       transform=transforms.ToTensor())\n",
    "test_loader = DataLoader(test_data, batch_size=1024)\n",
    "\n",
    "standalone_eval = EvalPipeline(handler=handler, trainer=trainer, test_loader=test_loader)\n",
    "standalone_eval.main()\n",
    "\n",
    "standalone_eval.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cross-process\n",
    "\n",
    "Due to the jupyter doesn't support multi-process program, we provide only the description of Cross-process mode and Hierachical mode in this part.\n",
    "\n",
    "For runable scripts, please see our examples in [cross-process](https://github.com/SMILELab-FL/FedLab/tree/master/examples/cross-process-mnist) and [scale](https://github.com/SMILELab-FL/FedLab/tree/master/examples/scale-mnist)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fedlab.core import DistNetwork\n",
    "from fedlab.core.client.manager import PassiveClientManager\n",
    "\n",
    "# Client side. Put your trainer into a network manager.\n",
    "\n",
    "args.ip = \"127.0.0.1\"\n",
    "args.port = 3002\n",
    "args.rank = 1\n",
    "args.world_size = 2 # world_size = the number of client manager + 1 (server)\n",
    "\n",
    "args.ethernet = None\n",
    "\n",
    "client_network = DistNetwork(\n",
    "    address=(args.ip, args.port),\n",
    "    world_size=args.world_size,\n",
    "    rank=args.rank,\n",
    "    ethernet=args.ethernet,\n",
    ")\n",
    "\n",
    "# trainer can be ordinary trainer or serial trainer.\n",
    "client_manager = PassiveClientManager(trainer=trainer,\n",
    "                                network=client_network)\n",
    "\n",
    "# Server side. Put your handler into a network manager.\n",
    "from fedlab.core.server import SynchronousServerManager\n",
    "\n",
    "server_network = DistNetwork(address=(args.ip, args.port),\n",
    "                      world_size=args.world_size,\n",
    "                      rank=0, # the rank of server is 0 as default\n",
    "                      ethernet=args.ethernet)\n",
    "\n",
    "server_manager = SynchronousServerManager(handler=handler,\n",
    "                                    network=server_network,\n",
    "                                    mode=\"GLOBAL\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hierachical\n",
    "\n",
    "Hierachical is a complex network topology implementation in FedLab. Hierarchical mode for FedLab is designed for situation tasks on multiple computer clusters (in different LAN) or the real-world scenes. To enable the inter-connection for different computer clusters, FedLab develops Scheduler as middle-server process to connect client groups. Each Scheduler manages the communication between the global server and clients in a client group. And server can communicate with clients in different LAN via corresponding Scheduler. The computation mode of a client group for each scheduler can be either standalone or cross-process.\n",
    "\n",
    "A hierarchical FL system with K client groups is depicted as below. For a runable scripts, please see [here](https://github.com/SMILELab-FL/FedLab/tree/master/examples/hierarchical-hybrid-mnist)\n",
    "\n",
    "![](./docs/imgs/fedlab-hierarchical.svg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "vscode": {
   "interpreter": {
    "hash": "019ae50596e3d4df627f3288be8543f4b17347150bdb9d2aa2e7c637014aee00"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}