time_series_seq2seq.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multivariate Time Series forecast using seq2seq in TensorFlow\n",
    "\n",
    "This notebook implements the cast study of applying seq2seq model for time series data. \n",
    "\n",
    "The purpose is to showcase the effectiveness of seq2seq to learn the true patterns under the noisy signals. In addition, we are able to implement models with flexibility such as: \n",
    "\n",
    "- Variable input and output sequence lengths\n",
    "- Variable numbers of input and output signals \n",
    "\n",
    "This tutorial is divided into four parts - first we will be demonstrating how to train a basicseq2seq model on univariate data. The model is then easily applied to multivariate cases. We will then discuss about situation with outliers. And finally, we will showcase a real-world dataset to forecast pollution (pm2.5) in Beijing. \n",
    "\n",
    "To see the comprehensive explanations of each step, can jump to the post associated with this study - \n",
    "weiminwang.blog/2017/09/29/multivariate-time-series-forecast-using-seq2seq-in-tensorflow/\n",
    "\n",
    "This is a long notebook - you can choose the session of your interests by clicking on the links below: \n",
    "\n",
    "## Contents\n",
    "\n",
    "### 1) <b>[Univariate time series](#session1)</b> \n",
    "\n",
    "### 2) <b>[Multivariate time series](#session2)</b> \n",
    "\n",
    "### 3) <b>[Seq2seq for outliers/extreme events](#session3)</b> \n",
    "\n",
    "### 4) <b>[A case study - Beijing pollution data](#session4)</b> \n",
    "data credits go to UCI - https://archive.ics.uci.edu/ml/datasets/Beijing+PM2.5+Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='session1'></a>\n",
    "# Univariate time series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np \n",
    "import random\n",
    "import math\n",
    "from matplotlib import pyplot as plt\n",
    "import os\n",
    "import copy\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<a id='session2'></a>\n",
    "# Multivariate time series: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XdcleX/x/HXhyWiCCogCiJuBbeomCNNrTRXw5V903Jk49fe89v8Vt922bCs1Kw0rRyZqamlKSqaiHsPHAwHggMZ1++Pc+sXjanAfYDP8/HgweG+r3Pu97k5fLjOdd/nusUYg1JKqbLLxe4ASimlipcWeqWUKuO00CulVBmnhV4ppco4LfRKKVXGaaFXSqkyTgu9KjIisldEepbwNv8tIt8Uon0nEdkhIqkiMrA4s+Ww7RBru64lud3CEpFNItKtBLazVERGF/d2lBb6UkVEhorIKhE5JSIJ1u17RETszpYfEflaRF65wsfoJiJxVxjlJeAjY0xlY8zPV/hYiMiDIrJbRE6KyCEReVdE3HJqa4zZb20380q3W4Bcl/1P1xgTboxZWsSRlI200JcSIvII8D7wXyAQqAGMAzoBHrncx6l7jtnlVhyLQR1g0+XcMZeMc4A2xpgqQDOgJXD/5cdTquhpoS8FRMQHR0/0HmPMDGNMinH42xgz3BiTZrX7WkQ+EZF5InIK6C4iPiIyWUQSRWSfiDwrIi5W+4uGPUQkVETM+YJmvbV+WUT+EpEUEVkgIn7Z2v/LesyjIvJMHvnHAsOBx62hiznW8r0i8oSIbABOiYibtf0G2e77tYi8IiKVgF+BWtZjpIpILauZh/UcU6xhh4hccuwC6gFzrPtXEJFaIjJbRI6JyE4RGZOt/b9FZIaIfCMiJ4GRlz6mMWaXMebE+bsAWUCDS9sVdv9mazvWeqdw2Ppnf9F+yfbzhXc7IjIFCMn2PB/PIYufiMwVkRPWc1+W7XVx4d2AiFQUkUkiclxEtojI49nfVVltHxWRDSKSLCLTRMTTWlfV2kaidf+5IhKcy75pICJ/WI+RJCLTcmqnLo8W+tKhI1ABmFWAtrcCrwLewHLgQ8AHR4G7GrgduKMQ277Vah+A453DowAiEgZ8AvwLqAVUB3L8IzbGTACmAm9aQxf9sq0eBtwA+BpjMnILYYw5BfQGDlmPUdkYc8ha3R/4HvAFZgMf5fIY9YH9QD/r/mnAd0Cc9RxuAV4TkR7Z7jYAmGE99tScHldEbrX+ESTh6NF/ltvzyEGO+zeb7kBD4FrgyYIMxxhj/sXFz/PNHJo9guN5++N4d/g0kNN8KC8AoTheP72A23JoMxi4HqgLtOB//xBdgK9wvIsKAc6Qy+8GeBlYAFTF8Tr6MI+nqApJC33p4AckZS+EIrLC6o2dEZGu2drOMsb8ZYzJAtKBIcBT1ruAvcDbOIpzQX1ljNlujDkDTAdaWctvAeYaY/60CuZzOHqzhfWBMeaA9fiXa7kxZp419j0FR7HNl4jUBjoDTxhjzhpj1gNfcPH+WWmM+dkYk5VbRmPMt9bQTSPgUyC+ENlz27/nvWiMOWWMicVRNIcV4rHzkg7UBOoYY9KNMctMzhNfDQZeM8YcN8bEAR/k0OYDY8whY8wxHENZrQCMMUeNMTONMaeNMSk4OiBX55GnDlDL+l0sv8Lnp7LRQl86HAX8so8RG2OuMsb4Wuuy/x4PZLvth6OXuC/bsn1AUCG2fSTb7dNAZet2rezbsnrcRwvxuOcdyL9Jvi7N6FnAMf9awDGrCJ136f4pcD5jzA4c4/8fF/Q+5L5/c9r+PhyZi8J/gZ3AAnEcTH4yl3YX/Z7JeX/k+BxExEtEPrOG904CfwK+kvOxo8dxDH2ttobf7izk81F50EJfOqwE0nAMI+Qne68sif/1lM4LAQ5at08BXtnWBRYi02Gg9vkfRMQLx/BNQXLltfx0HpmKeqrVQ0A1EfHOtiz7/rmcbboB9a80WDa1s90OwZEZ8v/d5Znbeof3iDGmHtAPePiSIavzDnPxkFztHNrk5hGgMdDBesdz/p3nP84SM8YcMcaMMcbUAu4CPs5+rEZdGS30pYB1sO9FHC/+W0Sksoi4iEgroFIe98vEMRzwqoh4i0gd4GHg/AHY9UBXcZzf7QM8VYhYM4C+ItJZRDxwHCzO6/UUj2OcNz/rgVtFxFVErufit/rxQHUr6xUzxhwAVgD/ERFPEWkBjCKXsficiMhoEQmwbofh2Ie/F0U+y3NWzzgcx1j++YOU64E+IlJNRAKBBy+5X577W0T6WgdABTgJZFpfl5oOPGUdWA0C7itEdm8c4/InRKQajvH+3PIMynag9jiOf1TFfhpqeaGFvpSwDqg9jOMtbgKOP+TPgCdwFKvc/B+O3t9uHAdnvwW+tB5zIY7CsQFYC8wtRJ5NwL3W4x3G8ceZ1znuE4Ew67hCXuevP4Cjh3kCx5k6F9oaY7biOHi623qcohjGGIbjYOMh4CfgBWu/FFQnIFYcZznNs76eLoJc5/2BY4jld+AtY8wCa/kUIAbYi+Mg5qVnqfwHeNbaT5ce4AXHAd5FQCqOd4wf53Lu/Es4fq97rPYzcLy7LIj3gIo43llGAfPzaNsOWCUiqTgOqD9gjNlTwO2ofIheeEQp5yMioTiKq3teZyOVNBG5GxhqjMntoKpyQtqjV0rlSkRqimPaCBcRaYxj3P0nu3OpwimpTyMqpUonDxxDhHVxDKd9T+HOKlJOQIdulFKqjNOhG6WUKuOcYujGz8/PhIaG2h1DKaVKlbVr1yYZY/zza+cUhT40NJTo6Gi7YyilVKkiIvvyb6VDN0opVeZpoVdKqTJOC71SSpVxTjFGr5Qqn9LT04mLi+Ps2bN2R3Fqnp6eBAcH4+7ufln310KvlLJNXFwc3t7ehIaGIs5/6WNbGGM4evQocXFx1K1b97IeQ4dulFK2OXv2LNWrV9cinwcRoXr16lf0rkcLvVLKVlrk83el+6hUF/oDUX+zctjdpJ8t6KypSilV/pTqQn9w0jQ6fv8pexq35mB0rN1xlFJlxOjRo9m8eXORP27lypdeKdLhzjvvJCAggGbNmhX5NqGUF/rIT15n3dufE5hwAJ9OkUS/qheOV0pduS+++IKwsLAS297IkSOZPz+v67JcmVJd6AHaPDya06vXcqB2AyKevZ81XfuSmnTc7lhKqVLg1KlT3HDDDbRs2ZJmzZoxbZrjQl3dunW7MC3LxIkTadSoEd26dWPMmDHcd5/jaoojR47k/vvv56qrrqJevXrMmDEDgNTUVHr06EGbNm1o3rw5s2bNyjdH165dqVatWjE9yzJyemVg80b4bVrDyrGP0n7KeA43ac7hSVNoeEN3u6MppQrqwQdh/fqifcxWreC993JdPX/+fGrVqsUvv/wCQHJy8kXrDx06xMsvv8y6devw9vbmmmuuoWXLlhfWHz58mOXLl7N161b69+/PLbfcgqenJz/99BNVqlQhKSmJyMhI+vfvb+tB51Lfoz/PrYIHHSd9wLbvZuOWcY7Q/r2IuucpsjL0+sJKqZw1b96cRYsW8cQTT7Bs2TJ8fC6+7vzq1au5+uqrqVatGu7u7gwaNOii9QMHDsTFxYWwsDDi4+MBx3nvTz/9NC1atKBnz54cPHjwwjq7lIkefXZhQ24gudMGYm+8lchPXmfDn0uo9fP3+DUItTuaUiovefS8i0ujRo1Yu3Yt8+bN46mnnuLaa6/l+eefv7A+vwszVahQ4R9tp06dSmJiImvXrsXd3Z3Q0FDbP/lbZnr02fkEB9J61SJWPfEqjbb9jbRqTczn39kdSynlZA4dOoSXlxe33XYbjz76KOvWrbtoffv27fnjjz84fvw4GRkZzJw5M9/HTE5OJiAgAHd3d5YsWcK+fQWaSbhYlclCDyAuLnR4/WmOLFrGySrVaDn2VqJuHEnaqdN2R1NKOYnY2Fjat29Pq1atePXVV3n22WcvWh8UFMTTTz9Nhw4d6NmzJ2FhYf8Y3rnU8OHDiY6OJiIigqlTp9KkSZN8cwwbNoyOHTuybds2goODmThx4hU9r0s5xTVjIyIiTHFeeOTsyVRiho2hw7zv2RnckAo/TKN2ZOti255SqmC2bNlC06ZN7Y6Rp9TUVCpXrkxGRgY33ngjd955JzfeeGOJ58hpX4nIWmNMRH73LbM9+uw8q1Smwy/f8fcHX+F39AjVu17F6hfewWRl2R1NKeXk/v3vf9OqVSuaNWtG3bp1GThwoN2RCq1AB2NFZC+QAmQCGcaYCBGpBkwDQoG9wGBjzHFxnEP0PtAHOA2MNMasy+lxS1rr/xtJfI9OHLxxKO1feoS1CxcQ8M5/tHevlMrVW2+9ZXeEK1aYHn13Y0yrbG8TngR+N8Y0BH63fgboDTS0vsYCnxRV2KJQI6whTTauYuWdD9Fi1SJqd2zDlgYtWf3MG6QkHLU7nlJKFbkrGboZAEyybk8CBmZbPtk4RAG+IlLzCrZT5Fzd3eg48R1ObN5O1F2P45VygvavPYlbUBDRXfqyccpPev69UqrMKGihN8ACEVkrImOtZTWMMYcBrO8B1vIg4EC2+8ZZyy4iImNFJFpEohMTEy8v/RXyb1yPyE/fIOTwHrb9vIAN1/Sn0ZqlNLv9JuL9g1j5r/s4uHaTLdmUUqqoFLTQdzLGtMExLHOviHTNo21On/P9x6k9xpgJxpgIY0yEv79/AWMUD3FxofGAXnT4bToeCUeIfu0jEoNC6fDNxwRFNGNTozasfuEdTh09YWtOpZS6HAUq9MaYQ9b3BOAnoD0Qf35IxvqeYDWPA2pnu3swcKioAhc3zyqViXjqXlpsjCIhdhtRox7G51gC7V96BKlZk9XdB7L5+zl6xo5SZVhJTlN84MABunfvTtOmTQkPD+f9998v8u3mW+hFpJKIeJ+/DVwLbARmAyOsZiOA81O0zQZuF4dIIPn8EE9pE9isIZFfvE1Qwn62/jCPjV16E75iAWHD+nMwoDZRD77A2ZOpdsdUShWxkpym2M3NjbfffpstW7YQFRXF+PHji/yfTEF69DWA5SISA6wGfjHGzAdeB3qJyA6gl/UzwDxgN7AT+By4p0gT20BcXGhyS2/a//4jcuQwa156jxRfPyLff4mU4DpEPfC8FnylSiFnmKa4Zs2atGnTBgBvb2+aNm3KwYMHi/R55nsevTFmN9Ayh+VHgR45LDfAvUWSzgl5VfWh3XMPwHMPsGnqLHjpJSI/eJmkL8ez/o57aPXKE3hWyfkqMkqp3D04/0HWHynaaYpbBbbivetLzzTFe/fu5e+//6ZDhw6X+YxzVi4+GVtcwocPIHzbWjZ9O4v4oLpEfvgKqUF1iPq/ZzlzIsXueEqpfDjTNMWpqancfPPNvPfee1SpUqXoniRlcJpiO4QP6w/D+rPpu9nw4ktEfvQqSV9/TMyIu2n5ypNU9PW2O6JSTi+vnndxcZZpitPT07n55psZPnw4N9100xU8o5xpj74IhQ/rT/jWaDZ/N5sjwfWIHP8ap4NDiLr3aU4fT87/AZRSJcoZpik2xjBq1CiaNm3Kww8/fEXPJzda6ItB2NB+NNsSzebv53I4pAGRH/+HM7VDibrnKS34SjkRZ5im+K+//mLKlCksXryYVq1a0apVK+bNm3fFzy27cjFNsd22/DCPjBf+TfMtazhWyYft/xpH81eepFJ1X7ujKWUrnaa44HSaYifXdFAfmm9ezdYf5nGwTmMiP30D18Aa/N3mala/8A4n9pfKjxkoVS6UhWmKtUdvg22zFnLi86+ps3whgcmJZIgL2xq1JqV3X0JHDSewWUO7IypVIkpDj95ZaI++lGk8oBcd5k6lxrEj7Ji7mDVDxlD5RBKR771IYPNG7KjTlKjRj7Bvefn556eUKj5a6G0kLi40vKE7Hb/7lDpH9rJ/xVpWjnkUI0LkxHeo06Ud+wNCWDlkLNvn/K5TJyulLoueR+9EQjq2IaRjG+C/JGzZxZ6J3+I1bw4RM77EffrnJFTxY89VPfAacjONB/fDw8vT7shKqVJAe/ROKqBpfTq89RzNN6/m9P5DrHnpPQ42bEaLRT/T/I7BpFetxrp217DmubdI2rnX7rhKKSemhb4U8AkKoN1zD9A6egkkJrL+o8ls7NaX4O2xtHvlMfwa1mVHnaas/Nf/seOXJTrEo9QVKslpis+ePUv79u1p2bIl4eHhvPDCC0W+XT3rphQzWVns/n0lCd/PpOqShTTaswkXDEmVq7I7oivuA/rR8LabqOxX1e6oSuWovJ11U7lyZVJTL57p1hjDqVOnqFy5Munp6XTu3Jn333+fyMjIi9rpWTfllLi4UL9XJzpOfIcmu2M5sXs/a156j33N2tFk5SJaPzQajxoBxIa1J+r+54hbvcHuyEo5FWeYplhELvT009PTSU9PL9BMl4WhB2PLkGp1g6lmTaGcfjaNTT/9RsqMn6n512Kaf/gKfPgK+wJCOHzzrTR+4v+oWqeW3ZGVuuDBB2F90c5STKtW8F4ec6U5yzTFmZmZtG3blp07d3LvvffqNMWqYNw9KxA+rD+RM7+kzpG9HIzeSNSDL3Da25fIT16nUv06RHfuw+Zpv+hlEVW55SzTFLu6urJ+/Xri4uJYvXo1GzduLNLnqT36ciKobThBbcPh3X+z94/VxP/3fZoumkWVob+y94FQjgy9naZP3IdPTXsv1K7Kr7x63sXFWaYpPs/X15du3boxf/58mjVrdhnPKGfaoy+HQq9uT4e5U3E/cpjVz73FOc+KRL7/Eh4htVndfSDbZi3UXr4qF5xhmuLExEROnDgBwJkzZ1i0aFG+M14Wlvboy7GKvt60f+kReOkRds7/g6PvfETzpXPxWjqLXUENSbptJOGP3q1n7agyKzY2lsceewwXFxfc3d355JNPLlqffZriWrVqFXia4n79+hEREUGrVq3yLdqHDx9mxIgRZGZmkpWVxeDBg+nbt+8VP7fs9PRKdZGUhKNsfutT/KZ+Tf1DOznlUZGN3fri98j/Uf/aLnbHU2VMaTi9UqcpVmWOd0B1Orz5DPUObGPrT7+x+apetFw8i/rXdWV7aBgrR9xP7NczSU06bndUpUqETlNcRLRH79ySDyaw5c2P8Pvxe+rF7cQFQ6a4sCeoPkdbtsOta2eC+/akRphOr6wKpzT06J3FlfTotdCrQjkZn8Teub9zevGfeK9dRd3dG/FKTwPgiG8AcWFtyOzYEb/rehDarQOu7noYSOVuy5YtNGnSpMg/IFTWGGPYunWrFnplj/Szaez9fQVHFyzBI2olwVv+JiDlKAApFbzY06A5p9p2wLvH1TQYeC2eVf4514cqv/bs2YO3tzfVq1fXYp8LYwxHjx4lJSWFunXrXrROC72yhcnK4nDMVg79sojM5csJ2LCWOof34ILhlEdFtkRcjcvgQYTdMViLviI9PZ24uLgCn2deXnl6ehIcHIy7u/tFy7XQK6eRfDiRvbMXkjbjRxquWEjV0ye16CtVBLTQK6eUfjaNrd/P5sw339Nw5SIt+kpdAS30yunlVfRdhwym6R2D8fSuZHdMpZyWFnpVquRa9Nt1w3XwIJrcfgsVfb3tjqmUU9FCr0qtnIp+hrhwIDCUpEbhZLZug2+nDtTu3pFK1X3tjquUbYq80IuIKxANHDTG9BWRusD3QDVgHfAvY8w5EakATAbaAkeBIcaYvXk9thZ6lZvzRT914RK8NsYQtHsLfqmOT+VmIRwIqE1iw3AyWrXGu2N7avfsRJUafjanVqpkFEehfxiIAKpYhX468KMx5nsR+RSIMcZ8IiL3AC2MMeNEZChwozFmSF6PrYVeFZTJyiJpx14OLv6LM1HReMaup9buLdRITrzQJq56EPENwjjXshWVO7aj9jWd8Q2paWNqpYpHkRZ6EQkGJgGvAg8D/YBEINAYkyEiHYF/G2OuE5HfrNsrRcQNOAL4mzw2pIVeXamjew5wcPEKTq9cjceG9QTu2kKtY4cvrD/i48+Ruk04G9acCu3aENAlkpotm+Di5mpjaqWuTEELfUE/n/4e8Dhw/mhYdeCEMSbD+jkOCLJuBwEHAKx/AslW+6RLAo4FxgKEhIQUMIZSOatetzbVRw2BUf9785gcd4QDi1eQunI1brEb8Nu1heCYv3D71jHXfkoFLw7UbkhK43BcWremaqd2BHeK0DN9VJmTb6EXkb5AgjFmrYh0O784h6amAOv+t8CYCcAEcPToC5RWqULwCQ7E5/ab4PabLiw7ezKVPcvWcHzFasz69fhs3UTzBT/i9cu3AKS7uLKnRh2SGjQls0VLKke2o0H/nnpuvyrVCtKj7wT0F5E+gCdQBUcP31dE3KxefTBwyGofB9QG4qyhGx/gWJEnV+oyeFapTMMbusMN3S8sy8rI5MDajSQsW8W5tevw2hxLnfVRBCz7BcbDWTcPNjRpw+mrryFw0ADqdIlAXHSGb1V6FOr0SqtH/6h1MPYHYGa2g7EbjDEfi8i9QPNsB2NvMsYMzutxdYxeOaOjew4Q99sfnPllPrVW/UlI4gEA4n382demE669r6PBsIH4BAfanFSVV8VyHv0lhb4e/zu98m/gNmNMmoh4AlOA1jh68kONMbvzelwt9Ko0OLxhKwe+n4XbogU02LCKKmmnyBQXdtZpwvGruuF7Y18a9OuBWwUPu6OqckI/MKVUMcpIO8fOX5Zw/Ke5VPtrCQ32bsHVZHGyQiV2Nm9Pes9rqT1kADVbNNZhHlVstNArVYKS446wa9ocMn6dT0j0MgKt8/qTPStzJDCElOC6pDdogEfTxvi0CCMwooVedF1dMS30StnEZGWxf8U6jvz4C2zZTKV9u/E7tO9C8T8v0bsaCYF1OBVSl6yGDakQ1oRqLcMJbBtOhUpeNqVXpYkWeqWczJkTKRz5eyPH/95E2tZtuO7YgfeBPQQc2U/1UycutMsUF+Kr1uBI/aakd7uGoFv6ERTRTIeA1D9ooVeqFEk+nEj82lhOxm4mffM2PHbtIGjzugvvAg771uBAm6twubYXdQf3pXrd2jYnVs5AC71SpZzJyiJudQyHZszFY+li6m+KpsrZVAB21WpAYofOePW5jgY398arqo+9YZUttNArVcZkpmew67c/OTprHt7Ll9JoRwwemRmcc3FjR/3mnOx0NVUH9KZB7256imc5oYVeqTLuzIkUdv48n9RffsM/ahn14nbggiGlghd7GjTnTO1QTL16VGjcEN/mTajRKkx7/mWMFnqlypnj+w6x+4e5ZP62gKrbNhKQeBAfa6jnvKTKVUkKCCYlKISMOnVxa9QA7/DG+LVoQvV6IXrAt5TRQq+UIvlgAgnrN3Fy01bO7diF6+7dVIrbR/X4AwQkJ+GSbb7B0+6exPvV4kStENIah+HeuiV+kW0IatdCh4KclBZ6pVSe0k6dJn7DVo5v2MqZrdth1y48D+yj2sG9BCUdxNU4pnNOc3UnLjCUY/UakRneDK+2rQjs1A7/xnX1HYDNtNArpS7b2ZOpxK1Yy/FV68iM2YDX9i3U3L8D/5T/TUR7oqI3h4Lrk9KwCTRvjk/7NgR1bod3QHUbk5cvWuiVUkXuxP7DHPxrDalr/oaNG/HZuZXgg7upfO70hTaHqtUkvm5jzoa3wDOiNYFXRxLYrJH2/ouBFnqlVIkwWVkc3rCNhL+iObPubzw2bSRg91aCEuMuHAM46VnZuppXGC6tW1Ptqghqd2mnUz1cIS30SilbnT6ezP4/VpG8Yg1siMF3+2Zqx+3EKz0NcFzNK65GHY7Wb0pGixZUatcW/4gW+Deui6t7Qa9yWr5poVdKOZ3M9AwOrY0lYfkax9W8tmyk5p5tBJz83yWlz7m4EV8tkOM1gjgTFEJWaCgeDepRJawx/s2b4BNcQ4eBLFrolVKlxvF9hzj4RxSnNm8la/cePPbvw/twHP5Jh6h6OvmitqkeXiT41SQ5MJi02nWgbl08G9WnavOmBLYOK1fDQQUt9Pr+SCllu6p1alE120Xcs0tNOk5CzBaSN28nbccuZPduKsTtp+qhfQTERl0YCgLHzJ8Hq9bgaK1QztSpC40a4dWsKdXbNKdGeMNyOySkPXqlVKllsrI4tu8giTFbSN20jfSt2/DYtROfA3uomXCASufOXGib5urOYb8gjgeFcrZefVwbN6Jys6YERDQvtZ8K1h69UqrMExcXqtet7Zi2eeC1F60zWVkk7d5PQnQsqbGbydq2nQp7dlP14F5qxSzHIzPjQtuTnpXZX7cJKc1bU6FDe2r27FymTgkt1T36tIw0lu9fTo96PYohlVKqrMpMzyB+43aS1sVyZtNW2LiRaltjqXNwFx5Zjn8Ax718OFAvjNMtWuPZsQNBvTrj37iezckvVi4Oxj7z+zO88dcbbLpnE439GhdDMqVUeXI25RT7/1jF8T9XINFr8d+6gZAjey9MB5HoXY2D9cM506I1Xp0jCe7Z2daLwJSLQh+fGk+DDxvQq14vfhzyYzEkU0qVd2dOpLBv8V+c+HMlruvWErAtltoJBy58GCzex5/EWo5r/5r6DfBs2oSqLZpSo3UYnt6VijVbuSj0AK/8+QrPLXmOZXcso3NI5yJOppRS/5SScJT9v/9FyvIo3GLWUyVuLzXiD1w0LXQWQoKvP0k1QzhVuy6mQQMqNGlE1RZhBLYJL5J/AuWm0J86d4qGHzakjm8dVty5AhEp4nRKKVUwyXFHOLI2lpSNW0nfuh33PbvwPriPGvEH8D2TcqFdFkKCjx9JNUPIfOBBWo677bK2V27OuqnkUYmXu7/M6DmjmbllJreE3WJ3JKVUOeUTHIhPcCAM6PWPdclxR4j/exMnN25x/BPYvRvvuD2kn0sv9lylvkcPkJmVSctPW5KWmcamezbh4aoXSVBKlX0F7dGXiZNEXV1cebPXm+w8tpPPoj+zO45SSjmVMlHoAXo36E330O689OdLJJ9Nzv8OSilVTpSZQi8i/LfXf0k6ncQbf71hdxyllHIaZabQA7St1ZbhzYfzbtS7HEg+YHccpZRyCvkWehHxFJHVIhIjIptE5EVreV0RWSUiO0Rkmoh4WMsrWD/vtNaHFu9TuNgr17xClsni+aUhd9cHAAAe4ElEQVTPl+RmlVLKaRWkR58GXGOMaQm0Aq4XkUjgDeBdY0xD4Dgwymo/CjhujGkAvGu1KzGhvqHc3/5+Jq2fRMyRmJLctFJKOaV8C71xOP9xL3frywDXADOs5ZOAgdbtAdbPWOt7SAl/iunpLk/j6+nLE4ueKMnNKqWUUyrQGL2IuIrIeiABWAjsAk4YY87P8xkHBFm3g4ADANb6ZKB6Do85VkSiRSQ6MTHxyp7FJapWrMqzXZ/lt12/sXDXwiJ9bKWUKm0KVOiNMZnGmFZAMNAeaJpTM+t7Tr33f3wqyxgzwRgTYYyJ8Pf3L2jeAru33b2E+oby2MLHyLJmnlNKqfKoUGfdGGNOAEuBSMBXRM5PoRAMHLJuxwG1Aaz1PsCxoghbGBXcKvDaNa8REx/DNxu+KenNK6WU0yjIWTf+IuJr3a4I9AS2AEuA8xPLjABmWbdnWz9jrV9sbJpnYUizIUTUiuDZxc9yJv1M/ndQSqkyqCA9+prAEhHZAKwBFhpj5gJPAA+LyE4cY/ATrfYTgerW8oeBJ4s+dsG4iAv/7fVfDpw8wAerPrArhlJK2apMTGqWn37f9ePPfX+y6/5d+Hn5Fdt2lFKqJJWrSc3y80bPN0g9l8orf75idxSllLog5kgMJdHZLheFPsw/jFGtR/Hxmo/ZdWyX3XGUUooVB1bQ7vN2vL3y7WLfVrko9AAvdnsRd1d3nl78tN1RlFLl3KGUQ9w8/WZCfEIY1XpU/ne4QuWm0Nf0rsmjHR9l+qbprIpbZXccpVQ5dS7zHLdMv4WTaSf5eejPVK1Ytdi3WW4KPcCjVz1KQKUAHlv4WImMiyml1KUe+PUBVsat5KsBX9EsoFmJbLNcFXrvCt682O1Flu1fxuxts+2Oo5QqZyaum8inaz/l8aseZ3D44BLbbrkq9ACj24ymcfXGPLHoCTKyMvK/g1JKFYHVB1dzz7x76FWvF6/1eK1Et13uCr2bixtv9HyDbUe38cW6L+yOo5QqB+JT47lp2k3U8q7Fdzd/h6uLa4luv9wVeoD+jfvTJaQLLyx9gZNpJ+2Oo5Qqw9Iz0xn0wyCOnTnGT0N+orrXPybzLXblstCLCG9d+xZJp5MY/uNwMrMy7Y6klCqjHl3wKMv2L+OL/l/QKrCVLRnKZaEHaB/Unvevf5+52+fy2MLH7I6jlCqDJsdM5oPVH/BQ5EPc2vxW23K45d+k7Lqv/X1sP7qdd6PepXH1xtwVcZfdkZRSZcTaQ2u5a+5ddAvtxpu93rQ1S7ku9ADvXPcOO4/t5N5591Kvaj161e9ldySlVCmXeCqRm6bfhL+XP9NvmY6bi72lttwO3Zzn5uLG97d8T1P/pgz6YRBbErfYHUkpVYplZGUwdOZQ4lPj+XHIj/hXKvor6BVWuS/0AFUqVGHusLl4unlyw7c3kHiqaK9hq5QqP55c9CSL9yzm076fElEr3xmES4QWeksd3zrMGjqLw6mHuXHajaRlpNkdSSlVynwX+x1vr3yb+9rdx8hWI+2Oc4EW+mw6BHdg0sBJ/HXgL0bPGa3z4SilCizmSAyjZo+ic0hn3rnuHbvjXKTcH4y91ODwwew4uoNnlzxLo2qNeO7q5+yOpJRycsfOHOPGaTdStWJVfhj0A+6u7nZHuogW+hw83eVpth/bzvNLn6dR9UYMaTbE7khKKSeVmZXJsJnDOJhykD9G/kFg5UC7I/2DDt3kQESY0HcCXUK6MOLnEUTFRdkdSSnlpJ5d/CwLdi1gfJ/xRAZH2h0nR1roc1HBrQI/DvmR4CrBDPh+AHtP7LU7klLKyczbMY/X/3qdsW3GMrrNaLvj5EoLfR78vPyYe+tczmWeo993/XQCNKXUBSlpKYybO45w/3A+6P2B3XHypIU+H038mjBj0Ay2Jm1lyIwhOoe9UgpwDNnEnYzj836fU8Gtgt1x8qSFvgB61OvBx30+Zv7O+Tw0/yG74yilbLb64Go+XP0hd0fcTcfaHe2Oky8966aAxrQdw/aj23lr5Vs09mvMfe3vszuSUsoG6ZnpjJ49mlretfhPz//YHadAtNAXwus9X2fHsR08MP8B6letT++Gve2OpJQqYW+teIvYhFh+HvIzVSpUsTtOgejQTSG4urgy9aaptKzRkiEzhrAxYaPdkZRSJWjH0R28+MeL3NT0JgY0GWB3nALTQl9IlTwqMWfYHCp5VGLojKGczThrdySlVAkwxjDul3F4unnyYe8P7Y5TKFroL0NQlSC+HvA1mxI38dSip+yOo5QqAZNiJrF4z2Le6PkGtbxr2R2nULTQX6brGlzHfe3u471V77Fw10K74yililHCqQQeWfAInUM6M6btGLvjFFq+hV5EaovIEhHZIiKbROQBa3k1EVkoIjus71Wt5SIiH4jIThHZICJtivtJ2OWNXm/Q1K8pI2eN5NiZY3bHUUoVk4d+e4jUc6lM6DsBFyl9/eOCJM4AHjHGNAUigXtFJAx4EvjdGNMQ+N36GaA30ND6Ggt8UuSpnYSXuxff3PQNCacSuGvuXTqtsVJl0K87fuXb2G95uvPTNPVvanecy5JvoTfGHDbGrLNupwBbgCBgADDJajYJGGjdHgBMNg5RgK+I1Czy5E6iTc02vNz9ZWZsnsGUDVPsjqOUKkKp51K5+5e7aerXlCc7P5n/HZxUod6DiEgo0BpYBdQwxhwGxz8DIMBqFgQcyHa3OGvZpY81VkSiRSQ6MbF0X7rvsaseo0tIF+6bd59OfqZUGfLCkhfYl7yPCf0mOP00B3kpcKEXkcrATOBBY0xes3tJDsv+MaZhjJlgjIkwxkT4+9t/8dwr4eriyuQbJwPwr5/+RWZWps2JlFJXKvpQNO+teo9xbcfROaSz3XGuSIEKvYi44yjyU40xP1qL488PyVjfE6zlcUDtbHcPBg4VTVznFeobyvg+41m+fzlv/vWm3XGUUlcgIyuDMXPGUKNSDV7v+brdca5YQc66EWAisMUYk/1CiLOBEdbtEcCsbMtvt86+iQSSzw/xlHW3tbiNQWGDeH7p86w7vM7uOEqpy/TuyndZf2Q9H/X5CB9PH7vjXDHJ70wREekMLANigSxr8dM4xumnAyHAfmCQMeaY9Y/hI+B64DRwhzEmOq9tREREmOjoPJuUGsfOHKP5J82pUqEKa8euxcvdy+5ISqlC2H18N80+bsa19a/lpyE/4ShpzklE1hpjIvJrl++kZsaY5eQ87g7QI4f2Brg334RlVLWK1Zg0cBK9pvTi8YWP81Gfj+yOpJQqIGMM4+aOw83FjfF9xjt1kS+M0nfmfynQs15PHuzwIOPXjOfXHb/aHUcpVUDfbPiGhbsX8nrP1wmq8o+TBUstLfTF5D89/0O4fzh3zr6TpNNJdsdRSuUj6XQSD/32EB2DOzIuYpzdcYqUFvpi4unmydSbpnLszDHGzBmjn5pVysk9/NvDnEw7yef9Pi+V0xzkpWw9GyfTMrAlr17zKj9v/Zmv1n9ldxylVC4W7FrAlA1TeLLzk4QHhNsdp8jle9ZNSShLZ91cKstk0WNyD9YcXEPMuBjqV6tvdySlVDan00/T7ONmeLh6sH7cejzdPO2OVGAFPetGe/TFzEVcmDRwEm4ubvzrp3+RkZVhdySllOVsxllu/+l29pzYw4R+E0pVkS8MLfQlIMQnhI9v+JiVcSv5z7LScTFhpcq6pNNJ9Jjcg5lbZvL2tW/TtU5XuyMVGy30JeTW5rcyrNkwXvzjRVYfXG13HKXKtR1Hd9BxYkfWHV7HD4N+4OGOD9sdqVhpoS9B4/uMp6Z3TW778TZOnTtldxylyqXl+5cTOTGS5LPJLBmxhFvCbrE7UrHTQl+CqlasyuSBk9l5bCcjZ43UC4srVcK+i/2OHpN74O/lT9ToKCKDI+2OVCK00Jew7nW782avN5mxeQZdv+rKwZMH7Y6kVJlnjOHVP1/l1h9vJTI4khWjVlCvaj27Y5UYLfQ2ePSqR/lpyE9sSdpCu8/bERUXZXckpcqs9Mx0Rs8ezbNLnmV48+EsuG0B1SpWsztWidJCb5OBTQayctRKPN08ufrrq/l6/dd2R1KqzEk+m0yfb/vw5fovea7rc0y5cUqpvlLU5dJCb6NmAc1YM2YNXUK6cMesO3hw/oN6nr1SRWTfiX10+rITS/cu5asBX/FS95fKzGyUhaWF3mbVvaoz/7b5PNDhAd5f9T69p/bm2JljdsdSqlRbe2gtkRMjiTsZx/zh8xnZaqTdkWylhd4JuLm48d717/Fl/y/5c9+ftPu8HZsSNtkdS6lSac62OXT9uisVXCuwYtQKetT7x2Uzyh0t9E7kjtZ3sHTEUk6nnyZyYiSzts7K/05KqQs+XPUhA6cNJMw/jKjRUYT5h9kdySlooXcyHWt3JHpMNE39mjJw2kBe/uNlneJYqXxkZmXy4PwHuX/+/fRr1I+lI5YSWDnQ7lhOQwu9EwqqEsQfI//gtha38fzS5xk8Y7B+klapXCSeSqT/9/15f9X7PNjhQWYOnkklj0p2x3Iq+V4zVtmjontFJg+cTKsarXh80eNsP7qdWUNnEeobanc0pZzGgl0LGPHzCI6fOc7HfT7m7nZ32x3JKWmP3omJCI9c9Qjzbp3H/uT9REyIYOnepXbHUsp2aRlpPPLbI1z3zXVUq1iN1WNWa5HPgxb6UuC6BtexevRqAioF0GtKLz5c9aGO26tya2vSViInRvJO1DvcE3EP0WOiaVGjhd2xnJoW+lKiYfWGRI2O4voG13P//Pvp820fnSdHlSvGGCasnUCbz9pwIPkAs4bOYvwN46noXtHuaE5PC30pUqVCFWYNncVHvT/ij71/0OyTZnwb+6327lWZd/T0UW6efjN3zb2LTiGd2HD3Bvo37m93rFJDC30p4yIu3Nv+XmLGxdDUrynDfxzO4BmDSTqdZHc0pYrFkj1LaPlpS+Zun8t/e/2X3277jVreteyOVapooS+lGlZvyLI7lvF6j9eZvW02zT5uxpxtc+yOpVSRSc9M56lFT9Fjcg8qeVQianQUj171KC6iZauwdI+VYq4urjzR+Qmix0QTWDmQ/t/3545Zd5B8NtnuaEpdkZ3HdtLpy068/tfrjGo9inVj19GmZhu7Y5VaWujLgOY1mrN6zGqe6fIMk2Mm0+LTFizes9juWEoVmjGGSesn0fqz1uw8tpMZg2bwef/P9QNQV0gLfRnh4erBK9e8woo7V+Dp5kmPyT24/9f7OZ1+2u5oShXIibMnGDZzGCNnjaRtzbbEjIvh5rCb7Y5VJmihL2M6BHfg77v+5v729/Ph6g9p9WkrvYKVcnqbEzfT+rPWzNg8g1eveZXfb/+d2j617Y5VZuRb6EXkSxFJEJGN2ZZVE5GFIrLD+l7VWi4i8oGI7BSRDSKig2o28HL34v3e77P49sWkZabR6ctOPP3706RlpNkdTal/WLp3KVdNvIoz6WdYfudynu7yNK4urnbHKlMK0qP/Grj+kmVPAr8bYxoCv1s/A/QGGlpfY4FPiiamuhzd63Yn9u5YRrYcyX+W/4f2X7Qn5kiM3bGUumDqhqlcO+VaannXImp0FJHBkXZHKpPyLfTGmD+BSy95NACYZN2eBAzMtnyycYgCfEWkZlGFVYVXpUIVJg6YyOyhs4lPjafd5+2YsHaC3bFUOWeM4bVlr3HbT7dxVe2r+OvOv3TCvmJ0uWP0NYwxhwGs7wHW8iDgQLZ2cdayfxCRsSISLSLRiYmJlxlDFVS/xv3YeM9Grql7DXfNvYu75tylQznKFhlZGdw19y6eWfwMtza/ld9u+42qFavaHatMK+qDsTldeTfHz+cbYyYYYyKMMRH+/v5FHEPlxM/Lj19u/YWnOj/FhHUT6DapG4dSDtkdS5UjKWkp9PuuH5+v+5xnujzDNzd+QwW3CnbHKvMut9DHnx+Ssb4nWMvjgOyHyoMBrSROxNXFldd6vMYPg34gNj6WthPasuLACrtjqXLgUMohun7dlYW7FjKh7wReueYVRHLqG6qidrmFfjYwwro9ApiVbfnt1tk3kUDy+SEe5VxuCbuFqNFRVHKvRLevu/FZ9Gc6OZoqNhsTNhL5RSQ7j+1kzrA5jGk7xu5I5UpBTq/8DlgJNBaROBEZBbwO9BKRHUAv62eAecBuYCfwOXBPsaRWRaJZQDPWjFlDj3o9GPfLOMbOGavj9qrI/b77dzp92YmMrAz+HPknvRv2tjtSuSPO0IuLiIgw0dHRdscotzKzMnl+yfO8tvw1OgR1YObgmQRVyfEYulKFMjlmMqNmj6Jx9cbMGz6PEJ8QuyOVKSKy1hgTkV87/WSswtXFlVd7vMqMQTPYmLCRthPasnz/crtjqVLMGMNLf7zEiJ9H0LVOV5bfuVyLvI200KsLbg67mVWjV+FdwZvuk7rzyZpPdNxeFVp6ZjqjZo/ihaUvcHvL2/l1+K/4evraHatc00KvLhIeEM6aMWu4tv613DPvHkbPHs3ZjLN2x1KlxMm0k9zw7Q18tf4rnu/6PF8P+BoPVw+7Y5V7WujVP/h6+jJ76Gye7fIsX67/kqu/vpq4k3F2x1JOzBjD/J3z6fxlZ5bsXcKX/b/kxe4v6umTTqJUH4x98EFYv74YAqkLkk4nsiVpK67iQrh/OD76Flxlk2UyiU+NJy4ljtPnTuPh6kETvyZUrVjN7milRqtW8N57l3ffgh6Mdbu8h1flhZ+XP21rerExYSPr42MIrFSDWt618K5Qxe5oykbnMs9xKOUgB1MOkZGZTiWPyjTxa0JApQBEL/XndEp1ob/c/4KqsCpx4mxjnlr0FFM2TGFd+ikiakVwd8TdDG02FC93L7sDqhISGx/Lu1HvMjV2KumZ6fRt1JeHOz7M1XXa6jCNEyvVQzeq5CWfTWbKhil8Ev0JmxM34+vpy4iWIxgXMY4mfk3sjqeKQZbJYv7O+bwb9S6Ldi/Cy92LkS1H8kDkAzSq3sjueOVaQYdutNCry2KMYdn+ZXwS/QkzN88kPSud7qHduTvibgY2GYi7q7vdEdUVOp1+mikxU3hv1XtsTdpKLe9a/F/7/2Ns27FU0zF4p6CFXpWY+NR4vvz7Sz5b+xn7kvcRWDmQMW3GMKbNGL0cXCl0OOUw49eM59PoTzl65ihta7blociHGBQ+SE+VdDJa6FWJy8zKZP7O+Xwc/TG/7vgVEaFfo37cHXE3ver3wkUP0jmtlLQU5myfw/RN05m3Yx4ZWRkMaDKAhyMfpnNIZx1/d1Ja6JWt9hzfw4S1E5j490QSTydSv2p97o64mztb36kXmXASKWkpzN0+l+mbp/Prjl9Jy0wjyDuIweGDuafdPTSo1sDuiCofWuiVU0jLSOPHLT/ySfQnLNu/jIpuFbmtxW3c1/4+WtRoYXe8cif1XCq/bP+F6ZsdPfezGWepWbkmg8IGMTh8MB1rd9R3XqWIFnrldGKOxDB+zXi+2fANZzLO0CWkC//X/v/04G0xO3XuFPN2zGP65un8sv0XzmScIbByILc0vYXB4YPpFNJJi3sppYVeOa1jZ47x1d9fMX7NePac2EMt71qMazuOsW3HUqNyDbvjlQmn00/z645fmb55OnO3z+V0+mlqVKrBzU1vZnD4YDqHdMbVxdXumOoKaaFXTi8zK5Nfd/7KR6s/4rddv+Hu4s7g8MHc1/4+OgR10AOAhXQ24yzzd85n2qZpzNk2h1Ppp/D38ueWMEfPvUtIFy3uZYwWelWqbD+6nfGrx/PV+q9IOZdCRK0I7mt3H0OaDcHTzdPueE7rXOY5Fu1exLRN0/h568+cTDtJ9YrVubnpzQxpNoSudbri5lKqPwCv8qCFXpVKKWkpTNkwhY9Wf8SWpC34efkxps0YhjcfThO/JtojBTKyMli6dynTNk5j5paZHD97HJ8KPtzU9CaGhA/hmrrX6DGPckILvSrVjDEs3rOYj9Z8xOxts8kyWXi6edI8oDkta7SkVWArWga2pEWNFlQpBxOsZZkslu9fzrSN05ixZQYJpxKo7FGZAY0HMCR8CNfWv5YKbhXsjqlKmBZ6VWbsT97P0r1LWX9kPTHxMaw/sp5jZ45dWF+vaj1H4T//D6BGS0J8Qkr9GL8xhlUHVzFt4zSmb57OoZRDVHSrSN9GfRkSPoQ+DftQ0b2i3TGVjbTQqzLLGMPBlIOOwn8khvXxju87j+3E4Hg9+3r60rJGS1rWaEl4QDg1K9ekRuUaBFYOpEalGk7V+83IyuBQyiH2ndjH/uT97E/ez94Te/lt12/sS96Hh6sHfRr2YUj4EPo26ktlj8p2R1ZOQgu9KndSz6USGx97Uc8/NiGW0+mn/9HW19OXGpWswl+5xv9uX7LMv5I/Hq4eV3SeefLZ5AsF/PzXvuT/FfWDKQfJMlkX3cfPy4/2Qe0ZEj6EAY0H4OPpc9nbV2WXFnqlcJzCGXcyjvhT8cSnxhN/Kp4jqUeIT43nyKkjFy07mXYyz8dyFVdcXVwL9N3NxQ0RIT41nuS05Isex93Fndo+tQnxCaGOTx1CfEIufNXxqUNtn9o6x78qEL3ClFKAq4srdXzrUMe3Tr5tz6Sf+cc/hKTTSaRnppNpMsnMyiTTZJKRlXHh9j++Z7udZbII8ApwFHDf/xX0wMqB+klUVaK00CtlqehekVDfUEJ9Q+2OolSR0m6FUkqVcVrolVKqjNNCr5RSZZwWeqWUKuO00CulVBmnhV4ppco4LfRKKVXGaaFXSqkyzimmQBCRRGDfZd7dD0gqwjhFRXMVjuYqPGfNprkK50py1THG+OfXyCkK/ZUQkeiCzPVQ0jRX4WiuwnPWbJqrcEoilw7dKKVUGaeFXimlyriyUOgn2B0gF5qrcDRX4TlrNs1VOMWeq9SP0SullMpbWejRK6WUyoMWeqWUKuNKTaEXketFZJuI7BSRJ3NYX0FEplnrV4lIaAlkqi0iS0Rki4hsEpEHcmjTTUSSRWS99fV8ceeytrtXRGKtbf7jOo3i8IG1vzaISJsSyNQ4235YLyInReTBS9qU2P4SkS9FJEFENmZbVk1EForIDut71VzuO8Jqs0NERhRzpv+KyFbr9/STiPjmct88f+fFlO3fInIw2++rTy73zfPvtxhyTcuWaa+IrM/lvsWyz3KrDba9vowxTv8FuAK7gHqABxADhF3S5h7gU+v2UGBaCeSqCbSxbnsD23PI1Q2Ya8M+2wv45bG+D/ArIEAksMqG3+kRHB/4sGV/AV2BNsDGbMveBJ60bj8JvJHD/aoBu63vVa3bVYsx07WAm3X7jZwyFeR3XkzZ/g08WoDfdZ5/v0Wd65L1bwPPl+Q+y6022PX6Ki09+vbATmPMbmPMOeB7YMAlbQYAk6zbM4AeIiLFGcoYc9gYs866nQJsAYKKc5tFaAAw2ThEAb4iUrMEt98D2GWMudxPRF8xY8yfwLFLFmd/HU0CBuZw1+uAhcaYY8aY48BC4PriymSMWWCMybB+jAKCi2JbhZXL/iqIgvz9FksuqwYMBr4rqu0VMFNutcGW11dpKfRBwIFsP8fxz4J6oY31R5EMVC+RdIA1VNQaWJXD6o4iEiMiv4pIeAlFMsACEVkrImNzWF+QfVqchpL7H58d++u8GsaYw+D4YwUCcmhj5767E8c7sZzk9zsvLvdZw0pf5jIUYef+6gLEG2N25LK+2PfZJbXBltdXaSn0OfXMLz0vtCBtioWIVAZmAg8aY05esnodjuGJlsCHwM8lkQnoZIxpA/QG7hWRrpest3N/eQD9gR9yWG3X/ioMW/adiDwDZABTc2mS3++8OHwC1AdaAYdxDJNcyrbXGjCMvHvzxbrP8qkNud4th2VXtL9KS6GPA2pn+zkYOJRbGxFxA3y4vLeZhSIi7jh+kVONMT9eut4Yc9IYk2rdnge4i4hfcecyxhyyvicAP+F4+5xdQfZpcekNrDPGxF+6wq79lU38+SEs63tCDm1KfN9ZB+T6AsONNZB7qQL8zoucMSbeGJNpjMkCPs9lm7a81qw6cBMwLbc2xbnPcqkNtry+SkuhXwM0FJG6Vm9wKDD7kjazgfNHp28BFuf2B1FUrPG/icAWY8w7ubQJPH+sQETa49jnR4s5VyUR8T5/G8fBvI2XNJsN3C4OkUDy+beUJSDXXpYd++sS2V9HI4BZObT5DbhWRKpaQxXXWsuKhYhcDzwB9DfGnM6lTUF+58WRLftxnRtz2WZB/n6LQ09gqzEmLqeVxbnP8qgN9ry+ivpoc3F94ThLZDuOo/fPWMtewvHiB/DEMRSwE1gN1CuBTJ1xvKXaAKy3vvoA44BxVpv7gE04zjSIAq4qgVz1rO3FWNs+v7+y5xJgvLU/Y4GIEvo9euEo3D7Zltmyv3D8szkMpOPoRY3CcVznd2CH9b2a1TYC+CLbfe+0Xms7gTuKOdNOHGO2519j588uqwXMy+t3XgL7a4r1+tmAo4jVvDSb9fM//n6LM5e1/Ovzr6tsbUtkn+VRG2x5fekUCEopVcaVlqEbpZRSl0kLvVJKlXFa6JVSqozTQq+UUmWcFnqllCrjtNArpVQZp4VeKaXKuP8H/BGDZjj2GccAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(0, 40, 130)\n",
    "x1 = 2 * np.sin(x)\n",
    "x2 = 2 * np.cos(x)\n",
    "pf = pd.read_csv(\"log.csv\")\n",
    "cols = [\"X\",\"Y0\",\"A\"]\n",
    "\n",
    "y1 = pf[cols[0]]\n",
    "y2 = pf[cols[1]]\n",
    "y3 = pf[cols[2]]\n",
    "plt.plot(pf[cols[0]])\n",
    "\n",
    "plt.title(\"Ground truth for 3 input signals\")\n",
    "l1, = plt.plot(y1, 'r', label = 'signal 1')\n",
    "l2, = plt.plot(y2, 'g', label = 'signal 2')\n",
    "l3, = plt.plot(y3, 'b', label = 'signal 3')\n",
    "\n",
    "plt.legend(handles = [l1, l2, l3], loc = 'upper right')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Both input and output sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1229fcb4e80>]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8XGW5wPHfk8m+p9naJm3TvaWFbim0ZatsAoIgixYVEVEuCir3ylW4KCIu4Aa44oWLirJVEAS5ygWLIFv3NN33Jk2aNEmbNvs2mff+cd4ZpmnSJm0mZ2byfD+ffHK2mXnmnDPPvPOe97yvGGNQSikVvWLcDkAppVRoaaJXSqkop4leKaWinCZ6pZSKcprolVIqymmiV0qpKKeJfoBE5Dci8i234zgRIvKmiHy+j3VjRaRZRDx9rL9XRJ48xnOXicgFgxXr8YjI2SKybQhep0hEjIjEDvBxIiK/E5FDIrIyVPGdiBN9T26zMU/qY12+iPxLRJpE5KdDHVu4i6gDHWoiUgbkA16gG9gM/AF41BjjAzDG3OJagCFkjNkLpLodR38ZY94GprodxzGcBVwIFBpjWtwOZhi4GTgApJuTuDlIRO4FJhljPj1YgYUDLdEf7XJjTBowDngA+AbwuLshDVxfJXM1ZMYBZX0l+UgrTQ+U/UUzlPllHLD5ZJJ8VDPG6J/9A8qAC3osOx3wATPt/O+B79npHOAV4DBQD7wNxNh1Y4AXgDrgIPBLuzwG+CZQDtTi/GLIsOteBW7r8fqlwFV2ehrwun2tbcDHg7b7PfAI8Degpef7sNu8CXwXeBdoAl4Dcuy6IsAAsXZ+PPCW3e514JfAk0HPdb19DweBu4P3nX2PdwK77Po/ASN6vM4NwF6cUtjdxzgml+L8smoC9gF32OWLgcqg7eYCJXa754ClQcdpMVAJfM3u82rgxqDHfsQ+thGoAO4NWtdzv3wW2G1fZw/wqV5ivglox/lV2Ax8JyiGbwD7gT/abb8A7LTH9GVgdNDzGOBLwA77et8FJgLv21j/BMT3sd+OdZ7539PNQJXdH1/rcc6vtq9RAzwYtG4B8B7OOV8KLO5xfn0f5/xqs6+/ukdc/w68bKcTgJ/Y86AG+A2QFLTtf9rYqoDP2Zgn9fJefw90AZ12f19g38P7Ns5qnPM3PugxM/jgs1QD/BdwsX2OLvs8pf095uH+53oA4fRHL4neLt8LfDHopPInkPvtyRln/84GBPDYD8FDQAqQCJxlH/M5+8GegFNV8gIffOg/A7wb9Lqn2BM1wT5PBXAjTpXbXJwkOSMorgbgTJwPeWIv7+NNnOQ7BUiy8w/YdUUcmdDeBx60r32OPcmfDIqr2S5PsNt5+SDR3w4sBwrt+v8GnunxOo/ZGGYBHcD0Po5JNXC2nc4C5trpxdhED8TjJLSv2uNwlf3ABid6L3CfXX8p0ApkBa0/1e6303A++Ff23C/2GDQCU+26Uf7930vcnwXeCZr3x/BDu0+SgPPsMZxrl/0C+FfQYwxO8k/HSUwdwDKccycD5wvwhj5e/1jnmf89PWPf06k4BZILgo799XY6FVhgpwtwvrgvtfvqQjufG3R+7bWxxtoYm4DJQXGtApbY6Yft+xsBpAF/Be636y62x2GmjfFp+kj0PT+Xdn4ezpdSrH2/W4Db7bo07JcbzmczDTjDrruXIws0/T7m4fznegDh9EffiX45ttTJkYn+PuClnicfsNB+cGJ7ea5lwJeC5qfilCBi7QnXAoyz674P/NZOfwJ4u8dz/Tfw7aC4/nCc9/cm8M2g+S8Br9rpIj5IaGNxklJK0LZP80Givwd4NmhdCk5i9SeKLcD5QetHBb1H/+sUBq1f6f/w9xLzXuDfcOpeg5cv5oNEfw5OaV+C1r/DkYm+Lfh44JRyF/Txmg8DD/WyX1JwvnivJqjk2cdzfJajE30nQV/AOFWCPwqaT7X7qcjOG+DMoPVrgG8Ezf8UeLiP1z/WeeZ/T9OC1v8IeNxO/wvnV0hOj+f8BvbLImjZ/2G/bOz5dV+P9U8C99jpyTiJPxmnQNQCTOzxudljp3+LLYTY+SkMINH3sv524EU7fR1Q0sd293J0ou/XMQ/nP62j758CnJ94Pf0Yp9T0mojsFpE77fIxQLkxxtvLY0bjlD79ynE+fPnGmCbgf4Eldt0S4Ck7PQ44Q0QO+/+ATwEjg56roh/vZX/QdCu9X4AdDRwyR9Yvl/dYH3gtu93BoPXjgBeD4tyCU42RP8A4wPmAXQqUi8hbIrKwj3j3GfvJtHrui4M9jkfgNUXkDBH5p4jUiUgDcAtOtdwR7Pv8hF1fLSL/KyLT+oi7N3XGmPYecQf2qzGmGWc/FgRtUxM03dbLfF/7rc/zLGhZRY/1o+30TTiJdauIrBKRy+zyccC1Pc7Bs3C+yHt7TnAKCNfZ6U8CfzHGtAK5OAl/TdBzvWqX++PvGV+/icgUEXlFRPaLSCPwAz44pmNwftke1yAc87Cgif44RGQ+zgfvnZ7rjDFNxpivGWMmAJcD/yEi5+OcoGP7uOBWhfOB8fOXnv0f4GeA62xCSwL+aZdXAG8ZYzKD/lKNMV8MDunE3+kRqoEsEUnpEWfw+jH+GRFJBrKD1lcAl/SINdEYs2+ggRhjVhljrgDygL/g1Ev3Fm+BiEjQsjG9bNeXp3GqEMYYYzJwquOktw2NMf9njLkQJ7ltxamC6q+ex+eIc8Hu72ycXycn63jnGRy5j8bax2CM2WGMuQ5nn/8QeN7GVoFTog8+rinGmAeCnqfne3wNyBGR2TgJ/2m7/ADOF9WMoOfKMMb4v7iOOMc48vzrj0dwjs9kY0w6Th28/5hW4Fzr6M1Rn6GTPOZhQRN9H0Qk3ZZknsX5Kbehl20uE5FJNsE04pRau3GqIqqBB0QkRUQSReRM+7BngH8XkfEikopT0lgaVNr8G84H9D673GeXvwJMEZHrRSTO/s0XkemD/d6NMeU4F+O+IyLxInIWzheZ3/PAZSJylojE21iDz6XfAN8XkXEAIpIrIlcMNA772p8SkQxjTBcf7OOe3rfLbxORWPtapw/gpdKAemNMu4icjlPy7C2efBH5qE16HTjXKXqLp7+eBm4UkdkikoBzLqwwxpSdxHP6He88A/iWiCSLyAycaz9LAUTk0yKSa8+9w3bbbpxqmMtF5MMi4rHn9WIRKewrCPt6z+P8+h2BcwEU+9yPAQ+JSJ593QIR+bB96J+Az4rIKbYg8e0Bvv80nPOl2ZbAgwtErwAjReR2EUkQkTQROcOuqwGK/C2GQnDMXaGJ/mh/FZEmnG/9u3EuNN7Yx7aTgX/gHPz3gV8bY940xnTjJMZJOHXMlTg//8Cpe/wjTj3oHpzWGV/2P6ExpgPnwtkFfFD6wVbrXIRTnVOFU/Xhv7AXCp8EzsCpsvo2TqsNfyybgFttfNXAIZz36PcznBLya3ZfLrfPdSKuB8rsz+9bgKPaNxtjOnEuwN6Ek5g+jfNh7ujna3wJuM/Geg+9/2oA5/PyNZz9Xw+cax97Qowxy4BvAX/G2Y8T+aDa7mQd8zyz3sKpelwG/MQY85pdfjGwSUSacY7lEmNMuzGmArgCp3Rch/MZ+U+On0eexjmfn+vxRfMN+/rL7fH9B/beCGPM33Gulbxht3ljQO8e7sA5h5twvlCW+lfYz9KFOJ/R/Titmj5kVz9n/x8UkbUM8jF3ixxZralUdBCRFcBvjDG/czsWpdymJXoVFUTkXBEZaatubsBpJvmq23EpFQ6i+u48NaxMxalyScVpUXGNMaba3ZCUCg9adaOUUlFOq26UUirKhUXVTU5OjikqKnI7DKWUiihr1qw5YIzJPd52YZHoi4qKWL16tdthKKVURBGRft0xrFU3SikV5TTRK6VUlNNEr5RSUU4TvVJKRTlN9EopFeU00SulVJTTRK+UUlEu4hN9eXs7nT7f8TdUSqlhKqITfafPx4WlpSzZvNntUJRSKmyFxZ2xJyo+Job7J0wgNy4OcAY6P3I0OaWUUhGd6AGuzv2gm4efVVaytrmZx6ZOJSEmon+sKKXUoImqbNjm89Hc3U28luqVUiog4kv0we4aNw6frb450NnJM7W13FpQQIwmfqXUMBZVJXogkNR/t38/d+zaxc62NpcjUkopd0Vdove7Y8wY1sybx5TkZAB2acJXSg1TUZvoRYSZqakAvNvQwJQVK1haW+tyVEopNfSiNtEHm5uayreLirgsOxtwmmEqpdRwMSwSfZLHwz1FRaR4PHh9Pi5ev56namrcDksppYZEvxK9iJSJyAYRWSciq+2yESLyuojssP+z7HIRkZ+LyE4RWS8ic0P5BgaqsbubTmPwt8PR0r1SKtoNpET/IWPMbGNMsZ2/E1hmjJkMLLPzAJcAk+3fzcAjgxXsYBgRF8eyWbP4ZH4+AL/at48z166lyet1OTKllAqNk6m6uQJ4wk4/AVwZtPwPxrEcyBSRUSfxOoMuuF19ZmwsYxISSIt1bil4pqaGNw8dcis0pZQadP1N9AZ4TUTWiMjNdlm+MaYawP7Ps8sLgIqgx1baZUcQkZtFZLWIrK6rqzux6AfBp0eO5NkZMwCnGufbZWX8ct++wPq6zk63QlNKqUHR3ztjzzTGVIlIHvC6iGw9xra93YZ6VEW4MeZR4FGA4uLisKgoFxFKi4s5ZKtxajs7KXz/fR6aNIlbC476rlJKqYjQrxK9MabK/q8FXgROB2r8VTL2v7+ReiUwJujhhUDVYAUcakkeD6MTEgCIFeHeoiLOz8wEoLS5mc9t3Uple7ubISql1IAcN9GLSIqIpPmngYuAjcDLwA12sxuAl+z0y8BnbOubBUCDv4on0oyIi+O/xo1jWkoKAFtaWvjrwYMkezwA7OvooK27280QlVLquPpTdZMPvGj7eY8FnjbGvCoiq4A/ichNwF7gWrv934BLgZ1AK3DjoEftkiX5+Vydm0uc7QL5pq1b2d/ZSUlxsfaDr5QKW8dN9MaY3cCsXpYfBM7vZbkBbh2U6MJQXFA/93eNG0dtZycigjGGp2pquDo3lyRb4ldKqXAQVd0UD7Vzbd09wIrGRq7fupV2n4/Pjx7tYlRKKXUkTfSDZEFGBm/Pns389HQA/vfgQba1tnJbQQHxOtqVUspFmoEG0VmZmYEhDP/34EEeq67GY+vutasFpZRbNNGHyK+nTOG9OXPwiNDl87GopIRntSM1pZQLNNGHUFZcHAAHu7pI83hItRdp27u7ada+dZRSQ0QT/RAYmZDAa7NmcVlODgCPVFWR+9571NjuFbRaRykVSproXXB2RgZ3jx1Lfnw8ALfv3MnlGzZowldKhYS2unFBcXo6xbZ1DkBRYiKxIoGbrr66YwenpqRoM02l1KDQRB8G/n3MB10DdRvD2uZmUmx9vjGGH+zdy1U5OUy3XTEopdRAaKIPMx4R3p4zB5+txtnV1sa9ZWWMio9nekoKTV4vyxsbOTczU9vnK6X6RTNFmPIPjjIpOZnaRYv4RJ7T3f/f6uu5aP16VjU1AU5/+fs7OlyLUykV/jTRR4CsuLhAVc7l2dn8deZMFtg6/v+uqqLg/fep7+oCoMHrDfwaUEop0KqbiJPs8QSaaQJcm5fHqIQERtg2+7ft2MG65mbW2x41fcYcMXSiUmr40UQf4aYmJzM1OTkw/7GcHBampwda8Jy7bh3zUlN5ePJkt0JUSrlMq26izFW5uXzJDntojGFRejozbWsdr8/HmWvX8oKLY/QqpYaeluijmIjww4kTA/MHvV5iRIi1pf3DXV1saGnhrIwMHThFqSimJfphJD8+nrfnzOHy7GwAnqyp4Zx169jY0uJyZEqpUNIS/TDkL71/btQoChISODU1FYB79uyhprOT30yZoiV8paKIluiHsWSPh4/l5gbmu4yh05hAkn+9vp4m7WVTqYinJXoVcP+ECYHpus5OLt2wgdsLC/lxUD2/UiryaIle9SonLo535szhNtuCp7S5mTPWrGF7a6vLkSmlBkoTveqViHBGejrjEhMBZ/CULmPIinV+BD6xfz/zVq+mzvap7/X5XItVKXVsmuhVv5yXlcXa4mJybR/6GR4PYxISyLF35N69Zw9TVqyg23a/cLCrKzCtlHKX1tGrE3Jlbi5XBl3IPT09nViRwGDoN23dyt6ODtYWFwOwrbWVsQkJJNk+e5RSQ0cTvRoUV+fmcnVQ4v/syJE0dncH5q/cuJGJiYm8ctppgNPvvkebcCo1JDTRq5AILu0DPDhxIsm2NN/a3c3EFSv4/vjxfG7UKDfCU2pY0Tp6NSQuyc7m3MxMAJq7u7k8O5spSUmAM7jKJzdvZoe26FEqJDTRqyGXFx/Po1OncpZN/NtbW3n90KHAiFnrmpr4S10dXdqSR6lBoYleue6S7Gz2L1oUaMr5WHU1n9m6NdBqp6ytjdag+n6l1MBooldhIfjC7MOTJvHenDkk2jr9G7Zu5YLS0sD6vx44QIkdSlEpdXz9TvQi4hGREhF5xc6PF5EVIrJDRJaKSLxdnmDnd9r1RaEJXUWruJgYZtqO1gDuKSrirrFjA/O3bN/Og5WVgfk7du7k5QMHhjRGpSLJQEr0XwW2BM3/EHjIGDMZOATcZJffBBwyxkwCHrLbKXXCzs/K4vKg4RNXzpvHfUVFAHT6fDxXV8e65mYAunw+ZqxcyZP79wPO4CsH7Xi6Sg1X/Ur0IlIIfAT4HzsvwHnA83aTJ4Ar7fQVdh67/nzRPm/VICpISGC8bbETHxND+cKF3D1uHOAMjj49OZkse8funvZ2ct59lz/axH+4q4s/19UFBlNXajjob4n+YeDrgL8ZRDZw2Bjj78O2Eiiw0wVABYBd32C3P4KI3Cwiq0VkdZ0ObadOkr+OPyc+nudnzuQjdnCVFI+HH02YwML0dABWNDVxzaZNrLe/ANY3N/OVHTuobG93J3ClhsBxE72IXAbUGmPWBC/uZVPTj3UfLDDmUWNMsTGmOLfHzTVKDZb8+Hj+c+xYJtkB1M/NyGDV3LnMt4l/W2srv9u/P9AH/5P793PqqlXU2M7aDnV10aYtflSE60+J/kzgoyJSBjyLU2XzMJApIv47awuBKjtdCYwBsOszgPpBjFmpE5bo8VCcnk6KbdFzbV4eDWedxWjbWVtWXBwTExMDnbX9uKKC/PfeCyT7Jq8Xo521qQhz3ERvjLnLGFNojCkClgBvGGM+BfwTuMZudgPwkp1+2c5j179h9JOhwliMSKBE/5HsbP5y6qmBqqDLsrO5r6go0BnbjVu3cmZJSeCxnXpTl4oAJ9PXzTeAZ0Xke0AJ8Lhd/jjwRxHZiVOSX3JyISrlnkUZGSzKyAjMX5WbS2PQ8Irz16zhnMxMfjF5MuC08tG2ByrcDCjRG2PeBN6007uB03vZph24dhBiUyrsfDI/PzDtM4arcnMDffZ0+HxMXL6cbxcV8YXRowNVPJr4ldv0zlilTlCMCN8uKuI6m/ybvF4uyc5mgk38W1tbGfHuu/z94EEADnR2srS2lgP2Qq9SQ0UTvVKDJCc+nsemTuX8rCwAEmJiuC4vjyLbh8/KpiaWbN7MtrY2AN4+fJhzS0oC4/DWdXayuaVFh2VUg04TvVIhMiEpiV9PmcL0lBQAzsvMZH1xMXNs9w6dxtBtDGn2Qu8LBw4wY9UqqmyJf2VjI7/Zt0+bd6qTpoleqSGS6PFwampqYACW87OyeGfuXEYlJABw8YgRPDl9OoV2/sUDB7hj165AC6C/HzzI0tpabd6pBkxHmFIqTIxLTAx01Qzwg/Hj+XJBQaCf/t9UVbGnvZ1P5OUBsLS2lry4OD5kq4qU6osmeqXClIgw2pbuAV6YOZPqjo7A/Df37GF2amog0f+2upritDROC+r5UynQqhulIoZHhMKgEn9pcTE/mzQJgJbubm7Zvp0/1dYCzo1cX9q+nfcaGgCnKWiL1vUPW5rolYpQyR5PoMSf4vFQtXAhXyksBGB/ZyfP1tayw7bwKWtvJ/Xtt3mqpgaA2s5O7t2zJzBOr88YrfuPYprolYoSOfHx5Nk+e8YmJlJ/1llcb9v4p3g8/GD8eObZap3tra3cV15Oha0KeruhgbS33w78Aqhob+e52toj7gJWkUsTvVJRLMa22MmPj+euceOYZpt6npWZSds553C27d4hNy6Oz48aFWjz/49Dh/j45s2BXjxfq6/nhi1bAoO4dPl8+gsggmiiV2qYSoiJIc626DklJYWHJ08OVAVdl5fH2nnzAnf57uvo4M3DhwNt/n+4dy+5775Lu633X9/czHsNDZr8w5QmeqXUURI9HuakpQXa8N84ahTlCxcGmnrOT0/nC6NHBwZwf7Cigqs3bQr06/NYVRU/2bvXneDVUTTRK6UG7MMjRnD/hAmB+e+NH88LM2YE5t84fJhXbB8/AB/ftIl/27YtML+nrY0O7ephyGg7eqXUSStMTDyi6eczp5xCd1A1zqSkpEC1D8B5paWckZbGs/bL4bfV1cxJTWVOWtrQBT2MaIleKRUSnqDumX8wYQJ32QHcjTH8eMIEbhk9GnC6d7552zb+bMeO7jaGU1au5LfV1YH5P9XWslfH9T1hmuiVUkNKRLgmL4/FQb187l+0KHAPQEt3N6empJBth3Os6ujgE5s38/d6Z0TS6o4OZq1aFej+ucnr5ZUDB7T752PQRK+Ucl3wPQDpsbEsnTGDK3JyABgZH09pcTEfs/PtPh9jExNJj3Vqnje0tHD5xo2sbGoCnBZAF5eWsqG5GYBGr5eytrYjqpKGG030SqmwFhcTw2mpqYEvgvFJSfz11FM5094DMCs1lffnzAnMN3q91HV1BVoIvVpfz/gVK9jU0gLA+w0NfH3XLuqH0T0BmuiVUhEtxeNhQUYGGbaEf1ZmJmuKi5manAzA6WlpPDZlCpPtPQEbWlr4eWUlcfYaws/37SP73Xc5bBN/WVsb5e3tUZX8NdErpaJaUVISnx89miTb6ufm0aNpOecc0uwXw9zUVG4aNYpMe03ghxUVzFq1Cn+af72+nr8FNRWNRBHdvPJgVxff2rOHrxYWBr69lVLqeIJbBH0oK+uIPv1vHT2aD2dlBbqP+FFFBYe6urg0OxuAB8rLyYyN5ZaCgqEN+iREdIm+2xiera3lHdsRk1JKnayZqalcmZsbmH9p5kz+FHQz2LLDh3k7KOd8uLSU+8rKAvNvHDpEeZg1BY3oRJ8XH0/ZggXcNGqU26EopaJUsscT6PMH4PVZs/jj9OmAc0/AyPh4cm21j88YPrJhA7/cty+w/tL163nOjhNgjGFzS0ugj6ChEtFVN0CgidWO1lYmJSUF+tpQSqlQ8VfriAhP2KTv99bs2WTZvNTU3U19V1dg0Je6ri5mrFrFw5Mm8VV738BQiPhED/DO4cOcu24dS085hWvseJpKKTXUYkQ4PT09MJ8eG8vyefMC88kxMTw1fTrFQ9zVQ1Qk+oUZGdw3fjyLMzPdDkUppfqUGhvLJ+1gMEMpKhK9R4S7bT8aSimljhTRF2N72tbaypJNm2jQ4c+UUiogqhJ9c3c3rx86xHrbx4VSSqkoqbrxm5eWRsXChSQH9XutlFLD3XFL9CKSKCIrRaRURDaJyHfs8vEiskJEdojIUhGJt8sT7PxOu74otG/hSMkeD8YYNmqpXimlgP5V3XQA5xljZgGzgYtFZAHwQ+AhY8xk4BBwk93+JuCQMWYS8JDdbkg9Wl3NaatXB7opVUqp4ey4id44/Bkzzv4Z4Dzgebv8CeBKO32FnceuP1+G+C6mj+fm8vCkSUzR/m+UUqp/F2NFxCMi64Ba4HVgF3DYGONv3lIJ+Hv4KQAqAOz6BiC7l+e8WURWi8jqOjuE2GDJiovjK4WFJMRE1bVmpZQ6If3KhMaYbmPMbKAQOB2Y3ttm9n9vpfejOnY2xjxqjCk2xhTnBnUgNJjeOHSI6zZvxhdF/UorpdRADajIa4w5DLwJLAAyRcTfaqcQqLLTlcAYALs+A6gfjGAHqqazk9VNTVR2dLjx8kopFRb60+omV0Qy7XQScAGwBfgncI3d7AbgJTv9sp3Hrn/DuDRUyyfy8tg0fz5jExPdeHmllAoL/WlHPwp4QkQ8OF8MfzLGvCIim4FnReR7QAnwuN3+ceCPIrITpyS/JARx90uMCPEieH0+NrW2Mis11a1QlFLKNcdN9MaY9cCcXpbvxqmv77m8Hbh2UKIbJF/ZuZOnamooW7CALNtvtFJKDRdRdWdsX75cUMBFWVlkxg6Lt6uUUkcYFu0Pp6ekcGVurg5KopQKGzWdnfygvHxIWgUOi0Tv93h1Nbdt3+52GEopRX58PAvT02kegmEFh1Wi39vezubWVjp8PrdDUUoNU50+H6W2e5YPZWUFhkMNpWGV6L81bhzLZs3SO2aVUq65r6yMM9asoby9fchec1hdnYy1Cb7B66WivZ2Z2txSKTXE/n3MGCYkJTFuCO/vGZZF28s2bOAT2jWCUmoIVba34zOG7Lg4Pjdq1JC+9rAq0fvdP348KR4PMdoKRyk1BA53dbGwpIQrc3L4xeTJQ/76wzLRn5WZGZjuNgaPJnylVAhlxMby9TFjODsjw5XXH5ZVN3737tnDVRs30q1VOEqpEGn0ehERvlxYyOy0NFdiGNaJPjc+npy4OK2rV0qFxB/372faypXsbG11NY5hWXXjd2tBAcYYRASfMVpnr5QaVHNSU7lkxAiKXO5Bd1iX6AFEhP0dHSxcu5bX613pNl8pFWW89qbMmampPD5tWqBpt1uGfaIHSPF4iBXBq1U4SqmT5PX5uHj9er5XVuZ2KAHDuurGLy02lnfmzAl0euavzlFKqYHqBsYlJlKYkOB2KAFaorf8if352louWr9e+8NRSg2YMYaEmBgenzaNzw7xTVHHoom+BwO0+3y0DEGPckqp6FHa3Mx5paVUDmEfNv2lVTc9XJuXx9W5ucSIaBWOUqrfqjo62N/ZSWwY5gwt0fciRoT27m4+uWULS2tr3Q5HKRUBLsnOZuP8+YwMo7p5P030fRARqjs6qOrocDsUpVQYu6+sjOdsgTBcu1PRqps+JMTE8I9Zs1xv/6qUCl+dPh//V19PVUcH1+bluR394jpNAAAXY0lEQVROnzSLHYM/yZc0NXFxaSmNXq/LESmlwkl8TAxvzJ7Nz1zokXIgNNH3Q73Xy462Nqo7O90ORSkVJv5SV0ez10tCTEzYj1oX3tGFifOzsthy+ulMTU52OxSlVBjY297ONZs2cf/evW6H0i9aR99P8TExGGO4f+9e0j0ebissdDskpZRLxiYm8tacOZyakuJ2KP2iiX4ADLCysZGM2FhtY6/UMNXk9ZIWG8uZLg0iciK06mYAYkRYOmMGv582TZO8UsPQjtZWxi5fzgt1dW6HMiCa6AcoISYGEaGyvZ1btm2jXbtKUGrYSI+N5cqcHBakp7sdyoBooj9Bm1pbeaq2lrXNzW6HopQaIvnx8fxu2jRGh+Hdr8eiif4EfXjECMoWLGBRBNXTKaVOTG1nJ5/dsiVi75Q/bqIXkTEi8k8R2SIim0Tkq3b5CBF5XUR22P9ZdrmIyM9FZKeIrBeRuaF+E27JjosD4NWDB6nv6nI5GqVUqKxsbOQvBw7QEKE3TfanRO8FvmaMmQ4sAG4VkVOAO4FlxpjJwDI7D3AJMNn+3Qw8MuhRh5HK9nY+unEjD0RIe1ql1MBdlpNDxcKFTI+Q5pQ9HTfRG2OqjTFr7XQTsAUoAK4AnrCbPQFcaaevAP5gHMuBTBEJnx74B1lhYiKvnnYa9xUVuR2KUmqQNXu9vHP4MOCMRBepBlRHLyJFwBxgBZBvjKkG58sA8PfoUwBUBD2s0i7r+Vw3i8hqEVldF2FNlXo6LyuLRI+HDp+PWu0mQamo8WBlJeeuW8eutja3Qzkp/f6KEpFU4M/A7caYxmO0I+9txVGjbhtjHgUeBSguLo74UbmNMVxUWooB3po9W9vZKxUFvjZmDDNTUpiYlOR2KCelX4leROJwkvxTxpgX7OIaERlljKm2VTP+EToqgTFBDy8EqgYr4HAlItxWUECibWevlIpcXp8PESHF4+Gq3Fy3wzlp/Wl1I8DjwBZjzINBq14GbrDTNwAvBS3/jG19swBo8FfxRLtr8/K4PCcHAJ+J+B8pSg1bD1ZWsnDt2qjpmrw/dfRnAtcD54nIOvt3KfAAcKGI7AAutPMAfwN2AzuBx4AvDX7Y4W1pbS0L1q6lVe+aVSoijU9MZFZqKmkej9uhDIrjVt0YY96h93p3gPN72d4At55kXBEtOzaWNI+Hpu5ukqPkRFFqOLk2Ly+sR4waqMhtLxTGLhgxgvOzsrSuXqkI83xtLW0+H5/Oz4+qz692gRAiIkKD18tXduzggDa5VCoi/KGmhkeqqo5uJhjhtEQfQuXt7TxeXc1ZGRl8PIp+BioVrV6cOZP6ri5ioqg0D5roQ+q01FTKFiwgNz7e7VCUUsewuaWFsQkJpMbGRuXnVatuQsx/0qxubGR3hN9dp1Q08vp8fHTDBq7etMntUEJGS/RDoLW7m0s3bOCcjAyenznT7XCUUkFiY2J4Yvp0orl9nCb6IZDs8fDnGTOYGaE93ykVjeo6O1nR2MhlOTkRNf7ridCqmyFydmYmWXFx+IyhOkIHL1AqmvzXnj18csuWYTGWhCb6IfaFbds4d9062vSuWaVcYWz3JD+ZOJFls2Yxwg4gFM000Q+x6/Pz+Y/CQhJjdNcrNdSeqanh2k2b8Pp8ZMTGMj/CBvk+UVpHP8QWZ2WxOCsLgDVNTYyMj6cgwgYaVipSHfZ6qevqotXnI30YFbaGzzsNM16fjyWbN3Pd5s1uh6JUVOvy+djW2grAFwsKWDZrFukRPFrUiRhe7zaMxMbE8OKMGYE78IwxUdW3hlLh4tYdO/jLgQNsP/10MuPiiB1GJXk/TfQumpmaGpj+1p49NHd38+CkSVF3+7VSbvrPMWM4OyODzGFw0bUvw++rLQwZY2jx+Wjx+frsD1op1X9rmpp4oLwcgMnJyVw/cqTLEblLS/RhQER4aNIkfLb6pqK9ncbubmboDVZKnZA/7N/PSwcOcMvo0cO6JO+nJfow4q+y+eL27Xy4tJQOn8/liJSKLM126L+fTpzIqnnzNMlbWqIPQ/89dSo7WltJsBeN9EKtUsd39+7dvHLwIO/OmRO1vVCeKE30YaggISHQtn5pbS3P19XxxLRpOiyhUsdwbmYm7T4fSfo5OYom+jBX29lJXWcnsVqiV+oor9XXU9vZyadHjuSiESO4aMQIt0MKS1pHH+a+XFjIstmziY+JobW7m/cbGtwOSamwYIzhwYoKfrZvHz4TbYP/DS5N9BHAY0vz3ykrY/G6dVS2t7sckVLu2drSwqGuLkSEJ6dP51+zZ+u9J8ehiT6C3D1uHE+fcgqFiYkAtGsPmGqYOdzVxYK1a/n67t0A5MTHa518P2iijyDpsbFcnZsLQGlzM0XLl/P24cMuR6VU6HXZpsaZcXE8OnUq3ykqcjegCKOJPkKlejwszMjgFL2pSkW59c3NTFm5kuX2+tTH8/IYrT2+Dogm+gg1MSmJF2fOJDsuDmMMN27dyl8PHHA7LKUG3bjERKYmJQXuK1EDp3suCtR7vZQ0NVGmF2lVlNjZ2srtO3bgM4aM2FhenTWLOWlpbocVsTTRR4HsuDhWzpvHrQUFALx1+DBvHDrkclRKnbh/NTTwh5oattt+5NXJ0RumokR80M/a75WXU9XRQWlx8bDse1tFpsNdXexoa2N+ejo3jhzJ5dnZ2o3BINFEH4VemjmT6s5OYmNi6PL52NDSwlz92avC3HVbtrC+uZldZ5xBosejSX4QHbe4JyK/FZFaEdkYtGyEiLwuIjvs/yy7XETk5yKyU0TWi8jcUAavepfs8TAxKQmAn1VWMn/NGra0tLgclVLH9svJk/nzjBkkarv4Qdef3/W/By7usexOYJkxZjKwzM4DXAJMtn83A48MTpjqRN08ejSPTp3KdNsM09+Nq1Lh4OmaGu62Nz9NTEpiQUaGyxFFp+MmemPMv4D6HouvAJ6w008AVwYt/4NxLAcyRWTUYAWrBi49NpabRjmHoKK9nQkrVvB8ba3LUSnleL+xkXcbGnTshRA70Tr6fGNMNYAxplpE8uzyAqAiaLtKu6y65xOIyM04pX7Gjh17gmGogUj2eDgvM1Pr65WrvD4f9V4vefHxPDxpEl5jtI18iA323u2tZ6Feu5UzxjxqjCk2xhTn2tv6VWhlx8Xx7IwZTLD197+orKSqo8PlqNRw85mtWznfjqDmEdEkPwROtERfIyKjbGl+FOCvC6gExgRtVwhUnUyAKjT2trdz1+7dHOjq4jvjx7sdjhpGPj9qFLva2jTBD6ET3dMvAzfY6RuAl4KWf8a2vlkANPireFR4GZuYyNriYr41bhwADXqRVoXQxubmwLWh87Ky+MLo0S5HNLz0p3nlM8D7wFQRqRSRm4AHgAtFZAdwoZ0H+BuwG9gJPAZ8KSRRq0ExJTmZ2JgYmr1eFq1dyzd27XI7JBWlvlVWxh27dmnX2i45btWNMea6Plad38u2Brj1ZINSQyvJ4+FjOTmcl5XldigqyviMIUaE302dSmN3t7aRd4lWkik8InxvwoRAon+qpoZ3dchCdZK+X17O1Zs20W0MmXFxjLUD5qihp4leHcHr83F/eTkP7N3rdigqwqV7PKR5PHTreK6u075u1BFiY2J4Z84c/LevNHm9xInoT27VL41eL3vb25mZmsqXCwsxxiA6nqvrtESvjpIZF8cIO6DJ9Vu2cEFpqZbKVL98bdcuLl6/njZ70VWTfHiI+ES/uKSE31c7LTi7fD4Wl5Tw5P79ALR2d7O4pISltllXg9fL4pISXqirA+BAZyeLS0oCIzPt7+hgcUkJrx48CDhdBiwuKeEf9U4PELvb2lhcUsJbdpzWba2tLC4p4T1bn72xuZnFJSWsamwEYF1TE4tLSljX1ATAqsZGFpeUsLG5GYD3GhpYXFLCNtvn9luHD7O4pITdbW0A/KO+nsUlJVTYAUVePXiQxSUl7Lc3Of31wAEWl5RwoLMTgBfq6lhcUhJoKrm0tpbFJSW02g/dk/v3s7ikJDD+5u+rq1lcUhLYl49VVXHBunWB+Ueqqtjb0cH1I0fiEeFnlZV8dMOGwPqf7N3L1RsDfd3xQHk5SzZtCsx/t6yMT2/eHJi/Z88ebty6NTB/1+7d3LxtW2D+jp07uXX79sD87Tt2cPuOHYH5W7dv546dOwPzN2/bxl22nxSAG7du5Z49ewLzn968me+WlQXml2zaxAPl5YH5qzdu5CdBVVQf3bCBn1VWBuYvWb+eX+/bF5i/YN06Hqv64LYQPfec9f9TVcUT9ty6OieH306bxst2+xM99369bx+XrF8fmI/2cy/UIj7Rq9DKj4/n32yb591tbaxrbuZwV5fLUalwUdbWxhe2b+d9+4WzMCODi0aMcDkq1ZOYMPhJXlxcbFavXu12GOo47t2zh+fr6iidPx+PCJtbWihKTCRZ6++HDWMMX9y+nbTYWH48cSIAJU1NzE5N1WoaF4jIGmNM8fG20xK96rd7x49nvU3yxhg+tnEjVwX9fFbRyWcMa20VkIjgETkiccxJS9MkH+a01Y0akJigD/RjU6cGerFr6+5myebN3Dl2LAu1T/Go8uOKCv5r9252nXEGRUlJ/GrKFLdDUgOkiV6dEBHhnMzMwPyutjbWt7QE+hWv7+qipbubMXqTTMQ53NXFTyoquCo3l7lpaXw6P5/ChARGJyS4HZo6QVp1owbFzNRUdp5xBufa5P+rffuYsGJFoJWGCm9NXi87bQucGBF+VVUVaOFTkJDAp/LzjxiAXkUWLdGrQeMJqta5Pj+fUfHxjLSlwO+UlZETF8etBQVuhad66DYmcMzOKikhNy6Of8yeTXpsLGULFpARq+khWuiRVCFRlJTE5+0AJ8YYljc2UhAfH1i/p62N8Xa9GnoPlJfzRE0Nm+fPR0S4f8IEMoMSuyb56KK/xVTIiQh/P+00HrEX8Xa1tTFxxYojbj5SoVXS1MSSTZsCN9NNTU7mgqwsWu01lUuzs1mkF9GjliZ6NWTibB1vTlwcP5k4kY9kZwOwsrGR75aV0aSDnwya9u5u/lxXxx57p2ubz8c/Dx8O3An7sdxcfjF5Mil6D8SwoIleDbmM2Fj+Y8yYQCuOZYcO8VBlZaC+uMHrJRxu5Is0HT4fNbY7jMNeL9du2sTTtguGhenpVC1axOnp6W6GqFyiiV657q5x49h1xhmBO2w/sn49nwjqp0QdnzGGU1au5D9sfywjExJYOXcu3xjjDOHsv9FJDU+a6FVYyIqLA5yE9an8fK7IyQGcliH3l5dTrc00j/JAeXmg4y8R4Z6iIm4aNSqwvjg9nVhtEqnQRK/CjIjwxYICPpWfD8Dapia+uWcPK2yvjDtbW/nCtm1st3XNw8nG5ma+tnMnXnsBNS02lpy4uECPkDeMHKnDQapeaaJXYW1+ejp7FizgQzaB7Wlv54W6OjptcnvpwAEmLl8euNmnrrOTve3tUVHHb4zh/YYG6m1vodvb2nikqorN9r3eWlDAH6dPD1zkVqoveoaosDc2MTHQrvvCESM4cOaZzEhJAWBEbCxz09ICF3Z/v38/45Yvp9624Hnn8GEeq6oKJP4mrzfQTUO48fp8PFVTwxrbgdimlhYWBfVpf1l2NrWLFnFaaqqbYaoIpIleRRwRCfSWeHZmJs/NmBG4kPvRnBx+N3Uq2bbO/+naWr6+e3dg+zt372b0e+8Fnuuhigr+LWgAincbGlh26FBgvtsYfCf566Ar6Ivlf6qqAoOLGGOYtHw5d9sBLGJE+NzWrfzJJvYZKSk8d8opfNJWY8XHxJCqNzKpE6BnjYoqU5OTmZqcHJj/xeTJfHPcuMD8x3JymGl/DQAc6OqiyjZJBPjR3r2UtbdTOn8+AFdu3EhtZycr5s0D4OLSUrqB12fNAuCy9etJjInh+ZkzAbhiwwZy4uJ4fNo0AGavWsWMlBSeOuUUAO7fu5eF6elcnJ2NiHBFTg6zbQk9RoTNp5/OKHsHsYhwTV7eoO4fNTxpoldRzSNyRK+LF4wYwQVB678/YcIR2z8yZQqNQTduXZObizeoRH9Vbu4R4+cuzsw8orOv2amppAeVuj87ciT5QV0/rJw374iuBn46adIRrz9Ru4VQIaAjTCmlVITSEaaUUkoBmuiVUirqaaJXSqkop4leKaWinCZ6pZSKcprolVIqymmiV0qpKKeJXimlolxY3DAlInVA+Qk+PAc4MIjhDBaNa2A0roEL19g0roE5mbjGGWNyj7dRWCT6kyEiq/tzZ9hQ07gGRuMauHCNTeMamKGIS6tulFIqymmiV0qpKBcNif5RtwPog8Y1MBrXwIVrbBrXwIQ8roivo1dKKXVs0VCiV0opdQya6JVSKspFTKIXkYtFZJuI7BSRO3tZnyAiS+36FSJSNAQxjRGRf4rIFhHZJCJf7WWbxSLSICLr7N89oY7Lvm6ZiGywr3nUqC7i+LndX+tFZO4QxDQ1aD+sE5FGEbm9xzZDtr9E5LciUisiG4OWjRCR10Vkh/2f1cdjb7Db7BCRG0Ic049FZKs9Ti+KSGYfjz3mMQ9RbPeKyL6g43VpH4895uc3BHEtDYqpTETW9fHYkOyzvnKDa+eXMSbs/wAPsAuYAMQDpcApPbb5EvAbO70EWDoEcY0C5trpNGB7L3EtBl5xYZ+VATnHWH8p8HdAgAXACheO6X6cGz5c2V/AOcBcYGPQsh8Bd9rpO4Ef9vK4EcBu+z/LTmeFMKaLgFg7/cPeYurPMQ9RbPcCd/TjWB/z8zvYcfVY/1PgnqHcZ33lBrfOr0gp0Z8O7DTG7DbGdALPAlf02OYK4Ak7/TxwvohIKIMyxlQbY9ba6SZgC1AQytccRFcAfzCO5UCmiIwawtc/H9hljDnRO6JPmjHmX0B9j8XB59ETwJW9PPTDwOvGmHpjzCHgdeDiUMVkjHnNGOMfyHY5UDgYrzVQfeyv/ujP5zckcdkc8HHgmcF6vX7G1FducOX8ipREXwBUBM1XcnRCDWxjPxQNQPaQRAfYqqI5wIpeVi8UkVIR+buIzBiikAzwmoisEZGbe1nfn30aSkvo+8Pnxv7yyzfGVIPzYQXyetnGzX33OZxfYr053jEPldtstdJv+6iKcHN/nQ3UGGN29LE+5PusR25w5fyKlETfW8m8Z7vQ/mwTEiKSCvwZuN0Y09hj9Vqc6olZwC+AvwxFTMCZxpi5wCXArSJyTo/1bu6veOCjwHO9rHZrfw2EK/tORO4GvMBTfWxyvGMeCo8AE4HZQDVONUlPrp1rwHUcuzQf0n12nNzQ58N6WXZS+ytSEn0lMCZovhCo6msbEYkFMjixn5kDIiJxOAfyKWPMCz3XG2MajTHNdvpvQJyI5IQ6LmNMlf1fC7yI8/M5WH/2aahcAqw1xtT0XOHW/gpS46/Csv9re9lmyPedvSB3GfApYytye+rHMR90xpgaY0y3McYHPNbHa7pyrtk8cBWwtK9tQrnP+sgNrpxfkZLoVwGTRWS8LQ0uAV7usc3LgP/q9DXAG319IAaLrf97HNhijHmwj21G+q8ViMjpOPv8YIjjShGRNP80zsW8jT02exn4jDgWAA3+n5RDoM9Slhv7q4fg8+gG4KVetvk/4CIRybJVFRfZZSEhIhcD3wA+aoxp7WOb/hzzUMQWfF3nY328Zn8+v6FwAbDVGFPZ28pQ7rNj5AZ3zq/Bvtocqj+cViLbca7e322X3Ydz8gMk4lQF7ARWAhOGIKazcH5SrQfW2b9LgVuAW+w2twGbcFoaLAcWDUFcE+zrldrX9u+v4LgE+JXdnxuA4iE6jsk4iTsjaJkr+wvny6Ya6MIpRd2Ec11nGbDD/h9hty0G/ifosZ+z59pO4MYQx7QTp87Wf475W5eNBv52rGM+BPvrj/b8WY+TxEb1jM3OH/X5DWVcdvnv/edV0LZDss+OkRtcOb+0CwSllIpykVJ1o5RS6gRpoldKqSiniV4ppaKcJnqllIpymuiVUirKaaJXSqkop4leKaWi3P8DxFEDI3inV1sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(0, 40, 130)\n",
    "x1 = 2 * np.sin(x)\n",
    "x2 = 2 * np.cos(x)\n",
    "\n",
    "y1 = y1\n",
    "y2 = y2\n",
    "y3 = y3\n",
    "\n",
    "plt.title(\"Discover hidden signals from observed facts\")\n",
    "l1, = plt.plot(y1, 'c:', label = 'input signals (observed facts)')\n",
    "plt.plot(y2, 'c:')\n",
    "plt.plot(y3, 'c:')\n",
    "\n",
    "#l4, = plt.plot(2 * x1, 'm', label = 'output signals (hidden signals)')\n",
    "#plt.plot(2 * x2, 'm') # multiplies by 2 just for visualization purpose\n",
    "\n",
    "#plt.legend(handles = [l1, l4], loc = 'upper right')\n",
    "\n",
    "#plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create function that generates training samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "#INPUT SEQ LEN MUST BE EQUAL TO THE AMOUNT OF ROWS MINUS THE HEADER\n",
    "input_seq_len =21\n",
    "output_seq_len = 20\n",
    "x = np.linspace(0, 40, 130)\n",
    "train_data_x = x[:110]\n",
    "\n",
    "def true_output_signals(x):\n",
    "    x1 = 2 * np.sin(x)\n",
    "    x2 = 2 * np.cos(x)\n",
    "    return x1, x2\n",
    "\n",
    "def true_input_signals(x):\n",
    "    x1, x2 = true_output_signals(x)\n",
    "    y1 = pf[cols[0]]\n",
    "    y2 = pf[cols[1]]\n",
    "    y3 = pf[cols[2]]\n",
    "    return y1, y2, y3\n",
    "\n",
    "def noise_func(x, noise_factor = 2):\n",
    "    return np.random.randn(len(x)) * noise_factor\n",
    "\n",
    "def generate_samples_for_output(x):\n",
    "    x1, x2 = true_output_signals(x)\n",
    "    return x1+noise_func(x1, 0.5), \\\n",
    "           x2+noise_func(x2, 0.5)\n",
    "\n",
    "def generate_samples_for_input(x):\n",
    "    y1, y2, y3 = true_input_signals(x)\n",
    "    return y1+noise_func(y1, 2), \\\n",
    "           y2+noise_func(y2, 2), \\\n",
    "           y3+noise_func(y3, 2)\n",
    "\n",
    "def generate_train_samples(x = train_data_x, batch_size = 10):\n",
    "\n",
    "    total_start_points = len(x) - input_seq_len - output_seq_len\n",
    "    start_x_idx = np.random.choice(range(total_start_points), batch_size)\n",
    "    \n",
    "    input_seq_x = [x[i:(i+input_seq_len)] for i in start_x_idx]\n",
    "    output_seq_x = [x[(i+input_seq_len):(i+input_seq_len+output_seq_len)] for i in start_x_idx]\n",
    "    \n",
    "    input_seq_y = [generate_samples_for_input(x) for x in input_seq_x]\n",
    "    output_seq_y = [generate_samples_for_output(x) for x in output_seq_x]\n",
    "    \n",
    "    ## return shape: (batch_size, time_steps, feature_dims)\n",
    "    return np.array(input_seq_y).transpose(0, 2, 1), np.array(output_seq_y).transpose(0, 2, 1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize one data sample from training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show(*args, **kw)>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3X10VPW97/H3lxCB+lgBqeUhgVuQUlDEyIO0SqVVQE9RKlWbdQ6hKvUB21Nvq/S6lqIL1tGjt2pXqy09KpxDalVOVWq9t3UprKOVqsGiKCggBcnFAqJwpEB4yPf+MTshCTOzJ5mZ7Jk9n9daWZn57Yf5zk7ynV+++7d/29wdERGJry5RByAiIvmlRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGnRC8iEnNK9CIiMdc16gAAevXq5ZWVlVGHISJSVFauXPmRu/cOW68gEn1lZSV1dXVRhyEiUlTMbHMm66l0IyISc0r0IiIxp0QvIhJzBVGjFylkBw8epL6+nv3790cdipSo7t27069fP8rLyzu0vRK9SIj6+nqOP/54KisrMbOow5ES4+7s3LmT+vp6Bg4c2KF9FG3pZtu2WlasqGT58i6sWFHJtm21UYckMbV//3569uypJC+RMDN69uyZ1X+URdmj37atlvfem0Vj414AGho28957swDo06c6ytAkppTkJUrZ/v4VZY9+48Zbm5N8k8bGvWzceGtEEYmIFK6iTPQNDR+0q11EpJQVZaLv1m1Au9pFOlM+zh+dc845OYistU2bNvHrX/865/uVhLlz53LvvfdGHQZQpIl+0KD5dOnymVZtXbp8hkGD5kcUkUhC0/mjhobNgDefP8o22b/yyiu5CbAFJfrSUZSJvk+fak47bQHdulUARrduFZx22gKdiJXI5ev80XHHHQfA8uXLmTBhApdddhlDhw6luroadwcSc0bdcsstjB49mtGjR7NhwwYAampqWLJkyVH7mjNnDi+99BIjR47kvvvuS/q677zzDqNHj2bkyJGcfvrprF+/HoDFixc3t3/3u9/l8OHDADz66KMMGTKE8847j2uuuYbZs2cD8OSTTzJ8+HDOOOMMzj333JTvc//+/cycOZMRI0Zw5plnsmzZMgAWLlzItGnTmDRpEoMHD+bmm29u3uaPf/wj48aNY9SoUUyfPp09e/ak3P+cOXMYNmwYp59+Oj/84Q8B+N3vfseYMWM488wz+drXvsa2bduARI98xowZXHDBBVRWVvLb3/6Wm2++mREjRjBp0iQOHjyY9ri39P777zNp0iTOOussvvKVr/Duu++mjDEv3D3yr7POOstFCtWaNWsyXnfZMvNly0jyZVnFcOyxxwb7X+YnnHCCb9myxQ8fPuxjx471l156yd3dKyoqfN68ee7uvmjRIr/ooovc3X3GjBn+5JNPJt1X0zqpzJ492xcvXuzu7g0NDb53715fs2aNX3zxxX7gwAF3d7/uuut80aJFvnXrVu/fv79v377dGxoa/JxzzvEbbrjB3d2HDx/u9fX17u7+ySefpHy9e++912tqatzdfe3atd6/f3/ft2+fP/rooz5w4EDftWuX79u3zwcMGOAffPCB79ixw7/yla/4nj173N39rrvu8jvuuCPpvnfu3OlDhgzxxsbGVnF8/PHHzW2/+tWv/KabbnJ399tvv93Hjx/vBw4c8FWrVnmPHj38ueeec3f3Sy65xJ966qm0x/3222/3e+65x93dzz//fF+3bp27u//5z3/2r371q2mPezLJfg+BOs8gxxbl8EqRQtWt24CgbHN0e66MHj2afv36ATBy5Eg2bdrEl7/8ZQCuvPLK5u8/+MEPsn6tcePGMX/+fOrr65k2bRqDBw/mhRdeYOXKlZx99tkA7Nu3j1NOOYVXX32VCRMm0Lt3Ytbcyy+/nHXr1gEwfvx4ampq+Na3vsW0adNSvt7LL7/MjTfeCMDQoUOpqKho3sfEiRM58cQTARg2bBibN29m165drFmzhvHjxwNw4MABxo0bl3TfJ5xwAt27d+fqq6/moosu4uKLLwYSF8RdfvnlfPjhhxw4cKDVRUmTJ0+mvLycESNGcPjwYSZNmgTAiBEj2LRpU/N66Y77nj17eOWVV5g+fXpzW0NDQ8pjkA9FWbrJhC6okih0xvmjbt26NT8uKyvj0KFDzc9bjrduety1a1caGxuBxH/wBw4cyPi1vv3tb7N06VJ69OjBhRdeyIsvvoi7M2PGDFatWsWqVat47733mDt37lGv39IvfvEL5s2bx5YtWxg5ciQ7d+5Mup4HZahkkr1vd+frX/96cyxr1qzh4YcfTrp9165dee211/jmN7/J008/3Zy0b7zxRmbPns3q1av55S9/2erCpKbX7NKlC+Xl5c3vr0uXLqHHvUljYyMnnXRSc4yrVq1i7dq1Kd9nPmSU6M1sk5mtNrNVZlYXtJ1sZs+b2frg+2eDdjOzn5rZBjN7y8xG5fMNJJOvE2IiYaI+f/T44483f2/q2VZWVrJy5UoAnnnmmeba8vHHH8+nn36adn8bN25k0KBBfO973+Mb3/gGb731FhMnTmTJkiVs374dgI8//pjNmzczZswYli9fzs6dOzl48CBPPvlk837ef/99xowZw5133kmvXr3YsmVL0tc799xzqa1N/J2uW7eODz74gNNOOy1lfGPHjuVPf/pTc1187969zf8BtLVnzx52797NlClTuP/++1m1ahUAu3fvpm/fvgAsWrQo7fFIJdlxb3LCCScwcODA5uPh7rz55psdep2Oak/p5qvu/lGL53OAF9z9LjObEzy/BZgMDA6+xgAPBd87TboTYjphK/nWp091ZL9nDQ0NjBkzhsbGRh577DEArrnmGqZOncro0aOZOHEixx57LACnn346Xbt25YwzzqCmpiZpqefxxx9n8eLFlJeX87nPfY7bbruNk08+mXnz5nHBBRfQ2NhIeXk5P//5zxk7dixz585l3LhxnHrqqYwaNar5JO2PfvQj1q9fj7szceJEzjjjjKTxX3/99Vx77bWMGDGCrl27snDhwlY9+bZ69+7NwoULufLKK5vLIfPmzWPIkCFHrfvpp58ydepU9u/fj7s3n4CeO3cu06dPp2/fvowdO5a//vWv7TjiCcmOe0u1tbVcd911zJs3j4MHD3LFFVekPAb5YOn+VWpeyWwTUNUy0ZvZe8AEd//QzE4Flrv7aWb2y+DxY23XS7X/qqoqz+UdppYv7wIke1/GhAmNOXsdKQ1r167li1/8YtRhhGq6U1uvXr2iDgVIjJSpq6vjZz/7WdSh5FVnHfdkv4dmttLdq8K2zbRG78AfzWylmc0K2vo0Je/g+ylBe1+g5f9l9UFbp9EFVSIiR2Rauhnv7lvN7BTgeTNLNwg02dmYo7rXwQfGLIABA3KbgAcNmt9q0jPQBVUSfy1HgbTXH/7wB2655ZZWbQMHDuSpp57q8D5ramqoqanptNdr6dJLLz2qBHP33Xdz4YUX5mT/LWVz3DtLRqWbVhuYzQX2ANdQoKUbSJyQ3bjxVhoaPqBbtwEMGjRf9XnpkGIp3Ui8ZVO6Ce3Rm9mxQBd3/zR4fAFwJ7AUmAHcFXx/JthkKTDbzH5D4iTs7nRJPl/CTojpg0BESkUmpZs+wFPB2NCuwK/d/f+a2evAE2Z2FfAB0HQ1wHPAFGADsBeYmfOos6T57EWklIQmenffCBw1DsjddwITk7Q7cENOossTDb8UkVIS2ytj09F89iJSSkoy0Wv4peRT7epaKu+vpMsdXai8v5La1Z1/RfbChQvZunVrh7fXFMb51dlz1Zdkotd89pIvtatrmfW7WWzevRnH2bx7M7N+N6vTk70SvbRUkok+6vlIJL5ufeFW9h5sff5n78G93PpC9vcz/slPfsLw4cMZPnw4999/P5s2bWL48OHNy++9917mzp3LkiVLqKuro7q6mpEjR7Jv3z7NVV/qc9VnMpdxvr8KbT76v/1tsb/ySoUvW2b+yisV/re/LY46JIlQe+ajt7nmzOWoL5ub3Xz0dXV1Pnz4cN+zZ49/+umnPmzYMH/jjTf8S1/6UvM699xzj99+++3u7n7eeef566+/3rxMc9UX/1z1mo8+hzT0UrIx4MQBbN599Hz0A07M7vzPyy+/zKWXXto8Idm0adN46aWX2rUPzVVfunPVl2TpJp183QpOSsP8ifP5THnr8z+fKf8M8ydmd/7Hk1zBvmvXruZ55oFW86gno7nqS3eueiX6NjIZeqmbmkgq1SOqWfAPC6g4sQLDqDixggX/sIDqEdn9N3juuefy9NNPs3fvXv7+97/z1FNPMXnyZLZv387OnTtpaGjg2WefbV4/2Vzzmqu+dOeqV+mmjbBbwam0I2GqR1RnndjbGjVqFDU1NYwePRqAq6++mrPPPpvbbruNMWPGMHDgQIYOHdq8fk1NDddeey09evRgxYoVgOaqh9Kdq77dk5rlQz4mNeuotokcEkMvm0blrFhRmeKDoIJx4zZ1YqTSWeIwqZnmqo9GLo97Z8xHXzLChl7qqloRKTYq3SSRbubLsNKOSCHSXPVHlOJc9SrdtFNYaUfTH8fP2rVrGTp0aMpRIyL55u68++67Kt10lnSlnaYPgUSP35tP1GpUTnHr3r07O3fuTDu8TyRf3J2dO3fSvXv3Du9DPfoc0onaeDp48CD19fWh49RF8qV79+7069eP8vLyVu05u8OUZE4nauOpvLy81RWRIsVGpZsc0vTHIlKIlOhzSNMfi0ghUqLPoUymP9b0CSLS2VSjz7F0Y/A1fYKIREE9+k6kmTFFJApK9J1IM2OKSBSU6DtR2KgcXXAlIvmgRN+JwkblqLQjIvmgRN+JNDOmiERBo246mWbGFJHOph59AdEFVyKSD0r0BSSstKMROSLSESrdFJhUpZ1MLrbSXPgikkzGPXozKzOzv5jZs8HzgWb2qpmtN7PHzeyYoL1b8HxDsLwyP6GXlrARORqaKSKptKd0831gbYvndwP3uftg4BPgqqD9KuATd/8CcF+wnmQpbESOhmaKSCoZJXoz6wdcBPxb8NyA84ElwSqLgEuCx1OD5wTLJ5ruwZa1sIutNDRTRFLJtEd/P3Az0Bg87wnscvdDwfN6oG/wuC+wBSBYvjtYvxUzm2VmdWZWt2PHjg6GXzrCRuRkctWtTuSKlKbQRG9mFwPb3X1ly+Ykq3oGy440uC9w9yp3r+rdu3dGwZaysBE56T4IVL8XKW2ZjLoZD3zDzKYA3YETSPTwTzKzrkGvvR+wNVi/HugP1JtZV+BE4OOcR16C0l1s1dSebNTNihWVKev3GpUjEn+hid7dfwz8GMDMJgA/dPdqM3sSuAz4DTADeCbYZGnwfEWw/EUvhDuQl4BUHwSZzpqpoZki8ZTNBVO3ADeZ2QYSNfiHg/aHgZ5B+03AnOxClGxp1kyR0tauRO/uy9394uDxRncf7e5fcPfp7t4QtO8Pnn8hWL4xH4FL5jRrpkhp0xQIJUCzZoqUNk2BUCKynTVTNXyR4qUevYSWdlTDFyluSvQSWtpRDV+kuKl0I0D60k5YDT+srKOyj0i0lOglVLoaftj0yZlMrywi+aXSjYRKV8MPK+uo7CMSPSV6CZWuhh9W1smk7KPJ1kTyS6UbyUiqGn7Y0Mxsyj4ikhvq0UtWwoZmZlP2EZHcUKKXrIQNzcym7CMiuaHSjWQt3dDMdMt1Ra5I51CPXiKTiytydTJXJJwSvUQm2ytyNTWDSGZUupFIZXNFbroPgqaLtVT2EVGPXgpY2A1T0n0QqOwjckTRJnr9kcZfWA0/3QeByj4iRxRlotcfaWkIq+Gn+yDIpuwjEjdFWaMPq81KfKSr4bc8adu2Dp9oSz10U2P4pZQUZaLXH6k0SfVBMGjQ/FbTK8DRZZ+wMfwicVGUpZuwk3Qi2ZR9QOeAJF6Kskcf1lsTgY6XfTTZmsSNuXvUMVBVVeV1dXXt2kZjpCVfVqyoTFHWqWDcuE2dH5BICma20t2rwtYryh49hM+vItJROgckcVOUNXqRfNI5IIkbJXqRNsJO1IoUGyV6kTbCRuyIFJuirdGL5JPOAUmchPbozay7mb1mZm+a2TtmdkfQPtDMXjWz9Wb2uJkdE7R3C55vCJZX5vctiIhIOpmUbhqA8939DGAkMMnMxgJ3A/e5+2DgE+CqYP2rgE/c/QvAfcF6IiISkdBE7wl7gqflwZcD5wNLgvZFwCXB46nBc4LlE83MchaxiIi0S0YnY82szMxWAduB54H3gV3ufihYpR7oGzzuC2wBCJbvBnom2ecsM6szs7odO3Zk9y5ERCSljBK9ux9295FAP2A08MVkqwXfk/Xej7r81t0XuHuVu1f17t0703hFRKSd2jW80t13AcuBscBJZtY0aqcfsDV4XA/0BwiWnwh8nItgRUSk/TIZddPbzE4KHvcAvgasBZYBlwWrzQCeCR4vDZ4TLH/RC2FCHRGREpXJOPpTgUVmVkbig+EJd3/WzNYAvzGzecBfgIeD9R8G/sPMNpDoyV+Rh7hF8kqT5kmchCZ6d38LODNJ+0YS9fq27fuB6TmJTiQCmqZY4kZTIIi0ofvJStwo0Yu0oWmKJW6U6EXa0DTFEjdK9CJtaJpiiRslepE2NE2xxI2mKRZJQtMUS5zEtke/bVstK1ZUsnx5F1asqGTbttqoQxIRiUQse/QaBy0ickQse/QaBy0ickQsE73GQYuIHBHLRK9x0CIiR8Qy0WsctIjIEbFM9BoHLSJyRCxH3YDGQYuINIllj15ERI5QohcRiTklehGRmFOiFxGJOSV6KVmaD0lKRWxH3Yiko/mQpJSoRy8lSfMhSSlRopeSpPmQpJSUbKJXfba0aT4kKSUlmeib6rMNDZsBb67PKtmXDs2HJKWkJBO96rOi+ZCklJTkqBvVZwU0H5KUjpLs0as+KyKlpCQTveqzIlJKQhO9mfU3s2VmttbM3jGz7wftJ5vZ82a2Pvj+2aDdzOynZrbBzN4ys1H5fhPtpfqsiJSSTGr0h4D/6e5vmNnxwEozex6oAV5w97vMbA4wB7gFmAwMDr7GAA8F3wuK6rMiUipCe/Tu/qG7vxE8/hRYC/QFpgKLgtUWAZcEj6cC/+4JfwZOMrNTcx65iIhkpF01ejOrBM4EXgX6uPuHkPgwAE4JVusLbGmxWX3QJiIiEcg40ZvZccB/Av/s7v+dbtUkbZ5kf7PMrM7M6nbs2JFpGCIi0k4ZJXozKyeR5Gvd/bdB87amkkzwfXvQXg/0b7F5P2Br2326+wJ3r3L3qt69e3c0fhERCZHJqBsDHgbWuvtPWixaCswIHs8AnmnR/k/B6JuxwO6mEo+IiHS+TEbdjAf+EVhtZquCtv8F3AU8YWZXAR8A04NlzwFTgA3AXmBmTiMWEZF2CU307v4yyevuABOTrO/ADVnGJSIiOVKSV8aKiJQSJXoRkZhTohcRiTkleokt3UVMJKEk56OX+Gu6i1jTDWaa7iIGaI4jKTnq0Uss6S5iIkco0Uss6S5iIkco0Seh2m7x013ERI5Qom+jqbbb0LAZ8ObarpJ9cdFdxESOUKJvQ7XdeNBdxESO0KibNlTbjQ/dRUwkQT36NlTbFZG4UaJvQ7VdEYkbJfo2VNsVkbhRjT4J1XZFJE7Uo5eipmseRMKpRy9FS/PZiGRGPfoOUC+yMOiaB5HMqEffTupFFg5d8yCSGfXo20m9yMKhax5EMqNE307qRRYOXfMgkhkl+nZSL7Jw6JoHkcyoRt9OgwbNb1WjB/Uio6RrHkTCqUffTupFikixUY++A9SLFJFioh69FDRdsyCSPfXopWDpmgWR3FCPXgqWrlkQyY3QRG9mj5jZdjN7u0XbyWb2vJmtD75/Nmg3M/upmW0ws7fMbFQ+gy9EKjXkjq5ZEMmNTHr0C4FJbdrmAC+4+2DgheA5wGRgcPA1C3goN2EWB91YPLd0zYJIboQmenf/L+DjNs1TgUXB40XAJS3a/90T/gycZGan5irYQqdSQ27pyleR3Ohojb6Pu38IEHw/JWjvC2xpsV590FYSVGrILV2zIJIbuR51Y0naPOmKZrNIlHcYMCAe/4p36zYgKNsc3S4do2sWRLLX0R79tqaSTPB9e9BeD/RvsV4/YGuyHbj7Anevcveq3r17dzCMwqJSg4gUoo4m+qXAjODxDOCZFu3/FIy+GQvsbirxlAKVGkSkEIWWbszsMWAC0MvM6oHbgbuAJ8zsKuADYHqw+nPAFGADsBeYmYeYC5pKDSJSaEITvbtfmWLRxCTrOnBDtkGJiEju6MrYAlNqF1wV6vst1LhEOkKJvoDk84KrsMSVzfKOJsVCvcCsUOMS6aiiTfTZJq58vnZHhV1w1dH3HJa4slmeTVLMxQVm2fwsUm2rC98kbopy9sqwWQ0zmfVw27ZaNm68lYaGD+jWbQCDBs3PeFnYa6faFmDduuvZunUBcBgo4/Ofn8WQIQ8G+0p9wVU27zld4urTpzqr5U2PU22b7ngmu+agKfYm+fpZpNtWF75J3BRlos82caX7IwfSJo+wpJdu20SSbzn9z+Hm50OGPJj2gqts3nNY4sp2ebpl6Y91GYkPvLbKQrfN9meRbltd+CZxU5Slm2wTU7o/8rDkkW7fYdsmevJHa2pPd8FVNu85bHKwbJaHbZv+mCRL8jS35/NnkW5bXfgmcVOUiT7bxJXujzwsoabbd3jPN31iS3fBVTbvOSxxZbM8bNv0H0AVKWKuCN226b0l3z78Z5FuW134JnFTlIk+28SVTQ813b7Dp9UtS/GOUrUfkc17Dktc2SwP2zabD6B8/izCXrtPn2rGjdvEhAmNjBu3SUleilpR1uib/uhSnWgLWz5o0PxW9Vto/UeeblnYvtNt+/nPz2pTo6e5HTK7dV5H33PYFbvZLE+3rGfPKUnfc8+eU7L+OWXzswjbViROLHExa7Sqqqq8rq6uU1+zo6NustkvpB91s2JFZYqTgBWMG7cp27cciWzfUz5/FiLFzsxWuntV6HqlmugL0fLlXUg+q7MxYUJjZ4eTE3F8TyKFItNEX5Q1+riK463z4vieRIqNEn0BKdZhfemuTi3W9yQSJ0r0BaQYh/WFTYFQjO9JJG5Uo5esxPEEskixUI0+hgpx6lzNCyNS+JToi0Qms0RG8UGgk60ihU+JvkhkMoVxPudQT/UhopOtIoVPib5IZDNRG2Q/b3uqDxGdbBUpfEU5BUIpCps6N5u57CH9VaRhUyTrhugihU09+iKRzQRg2ZZ9dMJVpLgp0ReJsBJJNnPZh30Q6ISrSHFT6aaIhM0gCclnY0y0dazsA+GzSIpIYVOij5FUHwRhiTqs/q8pfUWKmxJ9Cch23vemfSixixQnJfoS0dGyj4gUPyV6AdRjF4kzjboREYm5vCR6M5tkZu+Z2QYzm5OP1xCRo9WurqXy/kq63NGFyvsrqV1dm/HybLYFuP7319P1zq7YHUbXO7ty/e+vL4i4wpaXgpwnejMrA34OTAaGAVea2bBcv45IlNIlNYgmcdWuruU7z3yHzbs34zibd2/mO898J6Pl2WzbdDweqnuIw34YgMN+mIfqHuL6319P7epaZj49s9W2M5+e2WrfqZZns22my0vhAyjn89Gb2ThgrrtfGDz/MYC7/0uqbTQfvRSTpqTW1nVV1/HgRQ82J8UDhw80Lzum7BgemfoIQMpl1SOq024btvz7/+f77Ny386i4evboyUc3f0Svf+2VcjnQ4W0/uvkjut7ZtTnJt1RmZZzU/aTI4kq3/IHJDzDz6ZkcbDzY3F7epZxHL3m0+VinWg50eNtMlmcqspuDm9llwCR3vzp4/o/AGHefnWobJXopJumS2qHbDkWWuJK1N/HbHbvD0r6vjm6bz33nM65UxyzqD6CPbv4oZcxtZZro8zHqJtmRPerTxMxmAbMABgzQpfRSPJIl+ZbtqRJuukTctCxs247suzOUWVnKD79Uxytq+TjWhfpzzMfJ2Hqgf4vn/YCtbVdy9wXuXuXuVb17985DGCL5UWZl7WrvLE09zVTt6ZZnsy3ArLNmJV0+66xZkcYVtrxU5CPRvw4MNrOBZnYMcAWwNA+vIxKJdEkNoktcD0x+gPIu5a3ay7uU88DkBwDSLs9mW4AHL3qQ66qua/6wK7Oy5nMWD0x+gGPKjmm17TFlx7Tad6rl2WwbtryUPoBynujd/RAwG/gDsBZ4wt3fyfXriEQlXVKD/CbUdMurR1Tz6CWPUnFiBYZRcWJFq5N76ZZns23L43LotkP47c6h2w41H4/qEdU8MvWRVts2nVwOW57NtmHLC/UDKC/cPfKvs846y0XiZPFbi73ivgq3ueYV91X44rcWZ7QsF8slc9kc60L4OQJ1nkGOzfmom47QqBsRkfbLdNSNpkAQEYk5JXoRkZhTohcRiTklehGRmFOiFxGJuYIYdWNmO4Cjb1qamV5A5pNDdB7F1T6Kq/0KNTbF1T7ZxFXh7qFTCxREos+GmdVlMryosymu9lFc7VeosSmu9umMuFS6ERGJOSV6EZGYi0OiXxB1ACkorvZRXO1XqLEprvbJe1xFX6MXEZH04tCjFxGRNIo60ZvZJDN7z8w2mNmcqONpYmabzGy1ma0ys8hmazOzR8xsu5m93aLtZDN73szWB98/WyBxzTWz/xccs1VmNiWCuPqb2TIzW2tm75jZ94P2SI9ZmrgiPWZm1t3MXjOzN4O47gjaB5rZq8Hxejy4L0UhxLXQzP7a4niN7My4WsRXZmZ/MbNng+f5P16ZTHFZiF9AGfA+MAg4BngTGBZ1XEFsm4BeBRDHucAo4O0Wbf8KzAkezwHuLpC45gI/jPh4nQqMCh4fD6wDhkV9zNLEFekxI3Hb0OOCx+XAq8BY4AngiqD9F8B1BRLXQuCyKH/HgphuAn4NPBs8z/vxKuYe/Whgg7tvdPcDwG+AqRHHVFDc/b+Aj9s0TwUWBY8XAZd0alCkjCty7v6hu78RPP6UxI1z+hLxMUsTV6Q8YU/wtDz4cuB8YEnQHsXxShVX5MysH3AR8G/Bc6MTjlcxJ/q+wJYWz+spgF/+gAN/NLOVwU3QC0kfd/934F9pAAACaUlEQVQQEgkEOCXieFqabWZvBaWdTi8ptWRmlcCZJHqDBXPM2sQFER+zoAyxCtgOPE/iv+xdnrjTHET0d9k2LndvOl7zg+N1n5l16+y4gPuBm4HG4HlPOuF4FXOityRtBfGpDYx391HAZOAGMzs36oCKwEPA/wBGAh8C/zuqQMzsOOA/gX929/+OKo62ksQV+TFz98PuPhLoR+K/7C8mW61zozo6LjMbDvwYGAqcDZwM3NKZMZnZxcB2d1/ZsjnJqjk/XsWc6OuB/i2e9wO2RhRLK+6+Nfi+HXiKxB9AodhmZqcCBN+3RxwPAO6+LfjjbAR+RUTHzMzKSSTTWnf/bdAc+TFLFlehHLMgll3AchK18JPMrGuwKNK/yxZxTQpKYO7uDcCjdP7xGg98w8w2kSg1n0+ih5/341XMif51YHBwxvoY4ApgacQxYWbHmtnxTY+BC4C302/VqZYCM4LHM4BnIoylWVMiDVxKBMcsqJc+DKx195+0WBTpMUsVV9THzMx6m9lJweMewNdInD9YBlwWrBbF8UoW17stPqyNRB28U4+Xu//Y3fu5eyWJfPWiu1fTGccr6jPQWZ69nkJiBML7wK1RxxPENIjECKA3gXeijAt4jMS/9AdJ/Ad0FYma4AvA+uD7yQUS138Aq4G3SCTWUyOI68sk/m1+C1gVfE2J+piliSvSYwacDvwleP23gduC9kHAa8AG4EmgW4HE9WJwvN4GFhOMzIniC5jAkVE3eT9eujJWRCTmirl0IyIiGVCiFxGJOSV6EZGYU6IXEYk5JXoRkZhTohcRiTklehGRmFOiFxGJuf8PePLz5FB8ADoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "input_seq, output_seq = generate_train_samples(batch_size=100)\n",
    "#print(input_seq)\n",
    "#print('**********************************')\n",
    "#print(output_seq)\n",
    "print(input_seq_len)\n",
    "i1, i2, i3= plt.plot(range(input_seq_len), input_seq[0], 'yo', label = 'input_seqs_one_sample')\n",
    "o1, o2 = plt.plot(range(input_seq_len,(input_seq_len+output_seq_len)), output_seq[0], 'go', label = 'output_seqs_one_sample')\n",
    "plt.legend(handles = [i1, o1])\n",
    "plt.show\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "from build_model_multi_variate import * \n",
    "\n",
    "## Parameters\n",
    "learning_rate = 0.01\n",
    "lambda_l2_reg = 0.003  \n",
    "\n",
    "## Network Parameters\n",
    "# length of input signals\n",
    "input_seq_len = 15 \n",
    "# length of output signals\n",
    "output_seq_len = 20 \n",
    "# size of LSTM Cell\n",
    "hidden_dim = 64 \n",
    "# num of input signals\n",
    "input_dim = 3 \n",
    "# num of output signals\n",
    "output_dim = 2 \n",
    "# num of stacked lstm layers \n",
    "num_stacked_layers = 2 \n",
    "# gradient clipping - to avoid gradient exploding\n",
    "GRADIENT_CLIPPING = 2.5 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training losses: \n",
      "64.049706\n",
      "44.155674\n",
      "31.140558\n",
      "31.73363\n",
      "24.942982\n",
      "19.696169\n",
      "19.144077\n",
      "17.704527\n",
      "17.88693\n",
      "16.503464\n",
      "14.495393\n",
      "16.186893\n",
      "14.292113\n",
      "14.304227\n",
      "14.283773\n",
      "12.850826\n",
      "12.171898\n",
      "12.935734\n",
      "11.825614\n",
      "11.667006\n",
      "12.701382\n",
      "13.095595\n",
      "11.811556\n",
      "11.664649\n",
      "11.146628\n",
      "11.221983\n",
      "11.15009\n",
      "11.066132\n",
      "9.958694\n",
      "10.550867\n",
      "10.86588\n",
      "10.012608\n",
      "10.768536\n",
      "10.890377\n",
      "10.318668\n",
      "10.396291\n",
      "10.620685\n",
      "10.417299\n",
      "9.929403\n",
      "9.795001\n",
      "10.114215\n",
      "10.382643\n",
      "9.692801\n",
      "9.9228\n",
      "9.328102\n",
      "10.438609\n",
      "9.800053\n",
      "9.083695\n",
      "9.81049\n",
      "8.999154\n",
      "9.56674\n",
      "9.733073\n",
      "9.438743\n",
      "8.844038\n",
      "9.486154\n",
      "9.450358\n",
      "8.934301\n",
      "8.843366\n",
      "9.492493\n",
      "9.710927\n",
      "11.414139\n",
      "9.6475315\n",
      "9.108552\n",
      "9.009766\n",
      "9.574531\n",
      "10.318262\n",
      "10.263507\n",
      "9.134096\n",
      "9.32289\n",
      "9.066795\n",
      "8.026251\n",
      "10.083223\n",
      "9.919445\n",
      "8.713457\n",
      "10.054895\n",
      "9.382416\n",
      "8.679368\n",
      "8.648631\n",
      "8.705346\n",
      "8.117562\n",
      "8.662502\n",
      "9.188763\n",
      "8.425931\n",
      "9.901917\n",
      "8.623086\n",
      "10.127508\n",
      "9.052088\n",
      "8.2649765\n",
      "8.54978\n",
      "9.042678\n",
      "9.006031\n",
      "8.903379\n",
      "8.971109\n",
      "8.765441\n",
      "9.078067\n",
      "8.826898\n",
      "9.050326\n",
      "9.2550335\n",
      "8.381663\n",
      "8.935626\n",
      "Checkpoint saved at:  ./multivariate_ts_model0\n"
     ]
    }
   ],
   "source": [
    "total_iteractions = 100\n",
    "batch_size = 16\n",
    "KEEP_RATE = 0.5\n",
    "train_losses = []\n",
    "val_losses = []\n",
    "\n",
    "x = np.linspace(0, 40, 130)\n",
    "train_data_x = x[:110]\n",
    "\n",
    "rnn_model = build_graph(feed_previous=False)\n",
    "\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    sess.run(init)\n",
    "    \n",
    "    print(\"Training losses: \")\n",
    "    for i in range(total_iteractions):\n",
    "        batch_input, batch_output = generate_train_samples(batch_size=batch_size)\n",
    "        \n",
    "        feed_dict = {rnn_model['enc_inp'][t]: batch_input[:,t] for t in range(input_seq_len)}\n",
    "        feed_dict.update({rnn_model['target_seq'][t]: batch_output[:,t] for t in range(output_seq_len)})\n",
    "        _, loss_t = sess.run([rnn_model['train_op'], rnn_model['loss']], feed_dict)\n",
    "        print(loss_t)\n",
    "        \n",
    "    temp_saver = rnn_model['saver']()\n",
    "    save_path = temp_saver.save(sess, os.path.join('./', 'multivariate_ts_model0'))\n",
    "        \n",
    "print(\"Checkpoint saved at: \", save_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inference* end of train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./multivariate_ts_model0\n"
     ]
    }
   ],
   "source": [
    "test_seq_input = np.array(generate_samples_for_input(train_data_x[-15:])).transpose(1,0)\n",
    "\n",
    "rnn_model = build_graph(feed_previous=True)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    sess.run(init)\n",
    "    \n",
    "    saver = rnn_model['saver']().restore(sess, os.path.join('./', 'multivariate_ts_model0'))\n",
    "    \n",
    "    feed_dict = {rnn_model['enc_inp'][t]: test_seq_input[t].reshape(1, -1) for t in range(input_seq_len)}\n",
    "    feed_dict.update({rnn_model['target_seq'][t]: np.zeros([1, output_dim], dtype=np.float32) for t in range(output_seq_len)})\n",
    "    final_preds = sess.run(rnn_model['reshaped_outputs'], feed_dict)\n",
    "    \n",
    "    final_preds = np.concatenate(final_preds, axis = 0)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize predictions over true signals "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xt8FOW9+PHPNxiXEy9HBZqqkA1pKRe5qfFCQQpaPaB4rVQQrdh6cqx6Tttf9ZT+OK1oT/qzracXqy1Na4WWlHpptVRpvZVokShGD1huQQgBUjAiiJWCEcz398dM0s2yu9nNbHZ2dr7v12tf2Z19Zp5nZybPd+Z5ZuYRVcUYY0z4FPldAGOMMf6wAGCMMSFlAcAYY0LKAoAxxoSUBQBjjAkpCwDGGBNSFgAKlIiUi4iKyBHu5z+IyHU5yHeeiCzq7XzcvJpF5JO5yMuYQmQBwEduBXZARPaJSKuIPCAiR/dGXqo6VVUXplmmwFeqIrJARP7b73KkIz5Y59vyTOGyAOC/i1X1aOA04Azgv+ITiMO2VRZZ5WiMBYC8oap/Bf4AjAQQkToRqRaRF4D9QIWI/LOI3C8iO0XkryLy3yLSx03fR0TuFpG3RKQJuCh2+e7yboj5/K8isl5E3hWRdSJymoj8EigDfu+elfynm/ZsEVkhIntFZLWITIpZzmARec5dztNA/2S/UUSOF5HHRWSXiLztvh8YV8ZviMgL7vKeEpH+Md9fKyJbRWS3iMxNkU8VMAv4T/d3/N6d3iwiXxGR14C/i8gR7pHyR2Pm7XLmICLTRGSV+9tXiMjoFPl+XEReFpF33L8fj/muy5lVXFPZ8+7fvW55x4nIbHc9/NBd3gYROa+ny0tQ1jNFpEFE/uaefX435ru0t7eI3NuRr4hMEpGWuHw6yykiRSIyR0Q2u9vwIRE5wf2u46zlOhHZ5u7Hc2OW00dE/q8777si8oqIDHK/G+aWZY+INIrIp2Pmu9Ddv98V53/m1mTbL5RU1V4+vYBm4JPu+0HAWuAb7uc6YBtwCnAEUAw8BvwEOAr4ELAS+Dc3/Y3ABnc5JwDLAAWOiFneDe776cBfcc44BPgoEI0vk/v5ZGA3cCHOAcP57ucB7vf1wHeBCDAReBdYlOT39gM+BZQAxwAPA4/FfF8HbAY+BvyT+/ku97sRwD43j4ib56HYssbltQD47wTre5W7jv7JnabARxPNh3NW9iZwFtAHuM5dRiRBficAbwPXuttrpvu5X5L1Oq9jPQHlsdvKnTbb/X1fcrf9VcA7wAk9WV6C8tYD17rvjwbO9rq9gUlAS4p9/IvAi8BAd/6fAIvjyvxTd9uPAdqA4e73twF/AYbi7LNjcPano4DtwPXuej8NeAs4xZ1vJ3CO+/544DS//+/z6WVnAP57TET2AsuB54Bvxny3QFXXquohnApmKvBFVf27qr4JfA+Y4ab9NPB9Vd2uqnuA/5cizxuAb6vqy+rYpKpbk6S9BliqqktVtV1VnwYagAtFpAwniHxNVdtU9Xng98kyVdXdqvobVd2vqu8C1cAn4pI9oKobVfUA8BAw1p1+JfC4qj6vqm3A14D2FL8xmXvcdXQgjbT/CvxEVV9S1Q/U6UNpA85OkPYi4HVV/aWqHlLVxTgB+eIelLHDmzjb9KCqPgg0Endm58FB4KMi0l9V96nqi+70rG3vBP4NmKuqLe42nAdcKV2b4+5Q1QOquhpYjVPRg7PP/peqNrr77GpV3Q1MA5pV9QF3vb8K/AZnf+n4nSNE5FhVfdv93rgsAPjvMlU9TlWjqnpTXMW0PeZ9FOdIcKd7ar4X5wjqQ+73J8WlT1ahg3MEvDnN8kWB6R15uvlOAE5083xbVf+eTr4iUiIiP3Gbcf6G01RxnLjNWK43Yt7vxzk6hbjf5+a5O83fEGt790k6RYEvx/32QW5Z4p3E4b99K84RdU/9VVVjn9a4NUnePfE5nDOtDW5z1TR3eta2dwJR4NGY5a4HPgBKY9Ik2/7J9tkocFZceWcBH3a//xTO2cxWt+nqsOawMLOOsPwW+8+/Hefos797RhBvJ84/SYeyFMvdDnwkjTw70v5SVf81PqGIRIHjReSomEqhLMEyOnwZ5xT+LFV9Q0TGAv+Lc0rfnZ3A8Ji8S3CaAJJJVob46ftxmqQ6fBjoaMfeDlSranUa5duBUxnFKgP+6L7/e4J8uivrySIiMUGgDFjiYXn/SKD6OjBTnIsLrgAeEZF+eNveXcrkBvYBMYvYDnxWVV9IsOzyborcsc+uSTD9OVU9P8nvfBm4VESKgVtwzioHJUobRnYGEBCquhN4CvgfETnW7VD7iIh0NKE8BPyHiAwUkeOBOSkW9zPgVhE5XRwfdf+5AVqBipi0i4CLReRf3I64vm5n30C32agBuENEjhSRCaRu8jgGOIDTOXkCcHsGq+ARYJqITBCRI4E7Sb3/xv+OZFYBV7u/bQpdm6R+CtwoIme56+koEblIRI5JsJylwMdE5GpxOpevwum3eDwmnxkiUiwilfyjiQJgF05zVnx5P4SzTYtFZDpOAFzqYXmdROQaERmgqu3AXnfyB3jb3huBvu46Ksa5oi0S8/18oLpjXxORASJyabIyxvkZ8A0RGeJui9FuwHocZ71f666LYhE5Q0SGu2WcJSL/rKoHgb+5v9G4LAAEy2eAI4F1OB2Mj+CcmoNTWT2J0276KvDbZAtR1Ydx2t9/hdOJ9xhOHwM4fQf/5Z5O36qq24FLgf+LU7Fsx+mQ69h3rsbpJN2DU6H/IkX5v4/TwfcWTmfgH1OkjS/zWuBmt8w7cX5/S4pZ7sdp+90rIo+lSPcFnEqso+mgM62qNuD0A9zr5rcJp3M2Ufk62qO/jNM09Z/ANFV9y03yNZwj2LeBO9zf0THvfpzt8YJb3o4+hpeAITjrqxq40s2np8uLNQVYKyL7gB8AM1T1PS/bW1XfAW7Cqaz/inNGELuNfoBzBvOUiLyLsw+clWh9JvBdnIOcp3Aq8vtxOvLfBS7A6QvbgdOE9C3+EXiuBZrdJscbcfo4jEu6NjEaY/KBiMzGuWprgt9lSUVE5uFcRWUVawDZGYAxxoSUBQBjjAkpawIyxpiQsjMAY4wJqby+D6B///5aXl7udzGMMSYwXnnllbdUdUD3KfM8AJSXl9PQ0OB3MYwxJjBEJO27s60JyBhjQsoCgDHGhJQFAGOMCam87gNI5ODBg7S0tPDee+/5XRTTy/r27cvAgQMpLi72uyjGFCTPAUCcUXl+gfM0wnagRlV/EJdGcJ4DciHO0xdn9/S53C0tLRxzzDGUl5fjLNYUIlVl9+7dtLS0MHjwYL+LY0xBykYT0CHgy6o6HGegjJtFZERcmqk4D7UaAlQBP+5pZu+99x79+vWzyr/AiQj9+vWzM70M1ba2Ul5fT1FdHeX19dS2tvpdpEAJ2/rzHABUdWfH0bz7ZL71HD4IxqXAL9yRfF7EGQTkRHrIKv9wCON29lIB1ba2UtXYyNa2NhTY2tZGVWNjwVdi2RLG9ZfVTmB3UIdTcR5jG+tkuo7E1IK3kZKMKTheK6C5TU3sb+86Sub+9nbmNjX1Qmnzk5cAGsb1l7UAICJH44zF+UVV/Vv81wlmSfgQIhGpEpEGEWnYtWtXtoqX144+2hn1bseOHVx55ZUp037/+99n//79GS2/rq6OadOmdZ8wDd/85je7T9RNWVasWJGVshQarxXQtra2jKYXGq8BNIzrLysBwB395zdAraomGoikha7DsA3EGbzhMKpao6qVqlo5YEBadzOnVlsL5eVQVOT8ra31vsw0fPBB5gMPnXTSSTzyyCMp0/QkAGSTBYDe47UCKotEMppeaLwG0DCuP88BwL3C535gvap+N0myJcBn3KHczgbecYc47F21tVBVBVu3gqrzt6rKUxBobm5m2LBhXHfddYwePZorr7yys0IuLy/nzjvvZMKECTz88MNs3ryZKVOmcPrpp3POOeewYcMGALZs2cK4ceM444wz+NrXvtZl2SNHjgScAHLrrbcyatQoRo8ezQ9/+EPuueceduzYweTJk5k8eTIATz31FOPGjeO0005j+vTp7Nu3D4A//vGPDBs2jAkTJvDb3yYeHOy9997j+uuvZ9SoUZx66qksW7YMgAULFnDLLbd0pps2bRp1dXXMmTOHAwcOMHbsWGbNmtXtunjrLWcwrIaGBiZNmkRzczPz58/ne9/7HmPHjuXPf/4zDz/8MCNHjmTMmDFMnDixx9ulEHitgKorKigp6vovXVJURHVFOiNjBp/XABrK9aeqnl7ABJzmnNdwxildhXO5543AjW4aAe4DNgN/ASrTWfbpp5+u8datW3fYtKSiUVWn6u/6ikbTX0acLVu2KKDLly9XVdXrr79ev/Od77jZRfVb3/pWZ9pzzz1XN27cqKqqL774ok6ePFlVVS+++GJduHChqqree++9etRRR3Uu+5RTTlFV1R/96Ed6xRVX6MGDB1VVdffu3Z157Nq1S1VVd+3apeecc47u27dPVVXvuusuveOOO/TAgQM6cOBA3bhxo7a3t+v06dP1oosuOuy33H333Tp79mxVVV2/fr0OGjRIDxw4oA888IDefPPNnekuuugiXbZsmapqZ1nTWRcd5Xz55Zf1E5/4hKqq3n777Z1pVFVHjhypLS0tqqr69ttvH1bGjLZ3wC164w0tee45ZdmyzlfJc8/pojfeyGgZ0RUrVJYt0+iKFRnNG3TRFSu6rLuOV3TFirSXUQjrD2jQNOtvz/cBqOpyErfxx6ZRnPFcc2vbtsymp2nQoEGMHz8egGuuuYZ77rmHW2+9FYCrrroKgH379rFixQqmT5/eOV+beyTywgsv8Jvf/AaAa6+9lq985SuH5fHMM89w4403csQRziY64YQTDkvz4osvsm7dus6yvP/++4wbN44NGzYwePBghgwZ0lnGmpqaw+Zfvnw5//7v/w7AsGHDiEajbNy4MWvrIh3jx49n9uzZfPrTn+aKK67IKO9CM6u0FHCaMra1tVEWiVBdUdE5Pd1lZJK+kFRXVFDV2NilGSjTI/iwrb/A3QmckbIyp9kn0XQP4i9PjP181FFHAdDe3s5xxx3HqlWr0lpGPFVNK83555/P4sWLu0xftWpVWpdQapLBgI444gjaY/6JUl2Ln2xdxC4j1fzz58/npZde4oknnmDs2LGsWrWKfv36dVv2QhW2CiibshFAvaptbfU1/0wV9rOAqquhpKTrtJISZ7oH27Zto76+HoDFixczYcLh43Yfe+yxDB48mIcffhhwKtvVq1cDzlHvr3/9awBqk/RHXHDBBcyfP59Dhw4BsGfPHgCOOeYY3n33XQDOPvtsXnjhBTZt2gTA/v372bhxI8OGDWPLli1s3ry5s4yJTJw4sTP/jRs3sm3bNoYOHUp5eTmrVq2ivb2d7du3s3Llys55iouLOXjwYLfrory8nFdeeQWg82wnvvwAmzdv5qyzzuLOO++kf//+bN8ee7WwybWg3wg1q7SU5nHjaJ80ieZx43Je+QftPoLCDgCzZkFNDUSjIOL8ralxpnswfPhwFi5cyOjRo9mzZw+f//znE6arra3l/vvvZ8yYMZxyyin87ne/A+AHP/gB9913H2eccQbvvPNOwnlvuOEGysrKGD16NGPGjOFXv/oVAFVVVUydOpXJkyczYMAAFixYwMyZMxk9ejRnn302GzZsoG/fvtTU1HDRRRcxYcIEotFowjxuuukmPvjgA0aNGsVVV13FggULiEQijB8/nsGDBzNq1ChuvfVWTjvttM55qqqqGD16NLPcdZhsXdx+++184Qtf4JxzzqFPnz6d81988cU8+uijnZ3At912G6NGjWLkyJFMnDiRMWPGZLg1TLYEsQLLJ0G8jyCvxwSurKzU+AFh1q9fz/Dhw30qkXOlzrRp01izZo1vZcgXuVgXfm/vTAWtCSBWeX09WxNcMRONRGgeN86HEgVLUV1dwpubBGifNCln5RCRV1S1Mp20hX0GYEwOBf0IOh9uhApyE1QQ7yOwAJCh8vJyO/p32broKohNALH8rsCCHkCDeB+BBQBjsiQfjqC98LsCC3oAnVVaSs3QoUQjEQSn6axm6NC8bgIs7MtAjcmhskgkYRt6PjcBxPL7MsqgB1AI3mW8FgCMyZJs3IjkNz8rsKAH0CCyJiBjsiSITQD5xO8mqDCyAJChvXv38qMf/cjvYnTKRnkee+wx1q1bl6UShZufNyIFnQXQ3Cv4AJDty8pSVbg9eQS0VxYATCGxAJpbBR0AeuOysjlz5rB582bGjh3LbbfdRl1dHZMnT+bqq69m1KhRXR7pDHD33Xczb948gKSPh461Z88eLrvsss47e1977TUA5s2bx913392ZbuTIkTQ3Nycsz8SJE7n88ssZMWIEN954Y+czeToGngF45JFHmD17NitWrGDJkiXcdtttjB07ls2bN3PPPfcwYsQIRo8ezYwZM3q8rowx+a2gO4FTXVbW0yOLu+66izVr1nQ+5K2uro6VK1eyZs0aBg8eTHNzc9J5q6qqmD9/PkOGDOGll17ipptu4k9/+lOXNLfffjunnnoqjz32GH/605/4zGc+k/SBcqnKs27dOqLRKFOmTOG3v/1t0pHGPv7xj3PJJZcwbdq0zjR33XUXW7ZsIRKJsHfv3kxWjzEmQAo6AOTqsrIzzzyTwYMHp0yT6vHQsZYvX9758LRzzz2X3bt3J31eUKryVLgdZzNnzmT58uXdDjUZq+NZP5dddhmXXXZZRnkbY4KjoANAri4r63gENCR/lHJ3j4fukOjZTCKSlUc0x05PNf8TTzzB888/z5IlS/jGN77B2rVrO8clMMYUjmyNCfxzEXlTRBI+F0BEJonIOyKyyn19PRv5dqc3LiuLf5xxvNLSUt588012795NW1sbjz/+OJD68dCxYh/RXFdXR//+/Tn22GMpLy/n1VdfBeDVV19ly5YtScuzcuVKtmzZQnt7Ow8++GDnI5pLS0tZv3497e3tPProowl/U8cjoCdPnsy3v/1t9u7d2znMpDGmsGSrE3gBMKWbNH9W1bHu684s5ZtSb1xW1q9fP8aPH8/IkSO57bbbDvu+uLiYr3/965x11llMmzaNYcOGdX6X7PHQsebNm0dDQwOjR49mzpw5LFy4EIBPfepT7Nmzh7Fjx/LjH/+Yj33sY0nLM27cOObMmcPIkSMZPHgwl19+OeC07U+bNo1zzz2XE088sTPPGTNm8J3vfIdTTz2V119/nWuuuaZznOAvfelLHHfccT1eX8aY9OX6YXhZexy0iJQDj6vqyATfTQJuVdVpmSwzHx8Hne/q6uq4++67O888gs62d7AE+XHYfuu4ajH+TvJMD1rz9XHQ40RktYj8QUROyWG+xpgcCPrTPP3mx8PwchUAXgWiqjoG+CHwWLKEIlIlIg0i0rBr164cFa9wTJo0qWCO/k2wBP1pnn7z42F4OQkAqvo3Vd3nvl8KFItI/yRpa1S1UlUrBwwYkGx5vVdYkzdsOwdLITzN009+jMeQkwAgIh8W9xpEETnTzXd3T5bVt29fdu/ebZVDgVNVdu/eTd++fXOab5BHpPKb3wPKBJ0fD8PLysXdIrIYmAT0F5EW4HagGEBV5wNXAp8XkUPAAWCG9rAGHzhwIC0tLVjzUOHr27cvAwcOzFl+8Z1wHW3YgHVkpqG6ooLPbtjA+zH/2keK2NM80+THeAxZCQCqOrOb7+8F7s1GXsXFxd3edWtMT/TGo0PCJv64zs7UM5Pr8RgK+mFwxmTC2rC9mdvUxMG4aQfd6SY/WQAwxmVt2N5YAA0eCwDGuGxEKm8sgAaPBQBjXDYilTcWQIPHHvFoTAw/B0UPOj+uYjHeWAAwxmSNBdBgsSYgY4wJKQsAxhgTUhYAjDEmpCwAGGNMSFkAMMaYkLIAYIwxIWUBwBhjQsoCgDHGhJQFAGOMCSkLAMYYE1JZCQAi8nMReVNE1iT5XkTkHhHZJCKvichp2cjXGGNMz2XrDGABMCXF91OBIe6rCvhxlvI1xhjTQ1kJAKr6PLAnRZJLgV+o40XgOBE5MRt5G2MKR21rK+X19RTV1VFeX09ta6vfRSpoueoDOBnYHvO5xZ12GBGpEpEGEWmwgd+NCY/a1laqGhvZ2taGAlvb2qhqbLQg0ItyFQAkwbSEo0Wrao2qVqpq5YABA3q5WMaYfDG3qYn97e1dpu1vb7cxhXtRrgJACzAo5vNAYEeO8jbGBICNKZx7uQoAS4DPuFcDnQ28o6o7c5S3CRFrQw4uG1M497J1GehioB4YKiItIvI5EblRRG50kywFmoBNwE+Bm7KRbyJWAYSXtSEHm40pnHtZGRJSVWd2870CN2cjr1Q6KoCOdsSOCgCwYepCIFUbsm3/9NS2tvo2pq+NKZx7BTUmsFUA4WZtyN7kwwGUjSmcWwX1KAirAMLN2pC9satwwqegAoBVAOFmbcje2AFU+BRUALAKINxmlZZSM3Qo0UgEAaKRCDVDh1qTQprsACp8CqoPwDqRjLUh91x1RUWXPgCwA6hCV1ABAKwCMKan7AAqfAouABhjes4OoMKloPoAjDHGpM8CgDHGhJQFAGOMCSkLAMYYE1IWAIwxJqQsABhjTEhZADDGmJCyAGCMMSFlAcAYY0IqWyOCTRGRRhHZJCJzEnw/W0R2icgq93VDNvI1xhjTc54fBSEifYD7gPNxBn9/WUSWqOq6uKQPquotXvMzxhiTHdk4AzgT2KSqTar6PvBr4NIsLNcYY0wvykYAOBnYHvO5xZ0W71Mi8pqIPCIig5ItTESqRKRBRBp27dqVheIZY4xJJBsBQBJM07jPvwfKVXU08AywMNnCVLVGVStVtXLAgAFZKJ4xxphEshEAWoDYI/qBwI7YBKq6W1U7xpX7KXB6FvI1xpgualtbKa+vp6iujvL6empbW/0uUl7LRgB4GRgiIoNF5EhgBrAkNoGInBjz8RJgfRbyNcaYTrWtrVQ1NrK1rQ0Ftra1UdXYaEEgBc8BQFUPAbcAT+JU7A+p6loRuVNELnGT/YeIrBWR1cB/ALO95muMMbHmNjV1Gc4SYH97O3ObmnwqUf7LyohgqroUWBo37esx778KfDUbeRljTCLb2toymm7sTmBjTIEoi0Qymm4sAJg8Y514pqeqKyooKepapZUUFVFdUeFTifKfBYA4VgH5xzrxjBezSkupGTqUaCSCANFIhJqhQ22Q+xRENf6S/fxRWVmpDQ0NOcuvowKK7UgqKSqynShHyuvr2ZqgvTYaidA8bpwPJQqe2tZW5jY1sa2tjbJIhOqKCtt3Q0ZEXlHVynTS2hlADLuKwF/WieeNnUGZTFkAiJGVCqi2FsrLoajI+Vtbm5WyhYF14nljBzAmUxYAYniugGproaoKtm4FVedvVZUFgTRZJ543dgZlMmUBIEb1zp2UvPdel2kl771H9c6d6S1g7lzYv7/rtP37nenpCvEZhHXieZOVMyiv+1+I999AUtW8fZ1++umasUWLVKNRVRHn76JF6c8bjeqi887T6OLFKs8+q9HFi3XReec5y0mHiKpz7N/1JZJ+2UtKus5bUpLZbzChteiNN7Tk2WeVZcs6XyXPPquL3ngjzQV43P+ysf96+f81qqoKNGiadazvlXyqV8YBwOsO6LUCj0YTz59uAPE6vwk+LxXgokW6aOrUrgcwU6emvwy/918LIFkR3gAQ9B3YawAyweZ1//G6/3rd//w+gLIzaFUNcwDIhyYYj01QdgYQcH5uf78r4KCXX7UgziDCGwCCvgP4HYCMN36fAfp9BB30M5h8+P/Lwv9veANAIZwCemwDDvzv95ufR/B+V+Ady/CrAvM7gPi9/rP0/xveAKAa7iNga0Lyxu8j+HyowP3mNYAceWTX9XfkkekvI9G263ilw+8A1PkzwhwAwsw6kYN9BO+1/GG3aJFqcXHXdV9cnP467NMn8fbr0ye9+b0GkCz9/+Y8AABTgEZgEzAnwfcR4EH3+5dwBoi3AJBtYT8DKIQjeNNzXvd/rxW41wDiwxmA5zuBRaQPcB8wFRgBzBSREXHJPge8raofBb4HfMtrviaB6mooKek6raTEmZ4uv+/k9JK/1zuxy8oymx5v1iyoqYFoFEScvzU1znSTHi/bf9u2zKbHi0Yzmx7vgw8ymx7vwgszm54N6UaKZC9gHPBkzOevAl+NS/MkMM59fwTwFu6jqFO97AygB4LciWxH8OHmdyew3/kHsQ8AuBL4Wczna4F749KsAQbGfN4M9E+yvCqgAWgoKyvL6IerqrWheuF3G3Y+/APZ/uONn9vf7050vw9gXLkOANMTBIAfxqVZmyAA9Otu2aG8DNRPfh9B+52/8cbv7d9RBj8DeB7cCJrrAJA/TUBh7wT1yu8jcDuCD7Z82P5BFsT7ANwKvQkYDBwJrAZOiUtzMzDffT8DeCidZef8URBh5/UyOjuCDz4vAdS2v3dBvBMYuBDY6DbtzHWn3Qlc4r7vCzyMcxnoSqAineWG7lEQfvN6I02/fonXf79+mZUhrOvfb353gnaUwba/JzkPAL31ynkfQCEcgfjZBpmNAGC8CXonrPEsvAFA1f9OGD+PYPzuhLMmOH/5vf07ymBH8L4KdwDwIh/aMIN8GWXQA2jQ5cP2M76zANBTfv8D+X0E53cTmjUhWCeszxa98YZGV6xQWbZMoytWpD+cZh6xANBTflfAfgcgVe9H4GFugvOav3XC+mrRG29oyXPPdR1T+bnnAhcELAB44WcFFvYjuHz4/UGuwIO+/X0WXbGiS+Xf8YquWOF30TJiAcAvflcAHWUI6hGc32dAfm8/64T1lSSo/Fm2TGXZMr+LlpFMAoDnp4GaGF6fBpmNp3nOmgXNzdDe7vwN0pMovf5+r0+D9Po0Ua/5e30aKQR7+/usLBLJaHohsACQbV7+AQvgccK1ra2U19dTVFdHeX09ta2t6c/s9fd7rUD9rsCzcQBgeqy6ooKSoq5VYklREdUVFT6VqPdZAMgyTxUgBPoIrra1larGRra2taHA1rY2qhobMw8CPf39XitQvyvwAjgACLJZpaXUDB1KNBJBgGgkQs3QocygcsZPAAAOhklEQVQqLfW7aL1GnCaj/FRZWakNDQ1+FyNtHRXg/vb2zmklRUUFvxN1KK+vZ2tb22HTo5EIzePG5aYQtbVOk822bU7FXV2dfgVaWwtVVV2bgUpKMquEveSPsw/NbWpiW1sbZZEI1RUVodh3TPaIyCuqWplWWgsA2ZMXFaCPiurqSLQ3CdA+aVKOS9NDHitwT1mH/ADCZEcmAcCagLJoW4LKP9X0QlMQnWg+NsHNbWrqUvkD7G9vZ25TU87KYMLFAkAWFUQF6EEYO9GyKewHECb3LABkUdgrwDB2omVT2A8gTO4d4XcBCklHRRfmTrxZpaWh+r3ZVF1RkbAPICwHECb3LABkmVWApqfsAMLkmqcAICInAA8C5UAz8GlVfTtBug+Av7gft6nqJV7yNaZQ2QGEySWvfQBzgGdVdQjwrPs5kQOqOtZ9WeVvjDF5wGsAuBRY6L5fCFzmcXnGGGNyxGsAKFXVnQDu3w8lSddXRBpE5EURSRkkRKTKTduwa9cuj8UzxhiTTLd9ACLyDPDhBF+l+YhEAMpUdYeIVAB/EpG/qOrmRAlVtQaoAedO4AzyMMYYk4FuA4CqfjLZdyLSKiInqupOETkReDPJMna4f5tEpA44FUgYAIwxxuSG1yagJcB17vvrgN/FJxCR40Uk4r7vD4wH1nnM1xhjjEdeA8BdwPki8jpwvvsZEakUkZ+5aYYDDSKyGlgG3KWqFgCMMcZnnu4DUNXdwHkJpjcAN7jvVwCjvORjjDEm++xZQMYYE1IWAIwxJqQsABhjTEhZADDGmJCyAGCMMSFlAcAYY0LKAoAxxoSUBQBjjMmS2tZWyuvrKaqro7y+ntrWVr+LlJKNCGaMMVlQ29raZUjPrW1tVDU2AuTtID92BmCMMVkwt6mpy3jOAPvb25nb1ORTibpnAcAYY7JgW1tbRtPzgQUA00XQ2jCNyRdlkUhG0/OBBYA842cF3NGGubWtDeUfbZgWBIzpXnVFBSVFXavUkqIiqisqfCpR9ywA5BG/K+AgtmHmGzuDCq9ZpaXUDB1KNBJBgGgkQs3QoXnbAQx2FVBeSVUB52InCmIbZj4J4lUgJrtmlZYGalvbGUAe8bsCDmIbZj6xMygTNJ4CgIhMF5G1ItIuIpUp0k0RkUYR2SQic7zkWcj8roCD2IaZT/wO4MZkyusZwBrgCuD5ZAlEpA9wHzAVGAHMFJERHvMtSH5XwEFsw8wnfgdwYzLldUjI9QAikirZmcAmVW1y0/4auBQbGP4wHRXt3KYmtrW1URaJUF1RkdMKOGhtmPmkuqKiSx8A2BmUyW+56AQ+Gdge87kFOCtZYhGpAqoAysrKerdkecgq4ODKhwBuTCa6DQAi8gzw4QRfzVXV36WRR6LTA02WWFVrgBqAysrKpOmMyUcWwE2QdBsAVPWTHvNoAQbFfB4I7PC4TGOMMR7l4jLQl4EhIjJYRI4EZgBLcpCvMcaYFLxeBnq5iLQA44AnRORJd/pJIrIUQFUPAbcATwLrgYdUda23YhtjjPHK61VAjwKPJpi+A7gw5vNSYKmXvIwxxmSX3QlsjDEhZQHAGGNCygKAMcaElAUAY4wJKQsAxhgTUhYAjDEmpCwAGGNMSFkAMMaYkLIAYIwxeSLXY0rbmMDGGJMH/BhT2s4AjDEmD/gxprQFgAKT61NIY0x2+DGmtAWAAlLb2spnN2xga1sbinMK+dkNGywIGBMAfowpbQGggHzh9dd5X7sOova+Kl94/XWfShQ8dgZl/FJdUUFJUdcqubfHlLZO4AKy+9ChjKabrvzohDOmgx9jSlsAMMaVqhPOAoDJhVyPKe11RLDpIrJWRNpFpDJFumYR+YuIrBKRBi95muT69emT0XTTlR+dcMb4yWsfwBrgCuD5NNJOVtWxqpo0UBhvfvCxj1EcN63YnW6650cnnDF+8hQAVHW9qjZmqzDGm1mlpTwwfDjRSAQBopEIDwwfbs0XafKjE84YP+WqD0CBp0REgZ+oak2yhCJSBVQBlJWV5ah4hSPXbYiFxI9OOGP81G0AEJFngA8n+Gquqv4uzXzGq+oOEfkQ8LSIbFDVhM1GbnCoAaisrNREaYzpLRZATZh0GwBU9ZNeM1HVHe7fN0XkUeBM0us3MMYY00t6/UYwETlKRI7peA9cgNN5bIwxxkdeLwO9XERagHHAEyLypDv9JBFZ6iYrBZaLyGpgJfCEqv7RS77GGGO889QJrKqPAo8mmL4DuNB93wSM8ZKPMcaY7LNnARljTEhZADDGmJCyAGCMMSFlAcAYY0LKAoAxxoSUBYA84/eAJH7n71XQy29MLhXceAC1ra2BfZaL3wOS+J2/V0EvvzG5VlBnAB0VQOyYuFWNjYE5Ckw1IEkY8vcq6OU3JtcKKgAEvQLwe0ASv/P3KujlNybXCioABL0C8HtAEr/z9yro5Tcm1woqAAS9AvB7QBK/8/cq6OU3JtcKKgAEvQKYVVpKzdChXUb0qhk6NGcdmH7n71XQy29Mrolq/o65UllZqQ0NmY0hH+SrgIwxxisReSXdsdcL7jJQG9HJGGPSU1BNQPnAbkQyxgSFBYA4XirwoN+HkA1eA6AFUGNyx+uIYN8RkQ0i8pqIPCoixyVJN0VEGkVkk4jM8ZJnb/JagQf9PgSvvK4/C6DG5JbXM4CngZGqOhrYCHw1PoGI9AHuA6YCI4CZIjLCY769wmsFHvT7ELzyuv7CHkCNyTVPAUBVn1LVQ+7HF4GBCZKdCWxS1SZVfR/4NXCpl3x7i9cKPOj3IXjldf2FPYAak2vZ7AP4LPCHBNNPBrbHfG5xpyUkIlUi0iAiDbt27cpi8brntQIP+n0IXnldf2EPoMbkWrcBQESeEZE1CV6XxqSZCxwCahMtIsG0pDcfqGqNqlaqauWAAQPS+Q1Z47UCD/uNSF7XX9gDqDG51u19AKr6yVTfi8h1wDTgPE18V1kLMCjm80BgRyaFzJWOitrLjWRhvg/B6/rLxvo3xqTP053AIjIF+C7wCVVN2F4jIkfgdBCfB/wVeBm4WlXXdrf8ntwJbIwxYZbJncBe+wDuBY4BnhaRVSIy3y3ASSKyFMDtJL4FeBJYDzyUTuVvjDGmd3l6FISqfjTJ9B3AhTGflwJLveRljDEmu+xOYGOMCSkLAMYYE1IWAIwxJqTyejwAEdkFbO3h7P2Bt7JYnGyz8nlj5fPGyudNPpcvqqpp3USV1wHACxFpSPdSKD9Y+byx8nlj5fMm38uXLmsCMsaYkLIAYIwxIVXIAaDG7wJ0w8rnjZXPGyufN/levrQUbB+AMcaY1Ar5DMAYY0wKFgCMMSakAhcAROTnIvKmiKyJmTZPRP7qPpBulYhcmGTeXh+bOEn5HowpW7OIrEoyb7OI/MVN1yuPQRWRQSKyTETWi8haEfmCO/0EEXlaRF53/x6fZP7r3DSvu48Cz0XZ0h172s/1lxf7YIry5cU+KCJ9RWSliKx2y3eHO32wiLzk7lcPisiRSeb/qrvuGkXkX3JYvlo3zzXu/3hxkvk/iFnPS7JdvqxT1UC9gInAacCamGnzgFu7ma8PsBmoAI4EVgMjclG+uO//B/h6ku+agf69vP5OBE5z3x+D86juEcC3gTnu9DnAtxLMewLQ5P493n1/fA7KdgFwhDv9W4nKlgfrLy/2wWTly5d9EGeAqKPd98XAS8DZwEPADHf6fODzCeYd4a6zCDDYXZd9clS+C93vBFicqHzuPPt6c//L9itwZwCq+jywpwez5mRs4lTlExEBPo2zA/lCVXeq6qvu+3dxHtF9Ms66WOgmWwhclmD2fwGeVtU9qvo28DQwpbfLpumNPZ0TKdZfOnp9H+yufH7vg+rY534sdl8KnAs84k5Ptv9dCvxaVdtUdQuwCWed9nr5VHWp+50CK/FxH8ymwAWAFG5xmwh+nqT5IqOxiXvJOUCrqr6e5HsFnhKRV0SkqrcLIyLlwKk4RzmlqroTnEoE+FCCWXK2DuPKFivZ2NPg7/qDPNsHk6xD3/dBEenjNkG9iXMQsRnYGxPkk62XnKy/+PKp6ksx3xUD1wJ/TDJ7X3HGNH9RRBIFsbxSKAHgx8BHgLHATpxT3HgZjU3cS2aS+shrvKqeBkwFbhaRib1VEBE5GvgN8EVV/Vu6syWYlvV1mKxsknrsafB3/eXVPphi+/q+D6rqB6o6Fuco+kxgeKJkCablZP3Fl09ERsZ8/SPgeVX9c5LZy9R5RMTVwPdF5CPZLl82FUQAUNVWd6O1Az8l8Wmhr2MTizM05hXAg8nSqDOQDqr6JvAoWT69jSlLMU7lUKuqv3Unt4rIie73J+Ic/cTr9XWYpGyxY0/Pck/DD+Pn+sunfTDFOsybfdDNYy9Qh9PGfpxbPki+XnL6PxxTvikAInI7MAD4Pynm6Vh/Te68p/ZW+bKhIAJAR8XluhxYkyDZy8AQ92qDI4EZQC576T8JbFDVlkRfishRInJMx3ucjs9Ev8MTtw34fmC9qn435qslQMdVPdcBv0sw+5PABSJyvNvEcYE7rVfLJs7Y018BLlHV/Unm9XX95cs+mGL7Qh7sgyIyQNyruETkn9wyrQeWAVe6yZLtf0uAGSISEZHBwBCc9vjeLt8GEbkBpw9sphvkE817vIhE3Pf9gfHAumyWL+v87oXO9IVz+roTOIhzRPA54JfAX4DXcHaSE920JwFLY+a9EOeqiM3A3FyVz52+ALgxLm1n+XCuDFntvtb2Yvkm4Jw2vwascl8XAv2AZ4HX3b8nuOkrgZ/FzP9ZnM63TcD1OSrbJpy2345p8/Nw/eXFPpisfPmyDwKjgf91y7cG92okN++V7rZ+GIi40y8B7oyZf6677hqBqTks3yE334512jG98/8D+Li7D6x2/36uN/bBbL7sURDGGBNSBdEEZIwxJnMWAIwxJqQsABhjTEhZADDGmJCyAGCMMSFlAcAYY0LKAoAxxoTU/we9sMH9W9buuAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_seq_input = np.array(generate_samples_for_input(train_data_x[-15:])).transpose(1,0)\n",
    "test_seq_output = np.array(generate_samples_for_output(train_data_x[-20:])).transpose(1,0)\n",
    "plt.title(\"Predicted and true output sequences\")\n",
    "#i1, i2, i3, = plt.plot(range(15), np.array(true_input_signals(x[95:110])).transpose(1, 0), 'c:', label = 'true input sequence')\n",
    "p1, p2 = plt.plot(range(15, 35), final_preds, 'ro', label = 'predicted outputs')\n",
    "t1, t2 = plt.plot(range(15, 35), np.array(true_output_signals(x[110:])).transpose(1, 0), 'co', label = 'true outputs')\n",
    "plt.legend(handles = [p1, t1], loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}