nmt_python_scala_transpiler.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python to Scala Transpiler using Neural Machine Translation\n",
    "This project demonstrates how to convert simple Python expressions into Scala expressions. For example, consider the following Python expression.\n",
    "```python\n",
    "[a(x, z) for x in l if b(x, y)]\n",
    "```\n",
    "Within the context of this work, the equivalent Scala expressions is\n",
    "```scala\n",
    "l.filter(x => b(x, y)).map(x => a(x, z))\n",
    "```\n",
    "\n",
    "[Neural machine translation (NMT)](https://en.wikipedia.org/wiki/Neural_machine_translation)\n",
    "is applied using a\n",
    "[recurrent neural network](https://en.wikipedia.org/wiki/Recurrent_neural_network)\n",
    "(RNN)\n",
    "to implement this transpiler. The RNN is created by adapting the excellent work of Zafarali Ahmed in\n",
    "[keras-attention](https://github.com/datalogue/keras-attention),\n",
    "which was originally developed to convert dates from varied human readable format to machine format.\n",
    "\n",
    "For a discussion of results, see the corresponding blog post, [A Python to Scala transpiler using neural machine translation (NMT)](https://medium.com/@matthagy/a-python-to-scala-transpiler-using-neural-machine-translation-nmt-90d4d02afa70).\n",
    "\n",
    "See README.md for more details on this work and reproducing and extending it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = '0'\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from tqdm import tqdm\n",
    "\n",
    "import tensorflow as tf\n",
    "from keras.callbacks import ModelCheckpoint\n",
    "from keras.utils.training_utils import multi_gpu_model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# include companion project keras-attention libraries\n",
    "import sys\n",
    "sys.path.append('../keras-attention')\n",
    "\n",
    "from data.reader import Vocabulary, Data\n",
    "from models.NMT import simpleNMT\n",
    "from utils.metrics import all_acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load Data\n",
    "See details in README.md for generating data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "400000\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input</th>\n",
       "      <th>output</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>163704</th>\n",
       "      <td>[z(j) for j in m(g(t),[d for d in t if j(d)])]</td>\n",
       "      <td>m(g(t),t.filter(j)).map(z)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>216160</th>\n",
       "      <td>[m(f,b) for f in c]</td>\n",
       "      <td>c.map(f =&gt; m(f,b))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>317981</th>\n",
       "      <td>[j(k) for k in q(y(q,v),[q(v,f(z(g(k),v(q)),j(...</td>\n",
       "      <td>q(y(q,v),q.filter(v =&gt; v).map(v =&gt; q(v,f(z(g(k...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>194187</th>\n",
       "      <td>[b for b in c if b(b)]</td>\n",
       "      <td>c.filter(b)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>266994</th>\n",
       "      <td>[p(b) for b in o(i,x(b)) if b]</td>\n",
       "      <td>o(i,x(b)).filter(b =&gt; b).map(p)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55682</th>\n",
       "      <td>e(u,h(u))</td>\n",
       "      <td>e(u,h(u))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119073</th>\n",
       "      <td>[f for f in k([u(r) for r in l],u(l)) if f]</td>\n",
       "      <td>k(l.map(u),u(l)).filter(f =&gt; f)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298186</th>\n",
       "      <td>[c(r(j),u) for u in z]</td>\n",
       "      <td>z.map(u =&gt; c(r(j),u))</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    input  \\\n",
       "163704     [z(j) for j in m(g(t),[d for d in t if j(d)])]   \n",
       "216160                                [m(f,b) for f in c]   \n",
       "317981  [j(k) for k in q(y(q,v),[q(v,f(z(g(k),v(q)),j(...   \n",
       "194187                             [b for b in c if b(b)]   \n",
       "266994                     [p(b) for b in o(i,x(b)) if b]   \n",
       "55682                                           e(u,h(u))   \n",
       "119073        [f for f in k([u(r) for r in l],u(l)) if f]   \n",
       "298186                             [c(r(j),u) for u in z]   \n",
       "\n",
       "                                                   output  \n",
       "163704                         m(g(t),t.filter(j)).map(z)  \n",
       "216160                                 c.map(f => m(f,b))  \n",
       "317981  q(y(q,v),q.filter(v => v).map(v => q(v,f(z(g(k...  \n",
       "194187                                        c.filter(b)  \n",
       "266994                    o(i,x(b)).filter(b => b).map(p)  \n",
       "55682                                           e(u,h(u))  \n",
       "119073                    k(l.map(u),u(l)).filter(f => f)  \n",
       "298186                              z.map(u => c(r(j),u))  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "expressions = pd.read_csv('data/expressions', sep='|', names=['python', 'scala'])\n",
    "expressions = expressions.rename(columns={'python':'input', 'scala':'output'})\n",
    "print(len(expressions))\n",
    "\n",
    "expressions.sample(8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Determine Padding Size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "278"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_input_length = expressions['input'].str.len().max()\n",
    "max_input_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "282"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_output_length = expressions['output'].str.len().max()\n",
    "max_output_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "298"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "padding = max(max_input_length, max_output_length) + 16\n",
    "padding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom classes to present data to keras-attention code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ExpressionVocabulary(Vocabulary):\n",
    "    \"\"\"Hacked class to expose the vecabularly using an API required by keras-attention\n",
    "       for data in this project.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, vocabulary, padding):\n",
    "        self.vocabulary = vocabulary\n",
    "        self.padding = padding\n",
    "        self.reverse_vocabulary = {v: k for k, v in self.vocabulary.items()}\n",
    "        \n",
    "    @classmethod\n",
    "    def from_strings(cls, strs, padding):\n",
    "        strs = list(strs)\n",
    "        chars = sorted({c for s in strs for c in s})\n",
    "        chars = chars + ['<unk>', '<eof>']\n",
    "        return cls({c:i for i,c in enumerate(chars)}, padding)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_vocab = ExpressionVocabulary.from_strings(expressions['input'], padding)\n",
    "input_vocab.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "35"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_vocab = ExpressionVocabulary.from_strings(expressions['output'], padding)\n",
    "output_vocab.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ExpressionData(Data):\n",
    "    \"\"\"Hacked class to expose the expressions using an API required by keras-attention\n",
    "       for data in this project.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, inputs, targets, input_vocabulary, output_vocabulary):\n",
    "        self.inputs = list(inputs)\n",
    "        self.targets = list(targets)\n",
    "        self.input_vocabulary = input_vocabulary\n",
    "        self.output_vocabulary = output_vocabulary\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "320000"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_size = int(0.8 * len(expressions))\n",
    "training_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "training = ExpressionData(\n",
    "    expressions['input'].iloc[:training_size],\n",
    "    expressions['output'].iloc[:training_size],\n",
    "    input_vocab,\n",
    "    output_vocab\n",
    ")\n",
    "training.transform()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "validation = ExpressionData(\n",
    "    expressions['input'].iloc[training_size:],\n",
    "    expressions['output'].iloc[training_size:],\n",
    "    input_vocab,\n",
    "    output_vocab\n",
    ")\n",
    "validation.transform()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the NMT model using keras-attention"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "inputs shape: (?, ?, 1024)\n"
     ]
    }
   ],
   "source": [
    "model = simpleNMT(pad_length=padding,\n",
    "              n_chars=input_vocab.size(),\n",
    "              n_labels=output_vocab.size(),\n",
    "              embedding_learnable=False,\n",
    "              encoder_units=512,\n",
    "              decoder_units=512,\n",
    "              trainable=True,\n",
    "              return_probabilities=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 298)               0         \n",
      "_________________________________________________________________\n",
      "OneHot (Embedding)           (None, 298, 34)           1156      \n",
      "_________________________________________________________________\n",
      "bidirectional_1 (Bidirection (None, 298, 1024)         2240512   \n",
      "_________________________________________________________________\n",
      "attention_decoder_1 (Attenti (None, 298, 35)           3781356   \n",
      "=================================================================\n",
      "Total params: 6,023,024\n",
      "Trainable params: 6,021,868\n",
      "Non-trainable params: 1,156\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy', all_acc])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_cycle():\n",
    "    cp = ModelCheckpoint(\"./data/weights/NMT.{epoch:02d}-{val_loss:.2f}.hdf5\",\n",
    "                         monitor='val_loss',\n",
    "                         verbose=0,\n",
    "                         save_best_only=True,\n",
    "                         save_weights_only=True,\n",
    "                         mode='auto')\n",
    "\n",
    "    batch_size = 10\n",
    "    model.fit_generator(generator=training.generator(batch_size),\n",
    "                        steps_per_epoch=100,\n",
    "                        validation_data=validation.generator(batch_size),\n",
    "                        validation_steps=100,\n",
    "                        callbacks=[cp],\n",
    "                        workers=1,\n",
    "                        verbose=1,\n",
    "                        epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show Examples of Model on Validation Input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def encode(input_string):\n",
    "    \"Convert an input string in an array of numbers, on which modeling can be applied\"\n",
    "    return np.array([input_vocab.string_to_int(input_string)])\n",
    "\n",
    "def apply_model(input_string):\n",
    "    \"Run the model on a single input string\"\n",
    "    full_prediction = model.predict(encode(input_string))\n",
    "    prediction = np.argmax(full_prediction[0], axis=-1)\n",
    "    return output_vocab.int_to_string(prediction)\n",
    "    \n",
    "def show_example_ml_application(input_string):\n",
    "    \"Interpret the terminal and padding characters in raw results\"\n",
    "    results = apply_model(input_string)\n",
    "    pretty = ''.join('|' if x=='<eot>' else ('' if x == '<unk>' else x)\n",
    "                     for x in results)\n",
    "    print(repr(input_string), '->', repr(pretty))\n",
    "    \n",
    "\n",
    "examples = list(expressions['input'].iloc[training_size:].sample(10))\n",
    "\n",
    "def show_examples():\n",
    "    for example in examples:\n",
    "        show_example_ml_application(example)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train and Show Examples Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cycle 0\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 305s 3s/step - loss: 0.3820 - acc: 0.9250 - all_acc: 0.0000e+00 - val_loss: 0.2383 - val_acc: 0.9398 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2472 - acc: 0.9366 - all_acc: 0.0000e+00 - val_loss: 0.2193 - val_acc: 0.9428 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2250 - acc: 0.9417 - all_acc: 0.0000e+00 - val_loss: 0.2067 - val_acc: 0.9451 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2053 - acc: 0.9453 - all_acc: 0.0000e+00 - val_loss: 0.2003 - val_acc: 0.9449 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2039 - acc: 0.9451 - all_acc: 0.0000e+00 - val_loss: 0.2219 - val_acc: 0.9443 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'c(f(.)er(. (>(. (p(.((p))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'c(f).tel)er(>(. ()'\n",
      "'[l(n) for n in o]' -> 'a.map('\n",
      "'[p for p in n if b(p)]' -> 'f.filter(e ('\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'c(f(.(,)(.)())))()(((((((((((()))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'c(f).ter(e (>(.))))'\n",
      "'[a(e,c) for e in b]' -> 'a.ma.((p(p))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'c(f(.(,)(.)())))()(((((((((((((())))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'c(m).ter(. (>(p)))))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'c(f(.)il).)(>(. (p(.)()))))'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 1\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1856 - acc: 0.9486 - all_acc: 0.0000e+00 - val_loss: 0.1965 - val_acc: 0.9449 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1748 - acc: 0.9514 - all_acc: 0.0000e+00 - val_loss: 0.1938 - val_acc: 0.9489 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1915 - acc: 0.9493 - all_acc: 0.0000e+00 - val_loss: 0.1772 - val_acc: 0.9512 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 302s 3s/step - loss: 0.1887 - acc: 0.9497 - all_acc: 0.0000e+00 - val_loss: 0.1806 - val_acc: 0.9510 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1832 - acc: 0.9496 - all_acc: 0.0000e+00 - val_loss: 0.1635 - val_acc: 0.9550 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'd(f(l)er(e => p(=(((((()))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> '(f(l)ifilter(e '\n",
      "'[l(n) for n in o]' -> 'm.map(a)'\n",
      "'[p for p in n if b(p)]' -> 't.filter(e => '\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 't(f(q(,((((((((((((((((((((((((((((((((())))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 't(f(l),)i.)er)e)=> )'\n",
      "'[a(e,c) for e in b]' -> 'm.map(a => =( (,)))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 't(f(q(,(((((((((((((((((((((((((((((((((((((())))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'm(filter(. => p(=( (()))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 't(f(l),)).)ir).)ap .)'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 2\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1680 - acc: 0.9546 - all_acc: 0.0000e+00 - val_loss: 0.1565 - val_acc: 0.9574 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1539 - acc: 0.9586 - all_acc: 0.0000e+00 - val_loss: 0.1868 - val_acc: 0.9495 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1605 - acc: 0.9558 - all_acc: 0.0000e+00 - val_loss: 0.1522 - val_acc: 0.9587 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1470 - acc: 0.9607 - all_acc: 0.0000e+00 - val_loss: 0.1587 - val_acc: 0.9605 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1571 - acc: 0.9584 - all_acc: 0.0000e+00 - val_loss: 0.1629 - val_acc: 0.9558 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'q.map(e => =( (((((((((())))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'a(d(.).filter(e => '\n",
      "'[l(n) for n in o]' -> 'o.map(z)'\n",
      "'[p for p in n if b(p)]' -> 'e(f).ter(e)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'k(g(q(,))))))))))))))))))(((((((()))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'g(g(q(,(,))))).)).).)(.'\n",
      "'[a(e,c) for e in b]' -> 'o.map(s => p(g,,))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'k(g(l),))))))))))) ((((((((((((((((((((())))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'e.filter(e => p).map(t => p(.,a)'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'g(g(q(,(,)))))))))))))p)a))'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 3\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1295 - acc: 0.9657 - all_acc: 0.0000e+00 - val_loss: 0.1328 - val_acc: 0.9639 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1610 - acc: 0.9569 - all_acc: 0.0000e+00 - val_loss: 0.1486 - val_acc: 0.9598 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1403 - acc: 0.9627 - all_acc: 0.0000e+00 - val_loss: 0.1436 - val_acc: 0.9613 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1430 - acc: 0.9621 - all_acc: 0.0000e+00 - val_loss: 0.1274 - val_acc: 0.9663 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 303s 3s/step - loss: 0.1391 - acc: 0.9627 - all_acc: 0.0000e+00 - val_loss: 0.1313 - val_acc: 0.9643 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'q.ma.tar(> => ((((((((((())))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'm(e).f.filter(= => d(d,))'\n",
      "'[l(n) for n in o]' -> 'o.map(h)'\n",
      "'[p for p in n if b(p)]' -> 'j.filter(h)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'i(f(m(,)(.)=((. (( .((((((((((((((((((())))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(e(m).filter(a).map()'\n",
      "'[a(e,c) for e in b]' -> 'i.map(l => a(h,g))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'i(f(m(,)(. =( . (( ((((((((((((((((((((())))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'i.filter(m => m).map(v => h(z,v))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'm(e(m(,)())))).)).).pap('\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 4\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1397 - acc: 0.9632 - all_acc: 0.0000e+00 - val_loss: 0.1173 - val_acc: 0.9685 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1529 - acc: 0.9592 - all_acc: 0.0000e+00 - val_loss: 0.1337 - val_acc: 0.9648 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1194 - acc: 0.9683 - all_acc: 0.0000e+00 - val_loss: 0.1394 - val_acc: 0.9623 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1200 - acc: 0.9677 - all_acc: 0.0000e+00 - val_loss: 0.1315 - val_acc: 0.9650 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1272 - acc: 0.9657 - all_acc: 0.0000e+00 - val_loss: 0.1494 - val_acc: 0.9596 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(m(p(a(((((((((((((,,)))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 's(p(.),)).filte => =(,'\n",
      "'[l(n) for n in o]' -> 'n.map(i)'\n",
      "'[p for p in n if b(p)]' -> 'g.filter(p)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'p(s(p(,((((((()(((((((((((((((((((((((()))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 's(s(u),f).filter(p).m'\n",
      "'[a(e,c) for e in b]' -> 'b.map(b => (b,b))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'p(s(p(,((((((()()))))()()(((((((((((((((((()))))'\n",
      "'[i(c,z) for z in t if z]' -> 'z.map(z => =(z,z(,))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 's(s(q),(())))).)(p(.m'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 5\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100/100 [==============================] - 302s 3s/step - loss: 0.1236 - acc: 0.9663 - all_acc: 0.0000e+00 - val_loss: 0.1254 - val_acc: 0.9665 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1240 - acc: 0.9660 - all_acc: 0.0000e+00 - val_loss: 0.1164 - val_acc: 0.9693 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1165 - acc: 0.9684 - all_acc: 0.0000e+00 - val_loss: 0.1348 - val_acc: 0.9658 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1143 - acc: 0.9687 - all_acc: 0.0000e+00 - val_loss: 0.1076 - val_acc: 0.9701 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1139 - acc: 0.9689 - all_acc: 0.0000e+00 - val_loss: 0.1146 - val_acc: 0.9675 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k)l(k (> => (((((((((((((((,)))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(p,p),)).ter(e => t())'\n",
      "'[l(n) for n in o]' -> 'o.map(n)'\n",
      "'[p for p in n if b(p)]' -> 'b.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'k(k(q(q)((),))))))              ((((((((()))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'd(d(d),)).filter(l).map('\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => e(e,c),'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(n(l(e)(e)=)((((((((((((((((((((((((((((((((((((((((()))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'z.filter(z => z).map(z => c(c,))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'a(a(a(,)(y),y).mapmppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 6\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0996 - acc: 0.9718 - all_acc: 0.0000e+00 - val_loss: 0.0912 - val_acc: 0.9735 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0814 - acc: 0.9765 - all_acc: 0.0000e+00 - val_loss: 0.0934 - val_acc: 0.9734 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0977 - acc: 0.9717 - all_acc: 0.0000e+00 - val_loss: 0.0905 - val_acc: 0.9743 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1217 - acc: 0.9689 - all_acc: 0.0000e+00 - val_loss: 0.2086 - val_acc: 0.9473 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.3951 - acc: 0.9312 - all_acc: 0.0000e+00 - val_loss: 0.2536 - val_acc: 0.9406 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'p.f,lfirte)=> '\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'o.fi.fir(e)'\n",
      "'[l(n) for n in o]' -> 'o(n).ma'\n",
      "'[p for p in n if b(p)]' -> 'o.filter(e)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(f,.)ir(e)=> )'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'o.f,.fir(e)'\n",
      "'[a(e,c) for e in b]' -> 'o.filter(e)'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'p(f,l)irte)= '\n",
      "'[i(c,z) for z in t if z]' -> 'o.fi.fi)(e)'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'p(f,.)i)(e) '\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 7\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2070 - acc: 0.9473 - all_acc: 0.0000e+00 - val_loss: 0.2089 - val_acc: 0.9435 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1964 - acc: 0.9459 - all_acc: 0.0000e+00 - val_loss: 0.1894 - val_acc: 0.9477 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1842 - acc: 0.9491 - all_acc: 0.0000e+00 - val_loss: 0.1812 - val_acc: 0.9496 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1730 - acc: 0.9521 - all_acc: 0.0000e+00 - val_loss: 0.1710 - val_acc: 0.9530 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1529 - acc: 0.9572 - all_acc: 0.0000e+00 - val_loss: 0.1588 - val_acc: 0.9552 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k,k(i)(. (k(. => ,(=( ,(((),))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'e(f,.(i)te => =( ,))'\n",
      "'[l(n) for n in o]' -> 'o.m).(i'\n",
      "'[p for p in n if b(p)]' -> 'h.filter(p)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(.(,(((),((,((((,((((((((((()()))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'y(m,.)imt.mal(.m=>()'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e (> '\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(f,l(ilte (((( i((((i()())))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'f.filter(z => z).map()'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'y(y,y(ypyp)yp(p(yp(p))p)p)'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 8\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1503 - acc: 0.9585 - all_acc: 0.0000e+00 - val_loss: 0.1425 - val_acc: 0.9597 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1474 - acc: 0.9596 - all_acc: 0.0000e+00 - val_loss: 0.1331 - val_acc: 0.9628 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1337 - acc: 0.9633 - all_acc: 0.0000e+00 - val_loss: 0.1274 - val_acc: 0.9644 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1329 - acc: 0.9636 - all_acc: 0.0000e+00 - val_loss: 0.1196 - val_acc: 0.9669 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1305 - acc: 0.9636 - all_acc: 0.0000e+00 - val_loss: 0.1089 - val_acc: 0.9696 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k,.).)). er(> => s(s,s(s))))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'e(e,e)ef)eter(e => d(d,d))'\n",
      "'[l(n) for n in o]' -> 'o.o).map'\n",
      "'[p for p in n if b(p)]' -> 'i.filter(p)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q(,()(f,)).f))).)(( (((( (((((())))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'y(y,y).m).ter(e => p(,r)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => (,)'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(i(.)i)tef(>)e i>)e (> ((() ))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(a => (,))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'y(y,y),))p),p)pp(p(pp(p(ppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 9\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 302s 3s/step - loss: 0.1101 - acc: 0.9697 - all_acc: 0.0000e+00 - val_loss: 0.1073 - val_acc: 0.9699 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1080 - acc: 0.9701 - all_acc: 0.0000e+00 - val_loss: 0.1345 - val_acc: 0.9626 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1127 - acc: 0.9693 - all_acc: 0.0000e+00 - val_loss: 0.1171 - val_acc: 0.9672 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0897 - acc: 0.9751 - all_acc: 0.0000e+00 - val_loss: 0.1097 - val_acc: 0.9699 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0941 - acc: 0.9742 - all_acc: 0.0000e+00 - val_loss: 0.0983 - val_acc: 0.9731 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k).map(s => h(s(s(s(s)))))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'e(e,e),filter(d => d(d,d))'\n",
      "'[l(n) for n in o]' -> 'o.o).map()'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(p)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q(,(,())))))))))))))))))))))))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'y(m,y).milter(l)map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(a,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 't(a).)i)).ri) )ma)(( (( )))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(c => c(c,c))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'y(y(y(,(,y),))ppap(ppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 10\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100/100 [==============================] - 301s 3s/step - loss: 0.0900 - acc: 0.9750 - all_acc: 0.0000e+00 - val_loss: 0.0881 - val_acc: 0.9754 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0905 - acc: 0.9751 - all_acc: 0.0000e+00 - val_loss: 0.1041 - val_acc: 0.9715 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0782 - acc: 0.9785 - all_acc: 0.0000e+00 - val_loss: 0.0827 - val_acc: 0.9769 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0892 - acc: 0.9756 - all_acc: 0.0000e+00 - val_loss: 0.0893 - val_acc: 0.9753 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0838 - acc: 0.9772 - all_acc: 0.0000e+00 - val_loss: 0.0783 - val_acc: 0.9781 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k).map(s => h(d(k(k(k(k)))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e,e)efilter(d => t(d,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(a)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q(qqqqqq)))))zr()(zr() ( (( ((()()))))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(a,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 't(i).)i)(er(> =( ,((j(((()())))))))))))))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(z => c(c,c))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a(y(,(,y),y)y))p)pp(p(pp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 11\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0835 - acc: 0.9772 - all_acc: 0.0000e+00 - val_loss: 0.0884 - val_acc: 0.9757 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0802 - acc: 0.9779 - all_acc: 0.0000e+00 - val_loss: 0.0785 - val_acc: 0.9786 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0840 - acc: 0.9776 - all_acc: 0.0000e+00 - val_loss: 0.0775 - val_acc: 0.9787 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0770 - acc: 0.9788 - all_acc: 0.0000e+00 - val_loss: 0.0882 - val_acc: 0.9764 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0811 - acc: 0.9779 - all_acc: 0.0000e+00 - val_loss: 0.0649 - val_acc: 0.9821 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k).map(s => h(d(s(k(,(,,,,,)))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e,e).filter(d => t(s,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(k(q(qqqq(q),q)))f.)a.(.)a.(.,()(.(()(((()())()('\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e)(e,))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 't(t).filter(c).(ap(( => ,((,((,((())))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(c)'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a(y(y(,y,y),y).p)pppp(ppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 12\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0814 - acc: 0.9780 - all_acc: 0.0000e+00 - val_loss: 0.0688 - val_acc: 0.9817 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0781 - acc: 0.9783 - all_acc: 0.0000e+00 - val_loss: 0.0787 - val_acc: 0.9775 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.2648 - acc: 0.9460 - all_acc: 0.0000e+00 - val_loss: 0.1974 - val_acc: 0.9464 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1946 - acc: 0.9483 - all_acc: 0.0000e+00 - val_loss: 0.1655 - val_acc: 0.9550 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1617 - acc: 0.9566 - all_acc: 0.0000e+00 - val_loss: 0.1604 - val_acc: 0.9578 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(s,k),m)s((s(s(s((s(s('\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(f,d),fie)ed d>d'\n",
      "'[l(n) for n in o]' -> 'o.map(a)( '\n",
      "'[p for p in n if b(p)]' -> 'n.filter(p => p'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q(qq,q)qq)q)q))q)q))))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,b).mi.m(r(= ='\n",
      "'[a(e,c) for e in b]' -> 'b.map(a => '\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(n(n(,(i.).)).)())(((((((((())))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'i.filter(p => > '\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a,y),f,p(ep(p pp(ppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 13\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1813 - acc: 0.9516 - all_acc: 0.0000e+00 - val_loss: 0.1853 - val_acc: 0.9467 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.1795 - acc: 0.9488 - all_acc: 0.0000e+00 - val_loss: 0.1514 - val_acc: 0.9566 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1571 - acc: 0.9554 - all_acc: 0.0000e+00 - val_loss: 0.1405 - val_acc: 0.9610 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1388 - acc: 0.9611 - all_acc: 0.0000e+00 - val_loss: 0.1420 - val_acc: 0.9592 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1269 - acc: 0.9640 - all_acc: 0.0000e+00 - val_loss: 0.1176 - val_acc: 0.9669 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k).map(k => k(h,a((((()))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e(e)efieterter(r => )'\n",
      "'[l(n) for n in o]' -> 'o.map(n)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(p).p p('\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q(q(q),)))q))))q))))q))))))))))))))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'd(d(y).filter(e)=m v(.m(p)v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(a => a(a,a))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(e(.(e((())(((((()(()((())(((())))))))))))))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'f.filter(z => p).map(c)))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'a(a(x(,)(),,)).))p).p(p)pp(p)pp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 14\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1254 - acc: 0.9651 - all_acc: 0.0000e+00 - val_loss: 0.1122 - val_acc: 0.9677 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1122 - acc: 0.9691 - all_acc: 0.0000e+00 - val_loss: 0.1176 - val_acc: 0.9678 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1072 - acc: 0.9708 - all_acc: 0.0000e+00 - val_loss: 0.1044 - val_acc: 0.9714 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.1008 - acc: 0.9717 - all_acc: 0.0000e+00 - val_loss: 0.1217 - val_acc: 0.9676 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0989 - acc: 0.9728 - all_acc: 0.0000e+00 - val_loss: 0.0983 - val_acc: 0.9730 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k).map(k => h(s,s()))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'e(e,e).filter(d => d(,'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(q(q)q),q(q),qf,ff.f))).)))).))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'd(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(c,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(n).fil(e(((())).)))).)))))))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 'f.filter(z => z).map(c)=> (,'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'a(a(y),),y,y),y),y).map(p)'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 15\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0878 - acc: 0.9757 - all_acc: 0.0000e+00 - val_loss: 0.0796 - val_acc: 0.9780 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0793 - acc: 0.9783 - all_acc: 0.0000e+00 - val_loss: 0.0775 - val_acc: 0.9788 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0794 - acc: 0.9785 - all_acc: 0.0000e+00 - val_loss: 0.0752 - val_acc: 0.9791 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0824 - acc: 0.9773 - all_acc: 0.0000e+00 - val_loss: 0.1040 - val_acc: 0.9707 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0896 - acc: 0.9750 - all_acc: 0.0000e+00 - val_loss: 0.0722 - val_acc: 0.9791 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'k(k).map(k => h(d(s(k)))))))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'e(e,e).filter(d => t(t,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(k(q),q)q))))))).)))).)))).)))).)))).))))())))())))()))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'd(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(a,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'n(n).),l(u())))))))))))))))))))))))))))))))))())()())())))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(z => i(c,z))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'a(a(y),y,y,y)).))p(p)pp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 16\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0782 - acc: 0.9782 - all_acc: 0.0000e+00 - val_loss: 0.0805 - val_acc: 0.9770 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0941 - acc: 0.9754 - all_acc: 0.0000e+00 - val_loss: 0.1247 - val_acc: 0.9669 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0960 - acc: 0.9739 - all_acc: 0.0000e+00 - val_loss: 0.0816 - val_acc: 0.9776 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0918 - acc: 0.9747 - all_acc: 0.0000e+00 - val_loss: 0.0717 - val_acc: 0.9811 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0761 - acc: 0.9791 - all_acc: 0.0000e+00 - val_loss: 0.0731 - val_acc: 0.9802 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k).map(s => h(d(s(k(k)))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e,e).filter(d => s(s,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(k(q),,,q,)))))).filt.r=> => >(.map(.m)()p))())'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(a,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'e(t).filter(c).map(i => j(i,e(n(,)))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(c => c(c,z))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a(y(y)(y,y)).m)pmppp)'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 17\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0772 - acc: 0.9796 - all_acc: 0.0000e+00 - val_loss: 0.0748 - val_acc: 0.9792 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0653 - acc: 0.9821 - all_acc: 0.0000e+00 - val_loss: 0.0714 - val_acc: 0.9810 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 300s 3s/step - loss: 0.0847 - acc: 0.9782 - all_acc: 0.0000e+00 - val_loss: 0.0808 - val_acc: 0.9777 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0719 - acc: 0.9807 - all_acc: 0.0000e+00 - val_loss: 0.0600 - val_acc: 0.9833 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0780 - acc: 0.9792 - all_acc: 0.0000e+00 - val_loss: 0.0662 - val_acc: 0.9823 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k).map(s => h(d(s(k)k)(k),)))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e,e).filter(s => s(s,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(k(q),q(q,z))),)))z)()(z)()(=)))))))))))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,y).filter(r).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(e,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'z(t).fel(e ()(a)(i(e)())e))()))))))))))))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(z => i(c,z))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a(y(y)yy,y)).)ap)pppp(p)'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 18\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0632 - acc: 0.9832 - all_acc: 0.0000e+00 - val_loss: 0.0745 - val_acc: 0.9794 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0638 - acc: 0.9823 - all_acc: 0.0000e+00 - val_loss: 0.0503 - val_acc: 0.9857 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0648 - acc: 0.9825 - all_acc: 0.0000e+00 - val_loss: 0.0736 - val_acc: 0.9792 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0684 - acc: 0.9817 - all_acc: 0.0000e+00 - val_loss: 0.0698 - val_acc: 0.9819 - val_all_acc: 0.0000e+00\n",
      "Epoch 5/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0693 - acc: 0.9812 - all_acc: 0.0000e+00 - val_loss: 0.0461 - val_acc: 0.9871 - val_all_acc: 0.0000e+00\n",
      "'[h(d(s(k),f(y(k),w)),s) for s in l(k)]' -> 'l(k).map(s => h(d(s(s(k)(k(k),)))))))))'\n",
      "'[d for d in p(e,e) if t(s,d)]' -> 'p(e,e).filter(d => t(s,d))'\n",
      "'[l(n) for n in o]' -> 'o.map(l)'\n",
      "'[p for p in n if b(p)]' -> 'n.filter(b)'\n",
      "'[f(q,e(p,r)) for q in q(k(q),g(r(q),z)) if j(z(z,z),q)]' -> 'q(k(q),g(r,z)).)))fi)ier(r z)z)zz(z))z))))))=> ( => ( (( ( )))))'\n",
      "'[v(r) for r in m(d,y) if l(r)]' -> 'm(d,y).filter(l).map(v)'\n",
      "'[a(e,c) for e in b]' -> 'b.map(e => a(e,c))'\n",
      "'[j(i,v([o(e) for e in z(n)],u(k(l,x(j)),n))) for i in e(t) if c(i)]' -> 'z(n).(u(t(j(xj((x())))))))))iiii( ))))))))))(((((((((((((()(()())))))'\n",
      "'[i(c,z) for z in t if z]' -> 't.filter(z => z).map(z => c(c,z))'\n",
      "'[p(p) for p in z(a(x(y),h(y,p)),y)]' -> 'z(a(y(y),y,y(,),y))pmppppppp'\n",
      "--------------------------------------------------------------------------------\n",
      "cycle 19\n",
      "Epoch 1/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0588 - acc: 0.9838 - all_acc: 0.0000e+00 - val_loss: 0.0532 - val_acc: 0.9855 - val_all_acc: 0.0000e+00\n",
      "Epoch 2/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0569 - acc: 0.9844 - all_acc: 0.0000e+00 - val_loss: 0.0569 - val_acc: 0.9844 - val_all_acc: 0.0000e+00\n",
      "Epoch 3/5\n",
      "100/100 [==============================] - 301s 3s/step - loss: 0.0708 - acc: 0.9826 - all_acc: 0.0000e+00 - val_loss: 0.1151 - val_acc: 0.9698 - val_all_acc: 0.0000e+00\n",
      "Epoch 4/5\n",
      " 88/100 [=========================>....] - ETA: 26s - loss: 0.0713 - acc: 0.9815 - all_acc: 0.0000e+00"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-1e336c81ae51>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m100000\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'cycle'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0mtrain_cycle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m     \u001b[0mshow_examples\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'-'\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0;36m80\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-17-620fcdad904d>\u001b[0m in \u001b[0;36mtrain_cycle\u001b[0;34m()\u001b[0m\n\u001b[1;32m     15\u001b[0m                         \u001b[0mworkers\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m                         \u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m                         epochs=5)\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     89\u001b[0m                 warnings.warn('Update your `' + object_name + '` call to the ' +\n\u001b[1;32m     90\u001b[0m                               'Keras 2 API: ' + signature, stacklevel=2)\n\u001b[0;32m---> 91\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     92\u001b[0m         \u001b[0mwrapper\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_original_function\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     93\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mwrapper\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit_generator\u001b[0;34m(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)\u001b[0m\n\u001b[1;32m   1416\u001b[0m             \u001b[0muse_multiprocessing\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_multiprocessing\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1417\u001b[0m             \u001b[0mshuffle\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mshuffle\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1418\u001b[0;31m             initial_epoch=initial_epoch)\n\u001b[0m\u001b[1;32m   1419\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1420\u001b[0m     \u001b[0;34m@\u001b[0m\u001b[0minterfaces\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlegacy_generator_methods_support\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py\u001b[0m in \u001b[0;36mfit_generator\u001b[0;34m(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)\u001b[0m\n\u001b[1;32m    215\u001b[0m                 outs = model.train_on_batch(x, y,\n\u001b[1;32m    216\u001b[0m                                             \u001b[0msample_weight\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msample_weight\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 217\u001b[0;31m                                             class_weight=class_weight)\n\u001b[0m\u001b[1;32m    218\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    219\u001b[0m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mtrain_on_batch\u001b[0;34m(self, x, y, sample_weight, class_weight)\u001b[0m\n\u001b[1;32m   1215\u001b[0m             \u001b[0mins\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0msample_weights\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1216\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_train_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1217\u001b[0;31m         \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mins\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1218\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0munpack_singleton\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1219\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2713\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_legacy_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2714\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2715\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2716\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2717\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mpy_any\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mis_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2673\u001b[0m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_metadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2674\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2675\u001b[0;31m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2676\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mfetched\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2677\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1437\u001b[0m           ret = tf_session.TF_SessionRunCallable(\n\u001b[1;32m   1438\u001b[0m               \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_handle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatus\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1439\u001b[0;31m               run_metadata_ptr)\n\u001b[0m\u001b[1;32m   1440\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1441\u001b[0m           \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "for i in range(100000):\n",
    "    print('cycle', i)\n",
    "    train_cycle()\n",
    "    show_examples()\n",
    "    print('-'*80)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input</th>\n",
       "      <th>output</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>358683</th>\n",
       "      <td>[a(k,u(w,p(z(k),k))) for k in i(x) if y(k)]</td>\n",
       "      <td>i(x).filter(y).map(k =&gt; a(k,u(w,p(z(k),k))))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>330020</th>\n",
       "      <td>o</td>\n",
       "      <td>o</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>333910</th>\n",
       "      <td>[f(q,e(y(a),s(q))) for q in t if x(q)]</td>\n",
       "      <td>t.filter(x).map(q =&gt; f(q,e(y(a),s(q))))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334352</th>\n",
       "      <td>g</td>\n",
       "      <td>g</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384249</th>\n",
       "      <td>q(r)</td>\n",
       "      <td>q(r)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              input  \\\n",
       "358683  [a(k,u(w,p(z(k),k))) for k in i(x) if y(k)]   \n",
       "330020                                            o   \n",
       "333910       [f(q,e(y(a),s(q))) for q in t if x(q)]   \n",
       "334352                                            g   \n",
       "384249                                         q(r)   \n",
       "\n",
       "                                              output  \n",
       "358683  i(x).filter(y).map(k => a(k,u(w,p(z(k),k))))  \n",
       "330020                                             o  \n",
       "333910       t.filter(x).map(q => f(q,e(y(a),s(q))))  \n",
       "334352                                             g  \n",
       "384249                                          q(r)  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = expressions.iloc[training_size:].sample(200).copy()\n",
    "evaluation.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 200/200 [01:41<00:00,  1.98it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input</th>\n",
       "      <th>output</th>\n",
       "      <th>results</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>376389</th>\n",
       "      <td>[o for o in o(n) if r(z(b(o),a),o)]</td>\n",
       "      <td>o(n).filter(o =&gt; r(z(b(o),a),o))</td>\n",
       "      <td>o(n).filter(o =&gt; r(r(o(o),)),o))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334148</th>\n",
       "      <td>t</td>\n",
       "      <td>t</td>\n",
       "      <td>t</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328131</th>\n",
       "      <td>[e for e in r(v,i(f,q(e))) if y(e)]</td>\n",
       "      <td>r(v,i(f,q(e))).filter(y)</td>\n",
       "      <td>r(v,f(f)l(())).filter(e)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>386355</th>\n",
       "      <td>[p for p in q(s,q(p,a(f))) if h(p)]</td>\n",
       "      <td>q(s,q(p,a(f))).filter(h)</td>\n",
       "      <td>q(s,q(q,a(f))).filter(f)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393946</th>\n",
       "      <td>[m(b) for b in d(c) if r(b)]</td>\n",
       "      <td>d(c).filter(r).map(m)</td>\n",
       "      <td>d(c).filter(r).map(m)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      input                            output  \\\n",
       "376389  [o for o in o(n) if r(z(b(o),a),o)]  o(n).filter(o => r(z(b(o),a),o))   \n",
       "334148                                    t                                 t   \n",
       "328131  [e for e in r(v,i(f,q(e))) if y(e)]          r(v,i(f,q(e))).filter(y)   \n",
       "386355  [p for p in q(s,q(p,a(f))) if h(p)]          q(s,q(p,a(f))).filter(h)   \n",
       "393946         [m(b) for b in d(c) if r(b)]             d(c).filter(r).map(m)   \n",
       "\n",
       "                                 results  \n",
       "376389  o(n).filter(o => r(r(o(o),)),o))  \n",
       "334148                                 t  \n",
       "328131          r(v,f(f)l(())).filter(e)  \n",
       "386355          q(s,q(q,a(f))).filter(f)  \n",
       "393946             d(c).filter(r).map(m)  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def apply_convert(input_string):\n",
    "    results = apply_model(input_string)\n",
    "    terminate = results.index('<unk>')\n",
    "    return ''.join(results[:terminate:])\n",
    "\n",
    "evaluation['results'] = [apply_convert(input_string) for input_string in tqdm(evaluation['input'])]\n",
    "evaluation.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.565"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(evaluation['output'] == evaluation['results'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input</th>\n",
       "      <th>expected</th>\n",
       "      <th>results</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>398497</th>\n",
       "      <td>[c(q(l,d),h) for h in l if h]</td>\n",
       "      <td>l.filter(h =&gt; h).map(h =&gt; c(q(l,d),h))</td>\n",
       "      <td>l.filter(h =&gt; h).map(h =&gt; h(c,d),)))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381279</th>\n",
       "      <td>[r for r in t if r]</td>\n",
       "      <td>t.filter(r =&gt; r)</td>\n",
       "      <td>t.filter(r =&gt; r)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>320977</th>\n",
       "      <td>[x(l) for l in w(k,g)]</td>\n",
       "      <td>w(k,g).map(x)</td>\n",
       "      <td>w(k,g).map(x)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>367582</th>\n",
       "      <td>[s(n,b(m,f)) for n in e(y(p(s),o(w(s),n)),s) if u(n)]</td>\n",
       "      <td>e(y(p(s),o(w(s),n)),s).filter(u).map(n =&gt; s(n,b(m,f)))</td>\n",
       "      <td>e(y(s(s))s),((s(s(),)),filter(er(r .&gt; )(n (&gt; (((((</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>367794</th>\n",
       "      <td>[z(j,e(w)) for j in h]</td>\n",
       "      <td>h.map(j =&gt; z(j,e(w)))</td>\n",
       "      <td>h.map(j =&gt; z(j,e)e)))</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384249</th>\n",
       "      <td>q(r)</td>\n",
       "      <td>q(r)</td>\n",
       "      <td>q(r)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>364872</th>\n",
       "      <td>[x(f) for f in s(r(d(f),l),f)]</td>\n",
       "      <td>s(r(d(f),l),f).map(x)</td>\n",
       "      <td>s(r(f)f),lf,lter(l =&gt; f(f,a)).ma</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328018</th>\n",
       "      <td>b</td>\n",
       "      <td>b</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>333045</th>\n",
       "      <td>[n for n in p if n]</td>\n",
       "      <td>p.filter(n =&gt; n)</td>\n",
       "      <td>p.filter(n =&gt; n)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>373145</th>\n",
       "      <td>[i(h) for h in t(i,a(h))]</td>\n",
       "      <td>t(i,a(h)).map(i)</td>\n",
       "      <td>t(i,a(h)).map(i)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                        input  \\\n",
       "398497                          [c(q(l,d),h) for h in l if h]   \n",
       "381279                                    [r for r in t if r]   \n",
       "320977                                 [x(l) for l in w(k,g)]   \n",
       "367582  [s(n,b(m,f)) for n in e(y(p(s),o(w(s),n)),s) if u(n)]   \n",
       "367794                                 [z(j,e(w)) for j in h]   \n",
       "384249                                                   q(r)   \n",
       "364872                         [x(f) for f in s(r(d(f),l),f)]   \n",
       "328018                                                      b   \n",
       "333045                                    [n for n in p if n]   \n",
       "373145                              [i(h) for h in t(i,a(h))]   \n",
       "\n",
       "                                                      expected  \\\n",
       "398497                  l.filter(h => h).map(h => c(q(l,d),h))   \n",
       "381279                                        t.filter(r => r)   \n",
       "320977                                           w(k,g).map(x)   \n",
       "367582  e(y(p(s),o(w(s),n)),s).filter(u).map(n => s(n,b(m,f)))   \n",
       "367794                                   h.map(j => z(j,e(w)))   \n",
       "384249                                                    q(r)   \n",
       "364872                                   s(r(d(f),l),f).map(x)   \n",
       "328018                                                       b   \n",
       "333045                                        p.filter(n => n)   \n",
       "373145                                        t(i,a(h)).map(i)   \n",
       "\n",
       "                                                   results  \n",
       "398497                l.filter(h => h).map(h => h(c,d),)))  \n",
       "381279                                    t.filter(r => r)  \n",
       "320977                                       w(k,g).map(x)  \n",
       "367582  e(y(s(s))s),((s(s(),)),filter(er(r .> )(n (> (((((  \n",
       "367794                               h.map(j => z(j,e)e)))  \n",
       "384249                                                q(r)  \n",
       "364872                    s(r(f)f),lf,lter(l => f(f,a)).ma  \n",
       "328018                                                   b  \n",
       "333045                                    p.filter(n => n)  \n",
       "373145                                    t(i,a(h)).map(i)  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with pd.option_context('display.max_colwidth', 200):\n",
    "    display(evaluation.rename(columns={'output':'expected'}).sample(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a discussion of results, see the corresponding blog post, [A Python to Scala transpiler using neural machine translation (NMT)](https://medium.com/@matthagy/a-python-to-scala-transpiler-using-neural-machine-translation-nmt-90d4d02afa70)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}