final-lab_2018.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Final Lab\n",
    "\n",
    "## Main task\n",
    "\n",
    "In this notebook, we will apply transfer learning techniques to finetune the [MobileNet](https://arxiv.org/pdf/1704.04861.pdf) CNN on [Cifar-10](https://www.cs.toronto.edu/~kriz/cifar.html) dataset.\n",
    "\n",
    "## Procedures\n",
    "\n",
    "In general, the main steps that we will follow are:\n",
    "\n",
    "1. Load data, analyze and split in *training*/*validation*/*testing* sets.\n",
    "2. Load CNN and analyze architecture.\n",
    "3. Adapt this CNN to our problem.\n",
    "4. Setup data augmentation techniques.\n",
    "5. Add some keras callbacks.\n",
    "6. Setup optimization algorithm with their hyperparameters.\n",
    "7. Train model!\n",
    "8. Choose best model/snapshot.\n",
    "9. Evaluate final model on the *testing* set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup one GPU for tensorflow (don't be greedy).\n",
    "import os\n",
    "os.environ[\"CUDA_DEVICE_ORDER\"] = \"PCI_BUS_ID\"\n",
    "# The GPU id to use, \"0\", \"1\", etc.\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\" \n",
    "\n",
    "# https://keras.io/applications/#documentation-for-individual-models\n",
    "from keras.applications.mobilenet import MobileNet\n",
    "from keras.datasets import cifar10\n",
    "from keras.models import Model\n",
    "from keras.layers import Dense, GlobalAveragePooling2D\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras.utils import to_categorical\n",
    "from sklearn.model_selection import train_test_split\n",
    "import cv2\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "# Limit tensorflow gpu usage.\n",
    "# Maybe you should comment this lines if you run tensorflow on CPU.\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "config.gpu_options.per_process_gpu_memory_fraction = 0.3\n",
    "sess = tf.Session(config=config)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load data, analyze and split in *training*/*validation*/*testing* sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cifar-10 class names\n",
    "# We will create a dictionary for each type of label\n",
    "# This is a mapping from the int class name to \n",
    "# their corresponding string class name\n",
    "LABELS = {\n",
    "    0: \"airplane\",\n",
    "    1: \"automobile\",\n",
    "    2: \"bird\",\n",
    "    3: \"cat\",\n",
    "    4: \"deer\",\n",
    "    5: \"dog\",\n",
    "    6: \"frog\",\n",
    "    7: \"horse\",\n",
    "    8: \"ship\",\n",
    "    9: \"truck\"\n",
    "}\n",
    "\n",
    "# Load dataset from keras\n",
    "(x_train_data, y_train_data), (x_test_data, y_test_data) = cifar10.load_data()\n",
    "\n",
    "############\n",
    "# [COMPLETE] \n",
    "# Add some prints here to see the loaded data dimensions\n",
    "############\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Analyze the amount of images for each class\n",
    "# Plot some images to explore how they look\n",
    "############\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Split training set in train/val sets\n",
    "# Use the sampling method that you want\n",
    "############\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In order to use the MobileNet CNN pre-trained on imagenet, we have\n",
    "# to resize our images to have one of the following static square shape: [(128, 128),\n",
    "# (160, 160), (192, 192), or (224, 224)].\n",
    "# If we try to resize all the dataset this will not fit on memory, so we have to save all\n",
    "# the images to disk, and then when loading those images, our datagenerator will resize them\n",
    "# to the desired shape on-the-fly.\n",
    "\n",
    "def save_to_disk(x_data, y_data, usage, output_dir='cifar10_images'):\n",
    "    \"\"\"\n",
    "    This function will resize your data using the specified output_size and \n",
    "    save them to output_dir.\n",
    "    \n",
    "    x_data : np.ndarray\n",
    "        Array with images.\n",
    "    \n",
    "    y_data : np.ndarray\n",
    "        Array with labels.\n",
    "    \n",
    "    usage : str\n",
    "        One of ['train', 'val', 'test'].\n",
    "\n",
    "    output_dir : str\n",
    "        Path to save data.\n",
    "    \"\"\"\n",
    "    assert usage in ['train', 'val', 'test']\n",
    "    \n",
    "    # Set paths \n",
    "    if not os.path.exists(output_dir):\n",
    "        os.makedirs(output_dir)\n",
    "    for label in np.unique(y_data):\n",
    "        label_path = os.path.join(output_dir, usage, str(label))\n",
    "        if not os.path.exists(label_path):\n",
    "            os.makedirs(label_path)\n",
    "    \n",
    "    for idx, img in enumerate(x_data):\n",
    "        bgr_img = img[..., ::-1]  # RGB -> BGR\n",
    "        label = y_data[idx][0]\n",
    "        img_path = os.path.join(output_dir, usage, str(label), 'img_{}.jpg'.format(idx))\n",
    "\n",
    "        retval = cv2.imwrite(img_path, bgr_img)\n",
    "        assert retval, 'Problem saving image at index:{}'.format(idx)\n",
    "\n",
    "\n",
    "############\n",
    "# [COMPLETE] \n",
    "# Use the above function to save all your data, e.g.:\n",
    "#    save_to_disk(x_train, y_train, 'train', 'cifar10_images')\n",
    "#    save_to_disk(x_val, y_val, 'val', 'cifar10_images')\n",
    "#    save_to_disk(x_test, y_test, 'test', 'cifar10_images')\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load CNN and analyze architecture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Use the MobileNet class from Keras to load your base model, pre-trained on imagenet.\n",
    "# We wan't to load the pre-trained weights, but without the classification layer.\n",
    "# Check the notebook '3_transfer-learning' or https://keras.io/applications/#mobilenet to get more\n",
    "# info about how to load this network properly.\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Adapt this CNN to our problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Having the CNN loaded, now we have to add some layers to adapt this network to our\n",
    "# classification problem.\n",
    "# We can choose to finetune just the new added layers, some particular layers or all the layer of the\n",
    "# model. Play with different settings and compare the results.\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Setup data augmentation techniques"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Use data augmentation to train your model.\n",
    "# Use the Keras ImageDataGenerator class for this porpouse.\n",
    "# Note: Given that we want to load our images from disk, instead of using \n",
    "# ImageDataGenerator.flow method, we have to use ImageDataGenerator.flow_from_directory \n",
    "# method in the following way:\n",
    "#    generator_train = dataget_train.flow_from_directory('resized_images/train', \n",
    "#                                                        target_size=(128, 128), batch_size=32)\n",
    "#    generator_val = dataget_train.flow_from_directory('resized_images/val', \n",
    "#                                                      target_size=(128, 128), batch_size=32)\n",
    "# Note that we have to resize our images to finetune the MobileNet CNN, this is done using \n",
    "# the target_size argument in flow_from_directory. Remember to set the target_size to one of\n",
    "# the valid listed here: [(128, 128), (160, 160), (192, 192), or (224, 224)].\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Add some keras callbacks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Load and set some Keras callbacks here!\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Setup optimization algorithm with their hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Choose some optimization algorithm and explore different hyperparameters.\n",
    "# Compile your model.\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Train model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Use fit_generator to train your model.\n",
    "# e.g.:\n",
    "#    model.fit_generator(\n",
    "#        generator_train,\n",
    "#        epochs=50,\n",
    "#        validation_data=generator_val,\n",
    "#        steps_per_epoch=generator_train.n // 32,\n",
    "#        validation_steps=generator_val.n // 32)\n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Choose best model/snapshot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Analyze and compare your results. Choose the best model and snapshot, \n",
    "# justify your election. \n",
    "############\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Evaluate final model on the *testing* set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############\n",
    "# [COMPLETE] \n",
    "# Evaluate your model on the testing set.\n",
    "############\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}