C1_W3_Lab_3_unet_model__M.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# U-Net Model\n",
    "In this week's assignment, you'll be using a network architecture called \"U-Net\". The name of this network architecture comes from it's U-like shape when shown in a diagram like this (image from [U-net entry on wikipedia](https://en.wikipedia.org/wiki/U-Net)): \n",
    "\n",
    "<img src=\"images/U-net_example_wikipedia.png\" alt=\"U-net Image\" width=\"600\"/>\n",
    "\n",
    "U-nets are commonly used for image segmentation, which will be your task in the upcoming assignment. You won't actually need to implement U-Net in the assignment, but we wanted to give you an opportunity to gain some familiarity with this architecture here before you use it in the assignment. \n",
    "\n",
    "As you can see from the diagram, this architecture features a series of down-convolutions connected by max-pooling operations, followed by a series of up-convolutions connected by upsampling and concatenation operations. Each of the down-convolutions is also connected directly to the concatenation operations in the upsampling portion of the network. For more detail on the U-Net architecture, have a look at the original [U-Net paper by Ronneberger et al. 2015](https://arxiv.org/abs/1505.04597). \n",
    "\n",
    "In this lab, you'll create a basic U-Net using Keras. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# Import the elements you'll need to build your U-Net\n",
    "import keras\n",
    "from keras import backend as K\n",
    "from keras.engine import Input, Model\n",
    "from keras.layers import Conv3D, MaxPooling3D, UpSampling3D, Activation, BatchNormalization, PReLU, Deconvolution3D\n",
    "from keras.optimizers import Adam\n",
    "from keras.layers.merge import concatenate\n",
    "# Set the image shape to have the channels in the first dimension\n",
    "K.set_image_data_format(\"channels_first\")\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. The \"Depth\" of your U-Net\n",
    "The \"depth\" of your U-Net is equal to the number of down-convolutions you will use. In the image above, the depth is 4 because there are 4 down-convolutions running down the left side including the very bottom of the U.\n",
    "\n",
    "For this exercise, you'll use a U-Net depth of 2, meaning you'll have 2 down-convolutions in your network. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Input Layer and its \"Depth\"\n",
    "\n",
    "In this lab and in the assignment, you will be doing 3D image segmentation, which is to say that, in addition to \"height\" and \"width\", your input layer will also have a \"length\". We are deliberately using the word \"length\" instead of \"depth\" here to describe the third spatial dimension of the input so as not to confuse it with the depth of the network as defined above.\n",
    "\n",
    "The shape of the input layer is `(num_channels, height, width, length)`, where `num_channels` you can think of like color channels in an image, `height`, `width` and `length` are just the size of the input.\n",
    "\n",
    "For the assignment, the values will be:\n",
    "- num_channels: 4\n",
    "- height: 160\n",
    "- width: 160\n",
    "- length: 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'input_1:0' shape=(?, 4, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define an input layer tensor of the shape you'll use in the assignment\n",
    "input_layer = Input(shape=(4, 160, 160, 16))\n",
    "input_layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the tensor shape has a '?' as the very first dimension.  This will be the batch size. So the dimensions of the tensor are: (batch_size, num_channels, height, width, length)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Contracting (downward) Path \n",
    "Here you'll start by constructing the downward path in your network (the left side of the U-Net).  The `(height, width, length)` of the input gets smaller as you move down this path, and the number of channels increases.\n",
    "\n",
    "### 2.1 Depth 0\n",
    "\n",
    "By \"depth 0\" here, we're referring to the depth of the first down-convolution in the U-net.\n",
    "\n",
    "The number of filters is specified for each depth and for each layer within that depth.\n",
    "\n",
    "The formula to use for calculating the number of filters is:\n",
    "$$filters_{i} = 32 \\times (2^{i})$$\n",
    "\n",
    "Where $i$ is the current depth.\n",
    "\n",
    "So at depth $i=0$:\n",
    "$$filters_{0} = 32 \\times (2^{0}) = 32$$\n",
    "\n",
    "### 2.2 Layer 0\n",
    "There are two convolutional layers for each depth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cell to create the first 3D convolution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'conv3d_1/add:0' shape=(?, 32, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a Conv3D tensor with 32 filters\n",
    "down_depth_0_layer_0 = Conv3D(filters=32, \n",
    "                              kernel_size=(3,3,3),\n",
    "                              padding='same',\n",
    "                              strides=(1,1,1)\n",
    "                              )(input_layer)\n",
    "down_depth_0_layer_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that with 32 filters, the result you get above is a tensor with 32 channels.\n",
    "\n",
    "Run the next cell to add a relu activation to the first convolutional layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_1/Relu:0' shape=(?, 32, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a relu activation to layer 0 of depth 0\n",
    "down_depth_0_layer_0 = Activation('relu')(down_depth_0_layer_0)\n",
    "down_depth_0_layer_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Depth 0, Layer 1\n",
    "For layer 1 of depth 0, the formula for calculating the number of filters is:\n",
    "$$filters_{i} = 32 \\times (2^{i}) \\times 2$$\n",
    "\n",
    "Where $i$ is the current depth. \n",
    "- Notice that the '$\\times~2$' at the end of this expression isn't there for layer 0.\n",
    "\n",
    "\n",
    "So at depth $i=0$ for layer 1:\n",
    "$$filters_{0} = 32 \\times (2^{0}) \\times 2 = 64$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_2/Relu:0' shape=(?, 64, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a Conv3D layer with 64 filters and add relu activation\n",
    "down_depth_0_layer_1 = Conv3D(filters=64, \n",
    "                kernel_size=(3,3,3),\n",
    "                padding='same',\n",
    "                strides=(1,1,1)\n",
    "               )(down_depth_0_layer_0)\n",
    "down_depth_0_layer_1 = Activation('relu')(down_depth_0_layer_1)\n",
    "down_depth_0_layer_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Max pooling\n",
    "Within the U-Net architecture, there is a max pooling operation after each of the down-convolutions (not including the last down-convolution at the bottom of the U). In general, this means you'll add max pooling after each down-convolution up to (but not including) the `depth - 1` down-convolution (since you started counting at 0). \n",
    "\n",
    "For this lab exercise:\n",
    "- The overall depth of the U-Net you're constructing is 2\n",
    "- So the bottom of your U is at a depth index of: $2-1 = 1$.\n",
    "- So far you've only defined the $depth=0$ down-convolutions, so the next thing to do is add max pooling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cell to add a max pooling operation to your U-Net"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'max_pooling3d_1/transpose_1:0' shape=(?, 64, 80, 80, 8) dtype=float32>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a max pooling layer\n",
    "down_depth_0_layer_pool = MaxPooling3D(pool_size=(2,2,2))(down_depth_0_layer_1)\n",
    "down_depth_0_layer_pool"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.4.1 Depth 1, Layer 0\n",
    "\n",
    "At depth 1, layer 0, the formula for calculating the number of filters is:\n",
    "$$filters_{i} = 32 \\times (2^{i})$$\n",
    "\n",
    "Where $i$ is the current depth.\n",
    "\n",
    "So at depth $i=1$:\n",
    "$$filters_{1} = 32 \\times (2^{1}) = 64$$\n",
    "\n",
    "Run the next cell to add a Conv3D layer to your network with relu activation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_3/Relu:0' shape=(?, 64, 80, 80, 8) dtype=float32>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a Conv3D layer to your network with relu activation\n",
    "down_depth_1_layer_0 = Conv3D(filters=64, \n",
    "                kernel_size=(3,3,3),\n",
    "                padding='same',\n",
    "                strides=(1,1,1)\n",
    "               )(down_depth_0_layer_pool)\n",
    "down_depth_1_layer_0 = Activation('relu')(down_depth_1_layer_0)\n",
    "down_depth_1_layer_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.4.2 Depth 1,  Layer 1\n",
    "\n",
    "For layer 1 of depth 1 the formula you'll use for number of filters is:\n",
    "$$filters_{i} = 32 \\times (2^{i}) \\times 2$$\n",
    "\n",
    "Where $i$ is the current depth. \n",
    "- Notice that the '$\\times 2$' at the end of this expression isn't there for layer 0.\n",
    "\n",
    "So at depth $i=1$:\n",
    "$$filters_{0} = 32 \\times (2^{1}) \\times 2 = 128$$\n",
    "\n",
    "Run the next cell to add another Conv3D with 128 filters to your network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_4/Relu:0' shape=(?, 128, 80, 80, 8) dtype=float32>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add another Conv3D with 128 filters to your network.\n",
    "down_depth_1_layer_1 = Conv3D(filters=128, \n",
    "                kernel_size=(3,3,3),\n",
    "                padding='same',\n",
    "                strides=(1,1,1)\n",
    "               )(down_depth_1_layer_0)\n",
    "down_depth_1_layer_1 = Activation('relu')(down_depth_1_layer_1)\n",
    "down_depth_1_layer_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### No max pooling at depth 1 (the bottom of the U)\n",
    "\n",
    "When you get to the \"bottom\" of the U-net, you don't need to apply max pooling after the convolutions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Expanding  (upward) Path\n",
    "\n",
    "Now you'll work on the expanding path of the U-Net, (going up on the right side, when viewing the diagram).  The image's (height, width, length) all get larger in the expanding path.\n",
    "\n",
    "### 3.1 Depth 0, Up Sampling Layer 0\n",
    "\n",
    "You'll use a pool size of (2,2,2) for upsampling.\n",
    "- This is the default value for [tf.keras.layers.UpSampling3D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/UpSampling3D)\n",
    "- As input to the upsampling at depth 1, you'll use the last layer of the downsampling.  In this case, it's the depth 1 layer 1.\n",
    "\n",
    "Run the next cell to add an upsampling operation to your network. \n",
    "Note that you're not adding any activation to this upsampling layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'up_sampling3d_1/concat_2:0' shape=(?, 128, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add an upsampling operation to your network\n",
    "up_depth_0_layer_0 = UpSampling3D(size=(2,2,2))(down_depth_1_layer_1)\n",
    "up_depth_0_layer_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Concatenate Upsampled Depth 0 with Downsampled Depth 0\n",
    "\n",
    "Now you'll apply a concatenation operation using the layers that are both at the same depth of 0.\n",
    "- up_depth_0_layer_0: shape is (?, 128, 160, 160, 16)\n",
    "- depth_0_layer_1: shape is (?, 64, 160, 160, 16)\n",
    "\n",
    "- Double check that both of these layers have the same height, width and length.\n",
    "- If they're the same, then they can be concatenated along axis 1 (the channel axis).\n",
    "- The (height, width, length) is (160, 160, 16) for both.\n",
    "\n",
    "Run the next cell to check that the layers you wish to concatenate have the same height, width and length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"up_sampling3d_1/concat_2:0\", shape=(?, 128, 160, 160, 16), dtype=float32)\n",
      "\n",
      "Tensor(\"activation_2/Relu:0\", shape=(?, 64, 160, 160, 16), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Print the shape of layers to concatenate\n",
    "print(up_depth_0_layer_0)\n",
    "print()\n",
    "print(down_depth_0_layer_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cell to add a concatenation operation to your network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'concatenate_1/concat:0' shape=(?, 192, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a concatenation along axis 1\n",
    "up_depth_1_concat = concatenate([up_depth_0_layer_0,\n",
    "                                 down_depth_0_layer_1],\n",
    "                                axis=1)\n",
    "up_depth_1_concat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the upsampling layer had 128 channels, and the down-convolution layer had 64 channels so that when concatenated, the result has 128 + 64 = 192 channels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Up-convolution Layer 1\n",
    "\n",
    "The number of filters for this layer will be set to the number of channels in the down-convolution's layer 1 at the same depth of 0 (down_depth_0_layer_1).\n",
    "\n",
    "Run the next cell to have a look at the shape of the down-convolution depth 0 layer 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_2/Relu:0' shape=(?, 64, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "down_depth_0_layer_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the number of channels for `depth_0_layer_1` is 64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of filters: 64\n"
     ]
    }
   ],
   "source": [
    "print(f\"number of filters: {down_depth_0_layer_1._keras_shape[1]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_5/Relu:0' shape=(?, 64, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a Conv3D up-convolution with 64 filters to your network\n",
    "up_depth_1_layer_1 = Conv3D(filters=64, \n",
    "                            kernel_size=(3,3,3),\n",
    "                            padding='same',\n",
    "                            strides=(1,1,1)\n",
    "                           )(up_depth_1_concat)\n",
    "up_depth_1_layer_1 = Activation('relu')(up_depth_1_layer_1)\n",
    "up_depth_1_layer_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Up-convolution Depth 0, Layer 2\n",
    "\n",
    "At layer 2 of depth 0 in the up-convolution the next step will be to add another up-convolution. The number of filters you'll want to use for this next up-convolution will need to be equal to the number of filters in the down-convolution depth 0 layer 1.\n",
    "\n",
    "Run the next cell to remind yourself of the number of filters in down-convolution depth 0 layer 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"activation_2/Relu:0\", shape=(?, 64, 160, 160, 16), dtype=float32)\n",
      "number of filters: 64\n"
     ]
    }
   ],
   "source": [
    "print(down_depth_0_layer_1)\n",
    "print(f\"number of filters: {down_depth_0_layer_1._keras_shape[1]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the number of channels / filters in `down_depth_0_layer_1` is 64."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cell to add a Conv3D up-convolution with 64 filters to your network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_6/Relu:0' shape=(?, 64, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a Conv3D up-convolution with 64 filters to your network\n",
    "up_depth_1_layer_2 = Conv3D(filters=64, \n",
    "                            kernel_size=(3,3,3),\n",
    "                            padding='same',\n",
    "                            strides=(1,1,1)\n",
    "                           )(up_depth_1_layer_1)\n",
    "up_depth_1_layer_2 = Activation('relu')(up_depth_1_layer_2)\n",
    "up_depth_1_layer_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Final Convolution\n",
    "\n",
    "For the final convolution, you will set the number of filters to be equal to the number of classes in your input data.\n",
    "\n",
    "In the assignment, you will be using data with 3 classes, namely:\n",
    "\n",
    "- 1: edema\n",
    "- 2: non-enhancing tumor \n",
    "- 3: enhancing tumor\n",
    "\n",
    "Run the next cell to add a final Conv3D with 3 filters to your network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'conv3d_7/add:0' shape=(?, 3, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a final Conv3D with 3 filters to your network.\n",
    "final_conv = Conv3D(filters=3, #3 categories \n",
    "                    kernel_size=(1,1,1),\n",
    "                    padding='valid',\n",
    "                    strides=(1,1,1)\n",
    "                    )(up_depth_1_layer_2)\n",
    "final_conv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Activation for Final Convolution\n",
    "\n",
    "Run the next cell to add a sigmoid activation to your final convolution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'activation_7/Sigmoid:0' shape=(?, 3, 160, 160, 16) dtype=float32>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add a sigmoid activation to your final convolution.\n",
    "final_activation = Activation('sigmoid')(final_conv)\n",
    "final_activation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Create and Compile the Model\n",
    "\n",
    "In this example, you will be setting the loss and metrics to options that are pre-built in Keras.  However, in the assignment, you will implement better loss functions and metrics for evaluating the model's performance.\n",
    "\n",
    "Run the next cell to define and compile your model based on the architecture you created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define and compile your model\n",
    "model = Model(inputs=input_layer, outputs=final_activation)\n",
    "model.compile(optimizer=Adam(lr=0.00001),\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['categorical_accuracy']\n",
    "             )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_1\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            (None, 4, 160, 160,  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_1 (Conv3D)               (None, 32, 160, 160, 3488        input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "activation_1 (Activation)       (None, 32, 160, 160, 0           conv3d_1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_2 (Conv3D)               (None, 64, 160, 160, 55360       activation_1[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "activation_2 (Activation)       (None, 64, 160, 160, 0           conv3d_2[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling3d_1 (MaxPooling3D)  (None, 64, 80, 80, 8 0           activation_2[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_3 (Conv3D)               (None, 64, 80, 80, 8 110656      max_pooling3d_1[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "activation_3 (Activation)       (None, 64, 80, 80, 8 0           conv3d_3[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_4 (Conv3D)               (None, 128, 80, 80,  221312      activation_3[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "activation_4 (Activation)       (None, 128, 80, 80,  0           conv3d_4[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "up_sampling3d_1 (UpSampling3D)  (None, 128, 160, 160 0           activation_4[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, 192, 160, 160 0           up_sampling3d_1[0][0]            \n",
      "                                                                 activation_2[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_5 (Conv3D)               (None, 64, 160, 160, 331840      concatenate_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_5 (Activation)       (None, 64, 160, 160, 0           conv3d_5[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_6 (Conv3D)               (None, 64, 160, 160, 110656      activation_5[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "activation_6 (Activation)       (None, 64, 160, 160, 0           conv3d_6[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_7 (Conv3D)               (None, 3, 160, 160,  195         activation_6[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "activation_7 (Activation)       (None, 3, 160, 160,  0           conv3d_7[0][0]                   \n",
      "==================================================================================================\n",
      "Total params: 833,507\n",
      "Trainable params: 833,507\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Print out a summary of the model you created\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Congratulations! You've created your very own U-Net model architecture!\n",
    "Next, you'll check that you did everything correctly by comparing your model summary to the example model defined below.\n",
    "\n",
    "#### 4.2.1 Double Check Your Model\n",
    "\n",
    "To double check that you created the correct model, use a function that we've provided to create the same model, and check that the layers and the layer dimensions match!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import predefined utilities\n",
    "import util"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a model using a predefined function\n",
    "model_2 = util.unet_model_3d(depth=2,\n",
    "                                loss_function='categorical_crossentropy',\n",
    "                                metrics=['categorical_accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model_2\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_2 (InputLayer)            (None, 4, 160, 160,  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_8 (Conv3D)               (None, 32, 160, 160, 3488        input_2[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "activation_8 (Activation)       (None, 32, 160, 160, 0           conv3d_8[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_9 (Conv3D)               (None, 64, 160, 160, 55360       activation_8[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "activation_9 (Activation)       (None, 64, 160, 160, 0           conv3d_9[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling3d_2 (MaxPooling3D)  (None, 64, 80, 80, 8 0           activation_9[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_10 (Conv3D)              (None, 64, 80, 80, 8 110656      max_pooling3d_2[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "activation_10 (Activation)      (None, 64, 80, 80, 8 0           conv3d_10[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_11 (Conv3D)              (None, 128, 80, 80,  221312      activation_10[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_11 (Activation)      (None, 128, 80, 80,  0           conv3d_11[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "up_sampling3d_2 (UpSampling3D)  (None, 128, 160, 160 0           activation_11[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_2 (Concatenate)     (None, 192, 160, 160 0           up_sampling3d_2[0][0]            \n",
      "                                                                 activation_9[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_12 (Conv3D)              (None, 64, 160, 160, 331840      concatenate_2[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_12 (Activation)      (None, 64, 160, 160, 0           conv3d_12[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_13 (Conv3D)              (None, 64, 160, 160, 110656      activation_12[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_13 (Activation)      (None, 64, 160, 160, 0           conv3d_13[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv3d_14 (Conv3D)              (None, 3, 160, 160,  195         activation_13[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_14 (Activation)      (None, 3, 160, 160,  0           conv3d_14[0][0]                  \n",
      "==================================================================================================\n",
      "Total params: 833,507\n",
      "Trainable params: 833,507\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Print out a summary of the model created by the predefined function\n",
    "model_2.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Look at the model summary for the U-Net you created and compare it to the summary for the example model created by the predefined function you imported above. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### That's it for this exercise, we hope this have provided you with more insight into the network architecture you'll be working with in this week's assignment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}