2017-Jul-RMLL-Docker-Python/2017-July-3-RMLL-Python_and_Docker.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "  \n",
    "# Python User Group Grenoble\n",
    "\n",
    "## Docker + Python, Python + Docker\n",
    "\n",
    "### Michael Bright, 3 juillet 2017, RMLL, St-Etienne"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<a name=\"TOP\" />\n",
    "\n",
    "Agenda\n",
    "\n",
    "- <a href=\"#PYTHON_APPS\"> * </a> Running Python apps under Docker\n",
    "\n",
    "- <a href=\"#API\"> * </a>The Docker API\n",
    "\n",
    "- <a href=\"#MACHINE\"> * </a>Docker-machine from Python\n",
    "\n",
    "- <a href=\"#DOCKERPY\"> * </a>docker-py: Controlling Docker from Python\n",
    "\n",
    "- <a href=\"#OTHERPY\"> * </a>Other Modules\n",
    "\n",
    "- <a href=\"#ANSIBLE\"> * </a>Ansible Container"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"PYTHON_APPS\" />\n",
    "# Running Python apps under Docker\n",
    "<a href=\"#TOP\"> TOP </a>\n",
    "\n",
    "Of course we can run any Python app under Docker.\n",
    "\n",
    "This can be a very useful alternative to virtualenv\n",
    "- we can have complete filesystem isolation allowing to have complex dependancies beyond Python itself\n",
    "- we can easily launch multiple instances of a Python process in isolation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker stop $(docker ps -qa)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Flask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/mjb/Dropbox/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py/flask-app\n"
     ]
    }
   ],
   "source": [
    "cd ~/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py/flask-app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#!/usr/bin/env python\r\n",
      "\r\n",
      "from flask import Flask\r\n",
      "app = Flask(__name__)\r\n",
      "\r\n",
      "''' Create the default '/' route '''\r\n",
      "@app.route('/')\r\n",
      "def root_handler():\r\n",
      "    return 'Hello World from Flask - /'\r\n",
      "\r\n",
      "''' Create an alternative '/test' route '''\r\n",
      "@app.route('/test')\r\n",
      "def test_handler():\r\n",
      "    return 'Hello World from Flask - /test'\r\n",
      "\r\n",
      "if __name__ == '__main__':\r\n",
      "    app.run(debug=True,host='0.0.0.0')\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat flask_app.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FROM ubuntu:latest\r\n",
      "\r\n",
      "LABEL MAINTAINER Mike Bright \"dockerfile@mjbright.net\"\r\n",
      "\r\n",
      "# Install pip:\r\n",
      "RUN apt-get update -y && \\\r\n",
      "    apt-get install -y python-pip python-dev build-essential\r\n",
      "\r\n",
      "WORKDIR /app\r\n",
      "\r\n",
      "# Copy in requirements before the source files: as likely to be updated less frequently\r\n",
      "COPY    ./requirements.txt /app/\r\n",
      "\r\n",
      "# Install the Flask module:\r\n",
      "RUN pip install -r requirements.txt\r\n",
      "\r\n",
      "# Copy in the applicaton script:\r\n",
      "COPY    ./flask_app.py     /app/\r\n",
      "\r\n",
      "# Set the default command:\r\n",
      "ENTRYPOINT [\"python\"]\r\n",
      "CMD        [\"flask_app.py\"]\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "cat Dockerfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending build context to Docker daemon  4.608kB\n",
      "Step 1/9 : FROM ubuntu:latest\n",
      " ---> d355ed3537e9\n",
      "Step 2/9 : LABEL MAINTAINER Mike Bright \"dockerfile@mjbright.net\"\n",
      " ---> Using cache\n",
      " ---> 6ee2e95b312a\n",
      "Step 3/9 : RUN apt-get update -y &&     apt-get install -y python-pip python-dev build-essential\n",
      " ---> Using cache\n",
      " ---> 51c4db36528c\n",
      "Step 4/9 : WORKDIR /app\n",
      " ---> Using cache\n",
      " ---> 859a5b98bd66\n",
      "Step 5/9 : COPY ./requirements.txt /app/\n",
      " ---> Using cache\n",
      " ---> 7a681c19a582\n",
      "Step 6/9 : RUN pip install -r requirements.txt\n",
      " ---> Using cache\n",
      " ---> e0b779c11c92\n",
      "Step 7/9 : COPY ./flask_app.py /app/\n",
      " ---> Using cache\n",
      " ---> fe8991ce30d2\n",
      "Step 8/9 : ENTRYPOINT python\n",
      " ---> Using cache\n",
      " ---> 12cdb6f03a53\n",
      "Step 9/9 : CMD flask_app.py\n",
      " ---> Using cache\n",
      " ---> 2d4ffd4a9bbc\n",
      "Successfully built 2d4ffd4a9bbc\n",
      "Successfully tagged mjbright/flaskapp:latest\n"
     ]
    }
   ],
   "source": [
    "!docker build -t mjbright/flaskapp ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ecf09b28660f8269c8a436f8de2e612b307a4a507c072e10db65a56954944543\r\n"
     ]
    }
   ],
   "source": [
    "!docker run -d -p 5000:5000 mjbright/flaskapp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World from Flask - /"
     ]
    }
   ],
   "source": [
    "!wget -q -O - http://localhost:5000/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World from Flask - /test"
     ]
    }
   ],
   "source": [
    "!wget -q -O - http://localhost:5000/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Cherrypy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/mjb/Dropbox/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py/cherrypy-app\n"
     ]
    }
   ],
   "source": [
    "cd ~/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py/cherrypy-app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#!/usr/bin/env python\r\n",
      "\r\n",
      "import cherrypy\r\n",
      "\r\n",
      "class Root:\r\n",
      "\r\n",
      "    ''' Create the default '/' route '''\r\n",
      "    @cherrypy.expose\r\n",
      "    def index(self):\r\n",
      "        return 'Hello World from CherryPy - /'\r\n",
      "\r\n",
      "    ''' Create an alternative '/test' route '''\r\n",
      "    @cherrypy.expose\r\n",
      "    def test(self):\r\n",
      "        return 'Hello World from CherryPy - /test'\r\n",
      "\r\n",
      "if __name__ == '__main__':\r\n",
      "    cherrypy.config.update({'server.socket_host': '0.0.0.0'})\r\n",
      "    cherrypy.quickstart(Root())\r\n"
     ]
    }
   ],
   "source": [
    "!cat cherrypy_app.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FROM ubuntu:latest\r\n",
      "\r\n",
      "LABEL MAINTAINER Mike Bright \"dockerfile@mjbright.net\"\r\n",
      "\r\n",
      "# Install pip:\r\n",
      "RUN apt-get update -y && \\\r\n",
      "    apt-get install -y python-pip python-dev build-essential\r\n",
      "\r\n",
      "WORKDIR /app\r\n",
      "\r\n",
      "# Copy in requirements before the source files: as likely to be updated less frequently\r\n",
      "COPY    ./requirements.txt /app/\r\n",
      "\r\n",
      "# Install the CherryPy module:\r\n",
      "RUN pip install -r requirements.txt\r\n",
      "\r\n",
      "# Copy in the applicaton script:\r\n",
      "COPY    ./cherrypy_app.py     /app/\r\n",
      "\r\n",
      "# Set the default command:\r\n",
      "ENTRYPOINT [\"python\"]\r\n",
      "CMD        [\"cherrypy_app.py\"]\r\n"
     ]
    }
   ],
   "source": [
    "!cat Dockerfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending build context to Docker daemon  4.608kB\n",
      "Step 1/9 : FROM ubuntu:latest\n",
      " ---> d355ed3537e9\n",
      "Step 2/9 : LABEL MAINTAINER Mike Bright \"dockerfile@mjbright.net\"\n",
      " ---> Using cache\n",
      " ---> 6ee2e95b312a\n",
      "Step 3/9 : RUN apt-get update -y &&     apt-get install -y python-pip python-dev build-essential\n",
      " ---> Using cache\n",
      " ---> 51c4db36528c\n",
      "Step 4/9 : WORKDIR /app\n",
      " ---> Using cache\n",
      " ---> 859a5b98bd66\n",
      "Step 5/9 : COPY ./requirements.txt /app/\n",
      " ---> Using cache\n",
      " ---> 977309ea7e05\n",
      "Step 6/9 : RUN pip install -r requirements.txt\n",
      " ---> Using cache\n",
      " ---> 1d57541b9b7a\n",
      "Step 7/9 : COPY ./cherrypy_app.py /app/\n",
      " ---> Using cache\n",
      " ---> e6602881d27b\n",
      "Step 8/9 : ENTRYPOINT python\n",
      " ---> Using cache\n",
      " ---> 55f831e90de2\n",
      "Step 9/9 : CMD cherrypy_app.py\n",
      " ---> Using cache\n",
      " ---> eb2c01b4f1fb\n",
      "Successfully built eb2c01b4f1fb\n",
      "Successfully tagged mjbright/cherrpyapp:latest\n"
     ]
    }
   ],
   "source": [
    "!docker build -t mjbright/cherrpyapp ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b0991896976d694c5a97aa1f32a10b5af7943572d2b4b66d69385b58c712875d\r\n"
     ]
    }
   ],
   "source": [
    "!docker run -d -p 9191:8080 mjbright/cherrpyapp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World from CherryPy - /"
     ]
    }
   ],
   "source": [
    "!wget -q -O - http://localhost:9191/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                    NAMES\r\n",
      "b0991896976d        mjbright/cherrpyapp   \"python cherrypy_a...\"   45 seconds ago      Up 42 seconds       0.0.0.0:9191->8080/tcp   quizzical_yonath\r\n",
      "ecf09b28660f        mjbright/flaskapp     \"python flask_app.py\"    4 minutes ago       Up 4 minutes        0.0.0.0:5000->5000/tcp   cocky_cori\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World from CherryPy - /test"
     ]
    }
   ],
   "source": [
    "!wget -q -O - http://localhost:9191/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"API\" />\n",
    "# Docker API\n",
    "<a href=\"#TOP\"> TOP </a>\n",
    "\n",
    "Everything starts with the API whether that be\n",
    "- The docker cli\n",
    "- The docker-py module or other language bindings ...\n",
    "\n",
    "So let's look at that first.\n",
    "\n",
    "On my machine, the Docker daemon is listening on port *localhost:2475*\n",
    "\n",
    "(**Note**: that's unusual)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/mjb/Dropbox/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py\n"
     ]
    }
   ],
   "source": [
    "cd ~/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-py/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root     27835 27833  1 Jun30 pts/16   01:08:09 /home/mjb/DOCKER/docker-17.06.0-ce/dockerd --bridge=maindocker0 --exec-root=/home/mjb/var/run/maindocker/maindocker0.exec --graph=/home/mjb/var/run/maindocker/maindocker0.graph --host unix:///home/mjb/var/run/maindocker/maindocker0.sock --host tcp://127.0.0.1:2475 --pidfile=/home/mjb/var/run/maindocker/maindocker0.pid\r\n"
     ]
    }
   ],
   "source": [
    "!ps -fade | grep dockerd | grep -v grep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - monitoring the cli client\n",
    "\n",
    "**Note**: My Docker daemon is listening on *localhost:2475* without encryption for  experimental purposes - so we can examine  API traffic\n",
    "\n",
    "I'm running a Python script to monitor  *localhost:2475* and save request/responses to a file.\n",
    "\n",
    "Let's do some cli operations, and observe what goes out over the API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client:\r\n",
      " Version:      17.06.0-ce\r\n",
      " API version:  1.30\r\n",
      " Go version:   go1.8.3\r\n",
      " Git commit:   02c1d87\r\n",
      " Built:        Fri Jun 23 21:15:15 2017\r\n",
      " OS/Arch:      linux/amd64\r\n",
      "\r\n",
      "Server:\r\n",
      " Version:      17.06.0-ce\r\n",
      " API version:  1.30 (minimum version 1.12)\r\n",
      " Go version:   go1.8.3\r\n",
      " Git commit:   02c1d87\r\n",
      " Built:        Fri Jun 23 21:51:55 2017\r\n",
      " OS/Arch:      linux/amd64\r\n",
      " Experimental: false\r\n"
     ]
    }
   ],
   "source": [
    "!docker version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the request which was made to the Docker daemon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:32:55] REQUEST <<GET /v1.30/version HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the response which we got back from the Docker daemon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:32:55] RESPONSE <<HTTP/1.1 200 OK\r\n",
      "    Api-Version: 1.30\r\n",
      "    Content-Type: application/json\r\n",
      "    Docker-Experimental: false\r\n",
      "    Ostype: linux\r\n",
      "    Server: Docker/17.06.0-ce (linux)\r\n",
      "    Date: Mon, 03 Jul 2017 13:32:55 GMT\r\n",
      "    Content-Length: 231>>\r\n",
      "\r\n",
      "RESPONSE=<<{\r\n",
      "    \"ApiVersion\": \"1.30\", \r\n",
      "    \"Arch\": \"amd64\", \r\n",
      "    \"BuildTime\": \"2017-06-23T21:51:55.152028673+00:00\", \r\n",
      "    \"GitCommit\": \"02c1d87\", \r\n",
      "    \"GoVersion\": \"go1.8.3\", \r\n",
      "    \"KernelVersion\": \"4.11.6-101.fc24.x86_64\", \r\n",
      "    \"MinAPIVersion\": \"1.12\", \r\n",
      "    \"Os\": \"linux\", \r\n",
      "    \"Version\": \"17.06.0-ce\"\r\n",
      "}>>\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and we can compare this to the docker cli to see how it treats the information that it receives from the API.  Obviously this corresponds to the \"Server:\" information which comes via the API, not the \"Client:\" information which the client holds already"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!docker version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - monitoring 'docker ps'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                    NAMES\r\n",
      "b0991896976d        mjbright/cherrpyapp   \"python cherrypy_a...\"   10 minutes ago      Up 10 minutes       0.0.0.0:9191->8080/tcp   quizzical_yonath\r\n",
      "ecf09b28660f        mjbright/flaskapp     \"python flask_app.py\"    13 minutes ago      Up 13 minutes       0.0.0.0:5000->5000/tcp   cocky_cori\r\n"
     ]
    }
   ],
   "source": [
    "!docker container ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the request which was made to the Docker daemon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:38:25] REQUEST <<GET /v1.30/containers/json HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "and the request we got back"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:38:25] RESPONSE <<HTTP/1.1 200 OK\r\n",
      "    Api-Version: 1.30\r\n",
      "    Content-Type: application/json\r\n",
      "    Docker-Experimental: false\r\n",
      "    Ostype: linux\r\n",
      "    Server: Docker/17.06.0-ce (linux)\r\n",
      "    Date: Mon, 03 Jul 2017 13:38:25 GMT\r\n",
      "    Content-Length: 1857>>\r\n",
      "\r\n",
      "RESPONSE=<<[\r\n",
      "{\"Id\":\"b0991896976d694c5a97aa1f32a10b5af7943572d2b4b66d69385b58c712875d\",\r\n",
      "    \"Names\":[\r\n",
      "    \"/quizzical_yonath\"]\r\n",
      "      ,\r\n",
      "    \"Image\":\"mjbright/cherrpyapp\",\r\n",
      "    \"ImageID\":\"sha256:eb2c01b4f1fb83799ec7cf5666668baf07be4950653e9aea5eafcff6a55ba2d8\",\r\n",
      "    \"Command\":\"python.cherrypy_app.py\",\r\n",
      "    \"Created\":1499088488,\r\n",
      "    \"Ports\":[\r\n",
      "    {\"IP\":\"0.0.0.0\",\r\n",
      "        \"PrivatePort\":8080,\r\n",
      "        \"PublicPort\":9191,\r\n",
      "        \"Type\":\"tcp\"}\r\n",
      "        ],\r\n",
      "    \"Labels\":{\r\n",
      "    \"MAINTAINER\":\"Mike.Bright.dockerfile@mjbright.net\"}\r\n",
      "      ,\r\n",
      "    \"State\":\"running\",\r\n",
      "    \"Status\":\"Up.10.minutes\",\r\n",
      "    \"HostConfig\":{\r\n",
      "    \"NetworkMode\":\"default\"}\r\n",
      "      ,\r\n",
      "    \"NetworkSettings\":{\r\n",
      "    \"Networks\":{\r\n",
      "      \"bridge\":{\r\n",
      "        \"IPAMConfig\":null,\r\n",
      "          \"Links\":null,\r\n",
      "          \"Aliases\":null,\r\n",
      "          \"NetworkID\":\"151794f82880e59768799c567714ca3e8a15dd0f1c6d1957ded20a1cc67eb364\",\r\n",
      "          \"EndpointID\":\"251540abf2297c64a9cbabad4601495011bd77149bb0e78b14ac8f18ad00a3eb\",\r\n",
      "          \"Gateway\":\"10.9.0.10\",\r\n",
      "          \"IPAddress\":\"10.9.0.2\",\r\n",
      "          \"IPPrefixLen\":24,\r\n",
      "          \"IPv6Gateway\":\"\",\r\n",
      "          \"GlobalIPv6Address\":\"\",\r\n",
      "          \"GlobalIPv6PrefixLen\":0,\r\n",
      "          \"MacAddress\":\"02:42:0a:09:00:02\",\r\n",
      "          \"DriverOpts\":null}\r\n",
      "          }},\r\n",
      "    \"Mounts\":[\r\n",
      "    ]},\r\n",
      "  {\r\n",
      "  \"Id\":\"ecf09b28660f8269c8a436f8de2e612b307a4a507c072e10db65a56954944543\",\r\n",
      "    \"Names\":[\r\n",
      "    \"/cocky_cori\"]\r\n",
      "      ,\r\n",
      "    \"Image\":\"mjbright/flaskapp\",\r\n",
      "    \"ImageID\":\"sha256:2d4ffd4a9bbc404da4f1d5a1383d639b85757b85b3fdea22e4e769a998d8da07\",\r\n",
      "    \"Command\":\"python.flask_app.py\",\r\n",
      "    \"Created\":1499088282,\r\n",
      "    \"Ports\":[\r\n",
      "    {\"IP\":\"0.0.0.0\",\r\n",
      "        \"PrivatePort\":5000,\r\n",
      "        \"PublicPort\":5000,\r\n",
      "        \"Type\":\"tcp\"}\r\n",
      "        ],\r\n",
      "    \"Labels\":{\r\n",
      "    \"MAINTAINER\":\"Mike.Bright.dockerfile@mjbrigh>>\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try that query ourselves using wget:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;39m[\r\n",
      "  \u001b[1;39m{\r\n",
      "    \u001b[0m\u001b[34;1m\"Id\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"b0991896976d694c5a97aa1f32a10b5af7943572d2b4b66d69385b58c712875d\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Names\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[\r\n",
      "      \u001b[0;32m\"/quizzical_yonath\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m]\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Image\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"mjbright/cherrpyapp\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"ImageID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"sha256:eb2c01b4f1fb83799ec7cf5666668baf07be4950653e9aea5eafcff6a55ba2d8\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Command\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"python cherrypy_app.py\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Created\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m1499088488\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Ports\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[\r\n",
      "      \u001b[1;39m{\r\n",
      "        \u001b[0m\u001b[34;1m\"IP\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"0.0.0.0\"\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"PrivatePort\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m8080\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"PublicPort\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m9191\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"Type\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"tcp\"\u001b[0m\u001b[1;39m\r\n",
      "      \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m]\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Labels\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"MAINTAINER\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Mike Bright dockerfile@mjbright.net\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"State\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"running\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Status\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Up 12 minutes\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"HostConfig\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"NetworkMode\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"default\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"NetworkSettings\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"Networks\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "        \u001b[0m\u001b[34;1m\"bridge\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "          \u001b[0m\u001b[34;1m\"IPAMConfig\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Links\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Aliases\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"NetworkID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"151794f82880e59768799c567714ca3e8a15dd0f1c6d1957ded20a1cc67eb364\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"EndpointID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"251540abf2297c64a9cbabad4601495011bd77149bb0e78b14ac8f18ad00a3eb\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Gateway\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"10.9.0.10\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPAddress\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"10.9.0.2\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPPrefixLen\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m24\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPv6Gateway\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"GlobalIPv6Address\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"GlobalIPv6PrefixLen\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m0\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"MacAddress\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"02:42:0a:09:00:02\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"DriverOpts\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m\r\n",
      "        \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "      \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Mounts\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[]\u001b[0m\u001b[1;39m\r\n",
      "  \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "  \u001b[1;39m{\r\n",
      "    \u001b[0m\u001b[34;1m\"Id\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"ecf09b28660f8269c8a436f8de2e612b307a4a507c072e10db65a56954944543\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Names\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[\r\n",
      "      \u001b[0;32m\"/cocky_cori\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m]\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Image\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"mjbright/flaskapp\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"ImageID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"sha256:2d4ffd4a9bbc404da4f1d5a1383d639b85757b85b3fdea22e4e769a998d8da07\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Command\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"python flask_app.py\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Created\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m1499088282\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Ports\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[\r\n",
      "      \u001b[1;39m{\r\n",
      "        \u001b[0m\u001b[34;1m\"IP\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"0.0.0.0\"\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"PrivatePort\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m5000\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"PublicPort\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m5000\u001b[0m\u001b[1;39m,\r\n",
      "        \u001b[0m\u001b[34;1m\"Type\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"tcp\"\u001b[0m\u001b[1;39m\r\n",
      "      \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m]\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Labels\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"MAINTAINER\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Mike Bright dockerfile@mjbright.net\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"State\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"running\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Status\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Up 16 minutes\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"HostConfig\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"NetworkMode\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"default\"\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"NetworkSettings\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "      \u001b[0m\u001b[34;1m\"Networks\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "        \u001b[0m\u001b[34;1m\"bridge\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\r\n",
      "          \u001b[0m\u001b[34;1m\"IPAMConfig\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Links\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Aliases\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"NetworkID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"151794f82880e59768799c567714ca3e8a15dd0f1c6d1957ded20a1cc67eb364\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"EndpointID\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"978e969cfa40a2c2600626a54fa0c427692b8aeda5f9a51a7117fe27cf258575\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"Gateway\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"10.9.0.10\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPAddress\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"10.9.0.1\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPPrefixLen\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m24\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"IPv6Gateway\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"GlobalIPv6Address\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"GlobalIPv6PrefixLen\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m0\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"MacAddress\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"02:42:0a:09:00:01\"\u001b[0m\u001b[1;39m,\r\n",
      "          \u001b[0m\u001b[34;1m\"DriverOpts\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m\r\n",
      "        \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "      \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "    \u001b[1;39m}\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"Mounts\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[]\u001b[0m\u001b[1;39m\r\n",
      "  \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m]\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!wget -q -O - http://localhost:2475/v1.30/containers/json | jq ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - monitoring 'docker ps -a'\n",
    "\n",
    "Now let's specify the **-a** option to list all containers, not just running ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                    NAMES\r\n",
      "b0991896976d        mjbright/cherrpyapp   \"python cherrypy_a...\"   13 minutes ago      Up 13 minutes       0.0.0.0:9191->8080/tcp   quizzical_yonath\r\n",
      "ecf09b28660f        mjbright/flaskapp     \"python flask_app.py\"    16 minutes ago      Up 16 minutes       0.0.0.0:5000->5000/tcp   cocky_cori\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the request which was made to the Docker daemon\n",
    "\n",
    "Note that using the '-a' option has modified the request url to append '?all=1':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:41:13] REQUEST <<GET /v1.30/containers/json?all=1 HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "!cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!which docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://cloud.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/engine/userguide/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!docker run hello-world"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE                 COMMAND                  CREATED              STATUS                          PORTS                    NAMES\r\n",
      "58c1a1d09ee0        hello-world           \"/hello\"                 About a minute ago   Exited (0) About a minute ago                            amazing_ardinghelli\r\n",
      "b0991896976d        mjbright/cherrpyapp   \"python cherrypy_a...\"   16 minutes ago       Up 16 minutes                   0.0.0.0:9191->8080/tcp   quizzical_yonath\r\n",
      "ecf09b28660f        mjbright/flaskapp     \"python flask_app.py\"    19 minutes ago       Up 19 minutes                   0.0.0.0:5000->5000/tcp   cocky_cori\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps -a "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:42:40] REQUEST <<POST /v1.30/containers/58c1a1d09ee00477d56cf2ff80aa627fa7ab0d64e7a8af3146ac12f44bb2c018/start HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Content-Length: 0\r\n",
      "    Content-Type: text/plain\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - monitoring 'docker images'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "REPOSITORY                         TAG                 IMAGE ID            CREATED             SIZE\r\n",
      "mjbright/cherrpyapp                latest              eb2c01b4f1fb        22 hours ago        429MB\r\n",
      "<none>                             <none>              55f831e90de2        22 hours ago        429MB\r\n",
      "<none>                             <none>              e6602881d27b        22 hours ago        429MB\r\n",
      "<none>                             <none>              1d57541b9b7a        22 hours ago        429MB\r\n",
      "<none>                             <none>              977309ea7e05        22 hours ago        424MB\r\n",
      "mjbright/flaskapp                  latest              2d4ffd4a9bbc        22 hours ago        430MB\r\n",
      "<none>                             <none>              12cdb6f03a53        22 hours ago        430MB\r\n",
      "<none>                             <none>              fe8991ce30d2        22 hours ago        430MB\r\n",
      "<none>                             <none>              e0b779c11c92        22 hours ago        430MB\r\n",
      "<none>                             <none>              7a681c19a582        22 hours ago        424MB\r\n",
      "<none>                             <none>              859a5b98bd66        22 hours ago        424MB\r\n",
      "<none>                             <none>              1618df99522d        22 hours ago        424MB\r\n",
      "<none>                             <none>              f9d927d6e26b        22 hours ago        424MB\r\n",
      "<none>                             <none>              608fb355fab9        24 hours ago        429MB\r\n",
      "<none>                             <none>              9909d5c82237        24 hours ago        429MB\r\n",
      "<none>                             <none>              41561fcc23c0        24 hours ago        428MB\r\n",
      "<none>                             <none>              082fb8bc149a        24 hours ago        346MB\r\n",
      "<none>                             <none>              e6fdab049433        24 hours ago        193MB\r\n",
      "<none>                             <none>              6d1763b21b35        25 hours ago        597MB\r\n",
      "<none>                             <none>              2d05bc71ae18        25 hours ago        595MB\r\n",
      "<none>                             <none>              c4b6339dfc77        25 hours ago        513MB\r\n",
      "<none>                             <none>              9ffeefc9deb2        25 hours ago        119MB\r\n",
      "<none>                             <none>              cc347f6d4c45        4 days ago          429MB\r\n",
      "<none>                             <none>              137ad4d2c8e4        4 days ago          429MB\r\n",
      "<none>                             <none>              4fac431a12cd        4 days ago          429MB\r\n",
      "<none>                             <none>              7c8e46b39bec        4 days ago          424MB\r\n",
      "<none>                             <none>              5d003e518260        4 days ago          424MB\r\n",
      "<none>                             <none>              9c35f49a176a        4 days ago          424MB\r\n",
      "<none>                             <none>              16d368958fc4        4 days ago          430MB\r\n",
      "<none>                             <none>              42ef0492732e        4 days ago          430MB\r\n",
      "<none>                             <none>              2c0f0ef7ef07        4 days ago          430MB\r\n",
      "<none>                             <none>              1a163aeafd22        4 days ago          424MB\r\n",
      "<none>                             <none>              648c8743816e        4 days ago          424MB\r\n",
      "<none>                             <none>              bd56d01c5b21        4 days ago          424MB\r\n",
      "<none>                             <none>              51c4db36528c        4 days ago          424MB\r\n",
      "<none>                             <none>              6ee2e95b312a        4 days ago          119MB\r\n",
      "<none>                             <none>              80e5cdb184f0        7 days ago          638MB\r\n",
      "projectunik/binary                 latest              317dcab8e887        7 days ago          638MB\r\n",
      "<none>                             <none>              18aa216b0e9d        7 days ago          572MB\r\n",
      "<none>                             <none>              078ed50525a3        7 days ago          572MB\r\n",
      "<none>                             <none>              5a7a41a8c207        7 days ago          572MB\r\n",
      "<none>                             <none>              b5a4254b393a        7 days ago          567MB\r\n",
      "<none>                             <none>              d638e4e2dd6c        7 days ago          567MB\r\n",
      "<none>                             <none>              538048dc41ef        7 days ago          567MB\r\n",
      "<none>                             <none>              63197f704d0e        7 days ago          567MB\r\n",
      "<none>                             <none>              a9c46afdda2d        7 days ago          249MB\r\n",
      "unikernel/mirage                   latest              0333bb61e18a        9 days ago          2.14GB\r\n",
      "functions/webhookstash             latest              ca5deb1869d7        10 days ago         18.7MB\r\n",
      "ubuntu                             16.04               d355ed3537e9        12 days ago         119MB\r\n",
      "ubuntu                             latest              d355ed3537e9        12 days ago         119MB\r\n",
      "hello-world                        latest              1815c82652c0        2 weeks ago         1.84kB\r\n",
      "quay.io/prometheus/alertmanager    latest              822973257511        3 weeks ago         17.7MB\r\n",
      "alexellis2/faas-nodeinfo           latest              05a07204ab7d        3 weeks ago         49.4MB\r\n",
      "centos                             7                   3bee3060bfc8        3 weeks ago         193MB\r\n",
      "functions/gateway                  0.5.5               59f59685a096        8 weeks ago         13.3MB\r\n",
      "astefanutti/decktape               latest              a3b1965c17f1        2 months ago        199MB\r\n",
      "functions/alpine                   health              9c9a219c5c60        2 months ago        15.9MB\r\n",
      "mjbright/docker-demo               20                  35d3bc646d58        3 months ago        7.55MB\r\n",
      "mjbright/docker-demo               21                  b411c41d09ec        3 months ago        89.5MB\r\n",
      "alexellis2/faas-markdownrender     latest              4731497ee865        5 months ago        18.3MB\r\n",
      "alexellis2/faas-dockerhubstats     latest              5e81126a1f59        5 months ago        270MB\r\n",
      "projectunik/image-creator          fa560ac11cecae33    fa560ac11cec        7 months ago        901MB\r\n",
      "projectunik/boot-creator           d0665c8793b16c51    d0665c8793b1        7 months ago        243MB\r\n",
      "projectunik/compilers-rump-go-hw   d1bfbc13602e306d    d1bfbc13602e        7 months ago        2.69GB\r\n",
      "projectunik/qemu-util              6f5922561bbb86e3    6f5922561bbb        9 months ago        474MB\r\n",
      "projectunik/binary                 <none>              542dc074ba29        11 months ago       668MB\r\n",
      "includeos/build                    latest              be92c9c7365f        17 months ago       693MB\r\n"
     ]
    }
   ],
   "source": [
    "!docker images -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the request which was made to the Docker daemon\n",
    "\n",
    "Note that we use /images rather than /containers now.\n",
    "\n",
    "In fact just as the *new* docker cli commands operate on objects, those commands are more aligned with the API way of doing things"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:45:55] REQUEST <<GET /v1.30/images/json?all=1 HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at the response which we got back from the Docker daemon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - direct access\n",
    "\n",
    "Let's double-check by performing some requests directly from the command-line\n",
    "\n",
    "\n",
    "Note that 'docker search' actually queries the 'docker daemon' which then queries your registry (Docker hub by default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;39m[\r\n",
      "  \u001b[1;39m{\r\n",
      "    \u001b[0m\u001b[34;1m\"star_count\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m0\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"is_official\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39mfalse\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"name\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"mjbright/docker-demo\"\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"is_automated\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39mfalse\u001b[0m\u001b[1;39m,\r\n",
      "    \u001b[0m\u001b[34;1m\"description\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\"\u001b[0m\u001b[1;39m\r\n",
      "  \u001b[1;39m}\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m]\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!curl -s -XGET http://localhost:2475/images/search?term=mjbright/docker-demo | jq . "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:46:53] REQUEST <<GET /images/search?term=mjbright/docker-demo HTTP/1.1\r\n",
      "    Host: localhost:2475\r\n",
      "    User-Agent: curl/7.52.1\r\n",
      "    Accept: */*>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:46:54] RESPONSE <<HTTP/1.1 200 OK\r\n",
      "    Api-Version: 1.30\r\n",
      "    Content-Type: application/json\r\n",
      "    Docker-Experimental: false\r\n",
      "    Ostype: linux\r\n",
      "    Server: Docker/17.06.0-ce (linux)\r\n",
      "    Date: Mon, 03 Jul 2017 13:46:54 GMT\r\n",
      "    Content-Length: 107>>\r\n",
      "\r\n",
      "RESPONSE=<<[\r\n",
      "    {\r\n",
      "        \"description\": \"\", \r\n",
      "        \"is_automated\": false, \r\n",
      "        \"is_official\": false, \r\n",
      "        \"name\": \"mjbright/docker-demo\", \r\n",
      "        \"star_count\": 0\r\n",
      "    }\r\n",
      "]>>\r\n"
     ]
    }
   ],
   "source": [
    "cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Docker API - direct access\n",
    "\n",
    "Remember that '*docker run*' actually fetches an image if not present locally.\n",
    "\n",
    "This is the equivalent of '*docker image pull*'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;39m{\r\n",
      "  \u001b[0m\u001b[34;1m\"status\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Pulling from mjbright/docker-demo\"\u001b[0m\u001b[1;39m,\r\n",
      "  \u001b[0m\u001b[34;1m\"id\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"20\"\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m}\u001b[0m\r\n",
      "\u001b[1;39m{\r\n",
      "  \u001b[0m\u001b[34;1m\"status\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Digest: sha256:fdfb575bdfda156d5b06c7994deb770aac0efdacc3c91f285a35da8582772487\"\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m}\u001b[0m\r\n",
      "\u001b[1;39m{\r\n",
      "  \u001b[0m\u001b[34;1m\"status\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"Status: Image is up to date for mjbright/docker-demo:20\"\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!curl -XPOST http://localhost:2475/images/create?fromImage=mjbright/docker-demo:20  2>/dev/null| jq ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's look at recent requests made to our Docker daemon\n",
    "\n",
    "(Note the time that the request took - the Docker daemon checked with DockerHub that it has the latest image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:45:30] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:45:30] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:45:30] REQUEST <<GET /v1.30/images/json HTTP/1.1\r\n",
      "[2017-07-03 15:45:30] REQUEST <<GET /v1.30/images/json HTTP/1.1\r\n",
      "[2017-07-03 15:45:55] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:45:55] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:45:55] REQUEST <<GET /v1.30/images/json?all=1 HTTP/1.1\r\n",
      "[2017-07-03 15:45:55] REQUEST <<GET /v1.30/images/json?all=1 HTTP/1.1\r\n",
      "[2017-07-03 15:46:53] REQUEST <<GET /images/search?term=mjbright/docker-demo HTTP/1.1\r\n",
      "[2017-07-03 15:46:53] REQUEST <<GET /images/search?term=mjbright/docker-demo HTTP/1.1\r\n",
      "[2017-07-03 15:48:54] REQUEST <<POST /images/create?fromImage=mjbright/docker-demo:20 HTTP/1.1\r\n",
      "[2017-07-03 15:48:54] REQUEST <<POST /images/create?fromImage=mjbright/docker-demo:20 HTTP/1.1\r\n"
     ]
    }
   ],
   "source": [
    "!grep -aE \"GET|POST|PUT\" /tmp/dump.txt | tail -12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207\r\n"
     ]
    }
   ],
   "source": [
    "!docker run -d -p 3333:8080 mjbright/docker-demo:20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/start HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: Docker-Client/17.06.0-ce (linux)\r\n",
      "    Content-Length: 0\r\n",
      "    Content-Type: text/plain\r\n",
      "    Accept-Encoding: gzip>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "\r\n",
      "\u001b[1;36m\r\n",
      "                                                .---------.                                          \r\n",
      "                                               .///++++/:.                                          \r\n",
      "                                               .///+++//:.                                          \r\n",
      "                                               .///+++//:.                                          \r\n",
      "                             ``````````````````.:///////:.                       `                  \r\n",
      "                             .-///////:://+++//::///////-.                      .--.                \r\n",
      "                             .:::///:::///+++///:::///:::.                     .:ss+-`              \r\n",
      "                             .:::///:::///+++///:::///:::.                    `.ossss:.             \r\n",
      "                             .:///////:/+++oo++/:///////:.                    .-ssssss:.            \r\n",
      "                    .-:::::::--///////--:::::::--///////--:::::::-.           `-sssssso.........``  \r\n",
      "                    .::////:::://+++///:::///:::///+++//:/::////::.           `.+ssssss/++ooooo+/:.`\r\n",
      "                    .::////:::://+++///:::///:::///+++//:/::////::.            `.+sssooooooooooo/-` \r\n",
      "                    .::////:::://+++///:::///:::///+++//:/::////::.           ``.:osoooooooo+/:-.`  \r\n",
      "            ````````.-:::::::--///////--:::::::-:///////--:::::::-.``````...-:/+ssoo+:-----..``     \r\n",
      "            .-+++oooooooooooooooooooooooooooooooooooooooooooooooooooooooosssssssoooo-.              \r\n",
      "      `     .-ooossssssssssssssssssssssss+ssssssssssssssssossssssssssssssssssoooooo:.        `      \r\n",
      "    ``...```.-+++++/:--/+oosssssssoo++/:-.-:+oosssssso+/:..-:/+oossssssssoo+:--://-.`   ```...`     \r\n",
      " ````````````.-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::-..```````````````  \r\n",
      "             .-////++++++++++++++++++++++++++++++++++++++++++++++++++//////////:.`                  \r\n",
      "             `.:////++++++++++++++++oss+++++++++++++++++++++++++++///////////:-.                    \r\n",
      "              `.:////++++++++++++++os.+h+++++++++++++++++++++++/////////////-.`                     \r\n",
      "               `.://///++++++++++//+soos+++++++++++++++++++///////////////-.`                       \r\n",
      "                `.--------:::://+o/+++++++++++++++++++/////////////////:-.`                         \r\n",
      "                  `-sdmmmmmmmNNNNNmo+++++++++++/////////////////////:-.``                           \r\n",
      "                    `-+hmmmmmmmmmmmms///////////////////////////::-.``                              \r\n",
      "                      ``-+ydmmmmmmmmmds////////////////////::--.``                                  \r\n",
      "                         ``.-/oshdmmmmmmho+////////::::--..```                                      \r\n",
      "                              ````..-:::///---.....````                                             \r\n",
      "\r\n",
      "\u001b[0m\r\n",
      "\r\n",
      "\u001b[1;33mServed from host a3e90dba01c0\u001b[0;0m\r\n"
     ]
    }
   ],
   "source": [
    "!curl http://localhost:3333"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:46:53] REQUEST <<GET /images/search?term=mjbright/docker-demo HTTP/1.1\r\n",
      "[2017-07-03 15:46:53] REQUEST <<GET /images/search?term=mjbright/docker-demo HTTP/1.1\r\n",
      "[2017-07-03 15:48:54] REQUEST <<POST /images/create?fromImage=mjbright/docker-demo:20 HTTP/1.1\r\n",
      "[2017-07-03 15:48:54] REQUEST <<POST /images/create?fromImage=mjbright/docker-demo:20 HTTP/1.1\r\n",
      "[2017-07-03 15:50:28] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:50:28] REQUEST <<GET /_ping HTTP/1.1\r\n",
      "[2017-07-03 15:50:28] REQUEST <<POST /v1.30/containers/create HTTP/1.1\r\n",
      "[2017-07-03 15:50:28] REQUEST <<POST /v1.30/containers/create HTTP/1.1\r\n",
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/wait?condition=next-exit HTTP/1.1\r\n",
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/wait?condition=next-exit HTTP/1.1\r\n",
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/start HTTP/1.1\r\n",
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/start HTTP/1.1\r\n"
     ]
    }
   ],
   "source": [
    "!grep -a REQUEST /tmp/dump.txt | tail -12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;39m{\r\n",
      "  \u001b[0m\u001b[34;1m\"message\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"repository mjbright/nosuchimage not found: does not exist or no pull access\"\u001b[0m\u001b[1;39m\r\n",
      "\u001b[1;39m}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!curl -XPOST http://localhost:2475/images/create?fromImage=mjbright/nosuchimage 2>/dev/null | jq ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/start HTTP/1.1\r\n",
      "[2017-07-03 15:50:29] REQUEST <<POST /v1.30/containers/a3e90dba01c0ce36239a621ea4fb363383c5fcbaf0a8674b5b0180cabf743207/start HTTP/1.1\r\n",
      "[2017-07-03 15:53:06] REQUEST <<POST /images/create?fromImage=mjbright/nosuchimage HTTP/1.1\r\n",
      "[2017-07-03 15:53:06] REQUEST <<POST /images/create?fromImage=mjbright/nosuchimage HTTP/1.1\r\n"
     ]
    }
   ],
   "source": [
    "!grep -a REQUEST /tmp/dump.txt | tail -4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://cloud.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/engine/userguide/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!docker run hello-world"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b358bb76c6d1710156eeafb06e30c621494d631b6d3f2968f874ee36beb7ca18\r\n"
     ]
    }
   ],
   "source": [
    "!docker container create hello-world"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://cloud.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/engine/userguide/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!docker container start -a $(docker ps -ql)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3...3...POST /images/create?fromImage=mjbright/nosuchimage HTTP/1.1\r\n",
      "3.-.3.-.GET /_ping HTTP/1.1\r\n",
      "3.-.3.-.POST /v1.30/containers/create HTTP/1.1\r\n",
      "3.1.3.1.POST /v1.30/containers/efa634d4c0dd4710d1a8b3bbc4d4ec6f88877d8c1c3c7e4a3a7f8cc16ebbc269/attach?stderr=1&stdout=1&stream=1 HTTP/1.1\r\n",
      "3.1.3.1.POST /v1.30/containers/efa634d4c0dd4710d1a8b3bbc4d4ec6f88877d8c1c3c7e4a3a7f8cc16ebbc269/wait?condition=next-exit HTTP/1.1\r\n",
      "3.1.3.1.POST /v1.30/containers/efa634d4c0dd4710d1a8b3bbc4d4ec6f88877d8c1c3c7e4a3a7f8cc16ebbc269/start HTTP/1.1\r\n",
      "3.x.3.x.GET /_ping HTTP/1.1\r\n",
      "3.x.3.x.POST /v1.30/containers/create HTTP/1.1\r\n",
      "3..%3..%GET /_ping HTTP/1.1\r\n",
      "3..&3..&GET /v1.30/containers/json?limit=1 HTTP/1.1\r\n",
      "3../3../GET /_ping HTTP/1.1\r\n",
      "3..03..0GET /v1.30/containers/b358bb76c6d1/json HTTP/1.1\r\n"
     ]
    }
   ],
   "source": [
    "!grep -aE \"GET|POST|PUT\" tcpdump.log | tail -12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"Id\":\"f036cea7c90370329d6ef6116cc09a51b334f6c82489e17970c93ee51053b111\",\"Warnings\":null}\r\n"
     ]
    }
   ],
   "source": [
    "!curl -X POST -H \"User-Agent: Docker-Client/17.05.0-ce (linux)\" \\\n",
    "              -H \"Content-Type: application/json\" \\\n",
    "              -H \"Accept-Encoding: gzip\" \\\n",
    "      http://localhost:2475/containers/create \\\n",
    "    -g --data '{\\\n",
    "        \"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":true,\"AttachStderr\":true,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[],\"Cmd\":null,\"Image\":\"hello-world\",\"Volumes\":{},\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":{},\"HostConfig\":{\"Binds\":null,\"ContainerIDFile\":\"\",\"LogConfig\":{\"Type\":\"\",\"Config\":{}},\"NetworkMode\":\"default\",\"PortBindings\":{},\"RestartPolicy\":{\"Name\":\"no\",\"MaximumRetryCount\":0},\"AutoRemove\":false,\"VolumeDriver\":\"\",\"VolumesFrom\":null,\"CapAdd\":null,\"CapDrop\":null,\"Dns\":[],\"DnsOptions\":[],\"DnsSearch\":[],\"ExtraHosts\":null,\"GroupAdd\":null,\"IpcMode\":\"\",\"Cgroup\":\"\",\"Links\":null,\"OomScoreAdj\":0,\"PidMode\":\"\",\"Privileged\":false,\"PublishAllPorts\":false,\"ReadonlyRootfs\":false,\"SecurityOpt\":null,\"UTSMode\":\"\",\"UsernsMode\":\"\",\"ShmSize\":0,\"ConsoleSize\":[0,0],\"Isolation\":\"\",\"CpuShares\":0,\"Memory\":0,\"NanoCpus\":0,\"CgroupParent\":\"\",\"BlkioWeight\":0,\"BlkioWeightDevice\":null,\"BlkioDeviceReadBps\":null,\"BlkioDeviceWriteBps\":null,\"BlkioDeviceReadIOps\":null,\"BlkioDeviceWriteIOps\":null,\"CpuPeriod\":0,\"CpuQuota\":0,\"CpuRealtimePeriod\":0,\"CpuRealtimeRuntime\":0,\"CpusetCpus\":\"\",\"CpusetMems\":\"\",\"Devices\":[],\"DeviceCgroupRules\":null,\"DiskQuota\":0,\"KernelMemory\":0,\"MemoryReservation\":0,\"MemorySwap\":0,\"MemorySwappiness\":-1,\"OomKillDisable\":false,\"PidsLimit\":0,\"Ulimits\":null,\"CpuCount\":0,\"CpuPercent\":0,\"IOMaximumIOps\":0,\"IOMaximumBandwidth\":0},\"NetworkingConfig\":{\"EndpointsConfig\":{}}}' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://cloud.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/engine/userguide/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!docker start -a $(docker ps -ql)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!grep -aE \"GET|POST|PUT\" tcpdump.log | tail -12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Below**, we can see that our container is still present, but not running\n",
    "\n",
    "(See 2nd line:\n",
    "   \"Status\":\"exited\",\"Running\":false\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -X GET  -H \"User-Agent: Docker-Client/17.05.0-ce (linux)\" \\\n",
    "              -H \"Content-Type: application/json\" \\\n",
    "              -H \"Accept-Encoding: gzip\" \\\n",
    "      http://localhost:2475/v1.29/containers/$(docker ps -ql)/json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker start -a $(docker ps -ql)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!grep -aE \"GET|POST|PUT\" tcpdump.log | tail -12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!set -x; LAST_CONTAINER_ID=$(docker ps -ql --no-trunc); curl -X POST -H \"User-Agent: Docker-Client/17.05.0-ce (linux)\" \\\n",
    "              -H \"Content-Type: application/json\" \\\n",
    "              -H \"Accept-Encoding: gzip\" \\\n",
    "      \"http://localhost:2475/v1.29/containers/$LAST_CONTAINER_ID/attach?stderr=1&stdout=1&stream=1\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Docker-Python \"Swarm Mode\" Lab\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"MACHINE\" />\n",
    "# Docker-machine from Python\n",
    "<a href=\"#TOP\"> TOP </a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Define here if we should use docker-machines as is, or delete existing machines/create new machines\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME        ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER        ERRORS\r\n",
      "swmaster1   -        virtualbox   Running   tcp://192.168.99.100:2376           v17.03.2-ce   \r\n",
      "swnode1     -        virtualbox   Running   tcp://192.168.99.101:2376           v17.03.2-ce   \r\n",
      "swnode2     -        virtualbox   Running   tcp://192.168.99.102:2376           v17.03.2-ce   \r\n"
     ]
    }
   ],
   "source": [
    "!docker-machine ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# CREATE_MACHINES=True\n",
    "# DELETE_MACHINES=True\n",
    "\n",
    "CREATE_MACHINES=False\n",
    "DELETE_MACHINES=False\n",
    "\n",
    "# CREATE_MACHINES=True\n",
    "# DELETE_MACHINES=True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Undo existing cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "press() {\n",
    "    echo -e $*\n",
    "}\n",
    "\n",
    "cmd() {\n",
    "    echo -e $*\n",
    "    eval $*\n",
    "}\n",
    "\n",
    "leave_swarm() {\n",
    "    echo \"==== ${FUNCNAME[0]} $*\"\n",
    "    MASTER=$1; shift\n",
    "    NODE=$1;   shift\n",
    "\n",
    "    docker $(docker-machine config $MASTER) node ls | grep $NODE && {\n",
    "        press \"\\n---- $NODE swarm leave/node rm:\"\n",
    "        cmd docker $(docker-machine config $NODE) swarm leave\n",
    "        while docker $(docker-machine config $MASTER) node ls  | grep $NODE | grep down; do echo Waiting for $NODE to go down; sleep 1; done\n",
    "        cmd docker $(docker-machine config $MASTER) node rm $NODE\n",
    "\n",
    "        echo\n",
    "        cmd docker-machine ls\n",
    "        cmd docker $(docker-machine config $MASTER) node ls\n",
    "    }\n",
    "}\n",
    "\n",
    "#undo_swarm swmaster1 swnode1 swnode2\n",
    "undo_swarm() {\n",
    "    echo \"==== ${FUNCNAME[0]} $*\"\n",
    "    MASTER=$1; shift\n",
    "    NODES=\"$*\"\n",
    "\n",
    "    for NODE in $NODES; do\n",
    "        leave_swarm $MASTER $NODE\n",
    "    done\n",
    "\n",
    "    cmd docker $(docker-machine config $MASTER) swarm leave --force\n",
    "    cmd docker $(docker-machine config $MASTER) node ls\n",
    "}\n",
    "\n",
    "undo_swarm swmaster1 swnode1 swnode2\n",
    "\n",
    "docker $(docker-machine config swmaster1) node ls\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import pprint\n",
    "\n",
    "pp = pprint.PrettyPrinter(indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "# Requirements\n",
    "- Docker >= 1.12 (docker-17.06-ce recommended)\n",
    "- Docker machine\n",
    "- Virtualbox\n",
    "\n",
    "# Setup\n",
    "\n",
    "## Install docker and docker-machine modules\n",
    "\n",
    "## NOTE:\n",
    "#### python-docker-machine installed from https://github.com/mjbright/python-docker-machine\n",
    "\n",
    "Simple modification to depend upon 'docker' not 'docker-py'.\n",
    "\n",
    "Installed using\n",
    "\n",
    "'''python setup.py install'''\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Check that docker is installed, but not docker-py !!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/mjb/usr/anaconda3/bin/pip\n",
      "docker                             2.4.2      \n",
      "docker-pycreds                     0.2.1      \n",
      "python-docker-machine              0.2.2      \n",
      "docker-pycreds                     0.2.1      \n"
     ]
    }
   ],
   "source": [
    "!which pip\n",
    "!pip list --format=columns | grep -i docker\n",
    "\n",
    "!pip list --format=columns | grep docker-py || echo -e \"\\n\\n**** ERROR: you have Docker-py installed (not Docker)!!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "docker module version=2.4.2\n"
     ]
    }
   ],
   "source": [
    "import docker\n",
    "\n",
    "EXPECT_DOCKER_PY_VERSION='2.4.2'\n",
    "\n",
    "print(\"docker module version=\" + docker.version)\n",
    "\n",
    "if docker.version != EXPECT_DOCKER_PY_VERSION:\n",
    "    print(\"\\n\\nERROR: Bad docker.py version: {} != expected {}\".format(docker.version, EXPECT_DOCKER_PY_VERSION))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Using python-docker-machine wrapper module\n",
    "\n",
    "Here we will describe use of this Python module which provides Python access to the 'docker-machine' executable on your machine\n",
    "\n",
    "### Referencing the docker-machine executable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import machine\n",
    "import docker\n",
    "\n",
    "m = machine.Machine(path=\"/usr/local/bin/docker-machine\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<machine.machine.Machine object at 0x7fc7843e7940>\n"
     ]
    }
   ],
   "source": [
    "print(m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Cleanup:  Delete any existing machines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swmaster1',\n",
       "  'ResponseTime': '799ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.100:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode1',\n",
       "  'ResponseTime': '811ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.101:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode2',\n",
       "  'ResponseTime': '799ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.102:2376'},\n",
       " {'Name': ''}]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# If you want to delete existing machines:\n",
    "# !docker-machine rm -f swmaster swnode1 swnode2\n",
    "\n",
    "if DELETE_MACHINES:\n",
    "    for node in m.ls():\n",
    "        if node['Name'] != '':\n",
    "            print(\"Deleting node \" + node['Name'])\n",
    "            m.rm(machine=node['Name'],force=True)\n",
    "            \n",
    "m.ls()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Creating a new machine\n",
    "\n",
    "If creating a machine in some cloud provider you will need to first export some appropriate token or subscription id as\n",
    "an appropriately named environment variable, e.g.\n",
    "- DIGITALOCEAN_ACCESS_TOKEN for DigitalOcean\n",
    "- AZURE_SUBSCRIPTION_ID for Microsoft Azure\n",
    "\n",
    "Refer to https://docs.docker.com/machine/get-started-cloud/ for information,\n",
    "\n",
    "and to https://docs.docker.com/machine/drivers/gce/ (select cloud provider in left-hand sidebar) for specifics.\n",
    "\n",
    "**WARNING**: You may need to \"logon\" to the platform before using the python-docker-machine otherwise the python-docker-machine module will block asking you to validate - but **you will not see that message** !\n",
    "** This will not work with Azure ** because of the need to logon from the local terminal\n",
    "\n",
    "## Examples:\n",
    "\n",
    "### Digital Ocean:  \n",
    "Set DIGITALOCEAN_ACCESS_TOKEN as an environment variable \n",
    "\n",
    "e.g.\n",
    "```\n",
    "     os.environ['DIGITALOCEAN_ACCESS_TOKEN']='MY-TOKEN'\n",
    "     m.create('swmaster1', driver='digitalocean', blocking=True)\n",
    "```\n",
    "\n",
    "### Azure:\n",
    "Set AZURE_SUBSCRIPTION_ID as an environment variable \n",
    "\n",
    "e.g.\n",
    "```\n",
    "    os.environ['AZURE_SUBSCRIPTION_ID']='MY-ID'\n",
    "    m.create('swmaster1', driver='azure', blocking=True)\n",
    "```\n",
    "\n",
    "### Google Compute Engine:\n",
    "Use \"gcloud auth login\"\n",
    "\n",
    "e.g.\n",
    "```\n",
    "     m.create('swmaster1', driver='google', blocking=True)\n",
    "```\n",
    "\n",
    "## Example of local machine creation\n",
    "\n",
    "Below is an example of machine creation on a local machine, to be adapted as above to run in the cloud"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### NOTE: Temporary errors from \"*docker-machine ls*\"\n",
    "\n",
    "From the command-line you can see the progress of machine creation, using\n",
    "    ```$ docker-machine ls```\n",
    "    \n",
    "You may see errors during creation of machines before ssh connectivity is established and before docker host is started.\n",
    "\n",
    "![](images/docker-machine-errors.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swmaster1',\n",
       "  'ResponseTime': '1.125s',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.100:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode1',\n",
       "  'ResponseTime': '1.125s',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.101:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode2',\n",
       "  'ResponseTime': '1.142s',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.102:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'test-machine',\n",
       "  'ResponseTime': '1.124s',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.103:2376'},\n",
       " {'Name': ''}]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.create('test-machine', driver='virtualbox', blocking=True)\n",
    "\n",
    "m.ls()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Deletion of a local machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!docker-machine ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Below is how we would delete our test-machine, but we will keep it for the moment ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# m.rm(machine='test-machine')\n",
    "\n",
    "m.ls()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!docker-machine ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Creating a Docker client connection to the Docker engine\n",
    "\n",
    "During this lab we will use the installed Docker Client to talk to your Docker hosts.\n",
    "\n",
    "Those hosts may be running \n",
    "- locally      controlled via docker-machine\n",
    "- in the cloud controlled via docker-machine\n",
    "- under Docker Toolbox controlled via docker-machine (https://docs.docker.com/machine/get-started/#create-a-machine)\n",
    "- under Docker For Windows, controlled via docker-machine (https://docs.docker.com/docker-for-windows/#docker-settings)\n",
    "- Docker for Mac, where you will only be able to have a single-node swarm (unless bursting out to cloud)\n",
    "\n",
    "This notebook demonstrates the use of docker-machine to control the hosts.\n",
    "\n",
    "#### NOTE: The following command is specific to my environment as I have a special DOCKER_HOST value set:\n",
    "\n",
    "## Connection to a local Docker host\n",
    "\n",
    "#### NOTE: This is for information, in our lab we will obtain a client connection via the python-docker-machine module, see below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tcp://127.0.0.1:2475\r\n"
     ]
    }
   ],
   "source": [
    "# NOTE: specific to my environment\n",
    "#       if you have some special DOCKER_HOST setting, then check this and set in the Python code below\n",
    "\n",
    "! echo $DOCKER_HOST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Inspecting the DockerClient interface\n",
    "\n",
    "Let's look at what objects/methods are available using IPython introspection\n",
    "\n",
    "We can see that we have high-level objects in the same way the \"docker cli client\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Docker-py has 2 interfaces\n",
    "\n",
    "- low-level via Docker.APIClient interface\n",
    "  - This lower-level API corresponds to the actual API requests\n",
    "  - Use it only when you need it's extra features\n",
    "- high-level via Docker.DockerClient interface\n",
    "  - This higher-level API is what you should generally use\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Inspecting the 'low-level' APIClient interface\n",
    "\n",
    "Let's look at what objects/methods are available using IPython introspection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "testlo = docker.APIClient(**m.config(machine='test-machine'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "testlo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "testlo??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## So now let's discard our test-machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swmaster1',\n",
       "  'ResponseTime': '798ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.100:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode1',\n",
       "  'ResponseTime': '811ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.101:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode2',\n",
       "  'ResponseTime': '798ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.102:2376'},\n",
       " {'Name': ''}]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.rm(machine='test-machine')\n",
    "\n",
    "m.ls()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## We will use the 'high-level' DockerClient interface"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just as the docker cli has commands (since Docker 1.13) of the form\n",
    "    - docker images list\n",
    "    - docker containers list\n",
    "\n",
    "The DockerClient interface has a similar object-oriented interface,\n",
    "\n",
    "e.g. to list images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker.client."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### NOTE: You may comment the 'DOCKER_HOST' lines below if you have no special configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import docker, os\n",
    "\n",
    "# Open a client connection to your local Docker host, set DOCKER_HOST if you have special settings to use:\n",
    "\n",
    "# Use your DOCKER_HOST value here:\n",
    "# DOCKER_HOST='tcp://127.0.0.1:2475'\n",
    "\n",
    "#   os.environ['DOCKER_HOST']=DOCKER_HOST\n",
    "# e.g.\n",
    "#   os.environ['DOCKER_HOST']=\"tcp://127.0.0.1:2475\"\n",
    "\n",
    "# auto: use same api version as docker daemon\n",
    "client = docker.from_env(version='auto') #, url='tcp://127.0.0.1:2475')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the api of our client and compare it to the Docker cli client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Usage:\tdocker COMMAND\n",
      "\n",
      "A self-sufficient runtime for containers\n",
      "\n",
      "Options:\n",
      "      --config string      Location of client config files (default\n",
      "                           \"/home/mjb/.docker\")\n",
      "  -D, --debug              Enable debug mode\n",
      "      --help               Print usage\n",
      "  -H, --host list          Daemon socket(s) to connect to\n",
      "  -l, --log-level string   Set the logging level\n",
      "                           (\"debug\"|\"info\"|\"warn\"|\"error\"|\"fatal\")\n",
      "                           (default \"info\")\n",
      "      --tls                Use TLS; implied by --tlsverify\n",
      "      --tlscacert string   Trust certs signed only by this CA (default\n",
      "                           \"/home/mjb/.docker/ca.pem\")\n",
      "      --tlscert string     Path to TLS certificate file (default\n",
      "                           \"/home/mjb/.docker/cert.pem\")\n",
      "      --tlskey string      Path to TLS key file (default\n",
      "                           \"/home/mjb/.docker/key.pem\")\n",
      "      --tlsverify          Use TLS and verify the remote\n",
      "  -v, --version            Print version information and quit\n",
      "\n",
      "Management Commands:\n",
      "  config      Manage Docker configs\n",
      "  container   Manage containers\n",
      "  image       Manage images\n",
      "  network     Manage networks\n",
      "  node        Manage Swarm nodes\n",
      "  plugin      Manage plugins\n",
      "  secret      Manage Docker secrets\n",
      "  service     Manage services\n",
      "  stack       Manage Docker stacks\n",
      "  swarm       Manage Swarm\n",
      "  system      Manage Docker\n",
      "  volume      Manage volumes\n",
      "\n",
      "Commands:\n",
      "  attach      Attach local standard input, output, and error streams to a running container\n",
      "  build       Build an image from a Dockerfile\n",
      "  commit      Create a new image from a container's changes\n",
      "  cp          Copy files/folders between a container and the local filesystem\n",
      "  create      Create a new container\n",
      "  diff        Inspect changes to files or directories on a container's filesystem\n",
      "  events      Get real time events from the server\n",
      "  exec        Run a command in a running container\n",
      "  export      Export a container's filesystem as a tar archive\n",
      "  history     Show the history of an image\n",
      "  images      List images\n",
      "  import      Import the contents from a tarball to create a filesystem image\n",
      "  info        Display system-wide information\n",
      "  inspect     Return low-level information on Docker objects\n",
      "  kill        Kill one or more running containers\n",
      "  load        Load an image from a tar archive or STDIN\n",
      "  login       Log in to a Docker registry\n",
      "  logout      Log out from a Docker registry\n",
      "  logs        Fetch the logs of a container\n",
      "  pause       Pause all processes within one or more containers\n",
      "  port        List port mappings or a specific mapping for the container\n",
      "  ps          List containers\n",
      "  pull        Pull an image or a repository from a registry\n",
      "  push        Push an image or a repository to a registry\n",
      "  rename      Rename a container\n",
      "  restart     Restart one or more containers\n",
      "  rm          Remove one or more containers\n",
      "  rmi         Remove one or more images\n",
      "  run         Run a command in a new container\n",
      "  save        Save one or more images to a tar archive (streamed to STDOUT by default)\n",
      "  search      Search the Docker Hub for images\n",
      "  start       Start one or more stopped containers\n",
      "  stats       Display a live stream of container(s) resource usage statistics\n",
      "  stop        Stop one or more running containers\n",
      "  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE\n",
      "  top         Display the running processes of a container\n",
      "  unpause     Unpause all processes within one or more containers\n",
      "  update      Update configuration of one or more containers\n",
      "  version     Show the Docker version information\n",
      "  wait        Block until one or more containers stop, then print their exit codes\n",
      "\n",
      "Run 'docker COMMAND --help' for more information on a command.\n"
     ]
    }
   ],
   "source": [
    "!docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "REPOSITORY                         TAG                 IMAGE ID            CREATED             SIZE\r\n",
      "mjbright/cherrpyapp                latest              eb2c01b4f1fb        22 hours ago        429MB\r\n",
      "mjbright/flaskapp                  latest              2d4ffd4a9bbc        22 hours ago        430MB\r\n",
      "<none>                             <none>              1618df99522d        22 hours ago        424MB\r\n",
      "<none>                             <none>              f9d927d6e26b        22 hours ago        424MB\r\n",
      "<none>                             <none>              608fb355fab9        24 hours ago        429MB\r\n",
      "<none>                             <none>              9909d5c82237        25 hours ago        429MB\r\n",
      "<none>                             <none>              6d1763b21b35        25 hours ago        597MB\r\n",
      "<none>                             <none>              cc347f6d4c45        4 days ago          429MB\r\n",
      "<none>                             <none>              16d368958fc4        4 days ago          430MB\r\n",
      "projectunik/binary                 latest              317dcab8e887        7 days ago          638MB\r\n",
      "unikernel/mirage                   latest              0333bb61e18a        9 days ago          2.14GB\r\n",
      "functions/webhookstash             latest              ca5deb1869d7        10 days ago         18.7MB\r\n",
      "ubuntu                             16.04               d355ed3537e9        12 days ago         119MB\r\n",
      "ubuntu                             latest              d355ed3537e9        12 days ago         119MB\r\n",
      "hello-world                        latest              1815c82652c0        2 weeks ago         1.84kB\r\n",
      "quay.io/prometheus/alertmanager    latest              822973257511        3 weeks ago         17.7MB\r\n",
      "alexellis2/faas-nodeinfo           latest              05a07204ab7d        3 weeks ago         49.4MB\r\n",
      "centos                             7                   3bee3060bfc8        3 weeks ago         193MB\r\n",
      "functions/gateway                  0.5.5               59f59685a096        8 weeks ago         13.3MB\r\n",
      "astefanutti/decktape               latest              a3b1965c17f1        2 months ago        199MB\r\n",
      "functions/alpine                   health              9c9a219c5c60        2 months ago        15.9MB\r\n",
      "mjbright/docker-demo               20                  35d3bc646d58        3 months ago        7.55MB\r\n",
      "mjbright/docker-demo               21                  b411c41d09ec        3 months ago        89.5MB\r\n",
      "alexellis2/faas-markdownrender     latest              4731497ee865        5 months ago        18.3MB\r\n",
      "alexellis2/faas-dockerhubstats     latest              5e81126a1f59        5 months ago        270MB\r\n",
      "projectunik/image-creator          fa560ac11cecae33    fa560ac11cec        7 months ago        901MB\r\n",
      "projectunik/boot-creator           d0665c8793b16c51    d0665c8793b1        7 months ago        243MB\r\n",
      "projectunik/compilers-rump-go-hw   d1bfbc13602e306d    d1bfbc13602e        7 months ago        2.69GB\r\n",
      "projectunik/qemu-util              6f5922561bbb86e3    6f5922561bbb        9 months ago        474MB\r\n",
      "projectunik/binary                 <none>              542dc074ba29        11 months ago       668MB\r\n",
      "includeos/build                    latest              be92c9c7365f        17 months ago       693MB\r\n"
     ]
    }
   ],
   "source": [
    "!docker images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "client.images.list??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['mjbright/cherrpyapp:latest']\n",
      "['mjbright/flaskapp:latest']\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "['projectunik/binary:latest']\n",
      "['unikernel/mirage:latest']\n",
      "['functions/webhookstash:latest']\n",
      "['ubuntu:16.04', 'ubuntu:latest']\n",
      "['hello-world:latest']\n",
      "['quay.io/prometheus/alertmanager:latest']\n",
      "['alexellis2/faas-nodeinfo:latest']\n",
      "['centos:7']\n",
      "['functions/gateway:0.5.5']\n",
      "['astefanutti/decktape:latest']\n",
      "['functions/alpine:health']\n",
      "['mjbright/docker-demo:20']\n",
      "['mjbright/docker-demo:21']\n",
      "['alexellis2/faas-markdownrender:latest']\n",
      "['alexellis2/faas-dockerhubstats:latest']\n",
      "['projectunik/image-creator:fa560ac11cecae33']\n",
      "['projectunik/boot-creator:d0665c8793b16c51']\n",
      "['projectunik/compilers-rump-go-hw:d1bfbc13602e306d']\n",
      "['projectunik/qemu-util:6f5922561bbb86e3']\n",
      "[]\n",
      "['includeos/build:latest']\n"
     ]
    }
   ],
   "source": [
    "for image in client.images.list():\n",
    "    print(image.tags)\n",
    "    # was image.tag() in docker-py 2.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2017-07-03 16:13:47] REQUEST <<GET /v1.30/images/json?only_ids=0&all=0 HTTP/1.1\r\n",
      "    Host: 127.0.0.1:2475\r\n",
      "    User-Agent: docker-sdk-python/2.4.2\r\n",
      "    Accept-Encoding: gzip, deflate\r\n",
      "    Accept: */*\r\n",
      "    Connection: keep-alive>>\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  {\r\n",
      "  \"Containers\":-1,\r\n",
      "    \"Created\":1499012058,\r\n",
      "    \"Id\":\"sha256:2d4ffd4a9bbc404da4f1d5a1383d639b85757b85b3fdea22e4e769a998d8da07\",\r\n",
      "    \"Labels\":{\r\n",
      "    \"MAINTAINER\":\"Mike.Bright.dockerfile@mjbright.net\"}\r\n",
      "      ,\r\n",
      "    \"ParentId\":\"sha256:12cdb6f03a53ef9dad2c370574a0fc93d4d7818863dd1070f9fdad79b751e37b\",\r\n",
      "    \"RepoDigests\":null,\r\n",
      "    \"RepoTags\":[\r\n",
      "    \"mjbright/flaskapp:latest\"]\r\n",
      "      ,\r\n",
      "    \"SharedSize\":-1,\r\n",
      "    \"Size\":430002280,\r\n",
      "    \"VirtualSize\":430002280}\r\n",
      "    ,\r\n",
      "  {\r\n",
      "  \"Containers\":-1,\r\n",
      "    \"Created\":1499011914,\r\n",
      "    \"Id\":\"sha256:1618df99522d02589f79757aeaa3e648169ecf096c2681b025638a5b38019096\",\r\n",
      "    \"Labels\":{\r\n",
      "    \"MAINTAINER\":\"Mike.Bright.dockerfile@mjbright.net\"}\r\n",
      "      ,\r\n",
      "    \"ParentId\":\"sha256:51c4db36528cd50accc0856031cdec4e26d660be57ef0e63b44d8de8f4accf97\",\r\n",
      "    \"RepoDigests\":[\r\n",
      "    \"<none>@<none>\"]\r\n",
      "      ,\r\n",
      "    \"RepoTags\":[\r\n",
      "    \"<none>:<none>\"]\r\n",
      "      ,\r\n",
      "    \"SharedSize\":-1,\r\n",
      "    \"Size\":424281145,\r\n",
      "    \"VirtualSize\":424281145}\r\n",
      "    ,\r\n",
      "  {\r\n",
      "  \"Containers\":-1,\r\n",
      "    \"Created\":1499011859,\r\n",
      "    \"Id\":\"sha256:f9d927d6e26be3c67de944e236d281ad7d7a92ccae5e7220a5ce5683be05f6da\",\r\n",
      "    \"Labels\":{\r\n",
      "    \"MAINTAINER\":\"Mike.Bright.dockerfile@mjb>>\r\n"
     ]
    }
   ],
   "source": [
    "!tail -40 /tmp/dump.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "client.containers.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                    NAMES\r\n",
      "a3e90dba01c0        mjbright/docker-demo:20   \"/bin/docker-demo ...\"   24 minutes ago      Up 24 minutes       0.0.0.0:3333->8080/tcp   priceless_wescoff\r\n",
      "b0991896976d        mjbright/cherrpyapp       \"python cherrypy_a...\"   About an hour ago   Up About an hour    0.0.0.0:9191->8080/tcp   quizzical_yonath\r\n",
      "ecf09b28660f        mjbright/flaskapp         \"python flask_app.py\"    About an hour ago   Up About an hour    0.0.0.0:5000->5000/tcp   cocky_cori\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Image: 'mjbright/docker-demo:21'>"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note tcpdump/scapy output whilst we pull the image ...\n",
    "#      we see docker engine informing client of progress of pull ...\n",
    "\n",
    "client.images.pull(\"mjbright/docker-demo:21\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'hello Mon Jul 3 14:15:49 UTC 2017\\n'"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.run(\"ubuntu:latest\", command=\"bash -c 'echo hello $(date)'\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Container: f69604ad04>"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.run(\"ubuntu:latest\", command=\"bash -c 'while true; do echo hello  $(date); sleep 1; done'\", detach=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Container: f69604ad04>,\n",
       " <Container: a3e90dba01>,\n",
       " <Container: b099189697>,\n",
       " <Container: ecf09b2866>]"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "container = client.containers.list()[0]\n",
    "# container = client.containers.get(  \"0xid\" )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'f69604ad041d190f0fd64cfd534f4e77e6d855d0f80fd405ac9e75573f1bdd47'"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "container.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'hello Mon Jul 3 14:16:12 UTC 2017\\nhello Mon Jul 3 14:16:13 UTC 2017\\nhello Mon Jul 3 14:16:14 UTC 2017\\nhello Mon Jul 3 14:16:15 UTC 2017\\nhello Mon Jul 3 14:16:16 UTC 2017\\nhello Mon Jul 3 14:16:17 UTC 2017\\nhello Mon Jul 3 14:16:18 UTC 2017\\nhello Mon Jul 3 14:16:19 UTC 2017\\nhello Mon Jul 3 14:16:20 UTC 2017\\nhello Mon Jul 3 14:16:21 UTC 2017\\nhello Mon Jul 3 14:16:23 UTC 2017\\nhello Mon Jul 3 14:16:24 UTC 2017\\nhello Mon Jul 3 14:16:25 UTC 2017\\nhello Mon Jul 3 14:16:26 UTC 2017\\nhello Mon Jul 3 14:16:27 UTC 2017\\nhello Mon Jul 3 14:16:28 UTC 2017\\nhello Mon Jul 3 14:16:29 UTC 2017\\nhello Mon Jul 3 14:16:30 UTC 2017\\nhello Mon Jul 3 14:16:31 UTC 2017\\nhello Mon Jul 3 14:16:32 UTC 2017\\nhello Mon Jul 3 14:16:33 UTC 2017\\nhello Mon Jul 3 14:16:34 UTC 2017\\nhello Mon Jul 3 14:16:35 UTC 2017\\nhello Mon Jul 3 14:16:36 UTC 2017\\nhello Mon Jul 3 14:16:37 UTC 2017\\nhello Mon Jul 3 14:16:38 UTC 2017\\nhello Mon Jul 3 14:16:39 UTC 2017\\nhello Mon Jul 3 14:16:40 UTC 2017\\nhello Mon Jul 3 14:16:41 UTC 2017\\nhello Mon Jul 3 14:16:42 UTC 2017\\nhello Mon Jul 3 14:16:43 UTC 2017\\nhello Mon Jul 3 14:16:44 UTC 2017\\nhello Mon Jul 3 14:16:45 UTC 2017\\nhello Mon Jul 3 14:16:46 UTC 2017\\nhello Mon Jul 3 14:16:47 UTC 2017\\nhello Mon Jul 3 14:16:48 UTC 2017\\nhello Mon Jul 3 14:16:49 UTC 2017\\nhello Mon Jul 3 14:16:50 UTC 2017\\nhello Mon Jul 3 14:16:51 UTC 2017\\nhello Mon Jul 3 14:16:52 UTC 2017\\nhello Mon Jul 3 14:16:53 UTC 2017\\nhello Mon Jul 3 14:16:54 UTC 2017\\nhello Mon Jul 3 14:16:55 UTC 2017\\n'"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "container.logs()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "container.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "container.remove()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Container: a3e90dba01>, <Container: b099189697>, <Container: ecf09b2866>]"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Container: 5bcc2a339a>,\n",
       " <Container: f036cea7c9>,\n",
       " <Container: b358bb76c6>,\n",
       " <Container: efa634d4c0>,\n",
       " <Container: a3e90dba01>,\n",
       " <Container: 58c1a1d09e>,\n",
       " <Container: b099189697>,\n",
       " <Container: ecf09b2866>]"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.list(all=True,limit=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "for container in client.containers.list(all=True):\n",
    "    container.stop()\n",
    "    container.remove()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.containers.list(all=True,limit=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat /tmp/dump.last.req"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat /tmp/dump.last.rsp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Create nodes for your swarm cluster (1 master and 2 worker nodes)\n",
    "\n",
    "We will create 3 nodes using docker-machine/virtualbox.\n",
    "\n",
    "![](images/DockerSwarmArchi.svg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#### If not already created, we can create our machines here:\n",
    "import datetime\n",
    "import time\n",
    "import os\n",
    "\n",
    "def createNodes(nodes):\n",
    "    for node in nodes:\n",
    "        NOW=datetime.datetime.now().strftime(\"%H:%M:%S\")\n",
    "        SECS_BEFORE=int(time.time())\n",
    "        print(\"{} Creating node {} <on {}> ...\".format(NOW, node, MACHINE_DRIVER))\n",
    "        \n",
    "        m.create(node, driver=MACHINE_DRIVER, blocking=False)\n",
    "        \n",
    "        SECS_AFTER=int(time.time())\n",
    "        print(\"... took {} secs\".format(SECS_AFTER-SECS_BEFORE))\n",
    "\n",
    "MACHINE_DRIVER='virtualbox'\n",
    "MASTER_NODES=['swmaster1']\n",
    "WORKER_NODES=['swnode1','swnode2']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "if CREATE_MACHINES:\n",
    "    createNodes(MASTER_NODES + WORKER_NODES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Recovering from possible docker-machine/VirtualBox problems.\n",
    "\n",
    "It's possible that this machine creation fails, e.g. with error as below:\n",
    "\n",
    "```\n",
    "RuntimeError: cmd returned error 1: Error creating machine: Error in driver during machine creation: /usr/bin/VBoxManage modifyvm swmaster1 --natpf1 ssh,tcp,127.0.0.1,40886,,22 failed:\n",
    "VBoxManage: error: The machine 'swmaster1' is already locked for a session (or being unlocked)\n",
    "VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component MachineWrap, interface IMachine, callee nsISupports\n",
    "VBoxManage: error: Context: \"LockMachine(a->session, LockType_Write)\" at line 493 of file VBoxManageModifyVM.cpp\n",
    "```\n",
    "\n",
    "To recover, list current VMs known by VirtualBox using\n",
    "```VBoxManage list vms```\n",
    "\n",
    "For any VMs which are marked as 'inaccessible', delete them using their ID, e.g.\n",
    "```VBoxManage unregistervm 576f28c7-796e-47d4-a1b1-55dada4bba14```\n",
    "\n",
    "If other machines are listed in error seen by\n",
    "```docker-machine ls```\n",
    "\n",
    "try to delete theme with\n",
    "```docker-machine rm -f <machine-name>```\n",
    "\n",
    "If this fails, perform a \"VBoxManage unregistervm\", of the VM id or it's name\n",
    "e.g.\n",
    "```VBoxManage unregistervm 576f28c7-796e-47d4-a1b1-55dada4bba14```\n",
    "or\n",
    "```VBoxManage unregistervm swmaster1```\n",
    "\n",
    "Then try to create machines again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swmaster1',\n",
       "  'ResponseTime': '777ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.100:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode1',\n",
       "  'ResponseTime': '789ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.101:2376'},\n",
       " {'Active': '-',\n",
       "  'ActiveHost': 'false',\n",
       "  'ActiveSwarm': 'false',\n",
       "  'DockerVersion': 'v17.03.2-ce',\n",
       "  'DriverName': 'virtualbox',\n",
       "  'Error': '',\n",
       "  'Name': 'swnode2',\n",
       "  'ResponseTime': '777ms',\n",
       "  'State': 'Running',\n",
       "  'Swarm': '',\n",
       "  'URL': 'tcp://192.168.99.102:2376'},\n",
       " {'Name': ''}]"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.ls()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can obtain the \"config\" parameters (o/p of \"docker-machine config <machine name>\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'base_url': 'https://192.168.99.100:2376',\n",
       " 'tls': <docker.tls.TLSConfig at 0x7fc774e56d30>}"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.config(machine='swmaster1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can obtain the \"env\" parameters (o/p of \"docker-machine env <machine name>\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['export',\n",
       " 'DOCKER_TLS_VERIFY=\"1\"',\n",
       " 'export',\n",
       " 'DOCKER_HOST=\"tcp://192.168.99.100:2376\"',\n",
       " 'export',\n",
       " 'DOCKER_CERT_PATH=\"/home/mjb/.docker/machine/machines/swmaster1\"',\n",
       " 'export',\n",
       " 'DOCKER_MACHINE_NAME=\"swmaster1\"',\n",
       " '#',\n",
       " 'Run',\n",
       " 'this',\n",
       " 'command',\n",
       " 'to',\n",
       " 'configure',\n",
       " 'your',\n",
       " 'shell:',\n",
       " '#',\n",
       " 'eval',\n",
       " '$(/usr/local/bin/docker-machine',\n",
       " 'env',\n",
       " 'swmaster1)']"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.env(machine='swmaster1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"DOCKERPY\" />\n",
    "# docker-py: Controlling Docker from Python\n",
    "<a href=\"#TOP\"> TOP </a>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Creating DockerClients (docker-py)\n",
    "\n",
    "Now let's create docker-py client objects for each of our new nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{   'swmaster1': <docker.client.DockerClient object at 0x7fc774e5a208>,\n",
      "    'swnode1': <docker.client.DockerClient object at 0x7fc774ef5c50>,\n",
      "    'swnode2': <docker.client.DockerClient object at 0x7fc774ed9550>}\n"
     ]
    }
   ],
   "source": [
    "# Create Docker clients for each machine:\n",
    "nodes={}\n",
    "\n",
    "swmaster1 = docker.DockerClient(**m.config(machine='swmaster1'))\n",
    "\n",
    "nodes['swmaster1']=swmaster1\n",
    "swnode1 = docker.DockerClient(**m.config(machine='swnode1'))\n",
    "nodes['swnode1']=swnode1\n",
    "swnode2 = docker.DockerClient(**m.config(machine='swnode2'))\n",
    "nodes['swnode2']=swnode2\n",
    "\n",
    "pp.pprint(nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking that nodes are 'ping'able\n",
      "swmaster1: True\n",
      "swnode1: True\n",
      "swnode2: True\n"
     ]
    }
   ],
   "source": [
    "# Check connectivity to machines:\n",
    "\n",
    "print(\"Checking that nodes are 'ping'able\")\n",
    "\n",
    "for node in nodes.keys():\n",
    "    print(\"{}: {}\".format(node, nodes[node].ping()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ApiVersion': '1.27',\n",
       " 'Arch': 'amd64',\n",
       " 'BuildTime': '2017-06-27T01:35:00.165278597+00:00',\n",
       " 'GitCommit': 'f5ec1e2',\n",
       " 'GoVersion': 'go1.7.5',\n",
       " 'KernelVersion': '4.4.74-boot2docker',\n",
       " 'MinAPIVersion': '1.12',\n",
       " 'Os': 'linux',\n",
       " 'Version': '17.03.2-ce'}"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.version()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.images.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "swmaster1??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# swarm init\n",
    "\n",
    "Now that we have 3 nodes available, we will initialize our Swarm Cluster with 1 master node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've initially created 3 nodes, but we do not yet have a Swarm cluster created.\n",
    "\n",
    "![](images/SwarmNodes_3nodes.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Including these parameters on the docker command line will connect the client to the docker daemon running on node '*swmaster*'.\n",
    "\n",
    "### Networks before creation of swarm cluster\n",
    "Before going further let's look at the networks on your machine.\n",
    "\n",
    "Later, we'll see how a new network is created once the swarm cluster has been created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def show_networks(client):\n",
    "    FORMAT_STR=\"%-14s %-20s %-10s %-14s\"\n",
    "    print(FORMAT_STR % (\"NETWORK ID\", \"NAME\", \"DRIVER\", \"SCOPE\"))\n",
    "\n",
    "    for network in client.networks.list():\n",
    "        #print(dir(network))\n",
    "        print(FORMAT_STR % (network.short_id, network.name, network.attrs['Driver'], network.attrs['Scope']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NETWORK ID     NAME                 DRIVER     SCOPE         \n",
      "75062ff3ec     host                 host       local         \n",
      "49db0f8876     bridge               bridge     local         \n",
      "dd8ce398ea     none                 null       local         \n"
     ]
    }
   ],
   "source": [
    "  \n",
    "# NETWORK ID          NAME                DRIVER              SCOPE\n",
    "# ab973c7b9062        bridge              bridge              local\n",
    "# e953fc8fe3f2        host                host                local\n",
    "# d45eb5315d7c        none                null                local\n",
    "\n",
    "show_networks(swmaster1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Obtain ip addresses of nodes, esp. the 1st master node\n",
    "\n",
    "Now let's identify the ip address of our nodes.\n",
    "\n",
    "We need to note the ip address of the master node, so that when we initialize our swarm cluster, this node will advertise itself on this address so that other nodes can join the swarm cluster.\n",
    "\n",
    "We can see this through config or ip commands of docker-machine as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "swmaster1: 192.168.99.100\n",
      "swnode1: 192.168.99.101\n",
      "swnode2: 192.168.99.102\n"
     ]
    }
   ],
   "source": [
    "for node in nodes.keys():\n",
    "    ip = m.ip(machine=node)\n",
    "    print(\"{}: {}\".format(node, ip))\n",
    "\n",
    "master1ip = m.ip(machine='swmaster1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We could then provide the above ip address as parameter to --advertise-addr when initializing the swarm.\n",
    "\n",
    "However, it is quite convenient to run the above commands embedded, as below, as arguments to the swarm init command.\n",
    "\n",
    "docker-machine config swmaster provides the parameters to use when connecting to the appropriate docker engine for our machine \"swmaster\".\n",
    "\n",
    "The following command will run swarm init to generate the cluster with 'swmaster' as the Master node.\n",
    "You should see output similar to the below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Architecture': 'x86_64',\n",
       " 'BridgeNfIp6tables': True,\n",
       " 'BridgeNfIptables': True,\n",
       " 'CPUSet': True,\n",
       " 'CPUShares': True,\n",
       " 'CgroupDriver': 'cgroupfs',\n",
       " 'ClusterAdvertise': '',\n",
       " 'ClusterStore': '',\n",
       " 'ContainerdCommit': {'Expected': '4ab9917febca54791c5f071a9d1f404867857fcc',\n",
       "  'ID': '4ab9917febca54791c5f071a9d1f404867857fcc'},\n",
       " 'Containers': 0,\n",
       " 'ContainersPaused': 0,\n",
       " 'ContainersRunning': 0,\n",
       " 'ContainersStopped': 0,\n",
       " 'CpuCfsPeriod': True,\n",
       " 'CpuCfsQuota': True,\n",
       " 'Debug': True,\n",
       " 'DefaultRuntime': 'runc',\n",
       " 'DockerRootDir': '/mnt/sda1/var/lib/docker',\n",
       " 'Driver': 'aufs',\n",
       " 'DriverStatus': [['Root Dir', '/mnt/sda1/var/lib/docker/aufs'],\n",
       "  ['Backing Filesystem', 'extfs'],\n",
       "  ['Dirs', '0'],\n",
       "  ['Dirperm1 Supported', 'true']],\n",
       " 'ExperimentalBuild': False,\n",
       " 'HttpProxy': '',\n",
       " 'HttpsProxy': '',\n",
       " 'ID': '2DJW:E6MO:VV3H:RZNM:KE6E:JR6F:F3DM:VZ4U:ZMJX:HIY7:YYLA:Y32J',\n",
       " 'IPv4Forwarding': True,\n",
       " 'Images': 0,\n",
       " 'IndexServerAddress': 'https://index.docker.io/v1/',\n",
       " 'InitBinary': 'docker-init',\n",
       " 'InitCommit': {'Expected': '949e6fa', 'ID': '949e6fa'},\n",
       " 'Isolation': '',\n",
       " 'KernelMemory': True,\n",
       " 'KernelVersion': '4.4.74-boot2docker',\n",
       " 'Labels': ['provider=virtualbox'],\n",
       " 'LiveRestoreEnabled': False,\n",
       " 'LoggingDriver': 'json-file',\n",
       " 'MemTotal': 1044127744,\n",
       " 'MemoryLimit': True,\n",
       " 'NCPU': 1,\n",
       " 'NEventsListener': 0,\n",
       " 'NFd': 14,\n",
       " 'NGoroutines': 22,\n",
       " 'Name': 'swmaster1',\n",
       " 'NoProxy': '',\n",
       " 'OSType': 'linux',\n",
       " 'OomKillDisable': True,\n",
       " 'OperatingSystem': 'Boot2Docker 17.03.2-ce (TCL 7.2); HEAD : c019897 - Thu Jun 29 02:04:54 UTC 2017',\n",
       " 'Plugins': {'Authorization': None,\n",
       "  'Network': ['bridge', 'host', 'macvlan', 'null', 'overlay'],\n",
       "  'Volume': ['local']},\n",
       " 'RegistryConfig': {'IndexConfigs': {'docker.io': {'Mirrors': None,\n",
       "    'Name': 'docker.io',\n",
       "    'Official': True,\n",
       "    'Secure': True}},\n",
       "  'InsecureRegistryCIDRs': ['127.0.0.0/8'],\n",
       "  'Mirrors': []},\n",
       " 'RuncCommit': {'Expected': '54296cf40ad8143b62dbcaa1d90e520a2136ddfe',\n",
       "  'ID': '54296cf40ad8143b62dbcaa1d90e520a2136ddfe'},\n",
       " 'Runtimes': {'runc': {'path': 'docker-runc'}},\n",
       " 'SecurityOptions': ['name=seccomp,profile=default'],\n",
       " 'ServerVersion': '17.03.2-ce',\n",
       " 'SwapLimit': True,\n",
       " 'Swarm': {'Cluster': {'CreatedAt': '0001-01-01T00:00:00Z',\n",
       "   'ID': '',\n",
       "   'Spec': {'CAConfig': {},\n",
       "    'Dispatcher': {},\n",
       "    'EncryptionConfig': {'AutoLockManagers': False},\n",
       "    'Orchestration': {},\n",
       "    'Raft': {'ElectionTick': 0, 'HeartbeatTick': 0},\n",
       "    'TaskDefaults': {}},\n",
       "   'UpdatedAt': '0001-01-01T00:00:00Z',\n",
       "   'Version': {}},\n",
       "  'ControlAvailable': False,\n",
       "  'Error': '',\n",
       "  'LocalNodeState': 'inactive',\n",
       "  'Managers': 0,\n",
       "  'NodeAddr': '',\n",
       "  'NodeID': '',\n",
       "  'Nodes': 0,\n",
       "  'RemoteManagers': None},\n",
       " 'SystemStatus': None,\n",
       " 'SystemTime': '2017-07-03T14:24:27.568108146Z'}"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Cluster': {'CreatedAt': '0001-01-01T00:00:00Z',\n",
       "  'ID': '',\n",
       "  'Spec': {'CAConfig': {},\n",
       "   'Dispatcher': {},\n",
       "   'EncryptionConfig': {'AutoLockManagers': False},\n",
       "   'Orchestration': {},\n",
       "   'Raft': {'ElectionTick': 0, 'HeartbeatTick': 0},\n",
       "   'TaskDefaults': {}},\n",
       "  'UpdatedAt': '0001-01-01T00:00:00Z',\n",
       "  'Version': {}},\n",
       " 'ControlAvailable': False,\n",
       " 'Error': '',\n",
       " 'LocalNodeState': 'inactive',\n",
       " 'Managers': 0,\n",
       " 'NodeAddr': '',\n",
       " 'NodeID': '',\n",
       " 'Nodes': 0,\n",
       " 'RemoteManagers': None}"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()['Swarm']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Before creating the swarm cluster we have 0 nodes, and an inactive cluster\n",
    "\n",
    "![](images/SwarmNodes_3nodes.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "if 'Nodes' in swmaster1.info()['Swarm']:\n",
    "    swmaster1.info()['Swarm']['Nodes']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'inactive'"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()['Swarm']['LocalNodeState']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Note: the following is specific to the Jupyter notebook environment allowing to introspect class/method/variable definitions:\n",
    "\n",
    "swmaster1.swarm.init?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now let's initialize our swarm mode.\n",
    "\n",
    "This make this node a 'Manager' node of the cluster and creates new overlay network to be used by services running in the cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "swmaster1.swarm.init(advertise_addr=master1ip)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "We now have a Swarm cluster with 1 Master node.\n",
    "\n",
    "![](images/SwarmNodes_3nodes_1m.png)\n",
    "\n",
    "Let's investigate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{   'Cluster': {   'CreatedAt': '2017-07-03T14:26:24.577425478Z',\n",
      "                   'ID': '4u7032rg0zsazvjzfcfoccsex',\n",
      "                   'Spec': {   'CAConfig': {'NodeCertExpiry': 7776000000000000},\n",
      "                               'Dispatcher': {'HeartbeatPeriod': 5000000000},\n",
      "                               'EncryptionConfig': {'AutoLockManagers': False},\n",
      "                               'Name': 'default',\n",
      "                               'Orchestration': {   'TaskHistoryRetentionLimit': 5},\n",
      "                               'Raft': {   'ElectionTick': 3,\n",
      "                                           'HeartbeatTick': 1,\n",
      "                                           'KeepOldSnapshots': 0,\n",
      "                                           'LogEntriesForSlowFollowers': 500,\n",
      "                                           'SnapshotInterval': 10000},\n",
      "                               'TaskDefaults': {}},\n",
      "                   'UpdatedAt': '2017-07-03T14:26:25.201614598Z',\n",
      "                   'Version': {'Index': 10}},\n",
      "    'ControlAvailable': True,\n",
      "    'Error': '',\n",
      "    'LocalNodeState': 'active',\n",
      "    'Managers': 1,\n",
      "    'NodeAddr': '192.168.99.100',\n",
      "    'NodeID': 'x3f5shhedzq0ff0hqcq5xxole',\n",
      "    'Nodes': 1,\n",
      "    'RemoteManagers': [   {   'Addr': '192.168.99.100:2377',\n",
      "                              'NodeID': 'x3f5shhedzq0ff0hqcq5xxole'}]}\n"
     ]
    }
   ],
   "source": [
    "swarm_info = swmaster1.swarm.client.info()\n",
    "\n",
    "#pp.pprint(swarm_info)\n",
    "\n",
    "pp.pprint(swarm_info['Swarm'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### After creating the swarm cluster we have 1 nodes, and an active cluster\n",
    "\n",
    "\n",
    "![](images/SwarmNodes_3nodes_1m.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()['Swarm']['Nodes']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Addr': '192.168.99.100:2377', 'NodeID': 'x3f5shhedzq0ff0hqcq5xxole'}]"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()['Swarm']['RemoteManagers']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "A docker info should now show \"Swarm: active\" as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'active'"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.info()['Swarm']['LocalNodeState']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Networks created in swarm mode\n",
    "\n",
    "If we look at the networks we should now see new networks such as '*ingress*' an overlay network and docker_gwbridge for the swarm cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NETWORK ID     NAME                 DRIVER     SCOPE         \n",
      "jsux5dng3b     ingress              overlay    swarm         \n",
      "7ebbf5e663     docker_gwbridge      bridge     local         \n",
      "75062ff3ec     host                 host       local         \n",
      "49db0f8876     bridge               bridge     local         \n",
      "dd8ce398ea     none                 null       local         \n"
     ]
    }
   ],
   "source": [
    "show_networks(swmaster1)\n",
    "#show_networks(swmaster2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### We can obtain the join tokens allowing to join new nodes as manager or worker nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{   'Manager': 'SWMTKN-1-5wapsokhrncxkao84y7jj5pir3iz6vgf2fpfybwk2kq3xelovi-9i9h7zzmw3vm7zpgbq59avx3q',\n",
      "    'Worker': 'SWMTKN-1-5wapsokhrncxkao84y7jj5pir3iz6vgf2fpfybwk2kq3xelovi-a3qxt8jedveu1y1vjkx89feki'}\n"
     ]
    }
   ],
   "source": [
    "join_tokens = swmaster1.swarm.attrs['JoinTokens']\n",
    "\n",
    "pp.pprint(join_tokens)\n",
    "\n",
    "manager_token = join_tokens['Manager']\n",
    "worker_token = join_tokens['Worker']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# swarm join\n",
    "\n",
    "Now we wish to join Master and Worker nodes to our swarm cluster, to do this we need to obtain the token generated during the \"swarm init\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "print(manager_token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "pp.pprint(swmaster1.info()['Swarm'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "print(master1ip)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Add 2nd, 2rd master nodes if present"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "#if swmaster2 != None:\n",
    "#    swmaster2.swarm.join(join_token=manager_token, remote_addrs=[master1ip+':2377'],\n",
    "#                  listen_addr='0.0.0.0:5000', advertise_addr='eth0:5000')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "#if swmaster3 != None:\n",
    "#    swmaster3.swarm.join(join_token=manager_token, remote_addrs=[master1ip+':2377'],\n",
    "#                  listen_addr='0.0.0.0:5000', advertise_addr='eth0:5000')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) node list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID             HOSTNAME   STATE      AVAILABILITY   MANAGER STATUS\n",
      "x3f5shhedzq0   swmaster1  ready      active         leader    \n"
     ]
    }
   ],
   "source": [
    "def show_nodes(client):\n",
    "    FORMAT_STR=\"%-14s %-10s %-10s %-14s %-10s\"\n",
    "    print(FORMAT_STR % (\"ID\", \"HOSTNAME\", \"STATE\", \"AVAILABILITY\", \"MANAGER STATUS\"))\n",
    "    \n",
    "    for node in client.nodes.list():\n",
    "        #pp.pprint(node.attrs)\n",
    "        nodeSwarmStatus=''\n",
    "        state = node.attrs['Status']['State']\n",
    "        if 'ManagerStatus' in node.attrs:\n",
    "            if 'Leader' in node.attrs['ManagerStatus'] and node.attrs['ManagerStatus']['Leader']:\n",
    "                nodeSwarmStatus='leader'\n",
    "            elif 'Reachability' in node.attrs['ManagerStatus']:\n",
    "                nodeSwarmStatus=node.attrs['ManagerStatus']['Reachability']\n",
    "        print(FORMAT_STR % (node.id[:12], node.attrs['Description']['Hostname'], state, node.attrs['Spec']['Availability'], nodeSwarmStatus))\n",
    "\n",
    "        \n",
    "show_nodes(swmaster1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Add worker nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SWMTKN-1-5wapsokhrncxkao84y7jj5pir3iz6vgf2fpfybwk2kq3xelovi-a3qxt8jedveu1y1vjkx89feki\n"
     ]
    }
   ],
   "source": [
    "print(worker_token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now we can use this token to join nodes as a worker to this cluster\n",
    "\n",
    "Note: we could also join nodes as Master, but we have only 3 nodes available.\n",
    "\n",
    "Let's join swnode1 as a worker node"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "json_nodes = swmaster1.nodes.list()\n",
    "\n",
    "#print(type(json_nodes))\n",
    "\n",
    "for node in json_nodes:\n",
    "    pp.pprint(\"node {}.attrs=\".format(node.attrs['Description']['Hostname']))\n",
    "    pp.pprint(node.attrs)\n",
    "    print()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID             HOSTNAME   STATE      AVAILABILITY   MANAGER STATUS\n",
      "x3f5shhedzq0   swmaster1  ready      active         leader    \n"
     ]
    }
   ],
   "source": [
    "show_nodes(swmaster1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Node: x3f5shhedz>]"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.nodes.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Note: the following is specific to the Jupyter notebook environment allowing to introspect class/method/variable definitions:\n",
    "\n",
    "swnode1.swarm.join?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swnode1.swarm.join(join_token=worker_token, remote_addrs=[master1ip+':2377'],\n",
    "                  listen_addr='0.0.0.0:5000', advertise_addr='eth0:5000')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have 1 Master node and 1 Worker node\n",
    "\n",
    "![](images/SwarmNodes_3nodes_1m_1w.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Node: ihnkg8j6yk>, <Node: x3f5shhedz>]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.nodes.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "!docker-machine ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS\r\n",
      "ihnkg8j6yk0049nj6xhtuwjog     swnode1             Ready               Active              \r\n",
      "x3f5shhedzq0ff0hqcq5xxole *   swmaster1           Ready               Active              Leader\r\n"
     ]
    }
   ],
   "source": [
    "!docker $(docker-machine config swmaster1) node ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID             HOSTNAME   STATE      AVAILABILITY   MANAGER STATUS\n",
      "ihnkg8j6yk00   swnode1    ready      active                   \n",
      "x3f5shhedzq0   swmaster1  ready      active         leader    \n"
     ]
    }
   ],
   "source": [
    "show_nodes(swmaster1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swnode2.swarm.join(join_token=worker_token, remote_addrs=[master1ip+':2377'],\n",
    "                  listen_addr='0.0.0.0:5000', advertise_addr='eth0:5000')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have 1 Master node and 2 Worker nodes\n",
    "\n",
    "![](images/SwarmNodes_3nodes_1m_2w.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID             HOSTNAME   STATE      AVAILABILITY   MANAGER STATUS\n",
      "ihnkg8j6yk00   swnode1    ready      active                   \n",
      "me1v63cowcn0   swnode2    ready      active                   \n",
      "x3f5shhedzq0   swmaster1  ready      active         leader    \n"
     ]
    }
   ],
   "source": [
    "show_nodes(swmaster1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS\r\n",
      "ihnkg8j6yk0049nj6xhtuwjog     swnode1             Ready               Active              \r\n",
      "me1v63cowcn0djv0lu3zh8noe     swnode2             Ready               Active              \r\n",
      "x3f5shhedzq0ff0hqcq5xxole *   swmaster1           Ready               Active              Leader\r\n"
     ]
    }
   ],
   "source": [
    "!docker $(docker-machine config swmaster1) node ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# start service\n",
    "\n",
    "First we check for any running services - there should be none in our newly initialized cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swmaster1.services.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now we will create a new service based on the docker image mariolet/docker-demo\n",
    "\n",
    "We will expose this service on port 8080\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "swmaster1.services.create??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "docker.types.ContainerSpec?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Create a service\n",
    "\n",
    "Now let us use the Client API to create a new service\n",
    "\n",
    "Note how we create ContainerSpec and TaskTemplate types wich we pass to the Client.Services.create() method\n",
    "\n",
    "Here is a first version, which launches an alpine image which runs for 60 seconds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "IMAGE = 'ubuntu'\n",
    "\n",
    "container_spec = docker.types.ContainerSpec(\n",
    "    IMAGE, ['echo', 'hello']\n",
    ")\n",
    "\n",
    "\"\"\"\n",
    "endpoint_spec = docker.types.EndpointSpec(ports={\n",
    "    12357: (1990, 'udp'),cv\n",
    "    12562: (678,),\n",
    "    53243: 8080,\n",
    "})\n",
    "\"\"\"\n",
    "endpoint_spec = docker.types.EndpointSpec(ports={\n",
    "    4040: 8080,\n",
    "})\n",
    "\n",
    "task_tmpl = docker.types.TaskTemplate(container_spec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "IMAGE = 'alpine'\n",
    "\n",
    "container_spec = docker.types.ContainerSpec(\n",
    "    IMAGE, ['echo', 'hello']\n",
    ")\n",
    "\n",
    "\"\"\"\n",
    "endpoint_spec = docker.types.EndpointSpec(ports={\n",
    "    12357: (1990, 'udp'),cv\n",
    "    12562: (678,),\n",
    "    53243: 8080,\n",
    "})\n",
    "\"\"\"\n",
    "endpoint_spec = docker.types.EndpointSpec(ports={\n",
    "    4040: 8080,\n",
    "})\n",
    "\n",
    "task_tmpl = docker.types.TaskTemplate(container_spec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "swmaster1.services.create??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#swmaster1.services.create(task_tmpl, name='docker-demo', endpoint_spec=endpoint_spec)\n",
    "\n",
    "service = swmaster1.services.create(\n",
    "            # create arguments\n",
    "            name='alpine-test',\n",
    "            labels={'foo': 'bar'},\n",
    "            # ContainerSpec arguments\n",
    "            image=IMAGE,\n",
    "            command=\"sleep 20\",\n",
    "            container_labels={'container': 'label'}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<Service: p93mgwtydb>]\n",
      "Sleeping ...\r\n"
     ]
    }
   ],
   "source": [
    "pp.pprint( swmaster1.services.list() )\n",
    "\n",
    "!echo \"Sleeping ...\"; sleep 30\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "after 20 secs ...\n",
    "\n",
    "We might expect that after 1 minute, the service is no longer running.\n",
    "\n",
    "But in fact adter 1 minute although the container running the sleep command stops the service is still running and recognizing that the container exited it will restart a new container (\"*task*\") for this service\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<Service: p93mgwtydb>]\n"
     ]
    }
   ],
   "source": [
    "pp.pprint( swmaster1.services.list() )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Service=<Service: p93mgwtydb>\n",
      "\n",
      "Service attributes=\n",
      "{   'CreatedAt': '2017-07-03T14:34:10.535070457Z',\n",
      "    'Endpoint': {'Spec': {}},\n",
      "    'ID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Spec': {   'Labels': {'foo': 'bar'},\n",
      "                'Mode': {'Replicated': {'Replicas': 1}},\n",
      "                'Name': 'ubuntu-test',\n",
      "                'TaskTemplate': {   'ContainerSpec': {   'Command': [   'sleep',\n",
      "                                                                        '20'],\n",
      "                                                         'Image': 'ubuntu',\n",
      "                                                         'Labels': {   'container': 'label'}},\n",
      "                                    'ForceUpdate': 0}},\n",
      "    'UpdateStatus': {   'CompletedAt': '0001-01-01T00:00:00Z',\n",
      "                        'StartedAt': '0001-01-01T00:00:00Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:34:10.535070457Z',\n",
      "    'Version': {'Index': 21}}\n",
      "\n",
      "Tasks associated with the service:\n",
      "\n",
      "---- Task:\n",
      "{   'CreatedAt': '2017-07-03T14:35:06.278461781Z',\n",
      "    'DesiredState': 'shutdown',\n",
      "    'ID': 'edvrjeahvqylhby2fw96vsz5i',\n",
      "    'NodeID': 'x3f5shhedzq0ff0hqcq5xxole',\n",
      "    'ServiceID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Slot': 1,\n",
      "    'Spec': {   'ContainerSpec': {   'Command': ['sleep', '20'],\n",
      "                                     'Image': 'ubuntu',\n",
      "                                     'Labels': {'container': 'label'}},\n",
      "                'ForceUpdate': 0},\n",
      "    'Status': {   'ContainerStatus': {},\n",
      "                  'Err': 'No such image: ubuntu:latest',\n",
      "                  'Message': 'preparing',\n",
      "                  'PortStatus': {},\n",
      "                  'State': 'rejected',\n",
      "                  'Timestamp': '2017-07-03T14:35:21.4353765Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:35:21.735768518Z',\n",
      "    'Version': {'Index': 64}}\n",
      "\n",
      "---- Task:\n",
      "{   'CreatedAt': '2017-07-03T14:35:21.535440378Z',\n",
      "    'DesiredState': 'shutdown',\n",
      "    'ID': 'jbdgz0lcwy8m82m25mlxdkm2e',\n",
      "    'NodeID': 'ihnkg8j6yk0049nj6xhtuwjog',\n",
      "    'ServiceID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Slot': 1,\n",
      "    'Spec': {   'ContainerSpec': {   'Command': ['sleep', '20'],\n",
      "                                     'Image': 'ubuntu',\n",
      "                                     'Labels': {'container': 'label'}},\n",
      "                'ForceUpdate': 0},\n",
      "    'Status': {   'ContainerStatus': {},\n",
      "                  'Err': 'No such image: ubuntu:latest',\n",
      "                  'Message': 'preparing',\n",
      "                  'PortStatus': {},\n",
      "                  'State': 'rejected',\n",
      "                  'Timestamp': '2017-07-03T14:35:36.724512928Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:35:36.900381133Z',\n",
      "    'Version': {'Index': 73}}\n",
      "\n",
      "---- Task:\n",
      "{   'CreatedAt': '2017-07-03T14:35:01.252152802Z',\n",
      "    'DesiredState': 'shutdown',\n",
      "    'ID': 'to8x052gsx9kbgohh53pshdwv',\n",
      "    'NodeID': 'x3f5shhedzq0ff0hqcq5xxole',\n",
      "    'ServiceID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Slot': 1,\n",
      "    'Spec': {   'ContainerSpec': {   'Command': ['sleep', '20'],\n",
      "                                     'Image': 'ubuntu',\n",
      "                                     'Labels': {'container': 'label'}},\n",
      "                'ForceUpdate': 0},\n",
      "    'Status': {   'ContainerStatus': {},\n",
      "                  'Err': 'No such image: ubuntu:latest',\n",
      "                  'Message': 'preparing',\n",
      "                  'PortStatus': {},\n",
      "                  'State': 'rejected',\n",
      "                  'Timestamp': '2017-07-03T14:35:06.22409663Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:35:06.480505471Z',\n",
      "    'Version': {'Index': 56}}\n",
      "\n",
      "---- Task:\n",
      "{   'CreatedAt': '2017-07-03T14:34:46.059852681Z',\n",
      "    'DesiredState': 'shutdown',\n",
      "    'ID': 'u01sdzv2o5zdjo4b80qkseg8y',\n",
      "    'NodeID': 'x3f5shhedzq0ff0hqcq5xxole',\n",
      "    'ServiceID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Slot': 1,\n",
      "    'Spec': {   'ContainerSpec': {   'Command': ['sleep', '20'],\n",
      "                                     'Image': 'ubuntu',\n",
      "                                     'Labels': {'container': 'label'}},\n",
      "                'ForceUpdate': 0},\n",
      "    'Status': {   'ContainerStatus': {},\n",
      "                  'Err': 'No such image: ubuntu:latest',\n",
      "                  'Message': 'preparing',\n",
      "                  'PortStatus': {},\n",
      "                  'State': 'rejected',\n",
      "                  'Timestamp': '2017-07-03T14:35:01.222114262Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:35:01.455088804Z',\n",
      "    'Version': {'Index': 50}}\n",
      "\n",
      "---- Task:\n",
      "{   'CreatedAt': '2017-07-03T14:35:36.699067532Z',\n",
      "    'DesiredState': 'ready',\n",
      "    'ID': 'u23pvdgar7rnxrghbz6mi6k3l',\n",
      "    'NodeID': 'ihnkg8j6yk0049nj6xhtuwjog',\n",
      "    'ServiceID': 'p93mgwtydbrp260d9nprdburu',\n",
      "    'Slot': 1,\n",
      "    'Spec': {   'ContainerSpec': {   'Command': ['sleep', '20'],\n",
      "                                     'Image': 'ubuntu',\n",
      "                                     'Labels': {'container': 'label'}},\n",
      "                'ForceUpdate': 0},\n",
      "    'Status': {   'ContainerStatus': {},\n",
      "                  'Message': 'preparing',\n",
      "                  'PortStatus': {},\n",
      "                  'State': 'preparing',\n",
      "                  'Timestamp': '2017-07-03T14:35:36.88582371Z'},\n",
      "    'UpdatedAt': '2017-07-03T14:35:36.900269956Z',\n",
      "    'Version': {'Index': 73}}\n"
     ]
    }
   ],
   "source": [
    "service = swmaster1.services.list()[0]\n",
    "print(\"Service=\" + str(service) )\n",
    "service_id = service.id\n",
    "\n",
    "print(\"\\nService attributes=\")\n",
    "pp.pprint((service.attrs) )\n",
    "\n",
    "print(\"\\nTasks associated with the service:\")\n",
    "for task in service.tasks():\n",
    "    print(\"\\n---- Task:\")\n",
    "    pp.pprint(task)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's remove this service, before creating other services"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.remove()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Running containers on master node:\" + str(swmaster1.containers.list()) )\n",
    "\n",
    "print(\"Running services on master node:\" + str(swmaster1.services.list()))\n",
    "\n",
    "#swmaster1.services.create(image=IMAGE, name='docker-demo', labels={'label1': 'label2'}, command=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Launching \"*docker-demo*\" as a service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "IMAGE = 'mjbright/docker-demo:20'\n",
    "\n",
    "container_spec = docker.types.ContainerSpec(\n",
    "#    IMAGE, ['echo', 'hello']\n",
    "    IMAGE, ['/bin/docker-demo', '-listen=:8080']\n",
    ")\n",
    "\n",
    "endpoint_spec = docker.types.EndpointSpec(ports={\n",
    "    3030: 8080,\n",
    "})\n",
    "\n",
    "#task_tmpl = docker.types.TaskTemplate(container_spec)\n",
    "\n",
    "docker_demo_service = swmaster1.services.create(\n",
    "            # create arguments\n",
    "            name='docker-demo',\n",
    "            labels={'label1': 'value1'},\n",
    "            # ContainerSpec arguments\n",
    "            image=IMAGE,\n",
    "            #command=\"sleep 60\",\n",
    "            container_labels={'container': 'label'},\n",
    "            endpoint_spec=endpoint_spec\n",
    "        )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#BUT PORTS NOT SHOWING:\n",
    "    \n",
    "#!. /home/mjb/.docker.rc; docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#!. /home/mjb/.docker.rc; docker $(docker-machine config swmaster1) service create --replicas 1 --name docker-demo-via-cli -p 9090:8080 mariolet/docker-demo:20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#!. /home/mjb/.docker.rc; docker $(docker-machine config swmaster1) service ps docker-demo-via-cli\n",
    "#!. /home/mjb/.docker.rc; docker $(docker-machine config swmaster1) service rm docker-demo-via-cli"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we list services again and we should see our newly added docker-demo service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "swmaster1.services.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def showServices(master):\n",
    "    for service in master.services.list():\n",
    "        print(\"{}: {}\".format(service, service.name))\n",
    "\n",
    "showServices(swmaster1)\n",
    "\n",
    "service = swmaster1.services.list()[0]\n",
    "service_id = service.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#swmaster1.services.get(service_id=service_id).remove()\n",
    "#showServices(swmaster1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#!docker $(docker-machine config swmaster1) service rm docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and we can look at the service as seen by the cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and we can look at the service on the individual cluster nodes.\n",
    "\n",
    "Of course as we set replicas to 1 there is only 1 replica of the service for the moment, running on just 1 node of our cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "service = swmaster1.services.list()[0]\n",
    "\n",
    "service.tasks()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"tasks(containers) running on node swmaster1:\" + str(swmaster1.containers.list()) )\n",
    "print(\"tasks(containers) running on node swnode1:\" + str(swnode1.containers.list()) )\n",
    "print(\"tasks(containers) running on node swnode2:\" + str(swnode2.containers.list()) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swnode1) ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swnode2) ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accessing the service\n",
    "\n",
    "As we are working remotely we need to create an ssh tunnel through to the swarm cluster to access our service, exposing the port 8080 on your local machine.\n",
    "\n",
    "We have the ip address of the swarm master node which we obtained from docker-machine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "master1ip = m.ip(machine='swmaster1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then open your web browser at the page http://<master1ip>:8080 (replace <master1ip>) and you should see a lovely blue whale, as below:\n",
    "\n",
    "![](images/docker.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we can obtain a textual representation on the command-line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -O - http://$master1ip:3030"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!wget -O - http://192.168.99.102:3030"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scaling the service\n",
    "\n",
    "Now we can scale the service to 3 replicas:\n",
    "\n",
    "From the command line using:\n",
    "#### docker $(docker-machine config swmaster1) service scale docker-demo=3\n",
    "\n",
    "or using the update method of the service object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "service.update??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "service_mode = docker.types.ServiceMode('replicated', 3)\n",
    "print(\"Update service using:\" + str(service_mode))\n",
    "\n",
    "service.update(name=service.name, mode=service_mode, endpoint_spec=endpoint_spec, command=\"/bin/docker-demo\")\n",
    "#service.update(name=service.name, mode=service_mode, command=\"sleep 600\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#service.remove()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "service.attrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"tasks(containers) running on node swmaster1:\" + str(swmaster1.containers.list()) )\n",
    "print(\"tasks(containers) running on node swnode1:\" + str(swnode1.containers.list()) )\n",
    "print(\"tasks(containers) running on node swnode2:\" + str(swnode2.containers.list()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rolling update\n",
    "\n",
    "Now we will see how we can perform a rolling update.\n",
    "\n",
    "We initially deployed version 20 of the service, now we will upgrade our whole cluster to version 20 \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service update --detach=true --image mjbright/docker-demo:21 docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!docker $(docker-machine config swmaster1) service update --detach=false --image mjbright/docker-demo:21 docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#image=\"mjbright/docker-demo:21\"\n",
    "\n",
    "####container_spec = docker.types.ContainerSpec(image, ['echo', 'hello'])\n",
    "        \n",
    "####task_tmpl = docker.types.TaskTemplate(container_spec)\n",
    " \n",
    "#print(\"Update service using:\" + str(image))\n",
    "\n",
    "####service.update(name=service.name, task_template=task_tmpl, command=\"sleep 600\")\n",
    "#service.update(name=service.name, image=image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verifying the service has been updated\n",
    "\n",
    "Then open your web browser at the page http://localhost:8080 and you should now see a lovely **red** whale.\n",
    "\n",
    "\n",
    "![](images/docker_red.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -O - http://$master1ip:3030"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Leaving the Swarm Cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_nodes(swmaster1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "swmaster1.nodes.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for node in swmaster1.nodes.list():\n",
    "    #print(node)\n",
    "    hostname=node.attrs['Description']['Hostname']\n",
    "    print(str(node) + \" ==> \" + hostname + \" Spec: \" + str(node.attrs['Spec']))\n",
    "    #pp.pprint(node.attrs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for node in swmaster1.nodes.list():\n",
    "    #print(node)\n",
    "    hostname=node.attrs['Description']['Hostname']\n",
    "    print(str(node) + \" ==> \" + hostname)\n",
    "    pp.pprint(node.attrs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "swmaster1.nodes.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "swnode1.swarm.leave()\n",
    "swnode2.swarm.leave()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO: remove nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "node=swmaster1.nodes.get(\"ya6bpopa0m\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "node.attrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "node.remove?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for node in swmaster1.nodes.list():\n",
    "    hostname=node.attrs['Description']['Hostname']\n",
    "    print(str(node) + \" ==> \" + hostname)\n",
    "    #if hostname == 'swnode2':\n",
    "    #    #ret=swnode2.swarm.leave()\n",
    "    #    print(ret)\n",
    "    #if hostname == 'swnode1':\n",
    "        #ret=swnode1.swarm.leave()\n",
    "        #swmaster1.swarm.\n",
    "        #ret=swmaster1.swarm.leave(id=node.id)\n",
    "        #print(ret)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "swmaster1.swarm.leave(force=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#swnode2.swarm.leave(id=swnode2.node_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "#swnode2.swarm.leave??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for node in swmaster1.nodes.list():\n",
    "    pp.pprint(node.attrs['Status'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TODO:  drain a node\n",
    "\n",
    "We can drain a node effectively placing it in 'maintenance mode'.\n",
    "\n",
    "Draining a node means that it no longer has running tasks on it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) node ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's drain swnode1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker $(docker-machine config swmaster1) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) node update --availability drain swnode1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and now we see that all services on swnode1 are shutdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) service ps docker-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TODO:  remove a service\n",
    "\n",
    "Now let's cleanup by removing our service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) service rm docker-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that the service is no longer running:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) service ps docker-demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "docker $(docker-machine config swmaster) ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploying a stack\n",
    "\n",
    "**NOTE**: The stack concept exists in the docker client but not yet in the API, so we cannot do this from Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cd /home/mjb/z/bin/DEMOS_TUTORIALS/2017-May-PyconUS/docker-cli/alexellis.faas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker swarm init --advertise-addr 10.9.0.10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!grep -a REQUEST /tmp/dump.txt | tail -6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker stack deploy func --compose-file docker-compose.yml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that there is no \"stack\" operation in the API requests, but many lower level operations initiated by the Docker client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!grep -a REQUEST /tmp/dump.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Listing services we see that may are marked as 0/1 REPLICAS.\n",
    "\n",
    "This means that they have not yet been instanciated - probably we are still waiting for the appropriate images to be downloaded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker service ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eventually all services are available as here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker service ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker service logs func_gateway"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker stack ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker stack rm func"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!grep -a REQUEST /tmp/dump.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# TODO: working with networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# TODO: working with volumes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"OTHERPY\" />\n",
    "# Other Python  modules for Docker\n",
    "<a href=\"#TOP\"> TOP </a>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name=\"ANSIBLE\" />\n",
    "# Ansible Container\n",
    "<a href=\"#TOP\"> TOP </a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}