particle-swarm-optimization/particle-swarm-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Particle Swarm Optimization Tutorial\n",
    "\n",
    "This is a tutorial to implement a particle swarm optimization (PSO) algorithm using continuous numerical data. There are a few ways to approach an implementation of PSO but the most basic involves particles in a two-dimensional space--what this tutorial will implement. Feel free to distribute, edit, make changes, improve, etc. this tutorial. If you publish this tutorial elsewhere, please let me know and link back.\n",
    "\n",
    "## Particle Swarms\n",
    "\n",
    "PSO is a type of artificial intelligence algorithm which is inspired by behavior of flocks of birds. PSO falls broadly under the spectrum of swarm intelligence algorithms which include others like bee colony optimization, ant colony optimization, and so on.\n",
    "\n",
    "The goal of PSO is to optimize a set of solutions to a problem based on some fitness function. By encoding solutions as particles or point in `n`-space with a value for position and velocity, the swarm particles can reposition themselves to be closer to an optimization problem optima based on the fitness of the swarm and individual particles within the swarm. Given an initial random population (swarm) of solutions (particles) to the problem, members of the swarm will eventually converge to a global optima provided parameters are appropriate.\n",
    "\n",
    "To perform PSO, you need to know a few things things:\n",
    "\n",
    "- what is the search space?\n",
    "- what is the fitness function?\n",
    "- what are the PSO parameters?\n",
    "\n",
    "The search space being the possible space in which the solution lies and the fitness function being a heuristic to gauge how fit a solution is. There are three PSO parameters of import: inertial weight (denoted by $\\omega$), which is the resistance to a change in particle velocity; the cognitive coefficient (denoted by $c_1$), which allows for global exploration and local exploitation; and the social coefficient (denoted by $c_2$), which is for global exploitation and local exploration. Exploration means searching within a specific area and exploitation is focusing on promising areas as found from exploration.\n",
    "\n",
    "PSO is commonly referred to as a metaheuristic, or a procedure which generates a solution to an optimization problem. It's not guaranteed to be optimal, but it is a more \"informed\" search than random. Given the search space and fitness function, the algorithms generates generally good solution. Metaheuristics can be used problem agnostic and can be applied to almost anything. One interesting way to use PSO is to train a neural network.\n",
    "\n",
    "Just as a note on terminology, I say \"solution\", \"candidate\", \"particle\" interchangeably: they mean the same thing and is just a possible solution to the problem being optimized. The most proper term would be \"particle\".\n",
    "\n",
    "## Pseudocode\n",
    "\n",
    "A typical PSO algorithm may follow the below pseudocode:\n",
    "\n",
    "```\n",
    "procedure pso() is:\n",
    "  initialize random swarm s of particles\n",
    "  for each epoch do:\n",
    "    for each particle p in swarm s do:\n",
    "      for each dimension d in search space do:\n",
    "        new_vel, new_pos = [], []\n",
    "        new_vel[d] := w * p.vel\n",
    "        new_vel[d] += c1 * r1 * (p.best_pos[d] - p.pos[d])\n",
    "        new_vel[d] += c2 * r2 * (s.best_pos[d] - p.pos[d])\n",
    "        new_pos[d] := p.pos[d] + new_vel[d]\n",
    "      endfor\n",
    "      p.vel = new_vel\n",
    "      p.pos = new_pos\n",
    "    endfor\n",
    "    for each particle p in swarm s do:\n",
    "      if fitness(p.pos) < p.fit do:\n",
    "        p.fit := fitness(p.pos)\n",
    "      endif\n",
    "    endfor\n",
    "    update swarm best\n",
    "    update particle bests\n",
    "  endfor\n",
    "endprocedure\n",
    "```\n",
    "\n",
    "## Before you Begin\n",
    "\n",
    "This tutorial uses Python with one dependency which needs to be met:\n",
    "\n",
    "- Python 3.6\n",
    "- GNU/Linux (or WSL, LXSS, etc)\n",
    "- X11 or alternative\n",
    "- `matplotlib`\n",
    "\n",
    "You can install `matplotlib` using `pip` although it's not entirely necessary to have `matplotlib` as it's only for illustrative purposes for this tutorial. It can be omitted if desired.\n",
    "\n",
    "## Defining the Search Space and Fitness Function\n",
    "\n",
    "This tutorial will use a common optimization test problem, the Schwefel Function, which is defined as:\n",
    "\n",
    "$$f(x) = 418.9829 \\times d - \\sum_{i=1}^d (x_i \\times \\sin{\\sqrt{|x_i|}})$$\n",
    "\n",
    "And can be visualized in two dimensions as below:\n",
    "\n",
    "<img src=\"./images/schwefel.png\"/>\n",
    "\n",
    "This function can be applied to any dimensionality and has a global minimum of $f(x^*) = 0$ at $x^* = (420.9687, ..., 420.9687)$. For this tutorial, I'll be using a dimensionality of two to best visualize on a 2D plot. We can see this in a map:\n",
    "\n",
    "<img src=\"./images/schwefel-map.png\"/>\n",
    "\n",
    "Note the darkest blue region is around the global optima. This function was chosen specifically because it has many local minimum, something PSO excels at escaping. The `gnuplot` code to generate both can of these images can be found in `/code`.\n",
    "\n",
    "We have the fitness function, what about the search space? The interval in which our particles will reside is `[-500, 500]`. We'll use this information to also generate a swarm later. We wish to initialize a random swarm within these bounds as this is our search space. Particles in our swarm will be described as arrays of length `d`, where `d` is the dimensionality of the problem--in this case, two.\n",
    "\n",
    "We'll define the search space and fitness function now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sin, sqrt\n",
    "\n",
    "SEARCH_MIN = -500 # minimum in search space\n",
    "SEARCH_MAX = 500 # maximum in search space\n",
    "DIMENSIONS = 2 # dimensionality of the problem\n",
    "\n",
    "def schwefel(particle):\n",
    "    \"\"\"PSO fitness function.\n",
    "    Uses Schwefel function in d dimensions.\n",
    "    \n",
    "    Parameters:\n",
    "        particle : a particle in d-space.\n",
    "    \n",
    "    Returns:\n",
    "        The fitness of that particle.\n",
    "    \"\"\"\n",
    "    summ = 0\n",
    "    for d in particle:\n",
    "        summ += d * sin(sqrt(abs(d)))\n",
    "    return (len(particle) * 418.9829) - summ"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just to verify the function is working as intended, we'll generate a \"perfect particle\" with the known optimal position and evaluate its fitness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.545567497236334e-05\n"
     ]
    }
   ],
   "source": [
    "OPTIMAL_PARTICLE = [420.9687 for _ in range(DIMENSIONS)]\n",
    "print(schwefel(OPTIMAL_PARTICLE))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is not exact but $2.55\\times10^{-5}$ is arbitrarily close to the optima $0$, close enough for our purposes. We know our global minimum is around this point at the very least. We've initialized this as a global variable because soon we'll want to see where the minimum is in our plots.\n",
    "\n",
    "## Particle Representation\n",
    "\n",
    "Particles with be represented as a position in each dimension as well as a velocity in each dimension. To do this, we'll make a Particle class to store this information. We also need a way to store a particle's previous best position, its current fitness, and best fitness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Particle:\n",
    "    \"\"\"Particle class.\n",
    "    Containzerizes a position, velocity.\n",
    "    \n",
    "    Attributes:\n",
    "        pos : the position in n-space.\n",
    "        best_pos : the best position this particle has had.\n",
    "        fit : the fitness of the particle.\n",
    "        best_fit : the best fitness this particle has had.\n",
    "        vel : the velocity in n-space.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, pos, vel):\n",
    "        \"\"\"Particle constructor.\"\"\"\n",
    "        # initialize position and velocity as params\n",
    "        self.pos, self.vel = pos, vel\n",
    "        # find fitness at instantiation\n",
    "        self.fit = schwefel(self.pos)\n",
    "        # best so far is just initial\n",
    "        self.best_pos, self.best_fit = self.pos, self.fit\n",
    "    \n",
    "    def set_pos(self, pos):\n",
    "        \"\"\"Position mutator method.\"\"\"\n",
    "        self.pos = pos\n",
    "        # get fitness of new position if the new\n",
    "        # position is still within the search\n",
    "        # space. Otherwise don't update fitness\n",
    "        if not any(p < SEARCH_MIN for p in pos)\\\n",
    "        and not any(p > SEARCH_MAX for p in pos):\n",
    "            # get the fitness\n",
    "            fitness = schwefel(self.pos)\n",
    "            # if better\n",
    "            if fitness < self.best_fit:\n",
    "                self.fit = fitness\n",
    "                # update best fitness\n",
    "                self.best_fit = self.fit\n",
    "                # update best position\n",
    "                self.best_pos = self.pos\n",
    "    \n",
    "    def set_vel(self, vel):\n",
    "        \"\"\"Velocity mutator method.\"\"\"\n",
    "        self.vel = vel\n",
    "    \n",
    "    def get_pos(self):\n",
    "        \"\"\"Position accessor method.\"\"\"\n",
    "        return self.pos\n",
    "    \n",
    "    def get_vel(self):\n",
    "        \"\"\"Velocity accessor method.\"\"\"\n",
    "        return self.vel\n",
    "    \n",
    "    def get_best_pos(self):\n",
    "        \"\"\"Best position accessor method.\"\"\"\n",
    "        return self.best_pos\n",
    "\n",
    "    def get_fit(self):\n",
    "        \"\"\"Fitness accessor method.\"\"\"\n",
    "        return self.fit\n",
    "    \n",
    "    def get_best_fit(self):\n",
    "        \"\"\"Best fitness accessor method.\"\"\"\n",
    "        return self.best_fit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we now how how particles are represented, we can make a swarm of them.\n",
    "\n",
    "## Swarm Initialization\n",
    "\n",
    "First, we need to initialize an initial swarm. Since we know the search space boundaries, our swarm consists of particles in `2`-space where each axis is randomized within the bounds of the search space. Recall a particle is just an encoding of some solution to the problem: our solutions are in two dimensions so particles are also in two dimensions. Additionally, we'll start with a swarm size of `200`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "SWARM_SIZE = 200 # how large our swarm is\n",
    "\n",
    "def initialize_swarm(size, dim, min, max):\n",
    "    \"\"\"Swarm initialization function.\n",
    "    \n",
    "    Parameters:\n",
    "        size : the size of our swarm.\n",
    "        dim : the dimensionality of the problem.\n",
    "        min : the minimum in a dimension for the search space.\n",
    "        max : the maximum in a dimension for the search space.\n",
    "    \n",
    "    Returns:\n",
    "        A random swarm of that many Particles within the bounds of the space.\n",
    "    \"\"\"\n",
    "    swarm = [] # swarm stored as list\n",
    "    for _ in range(size): # for the size of the swarm\n",
    "        # position is random in every dimension\n",
    "        position = [random.uniform(min, max) for _ in range(dim)]\n",
    "        # velocity is initially zero in every dimension\n",
    "        velocity = [0 for _ in range(dim)]\n",
    "        # init a particle\n",
    "        particle = Particle(position, velocity)\n",
    "        swarm.append(particle) # add to swarm\n",
    "    return swarm\n",
    "\n",
    "SWARM = initialize_swarm(SWARM_SIZE, DIMENSIONS, SEARCH_MIN, SEARCH_MAX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To show our initialization is correct, we'll print out a few particles and examine their positions and fitnesses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[132.1122095054567, -345.59243830350596] = 865.3464638614537\n",
      "[-323.7980888932579, -102.02603846134105] = 529.7653393000207\n",
      "[106.30303354933301, -184.30057325146873] = 1076.2821135584174\n",
      "[135.7042670571525, 169.92525193245137] = 868.8671987423096\n",
      "[61.69931779065473, -420.4552134169011] = 1195.2161254015616\n"
     ]
    }
   ],
   "source": [
    "random.shuffle(SWARM)\n",
    "for i in range(5):\n",
    "    print(SWARM[i].get_pos(), \"=\", SWARM[i].get_fit())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can see our particles are in two dimensions and each axis is within the bounds of `[-500, 500]`. We'll be using `matplotlib` to plot these to see where particles lay within the search space. We've chosen to do this in two dimensions as it is difficult to visualize on a 2D plot otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEGCAYAAACZ0MnKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO3df6xcZ33n8ff3egKs6pT8uNd25Ouss90sUmCRFjs0dsjlElgSfohUaLsxuyqwRYp2CSsqkGiA1ULb/VHgDygqLZuWlWBFlR90uyCgmwZii8hOwA4QuiENcQNJbHxt3wZoSUXJvfPdP+aMPb6ZuffMzHPOeZ7nfF7SyDNnxjPPOfec5/s83+c555i7IyIiUsZM0wUQEZF0KGiIiEhpChoiIlKagoaIiJSmoCEiIqV1mi5AlWZnZ33nzp1NF6N1nnnmGc4777ymiyEiE3rggQeW3X1u2HtZB42dO3dy5MiRoe91u11Onz7Nli1bMLOaS9acKte72+3yile8gkOHDrF3717279/PzIw6syKpMbPHR73XyiO6X7nNz8+zuLhIt9ttuki1qHq9T58+zaFDh1hZWeHQoUOcPn066PdLPbrdLidPnkTncMkwrQwaba3cTp8+zcGDB1lZWeHgwYPB13vLli3s3buXTqfD3r172bJlS9Dvl+q1tUEl5bUyaLS1cpudnWXz5s0AbN68mdnZ2aDfb2bs37+fY8eOceDAgVal/XLR1gaVlNfKoNHWym15eZmnn34agKeffprl5eXgvzEzM8PWrVtbs03rVEfaqK0NKimvlUED2lm5qUJIV11po5gaVBpbiVNrg0YbxVQhyHiGpY2qqlRjaFBpbCVeChot00SFoBbj9Nb2EmdnZ7OuVDW2Ei8FDVnXtBW+WoxhrO0lLi8vZ12ptjGVmkrjSkFDRgpR4avFONwkFcRgLzH3SrVtqdSUGlcKGjJSiAo/t8otRGswRAXRhko1hrGVuqTUuFLQKCGVbmNoISr8nCq3UK3BUBVEmyrV3KXUuMo+aMTQKkxVqAo/l8otVGWfUgUh9UipcWU5t57PP/98/9nPfjbVxfNOnjzJ/Pw8KysrdDodjh07xtatWysorcTO3VlcXDxzQcZpDu62XjAzRvpbPJuZPeDuu4e9l3VP46c//alahRJMyNZgLr2v1CWRSXjiCbj33nOX3Xtvb3kDsg4amzdvnrqyT6nbKNVrqrJv67ha1ZIYgH78cXjjG+HAgd7rAwd6rx8fefXySmUdNF7wghdE1SrUgS+TSKI1nKgkMgnXXAN33gm/+qvwwQ/2/r3zzt7yBmQdNIBoUgChD3wFoPZIojWcqGQyCYuLcPPN8Fu/1ft3cbGxojQeNMxsk5l9y8y+WLy+zMy+bmZHzex2M3tOsfy5xeujxfs7myz3uEIe+Gp5titojmoN57INml6PJMaXDhyAT3wCPvCB3r/9VFUDGg8awDuBhwdefwj4qLv/U+BHwNuK5W8DflQs/2jxuWSE7Aa3veXZtqA5rDWcyzbIZT0qde+9Z1NSH/zg2VTV2sHxurh7Yw9gHvgqcC3wRcCAZaBTvL8HuKt4fhewp3jeKT5n633/rl27PCarq6u+tLTk3W53qu/pdru+sLDgnU7HFxYWpv6+tUKVsypLS0ve6XQc8E6n40tLS00XqXa5bINc1qNSjz/u/rWvnbvsa1/rLa8IcMRH1KtN9zQ+BrwH6DcvLgZ+7O4rxetjwPbi+XbgSYDi/Z8Unz+Hmd1kZkfM7EhsLfBQ3eAq87AptPySGLysWC7bIJf1qNSllz570Puaa3rLG9Bp5FcBM3s9cMrdHzCzxVDf6+63ArcC7N69O+1k7zr6ASi0Yamv2E5m7AfNNp+Qlcs2yGU92qTJnsbVwBvM7AfAbfRSVL8HXGBm/WA2Dxwvnh8HdgAU7z8f+Js6C9wGqbT8khi83MC0A8A5bAPIZz3aorGg4e7vdfd5d98J7APucfd/C+wH/lXxsbcAny+ef6F4TfH+PT7p0SYjhUh9NT0bJgUppAFTNbj/aV8Mr+kxjWF+E3iXmR2lN2bxqWL5p4CLi+XvAm5pqHzZm6blp8qwnLbPgKvK2v1vcXFR+2JgWV+wcPfu3X7kyJGmi9EqusBjOR7w4ody1tr9z91ZXV3Vvjim1l6wUOqXyphI05I5Ezkxa/c/7YvhqachwelS09Kk/jiGmTE3N8fy8rL2xSHWO07V05BaaTaMNG3fvn3s2LGDa6+9lrm5Oe2La0wz9qigISJZ0SSDjU2zjRQ0RCQrGlfb2DTbqLEzwkVEqqCzzDc2zTZST0NEspPiuFrdJyJOuo0UNEREGpbSSbEKGiIiDUtp8F5BQ5KkawpJTlIavFfQkOSk1JUXKSOlKwQoaEhyUurK10m9r7SlMnivoFEhHcTVSKkrXxf1vqQuChoVGfcgDhVgUgtUk5Q3pa58XdT7klFC1wkKGhUZ5yAO1UpMrbVZtrzDdvpUuvJ1Ue9LhqmkTnD3bB+7du3ypnS7XV9YWPBOp+MLCwve7XZHfnZpack7nY4D3ul0fGlpaaLfDPU9dSlT3tXV1XO24+rqagMlTcPq6qovLS2tu6/l+Nsy2qR1AnDER9Sr6mlUZJwUSqhWYkytzTJd4jLlVdqlvH7vy91rTVGm1sMtK7VU7zCV1AmjokkOj0l7Gk20mtb+5qRliKHFN07vYKPyjtNjk2Z6Zqn1cMsoux1jON42MkkZWaen0XjFXuVjkqARQzokhjJMI3QlksKBGYsmKvAcA3vbU6cKGmOIodUUQxmmkWMlkoqmtn1ugb3MdqzzOK17+64XNDSmsUYM4wIxlGEamhLbnKa2fW6z2cpsx7qO09jGjHSP8CFiuMd1DGUQkfXVcZyePHmS+fl5VlZW6HQ6HDt2jK1bt1byW326R/iYYmg1xVAGEVlfHcdpbJkH3blPREpR77cZsd2JUD0NEdlQbHn1tokp86CgIa2Tw0lbddNJltKnoCGVibFyHtVijrGsMYktry7NUdCQSsSazhjWYo61rDHRNGrpU9CIXKot4FjTGcNazLGWNTYx5dWlOQoaEUu5BRxrOmNYiznWsorESCf3RayJk3pCSmmKZkplFamaTu4jzTRP6i3glNIZKZVVpEmtCBqppnk0+Ji3FBsyIq0IGikPdDZ1Y51hVMmFk2pDRqSxoGFmO8xsv5l918weMrN3FssvMrO7zezR4t8Li+VmZh83s6Nm9h0ze0nZ30o9zRNDBRNDGXKSckNG2q3JnsYK8G53vwK4CrjZzK4AbgG+6u6XA18tXgO8Bri8eNwE/GHZH6oizVNnqzuGCiaGMoTWZM8p9YaMtFdjQcPdT7j7N4vnfwc8DGwHbgA+XXzs08CvFM9vAD5T3CPkfuACM7uk7O+FHOisu9UdQwUTsgwhKutpv6PpnpPGqyRZo+7OVOcD2Ak8Afwi8OOB5dZ/DXwReNnAe18Fdq/3vZPeI3wjTdxZL4Y7o4UoQ4hbZIb4jtTvjigyjnGPXWK+c5+ZbQb+FPgNd//bwfeKwo/VlDSzm8zsiJkdqSqF0kTLP4YpoSHKECLNFeI7yv4NY+gViUwjeK96VDSp4wGcB9wFvGtg2SPAJcXzS4BHiuf/A3jTsM+NelTV03CPo+WfohD3sA51H+yN/oax9IokTqnUAZP0qlmnp9FkwDDgM8DH1iz/CHBL8fwW4MPF89cBf178v6uAb2z0G1UGDZlcqDRX1QdsiBRWiO9IpXJqk5QaA5M0stYLGk2mp64Gfg241sy+XTxeC/wu8C/N7FHgVcVrgC8DjwFHgT8C3t5AmSWAEGmuVG6zufY7Zmdnx0pVNT1gH5tYUn0pzSYMPuliVDTJ4aGehkwrZK9oZWVl7NbpuD2VKnslTfd4Ymrdh0qR1mmcvx+R9jQkA7G0/KoSsle0vLw8dut0nN5Olb2SGHo8MbXuU5syHfTvNyqa5PBQT2O4UC3GmFp+KZi0dVr271XlNOIYpiin2LqPxbh/P9TTkL6QLY5xW36590o2MmnrtGxvp8qp4DGcYJpa6z4mIf9+up9Gy4S8R4e7s7i4yKFDh9i7d++6B3I/WPU/u3//fmZm1GYJrR+YzSz4RIEc7jmSwzpMapx11/005IygLY4xWn4x5aNzt2/fPnbs2BF87CGGE0ynEcO4TJNC/f0UNFomdBc/htSJnDVJcG5L2lANlzAUNFqoiRaj8tH1GDc4t6n1rYZLGBrTiESbc60S1jj7Uur3oR+XjrP19bfPtm3bNKYRs3Fbe21JJ8hkxulJtq31nfq4TJUG6yHgBaM+p6ARgZMnT5bOtbYpnSDVU9pQ+gbHfIBfGPU5BY2GdbtdbrzxRlZWVjCzDVt7GsyT0JpqfavHHJfBXifw9KjPKWhUbKMD4/Tp09x3331A7+C9/fbb1z1425ZOaKvcK1T1mOMz2Oukd+uJoRQ0KlTmwBgMAldfffWGg5BKJ+SvDRVqLD3m3IPzuPq9znU/U1NZWqnMgTFJENBgXt5iqVCrFEOPuQ3BuQoKGhUqe2DEFASqbHmpVVdODBVq1WLoMbchOFdBQaNCMRwY48j90tqpqGK/iTFgN91YakNwroJO7pMzqjzRq20nkQ3T1IlluljkaDrZbzhdsFBKqevS2nv27Dlzbf62aLKnpTTMaE33dlKkoCFnVJlO63/3E088gZkFuwprjGmXYZqsuJWGkZAUNOQcVba8ZmZmmJmZCVZ5pjROUrbiriIIpja2JnFT0JBahWz1ppR2KVNxVxkElYaRUBQ0pFYhW72ppV02qrirDoKppPIkbgoaUrtQrd7c0i5VBsGUUnkSt+yDhlpWeZs0AMXY6g4dBAfXMaVUnsQt66DxyCOPqGUlzxJzqztUL2ztOs7OzkaZyosxeMv6sj65z8wcaO3JZDJcG040HLaOc3NzUZ3IppMO49Xak/s2b94cXctKmpfaAPokhq1jbDOolDJLU9Y9jd27d/uXvvSlaFpWEo82XD4i9nV0dxYXF8/0NHKYzJCL9XoanboLU7fc0g4SRpn7BqQu9nXsD/zHHNjk2bJOT6UqxcHBFMs8rjasY91iS5nJxhQ0IhPzzJ5RYilz1fcCGVzHlZUVBRBpJQWNyKQ4OBhDmasOXIPrePDgQRYWFhoPkmWodyShKWhEJsWZPTGUuerANbiOV155JYcPH44+sMfSA5S8ZD97KsWbMMU+62WYpstcx0yc/jrOzc2dc35BrLN+2nA+ilRjvdlTChqSjToDV9NBsgxNaU1bk/tYVif3mdn1ZvaImR01s1uaLo/Eo86ZOCnM+sntgo5tEnNqMamgYWabgE8ArwGuAN5kZlc0WyqReKUQ3OTZYphcMsqGQcPM/qOZXVhHYUp4KXDU3R9z958DtwE3hPwBzTaROrRpP2vTuoYSw+SSUcr0NLYCh83sjiI11GSTZTvw5MDrY8WyM8zsJjM7YmZHxo3OMXcJJR9t2s/atK4hxZxa3DBouPt/Ai4HPgW8FXjUzP6bmf1SxWWbiLvf6u673X333NzcWP835i5haGr9NadN+1mb1jW0WFOLpcY0vFezLBWPFeBC4HNm9uEKyzbMcWDHwOv5YlkQMXcJQ1LrbzyhA2xb9jNo17q2xYZTbs3sncCbgWXgj4H/4+7PmNkM8Ki719bjMLMO8D3glfSCxWHg37j7Q8M+P8mU2xSmUk5r1Pz9Nqz7uKq650ObtnWb1jUX0065vQh4o7tf5+53uvszAO7eBV4fsJwbcvcV4B3AXcDDwB2jAsakYu0ShjSs9ZdL7yN0r6Cq9Eob9rO+jdY1p1RpTusySpkxjQ+4++Mj3ns4fJE2LM+X3f2fufsvuft/rfv3czBskC3V3PPgQVpF4FN6pVq5NFag3nVpNDi5e7aPXbt2uZTT7XZ9YWHBO52OLywseLfbbbpIG1pdXT2nzD/84Q+90+k44J1Ox5eWloL9ztLSUhLbJDVLS0uV/M2aUNe6rN3vV1dXg/8GcMRH1KtJndwn1Yl5it8oa3tHZlZJr6BNqaS65dSTq2tdms4K6NpTkiwfcm0ldx866KrB2Hjl9LepY12G7fehf0sXLJTKNH3Al/n9qmZAiTSl6uMuqwsWSjxiGMQskzpqujsvElqTKVMFDZlYKpVxannzNkzblHQpaMjEUqmMUxrkj6H3JrIejWmU0HTePmbaNmHpbnsSA41pTEEtv/VpOmpYqfTepL0UNDaQSt5e8pBSKi1VbR8zmnb9FTQ2oJaf1E29t+q0PXMQYv01plGC8vYieWj7mFHZ9deYxpTU8hPJQ9szByHWv1NBuUREotQfM2pr5iDE+qunkYG2D+yJjKPtmYNp119BI3FtH9gTkXopaCRucErwwYMHOXXqVNNFEpGMKWiMKbZU0JYtW9izZw/QK9uNN96o3oZIAmKrS8pS0BhDjKkgM+P222+n0+ng7uuegJjqTiqSm6rrkiqPdQWNMcR6dvi2bds2nEYXY8ATaasq65Kqj3UFjTHEOse7zKUnYg14Im1UZV1S9bGuoDGGmK8LtNE0ulgDXh2UlpPYVFmXVH2s6zIiLdLGy6HoVq/SRtMe67qMiADtPKlJaTlpoyqPdQUNyVoTaTmlwyRnChqStbrHoTRLTXKnoNEgtUjrUWdaTumwtNVxTKZ+3CtoNEQt0jzlOEst9UqurDqOyRyOe82eakjbbwaTs5xmqbVp9lkdx2Qqx71mT0Uoxxap9OQ0S61N6bY6jskcjnv1NBqUU4tU8uTuLC4unulpxHZSa2h1HJMpHPfr9TR0574G9VukG0lhJ5M8te1Od2WPydh/o0pKT0Uuh4EzidtGA905pdtkegoagYWYaTL4HW3KKUv91CiRcTUSNMzsI2b2V2b2HTP7MzO7YOC995rZUTN7xMyuG1h+fbHsqJnd0kS5NxLiAFz7HbOzs8kPnEm81CiRcTXV07gbeJG7vxj4HvBeADO7AtgHvBC4HvgDM9tkZpuATwCvAa4A3lR8NiohDsC137G8vBztlXUlfTnM5pF6NRI03P0v3H2leHk/MF88vwG4zd3/wd2/DxwFXlo8jrr7Y+7+c+C24rNRCXEADvsO5ZSlKjFf7l/iFMPsqV8Hbi+eb6cXRPqOFcsAnlyz/JerL9p4Qsw0adtsFWle6rN5pF6VBQ0z+wqwbchb73f3zxefeT+wAnw24O/eBNwEcOmll4b62tJCHIA6iEUkVpUFDXd/1Xrvm9lbgdcDr/SzU42OAzsGPjZfLGOd5Wt/91bgVuid3Dd2wUWk9UKeG5XbCYNNzZ66HngP8AZ3//uBt74A7DOz55rZZcDlwDeAw8DlZnaZmT2H3mD5F+oud1ltucBb7PR3kEmEnIac40UQm5o99fvA+cDdZvZtM/skgLs/BNwBfBf4v8DN7r5aDJq/A7gLeBi4o/hsdDTvPQ76O8ikQk5DrmNKc+3Tpt0928euXbu8bktLS97pdBzwTqfjS0tLtZdB9HeQyXW7XV9YWPBOp+MLCwve7Xaj+K46fwM44iPqVZ0RHpjmvcdBf4d0NZ1WDDENub8OQOVTmuueNq2gEZjmvcdBf4c0rU0rnjhxopHgMc25UWvXAaj8PKs6z+XSpdFFJBqDNykyM2ZmZrj66quTuvlTKjdaWo9uwiQiSRhMK7o7q6urzxrcbTp9tZHcU6MKGtI6sVc6bdZPKz755JNcc801z6p4U5gVl3tqVEFDWiWFSqftZmZm2LZtGwcOHHhWxZvKVXlzvl5c9kGjza3KNq/7KNNWOrFv09jLN45hFW/uqZ8UZB80xmlVVn3A1XlAq0U93DSVTuzbtO7yNRGgck/9JGHUCRw5PF784heXPsFrdXX1nBNkVldXNzr/ZSxVf/9a45zctrq66ktLS5WceBSjSdc35AmDVWzzOk9orHt/lnrR1pP7zjvvvNKtyqpzpXXnYsu2qGNvPVdh0nxzqNRIVdu8ztRNKmMLUoFR0SSHx65du0q36Ko+3X/Y91fdwi/z/brcxnhC/M2q3OZ19RrruDxGylLvvbNOT6Pxir3Kx7jXnqqzEn/mmWf8qquu8k2bNjXavdfBX79ctnnqFWNVckjdrRc0dEZ4A7rdLi972cu47777ANi0aRPHjx9v7KzROq/FLz3a5vnSGeES3OnTpzl8+PCZ11deeWWjUwdznlMeq5mZGebm5jh16hQ5N9zaKPdpwQoaDRjcqfbs2cPBgwezrrBzOncglDZOQGiL3KcFK2jUrNvtcurUKe655x6OHTvGwYMHk7kQ2yRUOQ6n2Ud5y7n3nm9tFaHBCvTaa69lbm6ukp0qppa9Ksfhck9hSFzHYUgKGhOYdGeoowKNrWWvynG4aVMYuVZIuYjtOAxJQWNM0+wMdVSgsbXsc8/vTmPSFEbOFVIuYjsOQ1LQGNM0O0MdFWiMLfuc87tNWG8fVA+kWmW3b4zHYSgKGmOadmeougJVyz5/o/ZB9UCqNc72zfk41Ml9jH+ilU7MktBC7INrTyp74oknmJmZYXZ2luXlZe2vU8rhpL2ydHLfOiZpnSndIiGF2gfX9kBuvPFGtm/fzsUXX5xl76PuVFzOKadxtL6nEar1oN6HTCpkC7a/H7o7O3bsYGVl5cx7ObWO+4H20KFD7N27l/3799dyvlNbjnP1NNYRovUQKpesQcx2CtmC7fdAtm7dyt69e9m0aRPPf/7zg1zOPaZ9s6nZScoyoKvcuk9/tc4Ql7rO4cqYMrkqrhjb/86VlZWpvjvGfTOXKwXHCl3ltlruzuLi4pmu8iSzJdo0yCZpiXXfbEuqqAlKT1UsxPQ6DbLFJ7aUTFNi3TeVKmqGehoRUcspHk0NtMZK+2a7qKeRCLWc4pHzZSAmoX1T+hQ0AlI6Ix+xpmREmqagEUgTl3AoG6QUzMaX82UgRKahoBFI3emMskFK1yOanFIyIs+moBFI3emMskFKuXkRCan1QSNU6qbudEbZIKXcvEh12pj6bTRomNm7zczNbLZ4bWb2cTM7ambfMbOXDHz2LWb2aPF4S4jfD526qTOdUTZIKTdfvzZWJG3U1tRvY0HDzHYArwaeGFj8GuDy4nET8IfFZy8CPgD8MvBS4ANmduG0ZUg9dVM2SCk3X58QFYmCThpSrz8m1WRP46PAe4DBI+MG4DPF5U/uBy4ws0uA64C73f0pd/8RcDdw/bQFUOpGQpu2Ism59ZpbMGxr/dFI0DCzG4Dj7v7gmre2A08OvD5WLBu1fNh332RmR8zsyEYHrFI36Uilwpm2Ism19ZpjMGxr/VFZ0DCzr5jZ/xvyuAF4H/Cfq/hdd7/V3Xe7++65ubkNPx9T6iaVirFuKVU401YkubZecw2GMdUfdaksaLj7q9z9RWsfwGPAZcCDZvYDYB74ppltA44DOwa+Zr5YNmp5NlKqGOuWWoUzTUWSa+u1ymCoxla9ak9PuftfuvsWd9/p7jvppZpe4u5LwBeANxezqK4CfuLuJ4C7gFeb2YXFAPiri2XZSK1irFMsre+6KqdJg07MlWdVwVCNrfrFdp7Gl+n1RI4CfwS8HcDdnwJ+BzhcPH67WJacUQd2LBVjjGJofcdeOcVePqgmlTNuYyvmwJqMUXdnyuFR9s59ddnoDmhV3L1Nwghxd8YqxV6+qoxzB78Y70AYK9a5c19sPY2sbdQqSm1QrU2ttth7grGXryrj9EKVAg5DQaNGOR3YKaRDQoohRbae2MtXlXFuDpXT8dck3bmvZrncAa2J+0bnsu0kjEnurqh9qBzdua9m66VtUktBjVJ3q61tPRvZ2CTpplyOv0mESicraATWlsqt7nSI8tGyltJN5YWsl5SeCqyJtE0buDuLi4tnUhFtytvLaEo3lTNuvaT0VI3U+qlG7AO9bZpJFpM2p5vGEbJeUk+jAmr9tMskA7IidRunXlJPo2Zq/bSLxlskBaHqJQUNkSkpJZkPpRk3pqAhMqXYx1tS0XSFPTjD6OUvfzknTpxQ8BhCQUMkAKUkpxPDVPXBNOO9997LpZdemt20+RCBWUFDRBoXw7hQP824adMmzCy7MapQgVlBQ0QaF8O40GCa8ZprrslujCpUYO4ELpeIyNj6FXbTU9VnZmbYtm1bFGUJrR+Y+1PDJw2GChoiEoX+uFAMYipLKKECc9Yn95nZaeDxpssBzALLTRciEtoWZ2lbnKVtcVYM2+Ifu/vcsDeyDhqxMLMjo86ubBtti7O0Lc7Stjgr9m2hgXARESlNQUNEREpT0KjHrU0XICLaFmdpW5ylbXFW1NtCYxoiIlKaehoiIlKagoaIiJSmoFEBM3u3mbmZzRavzcw+bmZHzew7ZvaSgc++xcweLR5vaa7UYZnZR8zsr4r1/TMzu2DgvfcW2+IRM7tuYPn1xbKjZnZLMyWvXlvWs8/MdpjZfjP7rpk9ZGbvLJZfZGZ3F/v+3WZ2YbF85PGSCzPbZGbfMrMvFq8vM7OvF+t8u5k9p1j+3OL10eL9nU2WG+jde1mPcA9gB3AXvZMKZ4tlrwX+HDDgKuDrxfKLgMeKfy8snl/Y9DoE2g6vBjrF8w8BHyqeXwE8CDwXuAz4a2BT8fhr4J8Azyk+c0XT61HBdmnFeq5Z50uAlxTPzwe+V+wHHwZuKZbfMrCPDD1ecnoA7wL+BPhi8foOYF/x/JPAfyievx34ZPF8H3B702VXTyO8jwLvAQZnGNwAfMZ77gcuMLNLgOuAu939KXf/EXA3cH3tJa6Au/+Fu68UL+8H5ovnNwC3ufs/uPv3gaPAS4vHUXd/zN1/DtxWfDY3bVnPM9z9hLt/s3j+d8DDwHZ66/3p4mOfBn6leD7qeMmCmc0DrwP+uHhtwLXA54qPrN0W/W30OeCV1vDFsBQ0AjKzG4Dj7v7gmre2A08OvD5WLBu1PDe/Tq/lCNoWbVnPoYr0yr8Avg5sdfcTxVtLQP9iT7lvo4/Ra1j2r01+MfDjgUbW4Pqe2RbF+z8pPt8YXbBwTGb2FWDbkLfeD7yPXlqmFdbbFu7++eIz7wdWgM/WWTaJj5ltBv4U+A13/9vBBrO7u5llP//fzF4PnHL3B8xssenyTEJBY0zu/qphy83sn9PL0T9YHAzzwDfN7KXAcXpjHX3zxbLjwOKa5QeCF54zu1QAAAI+SURBVLoio7ZFn5m9FXg98EovkrKM3hasszwn661/tszsPHoB47Pu/r+LxSfN7BJ3P1Gkn04Vy3PeRlcDbzCz1wLPA34R+D16KbhO0ZsYXN/+tjhmZh3g+cDf1F/sAU0PquT6AH7A2YHw13HuwN43iuUXAd+nNwh+YfH8oqbLHmj9rwe+C8ytWf5Czh0If4ze4HCneH4ZZweIX9j0elSwXVqxnmvW2YDPAB9bs/wjnDsQ/uHi+dDjJbcHvQZjfyD8Ts4dCH978fxmzh0Iv6PpcqunUY8v05sRchT4e+DfAbj7U2b2O8Dh4nO/7e5PNVPE4H6fXmC4u+h53e/u/97dHzKzO+gFlBXgZndfBTCzd9CbebYJ+J/u/lAzRa+Ou6+0YT3XuBr4NeAvzezbxbL3Ab8L3GFmb6M32/BfF+8NPV4y95vAbWb2X4BvAZ8qln8K+F9mdhR4il7gaJQuIyIiIqVp9pSIiJSmoCEiIqUpaIiISGkKGiIiUpqChoiIlKagISIipSloiIhIaQoaIjUysyuLe0Q8z8x+obi/xIuaLpdIWTq5T6RmxVm/zwP+EXDM3f97w0USKU1BQ6RmxV3ZDgM/A/b2L6MikgKlp0TqdzGwmd5d7J7XcFlExqKehkjNzOwL9O7Ydxlwibu/o+EiiZSmq9yK1MjM3gw84+5/YmabgENmdq2739N02UTKUE9DRERK05iGiIiUpqAhIiKlKWiIiEhpChoiIlKagoaIiJSmoCEiIqUpaIiISGn/H8mRSsfCwldSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_particles(swarm, opt):\n",
    "    x = [swarm[i].get_pos()[0] for i in range(len(swarm))]\n",
    "    y = [swarm[i].get_pos()[1] for i in range(len(swarm))]\n",
    "    plt.xlabel('x')\n",
    "    plt.ylabel('y')\n",
    "    plt.xlim(SEARCH_MIN, SEARCH_MAX)\n",
    "    plt.ylim(SEARCH_MIN, SEARCH_MAX)\n",
    "    plt.scatter(x, y, c='k', s=5, lw=1)\n",
    "    plt.scatter(opt[0], opt[1], c='r', marker='x', s=30, lw=1)\n",
    "    plt.show()\n",
    "    plt.clf()\n",
    "\n",
    "plot_particles(SWARM, OPTIMAL_PARTICLE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can see particles as a black dot and uniformly distributed within the search space and the optimum as a red 'x'. We'll use this function a few more times to visualize how points slowly converge upon the optimum.\n",
    "\n",
    "## Moving Particles\n",
    "\n",
    "Particles move within the search space based on the PSO parameters: the inertial weight and social and cognitive coefficients. These should be empirically found for each problem and shouldn't be known a priori but there are a few shortcuts that can be used: some parameter choices have been empirically found to work well if not great for most problems.\n",
    "\n",
    "Here we'll use some parameters found to be generally great for most problems:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = 0.729844 # intertial weight\n",
    "C_1 = 1.496180 # cognitive coefficient\n",
    "C_2 = 1.496180 # social coefficient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note `C_1` is equal to `C_2`, giving equivalent import on local exploration/global exploitation and local exploitation/global exploration.\n",
    "\n",
    "To actually move the particles, we need a way to find the swarm best position:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_swarm_best(swarm):\n",
    "    \"\"\"Finds the swarm best fitness and position.\n",
    "    \n",
    "    Parameters:\n",
    "        swarm : the swarm to search.\n",
    "        \n",
    "    Returns:\n",
    "        The swarm best fitness and swarm best position.\n",
    "    \"\"\"\n",
    "    # initially assume the first is the best\n",
    "    best_fit = swarm[0].get_fit()\n",
    "    best_pos = swarm[0].get_pos()\n",
    "    for particle in swarm: # for each particle\n",
    "        # if better fitness found\n",
    "        if particle.get_fit() < best_fit:\n",
    "            # update best fitness and position\n",
    "            best_fit = particle.get_fit()\n",
    "            best_pos = particle.get_pos()\n",
    "    return best_fit, best_pos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can try this out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best position: [397.8972501979873, 453.3622895984888]\n",
      "Best fitness: 192.09115213681207\n"
     ]
    }
   ],
   "source": [
    "def print_best(best):\n",
    "    print(f'Best position: {best[1]}')\n",
    "    print(f'Best fitness: {best[0]}')\n",
    "    \n",
    "print_best(get_swarm_best(SWARM))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall the best fitness is $0$, so we have a ways to go. We'll use this function several times when we update our positions. And about updating the positions, it is simply a function of the swarm best position, each particle's best position, their current positions and velocities, and the PSO parameters as defined previously. The math is below:\n",
    "\n",
    "$$p_i(t+1) = p_i(t) + v_i(t+1)$$\n",
    "\n",
    "Or, for a dimension $i$, the new position in that dimension is a function of its prior position and new velocity. Here $(t+1)$ means the current iterative step and $t$ is the prior. The new velocity is calculated as:\n",
    "\n",
    "$$v_i(t+1) = \\omega v_i(t) + c_1r_1(y(t) - x_i(t)) + c_2r_2(\\hat{y}(t) - x_i(t))$$\n",
    "\n",
    "Where $\\omega$ is the inertial weight, $c_1$ and $c_2$ are the cognitive and social coefficients respectively, $r_1$ and $r_2$ are stochastic multiplicands, and $y$ and $\\hat{y}$ is the particle best position and swarm best position respectively. It's useful to think of it split into three separate terms: a weight term ($w$), a cognitive term ($c$) and a social term ($s$):\n",
    "\n",
    "$$\\begin{align}\n",
    "w &= \\omega v_i(t) \\nonumber \\\\\n",
    "c &= c_1r_1 (y(t) - x_i(t)) \\nonumber \\\\\n",
    "s &= c_2r_2 (\\hat{y}(t) - x_i(t)) \\nonumber \n",
    "\\end{align}$$\n",
    "\n",
    "Since each term in order deals with the inertial weight, cognitive aspect, and social aspect of the velocity update. The new velocity is simply:\n",
    "\n",
    "$$v_i(t+1) = w + c + s$$\n",
    "\n",
    "We'll implement particle movement below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def move_particles(swarm):\n",
    "    \"\"\"Particle movement function.\n",
    "    \n",
    "    Parameters:\n",
    "        swarm : the swarm to move.\n",
    "    \"\"\"\n",
    "    # get swarm bests\n",
    "    swarm_best = get_swarm_best(swarm)\n",
    "    for particle in swarm: # for each particle\n",
    "        # new position and velocity is initially zero\n",
    "        new_pos = [0 for _ in range(DIMENSIONS)]\n",
    "        new_vel = [0 for _ in range(DIMENSIONS)]\n",
    "        for d in range(DIMENSIONS): # for each axis\n",
    "            # the social and cognitive coefficients \n",
    "            # take a stochastic multiplicand\n",
    "            R_1 = random.uniform(0.00, 1.00)\n",
    "            R_2 = random.uniform(0.00, 1.00)\n",
    "            # this is split for readability but the update is based\n",
    "            # on an addition of a weight, cognitive, and social term\n",
    "            weight = W * particle.get_vel()[d]\n",
    "            cognitive = C_1 * R_1\n",
    "            cognitive *= (particle.get_best_pos()[d] - particle.get_pos()[d])\n",
    "            social = C_2 * R_2\n",
    "            social *= (swarm_best[1][d] - particle.get_pos()[d])\n",
    "            # new velocity is simply weight + cognitive + social\n",
    "            new_vel[d] = weight + cognitive + social\n",
    "            # new position is just old position + velocity\n",
    "            new_pos[d] = particle.get_pos()[d] + new_vel[d]\n",
    "        # update particle with new position and velocity\n",
    "        particle.set_pos(new_pos)\n",
    "        particle.set_vel(new_vel)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above function will move particles in a way to reach a better fitness. We can verify that by running it, monitoring the swarm best position and fitness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before movement:\n",
      "Best position: [397.8972501979873, 453.3622895984888]\n",
      "Best fitness: 192.09115213681207\n",
      "\n",
      "After movement:\n",
      "Best position: [428.7136199467444, 437.2562098977089]\n",
      "Best fitness: 40.811954301692595\n"
     ]
    }
   ],
   "source": [
    "print(f'Before movement:')\n",
    "print_best(get_swarm_best(SWARM))\n",
    "\n",
    "move_particles(SWARM)\n",
    "\n",
    "print(f'\\nAfter movement:')\n",
    "print_best(get_swarm_best(SWARM))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can see the swarm best position closes in on the optimal position and the fitness does approach $0$. This is in essence what PSO accomplishes: the algorithm isn't that long or complex and we've pretty much finished. Before I make a main driver for the algorithm, I'll show what the new plot looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEGCAYAAACZ0MnKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAcp0lEQVR4nO3df4xcZ33v8fd3Z/KjwklxurZjZZ0bU+WiGwqiydp1HNY44VdIuaRSq6v0jxJopeiScC9VkWiAShHtvS0F6RZQ06ZpQSoVKHFpq0YRbWrSdSNlk+A1kISEm2YJhezau1k3LQWjFs/Ot3/MM7uz65nZM7Pnx3POfF7SyDNnxrPPOXPmfJ5f54y5OyIiIkmMFV0AEREpD4WGiIgkptAQEZHEFBoiIpKYQkNERBKrF12ALI2Pj/sVV1yx5fc5e/YsTz311Orj173udZx33nlbfl+RtDz33HP84Ac/YNu2bbz61a8uujgj5ezZs5kfD7L+fDeuw4kTJ067+46uL3b3yt6uueYaT0Oz2fRDhw55vV73Q4cOebPZTOV9RdKwuLjo9XrdAa/X6764uFh0kUbCysrKuuPCyspKJn8ny8+31zoAs97juKruqQTMjOnpaebn5zl27BhmVnSRRFbt3LmTgwcPUq/XOXjwIDt37iy6SCNheXmZmZkZGo0GMzMzLC8vp/K+zWaTpaUlPJxDl+XnO8w6KDQSGhsbY9euXQqMCtr4JS0bVWrOlcdnmsXBvNlscv311zMxMcHhw4dpNpuZfr7DrINCQ0Zaty9pGW1WqSl7MA4ir880i4N5r5p/VpXWYdZBoSEjbeOX9Nlnn63cgbUqwZhUVt1G3SQ9mCcN7SK6GgcNJIWGjLTOL+krXvEKXv/61/PGN76RU6dOpRYeRdfy8zyIthW5zrGN8SQJ7fb2AtbV/N09vhZirxHyKtzSmj01ylZWVnxxcbHSM8ZWVlb86aef9lqt5sDqLJU0ZsTkNcOmn7xn/xW1zp37ah77bdK/0W/208rKip88ebLr9ipy36HP7KnCD+xZ3hQaW5PnTjvMlzzNA0P7wFqr1dzMUpveuNkBI691zjP8i5gCnPcBdpC/1yu02+/Ruc/VajU/efKkuxc7lVqhIUPJa6cd5gufxUFiZWXFT506tWmtfJAD8GYHjKLXOQubtWyyCLC8D7CD/r12i+LUqVOr6935Hp23iy66yM+ePVvo+WEKDRlKXjvtMF/4rE942nhQay9rNBpDHew3vl9s65y2XsGQVfBlta/2Wo9B/1639e58j6uvvnpdcDz55JNd/3638MmCQkOGllatsN/7DPOFz7MW1vmFP3DgQCoH7tjXOSt5h/1WXrtZwA3y93qtd/s9FhYW1oVGu4uqW3naXVlTU1Ophe7GdVFoSKGS1C6LHtPop/MLX6vV/Nprr03lwB3zOmclhuBL2tpJM+A2W+9ms+lTU1Neq9V6bpeN3Vm1Wi2V0O22PYCvu0JDilKmbpVuNn7hG41GqQ/cRevs6itiO3bbH7uFcdoBt1ngb/Z8O1jaoXHxxRd7o9HYUpncu2+Pfi0NnachmYtt3vygNp41W6vVdEmZLRgbG2PHjh3ccMMNhZxwuHF/HB8f73oeRZKzpQc5H6XzJLphzmMxM+6//37GxlqH7TNnznD69Omhy9M28PezV5pU4aaWRjzK3q0i6dqs9Zn1/tL5/sO2hIcd1O/2//q+13e+4/7II+6+1vp5Y63mv7B//zmTNYadZKAxDYWGSNT6df3kPbV42G6oYcNm4xjZ008/7adOner9Xo884j4+7j497e7uKw8/7CuXXOLNf/iHVMrTjUJDRKLTqzVR1MmBg7Zshg2b9tjE2NiYX3zxxV6v131qaqr/e01Pt4LjrrvWBUga5emmX2hU+pf7ZHQ0m02Wl5fZuXOnxhoikOTzaPfxb9TuY5+Zmcn9on2QfF9qj3kMut+5O2aGmfH9738fd+exxx7ju9/9LmNjY93f6/BhuOMO+OhH4a67Wo9TKs+g353CB8LNrGZmXzOzB8PjvWb2hJnNmdn9ZnZ+WH5BeDwXnr+iyHJLPEbtKq6x2+rnkefvg2wcOB607MNcsrx9AcmVlZVWzT0MQF966aW93+vYMbj77lZg3H1363EK5Rnqs+rVBMnrBvwa8AXgwfD4CHBLuH8P8N5w/3bgnnD/FuD+zd5b3VOjoexTequmLJ9Ht7GTPMre2Y00NTW1+dndG8Y0VruqwuD4VvRaX2Id0wAmgIeBG4AHAQNOA/Xw/LXAQ+H+Q8C14X49vM76vb9CYzTEcMKYrInt80g6dtK+PMfU1NTAZR90TKTfpWrOeY+O2VOrHnmktXyLen1WMYfGF4FrgMMhNMaBuY7n9wDfCPe/AUx0PPctYLzLe94GzAKzl19++ZY3qvQW0zTamMoicXwe/S477n5ujb/zUjELCwsDBcBWZ3sVeTHKbp9VlKEBvAP4g3A/tdDovKmlkZ2yXHFVymsrwdPtsuP9zgfpnPIK+IEDBxLv02l0acXWpdcvNIocCL8OeKeZ/RNwH60uqk8BrzSz9qyuCWAh3F+gFSKE538c+Oc8Cyxrivg1OBkdWx1M7zXYvHEmVnvgeNeuXezbt291+fHjxxPv02lc8WB8fJzJyUlqtdqWZ4wNc1b4QHqlSZ43Qksj3P9z1g+E3x7u38H6gfAjm72vWhrZia3fumpi6N4p0lZr3gMPNrt7o9HwAwcO9L1oYC9ptIrq9bpfe+21W7qeVFo9AMTYPbWuEOtD41XAV4C5ECAXhOUXhsdz4flXbfa+Co1sjfqBLSvq+tt6pWTY350oYp9Os2sqrfeKPjSyuik0pIxi698uylZ+2rZMoZtmqz2t9+oXGtZ6vpomJyd9dna26GKIDMTdOXz48OoZ0Vmf4FY1S0tLTExM0Gg0qNfrzM/Pdz3zvGidZ2K7e9ezsoe50kEaV0cwsxPuPtntucLPCBeR9TY7Izrzgc6SK8Ol+DcO9APnnMk97GSAbmeFp7nPKDREItTrchC6ZMrm+oVuLIGbZPZhWjMU095nFBoiJaKpzsn0qm3HErhJWkPt19RqNSYnJ9mxY8dQfyvtfUahIVIiZeh6iVVMgZvkooxmxsMPP8z+/fuZnZ3l+uuvHyro0t5nNBAuUjK6DPxwippgsJXPK61B/UHLoIFwkQoZ5nLcku8l19u22iWWVishzX1GLQ0RkYyk0VIoomWploZIEMvsGYnLVvaLfv83aUuh33sM88NKWe7jCg0ZGTHNnpF4bGW/2Ph/G43GugN2ki6xRqPBG97whlT2yzz2cXVPycgoy5nCkq+t7Bcb/+/k5CSzs7McPHiQ6elpxsb618ubzSbXXXcdjz/+OMCmf3+zrqq09nF1T4mg6arS3bD7RbPZxN1X/+++ffs4fvz4QFN6l5eXOX78+Orjffv29e3C2qwVkcs+3uuiVFW46YKFspGuzJuPrLZzLO/beVHEqakpP3ny5DkXSkzyXu0LDNZqtU1/+CnphSzT2EZE+iNMIrnTdNXsZdWvnmV//aD7ReeJgo899hhjY2OMjY0NPKW3PeaxsLDAzMxM3+6spK2IrPdxjWmIjKispnJmNXYU05iUl/BEwUFoTENE1smy1p5Vv3pMY1JFnCgIcbSU1dIQGUFZ19qzqhHrEir5UEtDRNbJutaeVY04hpr2qKsXXQARyV+7e0W1dhmUQkNkRLVr7SKDUPeUiBRK1wMrF4WGVIoOQOWi64FlI8vvgUKjAnSgbNEBqHxi+jW9qsj6e6DQKDkdKNcsLy/z6KOP0mg0ePTRR3UAKoGYzr2oiqyDWKFRcqqprRkfH2fbtm0AbNu2jfHx8YJLJJsp6iS5rYq5dZ/5dOpU301yp5ramtOnT3PmzBkAzpw5w+nTpwsukSRRtnMvYm/dDxLEw4SfQqPkylpTy4ICVDbKokVQhtZ9kiAeNvwUGhVQtppaVhSg0imrFkFVKifDhp9CQ6KzldqhAlTasmoRVKVyMmz4KTQkKrH3F0t5ZNkiyLNyktWg+7Dhp9CQqJShv1iGk/eMoyq0CDI/52KI8FNoSFSq0l8s6xXVgix7d2UelahBw1yhIVGpQu1QzqUW5HB6VaLSarUNE+YKDYlO2WuHRYvxxDO1IIfTrRKVZqttmDAvLDTMbI+ZTZvZs2b2jJm9Pyy/xMyOmtnz4d/tYbmZ2afNbM7MnjKzq4squ5RDjAfPrMU6kSDvFmTZPvt+5d1YiUqz1TZMmBfZ0mgAH3D3q4ADwB1mdhVwJ/Cwu18JPBweA7wduDLcbgP+MP8iS1nEevDMWszdQHm1IMv22Q9a3jRbbUOFubtHcQP+GngL8BywOyzbDTwX7v8R8Isdr199Xa/bNddc4zKaFhcXvV6vO+D1et0XFxdzL8PKyoovLi56s9nM7W82m00/dOiQ1+t1P3ToUK5/OxYxfPaDGKa8We9bwKz3OK5GMaZhZlcAPw08Aexy91PhqUWg/dNilwEvdvy3+bBM5BxF96EXVdvVRILiP/tBDdVFVOC4X+E/92pm24C/AH7V3f+tcyO4u5vZQJ2SZnYbre4rLr/88jSLKiVS9G9gd+smyuunVUf9Z1yL/uwHtbG87s5LL70UbdkLbWmY2Xm0AuPz7v6XYfGSme0Oz+8GXgrLF4A9Hf99Iixbx93vdfdJd5/csWNHdoWX6BVZGytbbbdqyjYDr11ed49+PKbI2VMGfAb4prv/v46nHgBuDfdvpTXW0V7+rjCL6gDwvY5uLNmiss02iZ26iWQYMU9kaCuypXEd8EvADWb29XC7CfgY8BYzex54c3gM8CXgBWAO+GPg9gLKXEllm21SFmWr7Ur6Bq2MlaGFalWuWU5OTvrs7GzRxYje0tISExMTNBoN6vU68/PzI90nLqOp2WymOg7SrozNzMxw8OBBpqenGRvbvJ6edjmGYWYn3H2y23NRzJ6SYpWhdrMV6nqTzWTR2h769yoib6EqNKTS/e/qepMkshhLqGplTKEhQPy1m2HFPrCoVlAcsjjAV7UyptCQSou5tqdWUDyyOsBXsTKmgXCpvBgGFrvRBASJlQbCZaTFWtuLuRUk0kvhlxERGVVlu9yF5CfW1jGopRE1DZLGJYvPY5hWkPaLaot9rEuhEanYd5xRE8vnEUs5JDuxz/hTaEQq9h1n1MTyecRSDslO7GNdCo1Ixb7jjJpYPo9YyiHZif38Dk25jVjMg2GjKJbPI5ZyxETbJF2acltSsU4VHVWxfB6xlCMWGufJl0JDREpN4zz5UmiISKlpnCdfOrlPREpNJ0nmS6EhIqXXHueR7Kl7SkQk0Nn2m1NoiIigWVhJKTRERNAsrKQUGlJJ6maQQWkWVjIKDakcdTPIMGK/fEcsFBpSOepmkGHpbPvNKTSkctTNIFUTU3erQkMqR90MUiWxdbcqNKSS1M0gVRFbd6tCQ0QkYrF1t+oyIiIiEYvt2lpqaYhIqcQ0KJyXmLpbFRoiUhqxDQqPIoWGiJRGbIPCscqyNabQEJHSiG1QOEZZt8Y0EC4iA2s2m4UMzMY2KByjbq2xNH9rRC0NERlI0eMKMQ0Kxyjr1phaGiIykKxrsrI1WbfGStfSMLMbzew5M5szszuLLo/IqNG4QvyybI2VqqVhZjXgbuAtwDxw3MwecPdniy2ZyOjQuMJo27SlYWb/y8y251GYBPYDc+7+grv/CLgPuLngMomMHI0rjK4k3VO7aNXoj4SuoSL3ksuAFzsez4dlq8zsNjObNbNZzeEWkTIo01num4aGu/8GcCXwGeDdwPNm9ttm9pMZl20o7n6vu0+6++SOHTuKLo6ISF9Fz0YbVKKBcG/F32K4NYDtwBfN7OMZlq2bBWBPx+OJsExSVqaaj0iZ99eyneWeZEzj/WZ2Avg48CjwWnd/L3AN8PMZl2+j48CVZrbXzM4HbgEeyLkMlVe2mo+MtrLvr2WbjWabJbOZfRT4rLt/p8tz/83dv5lV4XqU5ybgk0AtlOv/9nrt5OSkz87O5la2qlhaWmJiYoJGo0G9Xmd+fl7z8CVaVdhfizrDvhczO+Huk92eSzKmcVe3wAjP5RoY4W9+yd3/q7v/ZL/AkOGVreYjo60K+2uZZqOV6jwNyYfm4UuZVGV/ja210UvpzgiXfJSp5iNS9v017XEZXRpdRKTC0pxBlfXEAIWGiEjB0hyXyXoKr0JDRKRg7XGZ+fl5jh07tqVutqQBNGwXlkJDRCQCaY3LJAmgrXRhKTRERCpmswDaSheWQkNEolfmy4TEaCtjKAoNEYla2S8TEqOtjKEoNEQkamW7oF9ZDDuGotAQkahV4TIhVaLLiIjIOrFdzqIqlwmpCrU0RGRVrOMHZb9MSJUoNERklcYPZDMKDRFZpfED2YzGNERklcYPZDMKDRFZpz1+INKNuqdERCQxhYZIienyGpI3hYZIScU6PVaqTaEhUlKaHitFUGiIlJSmx0oRNHtKpKQ0PVaKoNAQKTFNj5W8qXtKREQSU2iIiEhiCg0RiYrOPYmbQkNGgg5E5aBzT+Kn0JDK04GoPHTuSfwUGlJ5OhCVh849iZ9CQyov1gORuszO1T73ZH5+nmPHjo30uSex7h8KDam8GA9E6jLrTT/tmt3+kUYQKTRkJMR2IFKXWfUkOSAnPWhnsX+kFUQKDZECxNplJsNJckAe5KCdxf6RVhBZEf1lZvYJ4L8DPwK+BbzH3f81PPch4FeAFeB/u/tDYfmNwKeAGvAn7v6xzf7O5OSkz87OZrMSIlvUbDZ13aiKWFpaYmJigkajQb1eZ35+/pzLuyR5Tae09w935/Dhw8zMzHDw4MG+XbVmdsLdJ7s9V1RL4yjwU+7+OuAfgQ8BmNlVwC3Aa4AbgT8ws5qZ1YC7gbcDVwG/GF4rUlqxdZnJ8JK0DAZtPaS9f/Qb2xtkrKOQCxa6+991PHwc+IVw/2bgPnf/D+DbZjYH7A/Pzbn7CwBmdl947bM5FVlEpKckVxyO4arE3S5w2e42a7dApqen+79HlgVM6JeBvwn3LwNe7HhuPizrtVxEJApJWgYxti4HHevILDTM7Mtm9o0ut5s7XvMRoAF8PsW/e5uZzZrZrGakiIj0N2i3WWbdU+7+5n7Pm9m7gXcAb/K1jrQFYE/HyybCMvos3/h37wXuhdZA+MAFFxEZIYN2mxXSPRVmQn0QeKe7/7DjqQeAW8zsAjPbC1wJfAU4DlxpZnvN7Hxag+UP5F1uEZE85XVW+CDdZkWNafw+cBFw1My+bmb3ALj7M8ARWgPcfwvc4e4r7t4A3gc8BHwTOBJeKyJSSbFeNaCQ8zTyovM0RKSsBj2vI00xnqchIiJ9xHrVgELO0xARkf5iOK+jG4WGiEikup2MVzR1T4nISIv1dytipdAQkZEV6wylmCk0RGRk6XdNBqfQEJGRFesMpZhpIFxERlasM5RiptAQkZEW4wylmKl7SkREElNoiIhIYgoNERFJTKEhIiKJKTRERCQxhYaIiCSm0BARkcQUGiIikphCQ0REElNoiIhIYgoNERFJTKEhIiKJKTRERCQxhYaIiCSm0BARkcQUGiIikphCQ0REElNoiIhIYgoNERFJTKEhIiKJKTRERCQxhYaIiCSm0BARkcQUGiIikphCQ0REElNoiIhIYoWGhpl9wMzczMbDYzOzT5vZnJk9ZWZXd7z2VjN7PtxuLa7UIiKjq17UHzazPcBbge92LH47cGW4/Qzwh8DPmNklwF3AJODACTN7wN3/Jd9Si4iMtiJbGr8HfJBWCLTdDHzOWx4HXmlmu4G3AUfd/eUQFEeBG3MvsYjIiCskNMzsZmDB3Z/c8NRlwIsdj+fDsl7Lu733bWY2a2azy8vLKZZaREQy654ysy8Dl3Z56iPAh2l1TaXO3e8F7gWYnJz0TV4uIiIDyCw03P3N3Zab2WuBvcCTZgYwAXzVzPYDC8CejpdPhGULwOENy4+lXmgREekr9+4pd3/a3Xe6+xXufgWtrqar3X0ReAB4V5hFdQD4nrufAh4C3mpm281sO61WykN5l11EZNQVNnuqhy8BNwFzwA+B9wC4+8tm9lvA8fC633T3l4spoojI6Co8NEJro33fgTt6vO6zwGdzKpaIiHShM8JFRCQxhYaIiCSm0BARkcQUGiIikphCQ0REElNoiIhIYgoNERFJTKEhIiKJKTRERCQxhYaIiCSm0BARkcQUGiIikphCQ0REElNoiIhIYgoNERFJTKEhIiKJWet3j6rJzJaB7xRdDmAcOF10ISKhbbFG22KNtsWaGLbFf3H3Hd2eqHRoxMLMZt19suhyxEDbYo22xRptizWxbwt1T4mISGIKDRERSUyhkY97iy5ARLQt1mhbrNG2WBP1ttCYhoiIJKaWhoiIJKbQEBGRxBQaGTCzD5iZm9l4eGxm9mkzmzOzp8zs6o7X3mpmz4fbrcWVOl1m9gkz+/9hff/KzF7Z8dyHwrZ4zsze1rH8xrBszszuLKbk2RuV9Wwzsz1mNm1mz5rZM2b2/rD8EjM7Gvb9o2a2PSzv+X2pCjOrmdnXzOzB8HivmT0R1vl+Mzs/LL8gPJ4Lz19RZLkBcHfdUrwBe4CHaJ1UOB6W3QT8DWDAAeCJsPwS4IXw7/Zwf3vR65DSdngrUA/3fxf43XD/KuBJ4AJgL/AtoBZu3wJeBZwfXnNV0euRwXYZifXcsM67gavD/YuAfwz7wceBO8PyOzv2ka7flyrdgF8DvgA8GB4fAW4J9+8B3hvu3w7cE+7fAtxfdNnV0kjf7wEfBDpnGNwMfM5bHgdeaWa7gbcBR939ZXf/F+AocGPuJc6Au/+duzfCw8eBiXD/ZuA+d/8Pd/82MAfsD7c5d3/B3X8E3BdeWzWjsp6r3P2Uu3813P8+8E3gMlrr/afhZX8K/Fy43+v7UglmNgH8LPAn4bEBNwBfDC/ZuC3a2+iLwJvC6wuj0EiRmd0MLLj7kxueugx4sePxfFjWa3nV/DKtmiNoW4zKenYVuld+GngC2OXup8JTi8CucL/q2+iTtCqWzfD4J4B/7ahkda7v6rYIz38vvL4w9SL/eBmZ2ZeBS7s89RHgw7S6ZUZCv23h7n8dXvMRoAF8Ps+ySXzMbBvwF8Cvuvu/dVaY3d3NrPLz/83sHcBL7n7CzA4XXZ5hKDQG5O5v7rbczF5Lq4/+yfBlmAC+amb7gQVaYx1tE2HZAnB4w/JjqRc6I722RZuZvRt4B/AmD52y9N4W9FleJf3Wv7LM7DxagfF5d//LsHjJzHa7+6nQ/fRSWF7lbXQd8E4zuwm4ELgY+BStLrh6aE10rm97W8ybWR34ceCf8y92h6IHVap6A/6JtYHwn2X9wN5XwvJLgG/TGgTfHu5fUnTZU1r/G4FngR0blr+G9QPhL9AaHK6H+3tZGyB+TdHrkcF2GYn13LDOBnwO+OSG5Z9g/UD4x8P9rt+Xqt1oVRjbA+F/zvqB8NvD/TtYPxB+pOhyq6WRjy/RmhEyB/wQeA+Au79sZr8FHA+v+013f7mYIqbu92kFw9HQ8nrc3f+nuz9jZkdoBUoDuMPdVwDM7H20Zp7VgM+6+zPFFD077t4YhfXc4Drgl4CnzezrYdmHgY8BR8zsV2jNNvwf4bmu35eK+3XgPjP7P8DXgM+E5Z8B/szM5oCXaQVHoXQZERERSUyzp0REJDGFhoiIJKbQEBGRxBQaIiKSmEJDREQSU2iIiEhiCg0REUlMoSGSIzPbF34j4kIze0X4fYmfKrpcIknp5D6RnIWzfi8EfgyYd/ffKbhIIokpNERyFn6V7Tjw78DB9mVURMpA3VMi+fsJYButX7G7sOCyiAxELQ2RnJnZA7R+sW8vsNvd31dwkUQS01VuRXJkZu8Czrr7F8ysBsyY2Q3u/vdFl00kCbU0REQkMY1piIhIYgoNERFJTKEhIiKJKTRERCQxhYaIiCSm0BARkcQUGiIikth/AivWp8YrzMMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_particles(SWARM, OPTIMAL_PARTICLE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can see all of the particles have approached the optima to some degree. Were we to run this a few more times, it would get ever more closer. This leads well into our main driver.\n",
    "\n",
    "## Main Driver\n",
    "\n",
    "We'll combine everything thus far into one main driver which will move particles over many epochs. It will also output the swarm best fitness as well as what the final plot looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0, Best = 159.79625052137567\n",
      "Epoch 10, Best = 6.791653509486878\n",
      "Epoch 20, Best = 6.791653509486878\n",
      "Epoch 30, Best = 0.4872051415511578\n",
      "Epoch 40, Best = 0.07243380432009872\n",
      "Epoch 50, Best = 0.021283036086515494\n",
      "Epoch 60, Best = 0.004067141952418751\n",
      "Epoch 70, Best = 0.004067141952418751\n",
      "Epoch 80, Best = 0.0035295543207212177\n",
      "Epoch 90, Best = 0.00026627935983469797\n",
      "Epoch 100, Best = 2.6744321985461283e-05\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEGCAYAAACZ0MnKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAS5UlEQVR4nO3df4xdZZ3H8fd3OgIbWZeW9g61U7bdlX+qbgKOlQxkmYqRHxJqWDU1G8UfCVFxo9EEQTZxVze7isn6I7oaYklgg0FEDMRgsGor6AjSqkD4oYy4Lp2U4SIIukbd6f3uH/cZOp2dKQ9k5p758X4lN3PO8zyd+z0nM/3MOc9z743MRJKkGn1NFyBJWjoMDUlSNUNDklTN0JAkVTM0JEnV+psuYCGtXbs2N23a1HQZkrSk7Nu37/HMXDdb37IOjU2bNrF3796my5CkJSUifjVXn7enJEnVDA1JUjVDQ5JUzdCQpCWg0+kwMTFB02/9ZGhI0iLX6XTYtm0bg4ODjIyM0Ol0GqvF0JCkRa7dbjM6Osrk5CSjo6O02+3GajE0JGmRa7VaDA8P09/fz/DwMK1Wq7FalvXrNCRpNp1Oh3a7TavVIiKaLudZRQS7d+9mYmKi8Xq90pC0oiym+YHnaseOHWzcuLHRug0NSSvKYpofeC4WS92GhqQVZTHNDzwXvaq7XMHMOXURTa/5XUhDQ0Ppe09JmmmpzWl0Oh0OHDjA448/TqvV4oQTTliQuqdu3d12222ZmbNeVDgRLmnF6evrY2BgoOkyjmgq2NauXcsZZ5zBD37wAwCOPfZYnnzySfr75/+/76lbYMCcieTtKUlaZDqdDiMjI2zYsIHtJ59MlMAA+N3vfsfPd+6kvW8fBw8enNdXiU/dAgPm/IaGhiQ1bOZbhBw4cIDbb7+dgwcP8tS993IjcEYZewaw7l3v4g2vfCVr1qxhw4YN87aaamppL3DPXGMMDUlq0MwlwJOTk1xwwQXP9H8feCPwVeAj5esbgdsyefrppzl48OC8rqbq6+sDmJyzf16eRZL0vMxcSvvggw/+vw+P+x7weeCfytfvTeuLiJ6uAms8NCJiVUT8JCK+UfY3R8SdETEWEV+JiKNK+9Flf6z0b2qybkmaDzOX0m7ZsoXTTjvtsDFnABfTDY2LOXSr6oUvfCHj4+Ps2bOnZ6vAGg8N4H3AA9P2PwF8KjNfAjwJvLO0vxN4srR/qoyTpCVtah5h//797Nmzh76+Pvbs2cP4+Dhbt27lbyO4cdUq3gj8M91bU7e+6EU8dNVVPP3006xfv76ny4YbDY2IGAReB3yp7AfwauCGMuRq4PVle3vZp/SfGUthgbUkPYupJcBT/6X19fXx4he/mB/+8IfccNdd5A03cHt3roHb+/r4n2uu4SVnnjk1/9DbWnv+jIf7NHAJMDXtfzzwm8ycmoTZD2wo2xuARwBK/1Nl/GEi4qKI2BsRe5fK2wNI0mz6+vpY94pXsGb7dk4//XT6+/s5/fTTWX3++XDiic3U1MizAhFxHvBYZu6bz++bmVdm5lBmDq1bt24+v7UkNWLmLawmb7I0+Yrw04DzI+Jc4BjgRcBngOMior9cTQwC42X8OLAR2B8R/cBfAL/ufdmS1HuL5VXsjV1pZOZlmTmYmZuAHcB3M/Pvgd3AG8qwC4GbyvbNZZ/S/91czm+cJUmLUNNzGrP5EPCBiBijO2exs7TvBI4v7R8ALm2oPklasRbFGxZm5h5gT9l+GNg6y5g/0F1tJklqyGK80pAkLVKGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUlLRKfTYWJigsxsrAZDQ5KWgE6nw7Zt2xgcHGRkZIROp9NIHYaGJC0B7Xab0dFRJicnGR0dpd1uN1KHoSFJS0Cr1WJ4eJj+/n6Gh4dptVqN1NFYaETExojYHRH3R8R9EfG+0r4mInZFxEPl6+rSHhHx2YgYi4h7IuKUpmqXpF6LCHbv3s3+/fvZs2cPEdFIHU1eaUwCH8zMLcCpwMURsQW4FPhOZp4EfKfsA5wDnFQeFwFf6H3JktScvr4+BgYGGgsMaDA0MvNAZv64bP8WeADYAGwHri7DrgZeX7a3A9dk1x3AcRGxvsdlS9KKtijmNCJiE3AycCcwkJkHStejwEDZ3gA8Mu2f7S9tkqQeaTw0IuJY4GvA+zPz6el92V2M/JwWJEfERRGxNyL2NrW6QJKWq0ZDIyJeQDcwrs3MG0vzxNRtp/L1sdI+Dmyc9s8HS9thMvPKzBzKzKF169YtXPGStAI1uXoqgJ3AA5n579O6bgYuLNsXAjdNa39rWUV1KvDUtNtYkqQe6G/wuU8D3gLcGxE/LW0fBj4OXB8R7wR+Bbyp9N0CnAuMAb8H3t7bciVJjYVGZn4fmGvd2JmzjE/g4gUtSpJ0RI1PhEuSlg5DQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUjVDQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUjVDQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkKQlpNPpMDExQWY28vyGhiQtEZ1Oh23btjE4OMjIyAidTqfnNRgakrREtNttRkdHmZycZHR0lHa73fMaDA1JWiJarRbDw8P09/czPDxMq9XqeQ39PX9GSdLzEhHs3r2bdrtNq9UiInpew5K70oiIsyPiZxExFhGXNl2PJPVSX18fAwMDjQQGLLHQiIhVwOeBc4AtwJsjYkuzVUnSyvGsoRER/xARq3tRTIWtwFhmPpyZfwKuA7Y3XJMkrRg1VxoDwF0RcX25NdTMNVHXBuCRafv7S9szIuKiiNgbEXubWFkgScvZs4ZGZv4jcBKwE3gb8FBE/GtE/PUC1/a8ZOaVmTmUmUPr1q1ruhxJWlaq5jSy+9LDR8tjElgN3BARVyxgbbMZBzZO2x8sbZKkHqiZ03hfROwDrgB+ALw8M98NvAL4uwWub6a7gJMiYnNEHAXsAG7ucQ2StGLVvE5jDXBBZv5qemNmdiLivIUpa3aZORkR7wVuBVYBV2Xmfb2sQZJWsmcNjcz8yBH6Hpjfcp5dZt4C3NLr55UkLbHXaUiSmmVoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqo1EhoR8cmIeDAi7omIr0fEcdP6LouIsYj4WUScNa397NI2FhGXNlG3JK10TV1p7AJelpl/A/wcuAwgIrYAO4CXAmcD/xERqyJiFfB54BxgC/DmMlaS1EONhEZmfiszJ8vuHcBg2d4OXJeZf8zMXwJjwNbyGMvMhzPzT8B1ZawkqYcWw5zGO4Bvlu0NwCPT+vaXtrnaJUk91L9Q3zgivg2cMEvX5Zl5UxlzOTAJXDuPz3sRcBHAiSeeOF/fVpLEAoZGZr7mSP0R8TbgPODMzMzSPA5snDZssLRxhPaZz3slcCXA0NBQzjZGkvT8NLV66mzgEuD8zPz9tK6bgR0RcXREbAZOAn4E3AWcFBGbI+IoupPlN/e6bkla6RbsSuNZfA44GtgVEQB3ZOa7MvO+iLgeuJ/ubauLM/MgQES8F7gVWAVclZn3NVO6JK1ccejO0PIzNDSUe/fubboMSVpSImJfZg7N1rcYVk9JkpYIQ0OSVM3QkCRVMzQkSdUMjSPodDpMTEywnBcLSNJzYWjModPpsG3bNgYHBxkZGaHT6TRdkiQ1ztCYQ7vdZnR0lMnJSUZHR2m3202XJEmNMzTm0Gq1GB4epr+/n+HhYVqtVtMlSVLjmnpF+KIXEezevZt2u02r1aK8cl2SVjRD4wj6+voYGBhougxJWjS8PSVJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqtZoaETEByMiI2Jt2Y+I+GxEjEXEPRFxyrSxF0bEQ+VxYXNVS9LK1d/UE0fERuC1wH9Paz4HOKk8XgV8AXhVRKwBPgIMAQnsi4ibM/PJ3lYtSStbk1canwIuoRsCU7YD12TXHcBxEbEeOAvYlZlPlKDYBZzd84olaYVrJDQiYjswnpl3z+jaADwybX9/aZurfbbvfVFE7I2Ive12ex6rliQt2O2piPg2cMIsXZcDH6Z7a2reZeaVwJUAQ0ND+SzDJUnPwYKFRma+Zrb2iHg5sBm4OyIABoEfR8RWYBzYOG34YGkbB0ZmtO+Z96IlSUfU89tTmXlvZrYyc1NmbqJ7q+mUzHwUuBl4a1lFdSrwVGYeAG4FXhsRqyNiNd2rlFt7XbskrXSNrZ6awy3AucAY8Hvg7QCZ+UREfAy4q4z7aGY+0UyJkrRyNR4a5WpjajuBi+cYdxVwVY/KkiTNwleES5KqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSaoW3c89Wp4iog38quk6gLXA400XsUh4Lg7xXBziuThkMZyLv8zMdbN1LOvQWCwiYm9mDjVdx2LguTjEc3GI5+KQxX4uvD0lSapmaEiSqhkavXFl0wUsIp6LQzwXh3guDlnU58I5DUlSNa80JEnVDA1JUjVDYwFExAcjIiNibdmPiPhsRIxFxD0Rccq0sRdGxEPlcWFzVc+viPhkRDxYjvfrEXHctL7Lyrn4WUScNa397NI2FhGXNlP5wlspxzklIjZGxO6IuD8i7ouI95X2NRGxq/zs74qI1aV9zt+X5SIiVkXETyLiG2V/c0TcWY75KxFxVGk/uuyPlf5NTdYNQGb6mMcHsBG4le6LCteWtnOBbwIBnArcWdrXAA+Xr6vL9uqmj2GezsNrgf6y/QngE2V7C3A3cDSwGfgFsKo8fgH8FXBUGbOl6eNYgPOyIo5zxjGvB04p238O/Lz8HFwBXFraL532MzLr78tyegAfAL4MfKPsXw/sKNtfBN5dtt8DfLFs7wC+0nTtXmnMv08BlwDTVxhsB67JrjuA4yJiPXAWsCszn8jMJ4FdwNk9r3gBZOa3MnOy7N4BDJbt7cB1mfnHzPwlMAZsLY+xzHw4M/8EXFfGLjcr5TifkZkHMvPHZfu3wAPABrrHfXUZdjXw+rI91+/LshARg8DrgC+V/QBeDdxQhsw8F1Pn6AbgzDK+MYbGPIqI7cB4Zt49o2sD8Mi0/f2lba725eYddP9yBM/FSjnOWZXbKycDdwIDmXmgdD0KDJTt5X6OPk33D8tO2T8e+M20P7KmH+8z56L0P1XGN6a/ySdfiiLi28AJs3RdDnyY7m2ZFeFI5yIzbypjLgcmgWt7WZsWn4g4Fvga8P7MfHr6H8yZmRGx7Nf/R8R5wGOZuS8iRpqu5/kwNJ6jzHzNbO0R8XK69+jvLr8Mg8CPI2IrME53rmPKYGkbB0ZmtO+Z96IXyFznYkpEvA04Dzgzy01Z5j4XHKF9OTnS8S9bEfECuoFxbWbeWJonImJ9Zh4ot58eK+3L+RydBpwfEecCxwAvAj5D9xZcf7mamH68U+dif0T0A38B/Lr3ZU/T9KTKcn0A/8WhifDXcfjE3o9K+xrgl3QnwVeX7TVN1z5Px382cD+wbkb7Szl8IvxhupPD/WV7M4cmiF/a9HEswHlZEcc545gDuAb49Iz2T3L4RPgVZXvW35fl9qD7B+PURPhXOXwi/D1l+2IOnwi/vum6vdLojVvorggZA34PvB0gM5+IiI8Bd5VxH83MJ5opcd59jm4w7CpXXndk5rsy876IuJ5uoEwCF2fmQYCIeC/dlWergKsy875mSl84mTm5Eo5zhtOAtwD3RsRPS9uHgY8D10fEO+muNnxT6Zv192WZ+xBwXUT8C/ATYGdp3wn8Z0SMAU/QDY5G+TYikqRqrp6SJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUND6qGIeGX5jIhjIuKF5fMlXtZ0XVItX9wn9Vh51e8xwJ8B+zPz3xouSapmaEg9Vj6V7S7gD8Dw1NuoSEuBt6ek3jseOJbup9gd03At0nPilYbUYxFxM91P7NsMrM/M9zZcklTNd7mVeigi3gr8b2Z+OSJWAaMR8erM/G7TtUk1vNKQJFVzTkOSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnV/g+gB1FhbluZigAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The minimum for the fitness function was found to be 0.00.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "EPOCHS = 100\n",
    "\n",
    "def pso(dim, epochs):\n",
    "    \"\"\"Particle Swarm Optimization driver.\n",
    "    Initializes the swarm and improves it over time.\n",
    "    \n",
    "    Parameters:\n",
    "        dim : the dimensionality of the problem.\n",
    "        epochs : how many iterations.\n",
    "    \n",
    "    Returns:\n",
    "        Swarm best fitness after optimizing.\n",
    "    \"\"\"\n",
    "    # initialize the swarm based on problem scope and swarm size\n",
    "    swarm = initialize_swarm(SWARM_SIZE, dim, SEARCH_MIN, SEARCH_MAX)\n",
    "    # output initial best fitness\n",
    "    initial_best = get_swarm_best(swarm)[0]\n",
    "    print_stats(0, initial_best)\n",
    "    for e in range(1, epochs+1): # for each iteration\n",
    "        move_particles(swarm) # move particles to more fit neighborhood\n",
    "        if e % 10 == 0:\n",
    "            # every 10th epoch, print new best fitness\n",
    "            epoch_best = get_swarm_best(swarm)[0]\n",
    "            print_stats(e, epoch_best)\n",
    "    # plot the final particles\n",
    "    plot_particles(swarm, OPTIMAL_PARTICLE)\n",
    "    # return the swarm best fitness\n",
    "    return get_swarm_best(swarm)[0]\n",
    "    \n",
    "def print_stats(epoch, best):\n",
    "    print(f'Epoch {epoch}, Best = {best}')\n",
    "\n",
    "best = pso(DIMENSIONS, EPOCHS)\n",
    "print(f'The minimum for the fitness function was found to be {best:.2f}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can see after a hundred iterations, most of the particles congregate at or around the optimum, our intended behavior. There are a few outliers trapped in local minimum but that's okay. We only particularly care about the swarm best which does find the minimum of the optimization problem.\n",
    "\n",
    "For fun, here's an animation of a full run-through using the same parameters as above:\n",
    "\n",
    "<img width=\"432px\" src=\"./images/animation.gif\"/>\n",
    "\n",
    "Here you see how the swarm does zero in on the optimum. It's quite artistic in a way as it does really simulate something akin to birds converging onto prey.\n",
    "\n",
    "Other than that, this wraps up the particle swarm optimization tutorial.\n",
    "\n",
    "## Final Thoughts\n",
    "\n",
    "Particle swarm optimization can help you find global optima for a fitness function if it's not know a priori or is laborious to find manually. Provided you know the search space and the function to optimize, it's very fast and efficient.\n",
    "\n",
    "There's a lot more applications to PSO than the trivial example I've used here but since the algorithm is dimensionality and problem agnostic, it can be applied to anything without large changes. It's also a very short algorithm which makes it apt for optimization purposes, at least when compared to something like a genetic algorithm.\n",
    "\n",
    "Further reading might be on advanced forms of PSO. PSO is good at getting out of local minimum but it's not perfect. Over the years there's been numerous refinements to the algorithm to alleviate this. You can also split swarms into multiple packs (look up Grey Wolf Optimization for something akin to this), try different parameters, or add a local search when particles get trapped in local optima.\n",
    "\n",
    "## Finished Code\n",
    "\n",
    "Find below the full uncommented code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!/usr/bin/env python3\n",
    "\n",
    "from math import sin, sqrt\n",
    "import random\n",
    "\n",
    "class Particle:\n",
    "    def __init__(self, pos, vel):\n",
    "        self.pos, self.vel = pos, vel\n",
    "        self.fit = schwefel(self.pos)\n",
    "        self.best_pos, self.best_fit = self.pos, self.fit\n",
    "\n",
    "    def set_pos(self, pos):\n",
    "        self.pos = pos\n",
    "        if not any(p < SEARCH_MIN for p in pos)\\\n",
    "        and not any(p > SEARCH_MAX for p in pos):\n",
    "            fitness = schwefel(self.pos)\n",
    "            if fitness < self.best_fit:\n",
    "                self.fit = fitness\n",
    "                self.best_fit = self.fit\n",
    "                self.best_pos = self.pos\n",
    "\n",
    "    def set_vel(self, vel):\n",
    "        self.vel = vel\n",
    "\n",
    "    def get_pos(self):\n",
    "        return self.pos\n",
    "\n",
    "    def get_vel(self):\n",
    "        return self.vel\n",
    "\n",
    "    def get_best_pos(self):\n",
    "        return self.best_pos\n",
    "\n",
    "    def get_fit(self):\n",
    "        return self.fit\n",
    "\n",
    "def pso(dim, epochs):\n",
    "    swarm = initialize_swarm(SWARM_SIZE, dim, SEARCH_MIN, SEARCH_MAX)\n",
    "    for e in range(1, epochs+1):\n",
    "        move_particles(swarm)\n",
    "    best_fitness = get_swarm_best(swarm)[0]\n",
    "    print(f'Best minimum found was {best_fitness:.2f}.')\n",
    "\n",
    "def move_particles(swarm):\n",
    "    swarm_best = get_swarm_best(swarm)\n",
    "    for particle in swarm:\n",
    "        new_pos = [0 for _ in range(DIMENSIONS)]\n",
    "        new_vel = [0 for _ in range(DIMENSIONS)]\n",
    "        for d in range(DIMENSIONS):\n",
    "            R_1 = random.uniform(0.00, 1.00)\n",
    "            R_2 = random.uniform(0.00, 1.00)\n",
    "            weight = W * particle.get_vel()[d]\n",
    "            cognitive = C_1 * R_1\n",
    "            cognitive *= (particle.get_best_pos()[d] - particle.get_pos()[d])\n",
    "            social = C_2 * R_2\n",
    "            social *= (swarm_best[1][d] - particle.get_pos()[d])\n",
    "            new_vel[d] = weight + cognitive + social\n",
    "            new_pos[d] = particle.get_pos()[d] + new_vel[d]\n",
    "        particle.set_pos(new_pos)\n",
    "        particle.set_vel(new_vel)    \n",
    "\n",
    "def initialize_swarm(size, dim, min, max):\n",
    "    swarm = []\n",
    "    for _ in range(size):\n",
    "        position = [random.uniform(min, max) for _ in range(dim)]\n",
    "        velocity = [0 for _ in range(dim)]\n",
    "        particle = Particle(position, velocity)\n",
    "        swarm.append(particle)\n",
    "    return swarm\n",
    "\n",
    "def get_swarm_best(swarm):\n",
    "    best_fit = swarm[0].get_fit()\n",
    "    best_pos = swarm[0].get_pos()\n",
    "    for particle in swarm:\n",
    "        if particle.get_fit() < best_fit:\n",
    "            best_fit = particle.get_fit()\n",
    "            best_pos = particle.get_pos()\n",
    "    return best_fit, best_pos\n",
    "\n",
    "def schwefel(particle):\n",
    "    summ = 0\n",
    "    for d in particle:\n",
    "        summ += d * sin(sqrt(abs(d)))\n",
    "    return (len(particle) * 418.9829) - summ\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    SEARCH_MIN = -500\n",
    "    SEARCH_MAX = 500\n",
    "    DIMENSIONS = 2\n",
    "    SWARM_SIZE = 200\n",
    "    EPOCHS = 100\n",
    "    W = 0.729844\n",
    "    C_1 = 1.496180\n",
    "    C_2 = 1.496180\n",
    "    pso(DIMENSIONS, EPOCHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may wish to catch for errors (as I have done none here), and perhaps accept some of the parameters as command line arguments. This code, as-is, should run in a terminal but allows for zero customization unless you change the code manually. Some parameters might include inertial weight and cognitive/social coefficients, swarm size, how many epochs, etc.\n",
    "\n",
    "You can save this code as `particle_algorithm.py` and execute it as one of the below:\n",
    "\n",
    "```\n",
    "$ python3 particle_algorithm.py\n",
    "$ ./particle_algorithm.py\n",
    "```\n",
    "\n",
    "You can also find the code in `./code`. For a C++ implementation I did for optimizing the Rastrigin function, you can see it <a href=\"https://github.com/stratzilla/pso-rastrigin\">here</a>.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}