comparison_doc_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Goal : compare the documentation to the data collected with kernel compilation\n",
    "\n",
    "--\n",
    "\n",
    "Can we find the informations given by the documentation with the dataset?\n",
    "\n",
    "### Results : 51/73  features corresponding => DEAD END\n",
    "\n",
    "### TODO:\n",
    "- check if each modality is well represented => done, it's worse\n",
    "- read the doc again?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tuxml\n",
    "from scipy.stats import chi2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the different datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### I - Extracted from the documentation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>keyword</th>\n",
       "      <th>yes_increase_size</th>\n",
       "      <th>size_diff</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ZRAM</th>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KERNEL_GZIP</th>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KERNEL_BZIP2</th>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KERNEL_LZMA</th>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KERNEL_XZ</th>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               keyword  yes_increase_size size_diff\n",
       "features                                           \n",
       "ZRAM          compress              False       NaN\n",
       "KERNEL_GZIP   compress              False       NaN\n",
       "KERNEL_BZIP2  compress              False       NaN\n",
       "KERNEL_LZMA   compress              False       NaN\n",
       "KERNEL_XZ     compress              False       NaN"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc = pd.read_csv(r'C:\\Users\\llesoil\\Documents\\tuxml\\features_details.csv', index_col=0)\n",
    "doc.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- features => name\n",
    "- keyword => which csv give the result (useless here)\n",
    "- yes_increase_size => boolean, answering the question \"Does the kernel size increase (True) or Decrease (False) if I activate this feature?\"\n",
    "- size_diff => if it's mentioned in the documentation, what difference of size should be seen between a kernel with/without this feature activated?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "feat_doc = np.array(doc.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### II - Collected data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#features = pd.read_csv(\"dataset_before_encoding.csv\", low_memory = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = tuxml.load_dataset()\n",
    "#features = features.replace([2,-1],0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### III - Some tests/cleaning\n",
    "\n",
    "##### a] Are all the features from the documentation in our dataset?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "77 %  of the features of the documentation are present in the dataset\n"
     ]
    }
   ],
   "source": [
    "all_features = features.columns\n",
    "f_in_dataset = []\n",
    "for f in feat_doc:\n",
    "    if f in all_features:\n",
    "        f_in_dataset.append(f)\n",
    "print(int(len(f_in_dataset)/len(feat_doc)*100), \"%  of the features of the documentation are present in the dataset\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_common = features[f_in_dataset]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We keep them in a new dataset\n",
    "\n",
    "###### b] We remove all the features which aren' t well represented (lack of 'no'/'module' values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define our uniformity test (very \"large\" to keep enough features, keep all the features having vaguely an uniform distribution)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = features_common.shape[0]\n",
    "col = features_common.columns\n",
    "\n",
    "def is_nearly_uniform(feature_name):\n",
    "    my_feature = features_common[feature_name]\n",
    "    sum_yes = sum(my_feature==1)\n",
    "    sum_no = sum(my_feature==0)\n",
    "    sum_module = sum(my_feature==2)\n",
    "    d_xhi_square = ((sum_yes-n/3)**2 + (sum_no-n/3)**2 + (sum_module-n/3)**2)/(n/3)\n",
    "    #d_g_test = 2*(sum_yes*np.log(sum_yes/(n/3)) + sum_no*np.log(sum_no/(n/3)) + sum_module*np.log(sum_module/(n/3)))\n",
    "    return d_xhi_square < 1.5*chi2.ppf(0.95, n-1)\n",
    "\n",
    "unif = []\n",
    "for i in range(len(col)):\n",
    "    if is_nearly_uniform(col[i]):\n",
    "        unif.append(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "73"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(unif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Few bad examples of representation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feature 1\n",
      "yes : 0\n",
      "no : 92471\n",
      "module : 0\n",
      "should we keep it? False\n"
     ]
    }
   ],
   "source": [
    "num_f = 1\n",
    "print(\"feature\", num_f)\n",
    "my_feature = features_common[features_common.columns[num_f]]\n",
    "sum_yes = sum(my_feature==1)\n",
    "print(\"yes :\",sum_yes)\n",
    "sum_no = sum(my_feature==0)\n",
    "print(\"no :\",sum_no)\n",
    "sum_module = sum(my_feature==2)\n",
    "print(\"module :\",sum_module)\n",
    "print(\"should we keep it?\", is_nearly_uniform(features_common.columns[num_f]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feature 55\n",
      "yes : 92044\n",
      "no : 427\n",
      "module : 0\n",
      "should we keep it? False\n"
     ]
    }
   ],
   "source": [
    "num_f = 55\n",
    "print(\"feature\", num_f)\n",
    "my_feature = features_common[features_common.columns[num_f]]\n",
    "sum_yes = sum(my_feature==1)\n",
    "print(\"yes :\",sum_yes)\n",
    "sum_no = sum(my_feature==0)\n",
    "print(\"no :\",sum_no)\n",
    "sum_module = sum(my_feature==2)\n",
    "print(\"module :\",sum_module)\n",
    "print(\"should we keep it?\", is_nearly_uniform(features_common.columns[num_f]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Few \"good enough\" examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feature 0\n",
      "yes : 7538\n",
      "no : 80425\n",
      "module : 4508\n",
      "should we keep it? True\n"
     ]
    }
   ],
   "source": [
    "num_f = 0\n",
    "print(\"feature\", num_f)\n",
    "my_feature = features_common[features_common.columns[num_f]]\n",
    "sum_yes = sum(my_feature==1)\n",
    "print(\"yes :\",sum_yes)\n",
    "sum_no = sum(my_feature==0)\n",
    "print(\"no :\",sum_no)\n",
    "sum_module = sum(my_feature==2)\n",
    "print(\"module :\",sum_module)\n",
    "print(\"should we keep it?\", is_nearly_uniform(features_common.columns[num_f]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feature 14\n",
      "yes : 22911\n",
      "no : 69560\n",
      "module : 0\n",
      "should we keep it? True\n"
     ]
    }
   ],
   "source": [
    "num_f = 14\n",
    "print(\"feature\", num_f)\n",
    "my_feature = features_common[features_common.columns[num_f]]\n",
    "sum_yes = sum(my_feature==1)\n",
    "print(\"yes :\",sum_yes)\n",
    "sum_no = sum(my_feature==0)\n",
    "print(\"no :\",sum_no)\n",
    "sum_module = sum(my_feature==2)\n",
    "print(\"module :\",sum_module)\n",
    "print(\"should we keep it?\", is_nearly_uniform(features_common.columns[num_f]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Then we keep the rest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_common = features_common[features_common.columns[unif]]\n",
    "\n",
    "# we add the kernel size\n",
    "features_common['vmlinux'] = features['vmlinux']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "74"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(features_common.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparison doc vs data : Does the feature increase the kernel size?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### I - On the first example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ZRAM\n",
       "0    4.899161e+07\n",
       "1    6.269718e+07\n",
       "2    3.960957e+07\n",
       "Name: vmlinux, dtype: float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f_com = features_common.columns\n",
    "f_com = f_com[0: len(f_com)-1]\n",
    "\n",
    "size_mod = features_common.groupby(f_com[0])['vmlinux'].mean()\n",
    "size_mod"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each features, we compute the the average size for all the modalities ('y', 'm', 'n').\n",
    "We keep the modality correponding to the biggest value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "size_mod.index[np.argmax(np.array(size_mod))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our case, the 'yes' modality has a bigger size than others, which means that we can find a correlation between a bigger kernel size and the activation of the feature 'ZRAM'. This does not prove the causality (i.e. check 'yes' implies always a bigger size).\n",
    "\n",
    "At least we want to find correlations correponding to the documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the rest of the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "mod = []\n",
    "for f in f_com:\n",
    "    size_mod = features_common.groupby(f)['vmlinux'].mean()\n",
    "    mod.append(size_mod.index[np.argmax(np.array(size_mod))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We join the results to the dataframe with documentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>biggest_size_mod</th>\n",
       "      <th>keyword</th>\n",
       "      <th>yes_increase_size</th>\n",
       "      <th>size_diff</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ZRAM</th>\n",
       "      <td>1</td>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MODULE_COMPRESS</th>\n",
       "      <td>0</td>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZPOOL</th>\n",
       "      <td>1</td>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZBUD</th>\n",
       "      <td>1</td>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Z3FOLD</th>\n",
       "      <td>1</td>\n",
       "      <td>compress</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                biggest_size_mod   keyword  yes_increase_size size_diff\n",
       "features                                                               \n",
       "ZRAM                           1  compress              False       NaN\n",
       "MODULE_COMPRESS                0  compress              False       NaN\n",
       "ZPOOL                          1  compress              False       NaN\n",
       "ZBUD                           1  compress              False       NaN\n",
       "Z3FOLD                         1  compress              False       NaN"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = pd.DataFrame(np.transpose([f_com ,  mod]), columns = [\"features\", \"biggest_size_mod\"]).set_index(\"features\").join(doc)\n",
    "res.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For 51 features on the 73 we can compare, the documentation corresponds to the dataset.\n"
     ]
    }
   ],
   "source": [
    "print(\"For\", np.sum((res[\"biggest_size_mod\"]==1)==res[\"yes_increase_size\"]), \"features on the\", res.shape[0], \n",
    "      \"we can compare, the documentation corresponds to the dataset.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}