randomflorest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Descrição das Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Number |Attribute Information\n",
    "--|--\n",
    "1...50 | Average, standard deviation, min, max and median of the Attributes 51...60 for the source of the current blog post. With source we mean the blog on which the post appeared. For example, myblog.blog.org would be the source of the post myblog.blog.org/post_2010_09_10 \n",
    "51| Total number of comments before basetime \n",
    "52| Number of comments in the last 24 hours before the basetime \n",
    "53| Let T1 denote the datetime 48 hours before basetime, Let T2 denote the datetime 24 hours before basetime. This attribute is the number of comments in the time period between T1 and T2 \n",
    "54| Number of comments in the first 24 hours after the publication of the blog post, but before basetime \n",
    "55| The difference of Attribute 52 and Attribute 53 \n",
    "56...60| The same features as the attributes 51...55, but features 56...60 refer to the number of links (trackbacks), while features 51...55 refer to the number of comments. \n",
    "61| The length of time between the publication of the blog post and basetime \n",
    "62| The length of the blog post \n",
    "63...262| The 200 bag of words features for 200 frequent words of the text of the blog post \n",
    "263...269| binary indicator features (0 or 1) for the weekday (Monday...Sunday) of the basetime \n",
    "270...276| binary indicator features (0 or 1) for the weekday (Monday...Sunday) of the date of publication of the blog post \n",
    "277| Number of parent pages: we consider a blog post P as a parent of blog post B, if B is a reply (trackback) to  blog post P. \n",
    "278...280| Minimum, maximum, average number of comments that the parents received \n",
    "281| The target: the number of comments in the next 24 hours (relative to basetime)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Carregando Dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data_train = pd.read_csv('blogdata/train.csv',header=None)\n",
    "data_test = pd.read_csv('blogdata/test1.csv', header=None)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_train = data_train.iloc[:,0:280]\n",
    "y_train = data_train.iloc[:,-1]\n",
    "\n",
    "x_test = data_test.iloc[:, 0:280]\n",
    "y_test = data_test.iloc[:,-1] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Selection "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Variance Theshold** é um algoritmo de seleção de características que remove features que não atendem a certa variância.\n",
    "\n",
    "**RFE** elimina features menos importante por seleção recursiva \n",
    "\n",
    "**K-best** Seleciona a k-best features utilizando método univariados\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import VarianceThreshold, RFECV, SelectKBest, f_regression\n",
    "from sklearn.svm import SVR\n",
    "\n",
    "vt = VarianceThreshold(threshold=(.8 * (1 - .8))) #retira todos que a variância em 80% dos exemplos é 0\n",
    "rfecv = RFECV(SVR(kernel='linear'))\n",
    "selbest = SelectKBest(f_regression,10)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Processamento"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True  True  True  True False  True  True  True  True\n",
      " False  True  True  True  True  True  True  True  True  True  True  True\n",
      "  True  True  True False  True  True False  True False  True False False\n",
      "  True False  True False  True  True False  True False False  True  True\n",
      "  True False  True  True  True  True  True  True  True  True  True  True\n",
      "  True  True False False False False  True False  True False False False\n",
      " False False False False False False  True False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False  True False False False False False False False\n",
      " False False False False False False False False False False False  True\n",
      " False  True False False False False False False False False False False\n",
      " False False False False False False  True False False False  True False\n",
      " False False False False False False  True False False False False False\n",
      " False  True False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False  True False False  True False False False\n",
      " False False False False False False False False False  True False  True\n",
      " False False False  True False False False False False False False False\n",
      " False False False False False  True False  True False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False  True  True False False False False False\n",
      "  True False False False]\n"
     ]
    }
   ],
   "source": [
    "x_train_vt = vt.fit_transform(x_train, y_train)\n",
    "x_test_vt = vt.fit_transform(x_test,y_test)\n",
    "\n",
    "print(vt.get_support())\n",
    "\n",
    "#x_train_rfecv = rfecv.fit_transform(x_train, y_train)\n",
    "#x_train_selbest = selbest.fit_transform(x_train,y_train)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(52397, 68)\n"
     ]
    }
   ],
   "source": [
    "print(x_train_vt.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "model_rfg = RandomForestRegressor(n_estimators=25)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fit Model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n",
       "           max_features='auto', max_leaf_nodes=None,\n",
       "           min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "           min_samples_leaf=1, min_samples_split=2,\n",
       "           min_weight_fraction_leaf=0.0, n_estimators=25, n_jobs=1,\n",
       "           oob_score=False, random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_rfg.fit(x_train_vt,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reference method: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.explained_variance_score.html#sklearn.metrics.explained_variance_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.52516145  0.36663188 -0.26089937]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "import numpy as np \n",
    "y = np.array(y_test)\n",
    "x = np.array(x_test)\n",
    "print(cross_val_score(model_rfg,x_test_vt,y_test,scoring='r2'))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['randonforest_and_variancthreshold.sav']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.externals import joblib\n",
    "\n",
    "joblib.dump(model_rfg,'randonforest_and_variancthreshold.sav')\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ml",
   "language": "python",
   "name": "ml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}