titanic-machine-learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Titanic: Machine Learning from Disaster\n",
    "\n",
    "Description of the challenge [here](https://www.kaggle.com/c/titanic)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our data is in .csv format. Each row represents a passenger on the titanic, and some information about them. Let's take a look at the columns:\n",
    "\n",
    " - PassengerId -- A numerical id assigned to each passenger.\n",
    " - Survived -- Whether the passenger survived (1), or didn't (0). We'll be making predictions for this column.\n",
    " - Pclass -- The class the passenger was in -- first class (1), second class (2), or third class (3).\n",
    " - Name -- the name of the passenger.\n",
    " - Sex -- The gender of the passenger -- male or female.\n",
    " - Age -- The age of the passenger. Fractional.\n",
    " - SibSp -- The number of siblings and spouses the passenger had on board.\n",
    " - Parch -- The number of parents and children the passenger had on board.\n",
    " - Ticket -- The ticket number of the passenger.\n",
    " - Fare -- How much the passenger paid for the ticker.\n",
    " - Cabin -- Which cabin the passenger was in.\n",
    " - Embarked -- Where the passenger boarded the Titanic.\n",
    " \n",
    "A good first step is to think logically about the columns and what we're trying to predict. What variables might logically affect the outcome of survived? (reading more about the titanic might help here).\n",
    "\n",
    "**Exercise.** Discuss which variables are more likely to have had an impact on the survival odds of the passengers. \n",
    "\n",
    "## Looking at the data\n",
    "\n",
    "Using your recent knowledge of pandas, load the train data into a dataframe and perform some first basic exploratory operations on it to make yourself aquainted with its features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Handling missing data\n",
    "\n",
    "You might have noticed that not every field for all of the records contains data. This means that we will need to deal with handling missing data. There are many strategies for cleaning up missing data, but a simple one is to just fill in all the missing values with the median of all the values in the column. (Question: why the median and not the average?)\n",
    "\n",
    "Implement a solution to handle missing data on the Titanic dataframe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Non-numeric columns\n",
    "\n",
    "Several of our columns are non-numeric, which is a problem when it comes time to make predictions -- we can't feed non-numeric columns into a machine learning algorithm and expect it to make sense of them.\n",
    "\n",
    "We have to either exclude our non-numeric columns when we train our algorithm (Name, Sex, Cabin, Embarked, and Ticket), or find a way to convert them to numeric columns.\n",
    "\n",
    "Decide which non-numeric columns to keep for the analysis, based on your domain knowledge of the problem. Then, turn them into numeric values (categorical variables)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cross validation\n",
    "\n",
    "We want to train the algorithm on different data than we make predictions on. This is critical if we want to avoid overfitting. Overfitting is what happens when a model fits itself to \"noise\", not signal. Every dataset has its own quirks that don't exist in the full population. For example, if I asked you to predict the top speed of a car from its horsepower and other characteristics, and gave you a dataset that randomly had cars with very high top speeds, you would create a model that overstated speed. The way to figure out if your model is doing this is to evaluate its performance on data it hasn't been trained using.\n",
    "\n",
    "Every machine learning algorithm can overfit, although some are much less prone to it. If you evaluate your algorithm on the same dataset that you train it on, it's impossible to know if it's performing well because it overfit itself to the noise, or if it actually is a good algorithm.\n",
    "\n",
    "Luckily, cross validation is a simple way to avoid overfitting. To cross validate, you split your data into some number of parts (or \"folds\"). Lets use 3 as an example. You then do this:\n",
    "\n",
    " - Combine the first two parts, train a model, make predictions on the third.\n",
    " - Combine the first and third parts, train a model, make predictions on the second.\n",
    " - Combine the second and third parts, train a model, make predictions on the first.\n",
    "\n",
    "This way, we generate predictions for the whole dataset without ever evaluating accuracy on the same data we train our model using.\n",
    "\n",
    "Familiarize yourself with cross-validation in scikit-learn reading about [KFold](http://scikit-learn.org/stable/modules/generated/sklearn.cross_validation.KFold.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying scikit-learn ML algorithms to the Titanic dataset\n",
    "\n",
    " - Follow the scikit-learn API and apply the ML algorithm of your choice to a subset of the Titanic dataset.\n",
    " - Using the wrappers described [here](http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation), compute cross-validated metrics for said algorithm\n",
    " - If your algorithm takes various parameters as input, consider tuning them to improve performance\n",
    " - Repeat with various algorithms and compare its performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating new features\n",
    "\n",
    "If your data is rich, sometimes new features can be extracted from it that can better inform your machine learning algorithm. You can try generating new features from the name of the passengers, the members of their families present, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding the best features\n",
    "\n",
    "Feature engineering is the most important part of any machine learning task, and there are lots more features we could calculate. But we also need a way to figure out which features are the best.\n",
    "\n",
    "One way to do this is to use univariate feature selection. This essentially goes column by column, and figures out which columns correlate most closely with what we're trying to predict (Survived).\n",
    "\n",
    "As usual, scikit-learn has a function that will help us with feature selection, [SelectKBest](http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectKBest.html). This selects the best features from the data, and allows us to specify how many it selects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}