ml_projects.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to Approach a Machine Learning Project \n",
    "\n",
    "Let's explore a step-by-step process for approaching ML problems in Real life:\n",
    "\n",
    "1. Understanding the Business Requirements and the Nature of the Available Data.\n",
    "2. Classify the problem as Supervised/Unsupervised as well as Regression/Classification (in advance).\n",
    "3. Download, Clean & Explore data and Create New Features (if required) that may improve models.\n",
    "4. Create Training/Validation/Test sets of data and prepare the data for training ML models.\n",
    "5. Create a quick & easy baseline model to evaluate and benchmark future models.\n",
    "6. Select a modeling strategy, train a model, and tune hyperparameters to achieve optimal fit.\n",
    "7. Experiment and combine results from multiple strategies to get a better result.\n",
    "8. Interpret models, study individual predictions, and present your findings to the stakeholders. \n",
    "\n",
    "## Step 1 - Understand Business Requirements & Nature of Data\n",
    "\n",
    "Most machine learning models are trained to serve a real-world use case. It's important to understand the business requirements, modeling objectives and the nature of the data available before you start building a machine learning model.\n",
    "\n",
    "### Understanding the Big Picture\n",
    "\n",
    "The first step in any machine learning problem is to read the given documentation, talk to various stakeholders and identify the following:\n",
    "\n",
    "1. What business problem are you trying to solve with machine learning?\n",
    "2. Why do we want to solve this problem, and how will it benefit the business?\n",
    "3. How is this problem currently addressed without using machine learning tools?\n",
    "4. Can we see some sample rows from the dataset? Do they accurately represent the entire dataset?\n",
    "5. Where is the data stored, and how will you access it?\n",
    "6. How much historical data do we have, and how was it collected?\n",
    "7. What information/features does the historical data include? Does it have the past values of the feature for what we're trying to predict?\n",
    "8. Are there any known issues with the data, like mistakes in data entry, missing information, or differences in units?\n",
    "9. Who will use the results of the model, and how does it fit into other business processes?\n",
    "10. Are there any privacy or security considerations with the data, and how will they be addressed?\n",
    "11. ...\n",
    "\n",
    "Gather as much information about the problem as possible, so that you're clear understanding of the objective and feasibility of the project.\n",
    "\n",
    "## Step 2 - Classify the problem as supervised/unsupervised & regression/classification\n",
    "\n",
    "If you're new to Data Science, make sure to visit ([this](https://vas3k.com/blog/machine_learning/))  to upgrade your skills.\n",
    "\n",
    "And if you're already an expert, make sure not to overlook enhancing your skills by checking out this ([Machine Learning in 10 minutes Blog](https://medium.datadriveninvestor.com/machine-learning-in-10-minutes-354d83e5922e))\n",
    "\n",
    "### Loss Functions and Evaluation Metrics\n",
    "\n",
    "Once you've figured out the type of problem you're solving, it's crucial to choose an appropriate evaluation metric. Also, depending on the kind of model you train, your model will also use a loss/cost function to optimize during the training process.\n",
    "\n",
    "* **Evaluation metrics** - used by industry experts to evaluate the machine learning model\n",
    "\n",
    "* **Loss functions** - used by computers to fine-tune/optimize the ML model during the training process\n",
    "\n",
    "They are often the same (e.g. RMSE for regression problems), but they can be different for instance, Classification problems could use Cross Entropy for the loss function and Accuracy for the evaluation metric.\n",
    "\n",
    "Check out this article for a comprehensive overview of common loss functions and evaluation metrics: ([11 Evaluation Metrics Data Scientists Should Be Familiar with Blog](https://towardsdatascience.com/11-evaluation-metrics-data-scientists-should-be-familiar-with-lessons-from-a-high-rank-kagglers-8596f75e58a7))\n",
    "\n",
    "## Step 3 - Download, clean & explore the data and create new features\n",
    "\n",
    "### Downloading Data\n",
    "\n",
    "There may be different sources to get the data:\n",
    "\n",
    "* CSV files\n",
    "* SQL databases\n",
    "* Raw File URLs\n",
    "* Kaggle datasets \n",
    "* Google Drive\n",
    "* Dropbox\n",
    "* etc.\n",
    "\n",
    "Identify the right tool/library to get the data. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploratory Data Analysis and Visualization\n",
    "\n",
    "Objectives of exploratory data analysis:\n",
    "\n",
    "- Study the patterns/distributions/spread of individual columns (uniform, normal, exponential)\n",
    "- Identify any unusual or incorrect data points(e.g. missing/incorrect values)\n",
    "- Explore how the target column relates to other columns, (linear, non-linear etc.)\n",
    "- Gather valuable insights about  both the problem you're solving and the dataset itself\n",
    "- Basically, come up with ideas for Preprocessing and Feature Engineering of data\n",
    "\n",
    "### Feature Engineering\n",
    "\n",
    "Feature engineer is the process of creating new features (columns) by transforming/combining existing features or by incorporating data from external sources. \n",
    "\n",
    "For example, here are some features that can be extracted from the \"Date\" column:\n",
    "\n",
    "1. Day of week\n",
    "2. Day or month\n",
    "3. Month\n",
    "4. Year\n",
    "5. Weekend/Weekday\n",
    "6. Month/Quarter End\n",
    "\n",
    "Using date information, we can also create new current columns like: \n",
    "\n",
    "1. Weather on each day\n",
    "2. Whether the date was a public holiday\n",
    "3. Whether the store was running a promotion on that day\n",
    "4. Social media trends or online activities on specific dates\n",
    "5. Seasonal factors influencing sales or activities\n",
    "\n",
    "These additional columns can provide a more comprehensive understanding of the context surrounding the date information.\n",
    "\n",
    "\n",
    "## Step 4 - Create a training/validation/test split and prepare the data for training\n",
    "\n",
    "### Train/Test/Validation Split\n",
    "\n",
    "In data analysis, we split our dataset into three parts for effective model training and evaluation. The training set, typically 70-75% of the data, train the model. The test set (15-20%) checks how well the model generalizes to new, unseen data. Validation set (10-15%) helps fine-tune the model's parameters without influencing the test set, ensuring unbiased evaluation.\n",
    "\n",
    "This three-way split ensures our model is both trained well and performs reliably on real-world scenarios.\n",
    "\n",
    "### Input and Target columns\n",
    "\n",
    "Also identify input and target columns. We needn't use all the available columns, we can start out with just a small subset.\n",
    "\n",
    "Note: We can't use the no. of customers as an input, because this information isn't available beforehand. \n",
    "\n",
    "### Imputation, Scaling and Encode\n",
    "\n",
    "\n",
    "In data preparation, we often encounter missing values, differing scales, and categorical variables. \n",
    "\n",
    "- Imputation is the process of filling in missing data, ensuring a complete dataset. \n",
    "- Scaling involves adjusting numerical values to a common scale, preventing one variable from dominating others. \n",
    "- Encoding tackles categorical data, transforming it into numerical format for machine learning algorithms to comprehend. \n",
    "\n",
    "Note: We can apply a different imputation strategy to different columns depending on their distributions (e.g. mean for normally distribute and median for exponentially distributed).\n",
    "\n",
    "And also encode categorical columns as one-hot vectors.\n",
    "\n",
    "Explore the ([scikit-learn](https://scikit-learn.org/stable/modules/preprocessing.html) preprocessing module))\n",
    "\n",
    "## Step 5 - Create quick & easy baseline models to benchmark future models\n",
    "\n",
    "A quick baseline model helps establish the minimum score any ML model you train should achieve.\n",
    "\n",
    "### Fixed/Random Guess\n",
    "\n",
    "Let's define a model that always returns the mean value of Sales as the prediction.\n",
    "\n",
    "def return_mean(inputs):\n",
    "    return np.full(len(inputs), merged_df.Sales.mean())\n",
    "\n",
    "Let's evaluate this to using the RMSE score.\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "### Baseline ML model\n",
    "\n",
    "Let's train a simple `LinearRegression` model, with no customization.\n",
    "\n",
    "`model.fit` uses the following workflow for training the model ([From Basic Machine Learning to Deep Learning in 5 Minutes](https://www.deepnetts.com/blog/from-basic-machine-learning-to-deep-learning-in-5-minutes.html)):\n",
    "\n",
    "1. We initialize a model with random parameters (weights & biases).\n",
    "2. We pass some inputs into the model to obtain predictions.\n",
    "3. We compare the model's predictions with the actual targets using the loss function.  \n",
    "4. We use an optimization technique (like least squares, gradient descent etc.) to reduce the loss by adjusting the weights & biases of the model\n",
    "5. We repeat steps 1 to 4 till the predictions from the model are good enough.\n",
    "\n",
    "\n",
    "Note: A simple linear regression model isn't much better than our fixed baseline model which always predicts the mean.\n",
    "\n",
    "Based on the above baselines, we now know that any model we train should have ideally have a RMSE score lower than $2800. This baseline can also be conveyed to other stakeholders to get a sense of whether the range of loss makes sense. \n",
    "\n",
    "## Step 6 - Pick a strategy, train a model & tune hyperparameters\n",
    "\n",
    "### Systematically Exploring Modeling Strategies\n",
    "\n",
    "Scikit-learn offers the following cheatsheet to decide which model to pick.\n",
    "\n",
    "Here's the general strategy to follow:\n",
    "\n",
    "- Find out which models are applicable to the problem you're solving.\n",
    "- Train a basic version for each type of model that's applicable\n",
    "- Identify the modeling approaches that work well and tune their hypeparameters\n",
    "- Use a worksheet to keep track of your experiments and results.\n",
    "\n",
    "Let's define a function `try_model`, which takes a model, then performs training and evaluation.\n",
    "\n",
    "### Linear Models\n",
    "\n",
    "Read about ([linear models](https://scikit-learn.org/stable/modules/linear_model.html))\n",
    "\n",
    "from sklearn.linear_model import LinearRegression, Ridge(L2 Regularization), Lasso(L1 Regularization), ElasticNet, SGDRegressor\n",
    "\n",
    "\n",
    "### Tree Based Models\n",
    "\n",
    "* ([Decision Trees](https://scikit-learn.org/stable/modules/tree.html))\n",
    "* ([Random Forests and Gradient Boosting](https://scikit-learn.org/stable/modules/ensemble.html))\n",
    "\n",
    "### Other Supervised Learning Models\n",
    "\n",
    "Checkout ([Spervised Learning Models](https://scikit-learn.org/stable/supervised_learning.html))\n",
    "\n",
    "Try some other supervised learning algorithms and see if you can get a better result.\n",
    "\n",
    "### Unsupervised Learning Techniques\n",
    "\n",
    "Checkout ([Spervised Learning Models](https://scikit-learn.org/stable/unsupervised_learning.html))\n",
    "\n",
    "\n",
    "## Step 7 - Experiment and combine results from multiple strategies\n",
    "\n",
    "In general, the following strategies can be used to improve the performance of a model:\n",
    "\n",
    "- Gather more data. A greater amount of data can let you learn more relationships and generalize the model better.\n",
    "- Include more features. The more relevant the features for predicting the target, the better the model gets.\n",
    "- Tune the hyperparameters of the model. Increase the capacity of the model while ensuring that it doesn't overfit.\n",
    "- Look at the specific examples where the model make incorrect or bad predictions and gather some insights\n",
    "- Try strategies like grid search for hyperparameter optimization and K-fold cross validation\n",
    "- Combine results from different types of models (ensembling), or train another model using their results.\n",
    "\n",
    "### Hyperparameter Optimization & Grid Search\n",
    "\n",
    "You can hypertune parameters manually, our use an automated tuning strategy like random search or Grid search. Follow this tutorial for hyperparameter tuning using ([Grid search](https://machinelearningmastery.com/hyperparameter-optimization-with-random-search-and-grid-search/))\n",
    "   \n",
    "\n",
    "### K-Fold Cross Validation\n",
    "\n",
    "- Apply K fold cross validation for the random forest regressor trained earlier.\n",
    "\n",
    "Here's what ([K-fold cross validation](https://vitalflux.com/k-fold-cross-validation-python-example/)) looks like visually \n",
    "\n",
    "\n",
    "Follow this tutorial to apply ([K-fold cross validation] (https://machinelearningmastery.com/repeated-k-fold-cross-validation-with-python/))\n",
    "\n",
    "### Ensembling and Stacking\n",
    "\n",
    "Ensembling refers to combining the results of multiple models. Here's what ([ensembling](https://www.kdnuggets.com/2019/01/ensemble-learning-5-main-approaches.html)) looks like visually\n",
    "\n",
    "\n",
    "Stacking is a more advanced version of ensembling, where we train another model using the results from multiple models. Here's what ([stacking](https://medium.com/ml-research-lab/stacking-ensemble-meta-algorithms-for-improve-predictions-f4b4cf3b9237)) looks like visually \n",
    "\n",
    "\n",
    "Here's a tutorial on ([stacking] (https://machinelearningmastery.com/stacking-ensemble-machine-learning-with-python/))\n",
    "\n",
    "\n",
    "## Step 8 - Interpret models, study individual predictions & present your findings\n",
    "\n",
    "### Feature Importance\n",
    "\n",
    "You'll need to explain why your model returns a particular result. Most scikit-learn models offer some kind of \"feature importance\" score.\n",
    "\n",
    "rf.feature_importances_\n",
    "\n",
    "importance_df = pd.DataFrame({ \n",
    "    'feature': numeric_cols+encoded_cols, \n",
    "    'importance': rf.feature_importances_\n",
    "}).sort_values('importance', ascending=False)\n",
    "importance_df.head(10)\n",
    "\n",
    "sns.barplot(data=importance_df.head(10), x='importance', y='feature');\n",
    "\n",
    "The above chart can be presented to non-technical stakeholders to explain how the model arrives at its result. For greater explainability, a single decision tree can be used.\n",
    "\n",
    "### Presenting your results\n",
    "\n",
    "* Create a presentation for non-technical stakeholders\n",
    "* Understand your audience - figure out what they care about most\n",
    "* Skip the code or technical jargon, include visualizations\n",
    "* Highlight metrics important for the business.\n",
    "* Discuss about feature importance and how to interpret results\n",
    "* Explain the strengths and limitations of the model\n",
    "* Explain how the model can be improved over time\n",
    "\n",
    "### Model Deployment\n",
    "\n",
    "At this point, the model can be handed over to a software developer / ML engineer who can put the model into production, as part of an existing software system. It's important to monitor the results of the model, and make improvements from time to time.\n",
    "\n",
    "Check out this tutorial on how to deploy a model to the Heroku platform using the ([Flask framework]( https://towardsdatascience.com/create-an-api-to-deploy-machine-learning-models-using-flask-and-heroku-67a011800c50))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}