003.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPHLVMk6a7ApO05f2xXlzS3",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/johnsanterre/santerreAI/blob/main/003.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Tutorial 3: Cross-Validation\n",
        "\n",
        "Cross-validation is a technique for evaluating the performance of a machine learning algorithm. It helps to prevent overfitting, which is when a model performs well on the training data but poorly on unseen data. Cross-validation involves splitting the data into a number of folds, training the model on some of the folds, and evaluating it on the remaining folds. This process is repeated a number of times, with different folds being used as the evaluation set each time. The performance of the model is then averaged across all of the folds.\n",
        "Here is a tutorial on how to perform cross-validation in Python:\n",
        "First, we will start by loading the necessary libraries and creating some sample data. For this tutorial, we will use the load_diabetes function from the sklearn.datasets module to load a sample dataset.\n"
      ],
      "metadata": {
        "id": "HofU0cZad9aw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.datasets import load_diabetes\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "from sklearn.model_selection import cross_val_score\n",
        "\n",
        "X, y = load_diabetes(return_X_y=True)"
      ],
      "metadata": {
        "id": "31-wd-0Td8uv"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we will create a model that we want to evaluate using cross-validation. For this tutorial, we will use a random forest regressor."
      ],
      "metadata": {
        "id": "o687NlnweCpH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = RandomForestRegressor()"
      ],
      "metadata": {
        "id": "TQiTc5y2eFAr"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, we can use the cross_val_score function from the sklearn.model_selection module to perform cross-validation. The cross_val_score function takes the model, the input data, and the target data as input, and returns an array of evaluation scores. By default, the cross_val_score function uses a 5-fold cross-validation, but this can be changed using the cv parameter."
      ],
      "metadata": {
        "id": "k-CUvKf8eHTR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "scores = cross_val_score(model, X, y)\n",
        "print(scores)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HMIwMYoFeJjD",
        "outputId": "2595a5a7-c9ff-49fe-eb1a-2d6a89950e0c"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0.38502019 0.50405881 0.43881749 0.37877011 0.40456854]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The scores array contains the evaluation scores for each fold of the cross-validation. We can calculate the mean and standard deviation of these scores to get a sense of the model's performance.\n",
        "Finally, we can use the mean and standard deviation to get a sense of the model's performance. If the mean score is high and the standard deviation is low, it means that the model is performing well and is consistent across different folds. If the standard deviation is high, it means that the model's performance is more variable across different folds, which could indicate overfitting or underfitting."
      ],
      "metadata": {
        "id": "NJqziIm7eMYk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(f\"Mean: {scores.mean()}\")\n",
        "print(f\"Standard Deviation: {scores.std()}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tJOip0eleQMT",
        "outputId": "05584ec9-2d1a-4aa1-c800-b173853cd463"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mean: 0.42224702744364784\n",
            "Standard Deviation: 0.045948572219829495\n"
          ]
        }
      ]
    }
  ]
}