08B - Using Automated Machine Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Automated Machine Learning\n",
    "\n",
    "There are many kinds of machine learning algorithm that you can use to train a model, and sometimes it's not easy to determine the most effective algorithm for your particular data and prediction requirements. Additionally, you can significantly affect the predictive performance of a model by preprocessing the training data, using techniques such as normalization, missing feature imputation, and others. In your quest to find the *best* model for your requirements, you may need to try many combinations of algorithms and preprocessing transformations; which takes a lot of time and compute resources.\n",
    "\n",
    "Azure Machine Learning enables you to automate the comparison of models trained using different algorithms and preprocessing options. You can use the visual interface in [Azure Machine Learning studio](https://ml/azure.com) or the SDK to leverage this capability. he SDK gives you greater control over the settings for the automated machine learning experiment, but the visual interface is easier to use. In this lab, you'll explore automated machine learning using the SDK.\n",
    "\n",
    "## Connect to Your Workspace\n",
    "\n",
    "The first thing you need to do is to connect to your workspace using the Azure ML SDK.\n",
    "\n",
    "> **Note**: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare Data for Automated Machine Learning\n",
    "\n",
    "You don't need to create a training script for automated machine learning, but you do need to load the training data. In this case, you'll create a dataset containing details of diabetes patients (just as you did in previous labs), and then split this into two datasets: one for training, and another for model validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Dataset\n",
    "\n",
    "default_ds = ws.get_default_datastore()\n",
    "\n",
    "if 'diabetes dataset' not in ws.datasets:\n",
    "    default_ds.upload_files(files=['./data/diabetes.csv', './data/diabetes2.csv'], # Upload the diabetes csv files in /data\n",
    "                        target_path='diabetes-data/', # Put it in a folder path in the datastore\n",
    "                        overwrite=True, # Replace existing files of the same name\n",
    "                        show_progress=True)\n",
    "\n",
    "    #Create a tabular dataset from the path on the datastore (this may take a short while)\n",
    "    tab_data_set = Dataset.Tabular.from_delimited_files(path=(default_ds, 'diabetes-data/*.csv'))\n",
    "\n",
    "    # Register the tabular dataset\n",
    "    try:\n",
    "        tab_data_set = tab_data_set.register(workspace=ws, \n",
    "                                name='diabetes dataset',\n",
    "                                description='diabetes data',\n",
    "                                tags = {'format':'CSV'},\n",
    "                                create_new_version=True)\n",
    "        print('Dataset registered.')\n",
    "    except Exception as ex:\n",
    "        print(ex)\n",
    "else:\n",
    "    print('Dataset already registered.')\n",
    "\n",
    "\n",
    "# Split the dataset into training and validation subsets\n",
    "diabetes_ds = ws.datasets.get(\"diabetes dataset\")\n",
    "train_ds, test_ds = diabetes_ds.random_split(percentage=0.7, seed=123)\n",
    "print(\"Data ready!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configure Automated Machine Learning\n",
    "\n",
    "Now you're ready to configure the automated machine learning experiment. To do this, you'll need an AutoML configuration that specifies options like the data to use, how many combinations to try, which metric to use when evaluating models, and so on.\n",
    "\n",
    "> **Note**: In this example, you'll restrict the experiment to 6 iterations to reduce the amount of time taken. In reality, you'd likely try many more iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.automl import AutoMLConfig\n",
    "\n",
    "automl_config = AutoMLConfig(name='Automated ML Experiment',\n",
    "                             task='classification',\n",
    "                             compute_target='local',\n",
    "                             training_data = train_ds,\n",
    "                             validation_data = test_ds,\n",
    "                             label_column_name='Diabetic',\n",
    "                             iterations=6,\n",
    "                             primary_metric = 'AUC_weighted',\n",
    "                             max_concurrent_iterations=4,\n",
    "                             featurization='auto'\n",
    "                             )\n",
    "\n",
    "print(\"Ready for Auto ML run.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run an Automated Machine Learning Experiment\n",
    "\n",
    "OK, you're ready to go. Let's run the automated machine learning experiment.\n",
    "\n",
    "> **Note**: This may take some time!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from azureml.core.experiment import Experiment\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "print('Submitting Auto ML experiment...')\n",
    "automl_experiment = Experiment(ws, 'diabetes_automl')\n",
    "automl_run = automl_experiment.submit(automl_config)\n",
    "RunDetails(automl_run).show()\n",
    "automl_run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Determine the Best Performing Model\n",
    "\n",
    "When the experiment has completed, view the output in the widget, and click the run that produced the best result to see its details.\n",
    "Then click the link to view the experiment details in the Azure portal and view the overall experiment details before viewing the details for the individual run that produced the best result. There's lots of information here about the performance of the model generated.\n",
    "\n",
    "Let's get the best run and the model that it produced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_run, fitted_model = automl_run.get_output()\n",
    "print(best_run)\n",
    "print(fitted_model)\n",
    "best_run_metrics = best_run.get_metrics()\n",
    "for metric_name in best_run_metrics:\n",
    "    metric = best_run_metrics[metric_name]\n",
    "    print(metric_name, metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Automated machine learning includes the option to try preprocessing the data, which is accomplished through the use of [Scikit-Learn transformation pipelines](https://scikit-learn.org/stable/modules/compose.html#combining-estimators) (not to be confused with Azure Machine Learning pipelines!). These produce models that include steps to transform the data before inferencing. You can view the steps in a model like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for step in fitted_model.named_steps:\n",
    "    print(step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, having found the best performing model, you can register it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register model\n",
    "best_run.register_model(model_path='outputs/model.pkl', model_name='diabetes_model_automl',\n",
    "                        tags={'Training context':'Auto ML'},\n",
    "                        properties={'AUC': best_run_metrics['AUC_weighted'], 'Accuracy': best_run_metrics['accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **More Information**: For more information Automated Machine Learning, see the [Azure ML documentation](https://docs.microsoft.com/azure/machine-learning/how-to-configure-auto-train)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}