Skip to content

Commit 0514eee

Browse files
rastalarastala
authored
Merge pull request Azure#182 from rastala/master
version 1.0.10
2 parents 06aba38 + 4b6e34f commit 0514eee

File tree

43 files changed

+703
-277
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+703
-277
lines changed

‎NBSETUP.md

Lines changed: 16 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
1-
# Setting up environment
1+
# Notebook setup
22

33
---
44

5-
To run the notebooks in this repository use one of the two options.
5+
To run the notebooks in this repository use one of these methods:
66

7-
## Option 1: Use Azure Notebooks
8-
Azure Notebooks is a hosted Jupyter-based notebook service in the Azure cloud. Azure Machine Learning Python SDK is already pre-installed in the Azure Notebooks `Python 3.6` kernel.
7+
## Use Azure Notebooks - Jupyter based notebooks in the Azure cloud
98

109
1. [![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://aka.ms/aml-clone-azure-notebooks)
1110
[Import sample notebooks ](https://aka.ms/aml-clone-azure-notebooks) into Azure Notebooks
@@ -16,40 +15,20 @@ Azure Notebooks is a hosted Jupyter-based notebook service in the Azure cloud. A
1615

1716
![set kernel to Python 3.6](images/python36.png)
1817

19-
## **Option 2: Use your own notebook server**
18+
## **Use your own notebook server**
2019

21-
### Quick installation
22-
We recommend you create a Python virtual environment ([Miniconda](https://conda.io/miniconda.html) preferred but [virtualenv](https://virtualenv.pypa.io/en/latest/) works too) and install the SDK in it.
23-
```sh
24-
# install just the base SDK
25-
pip install azureml-sdk
26-
27-
# clone the sample repoistory
28-
git clone https://github.com/Azure/MachineLearningNotebooks.git
29-
30-
# below steps are optional
31-
# install the base SDK and a Jupyter notebook server
32-
pip install azureml-sdk[notebooks]
33-
34-
# install the data prep component
35-
pip install azureml-dataprep
36-
37-
# install model explainability component
38-
pip install azureml-sdk[explain]
39-
40-
# install automated ml components
41-
pip install azureml-sdk[automl]
42-
43-
# install experimental features (not ready for production use)
44-
pip install azureml-sdk[contrib]
45-
```
46-
47-
### Full instructions
48-
[Install the Azure Machine Learning SDK](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-create-workspace-with-python)
49-
50-
Please make sure you start with the [Configuration](configuration.ipynb) notebook to create and connect to a workspace.
20+
Video walkthrough:
5121

22+
[![Get Started video](images/yt_cover.png)](https://youtu.be/VIsXeTuW3FU)
5223

53-
### Video walkthrough:
24+
1. Setup a Jupyter Notebook server and [install the Azure Machine Learning SDK](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-create-workspace-with-python)
25+
1. Clone [this repository](https://aka.ms/aml-notebooks)
26+
1. You may need to install other packages for specific notebook
27+
- For example, to run the Azure Machine Learning Data Prep notebooks, install the extra dataprep SDK:
28+
```bash
29+
pip install azureml-dataprep
30+
```
5431

55-
[![Get Started video](images/yt_cover.png)](https://youtu.be/VIsXeTuW3FU)
32+
1. Start your notebook server
33+
1. Follow the instructions in the [Configuration](configuration.ipynb) notebook to create and connect to a workspace
34+
1. Open one of the sample notebooks

‎README.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,5 +55,4 @@ The [How to use Azure ML](./how-to-use-azureml) folder contains specific example
5555
Visit following repos to see projects contributed by Azure ML users:
5656

5757
- [Fine tune natural language processing models using Azure Machine Learning service](https://github.com/Microsoft/AzureML-BERT)
58-
- [Fashion MNIST with Azure ML SDK](https://github.com/amynic/azureml-sdk-fashion)
59-
58+
- [Fashion MNIST with Azure ML SDK](https://github.com/amynic/azureml-sdk-fashion)

‎configuration.ipynb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@
9696
"source": [
9797
"import azureml.core\n",
9898
"\n",
99-
"print(\"This notebook was created using version 1.0.6 of the Azure ML SDK\")\n",
99+
"print(\"This notebook was created using version 1.0.10 of the Azure ML SDK\")\n",
100100
"print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
101101
]
102102
},
@@ -373,4 +373,4 @@
373373
},
374374
"nbformat": 4,
375375
"nbformat_minor": 2
376-
}
376+
}

‎how-to-use-azureml/automated-machine-learning/automl_setup.cmd

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
@echo off
22
set conda_env_name=%1
33
set automl_env_file=%2
4+
set options=%3
45
set PIP_NO_WARN_SCRIPT_LOCATION=0
56

67
IF "%conda_env_name%"=="" SET conda_env_name="azure_automl"
@@ -32,10 +33,12 @@ echo.
3233
echo ***************************************
3334
echo * AutoML setup completed successfully *
3435
echo ***************************************
35-
echo.
36-
echo Starting jupyter notebook - please run the configuration notebook
37-
echo.
38-
jupyter notebook --log-level=50 --notebook-dir='..\..'
36+
IF NOT "%options%"=="nolaunch" (
37+
echo.
38+
echo Starting jupyter notebook - please run the configuration notebook
39+
echo.
40+
jupyter notebook --log-level=50 --notebook-dir='..\..'
41+
)
3942

4043
goto End
4144

‎how-to-use-azureml/automated-machine-learning/automl_setup_linux.sh

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
CONDA_ENV_NAME=$1
44
AUTOML_ENV_FILE=$2
5+
OPTIONS=$3
56
PIP_NO_WARN_SCRIPT_LOCATION=0
67

78
if [ "$CONDA_ENV_NAME" == "" ]
@@ -34,10 +35,13 @@ else
3435
echo "***************************************" &&
3536
echo "* AutoML setup completed successfully *" &&
3637
echo "***************************************" &&
37-
echo "" &&
38-
echo "Starting jupyter notebook - please run the configuration notebook" &&
39-
echo "" &&
40-
jupyter notebook --log-level=50 --notebook-dir '../..'
38+
if [ "$OPTIONS" != "nolaunch" ]
39+
then
40+
echo "" &&
41+
echo "Starting jupyter notebook - please run the configuration notebook" &&
42+
echo "" &&
43+
jupyter notebook --log-level=50 --notebook-dir '../..'
44+
fi
4145
fi
4246

4347
if [ $? -gt 0 ]

‎how-to-use-azureml/automated-machine-learning/automl_setup_mac.sh

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
CONDA_ENV_NAME=$1
44
AUTOML_ENV_FILE=$2
5+
OPTIONS=$3
56
PIP_NO_WARN_SCRIPT_LOCATION=0
67

78
if [ "$CONDA_ENV_NAME" == "" ]
@@ -36,10 +37,13 @@ else
3637
echo "***************************************" &&
3738
echo "* AutoML setup completed successfully *" &&
3839
echo "***************************************" &&
39-
echo "" &&
40-
echo "Starting jupyter notebook - please run the configuration notebook" &&
41-
echo "" &&
42-
jupyter notebook --log-level=50 --notebook-dir '../..'
40+
if [ "$OPTIONS" != "nolaunch" ]
41+
then
42+
echo "" &&
43+
echo "Starting jupyter notebook - please run the configuration notebook" &&
44+
echo "" &&
45+
jupyter notebook --log-level=50 --notebook-dir '../..'
46+
fi
4347
fi
4448

4549
if [ $? -gt 0 ]

‎how-to-use-azureml/automated-machine-learning/forecasting-energy-demand/auto-ml-forecasting-energy-demand.ipynb

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,31 @@
4444
"cell_type": "markdown",
4545
"metadata": {},
4646
"source": [
47-
"## Setup\n",
48-
"\n",
49-
"As part of the setup you have already created a <b>Workspace</b>. For AutoML you would need to create an <b>Experiment</b>. An <b>Experiment</b> is a named object in a <b>Workspace</b>, which is used to run experiments."
47+
"## Setup\n"
48+
]
49+
},
50+
{
51+
"cell_type": "markdown",
52+
"metadata": {},
53+
"source": [
54+
"To use the *forecasting* task in AutoML, you need to have the **azuremlftk** package installed in your environment. The following cell tests whether this package is installed locally and, if not, gives you instructions for installing it. "
55+
]
56+
},
57+
{
58+
"cell_type": "code",
59+
"execution_count": null,
60+
"metadata": {},
61+
"outputs": [],
62+
"source": [
63+
"try:\n",
64+
" import ftk\n",
65+
" print('Using FTK version ' + ftk.__version__)\n",
66+
"except ImportError:\n",
67+
" print(\"Unable to import forecasting package. This notebook does not work without this package.\\n\"\n",
68+
" + \"Please open a command prompt and run `pip install azuremlftk` to install the package. \\n\"\n",
69+
" + \"Make sure you install the package into AutoML's Python environment.\\n\\n\"\n",
70+
" + \"For instance, if AutoML is installed in a conda environment called `python36`, run:\\n\"\n",
71+
" + \"> activate python36\\n> pip install azuremlftk\")"
5072
]
5173
},
5274
{
@@ -71,6 +93,13 @@
7193
"from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score"
7294
]
7395
},
96+
{
97+
"cell_type": "markdown",
98+
"metadata": {},
99+
"source": [
100+
"As part of the setup you have already created a <b>Workspace</b>. For AutoML you would need to create an <b>Experiment</b>. An <b>Experiment</b> is a named object in a <b>Workspace</b>, which is used to run experiments."
101+
]
102+
},
74103
{
75104
"cell_type": "code",
76105
"execution_count": null,
@@ -368,7 +397,7 @@
368397
"name": "python",
369398
"nbconvert_exporter": "python",
370399
"pygments_lexer": "ipython3",
371-
"version": "3.6.6"
400+
"version": "3.6.8"
372401
}
373402
},
374403
"nbformat": 4,

‎how-to-use-azureml/automated-machine-learning/forecasting-orange-juice-sales/auto-ml-forecasting-orange-juice-sales.ipynb

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,31 @@
4545
"cell_type": "markdown",
4646
"metadata": {},
4747
"source": [
48-
"## Setup\n",
49-
"\n",
50-
"As part of the setup you have already created a <b>Workspace</b>. To run AutoML, you also need to create an <b>Experiment</b>. An Experiment is a named object in a Workspace which represents a predictive task, the output of which is a trained model and a set of evaluation metrics for the model. "
48+
"## Setup"
49+
]
50+
},
51+
{
52+
"cell_type": "markdown",
53+
"metadata": {},
54+
"source": [
55+
"To use the *forecasting* task in AutoML, you need to have the **azuremlftk** package installed in your environment. The following cell tests whether this package is installed locally and, if not, gives you instructions for installing it."
56+
]
57+
},
58+
{
59+
"cell_type": "code",
60+
"execution_count": null,
61+
"metadata": {},
62+
"outputs": [],
63+
"source": [
64+
"try:\n",
65+
" import ftk\n",
66+
" print('Using FTK version ' + ftk.__version__)\n",
67+
"except ImportError:\n",
68+
" print(\"Unable to import forecasting package. This notebook does not work without this package.\\n\"\n",
69+
" + \"Please open a command prompt and run `pip install azuremlftk` to install the package. \\n\"\n",
70+
" + \"Make sure you install the package into AutoML's Python environment.\\n\\n\"\n",
71+
" + \"For instance, if AutoML is installed in a conda environment called `python36`, run:\\n\"\n",
72+
" + \"> activate python36\\n> pip install azuremlftk\")"
5173
]
5274
},
5375
{
@@ -71,6 +93,13 @@
7193
"from sklearn.metrics import mean_absolute_error, mean_squared_error"
7294
]
7395
},
96+
{
97+
"cell_type": "markdown",
98+
"metadata": {},
99+
"source": [
100+
"As part of the setup you have already created a <b>Workspace</b>. To run AutoML, you also need to create an <b>Experiment</b>. An Experiment is a named object in a Workspace which represents a predictive task, the output of which is a trained model and a set of evaluation metrics for the model. "
101+
]
102+
},
74103
{
75104
"cell_type": "code",
76105
"execution_count": null,
@@ -404,7 +433,7 @@
404433
"name": "python",
405434
"nbconvert_exporter": "python",
406435
"pygments_lexer": "ipython3",
407-
"version": "3.6.6"
436+
"version": "3.6.8"
408437
}
409438
},
410439
"nbformat": 4,

‎how-to-use-azureml/automated-machine-learning/remote-execution/auto-ml-remote-execution.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@
150150
" dsvm_compute = DsvmCompute.create(ws, name = dsvm_name, provisioning_configuration = dsvm_config)\n",
151151
" dsvm_compute.wait_for_completion(show_output = True)\n",
152152
" print(\"Waiting one minute for ssh to be accessible\")\n",
153-
" time.sleep(60) # Wait for ssh to be accessible"
153+
" time.sleep(90) # Wait for ssh to be accessible"
154154
]
155155
},
156156
{

‎how-to-use-azureml/azure-databricks/README.md

Lines changed: 1 addition & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -10,52 +10,7 @@ In this section, you will find sample notebooks on how to use Azure Machine Lear
1010
- Every run (including the best run) is available as a pipeline, which you can tune further if needed.
1111
- The model trained using Azure Databricks can be registered in Azure ML SDK workspace and then deployed to Azure managed compute (ACI or AKS) using the Azure Machine learning SDK.
1212

13-
14-
**Create Azure Databricks Cluster:**
15-
16-
Select New Cluster and fill in following detail:
17-
- Cluster name: _yourclustername_
18-
- Databricks Runtime: Any **non ML** runtime (non ML 4.x, 5.x)
19-
- Python version: **3**
20-
- Workers: 2 or higher.
21-
22-
These settings are only for using Automated Machine Learning on Databricks.
23-
- Max. number of **concurrent iterations** in Automated ML settings is **<=** to the number of **worker nodes** in your Databricks cluster.
24-
- Worker node VM types: **Memory optimized VM** preferred.
25-
- Uncheck _Enable Autoscaling_
26-
27-
28-
It will take few minutes to create the cluster. Please ensure that the cluster state is running before proceeding further.
29-
30-
**Install Azure ML SDK without Automated ML capability on your Azure Databricks cluster**
31-
32-
- Select Import library
33-
34-
- Source: Upload Python Egg or PyPI
35-
36-
- PyPi Name: **azureml-sdk[databricks]**
37-
38-
**Install Azure ML with Automated ML SDK on your Azure Databricks cluster**
39-
40-
- Select Import library
41-
42-
- Source: Upload Python Egg or PyPI
43-
44-
- PyPi Name: **azureml-sdk[automl_databricks]**
45-
46-
**For installation with or without Automated ML**
47-
48-
- Click Install Library
49-
50-
- Do not select _Attach automatically to all clusters_. In case you selected this earlier, please go to your Home folder and deselect it.
51-
52-
- Select the check box _Attach_ next to your cluster name
53-
54-
(More details on how to attach and detach libs are here - [https://docs.databricks.com/user-guide/libraries.html#attach-a-library-to-a-cluster](https://docs.databricks.com/user-guide/libraries.html#attach-a-library-to-a-cluster) )
55-
56-
- Ensure that there are no errors until Status changes to _Attached_. It may take a couple of minutes.
57-
58-
**Note** - If you have an old SDK version, please deselect it from cluster’s installed libs > move to trash. Install the new SDK verdion and restart the cluster. If there is an issue after this, please detach and reattach your cluster.
13+
Please follow our [Azure doc](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-environment#azure-databricks) to install the sdk in your Azure Databricks cluster before trying any of the sample notebooks.
5914

6015
**Single file** -
6116
The following archive contains all the sample notebooks. You can the run notebooks after importing [DBC](Databricks_AMLSDK_1-4_6.dbc) in your Databricks workspace instead of downloading individually.

0 commit comments

Comments
 (0)