new sample - INC Quantization with PyTorch (#1550)

* new sample - INC Quantization with PyTorch

Signed-off-by: devpramod-intel <[email protected]>

* add additional instructions

* readme & json changes

---------

Signed-off-by: devpramod-intel <[email protected]>

Author: devpramod-intel

AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/.ipynb_checkpoints/quantize_with_inc-checkpoint.ipynb

@@ -0,0 +1 @@
+neural_compressor==2.1

AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/.ipynb_checkpoints/requirements-checkpoint.txt

@@ -0,0 +1,7 @@
+Copyright Intel Corporation
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/License.txt

@@ -0,0 +1,140 @@
+# `Getting Started with Intel® Neural Compressor for Quantization` Sample
+
+The sample is a getting started tutorial for the Intel® Neural Compressor (INC), and demonstrates how to perform INT8 quantization on a Hugging Face BERT model. This sample shows how to achieve performance boosts using Intel hardware.
+
+| Area                  | Description
+|:---                   |:---
+| What you will learn   | How to quantize a BERT model using Intel® Neural Compressor
+| Time to complete      | 20 minutes
+| Category              | Code Optimization
+
+## Purpose
+
+Intel® Neural Compressor comes with many options for deep learning model compression, one of them being INT8 Quantization. Quantization help to reduce the size of the model, which enables faster inference. The approach requires a trade-off in reduced accuracy for the reduced size; however, Intel® Neural Compressor provides automated accuracy-driven tuning recipes that will allow you to quantize your model and maintain your model accuracy goals.
+
+The sample starts by loading a BERT model from Hugging Face. After loading the model, we set up an evaluation function that we care about using PyTorch* Dataset and DataLoader classes. Using this evaluation function, Intel® Neural Compressor can perform both post training static and dynamic quantization to achieve the speedups.
+
+## Prerequisites
+
+| Optimized for           | Description
+|:---                     |:---
+| OS                      | Ubuntu* 20.04 (or newer)
+| Hardware                | Intel® Xeon® Scalable processor family
+| Software                | Intel® AI Analytics Toolkit (AI Kit)
+
+### For Local Development Environments
+
+You will need to download and install the following toolkits, tools, and components to use the sample.
+
+- **Intel® AI Analytics Toolkit (AI Kit)**
+
+  You can get the AI Kit from [Intel® oneAPI Toolkits](https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html#analytics-kit). <br> See [*Get Started with the Intel® AI Analytics Toolkit for Linux**](https://www.intel.com/content/www/us/en/develop/documentation/get-started-with-ai-linux) for AI Kit installation information and post-installation steps and scripts.
+
+- **Jupyter Notebook**
+
+  Install using PIP: `$pip install notebook`. <br> Alternatively, see [*Installing Jupyter*](https://jupyter.org/install) for detailed installation instructions.
+
+### For Intel® DevCloud
+
+The necessary tools and components are already installed in the environment. You do not need to install additional components. See [Intel® DevCloud for oneAPI](https://devcloud.intel.com/oneapi/get_started/) for information.
+
+### **Additional Packages**
+
+  You will need to install these additional packages in *requirements.txt*.
+  ```
+  python -m pip install -r requirements.txt
+  ```
+
+## Key Implementation Details
+
+The sample contains one Jupyter Notebook and one Python script.
+
+### Jupyter Notebook
+
+|Notebook                  |Description
+|:---                      |:---
+|`quantize_with_inc.ipynb` | Get started tutorial for using Intel® Neural Compressor for PyTorch*
+
+### Python Script
+
+|Script                    |Description
+|:---                      |:---
+|`dataset.py`              | The script provides a PyTorch* Dataset class that tokenizes text data 
+
+
+## Run the `Getting Started with Intel® Neural Compressor for Quantization` Sample
+
+> **Note**: If you have not already done so, set up your CLI
+> environment by sourcing  the `setvars` script in the root of your oneAPI installation.
+>
+> Linux*:
+> - For system wide installations: `. /opt/intel/oneapi/setvars.sh`
+> - For private installations: ` . ~/intel/oneapi/setvars.sh`
+> - For non-POSIX shells, like csh, use the following command: `bash -c 'source <install-dir>/setvars.sh ; exec csh'`
+>
+> For more information on configuring environment variables, see [Use the setvars Script with Linux* or macOS*](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-linux-or-macos.html).
+
+### On Linux*
+
+#### Activate Conda
+
+1. Activate the Conda environment.
+
+    ```
+    conda activate pytorch
+    ```
+
+   By default, the AI Kit is installed in the `/opt/intel/oneapi` folder and requires root privileges to manage it.
+
+   You can choose to activate Conda environment without root access. To bypass root access to manage your Conda environment, clone and activate your desired Conda environment using the following commands similar to the following.
+
+#### Run the NoteBook
+
+1. Launch Jupyter Notebook.
+   ```
+   jupyter notebook --ip=0.0.0.0
+   ```
+2. Follow the instructions to open the URL with the token in your browser.
+3. Locate and select the Notebook.
+   ```
+   optimize_pytorch_models_with_ipex.ipynb
+   ```
+4. Change the kernel to **pytorch**.
+5. Run every cell in the Notebook in sequence.
+
+#### Troubleshooting
+
+If you receive an error message, troubleshoot the problem using the **Diagnostics Utility for Intel® oneAPI Toolkits**. The diagnostic utility provides configuration and system checks to help find missing dependencies, permissions errors, and other issues. See the [Diagnostics Utility for Intel® oneAPI Toolkits User Guide](https://www.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html) for more information on using the utility.
+
+### Run the Sample on Intel® DevCloud (Optional)
+
+1. If you do not already have an account, request an Intel® DevCloud account at [*Create an Intel® DevCloud Account*](https://intelsoftwaresites.secure.force.com/DevCloud/oneapi).
+2. On a Linux* system, open a terminal.
+3. SSH into Intel® DevCloud.
+   ```
+   ssh DevCloud
+   ```
+   > **Note**: You can find information about configuring your Linux system and connecting to Intel DevCloud at Intel® DevCloud for oneAPI [Get Started](https://devcloud.intel.com/oneapi/get_started).
+
+4. Follow the instructions to open the URL with the token in your browser.
+3. Locate and select the Notebook.
+   ```
+   quantize_with_inc.ipynb
+   ```
+4. Change the kernel to **PyTorch (AI Kit)**.
+7. Run every cell in the Notebook in sequence.
+
+## Example Output
+
+You should see an image showing the performance comparison and analysis between FP32 and INT8.
+
+>**Note**: The image shown below is an example of a general performance comparison for inference speedup obtained by quantization. (Your results might be different.)
+
+![Performance Numbers](images/inc_speedup.png)
+
+## License
+
+Code samples are licensed under the MIT license. See
+[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details.
+
+Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt).

AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/README.MD

@@ -0,0 +1,26 @@
+import os
+
+def runJupyterNotebook(input_notebook_filename, output_notebook_filename, conda_env, fdpath='./'):
+    import nbformat
+    import os
+    from nbconvert.preprocessors import ExecutePreprocessor
+    from nbconvert.preprocessors import CellExecutionError
+    if os.path.isfile(input_notebook_filename) is False:
+        print("No Jupyter notebook found : ",input_notebook_filename)
+    try:
+        with open(input_notebook_filename) as f:
+            nb = nbformat.read(f, as_version=4)
+            ep = ExecutePreprocessor(timeout=6000, kernel_name=conda_env, allow_errors=True)
+            ep.preprocess(nb, {'metadata': {'path': fdpath}})
+            with open(output_notebook_filename, 'w', encoding='utf-8') as f:
+                nbformat.write(nb, f)
+            return 0
+    except CellExecutionError:
+        print("Exception!")
+        return -1
+
+
+runJupyterNotebook(os.path.join(os.path.dirname(os.path.realpath(__file__)),
+                                'quantize_with_inc.ipynb'),
+                                'quantize_with_inc.ipynb', 
+                                'workshop')

AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/ci_test.py

@@ -0,0 +1,41 @@
+from torch.utils.data import Dataset
+from typing import List
+from transformers import AutoTokenizer
+import torch
+
+class IMDBDataset(Dataset):
+    """Dataset with strings to predict pos/neg
+    Args:
+        text (List[str]): list of strings
+        label (List[str]): list of corresponding labels (spam/ham)
+        data_size (int): number of data rows to use
+    """
+
+
+    def __init__(
+            self,
+            text: List[str],
+            label: List[str],
+            tokenizer: AutoTokenizer,
+            max_length: int = 64,
+            data_size: int = 1000):
+        
+        if data_size > len(text):
+            raise ValueError(f"Maximum rows in dataset {len(text)}")
+        self.text = text[:data_size]
+        self.label = label
+        self.tokenizer = tokenizer
+        self.max_length = max_length
+
+    def __len__(self):
+        return len(self.text)
+
+    def __getitem__(self, idx):
+        encoding = self.tokenizer(
+            self.text[idx],
+            max_length=self.max_length,
+            padding='max_length',
+            truncation=True)
+        item = {key: torch.as_tensor(val) for key, val in encoding.items()}
+            
+        return (item, self.label[idx])