[Intel GPU] Docs of XPUInductorQuantizer #3293
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3293
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| :: | ||
| quantizer = XPUInductorQuantizer() | ||
| quantizer.set_global(get_xpu_inductor_symm_quantization_config()) | ||
|
|
There was a problem hiding this comment.
The code format has not taken effect.
There was a problem hiding this comment.
thanks for reminding, added the fix.
prototype_source/prototype_index.rst
Outdated
| @@ -96,6 +96,13 @@ Prototype features are not available as part of binary distributions like PyPI o | |||
| :link: ../prototype/pt2e_quant_x86_inductor.html | |||
| :tags: Quantization | |||
|
|
|||
| .. customcarditem:: | |||
| :header: PyTorch 2 Export Quantization with Intel GPU Backend through Inductor | |||
There was a problem hiding this comment.
At previous stage when we upload RFCs, we recommend using GPU instead of XPU for readability for users. Do we have some changes on this description desicsion?
| @@ -0,0 +1,234 @@ | |||
| PyTorch 2 Export Quantization with Intel GPU Backend through Inductor | |||
| utilizes PyTorch 2 Export Quantization flow and lowers the quantized model into the inductor. | ||
|
|
||
| The pytorch 2 export quantization flow uses the torch.export to capture the model into a graph and perform quantization transformations on top of the ATen graph. | ||
| This approach is expected to have significantly higher model coverage, better programmability, and a simplified UX. |
There was a problem hiding this comment.
This approach is expected to have significantly higher model coverage with better programmability and a simplified user experience.
There was a problem hiding this comment.
Thanks for suggestions, modified.
| The quantization flow mainly includes three steps: | ||
|
|
||
| - Step 1: Capture the FX Graph from the eager Model based on the `torch export mechanism <https://pytorch.org/docs/main/export.html>`_. | ||
| - Step 2: Apply the Quantization flow based on the captured FX Graph, including defining the backend-specific quantizer, generating the prepared model with observers, |
There was a problem hiding this comment.
Apply the quantization flow based on the captured FX Graph, including defining the backend-specific quantizer, generating the prepared model with observers,
There was a problem hiding this comment.
Thanks for suggestions, has changed the description here.
| performing the prepared model's calibration, and converting the prepared model into the quantized model. | ||
| - Step 3: Lower the quantized model into inductor with the API ``torch.compile``. | ||
|
|
||
| During Step 3, the inductor would decide which kernels are dispatched into. There are two kinds of kernels the Intel GPU would obtain benefits, oneDNN kernels and triton kernels. `Intel oneAPI Deep Neural Network Library (oneDNN) <https://github.com/uxlfoundation/oneDNN>`_ contains |
There was a problem hiding this comment.
If a end-user documentation, I think we could focus on PyTorch itself, and remove this section explanation.
There was a problem hiding this comment.
Thanks for suggestion, I removed the prolonged description over oneDNN and triton. Instead, I add a simple mention at Step 3 above.
| Post Training Quantization | ||
| ---------------------------- | ||
|
|
||
| Static quantization is the only method we support currently. QAT and dynamic quantization will be available in later versions. |
There was a problem hiding this comment.
remove the further ready context from current introduction - "QAT and dynamic quantization will be available in later versions."
There was a problem hiding this comment.
Thanks for suggestion, removed.
|
|
||
| :: | ||
|
|
||
| pip install torchvision pytorch-triton-xpu --index-url https://download.pytorch.org/whl/nightly/xpu |
There was a problem hiding this comment.
Let's use standard "pip install torch torchvision torchaudio", not separate internal commands to highlight the internal dependencies command.
There was a problem hiding this comment.
Lets keep using our own channels, since torchvision is customized on XPU, we need let user could run example in this doc successfully. Synced with @jingxu10 let's us pip3 install torch torchvision torchaudio pytorch-triton-xpu --index-url https://download.pytorch.org/whl/xpu, instead of nightly wheel.
Description
Add tutorials for XPUInductorQuantzer, which serves as the INT8 quantization backend for Intel GPU inside PT2E.