Add transpose variants to XNNPACK shared-qparam op list#19836
Open
Hyungkeun-Park-Nota wants to merge 2 commits into
Open
Add transpose variants to XNNPACK shared-qparam op list#19836Hyungkeun-Park-Nota wants to merge 2 commits into
Hyungkeun-Park-Nota wants to merge 2 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19836
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Contributor
Author
|
@pytorchbot label "release notes: xnnpack" |
54027d1 to
b64ad84
Compare
aten.transpose.int and its siblings (transpose_copy.int, t.default, t_copy.default, swapaxes.default) were missing from _is_share_obs_or_fq_op in xnnpack_quantizer_utils.py, even though the value-preserving sibling permute/permute_copy was already present. Without this fix, propagate_annotation does not attach a SharedQuantizationSpec to transpose, so the input and output activations can be observed independently and receive different (scale, zero_point). After to_edge decomposes aten.transpose.int to aten.permute_copy.default, XNNPACK lowers it to XNNStaticTranspose, which hard-requires identical input/output quantization parameters for qint8/quint8 and rejects a mismatch with xnn_status_invalid_parameter. ARM (TOSA) and QNN quantizers already treat transpose as a shared-qparam op; this aligns the XNNPACK quantizer with them. Test: test_transpose_shared_qparams verifies that after prepare_pt2e the transpose node's input and output share the same observer instance.
b64ad84 to
bc89f05
Compare
GregoryComer
approved these changes
May 28, 2026
Contributor
Author
|
@GregoryComer Just checking, is there anything blocking this PR, or are there any changes you'd like me to make before it's merged? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aten.transpose.intand its siblings (transpose_copy.int,t.default,t_copy.default,swapaxes.default) were missing from_is_share_obs_or_fq_opinbackends/xnnpack/quantizer/xnnpack_quantizer_utils.py, even though the value-preserving siblingpermute/permute_copywas already present.Without this fix,
propagate_annotationdoes not attach aSharedQuantizationSpecto a transpose node, so its input and output activations can be observed independently and receive different(scale, zero_point). Afterto_edgedecomposesaten.transpose.int→aten.permute_copy.default, XNNPACK lowers it toXNNStaticTranspose, which hard-requires identical input/output quantization parameters forqint8/quint8and rejects a mismatch withxnn_status_invalid_parameteratload_methodtime.The ARM (TOSA/Ethos-U) and Qualcomm (QNN) quantizers already treat transpose as a shared-qparam op. This aligns the XNNPACK quantizer with them.
Why XNNPACK rejects the mismatch
XNNStaticTransposeis a pure data-movement op with no requantization path. Its subgraph builder callsxnn_subgraph_check_quantization_parameter_matches(XNNPACK/src/subgraph/validation.c), which hard-rejects any scale or zero-point difference:The same check guards other value-preserving ops (
static-slice,concatenate,max-pooling, etc.). XNNPACK has no PTQ quantizer of its own; it only consumes qparams and validates this constraint, so the fix must live in the framework-side quantizer.When it triggers (natural, no tampering)
A plain
Linear → transpose → Linearstays latent because transpose is value-preserving and both observers see identical min/max. The bug fires naturally when the transpose output is re-anchored to a different scale — the clearest case is transpose feeding intocatalongside a wider-range branch:XNNPACK's
catannotator ties the transpose output to the combined-range scale, while the transpose input keeps the upstream narrow scale. Because transpose is not in the shared-op list, nothing reconciles the two, leading toxnn_status_invalid_parameter.Changes
backends/xnnpack/quantizer/xnnpack_quantizer_utils.py: addtranspose.int,transpose_copy.int,t.default,t_copy.default,swapaxes.defaultto_is_share_obs_or_fq_opbackends/xnnpack/test/quantizer/test_xnnpack_quantizer.py: addtest_transpose_cat_shared_qparams— buildsLinear → transpose → cat ← Linear(large-range second branch), calibrates, runsconvert_pt2e, and asserts Q/DQ flanking transpose carry identicalscaleandzero_pointTest plan
python -m pytest backends/xnnpack/test/quantizer/test_xnnpack_quantizer.py::TestXNNPACKQuantizer::test_transpose_cat_shared_qparams -xvspassespython -m pytest backends/xnnpack/test/quantizer/test_xnnpack_quantizer.py -xpassescc @GregoryComer @digantdesai @cbilgin @kimishpatel @jerryzh168