Skip to content

Add transpose variants to XNNPACK shared-qparam op list#19836

Open
Hyungkeun-Park-Nota wants to merge 2 commits into
pytorch:mainfrom
Hyungkeun-Park-Nota:fix/xnnpack-transpose-shared-qparam
Open

Add transpose variants to XNNPACK shared-qparam op list#19836
Hyungkeun-Park-Nota wants to merge 2 commits into
pytorch:mainfrom
Hyungkeun-Park-Nota:fix/xnnpack-transpose-shared-qparam

Conversation

@Hyungkeun-Park-Nota

@Hyungkeun-Park-Nota Hyungkeun-Park-Nota commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

aten.transpose.int and its siblings (transpose_copy.int, t.default, t_copy.default, swapaxes.default) were missing from _is_share_obs_or_fq_op in backends/xnnpack/quantizer/xnnpack_quantizer_utils.py, even though the value-preserving sibling permute/permute_copy was already present.

Without this fix, propagate_annotation does not attach a SharedQuantizationSpec to a transpose node, so its input and output activations can be observed independently and receive different (scale, zero_point). After to_edge decomposes aten.transpose.intaten.permute_copy.default, XNNPACK lowers it to XNNStaticTranspose, which hard-requires identical input/output quantization parameters for qint8/quint8 and rejects a mismatch with xnn_status_invalid_parameter at load_method time.

The ARM (TOSA/Ethos-U) and Qualcomm (QNN) quantizers already treat transpose as a shared-qparam op. This aligns the XNNPACK quantizer with them.

Why XNNPACK rejects the mismatch

XNNStaticTranspose is a pure data-movement op with no requantization path. Its subgraph builder calls xnn_subgraph_check_quantization_parameter_matches (XNNPACK/src/subgraph/validation.c), which hard-rejects any scale or zero-point difference:

enum xnn_status xnn_subgraph_check_quantization_parameter_matches(...) {
  if (output_value->datatype == xnn_datatype_qint8 ||
      output_value->datatype == xnn_datatype_quint8) {
    if (input_value->quantization.zero_point != output_value->quantization.zero_point)
      return xnn_status_invalid_parameter;
    if (input_value->quantization.scale != output_value->quantization.scale)
      return xnn_status_invalid_parameter;
  }
}

The same check guards other value-preserving ops (static-slice, concatenate, max-pooling, etc.). XNNPACK has no PTQ quantizer of its own; it only consumes qparams and validates this constraint, so the fix must live in the framework-side quantizer.

When it triggers (natural, no tampering)

A plain Linear → transpose → Linear stays latent because transpose is value-preserving and both observers see identical min/max. The bug fires naturally when the transpose output is re-anchored to a different scale — the clearest case is transpose feeding into cat alongside a wider-range branch:

a = self.l1(x).transpose(1, 2)   # narrow range
b = self.l2(y)                    # large range
return torch.cat([a, b], dim=1)

XNNPACK's cat annotator ties the transpose output to the combined-range scale, while the transpose input keeps the upstream narrow scale. Because transpose is not in the shared-op list, nothing reconciles the two, leading to xnn_status_invalid_parameter.

Changes

  • backends/xnnpack/quantizer/xnnpack_quantizer_utils.py: add transpose.int, transpose_copy.int, t.default, t_copy.default, swapaxes.default to _is_share_obs_or_fq_op
  • backends/xnnpack/test/quantizer/test_xnnpack_quantizer.py: add test_transpose_cat_shared_qparams — builds Linear → transpose → cat ← Linear (large-range second branch), calibrates, runs convert_pt2e, and asserts Q/DQ flanking transpose carry identical scale and zero_point

Test plan

  • python -m pytest backends/xnnpack/test/quantizer/test_xnnpack_quantizer.py::TestXNNPACKQuantizer::test_transpose_cat_shared_qparams -xvs passes
  • python -m pytest backends/xnnpack/test/quantizer/test_xnnpack_quantizer.py -x passes

cc @GregoryComer @digantdesai @cbilgin @kimishpatel @jerryzh168

@pytorch-bot

pytorch-bot Bot commented May 28, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19836

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@Hyungkeun-Park-Nota

Copy link
Copy Markdown
Contributor Author

@pytorchbot label "release notes: xnnpack"

@pytorch-bot pytorch-bot Bot added the release notes: xnnpack Changes to the XNNPack backend delegate label May 28, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 28, 2026
@Hyungkeun-Park-Nota Hyungkeun-Park-Nota force-pushed the fix/xnnpack-transpose-shared-qparam branch from 54027d1 to b64ad84 Compare May 28, 2026 04:18
aten.transpose.int and its siblings (transpose_copy.int, t.default,
t_copy.default, swapaxes.default) were missing from _is_share_obs_or_fq_op
in xnnpack_quantizer_utils.py, even though the value-preserving sibling
permute/permute_copy was already present.

Without this fix, propagate_annotation does not attach a
SharedQuantizationSpec to transpose, so the input and output activations
can be observed independently and receive different (scale, zero_point).
After to_edge decomposes aten.transpose.int to aten.permute_copy.default,
XNNPACK lowers it to XNNStaticTranspose, which hard-requires identical
input/output quantization parameters for qint8/quint8 and rejects a
mismatch with xnn_status_invalid_parameter.

ARM (TOSA) and QNN quantizers already treat transpose as a shared-qparam
op; this aligns the XNNPACK quantizer with them.

Test: test_transpose_shared_qparams verifies that after prepare_pt2e the
transpose node's input and output share the same observer instance.
@Hyungkeun-Park-Nota Hyungkeun-Park-Nota force-pushed the fix/xnnpack-transpose-shared-qparam branch from b64ad84 to bc89f05 Compare May 28, 2026 04:32
@mergennachin mergennachin requested a review from GregoryComer May 28, 2026 15:07
@nil-is-all nil-is-all added module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/ module: quantization Issues related to quantization labels Jun 1, 2026
@Hyungkeun-Park-Nota

Copy link
Copy Markdown
Contributor Author

@GregoryComer Just checking, is there anything blocking this PR, or are there any changes you'd like me to make before it's merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: quantization Issues related to quantization module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/ release notes: xnnpack Changes to the XNNPack backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants