Skip to content

[MLAS] Add fp16 extreme-value coverage for ARM64 HGEMM NEON - #31761

Open
kjg0724 (kjg0724) wants to merge 1 commit into
microsoft:mainfrom
kjg0724:hgemm-neon-extreme-value-test
Open

[MLAS] Add fp16 extreme-value coverage for ARM64 HGEMM NEON#31761
kjg0724 (kjg0724) wants to merge 1 commit into
microsoft:mainfrom
kjg0724:hgemm-neon-extreme-value-test

Conversation

@kjg0724

Copy link
Copy Markdown
Contributor

Summary

test_hgemm_neon.cpp's four arithmetic test classes (MlasNeonHGemmTransposedBTest, MlasNeonHGemmBTest, MlasNeonHGemmPackedBTest, MlasNeonHGemmTest) only ever fill operands via a uniform_real_distribution restricted to [-1,1] or [-0.25,0.25]. Even the largest shapes (K up to 513) never come close to fp16 overflow (max ~65504) since the accumulated magnitude stays well under that.

FloatEqual (duplicated identically across all four classes) also mishandles Inf/NaN:

return std::abs(f0 - f1) <= std::abs(f1 * rtol) + atol;

Inf == Inf computes abs(Inf-Inf) = NaN, and NaN <= Inf is false, so two matching infinities fail the comparison. Conversely, a finite kernel output against an Inf reference computes abs(finite-Inf) = Inf <= Inf, which is true -- a dropped overflow passes silently. Extracted the function into a single free function with explicit Inf/NaN handling; the existing finite-value branch is untouched.

Added TestHGemmFixed<> to each class, filling operands with fixed extreme fp16 values instead of the random distribution: near max magnitude, alpha/beta scaling that pushes into overflow, and subnormal-range products. Values were chosen to land in ranges where the kernel's native fp16 accumulation and the reference's fp32-then-round accumulation agree. Large-magnitude operands combined with large K diverge past the existing rtol=0.02/atol=0.055 tolerance from fp16's accumulation error alone (confirmed by measurement, e.g. a=10,b=1,K=513 gives 4928 vs a 5128 reference), so those combinations are deliberately avoided rather than loosening the tolerance.

MlasNeonHGemmPackBTest is left untouched: HPackB_TransposedB_Kernel/HPackB_B_Kernel only move data, and Check<N,K> already compares raw bit patterns (ASSERT_EQ(packed[...].val, ref[...].val)), so FloatEqual's defect doesn't apply there.

Test-only change; no file under onnxruntime/core/mlas/lib/ is touched.

Testing

test_hgemm_neon.cpp only builds under MLAS_F16VEC_INTRINSICS_SUPPORTED && MLAS_TARGET_ARM64; under the default build configuration that excludes Apple targets (mlas.h disables fp16 vector intrinsics under __APPLE__), so verification was done on a non-Apple ARM64 host.

Neoverse-N1 (Oracle Cloud A1, Ubuntu):

  • *NeonHGemm*: 5/5 passed (all five suites, including the new extreme-value cases)
  • Full onnxruntime_mlas_test suite: 36020/36020 passed, no regressions

MlasNeonHGemmTransposedBTest/BTest/PackedBTest/HGemmTest only ever
fill A/B/C via a uniform_real_distribution restricted to [-1,1] or
[-0.25,0.25], so overflow, underflow, and Inf/NaN never get
exercised even at the largest shapes (K up to 513).

FloatEqual (duplicated across all four classes) also mishandles
Inf/NaN: abs(f0-f1) <= abs(f1*rtol)+atol makes Inf==Inf compare as
NaN<=Inf (fails), while a finite kernel output against an Inf
reference compares as finite<=Inf (passes, silently missing a
dropped overflow). Extracted into a single free function with
explicit Inf/NaN handling; existing finite-value comparisons are
untouched.

Added TestHGemmFixed<> to each arithmetic test class, filling
operands with fixed extreme fp16 values (near max magnitude,
alpha/beta scaling into overflow, subnormal-range products) chosen
to land in ranges where the kernel's native fp16 accumulation and
the reference's fp32 accumulation agree -- large-magnitude operands
combined with large K diverge past the existing rtol/atol tolerance
because of fp16's accumulation error, which isn't this file's
concern to fix.

MlasNeonHGemmPackBTest (pure data movement, already compared via
raw bit equality) is left as-is.

Test-only change, no kernel code touched.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant