Aarch64 performance: vld1q_u8 intrinsic can cause single-byte loads

[hkratz]

While adding aarch64 support to simdutf8 I encountered an unexpected eight times slowdown when hand-unrolling a loop. This slowdown was the result of the compiler deciding to suddenly load 128-bit uint8x16_t values with single-byte load instructions instead of 128-bit loads.

It turns out, that the vld1q_u8 intrinsic is at fault. The code generator thinks it can "optimize" loads by loading bytes individually if a SIMD shuffle instruction follows. According to the ARM docs this intrinsic should always be coded as one instruction. I fixed it by doing the load similar to how it is currently done for SSE2.

Testcase and proposed fix on Godbolt

The same issue likely applies is to the other vld1q intrinsics.

[Amanieu]

Comparing to the IR generated by Clang, the real issue is that we should be calling llvm.aarch64.neon.ld1x4.v16i8.p0i8 instead of doing the loads manually.

cc @SparrowLii

[Amanieu]

Ah but that's currently blocked on #1143, which is a rustc limitation on returning tuples from intrinsics.

[SparrowLii]

In general, these implementations will be optimized by the compiler to a ldr instruction:
godbolt
In Clang, the implementation also directly uses the load instruction:
godbolt
Although I don’t know much about the optimization reasons of the compiler, in view of the current clear improvement scenarios, I think it is reasonable to change the implementation of vld1* instructions. For us, these instructions do not need to call llvm.aarch64.neon.*

[hkratz]

In Clang, the implementation also directly uses the load instruction:
godbolt

That is really interesting...

However with clang it does not break up the loads when the vector register is actually used:
https://godbolt.org/z/ozMev64sz

In contrast to Rust:
https://godbolt.org/z/rj1zv8PjW

That means it is likely not the fault of the vld1q_u8 intrinsic at all.