ICE when using `feature(stdsimd)` and IFMA intrinsics on non-IFMA platforms #61530

hdevalence · 2019-06-04T22:00:23Z

Hi, while working on dalek-cryptography/curve25519-dalek#257 , I ran into an ICE.

On a Cannonlake CPU with target_feature=+avx512ifma, the branch in the linked PR works fine. On a Skylake CPU (without IFMA), the feature-gating should disable all of the IFMA code at compile-time. But instead, running cargo test causes an ICE:

~/c/curve25519-dalek (use_upstream_intrinsics|✔) $ cargo test --features "simd_backend"
   Compiling curve25519-dalek v1.1.4 (/home/hdevalence/code/curve25519-dalek)
warning: unused import: `ValidityCheck`
    --> src/ristretto.rs:1084:28
     |
1084 |     use traits::{Identity, ValidityCheck};
     |                            ^^^^^^^^^^^^^
     |
     = note: #[warn(unused_imports)] on by default

thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 8], val: ByRef(AllocId(4713).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [4294967295], len: Size { raw: 32 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(eb99858e4874f291-9a55d263e1a1abdf)', src/librustc/ty/query/plumbing.rs:548:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (6ffb8f53e 2019-06-03) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental -C target_cpu=skylake --crate-type lib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `curve25519-dalek`.
warning: build failed, waiting for other jobs to finish...
thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 8], val: ByRef(AllocId(6310).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [4294967295], len: Size { raw: 32 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(eb99858e4874f291-9a55d263e1a1abdf)', src/librustc/ty/query/plumbing.rs:548:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (6ffb8f53e 2019-06-03) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental -C target_cpu=skylake

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `curve25519-dalek`.

To learn more, run the command again with --verbose.

(I think the reason the error appears twice is just because the crate loads itself into its own build.rs, not anything else.)

The text was updated successfully, but these errors were encountered:

mati865 · 2019-06-05T11:59:42Z

Could you post exact reproduction steps?
This is what I tried:

 mateusz@ubuntu  /tmp  git clone https://github.com/dalek-cryptography/curve25519-dalek.git                                          
Cloning into 'curve25519-dalek'...
remote: Enumerating objects: 89, done.
remote: Counting objects: 100% (89/89), done.
remote: Compressing objects: 100% (55/55), done.
remote: Total 5149 (delta 51), reused 59 (delta 34), pack-reused 5060
Receiving objects: 100% (5149/5149), 2.26 MiB | 3.34 MiB/s, done.
Resolving deltas: 100% (3465/3465), done.
 mateusz@ubuntu  /tmp  curve25519-dalek/
 mateusz@ubuntu  /tmp/curve25519-dalek   master  git checkout use_upstream_intrinsics 
Branch 'use_upstream_intrinsics' set up to track remote branch 'use_upstream_intrinsics' from 'origin'.
Switched to a new branch 'use_upstream_intrinsics'
 mateusz@ubuntu  /tmp/curve25519-dalek   use_upstream_intrinsics  cargo test --features "simd_backend"

[long output]

test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

CPU: Ryzen 2700X without IFMA

hdevalence · 2019-06-06T03:00:12Z

Hmm, following exactly those steps on my machine (i7-8650U) reproduces the ICE, using rustc 1.37.0-nightly (6ffb8f53e 2019-06-03).

My machine builds with -C target_cpu=native by default, but I can also reproduce using -C target_cpu=skylake. Does setting that change anything?

mati865 · 2019-06-06T14:37:42Z

I was able to reproduce it on your branch with -C target_cpu=native and reduced it to https://github.com/mati865/curve25519-dalek/blob/ice/src/backend/vector/avx2/field.rs (it crashes without changing target_cpu but still depends on packed_simd).

Reproduction steps:

git clone https://github.com/mati865/curve25519-dalek -b ice
cd curve25519-dalek
cargo build

Click here for build output:

Updating crates.io index
   Compiling packed_simd v0.3.3
   Compiling cfg-if v0.1.9
   Compiling curve25519-dalek v1.1.4 (/tmp/tmp/curve25519-dalek)
thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 8], val: ByRef(AllocId(172).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [4294967295], len: Size { raw: 32 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(eb99858e4874f291-9a55d263e1a1abdf)', src/librustc/ty/query/plumbing.rs:548:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (7cdaffd79 2019-06-05) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `curve25519-dalek`.

To learn more, run the command again with --verbose.

This is recent regression and bisection leads to #61062
cc @mark-i-m @eddyb @oli-obk

gnzlbg · 2019-06-07T11:26:11Z

This fails on x86_64-apple-darwin, but I can't reproduce on the playground:

#![feature(
    platform_intrinsics,
    repr_simd,
)]

extern "platform-intrinsic" {
    pub fn simd_shuffle2<T, U>(x: T, y: T, idx: [u32; 2]) -> U;
}

#[repr(simd)]
#[derive(Copy, Clone)]
pub struct T(f32, f32);

pub unsafe fn foo(b: T) -> T {
    simd_shuffle2(b, b, [0, 1])
}
pub unsafe fn bar(b: T) -> T {
    simd_shuffle2(b, b, [0, 1])
}

with

thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 2], val: ByRef(AllocId(3).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [255], len: Size { raw: 8 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(67586bc8d6719bbd-8210aca0f024cb22)', src/librustc/ty/query/plumbing.rs:548:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (5eeb567a2 2019-06-06) running on x86_64-apple-darwin

note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

gnzlbg · 2019-06-07T11:30:06Z

FYI rust-lang/rust is not running stdsimd tests, and stdsimd functions are all #[inline], so they don't trigger this issue when just trying to build the crate. The moment a user tries to call one of the functions, these errors pop up.

eddyb · 2019-06-10T08:21:02Z

cc @Zoxc (sadly, @michaelwoerister is on leave)

gnzlbg · 2019-06-10T08:40:06Z

@eddyb Maybe this should be P-High, Nominated, etc. This does not compile on nightly anymore:

use core::{arch::x86_64::*, mem};
pub unsafe fn foo() {
    let x: __m128 = mem::zeroed();
    let x = _mm_cmpgt_ss(x, x);
    let x = _mm_cmpge_ss(x, x);
}

Note that this does not use any feature(...). That is, whatever broke this, broke core::arch on stable Rust.

eddyb · 2019-06-10T08:46:42Z

@gnzlbg Given const_field was added as a query in #61062, I suspect it's that PR.
~~It, and type_name, were added as eval_always and no_force, and I don't think either flag is needed.~~
Turns out that's not actually helping. Maybe there's a bug somewhere else ("stable hashing" vs interning for ty::Const? since this has an Allocation, maybe "stable hashing" is by value?)

eddyb · 2019-06-11T07:59:53Z

Self-contained reproduction:

#![feature(repr_simd, platform_intrinsics)]

#[repr(simd)]
struct I32x2(i32, i32);

extern "platform-intrinsic" {
    fn simd_shuffle2<T, U>(x: T, y: T, idx: [u32; 2]) -> U;
}

fn main() {
    unsafe {
        let _: I32x2 = simd_shuffle2(I32x2(1, 2), I32x2(3, 4), [0, 0]);
        let _: I32x2 = simd_shuffle2(I32x2(1, 2), I32x2(3, 4), [0, 0]);
    }
}

However, the tricky bit is that you need to pass -Cincremental=some-directory if you want to use rustc directly (and that also means it doesn't repro on the playground).
That's because the bit that's ICE-ing is specific to incremental (re)compilation (but it triggers right away).

This confirms my suspicion that it is const_field(a) and const_field(b) where a != b (compares AllocIds which are distinct) but hash_stable(a) == hash_stable(b) (hashes the contents).

Fix a hash collision issue on the `const_field` query fixes #61530

hdevalence mentioned this issue Jun 4, 2019

Remove simd_ffi for IFMA backend dalek-cryptography/curve25519-dalek#256

Closed

estebank added A-SIMD Area: SIMD (Single Instruction Multiple Data) I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ labels Jun 4, 2019

mati865 mentioned this issue Jun 7, 2019

ICE on nightly: "Forcing query with already existing DepNode." #61594

Closed

gnzlbg mentioned this issue Jun 7, 2019

Update stdsimd #61540

Closed

oli-obk self-assigned this Jun 11, 2019

oli-obk mentioned this issue Jun 19, 2019

Fix a hash collision issue on the const_field query #61959

Merged

bors added a commit that referenced this issue Jun 21, 2019

Auto merge of #61959 - oli-obk:const_field_ice, r=eddyb

38cd948

Fix a hash collision issue on the `const_field` query fixes #61530

bors closed this as completed in #61959 Jun 21, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ICE when using `feature(stdsimd)` and IFMA intrinsics on non-IFMA platforms #61530

ICE when using `feature(stdsimd)` and IFMA intrinsics on non-IFMA platforms #61530

hdevalence commented Jun 4, 2019

mati865 commented Jun 5, 2019

hdevalence commented Jun 6, 2019

mati865 commented Jun 6, 2019

gnzlbg commented Jun 7, 2019

gnzlbg commented Jun 7, 2019 •

edited

Loading

eddyb commented Jun 10, 2019

gnzlbg commented Jun 10, 2019 •

edited

Loading

eddyb commented Jun 10, 2019 •

edited

Loading

eddyb commented Jun 11, 2019

ICE when using feature(stdsimd) and IFMA intrinsics on non-IFMA platforms #61530

ICE when using feature(stdsimd) and IFMA intrinsics on non-IFMA platforms #61530

Comments

hdevalence commented Jun 4, 2019

mati865 commented Jun 5, 2019

hdevalence commented Jun 6, 2019

mati865 commented Jun 6, 2019

gnzlbg commented Jun 7, 2019

gnzlbg commented Jun 7, 2019 • edited Loading

eddyb commented Jun 10, 2019

gnzlbg commented Jun 10, 2019 • edited Loading

eddyb commented Jun 10, 2019 • edited Loading

eddyb commented Jun 11, 2019

ICE when using `feature(stdsimd)` and IFMA intrinsics on non-IFMA platforms #61530

ICE when using `feature(stdsimd)` and IFMA intrinsics on non-IFMA platforms #61530

gnzlbg commented Jun 7, 2019 •

edited

Loading

gnzlbg commented Jun 10, 2019 •

edited

Loading

eddyb commented Jun 10, 2019 •

edited

Loading