Issue bootstrapping x86_64-unknown-linux-musl - using -static when linking llvm dynamic library

[lancethepants]

I'm using Debian 10 trying to bootstrap x86_64-unknown-linux-musl.
https://github.com/lancethepants/rust-bootstrap-x86_64-unknown-linux-musl

I've bootstrapped other platforms using musl libc but haven't encountered this issue. Near the end of compiling llvm for the target when everything is linking I get this and other issues like this. The issue seems to be that -static is being passed to the link invocation when creating a dynamic library which is pulling in several static libraries. The -static shouldn't be in there and when I manually run the command without it it links just fine. The -static is causing the linker to want to pull in crtbeginT.o when what we really need is crtbeginS.o. -shared is also provided later on in the link, but that doesn't seem to counter the -static.

Looking through the source I'm not sure where the -static is being pulled in.
In compiler/rustc_codegen_ssa/src/back/linker.rs there is a reference to StaticDylib that invokes -static. I've tried removing that but that didn't seem to work, though I don't know if it's using the pre-compiled toolchain downloaded when bootstrapping or the newly created x86_64-unknown-linux-gnu version that is built during this bootstrap process. The x86_64-unknown-linux-gnu toolchain it creates in the bootstrap process doesn't have -static when llvm is being linked for that target, so seems like a musl thing maybe.

/opt/tomatoware/x86_64-musl-mmc/bin/x86_64-linux-g++ -fPIC -ffunction-sections -fdata-sections -fPIC -m64 -static -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-Bsymbolic -static-libstdc++ -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/lance/rust-bootstrap-x86_64-unknown-linux-musl/src/rust/rust/build/x86_64-unknown-linux-musl/llvm/build/./lib  -Wl,--gc-sections  -Wl,--version-script,"/home/lance/rust-bootstrap-x86_64-unknown-linux-musl/src/rust/rust/build/x86_64-unknown-linux-musl/llvm/build/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.14-rust-1.64.0-stable -o lib/libRemarks.so.14-rust-1.64.0-stable tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMSupport.a -lrt -ldl -lm lib/libLLVMDemangle.a && :
/opt/tomatoware/x86_64-musl-mmc/lib/gcc/x86_64-tomatoware-linux-musl/12.2.0/../../../../x86_64-tomatoware-linux-musl/bin/ld: /opt/tomatoware/x86_64-musl-mmc/lib/gcc/x86_64-tomatoware-linux-musl/12.2.0/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/opt/tomatoware/x86_64-musl-mmc/lib/gcc/x86_64-tomatoware-linux-musl/12.2.0/../../../../x86_64-tomatoware-linux-musl/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status

[jyn514]

This looks related to #108878. In general, our linkage situation on MUSL is ... dubious. See https://rust-lang.zulipchat.com/#narrow/stream/233931-xxx/topic/Update.20the.20existing.20musl.20targets.20to.20be.20dy.E2.80.A6.20compiler-team.23422 for some past discussion - looks like that never ended up happening, not sure what the current status is.

[jyn514]

I'm curious to know if adding -Ctarget-feature=-crt-static -Clink-self-contained=no in RUSTFLAGS helps at all.

[lancethepants]

I threw in -Clink-self-contained=no in with -Ctarget-feature=-crt-static that I already was using but I still get the same outcome. Somehow that -static is being thrown into the mix there.

[catamorphism]

What happens if you add crt-static = false under [target.x86_64-unknown-linux-musl] in your config.toml? I just ran into a similar problem on riscv64gc-unknown-linux-musl, and that fixed it.

The relevant code that adds -static is in src/bootstrap/cc_detect.rs: https://github.com/rust-lang/rust/blob/master/src/bootstrap/cc_detect.rs#L66 - this doesn't appear to be affected by RUSTFLAGS but does take a Build as an argument that reflects the contents of your config.toml.

[catamorphism]

That being said, there should really be some sort of check before LLVM is even configured that errors out if llvm.link-shared = true and <target>.crt-static = true, or at least, the cc_detect code implies that these are incompatible, and if they are, it should be detected early.

[lancethepants]

What happens if you add crt-static = false under [target.x86_64-unknown-linux-musl] in your config.toml? I just ran into a similar problem on riscv64gc-unknown-linux-musl, and that fixed it.

That was the trick, now I'm able to complete the compilation. I never had to change that value for other platforms. I also had to take out the -Clink-self-contained=no because during build the rust analyzer it said it was unsupported for this platform.
Thanks a bunch!

[catamorphism]

What happens if you add crt-static = false under [target.x86_64-unknown-linux-musl] in your config.toml? I just ran into a similar problem on riscv64gc-unknown-linux-musl, and that fixed it.

That was the trick, now I'm able to complete the compilation. I never had to change that value for other platforms.

Yes, Rust defaults to crt-static = true for musl platforms (and maybe a few others). That's probably why you didn't have to change it for other platforms. This has caused me a great deal of confusion as well. Glad it worked for you!

I think there's a potential diagnostics enhancement to be extracted from this; haven't thought hard enough about whether that would be the same enhancement(s) suggested in #110511 or not.