Rebuilding webrender/wrench takes a long time

[jrmuizel]

$ git clone https://github.com/servo/webrender/
$ cd wrench
$ cargo build

$ [do some small change to wrench/src/rawtest.rs]
$ cargo build
   Compiling wrench v0.3.0 (file:///tmp/webrender/wrench)
    Finished dev [unoptimized + debuginfo] target(s) in 5.15 secs

These timings are from linux. It's quite a bit worse on Mac, but a lot of that time probably comes from dsymutil (#47784)

[ishitatsuyuki]

@kennytm Issues with a mixture of root cause shouldn't be tagged for the WG; a WG issue normally has a concrete target to achieve, and is normally tagged by a WG lead.

[michaelwoerister]

@ishitatsuyuki There's no harm in marking issues like this one with the WG-compiler-performance label. I personally view the label as a tool for discoverability, so it's better to tag too much than too little.

[michaelwoerister]

I can reproduce this. Rebuilding takes 5.6 seconds on my system. Of that, 2.8 seconds are spent in the linker. The compiler also spends 0.780 seconds in LLVM, but it only re-compiles the two object files corresponding to rawtest.rs, so that's pretty much the best case.

Note that using different linkers gives different results:

	rebuild time
ld	9.38s
gold	5.65s
lld	3.41s

[nikomatsakis]

@michaelwoerister can you clarify why that is the best case? =) that is, can we break down a bit more where the time is going?

[michaelwoerister]

It is the best case in the sense that only the two object files get re-compiled that correspond to the changed module while all the others are re-used. Iow, the best case with respect to object file re-use. I'll post some -Ztime-passes and perf focus numbers later, so we can take a closer look.

[michaelwoerister]

By the way, in order to compile with LLD, I had to use clang as the linker:

RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=lld" cargo build

[nnethercote]

On my Linux box, using Nightly rustc (from 2018-05-22), linking takes about 4.8s out of a total time of 6.3s. That's with whatever the default linker is.

[michaelwoerister]

That's with whatever the default linker is.

Really? That's very surprising. Maybe it's bottlenecked on I/O on that system?

[nnethercote]

@michaelwoerister: how can I tell which linker I am using? And how can I change the linker?

[michaelwoerister]

@nnethercote You can change the linker as described above. You can use -Zprint-link-args to view the exact linker invocation.

[nnethercote]

I just re-measured on my Linux box. For "some small change to wrench/src/rawtest.rs" I just touched the file. Results:

• RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=ld" time cargo build: 7.6 seconds
• RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=gold" time cargo build: 4.4 seconds
• RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=lld" time cargo build: 2.9 seconds

I had to run sudo apt install lld because lld wasn't installed by default on my system.

Right, so what can we do about getting lld as the default linker?

[nnethercote]

Right, so what can we do about getting lld as the default linker?

#39915 is the PR for that.

[alexcrichton]

@nnethercote we're actually currently shipping LLD on nightly, but it's currently only intended for wasm and "accidentally" also works for MSVC. For targets like OSX and Linux the story is a bit trickier due to the other object files that clang/gcc typically insert.

The next step for stabilizing LLD would be to get a flag, like -Z linker-flavor=lld, working for all targets (Windows + Mac + Linux). It'd do whatever it needs to do to work across the various platforms.

Once that's done we can advertise it to the community, asking for feedback. Here we can gain both timing information as well as bug reports to send to LLD. If everything goes smoothly (which is sort of doubtful with a whole brand new linker, but hey you never know!) we can turn it on by default, otherwise we can work to stabilize the selection of LLD and then add an option to Cargo.toml so projects can at least opt-in to it.

[steveklabnik]

Triage: I'm not entirely sure, but I think this issue could be closed? There's been no comments for a year, and it seems like a linker issue, and we've been making progress towards LLD (Used on some targets by default, even).

[jrmuizel]

This is still pretty bad

$ touch src/rawtest.rs
$ cargo build
   Compiling wrench v0.3.0 (/Users/jrmuizel/src/webrender/wrench)
    Finished dev [unoptimized + debuginfo] target(s) in 12.56s

[Mark-Simulacrum]

It does seem like an odd thing to track a particular crate though -- if this was reopened as "X pattern makes compiler super slow" then that would make more sense to me.

We also have a somewhat long-standing issue I believe on rustc-perf to add webrender/wrench as a benchmark so I'm going to loosely close in favor of that.