From 7f71c091b5c277f399ec7d4862d6fb71f489046f Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 21 Jul 2020 21:02:11 -0400 Subject: [PATCH 1/3] Name `rustbuild` instead of saying 'the bootstrap binary' --- src/building/bootstrapping.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/building/bootstrapping.md b/src/building/bootstrapping.md index cce9fac49..1b1337e1a 100644 --- a/src/building/bootstrapping.md +++ b/src/building/bootstrapping.md @@ -57,13 +57,13 @@ When you use the bootstrap system, you'll call it through `x.py`. However, most of the code lives in `src/bootstrap`. `bootstrap` has a difficult problem: it is written in Rust, but yet it is run before the rust compiler is built! To work around this, there are two -components of bootstrap: the main one written in rust, and `bootstrap.py`. +components of bootstrap: the main one written in Rust, called `rustbuild`, +and `bootstrap.py`. `bootstrap.py` is what gets run by x.py. It takes care of downloading the -`stage0` compiler, which will then build the bootstrap binary written in -Rust. +bootstrap compiler, which will then compile `rustbuild`. Because there are two separate codebases behind `x.py`, they need to -be kept in sync. In particular, both `bootstrap.py` and the bootstrap binary +be kept in sync. In particular, both `bootstrap.py` and `rustbuild` parse `config.toml` and read the same command line arguments. `bootstrap.py` keeps these in sync by setting various environment variables, and the programs sometimes to have add arguments that are explicitly ignored, to be From d2390434483cb8270eee30e85e281c3f9a3ad836 Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 21 Jul 2020 21:03:54 -0400 Subject: [PATCH 2/3] Fix and clean up the terminology for stages - Use 'stage0 rustc' and 'stage0 compiler' to mean `stage0-rustc` - Use 'bootstrap' to mean `build/$target/stage0` (i.e. the beta compiler) - Mention that 'ignore-stage1' means 'ignore programs linked to stage1', not built by it. --- src/building/bootstrapping.md | 41 ++++++++++++++++++++++++++++ src/building/how-to-build-and-run.md | 32 ---------------------- 2 files changed, 41 insertions(+), 32 deletions(-) diff --git a/src/building/bootstrapping.md b/src/building/bootstrapping.md index 1b1337e1a..8f566864a 100644 --- a/src/building/bootstrapping.md +++ b/src/building/bootstrapping.md @@ -78,6 +78,47 @@ contribution [here][bootstrap-build]. ## Stages of bootstrap +Like most other bootstrapping compilers, `rustc` is compiled in stages. +_Unlike_ most other compilers, where `stage0` refers to the bootstrap compiler, +`stage0` refers to the first compiler built by bootstrap. So the following command: + +```sh +x.py build --stage 0 src/rustc +``` + +will actually perform a full build of rustc. Confusingly, the `build/$target/stageN` directories are named after the compiler they are *used to build*, not the commands you need to build them. + +- **Stage 0:** The stage0 compiler is built by the bootstrap compiler. + The bootstrap compiler is usually (you can configure `x.py` to use + something else) the current beta `rustc` compiler and its associated dynamic + libraries (which `x.py` will download for you). This bootstrap compiler is then + used to compile `rustbuild`, `std`, and `rustc` (plus a few other tools, like `tidy`). When compiling + `rustc`, this bootstrap compiler uses the freshly compiled `std`. + There are two concepts at play here: a compiler (with its set of dependencies) + and its 'target' or 'object' libraries (`std` and `rustc`). + Both are staged, but in a staggered manner. + The `stage0` standard library is the one _linked_ to `stage0` rustc + (allowing you to `use std::vec::Vec` from within the compiler itself), + while `stage1 libstd` is the library _built_ by stage1. `libstd` also include `libcore`, + so without it there are very few programs that rustc can compile. + +- **Stage 1:** In theory, the stage0 compiler is functionally identical to the + stage1 compiler, but in practice there are subtle differences. In + particular, the stage0 compiler was built by bootstrap and + hence not by the source in your working directory: this means that + the symbol names used in the compiler source may not match the + symbol names that would have been made by the stage1 compiler. This is + important when using dynamic linking because Rust does not have ABI compatibility + between versions. This primarily manifests when tests try to link with any + of the `rustc_*` crates or use the (now deprecated) plugin infrastructure. + These tests are marked with `ignore-stage1`. The `stage1` is because + these plugins *link* to the stage1 compiler, even though they are being + *built* by stage0. Rebuilding again also gives us the benefit of the latest optimizations (i.e. those added since the beta fork). + +- _(Optional)_ **Stage 2**: to sanity check our new compiler, we + can build the libraries with the stage1 compiler. The result ought + to be identical to before, unless something has broken. + This is a detailed look into the separate bootstrap stages. When running `x.py` you will see output such as: diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index 155c84972..cc83a78cc 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -109,38 +109,6 @@ to build it, such as `libstd` and other tooling, may use some unstable features internally, requiring a specific version which understands these unstable features. -The result is that compiling `rustc` is done in stages: - -- **Stage 0:** the stage0 compiler is usually (you can configure `x.py` to use - something else) the current _beta_ `rustc` compiler and its associated dynamic - libraries (which `x.py` will download for you). This stage0 compiler is then - used only to compile `rustbuild`, `std`, and `rustc`. When compiling - `rustc`, this stage0 compiler uses the freshly compiled `std`. - There are two concepts at play here: a compiler (with its set of dependencies) - and its 'target' or 'object' libraries (`std` and `rustc`). - Both are staged, but in a staggered manner. -- **Stage 1:** the code in your clone (for new version) is then - compiled with the stage0 compiler to produce the stage1 compiler. - However, it was built with an older compiler (stage0), so to - optimize the stage1 compiler we go to next the stage. - - In theory, the stage1 compiler is functionally identical to the - stage2 compiler, but in practice there are subtle differences. In - particular, the stage1 compiler itself was built by stage0 and - hence not by the source in your working directory: this means that - the symbol names used in the compiler source may not match the - symbol names that would have been made by the stage1 compiler. This is - important when using dynamic linking and the lack of ABI compatibility - between versions. This primarily manifests when tests try to link with any - of the `rustc_*` crates or use the (now deprecated) plugin infrastructure. - These tests are marked with `ignore-stage1`. -- **Stage 2:** we rebuild our stage1 compiler with itself to produce - the stage2 compiler (i.e. it builds itself) to have all the _latest - optimizations_. (By default, we copy the stage1 libraries for use by - the stage2 compiler, since they ought to be identical.) -- _(Optional)_ **Stage 3**: to sanity check our new compiler, we - can build the libraries with the stage2 compiler. The result ought - to be identical to before, unless something has broken. - To read more about the bootstrap process, [read this chapter][bootstrap]. [bootstrap]: ./bootstrapping.md From a237571e63fb7ca02a73c26a5734d49144bc8060 Mon Sep 17 00:00:00 2001 From: Joshua Nelson Date: Tue, 21 Jul 2020 21:38:49 -0400 Subject: [PATCH 3/3] Fix incorrect/misleading summary in the middle of the wall of text --- src/building/bootstrapping.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/building/bootstrapping.md b/src/building/bootstrapping.md index 8f566864a..bfed53b28 100644 --- a/src/building/bootstrapping.md +++ b/src/building/bootstrapping.md @@ -204,11 +204,11 @@ The following tables indicate the outputs of various stage actions: `--stage=2` stops here. Note that the convention `x.py` uses is that: -- A "stage N artifact" is an artifact that is _produced_ by the stage N compiler. -- The "stage (N+1) compiler" is assembled from "stage N artifacts". -- A `--stage N` flag means build _with_ stage N. +- A "stage N artifact" is an artifact that is part of stage N. +- The compiler in `build/$target/stage(N+1)` is assembled from "stage N artifacts". +- A `--stage N` flag means build a stage N artifact. -In short, _stage 0 uses the stage0 compiler to create stage0 artifacts which +In short, _stage 0 uses the bootstrap compiler to create stage0 artifacts which will later be uplifted to stage1_. Every time any of the main artifacts (`std` and `rustc`) are compiled, two