Update documentation for LLVM CFI support

Updates documentation for LLVM CFI support with recommended information
since the user can now rebuild and use both core and std with CFI
enabled using the Cargo build-std feature.

Author: rcvalle

src/doc/unstable-book/src/compiler-flags/sanitizer.md

@@ -197,22 +197,26 @@ Shadow byte legend (one shadow byte represents 8 application bytes):
 
 # ControlFlowIntegrity
 
-The LLVM Control Flow Integrity (CFI) support in the Rust compiler provides
-forward-edge control flow protection for both Rust-compiled code only and for C
-or C++ and Rust -compiled code mixed-language binaries, also known as “mixed
-binaries” (i.e., for when C or C++ and Rust -compiled code share the same
-virtual address space), by aggregating function pointers in groups identified by
-their return and parameter types.
-
-LLVM CFI can be enabled with `-Zsanitizer=cfi` and requires LTO (i.e., `-Clto`).
-Cross-language LLVM CFI can be enabled with `-Zsanitizer=cfi`, and requires the
-`-Zsanitizer-cfi-normalize-integers` option to be used with Clang
-`-fsanitize-cfi-icall-normalize-integers` for normalizing integer types, and
-proper (i.e., non-rustc) LTO (i.e., `-Clinker-plugin-lto`).
+The LLVM CFI support in the Rust compiler provides forward-edge control flow
+protection for both Rust-compiled code only and for C or C++ and Rust -compiled
+code mixed-language binaries, also known as “mixed binaries” (i.e., for when C
+or C++ and Rust -compiled code share the same virtual address space), by
+aggregating function pointers in groups identified by their return and parameter
+types.
+
+LLVM CFI can be enabled with `-Zsanitizer=cfi` and requires LTO (i.e.,
+`-Clinker-plugin-lto` or `-Clto`). Cross-language LLVM CFI can be enabled with
+`-Zsanitizer=cfi`, and requires the `-Zsanitizer-cfi-normalize-integers` option
+to be used with Clang `-fsanitize-cfi-icall-experimental-normalize-integers`
+option for cross-language LLVM CFI support, and proper (i.e., non-rustc) LTO
+(i.e., `-Clinker-plugin-lto`).
+
+It is recommended to rebuild the standard library with CFI enabled by using the
+Cargo build-std feature (i.e., `-Zbuild-std`) when enabling CFI.
 
 See the [Clang ControlFlowIntegrity documentation][clang-cfi] for more details.
 
-## Example
+## Example 1: Redirecting control flow using an indirect branch/call to an invalid destination
 
 ```rust,ignore (making doc tests pass cross-platform is hard)
 #![feature(naked_functions)]
@@ -239,7 +243,7 @@ pub extern "C" fn add_two(x: i32) {
              nop
              nop
              nop
-             lea eax, [edi+2]
+             lea eax, [rdi+2]
              ret
         ",
             options(noreturn)
@@ -258,47 +262,50 @@ fn main() {
 
     println!("With CFI enabled, you should not see the next answer");
     let f: fn(i32) -> i32 = unsafe {
-        // Offsets 0-8 make it land in the landing pad/nop block, and offsets 1-8 are
-        // invalid branch/call destinations (i.e., within the body of the function).
+        // Offset 0 is a valid branch/call destination (i.e., the function entry
+        // point), but offsets 1-8 within the landing pad/nop block are invalid
+        // branch/call destinations (i.e., within the body of the function).
         mem::transmute::<*const u8, fn(i32) -> i32>((add_two as *const u8).offset(5))
     };
     let next_answer = do_twice(f, 5);
 
     println!("The next answer is: {}", next_answer);
 }
 ```
-Fig. 1. Modified example from the [Advanced Functions and
-Closures][rust-book-ch19-05] chapter of the [The Rust Programming
-Language][rust-book] book.
+Fig. 1. Redirecting control flow using an indirect branch/call to an invalid
+destination (i.e., within the body of the function).
 
 ```shell
 $ cargo run --release
    Compiling rust-cfi-1 v0.1.0 (/home/rcvalle/rust-cfi-1)
-    Finished release [optimized] target(s) in 0.76s
+    Finished release [optimized] target(s) in 0.42s
      Running `target/release/rust-cfi-1`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 The next answer is: 14
 $
 ```
-Fig. 2. Build and execution of the modified example with LLVM CFI disabled.
+Fig. 2. Build and execution of Fig. 1 with LLVM CFI disabled.
 
 ```shell
-$ RUSTFLAGS="-Zsanitizer=cfi -Cembed-bitcode=yes -Clto" cargo run --release
+$ RUSTFLAGS="-Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld -Zsanitizer=cfi" cargo run -Zbuild-std -Zbuild-std-features --release --target x86_64-unknown-linux-gnu
+   ...
    Compiling rust-cfi-1 v0.1.0 (/home/rcvalle/rust-cfi-1)
-    Finished release [optimized] target(s) in 3.39s
-     Running `target/release/rust-cfi-1`
+    Finished release [optimized] target(s) in 1m 08s
+     Running `target/x86_64-unknown-linux-gnu/release/rust-cfi-1`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 Illegal instruction
 $
 ```
-Fig. 3. Build and execution of the modified example with LLVM CFI enabled.
+Fig. 3. Build and execution of Fig. 1 with LLVM CFI enabled.
 
 When LLVM CFI is enabled, if there are any attempts to change/hijack control
 flow using an indirect branch/call to an invalid destination, the execution is
 terminated (see Fig. 3).
 
+## Example 2: Redirecting control flow using an indirect branch/call to a function with a different number of parameters
+
 ```rust
 use std::mem;
 
@@ -327,39 +334,42 @@ fn main() {
     println!("The next answer is: {}", next_answer);
 }
 ```
-Fig. 4. Another modified example from the [Advanced Functions and
-Closures][rust-book-ch19-05] chapter of the [The Rust Programming
-Language][rust-book] book.
+Fig. 4. Redirecting control flow using an indirect branch/call to a function
+with a different number of parameters than arguments intended/passed in the
+call/branch site.
 
 ```shell
 $ cargo run --release
    Compiling rust-cfi-2 v0.1.0 (/home/rcvalle/rust-cfi-2)
-    Finished release [optimized] target(s) in 0.76s
+    Finished release [optimized] target(s) in 0.43s
      Running `target/release/rust-cfi-2`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 The next answer is: 14
 $
 ```
-Fig. 5. Build and execution of the modified example with LLVM CFI disabled.
+Fig. 5. Build and execution of Fig. 4 with LLVM CFI disabled.
 
 ```shell
-$ RUSTFLAGS="-Cembed-bitcode=yes -Clto -Zsanitizer=cfi" cargo run --release
+$ RUSTFLAGS="-Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld -Zsanitizer=cfi" cargo run -Zbuild-std -Zbuild-std-features --release --target x86_64-unknown-linux-gnu
+   ...
    Compiling rust-cfi-2 v0.1.0 (/home/rcvalle/rust-cfi-2)
-    Finished release [optimized] target(s) in 3.38s
-     Running `target/release/rust-cfi-2`
+    Finished release [optimized] target(s) in 1m 08s
+     Running `target/x86_64-unknown-linux-gnu/release/rust-cfi-2`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 Illegal instruction
 $
 ```
-Fig. 6. Build and execution of the modified example with LLVM CFI enabled.
+Fig. 6. Build and execution of Fig. 4 with LLVM CFI enabled.
 
 When LLVM CFI is enabled, if there are any attempts to change/hijack control
 flow using an indirect branch/call to a function with different number of
 parameters than arguments intended/passed in the call/branch site, the
 execution is also terminated (see Fig. 6).
 
+## Example 3: Redirecting control flow using an indirect branch/call to a function with different return and parameter types
+
 ```rust
 use std::mem;
 
@@ -388,42 +398,46 @@ fn main() {
     println!("The next answer is: {}", next_answer);
 }
 ```
-Fig. 7. Another modified example from the [Advanced Functions and
-Closures][rust-book-ch19-05] chapter of the [The Rust Programming
-Language][rust-book] book.
+Fig. 7. Redirecting control flow using an indirect branch/call to a function
+with different return and parameter types than the return type expected and
+arguments intended/passed at the call/branch site.
 
 ```shell
 $ cargo run --release
    Compiling rust-cfi-3 v0.1.0 (/home/rcvalle/rust-cfi-3)
-    Finished release [optimized] target(s) in 0.74s
+    Finished release [optimized] target(s) in 0.44s
      Running `target/release/rust-cfi-3`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 The next answer is: 14
 $
 ```
-Fig. 8. Build and execution of the modified example with LLVM CFI disabled.
+Fig. 8. Build and execution of Fig. 7 with LLVM CFI disabled.
 
 ```shell
-$ RUSTFLAGS="-Cembed-bitcode=yes -Clto -Zsanitizer=cfi" cargo run --release
+$ RUSTFLAGS="-Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld -Zsanitizer=cfi" cargo run -Zbuild-std -Zbuild-std-features --release --target x86_64-unknown-linux-gnu
+   ...
    Compiling rust-cfi-3 v0.1.0 (/home/rcvalle/rust-cfi-3)
-    Finished release [optimized] target(s) in 3.40s
-     Running `target/release/rust-cfi-3`
+    Finished release [optimized] target(s) in 1m 07s
+     Running `target/x86_64-unknown-linux-gnu/release/rust-cfi-3`
 The answer is: 12
 With CFI enabled, you should not see the next answer
 Illegal instruction
 $
 ```
-Fig. 9. Build and execution of the modified example with LLVM CFI enabled.
+Fig. 9. Build and execution of Fig. 7 with LLVM CFI enabled.
 
 When LLVM CFI is enabled, if there are any attempts to change/hijack control
 flow using an indirect branch/call to a function with different return and
 parameter types than the return type expected and arguments intended/passed in
 the call/branch site, the execution is also terminated (see Fig. 9).
 
+## Example 4: Redirecting control flow using an indirect branch/call to a function with different return and parameter types across the FFI boundary
+
 ```ignore (cannot-test-this-because-uses-custom-build)
 int
-do_twice(int (*fn)(int), int arg) {
+do_twice(int (*fn)(int), int arg)
+{
     return fn(arg) + fn(arg);
 }
 ```
@@ -459,54 +473,49 @@ fn main() {
     println!("The next answer is: {}", next_answer);
 }
 ```
-Fig. 11. Another modified example from the [Advanced Functions and
-Closures][rust-book-ch19-05] chapter of the [The Rust Programming
-Language][rust-book] book.
+Fig. 11. Redirecting control flow using an indirect branch/call to a function
+with different return and parameter types than the return type expected and
+arguments intended/passed in the call/branch site, across the FFI boundary.
 
 ```shell
 $ make
-mkdir -p target/debug
-clang -I. -Isrc -Wall -flto -fvisibility=hidden -c -emit-llvm src/foo.c -o target/debug/libfoo.bc
-llvm-ar rcs target/debug/libfoo.a target/debug/libfoo.bc
-RUSTFLAGS="-L./target/debug -Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld" cargo build
-   Compiling main v0.1.0 (/home/rcvalle/rust-cross-cfi-1)
-    Finished dev [unoptimized + debuginfo] target(s) in 0.45s
-$ ./target/debug/main
+mkdir -p target/release
+clang -I. -Isrc -Wall -c src/foo.c -o target/release/libfoo.o
+llvm-ar rcs target/release/libfoo.a target/release/libfoo.o
+RUSTFLAGS="-L./target/release -Clinker=clang -Clink-arg=-fuse-ld=lld" cargo build --release
+   Compiling rust-cfi-4 v0.1.0 (/home/rcvalle/rust-cfi-4)
+    Finished release [optimized] target(s) in 0.49s
+$ ./target/release/rust-cfi-4
 The answer is: 12
 With CFI enabled, you should not see the next answer
 The next answer is: 14
 $
 ```
-Fig. 12. Build and execution of the modified example with LLVM CFI disabled.
+Fig. 12. Build and execution of Figs. 10–11 with LLVM CFI disabled.
 
 ```shell
 $ make
-mkdir -p target/debug
-clang -I. -Isrc -Wall -flto -fvisibility=hidden -fsanitize=cfi -fsanitize-cfi-icall-normalize-integers -c -emit-llvm src/foo.c -o target/debug/libfoo.bc
-llvm-ar rcs target/debug/libfoo.a target/debug/libfoo.bc
-RUSTFLAGS="-L./target/debug -Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld -Zsanitizer=cfi -Zsanitizer-cfi-normalize-integers" cargo build
-   Compiling main v0.1.0 (/home/rcvalle/rust-cross-cfi-1)
-    Finished dev [unoptimized + debuginfo] target(s) in 0.45s
-$ ./target/debug/main
+mkdir -p target/release
+clang -I. -Isrc -Wall -flto -fsanitize=cfi -fsanitize-cfi-icall-experimental-normalize-integers -fvisibility=hidden -c -emit-llvm src/foo.c -o target/release/libfoo.bc
+llvm-ar rcs target/release/libfoo.a target/release/libfoo.bc
+RUSTFLAGS="-L./target/release -Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld -Zsanitizer=cfi -Zsanitizer-cfi-normalize-integers" cargo build -Zbuild-std -Zbuild-std-features --release --target x86_64-unknown-linux-gnu
+   ...
+   Compiling rust-cfi-4 v0.1.0 (/home/rcvalle/rust-cfi-4)
+    Finished release [optimized] target(s) in 1m 06s
+$ ./target/x86_64-unknown-linux-gnu/release/rust-cfi-4
 The answer is: 12
 With CFI enabled, you should not see the next answer
 Illegal instruction
 $
 ```
-Fig. 13. Build and execution of the modified example with LLVM CFI enabled.
-
-When LLVM CFI is enabled, if there are any attempts to change/hijack control
-flow using an indirect branch/call to a function with different return and
-parameter types than the return type expected and arguments intended/passed in
-the call/branch site, even across the FFI boundary and for extern "C" function
-types indirectly called (i.e., callbacks/function pointers) across the FFI
-boundary, in C or C++ and Rust -compiled code mixed-language binaries, also
-known as “mixed binaries” (i.e., for when C or C++ and Rust -compiled code share
-the same virtual address space), the execution is also terminated (see Fig. 13).
-
-
-[rust-book-ch19-05]: https://doc.rust-lang.org/book/ch19-05-advanced-functions-and-closures.html
-[rust-book]: https://doc.rust-lang.org/book/title-page.html
+Fig. 13. Build and execution of FIgs. 10–11 with LLVM CFI enabled.
+
+When LLVM CFI is enabled, if there are any attempts to redirect control flow
+using an indirect branch/call to a function with different return and parameter
+types than the return type expected and arguments intended/passed in the
+call/branch site, even across the FFI boundary and for extern "C" function types
+indirectly called (i.e., callbacks/function pointers) across the FFI boundary,
+the execution is also terminated (see Fig. 13).
 
 # HWAddressSanitizer