1.24.1 announcement

_posts/2018-03-01-Rust-1.24.1.md

@@ -0,0 +1,268 @@
+---
+layout: post
+title: "Announcing Rust 1.24.1"
+author: The Rust Core Team
+---
+
+The Rust team is happy to announce a new version of Rust, 1.24.1. Rust is a
+systems programming language focused on safety, speed, and concurrency.
+
+If you have a previous version of Rust installed via rustup, getting Rust
+1.24.1 is as easy as:
+
+```bash
+$ rustup update stable
+```
+
+If you don't have it already, you can [get `rustup`][install] from the
+appropriate page on our website, and check out the [detailed release notes for
+1.24.1][notes] on GitHub.
+
+[install]: https://www.rust-lang.org/install.html
+[notes]: https://github.com/rust-lang/rust/blob/stable/RELEASES.md#version-1241-2018-03-01
+
+## What's in 1.24.1 stable
+
+As you know, we tend not to release point releases. In this case, Several minor
+regressions were found in 1.24.0 which collectively merited a release.
+
+A quick summary of the changes: there are four.
+
+* Do not abort when unwinding through FFI (this reverts behavior from 1.24.0)
+* Emit UTF-16 files for linker arguments on Windows
+* Make the error index generator work again
+* Cargo will warn on Windows 7 if an update is needed.
+
+If your code is continuing to build, then the only issue that may affect you is
+the unwinding issue. We plan on bringing this behavior back in 1.25 or 1.26,
+depending on how smoothly the new strategy goes.
+
+With that, let's dig into the details!
+
+### Do not abort when unwinding through FFI
+
+TL;DR: the behavior in 1.24.0 broke the `rlua` crate, and is being reverted. If
+you have since changed your code to take advantage of the behavior in 1.24.0,
+you'll need to revert it for now.
+
+Quoting [the 1.24 annoucement](https://blog.rust-lang.org/2018/02/15/Rust-1.24.html):
+
+> There’s one other change we’d like to talk about here: undefined behavior.
+> Rust generally strives to minimize undefined behavior, having none of it in
+> safe code, and as little as possible in unsafe code. One area where you could
+> invoke UB is when a panic! goes across an FFI boundary. In other words, this:
+
+```rust
+extern "C" fn panic_in_ffi() {
+    panic!("Test");
+}
+```
+
+> This cannot work, as the exact mechanism of how panics work would have to
+> be reconciled with how the "C" ABI works, in this example, or any other ABI
+> in other examples.
+>
+> In Rust 1.24, this code will now abort instead of producing undefined behavior.
+
+As implemented, this functionality broke some users, notably, integration with
+the Lua language. We have reverted this behavior, and 1.24.1 will reintroduce
+the undefined behavior. Fundamentally, the aborting behavior we originally
+rolled out is a soundness fix, and we still plan on reintroducing it, but are
+going to be rolling it out a bit more slowly, and with a different
+implementation.
+
+It started with [a bug filed against the `rlua`
+crate](https://github.com/chucklefish/rlua/issues/71). `rlua` is a package
+that provides high level bindings between Rust and the [Lua programming
+language](https://www.lua.org/).
+
+> Side note: `rlua` is maintained by [Chucklefish](https://chucklefish.org/),
+> a game development studio from London that's using Rust. Lua is a very
+> popular language to use for extending and scripting games. We care deeply about
+> production Rust users, and so handling this was a very high priority for the
+> Rust team.
+
+On Windows, and only on Windows, any attempt to handle errors from Lua would
+simply abort. This makes `rlua` unusable, as any error of any kind within Lua
+causes your program to die.
+
+After digging in, the culpurit was found: `setjmp`/`longjmp`. These functions
+are provided by the C standard library as a means of handling errors. You
+first call `setjmp`, and then, at some later point in time, call `longjmp`.
+When you do, control flow returns to where you had previously called
+`setjmp`. This is often used as a way to implement exceptions, and sometimes,
+even coroutines. Lua's implementation uses `setjmp`/`longjmp` [to implement
+exceptions](https://www.lua.org/pil/24.3.html):
+
+> Unlike C++ or Java, the C language does not offer an exception handling
+> mechanism. To ameliorate this difficulty, Lua uses the setjmp facility from
+> C, which results in a mechanism similar to exception handling. (If you
+> compile Lua with C++, it is not difficult to change the code so that it uses
+> real exceptions instead.)
+
+The issue is this: what happens when some C code `setjmp`/`longjmp`'s through
+Rust stack frames? Because drop checking and borrow checking know nothing
+about this style of control flow, if you `longjmp` across a Rust stack
+frame that has any type that's not `Copy` on its stack, undefined
+behavior will result. However, if the jump happens entirely in C, this
+should work just fine. This is how `rlua` was managing it: every call
+into Lua is [wrapped with `lua_pcall`](https://www.lua.org/pil/24.3.2.html):
+
+> When you write library functions for Lua, however, there is a standard way
+> to handle errors. Whenever a C function detects an error, it simply calls
+> `lua_error`, (or better yet `luaL_error`, which formats the error message and
+> then calls `lua_error`). The `lua_error` function clears whatever needs to be
+> cleared in Lua and jumps back to the `lua_pcall` that originated that
+> execution, passing along the error message.
+
+So, the question becomes: Why does this break? And why does it break on
+Windows?
+
+When we talked about `setjmp`/`longjmp` inititally, a key phrase here wasn't
+highlighted. Here it is:
+
+> After digging in, the culpurit was found: `setjmp`/`longjmp`. These functions
+> are *provided by the C standard library* as a means of handling errors.
+
+These functions aren't part of the C language, but part of the standard
+library. That means that platform authors implement these functions, and
+their implementations may differ.
+
+Windows has a concept called SEH, short for ["Structured Exception
+Handling"](https://msdn.microsoft.com/en-us/library/windows/desktop/ms680657(v=vs.85).aspx).
+Windows uses SEH to implement `setjmp`/`longjmp`, as the whole idea of SEH
+is to have uniform error handling. For similar reasons, C++ exceptions use
+SEH, as do Rust panics.
+
+Before we can sort the exact details of what's happening, let's look at how `rlua`
+works. `rlua` has an internal function, `protect_lua_call`, used to call into
+Lua. Using it looks like this:
+
+```rust
+protect_lua_call(self.state, 0, 1, |state| {
+    ffi::lua_newtable(state);
+})?;
+```
+
+That is, `protect_lua_call` takes some arguments, one of which is a closure. This
+closure is passed to `lua_pcall`, which catches any `longjmp`s that may be thrown
+by the code passed to it, aka, that closure.
+
+Consider the code above, and imagine that `lua_newtable` here could call
+`longjmp`. Here's what should happen:
+
+1. `protect_lua_call` takes our closure, and passes it to `lua_pcall`.
+2. `lua_pcall` calls `setjmp` to handle any errors, and invokes our closure.
+2. Inside our closure, `lua_newtable` has an error, and calls `longjmp`.
+3. The initial `lua_pcall` catches the `longjmp` with the `setjmp` it called earlier.
+4. Everyone is happy.
+
+However, the implementation of `protect_lua_call` converts our closure to an
+`extern fn`, since that's what Lua needs. So, with the changes in 1.24.0, it
+sets up a panic handler that will cause an abort. In other words, the code
+sorta looks something like this pseudo code now:
+
+```rust
+protect_lua_call(self.state, 0, 1, |state| {
+    let result = panic::catch_unwind(|| {
+        ffi::lua_newtable(state);
+    });
+
+    if result.is_err() {
+        process::abort();
+    }
+})?;
+```
+
+Earlier, when discussing `setjmp`/`longjmp`, we said that the issue with it in
+Rust code is that it doesn't handle Rust destructors. So, on every platform but
+Windows, the above `catch_unwind` shenanigans is effectively ignored, so
+everything works. However, on Windows, since both `setjmp`/`longjmp` and Rust
+panics use SEH, the `longjmp` gets "caught", and runs the new abort code!
+
+The [solution here](https://github.com/rust-lang/rust/pull/48572) is to
+generate the abort handler, but in a way that `longjmp` won't trigger it.  It's
+not 100% clear if this will make it into Rust 1.25; if the landing is smooth,
+we may backport, otherwise, this functionality will be back in 1.26.
+
+### Emit UTF-16 files for linker arguments on Windows
+
+TL;DR: `rustc` stopped working for some Windows users. If it's been working for you,
+you were not affected by this bug.
+
+In constrast with the previous bug, which is very complex and tough to understand,
+this bug's impact is simple: if you have non-ASCII paths in the directory where
+you invoke `rustc`, in 1.24, it would incorrectly error with a message like
+
+> fatal error LNK1181: cannot open input file
+
+The PR that caused it, [#47507](https://github.com/rust-lang/rust/pull/47507),
+has a good explanation of the behavior that ended up causing the problem:
+
+> When spawning a linker rustc has historically been known to blow OS limits
+> for the command line being too large, notably on Windows. This is especially
+> true of incremental compilation where there can be dozens of object files per
+> compilation. The compiler currently has logic for detecting a failure to
+> spawn and instead passing arguments via a file instead, but this failure
+> detection only triggers if a process actually fails to spawn.
+
+However, when generating that file, we were doing it incorrectly. As [the
+docs state](https://docs.microsoft.com/en-gb/cpp/build/reference/unicode-support-in-the-compiler-and-linker#linker-response-files-and-def-files):
+
+> Response files and DEF files can be either UTF-16 with a BOM, or ANSI.
+
+We were providing a UTF-8 encoded file, with no
+[BOM](https://en.wikipedia.org/wiki/Byte_order_mark). The fix is therefore
+straightforward: produce a UTF-16 file with a BOM.
+
+### Make the error index generator work again
+
+TL;DR: building Rust 1.24.0 with Rust 1.24.0 broke in some circumstances.
+If you weren't building Rust yourself, you were not affected by this bug.
+
+When packaging Rust for various Linux distros, it was found that [building
+1.24 with 1.24 fails](https://github.com/rust-lang/rust/issues/48308).
+This was caused by an incorrect path, causing certain metadata to not
+be generated properly.
+
+### Cargo will warn on Windows 7 if an update is needed.
+
+TL;DR: Cargo couldn't fetch the index from crates.io if you were using an older
+Windows without having applied security fixes. If you are using a newer
+Windows, or a patched Windows, you are not affected by this bug.
+
+In February of 2017, [GitHub announced that they were dropping support for
+weak cryptographic
+standards](https://githubengineering.com/crypto-deprecation-notice/). One
+year later, in February of 2018, [the deprecation period is over, and support
+is
+removed](https://blog.github.com/2018-02-23-weak-cryptographic-standards-removed/).
+In general, this is a great thing.
+
+Cargo uses GitHub to store the index of Crates.io, our package repository.
+It also uses `libgit2` for `git` operations. `libgit2` uses
+[WinHTTP](https://msdn.microsoft.com/en-us/library/windows/desktop/aa382925(v=vs.85).aspx)
+for making HTTP calls. As part of the OS, its feature set depends on the OS you're using.
+
+> This section uses "Windows 7" to mean "Windows 7, Windows Server 2018, and Windows Server 2012",
+> because it's much shorter. The following applies to all three of these editions of Windows,
+> however.
+
+Windows 7 [received an update](https://support.microsoft.com/en-us/help/3140245/update-to-enable-tls-1-1-and-tls-1-2-as-a-default-secure-protocols-in)
+in June of 2016 regarding TLS. Before the patch, Windows 7 would use TLS 1.0 by default. This update
+allows for applications to use TLS 1.1 or 1.2 natively instead.
+
+If your system has not received that update, then you'd still be using TLS 1.0. This means
+that accessing GitHub would start to fail.
+
+`libgit2` [created a fix](https://github.com/libgit2/libgit2/pull/4550), using the `WinHTTP` API
+to request TLS 1.2. On master, we've [updated to fix this](https://github.com/rust-lang/cargo/pull/5091),
+but for 1.24.1 stable, we're [issuing a warning](https://github.com/rust-lang/cargo/pull/5069),
+suggesting that they upgrade their Windows version. Although the `libgit2` fix
+could have been backported, we felt that the code change footprint was too
+large for the point release, especially since the issue does not affect patched
+systems.
+
+## Contributors to 1.24.1
+
+[Thanks!](https://thanks.rust-lang.org/rust/1.24.1)