Auto merge of #56066 - jethrogb:jb/sgx-target, r=alexcrichton

Add SGX target to std and dependencies

This PR adds tier 3 `std` support for the `x86_64-fortanix-unknown-sgx` target.

### Background

Intel Software Guard Extensions (SGX) is an instruction set extension for x86 that allows executing code in fully-isolated *secure enclaves*. These enclaves reside in the address space of a regular user process, but access to the enclave's address space from outside (by e.g. the OS or a hypervisor) is blocked.

From within such enclaves, there is no access to the operating system or hardware peripherals. In order to communicate with the outside world, enclaves require an untrusted “helper” program that runs as a normal user process.

SGX is **not** a sandboxing technology: code inside SGX has full access to all memory belonging to the process it is running in.

### Overview

The Fortanix SGX ABI (compiler target `x86_64-fortanix-unknown-sgx`) is an interface for Intel SGX enclaves. It is a small yet functional interface suitable for writing larger enclaves. In contrast to other enclave interfaces, this interface is primarly designed for running entire applications in an enclave. The interface has been under development since early 2016 and builds on Fortanix's significant experience running enclaves in production.

Also unlike other enclave interfaces, this is the only implementation of an enclave interface that is nearly pure-Rust (except for the entry point code).

A description of the ABI may be found at https://docs.rs/fortanix-sgx-abi/ and https://github.com/fortanix/rust-sgx/blob/master/doc/FORTANIX-SGX-ABI.md.

The following parts of `std` are not supported and most operations will error when used:

* `std::fs`
* `std::process`
* `std::net::UdpSocket`

### Future plans

A separate PR (#56067) will add the SGX target to the rust compiler. In the very near future, I expect to upgrade this target to tier 2.

This PR is just the initial support to make things mostly work. There will be more work coming in the future, for example to add interfaces to the native SGX primitives, implement unwinding, optimize usercalls.

UDP and some form of filesystem support may be added in the future, but process support seems unlikely given the platform's constraints.

### Testing build

1. Install [Xargo](https://github.com/japaric/xargo): `cargo install xargo`
2. Create a new Cargo project, for example: `cargo new --bin sgxtest`.
3. Put the following in a file `Xargo.toml` next to your `Cargo.toml`:

```toml
[target.x86_64-fortanix-unknown-sgx.dependencies.std]
git = "https://github.com/jethrogb/rust"
branch = "jb/sgx-target"
```

NB. This can be quite slow. Instead, you can have a local checkout of that branch and use `path = "/path/to/rust/src/libstd"` instead. Don't forget to checkout the submodules too!

4. Build:

```sh
xargo build --target x86_64-fortanix-unknown-sgx
```

### Testing execution

Execution is currently only supported on x86-64 Linux, but support for Windows is planned.

1. Install pre-requisites. In order to test execution, you'll need to have a CPU with Intel SGX support. SGX support needs to be enabled in the BIOS. You'll also need to install the SGX driver and Platform Software (PSW) from [Intel](https://01.org/intel-software-guard-extensions).

2. Install toolchain, executor:
```sh
cargo install sgxs-tools --version 0.6.0-rc1
cargo install fortanix-sgx-tools --version 0.1.0-rc1
```

3. Start the enclave:

```sh
ftxsgx-elf2sgxs target/x86_64-fortanix-unknown-sgx/debug/sgxtest --heap-size 0x20000 --ssaframesize 1 --stack-size 0x20000 --threads 1 --debug
sgxs-append -i target/x86_64-fortanix-unknown-sgx/debug/sgxtest.sgxs
ftxsgx-runner target/x86_64-fortanix-unknown-sgx/debug/sgxtest.sgxs
```

Author: bors

.gitmodules

@@ -61,10 +61,12 @@
 	path = src/tools/clang
 	url = https://github.com/rust-lang-nursery/clang.git
 	branch = rust-release-80-v2
-
 [submodule "src/doc/rustc-guide"]
 	path = src/doc/rustc-guide
 	url = https://github.com/rust-lang/rustc-guide.git
 [submodule "src/doc/edition-guide"]
 	path = src/doc/edition-guide
 	url = https://github.com/rust-lang-nursery/edition-guide
+[submodule "src/rust-sgx"]
+	path = src/rust-sgx
+	url = https://github.com/fortanix/rust-sgx

Cargo.lock

@@ -797,6 +797,14 @@ name = "foreign-types-shared"
 version = "0.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
+[[package]]
+name = "fortanix-sgx-abi"
+version = "0.0.0"
+dependencies = [
+ "compiler_builtins 0.0.0",
+ "core 0.0.0",
+]
+
 [[package]]
 name = "fs2"
 version = "0.4.3"
@@ -2773,6 +2781,7 @@ dependencies = [
  "compiler_builtins 0.0.0",
  "core 0.0.0",
  "dlmalloc 0.0.0",
+ "fortanix-sgx-abi 0.0.0",
  "libc 0.0.0",
  "panic_abort 0.0.0",
  "panic_unwind 0.0.0",

src/bootstrap/dist.rs

@@ -874,6 +874,7 @@ impl Step for Src {
             "src/rustc/compiler_builtins_shim",
             "src/rustc/libc_shim",
             "src/rustc/dlmalloc_shim",
+            "src/rustc/fortanix-sgx-abi_shim",
             "src/libtest",
             "src/libterm",
             "src/libprofiler_builtins",

src/dlmalloc

@@ -66,6 +66,12 @@ pub unsafe extern fn __rust_start_panic(_payload: usize) -> u32 {
     unsafe fn abort() -> ! {
         core::intrinsics::abort();
     }
+
+    #[cfg(target_env="sgx")]
+    unsafe fn abort() -> ! {
+        extern "C" { pub fn panic_exit() -> !; }
+        panic_exit();
+    }
 }
 
 // This... is a bit of an oddity. The tl;dr; is that this is required to link

src/libcompiler_builtins

@@ -62,7 +62,7 @@ cfg_if! {
     if #[cfg(target_os = "emscripten")] {
         #[path = "emcc.rs"]
         mod imp;
-    } else if #[cfg(target_arch = "wasm32")] {
+    } else if #[cfg(any(target_arch = "wasm32", target_env = "sgx"))] {
         #[path = "dummy.rs"]
         mod imp;
     } else if #[cfg(all(target_env = "msvc", target_arch = "aarch64"))] {

src/liblibc

@@ -35,9 +35,12 @@ rustc_lsan = { path = "../librustc_lsan" }
 rustc_msan = { path = "../librustc_msan" }
 rustc_tsan = { path = "../librustc_tsan" }
 
-[target.'cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))'.dependencies]
+[target.'cfg(any(all(target_arch = "wasm32", not(target_os = "emscripten")), target_env = "sgx"))'.dependencies]
 dlmalloc = { path = '../rustc/dlmalloc_shim' }
 
+[target.x86_64-fortanix-unknown-sgx.dependencies]
+fortanix-sgx-abi = { path = "../rustc/fortanix-sgx-abi_shim" }
+
 [build-dependencies]
 cc = "1.0"
 build_helper = { path = "../build_helper" }

src/libpanic_abort/lib.rs

@@ -184,7 +184,7 @@ pub enum ErrorKind {
 }
 
 impl ErrorKind {
-    fn as_str(&self) -> &'static str {
+    pub(crate) fn as_str(&self) -> &'static str {
         match *self {
             ErrorKind::NotFound => "entity not found",
             ErrorKind::PermissionDenied => "permission denied",

src/libpanic_unwind/lib.rs

@@ -26,6 +26,7 @@ const fn done<T>() -> *mut Arc<T> { 1_usize as *mut _ }
 unsafe impl<T> Sync for Lazy<T> {}
 
 impl<T> Lazy<T> {
+    #[unstable(feature = "sys_internals", issue = "0")] // FIXME: min_const_fn
     pub const fn new() -> Lazy<T> {
         Lazy {
             lock: Mutex::new(),

src/libstd/Cargo.toml

@@ -312,6 +312,8 @@
 #![feature(non_exhaustive)]
 #![feature(alloc_layout_extra)]
 #![feature(maybe_uninit)]
+#![cfg_attr(target_env = "sgx", feature(global_asm, range_contains, slice_index_methods,
+                                        decl_macro, coerce_unsized))]
 
 #![default_lib_allocator]
 
@@ -354,6 +356,12 @@ extern crate unwind;
 // testing gives test-std access to real-std lang items and globals. See #2912
 #[cfg(test)] extern crate std as realstd;
 
+#[cfg(target_env = "sgx")]
+#[macro_use]
+#[allow(unused_imports)] // FIXME: without `#[macro_use]`, get error: “cannot
+                         // determine resolution for the macro `usercalls_asm`”
+extern crate fortanix_sgx_abi;
+
 // The standard macros that are not built-in to the compiler.
 #[macro_use]
 mod macros;

src/libstd/io/error.rs

@@ -16,10 +16,11 @@ use net::{ntoh, hton, IpAddr, Ipv4Addr, Ipv6Addr};
 use option;
 use sys::net::netc as c;
 use sys_common::{FromInner, AsInner, IntoInner};
-use sys_common::net::lookup_host;
+use sys_common::net::LookupHost;
 use vec;
 use iter;
 use slice;
+use convert::TryInto;
 
 /// An internet socket address, either IPv4 or IPv6.
 ///
@@ -863,9 +864,9 @@ impl ToSocketAddrs for (Ipv6Addr, u16) {
     }
 }
 
-fn resolve_socket_addr(s: &str, p: u16) -> io::Result<vec::IntoIter<SocketAddr>> {
-    let ips = lookup_host(s)?;
-    let v: Vec<_> = ips.map(|mut a| { a.set_port(p); a }).collect();
+fn resolve_socket_addr(lh: LookupHost) -> io::Result<vec::IntoIter<SocketAddr>> {
+    let p = lh.port();
+    let v: Vec<_> = lh.map(|mut a| { a.set_port(p); a }).collect();
     Ok(v.into_iter())
 }
 
@@ -885,7 +886,7 @@ impl<'a> ToSocketAddrs for (&'a str, u16) {
             return Ok(vec![SocketAddr::V6(addr)].into_iter())
         }
 
-        resolve_socket_addr(host, port)
+        resolve_socket_addr((host, port).try_into()?)
     }
 }
 
@@ -899,22 +900,7 @@ impl ToSocketAddrs for str {
             return Ok(vec![addr].into_iter());
         }
 
-        macro_rules! try_opt {
-            ($e:expr, $msg:expr) => (
-                match $e {
-                    Some(r) => r,
-                    None => return Err(io::Error::new(io::ErrorKind::InvalidInput,
-                                                      $msg)),
-                }
-            )
-        }
-
-        // split the string by ':' and convert the second part to u16
-        let mut parts_iter = self.rsplitn(2, ':');
-        let port_str = try_opt!(parts_iter.next(), "invalid socket address");
-        let host = try_opt!(parts_iter.next(), "invalid socket address");
-        let port: u16 = try_opt!(port_str.parse().ok(), "invalid port value");
-        resolve_socket_addr(host, port)
+        resolve_socket_addr(self.try_into()?)
     }
 }
 

src/libstd/io/lazy.rs

@@ -112,11 +112,15 @@ fn hton<I: NetInt>(i: I) -> I { i.to_be() }
 fn ntoh<I: NetInt>(i: I) -> I { I::from_be(i) }
 
 fn each_addr<A: ToSocketAddrs, F, T>(addr: A, mut f: F) -> io::Result<T>
-    where F: FnMut(&SocketAddr) -> io::Result<T>
+    where F: FnMut(io::Result<&SocketAddr>) -> io::Result<T>
 {
+    let addrs = match addr.to_socket_addrs() {
+        Ok(addrs) => addrs,
+        Err(e) => return f(Err(e))
+    };
     let mut last_err = None;
-    for addr in addr.to_socket_addrs()? {
-        match f(&addr) {
+    for addr in addrs {
+        match f(Ok(&addr)) {
             Ok(l) => return Ok(l),
             Err(e) => last_err = Some(e),
         }

src/libstd/lib.rs

@@ -29,7 +29,7 @@ use intrinsics;
 use mem;
 use ptr;
 use raw;
-use sys::stdio::{Stderr, stderr_prints_nothing};
+use sys::stdio::panic_output;
 use sys_common::rwlock::RWLock;
 use sys_common::thread_info;
 use sys_common::util;
@@ -193,7 +193,6 @@ fn default_hook(info: &PanicInfo) {
             None => "Box<Any>",
         }
     };
-    let mut err = Stderr::new().ok();
     let thread = thread_info::current_thread();
     let name = thread.as_ref().and_then(|t| t.name()).unwrap_or("<unnamed>");
 
@@ -215,17 +214,14 @@ fn default_hook(info: &PanicInfo) {
         }
     };
 
-    let prev = LOCAL_STDERR.with(|s| s.borrow_mut().take());
-    match (prev, err.as_mut()) {
-       (Some(mut stderr), _) => {
-           write(&mut *stderr);
-           let mut s = Some(stderr);
-           LOCAL_STDERR.with(|slot| {
-               *slot.borrow_mut() = s.take();
-           });
-       }
-       (None, Some(ref mut err)) => { write(err) }
-       _ => {}
+    if let Some(mut local) = LOCAL_STDERR.with(|s| s.borrow_mut().take()) {
+       write(&mut *local);
+       let mut s = Some(local);
+       LOCAL_STDERR.with(|slot| {
+           *slot.borrow_mut() = s.take();
+       });
+    } else if let Some(mut out) = panic_output() {
+        write(&mut out);
     }
 }
 
@@ -485,7 +481,7 @@ fn rust_panic_with_hook(payload: &mut dyn BoxMeUp,
             // Some platforms know that printing to stderr won't ever actually
             // print anything, and if that's the case we can skip the default
             // hook.
-            Hook::Default if stderr_prints_nothing() => {}
+            Hook::Default if panic_output().is_none() => {}
             Hook::Default => {
                 info.set_payload(payload.get());
                 default_hook(&info);
@@ -494,7 +490,7 @@ fn rust_panic_with_hook(payload: &mut dyn BoxMeUp,
                 info.set_payload(payload.get());
                 (*ptr)(&info);
             }
-        }
+        };
         HOOK_LOCK.read_unlock();
     }
 

src/libstd/net/addr.rs

@@ -13,13 +13,14 @@ use io;
 use net::{Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr};
 use time::Duration;
 use sys::{unsupported, Void};
+use convert::TryFrom;
 
 pub extern crate libc as netc;
 
 pub struct TcpStream(Void);
 
 impl TcpStream {
-    pub fn connect(_: &SocketAddr) -> io::Result<TcpStream> {
+    pub fn connect(_: io::Result<&SocketAddr>) -> io::Result<TcpStream> {
         unsupported()
     }
 
@@ -105,7 +106,7 @@ impl fmt::Debug for TcpStream {
 pub struct TcpListener(Void);
 
 impl TcpListener {
-    pub fn bind(_: &SocketAddr) -> io::Result<TcpListener> {
+    pub fn bind(_: io::Result<&SocketAddr>) -> io::Result<TcpListener> {
         unsupported()
     }
 
@@ -155,7 +156,7 @@ impl fmt::Debug for TcpListener {
 pub struct UdpSocket(Void);
 
 impl UdpSocket {
-    pub fn bind(_: &SocketAddr) -> io::Result<UdpSocket> {
+    pub fn bind(_: io::Result<&SocketAddr>) -> io::Result<UdpSocket> {
         unsupported()
     }
 
@@ -271,7 +272,7 @@ impl UdpSocket {
         match self.0 {}
     }
 
-    pub fn connect(&self, _: &SocketAddr) -> io::Result<()> {
+    pub fn connect(&self, _: io::Result<&SocketAddr>) -> io::Result<()> {
         match self.0 {}
     }
 }
@@ -284,13 +285,31 @@ impl fmt::Debug for UdpSocket {
 
 pub struct LookupHost(Void);
 
+impl LookupHost {
+    pub fn port(&self) -> u16 {
+        match self.0 {}
+    }
+}
+
 impl Iterator for LookupHost {
     type Item = SocketAddr;
     fn next(&mut self) -> Option<SocketAddr> {
         match self.0 {}
     }
 }
 
-pub fn lookup_host(_: &str) -> io::Result<LookupHost> {
-    unsupported()
+impl<'a> TryFrom<&'a str> for LookupHost {
+    type Error = io::Error;
+
+    fn try_from(_v: &'a str) -> io::Result<LookupHost> {
+        unsupported()
+    }
+}
+
+impl<'a> TryFrom<(&'a str, u16)> for LookupHost {
+    type Error = io::Error;
+
+    fn try_from(_v: (&'a str, u16)) -> io::Result<LookupHost> {
+        unsupported()
+    }
 }

src/libstd/net/mod.rs

@@ -78,6 +78,6 @@ pub fn is_ebadf(err: &io::Error) -> bool {
 
 pub const STDIN_BUF_SIZE: usize = ::sys_common::io::DEFAULT_BUF_SIZE;
 
-pub fn stderr_prints_nothing() -> bool {
-    false
+pub fn panic_output() -> Option<impl io::Write> {
+    Stderr::new().ok()
 }

src/libstd/panicking.rs

@@ -32,7 +32,8 @@ unsafe impl Send for Thread {}
 unsafe impl Sync for Thread {}
 
 impl Thread {
-    pub unsafe fn new<'a>(stack: usize, p: Box<dyn FnBox() + 'a>) -> io::Result<Thread> {
+    // unsafe: see thread::Builder::spawn_unchecked for safety requirements
+    pub unsafe fn new(stack: usize, p: Box<dyn FnBox()>) -> io::Result<Thread> {
         let p = box p;
         let mut native: libc::pthread_t = mem::zeroed();
         let mut attr: libc::pthread_attr_t = mem::zeroed();

src/libstd/sys/cloudabi/shims/net.rs

@@ -48,6 +48,9 @@ cfg_if! {
     } else if #[cfg(target_arch = "wasm32")] {
         mod wasm;
         pub use self::wasm::*;
+    } else if #[cfg(target_env = "sgx")] {
+        mod sgx;
+        pub use self::sgx::*;
     } else {
         compile_error!("libstd doesn't compile for this platform yet");
     }