Skip to content

Commit 47d7525

Browse files
author
QuineDot
committed
Add a section about higher-ranked types with related notes elsewhere
1 parent 54d0a3d commit 47d7525

File tree

7 files changed

+325
-13
lines changed

7 files changed

+325
-13
lines changed

src/SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
- [`dyn` safety (object safety)](dyn-safety.md)
5050
- [`dyn Trait` lifetimes](dyn-trait-lifetime.md)
5151
- [Variance](dyn-covariance.md)
52+
- [Higher-ranked types](dyn-hr.md)
5253
- [Elision rules](dyn-elision.md)
5354
- [Basic guidelines](dyn-elision-basic.md)
5455
- [Advanced guidelines](dyn-elision-advanced.md)

src/dyn-any.md

Lines changed: 39 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ not unimaginable that all vtables will gain some lifetime-erased version of
5252
`TypeId`, but [related to some discussion below,](#why-static) this may not
5353
be as straightforward as it may sound.
5454

55-
## Dowcasting methods are not trait methods
55+
## Downcasting methods are not trait methods
5656

5757
Note that the *only* method available in the `Any` trait is `type_id`.
5858
[All of the downcasting methods](https://doc.rust-lang.org/std/any/trait.Any.html#implementations)
@@ -303,9 +303,9 @@ but an in-depth exploration is out of scope for this guide.
303303

304304
## A potential footgun around subtypes (subtitle: why not `const`?)
305305

306-
Let's take a minute to finally talk about types that *do* have a sub and supertype
307-
relationship in Rust! Types which are higher-ranked have this relationship. For
308-
example:
306+
Let's take a minute to talk about types that *do* have a sub and supertype
307+
relationship in Rust! Types which are [higher-ranked](./dyn-hr.md) have this
308+
relationship. For example:
309309
```rust
310310
// More explicitly, this is a `for<'any> fn(&'any str)` function pointer.
311311
// The type is higher-ranked over the lifetime.
@@ -319,7 +319,7 @@ let fp: fn(&'static str) = fp;
319319
```
320320

321321
And as it turns out, it is possible for two Rust types which are more than
322-
superficially syntactically different to be subtypes of one another. Some
322+
superficially syntactically different to be *subtypes of one another.* Some
323323
parts of the language consider the existence of such a relationship to mean
324324
that the two types are equal. Let's say they are semantically equal.
325325

@@ -349,6 +349,40 @@ bad enough that [some version with caveats about false negatives](https://github
349349
may be pursued. Personally I feel making the type system consistent would be the
350350
better solution and worth waiting for.
351351

352+
## More considerations around higher-ranked types
353+
354+
Even if the issue discussed above gets resolved and Rust becomes consistent about
355+
what types are equal, [higher-ranked types](./dyn-hr.md) introduce some nuance to
356+
be aware of. For example, when considering these two types:
357+
```rust
358+
trait Look<'s> {}
359+
type HR = dyn for<'any> Look<'any> + 'static;
360+
type ST = dyn Look<'static> + 'static;
361+
```
362+
`HR` is a *subtype* of `ST`, but not *the same type*.
363+
However, they both satisfy a `'static` bound:
364+
```rust
365+
#trait Look<'s> {}
366+
#type HR = dyn for<'any> Look<'any> + 'static;
367+
#type ST = dyn Look<'static> + 'static;
368+
fn assert_static<T: ?Sized + 'static>() {}
369+
370+
assert_static::<HR>();
371+
assert_static::<ST>();
372+
```
373+
As `'static` types, they have `TypeId`s. As distinct types, their
374+
`TypeId`s are different, even though one is a subtype of the other.
375+
376+
And this in turn means that you can't stop thinking about sub-and-super types
377+
by simply applying a `'static` bound. If you need to "disable" sub/super type
378+
coercions in a generic context for soundness, you must make that context invariant
379+
or take other steps to avoid a soundness hole, even if you have a `'static` bound.
380+
381+
[See this issue](https://github.com/rust-lang/rust/issues/85863) for a real-life
382+
example of such a soundness hole, and
383+
[this comment in particular](https://github.com/rust-lang/rust/issues/85863#issuecomment-872536139)
384+
exploring the sub/super type relationships of higher-ranked function pointers.
385+
352386
## The representation of `TypeId`
353387

354388
`TypeId` is intentionally opaque and subject to change. It was internally represented

src/dyn-covariance.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,20 @@
33
The `dyn Trait` lifetime is covariant, like the outer lifetime of a
44
reference. This means that whenever it is in a covariant type position,
55
longer lifetimes can be coerced into shorter lifetimes.
6-
76
```rust
87
# trait Trait {}
98
fn why_be_static<'a>(bx: Box<dyn Trait + 'static>) -> Box<dyn Trait + 'a> {
109
bx
1110
}
1211
```
13-
14-
The idea is that the lifetime represents the region where it is still valid
15-
to call methods on the trait object. Since it's valid to call methods anywhere
16-
in that region, it's also valid to restrict the region to some subset of itself
17-
-- i.e. to coerce the lifetime to be shorter.
12+
The trait object with the longer lifetime is a subtype of the trait object with
13+
the shorter lifetime, so this is a form of supertype coercion. [In the next
14+
section,](./dyn-hr.md) we'll look at another form of trait object subtyping.
15+
16+
The idea behind *why* trait object lifetimes are covariant is that the lifetime
17+
represents the region where it is still valid to call methods on the trait object.
18+
Since it's valid to call methods anywhere in that region, it's also valid to restrict
19+
the region to some subset of itself -- i.e. to coerce the lifetime to be shorter.
1820

1921
However, it turns out that the `dyn Trait` lifetime is even more flexible than
2022
your typical covariant lifetime.

src/dyn-hr.md

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# Higher-ranked types
2+
3+
Another feature of trait objects is that they can be *higher-ranked* over
4+
lifetime parameters of the trait:
5+
```rust
6+
// A trait with a lifetime parameter
7+
trait Look<'s> {
8+
fn method(&self, s: &'s str);
9+
}
10+
11+
// An implementation that works for any lifetime
12+
impl<'s> Look<'s> for () {
13+
fn method(&self, s: &'s str) {
14+
println!("Hi there, {s}!");
15+
}
16+
}
17+
18+
fn main() {
19+
// A higher-ranked trait object
20+
// vvvvvvvvvvvvvvvvvvvvvvvv
21+
let _bx: Box<dyn for<'any> Look<'any>> = Box::new(());
22+
}
23+
```
24+
The `for<'x>` part is a *lifetime binder* that introduces higher-ranked
25+
lifetimes. There can be more than one lifetime, and you can give them
26+
arbitrary names just like lifetime parameters on functions, structs,
27+
and so on.
28+
29+
You can only coerce to a higher-ranked trait object if you implement
30+
the trait in question for *all* lifetimes. For example, this doesn't
31+
work:
32+
```rust,compile_fail
33+
# trait Look<'s> { fn method(&self, s: &'s str); }
34+
impl<'s> Look<'s> for &'s i32 {
35+
fn method(&self, s: &'s str) {
36+
println!("Hi there, {s}!");
37+
}
38+
}
39+
40+
fn main() {
41+
let _bx: Box<dyn for<'any> Look<'any>> = Box::new(&0);
42+
}
43+
```
44+
`&'s i32` only implements `Look<'s>`, not `Look<'a>` for all lifetimes `'a`.
45+
46+
Similarly, this won't work either:
47+
```rust,compile_fail
48+
# trait Look<'s> { fn method(&self, s: &'s str); }
49+
impl Look<'static> for i32 {
50+
fn method(&self, s: &'static str) {
51+
println!("Hi there, {s}!");
52+
}
53+
}
54+
55+
fn main() {
56+
let _bx: Box<dyn for<'any> Look<'any>> = Box::new(0);
57+
}
58+
```
59+
60+
Implementing the trait with `'static` as the lifetime parameter is not the
61+
same thing as implementing the trait for any lifetime as the parameter.
62+
Traits and trait implementations don't have something like variance; the
63+
parameters of traits are always invariant and thus implementations are
64+
always for the explicit lifetime(s) only.
65+
66+
## Subtyping
67+
68+
There's a relationship between higher-ranked types like `dyn for<'any> Look<'any>`
69+
and non-higher-ranked types like `dyn Look<'x>` (for a single lifetime `'x`): the
70+
higher-ranked type is a subtype of the non-higher-ranked types. Thus you can
71+
coerce a higher-ranked type to a non-higher-ranked type with any concrete lifetime:
72+
```rust
73+
# trait Look<'s> { fn method(&self, s: &'s str); }
74+
fn as_static(bx: Box<dyn for<'any> Look<'any>>) -> Box<dyn Look<'static>> {
75+
bx
76+
}
77+
78+
fn as_whatever<'w>(bx: Box<dyn for<'any> Look<'any>>) -> Box<dyn Look<'w>> {
79+
bx
80+
}
81+
```
82+
83+
Note that this still isn't a form of variance for the *lifetime parameter* of the
84+
trait. This fails for example, because you can't coerce from `dyn Look<'static>`
85+
to `dyn Look<'w>`:
86+
```rust
87+
# trait Look<'s> { fn method(&self, s: &'s str); }
88+
# fn as_static(bx: Box<dyn for<'any> Look<'any>>) -> Box<dyn Look<'static>> { bx }
89+
fn as_whatever<'w>(bx: Box<dyn for<'any> Look<'any>>) -> Box<dyn Look<'w>> {
90+
as_static(bx)
91+
}
92+
```
93+
94+
As a supertype coercion, going from higher-ranked to non-higher-ranked can
95+
apply even in a covariant nested context,
96+
[just like non-higher-ranked supertype coercions:](./dyn-covariance.md#variance-in-nested-context)
97+
```rust
98+
# trait Look<'s> {}
99+
fn foo<'l: 's, 's, 'p>(v: *const Box<dyn for<'any> Look<'any> + 'l>) -> *const Box<dyn Look<'p> + 's> {
100+
v
101+
}
102+
```
103+
104+
## `Fn` traits and `fn` pointers
105+
106+
The `Fn` traits ([`FnOnce`](https://doc.rust-lang.org/std/ops/trait.FnOnce.html),
107+
[`FnMut`](https://doc.rust-lang.org/std/ops/trait.FnMut.html),
108+
and [`Fn`](https://doc.rust-lang.org/std/ops/trait.Fn.html))
109+
have special-cased syntax. For one, you write them out to look more like
110+
a function, using `(TypeOne, TypeTwo)` to list the input parameters and
111+
`-> ResultType` to list the associated type. But for another, elided
112+
input lifetimes are sugar that introduces higher-ranked bindings.
113+
114+
For example, these two trait object types are the same:
115+
```rust
116+
fn identity(bx: Box<dyn Fn(&str)>) -> Box<dyn for<'any> Fn(&'any str)> {
117+
bx
118+
}
119+
```
120+
121+
This is similar to how elided lifetimes work for function declarations
122+
as well, and indeed, the same output lifetime elision rules also apply:
123+
```rust
124+
// The elided input lifetime becomes a higher-ranked lifetime
125+
// The elided output lifetime is the same as the single input lifetime
126+
// (underneath the binder)
127+
fn identity(bx: Box<dyn Fn(&str) -> &str>) -> Box<dyn for<'any> Fn(&'any str) -> &'any str> {
128+
bx
129+
}
130+
```
131+
```rust,compile_fail
132+
// Doesn't compile as what the output lifetime should be is
133+
// considered ambiguous
134+
fn ambiguous(bx: Box<dyn Fn(&str, &str) -> &str>) {}
135+
136+
// Here's a possible fix, which is also an example of
137+
// multiple lifetimes in the binder
138+
fn first(bx: Box<dyn for<'a, 'b> Fn(&'a str, &'b str) -> &'a str>) {}
139+
```
140+
141+
Function pointers are another example of types which can be higher-ranked
142+
in Rust. They have analogous syntax and sugar to function declarations
143+
and the `Fn` traits.
144+
```rust
145+
fn identity(fp: fn(&str) -> &str) -> for<'any> fn(&'any str) -> &'any str {
146+
fp
147+
}
148+
```
149+
150+
### Syntactic inconsistencies
151+
152+
There are some inconsistencies around the syntax for function declarations,
153+
function pointer types, and the `Fn` traits involving the "names" of the
154+
input arguments.
155+
156+
First of all, only function (method) declarations can make use of the
157+
shorthand `self` syntaxes for receivers, like `&self`:
158+
```rust
159+
# struct S;
160+
impl S {
161+
fn foo(&self) {}
162+
// ^^^^^
163+
}
164+
```
165+
This exception is pretty unsurprising as the `Self` alias only exists
166+
within those implementation blocks.
167+
168+
Each non-`self` argument in a function declaration is an
169+
[irrefutable pattern](https://doc.rust-lang.org/reference/items/functions.html#function-parameters)
170+
followed by a type annotation. It is an error to leave out the pattern;
171+
if you don't use the argument (and thus don't need to name it), you
172+
still need to use at least the wildcard pattern.
173+
```rust,compile_fail
174+
fn this_works(_: i32) {}
175+
fn this_fails(i32) {}
176+
```
177+
There is
178+
[an accidental exception](https://rust-lang.github.io/rfcs/1685-deprecate-anonymous-parameters.html)
179+
to this rule, but it was removed in Edition 2018 and thus is only
180+
available on Edition 2015.
181+
182+
In contrast, each argument in a function pointer can be
183+
- An *identifier* followed by a type annotation (`i: i32`)
184+
- `_` followed by a type annotation (`_: i32`)
185+
- Just a type name (`i32`)
186+
187+
So these all work:
188+
```rust
189+
let _: fn(i32) = |_| {};
190+
let _: fn(i: i32) = |_| {};
191+
let _: fn(_: i32) = |_| {};
192+
```
193+
But *actual* patterns [are not allowed:](https://doc.rust-lang.org/stable/error_codes/E0561.html)
194+
```rust,compile_fail
195+
let _: fn(&i: &i32) = |_| {};
196+
```
197+
The idiomatic form is to just use the type name.
198+
199+
It's also allowed [to have colliding names in function pointer
200+
arguments,](https://github.com/rust-lang/rust/issues/33995) but this
201+
is a property of having no function body -- so it's also possible in
202+
a trait method declaration, for example. It is also related to the
203+
Edition 2015 exception for anonymous function arguments mentioned
204+
above, and may be deprecated eventually.
205+
```rust
206+
trait Trait {
207+
fn silly(a: u32, a: i32);
208+
}
209+
210+
let _: fn(a: u32, a: i32) = |_, _| {};
211+
```
212+
213+
Finally, each argument in the `Fn` traits can *only* be a type name:
214+
no identifiers, `_`, or patterns allowed.
215+
```rust,compile_fail
216+
// None of these compile
217+
let _: Box<dyn Fn(i: i32)> = Box::new(|_| {});
218+
let _: Box<dyn Fn(_: i32)> = Box::new(|_| {});
219+
let _: Box<dyn Fn(&_: &i32)> = Box::new(|_| {});
220+
```
221+
222+
Why the differences? One reason is that
223+
[patterns are gramatically incompatible with anonymous arguments,
224+
apparently.](https://github.com/rust-lang/rust/issues/41686#issuecomment-366611096)
225+
I'm uncertain as to why identifiers are accepted on function pointers,
226+
however, or more generally why the `Fn` sugar is inconsistent with
227+
function pointer types. But the simplest explanation is that function
228+
pointers existed first with nameable parameters for whatever reason,
229+
whereas the `Fn` sugar is for trait input type parameters which also
230+
do not have names.
231+
232+
## Higher-ranked trait bounds
233+
234+
You can also apply higher-ranked trait bounds (HRTBs) to generic
235+
type parameters, using the same syntax:
236+
```rust
237+
# trait Look<'s> { fn method(&self, s: &'s str); }
238+
fn box_it_up<'t, T>(t: T) -> Box<dyn for<'any> Look<'any> + 't>
239+
where
240+
T: for<'any> Look<'any> + 't,
241+
{
242+
Box::new(t)
243+
}
244+
```
245+
246+
The sugar for `Fn` like traits applies here as well. You've probably
247+
already seen bounds like this on methods that take closures:
248+
```rust
249+
# struct S;
250+
# impl S {
251+
fn map<'s, F, R>(&'s self, mut f: F) -> impl Iterator<Item = R> + 's
252+
where
253+
F: FnMut(&[i32]) -> R + 's
254+
{
255+
// This part isn't the point ;-)
256+
[].into_iter().map(f)
257+
}
258+
# }
259+
```
260+
261+
That bound is actually `F: for<'x> FnMut(&'x [i32]) -> R + 's`.
262+
263+
## That's all about higher-ranked types for now
264+
265+
Hopefully this has given you a decent overview of higher-ranked
266+
types, HRTBs, and how they relate to the `Fn` traits. There
267+
are a lot more details and nuances to those topics and related
268+
concepts such as closures, as you might imagine. However, an
269+
exploration of those topics deserves its own dedicated guide, so
270+
we won't see too much more about higher-ranked types in this
271+
tour of `dyn Trait`.

src/dyn-trait-box-impl.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Oh yeah, that last one. [Remember what we said before?](dyn-trait-impls.md#boxd
5757
The compiler isn't going to just guess what to do here (and couldn't if,
5858
say, we needed a return value). We can't move the `dyn Trait` out of
5959
the `Box` because it's unsized. And we can't
60-
[downcast from `dyn Trait`](./dyn-any.md#dowcasting-methods-are-not-trait-methods)
60+
[downcast from `dyn Trait`](./dyn-any.md#downcasting-methods-are-not-trait-methods)
6161
either; even if we could, it would rarely help here, as we'd have to both
6262
impose a `'static` constraint and also know every type that implements our
6363
trait to attempt downcasting on each one (or have some other clever scehme

src/dyn-trait-coercions.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,9 @@ Despite that, this unsizing coercion can still [not happen in a nested context.]
202202

203203
However, [in a future section](http://127.0.0.1:3000/dyn-covariance.html) we'll see
204204
how variance can allow shortening the trait object lifetime even in nested context,
205-
provided that context is also covariant.
205+
provided that context is also covariant. [The section after that about higher-ranked
206+
types](./dyn-hr.md) explores another lifetime-related coercion which could also be
207+
considered reflexive.
206208

207209
## Supertrait upcasting
208210

0 commit comments

Comments
 (0)