Subtyping Reconstruction

[e2e4b6b7]

This issue is for discussion of the proposal to introduce Subtyping Reconstruction into the Kotlin language.
The full proposal text is available here: proposals/subtyping-reconstruction.md

The KEEP presents the Subtyping Reconstruction technique that introduces smart casts for generics.

Please, use this issue for the discussion on the substance of the proposal. For minor corrections to the text, please open comment directly in the PR: #410.

[kyay10]

Just clarifying, this would also be fine, right?

fun <T> evalWhen(e: Expr<T>): T = when (e) {
    is IntLit -> 42
}

[kyay10]

From "Can we fix these problems":

Fundamentally, this means we may need to add ways to encode recursive types (aka µ types) or some other ways to propagate such complex (e.g., recursive) constraints in the type system.

We leave the solutions to these problems for possible future work.

Would introducing Shape interfaces do the trick here? Just food for thought

[e2e4b6b7]

@kyay10,

Just clarifying, this would also be fine, right?

fun <T> evalWhen(e: Expr<T>): T = when (e) {
    is IntLit -> 42
}

Yes, it works in the same way as with any other Int-typed expression on the rhs. More precisely, in this case it is: T :> Int :> ILT and everything is fine.

Would introducing Shape interfaces do the trick here?

Yes, inferring only material interfaces will simplify everything, but there are some other issues. Therefore, we decided to address them separately to keep things manageable.

[RossTate]

I'm concerned about the approach to supporting this feature. The constraint resolution algorithm seems to have a lot of corner cases. For example, when resolving a constraint on a type variable, it goes through all the bounds on the type variable, but then it seems it can eventually generate a new bound on that same variable, which seems to create a non-terminating loop (or a loop that gets cut off arbitrarily). Even more worryingly, it seems hard to ensure the newly generated bounds conform to the invariants we can use to make typing efficient. For example, if every type variable has at most one upper bound, we can take advantage of this to keep subtyping and inference polynomial-time. But the proposed approach seems to enable adding arbitrary upper bounds to type variables, making these problems exponential-time. (I know some of these problems currently arise with smart casts, but there might be ways to fix this, whereas the proposed approach seems to make them unfixable.)

This feature seems to be expected to be used by advanced users with low frequency. Given that, would it make sense to look into decidable approaches that require a little more guidance/annotation from the user? For example, here's my stab at adapting the syntax for a known decidable approach:

when (variable of SuperClass<T>): WhenType {
   is SubClass<U> -> expr
   ...
}

Here we add this of SuperClass<T> clause, which declares type variables T to represent variable's type argument for SuperClass.
The WhenType then uses T to describe the type of the when expression, e.g. List<T>.
In the case is SubClass<U>, the U is yet another type variable, this time representing the value's type argument for SubClass (if it belongs to SubClass).
Suppose we have SubClass<X> : SuperClass<Set<X>>; then in this case the T in WhenType gets replaced by Set<U>, essentially incorporating the knowledge we gain about T from the fact that variable belongs to SubClass.
So, if WhenType were List<T>, then expr would need to have type List<Set<U>> assuming variable has type SubClass<U>.
We do this check for each case.
(When the U would not be used in a case, the user can elide them.)
Afterwards, if know that variable has type SuperClass<A>, and if WhenType were List<T>, then the when expression would have type List<A>.

In other words, the WhenType annotation tells us how to translate between the (possibly conflicting) information ensured by the type of variable and the invariants guaranteed by the various subclasses of SuperClass.
(I'm shoving details with variance and such under the rug.)

That's probably not very clear, so here is how it looks applied to various examples in the KEEP:

private val caller: Caller<M> = when (oldCaller of Caller<T>): Caller<T> {
    is CallerImpl.Method.BoundStatic -> { // Here T is substituted with ReflectMethod
        val receiverType = (descriptor.extensionReceiverParameter ?: descriptor.dispatchReceiverParameter)?.type
        if (receiverType != null && receiverType.needsMfvcFlattening()) {
            val unboxMethods = getMfvcUnboxMethods(receiverType.asSimpleType())!!
            val boundReceiverComponents = unboxMethods.map { it.invoke(oldCaller.boundReceiver) }.toTypedArray()
            CallerImpl.Method.BoundStaticMultiFieldValueClass(oldCaller.member, boundReceiverComponents)
        } else {
            oldCaller
        }
    }
    else -> oldCaller
}

sealed interface Expr<out T>
class IntLit(val i: Int) : Expr<Int>

fun <T> evalWhen(e: Expr<T>): T = when (e of Expr<X>): X {
    is IntLit -> e.i
}

sealed interface EqT<A, B> {
  class Evidence<X> : EqT<X, X>
}

class Data<S, D> {
  val value: D = ...
  fun coerceJsonToString(ev: EqT<D, JsonNode>): Data<S, String> =
    Data(when (ev of EqT<A,B>): (A) -> B { is Evidence -> { it } } (this.value))
}

// The above pattern of converting a value is pretty common, so here's a shorthand syntax for it:
class Data<S, D> {
  val value: D = ...
  fun coerceJsonToString(ev: EqT<D, JsonNode>): Data<S, String> =
    Data(when (ev of EqT<A,B>): (val x: A = this.value) -> B { is Evidence -> x })
}

sealed interface Chart<A> {
    fun draw(chartData: A)
}
class PieChart : Chart<PieData>
class XYChart : Chart<XYData>
fun modifyPieData(PieData): PieData {...}

fun <A> Chart<A>.customDraw(chartData: A): Unit =
  when (this of Chart<X>): (chartData: X) -> Unit {
    is PieChart -> {
      draw(modifyPieData(chartData))
    }
    else -> draw(chartData)
  }

interface A<in T, in V>
interface A1<in V> : A<Int, V>
interface A2<in T> : A<T, Int>

fun f(v: A1<*>): A2<Int> =
    when (v of A<T,V>): A2<T> {
        is A2 -> v
        else -> throw Exception()
    }
}

(I believe this is known as indexed pattern matching, albeit adapted to subtyping and inheritance rather than cases of an inductive data type.)

[kyay10]

@RossTate would this also basically allow existential types in subexpressions? What I mean is, Kotlin seems to have recently gained the ability to do something like:

data class Foo<T>(val list: MutableList<T>, val set: MutableSet<T>)
fun <T> bar(list: MutableList<T>, set: MutableSet<T>) { }
fun Any.baz() = when (this) {
  is Foo<*> -> bar(list, set)
  else -> {}
}

Where internally the type T of this@baz is the type argument to bar. Weirdly, the very similar

fun baz(foo: Any) = when (foo) {
  is Foo<*> -> bar(foo.list, foo.set)
  else -> {}
}

Does not compile. It sounds like your proposed idea would cover this usecase nicely too by allowing:

fun baz(foo: Any) = when (foo) {
  is Foo<T> -> bar<T>(foo.list, foo.set)
  else -> {}
}

Where T is a fresh type parameter (and perhaps even the type argument to bar would be unnecessary and inferred automatically)
Is that correct? Or would this be limited to only superclass-subclass situations?

[RossTate]

@kyay10 What you're touching upon is type capturing. There's a lot I could go into, but I think I'll just focus on the specific question you asked: yes, what I described would support "opening" the existential type. All of the following would type-check:

fun baz1(foo: Any) = when (foo of Any) {
  is Foo -> bar(foo.list, foo.set)
  else -> {}
}

and

fun baz2(foo: Foo<*>) = when (foo of Foo<T>) {
  is Foo -> bar(foo.list, foo.set)
}

and

fun baz3(foo: Foo<*>) = when (foo of Foo<T>) {
  is Foo<U> -> bar<U>(foo.list, foo.set)
}

They might be overkill for that particular example, but hopefully you can see how they can generalize to more complex examples.

[kyay10]

Apologies for the tangential conversation @RossTate. Would something like

fun baz4(foo: Any) = when (foo of Any) {
  is Foo<U> -> bar<U>(foo.list, foo.set)
  else -> {}
}

also be supported? Or would it have to instead be the mouthful of:

fun baz4(foo: Any) = when (foo) {
  is Foo<*> -> when(foo of Foo<T>) { is Foo -> bar<T>(foo.list, foo.set) }
  else -> {}
}

If the automatic approach in the KEEP cannot be made polynomial (maybe there's a way to modify the algorithm?), then I would like this more-explicit syntax, especially because it allows that explicit type-capturing. In fact, even if the automatic approach works, I like the explicit syntax anyways since it clarifies what's going on for the reader.

[RossTate]

Yes, your first (more concise) version of baz4 would work too.