Fix #16018: Existential widening for wildcard arguments

He-Pin · He-Pin · commit dc0d66baf017 · 2026-05-24T05:34:03.000+08:00
The Scala 3 typer was rejecting subtypings of the form F[? >: lo <: hi] <: F[X] for covariant or contravariant `F`, even when `X = hi` (covariant) or `X = lo` (contravariant). This is a regression against Scala 2 and surfaced as the akka-derived report in #16018: def f[M](xs: java.lang.Iterable[? <: Container[Any, ? <: M]]) : Seq[Container[Any, M]] = seqOf(xs).collect { case g: SubContainer[Any, M] @unchecked => g case other => other } Once `Inferencing.captureWildcards` lifted the Java wildcard to a `TypeBox.CAP` skolem and pattern matching narrowed the first case body to `pat ∩ ?N.CAP`, the inferred result reduced to a chain of subtype checks that ultimately required `F[? <: M] <:< F[M]` for covariant `F`. That check went through `TypeComparer.compareCaptured` and was answered in the negative. Type-theoretic justification ---------------------------- For a covariant type constructor `F` and an existential `∃X. X <: hi`, ⨆{F[X] | X <: hi} = F[hi] (covariant supremum) — the supremum is attained at `X = hi`, by covariance. The compiler must admit that supremum as a subtype of `F[hi]`. Dually for contravariance with the lower bound: ⨅{G[X] | X >: lo} = G[lo] (contravariant infimum) This is the standard existential-elimination rule for variant occurrences. Bug --- `compareCaptured` was checking v > 0: isSubType(paramBounds(tparam).hi, arg2) v < 0: isSubType(arg2, paramBounds(tparam).lo) i.e. it asked whether the *declared parameter bound* (which collapses to `Any` / `Nothing` for unconstrained type parameters) conforms to `arg2`, instead of asking whether the *wildcard's own bound* (`arg1.hi` / `arg1.lo`) does. That rejected every interesting case. Fix --- In `TypeComparer.compareCaptured` use the wildcard's own hi/lo, intersected (resp. unioned) with the declared parameter bounds to preserve soundness in the corner case where a wildcard is wider than its parameter permits: v > 0: isSubType(arg1.hi & paramBounds(tparam).hi, arg2) v < 0: isSubType(arg2, arg1.lo | paramBounds(tparam).lo) Tests (directional matrix) -------------------------- - `tests/pos/i16018.scala` — mbovel's minimization plus a pure-Scala matrix that exercises subtype patterns + fallback `case other` over wildcards. - `tests/pos/i16018b.scala` — the akka-style shapes that motivated the ticket: `java.lang.Iterable[? <: G[? <: M]]` / `java.util.List[…]` routed through `seqOf`, then `collect` / `map` with subtype + fallback patterns. - `tests/pos/i16018c.scala` — positive half of the variance × bound matrix for the fix itself: covariant + upper, contravariant + lower, nested, via method type params, with parameter-bound tightening. - `tests/neg/i16018c.scala` — negative half of the matrix: invariant containers, covariant + lower, contravariant + upper, and the soundness guard against wildcards wider than the required type. These must remain rejected after the fix. Verified locally ---------------- - All four `i16018*` tests: 16/16 pass via `sbt 'scala3-compiler-bootstrapped/testOnly … CompilationTests'` with `-Ddotty.tests.filter=i16018`. - `wildcard` / `match` / `variance` filter subsets: 16/16 each. - Full `CompilationTests` run: the remaining 4 failures (`tests/run/i13358.scala`, `tests/run/lazy-*.scala`, `tests/run/t5552.scala`, `tests/run/t7406.scala`, `tests/run/isInstanceOf-eval.scala`, `tests/run/i24553.scala`, `tests/pos-custom-args/captures/fill-cbn.scala`, and the capture-checking neg suite) reproduce on `main` without this change and are caused by JDK 25 environment drift (`sun.misc.Unsafe` deprecation messages on stdout, the introduction of `java.lang.IO`, and removal of `native` from `Object.wait`).
diff --git a/compiler/src/dotty/tools/dotc/core/TypeComparer.scala b/compiler/src/dotty/tools/dotc/core/TypeComparer.scala
@@ -1890,10 +1890,25 @@ class TypeComparer(@constructorOnly initctx: Context) extends ConstraintHandling
                 // The captured reference could be illegal and cause a
                 // TypeError to be thrown in argDenot
                 false
+            // Existential widening for wildcard arguments (issue #16018):
+            // `arg1` is the bounds carried by a `? >: lo <: hi` wildcard. For
+            // `arg2` to be a (covariant / contravariant) supertype of `arg1`,
+            // we only need the *wildcard's own* hi/lo to conform — the
+            // declared parameter bounds (`paramBounds(tparam)`) only ever
+            // *constrain* the wildcard further, never relax it. The earlier
+            // formulation used `paramBounds(tparam).hi`, which collapses to
+            // `Any` for unconstrained type parameters and rejects valid
+            // subtypings such as `F[? <: M] <: F[M]` for covariant `F`
+            // (mathematically: ⨆{F[X] | X <: M} = F[M] by covariance, since
+            // the supremum is attained at X = M). Intersecting with
+            // `paramBounds(tparam)` keeps soundness in the rare case where
+            // the wildcard's bound is *wider* than what `tparam` permits.
             else if (v > 0)
-              isSubType(paramBounds(tparam).hi, arg2)
+              val effectiveHi = arg1.hi & paramBounds(tparam).hi
+              isSubType(effectiveHi, arg2)
             else if (v < 0)
-              isSubType(arg2, paramBounds(tparam).lo)
+              val effectiveLo = arg1.lo | paramBounds(tparam).lo
+              isSubType(arg2, effectiveLo)
             else
               false
           case _ =>
diff --git a/tests/neg/i16018c.check b/tests/neg/i16018c.check
@@ -0,0 +1,49 @@
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:18:50 -----------------------------------------------------------
+18 |  def inv_upper_bad[M](xs: Inv[? <: M]): Inv[M] = xs   // error
+   |                                                  ^^
+   |                                                  Found:    (xs : Test.Inv[? <: M])
+   |                                                  Required: Test.Inv[M]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:19:50 -----------------------------------------------------------
+19 |  def inv_lower_bad[M](xs: Inv[? >: M]): Inv[M] = xs   // error
+   |                                                  ^^
+   |                                                  Found:    (xs : Test.Inv[? >: M])
+   |                                                  Required: Test.Inv[M]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:26:47 -----------------------------------------------------------
+26 |  def co_lower_bad[M](xs: Co[? >: M]): Co[M] = xs       // error
+   |                                               ^^
+   |                                               Found:    (xs : Test.Co[? >: M])
+   |                                               Required: Test.Co[M]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:32:59 -----------------------------------------------------------
+32 |  def contra_upper_bad[M](xs: Contra[? <: M]): Contra[M] = xs // error
+   |                                                           ^^
+   |                                                           Found:    (xs : Test.Contra[? <: M])
+   |                                                           Required: Test.Contra[M]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:40:50 -----------------------------------------------------------
+40 |  def co_too_wide(xs: Co[? <: Any]): Co[Holder] = xs    // error
+   |                                                  ^^
+   |                                                  Found:    (xs : Test.Co[?])
+   |                                                  Required: Test.Co[Test.Holder]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:56:73 -----------------------------------------------------------
+56 |  def co_guard_upper_boundary(xs: CoBounded[? <: Any]): CoBounded[Sub] = xs // error
+   |                                                                         ^^
+   |                                                                         Found:    (xs : Test.CoBounded[?])
+   |                                                                         Required: Test.CoBounded[Test.Sub]
+   |
+   | longer explanation available when compiling with `-explain`
+-- [E007] Type Mismatch Error: tests/neg/i16018c.scala:68:91 -----------------------------------------------------------
+68 |  def contra_guard_lower_boundary(xs: ContraBounded[? >: Nothing]): ContraBounded[Super] = xs // error
+   |                                                                                           ^^
+   |                                                                        Found:    (xs : Test.ContraBounded[?])
+   |                                                                        Required: Test.ContraBounded[Test.Super]
+   |
+   | longer explanation available when compiling with `-explain`
diff --git a/tests/neg/i16018c.scala b/tests/neg/i16018c.scala
@@ -0,0 +1,68 @@
+// Negative half of the directional matrix for the fix to
+// `TypeComparer.compareCaptured` (#16018 / Existential widening for wildcard
+// arguments).
+//
+// These shapes must remain *rejected* after the fix: the fix only adds
+// widening that is justified by variance — covariant + upper bound,
+// contravariant + lower bound. Any other combination is unsound and must
+// keep the same diagnostic it had before. If any of these `//<space>error`
+// markers stops firing, the fix has overreached.
+
+object Test:
+
+  class Co[+T]
+  class Contra[-T]
+  class Inv[T]
+
+  // ---- A. Invariant containers never widen the wildcard ---------------
+  def inv_upper_bad[M](xs: Inv[? <: M]): Inv[M] = xs   // error
+  def inv_lower_bad[M](xs: Inv[? >: M]): Inv[M] = xs   // error
+
+  // ---- B. Covariant with a *lower*-bounded wildcard does not widen ----
+  //
+  // For covariant F, `F[? >: lo]` is the type of F[X] for some X >: lo.
+  // That has no useful relationship to F[lo] — picking X = Any is
+  // permitted yet gives F[Any], not F[lo].
+  def co_lower_bad[M](xs: Co[? >: M]): Co[M] = xs       // error
+
+  // ---- C. Contravariant with an *upper*-bounded wildcard does not widen
+  //
+  // Dual to (B): for contravariant G, `G[? <: hi]` makes no commitment
+  // that the picked X is `hi` itself; it could be `Nothing`.
+  def contra_upper_bad[M](xs: Contra[? <: M]): Contra[M] = xs // error
+
+  // ---- D. The required type's hi is strictly tighter than the wildcard
+  //
+  // `Co[? <: Any]` is the trivial widening shape (any T). It must not be
+  // accepted at `Co[Holder]` because the wildcard's hi (`Any`) is wider
+  // than the target.
+  trait Holder
+  def co_too_wide(xs: Co[? <: Any]): Co[Holder] = xs    // error
+
+  // ---- E. Upper-boundary check, covariant ------------------------------
+  //
+  // Boundary companion to `soundness_guard_co` in tests/pos/i16018c.scala.
+  // For `CoBounded[+T <: Holder]` fed `CoBounded[? <: Any]`, the effective
+  // wildcard hi computed by the fix is `Any & Holder = Holder`. The
+  // *positive* counterpart pins conformance at `CoBounded[Holder]`
+  // (admits THE FIX, rejects NAIVE). This negative pins that conformance
+  // stops there: `CoBounded[Sub]` for `Sub <: Holder` must still be
+  // rejected. Note that all three of OLD / NAIVE / THE FIX reject this
+  // case (each computes a different LHS but each LHS is ⊄ Sub), so this
+  // case is a *boundary* test, not a distinguishing one — read the pos
+  // counterpart for the implementation-distinguishing check.
+  class Sub extends Holder
+  class CoBounded[+T <: Holder]
+  def co_guard_upper_boundary(xs: CoBounded[? <: Any]): CoBounded[Sub] = xs // error
+
+  // ---- F. Lower-boundary check, contravariant --------------------------
+  //
+  // Dual to (E). The positive counterpart `soundness_guard_contra` pins
+  // conformance at `ContraBounded[Holder]`. This negative pins that a
+  // *strict supertype* of `Holder` is not reached (variance flips the
+  // direction). Same caveat as (E): all three implementations reject,
+  // so this is a boundary check.
+  class Super
+  class HolderS extends Super
+  class ContraBounded[-T >: HolderS]
+  def contra_guard_lower_boundary(xs: ContraBounded[? >: Nothing]): ContraBounded[Super] = xs // error
diff --git a/tests/pos/i16018-orig.scala b/tests/pos/i16018-orig.scala
@@ -0,0 +1,39 @@
+// The original akka-minimized reproduction from #16018 (verbatim, modulo
+// `_` → `?` syntax migration). Posted by @He-Pin on 2022-09-17:
+//
+//   https://github.com/scala/scala3/issues/16018#issuecomment-1250066489
+//
+// Pre-fix this rejected with
+//
+//   Found:    (other : ?1.CAP)
+//   Required: Main.Source[T, M] & ?1.CAP
+//   where:    ?1 is an unknown value of type
+//             scala.runtime.TypeBox[Nothing, Main.Graph[Main.SourceShape[T], ? <: M]]
+//
+// The directional minimizations of this same shape live in
+// tests/pos/i16018b.scala (the matrix); this file pins the exact
+// reproduction so that the original ticket cannot regress in either
+// direction.
+
+import scala.collection.immutable
+
+object Main:
+
+  class Source[+Out, +Mat] extends Graph[SourceShape[Out], Mat]
+
+  class Shape
+  class SourceShape[+T] extends Shape
+  class Graph[+S <: Shape, +M]
+
+  def combine[T, U, M](sources: java.util.List[? <: Graph[SourceShape[T], ? <: M]])
+      : Source[U, java.util.List[M]] =
+    val seq: immutable.Seq[Graph[SourceShape[T], M]] =
+      if sources != null then
+        immutableSeq(sources).collect {
+          case source: Source[T, M] @unchecked => source
+          case other                           => other
+        }
+      else immutable.Seq()
+    ???
+
+  def immutableSeq[T](iterable: java.lang.Iterable[T]): immutable.Seq[T] = ???
diff --git a/tests/pos/i16018.scala b/tests/pos/i16018.scala
@@ -0,0 +1,58 @@
+// Regression test for #16018: type inference with `_ <: T` wildcards in
+// pattern matching over capture-converted scrutinees.
+//
+// Coverage axes for the positive matrix:
+//   1. The mbovel minimization (single wildcard via List.map) — was fixed
+//      since 3.2.0, kept here as a pin.
+//   2. Pure-Scala collections — never trigger the capture-conversion path.
+//   3. Identity / generic Scala helpers over the wildcard-bearing collection.
+//
+// The dual cases reached through Java-generic interop (java.lang.Iterable /
+// java.util.List), which were the original akka regression, are in
+// tests/pos/i16018b.scala.
+
+object Test:
+
+  class Box[T](val value: T)
+  class Container[+S, +M]
+  class SubContainer[+S, +M] extends Container[S, M]
+
+  // ---- 1. mbovel's minimized case from #16018 ------------------------
+  def f1[T](l: List[Box[? <: T]]): List[T] = l.map(_.value)
+
+  // 1b. Variant: returning the wildcard-bearing element preserved.
+  def f1b[T](l: List[Box[? <: T]]): List[Box[? <: T]] = l.map(identity)
+
+  // ---- 2. Pure-Scala scrutinee — never goes through capture conversion -
+  def f2[M](xs: scala.collection.immutable.Seq[Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    xs.collect {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
+
+  // 2b. Variant: `.map` instead of `.collect`.
+  def f2b[M](xs: scala.collection.immutable.Seq[Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    xs.map {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
+
+  // 2c. Variant: List instead of Seq.
+  def f2c[M](xs: List[Container[Any, ? <: M]])
+      : List[Container[Any, M]] =
+    xs.collect {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
+
+  // ---- 3. Generic helper that stays pure-Scala does not skolemize ----
+  def idSeq[T](it: Iterable[T]): scala.collection.immutable.Seq[T] = ???
+
+  def f3[M](xs: List[Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    idSeq(xs).collect {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
diff --git a/tests/pos/i16018b.scala b/tests/pos/i16018b.scala
@@ -0,0 +1,47 @@
+// Regression test for #16018: the akka-derived shapes.
+//
+// These exercise the path that motivated the original ticket: a value of
+// type `J[? <: G[? <: M]]` (J a Java generic) flows through a method that
+// re-projects it as a Scala collection, and is then `collect`/`map`-ed with
+// a subtype pattern + a fallback `case other`. The combination triggers
+//   (i)  `Inferencing.captureWildcards` — the Java-generic argument becomes
+//        a TypeBox.CAP skolem `?N.CAP`, and
+//   (ii) `Match` typing — pattern narrowing turns the first case body into
+//        `pat ∩ ?N.CAP`, while `case other` keeps the raw `?N.CAP`.
+//
+// Until the fix in `TypeComparer.compareCaptured` for wildcard-bound
+// existential widening, the inferred result type could not escape the
+// skolem cleanly. See:
+//   - tests/pos/i16018.scala — companion pos matrix (pure-Scala)
+//   - scala/scala3 #16018 — original bug report
+
+object Test:
+
+  class Container[+S, +M]
+  class SubContainer[+S, +M] extends Container[S, M]
+
+  def seqOf[T](it: java.lang.Iterable[T]): scala.collection.immutable.Seq[T] = ???
+
+  // ---- f1: trigger via java.lang.Iterable ----------------------------
+  def f1[M](xs: java.lang.Iterable[? <: Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    seqOf(xs).collect {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
+
+  // ---- f2: same trigger via java.util.List ---------------------------
+  def f2[M](xs: java.util.List[? <: Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    seqOf(xs).collect {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
+
+  // ---- f3: same trigger via .map instead of .collect -----------------
+  def f3[M](xs: java.lang.Iterable[? <: Container[Any, ? <: M]])
+      : scala.collection.immutable.Seq[Container[Any, M]] =
+    seqOf(xs).map {
+      case g: SubContainer[Any, M] @unchecked => g
+      case other                              => other
+    }
diff --git a/tests/pos/i16018c.scala b/tests/pos/i16018c.scala
@@ -0,0 +1,86 @@
+// Directional matrix for the fix to `TypeComparer.compareCaptured`
+// (#16018 / Existential widening for wildcard arguments).
+//
+// The fix replaces `paramBounds(tparam).hi` (the *declared* parameter
+// bound) with `arg1.hi & paramBounds(tparam).hi` (the wildcard's own
+// upper bound, intersected with the declared one to preserve soundness).
+// Dually for contravariance, with `arg1.lo | paramBounds(tparam).lo`.
+//
+// Three implementations are distinguishable on this matrix:
+//
+//   OLD     : isSubType( paramBounds(tparam).hi,                 arg2 )
+//   NAIVE   : isSubType( arg1.hi,                                arg2 )
+//   THE FIX : isSubType( arg1.hi & paramBounds(tparam).hi,       arg2 )
+//
+// Sections 1–3 below distinguish OLD from THE FIX (these are the cases
+// the original ticket cares about). Sections 4–5 distinguish a hypothetical
+// NAIVE refactor (drop the intersection) from THE FIX — they pin the
+// soundness guard so that future code archaeology cannot quietly delete it.
+
+object Test:
+
+  class Co[+T]
+  class Contra[-T]
+
+  // ---- 1. Covariant + upper-bounded wildcard widens to its hi --------
+  //
+  // For `Co[+T]`, `paramBounds(tparam) = [Nothing, Any]`. OLD asked
+  // `Any <: M` (false). THE FIX asks `M & Any <: M` (true).
+
+  def co_upper_direct[M](xs: Co[? <: M]): Co[M] = xs
+  def co_upper_in_co[M](xs: Co[Co[? <: M]]): Co[Co[M]] = xs
+  def co_upper_in_seq[M](xs: scala.collection.immutable.Seq[Co[? <: M]])
+      : scala.collection.immutable.Seq[Co[M]] = xs
+  def co_upper_via_tparam[A, M <: A](xs: Co[? <: M]): Co[M] = xs
+
+  // ---- 2. Contravariant + lower-bounded wildcard widens to its lo ----
+  //
+  // Dual to (1). For `Contra[-T]`, `paramBounds(tparam) = [Nothing, Any]`,
+  // so `paramBounds(tparam).lo = Nothing`. OLD asked `M <: Nothing`
+  // (false). THE FIX asks `M <: Nothing | M = M` (true).
+
+  def contra_lower_direct[M](xs: Contra[? >: M]): Contra[M] = xs
+  def contra_lower_in_co[M](xs: Co[Contra[? >: M]]): Co[Contra[M]] = xs
+
+  // ---- 3. The headline regression --------------------------------------
+  //
+  // The wildcard hi is reached through a covariant container; this is
+  // the shape that drove the akka report and the pure-Scala minimization
+  // in tests/pos/i16018.scala.
+
+  trait Holder
+  def headline(xs: Co[? <: Holder]): Co[Holder] = xs
+
+  // ---- 4. Soundness guard, covariant (intersection in action) ----------
+  //
+  // For `CoBounded[+T <: Holder]`, `paramBounds(tparam).hi = Holder`.
+  // We feed a wildcard `? <: Any` whose hi (`Any`) is *wider* than the
+  // declared bound. The three candidates diverge:
+  //
+  //   OLD     : isSubType( Holder,        Holder )  =  true   (would pass)
+  //   NAIVE   : isSubType( Any,           Holder )  =  false  (would fail)
+  //   THE FIX : isSubType( Any & Holder = Holder,    Holder ) = true
+  //
+  // The case must pass: a value of `CoBounded[? <: Any]` is, by the
+  // parameter's own constraint, a value of `CoBounded[X]` for some
+  // `X <: Holder`, hence (by covariance) of `CoBounded[Holder]`.
+  // This is what guards `& paramBounds(tparam).hi` from being deleted.
+
+  class CoBounded[+T <: Holder]
+  def soundness_guard_co(xs: CoBounded[? <: Any]): CoBounded[Holder] = xs
+
+  // ---- 5. Soundness guard, contravariant (union in action) -------------
+  //
+  // Dual to (4). For `ContraBounded[-T >: Holder]`,
+  // `paramBounds(tparam).lo = Holder`. We feed a wildcard `? >: Nothing`
+  // whose lo (`Nothing`) is *lower* than the declared bound. The three
+  // candidates diverge:
+  //
+  //   OLD     : isSubType( Holder, Holder            ) = true   (would pass)
+  //   NAIVE   : isSubType( Holder, Nothing           ) = false  (would fail)
+  //   THE FIX : isSubType( Holder, Nothing | Holder = Holder ) = true
+  //
+  // This is what guards `| paramBounds(tparam).lo` from being deleted.
+
+  class ContraBounded[-T >: Holder]
+  def soundness_guard_contra(xs: ContraBounded[? >: Nothing]): ContraBounded[Holder] = xs
