SIP-67 - Improve strictEquality feature for better compatibility with existing code bases

[mberndt123]

Hi there,

I'd like to use the strictEquality feature for the improved type safety it provides, but currently find it too inconvenient to use due to an unfortunate interaction with pattern matching. This SIP is my attempt to fix that.
There have been no comments in the Pre-SIP thread for the past two weeks, and it's a very small (though impactful) change to the language, so I felt it was time to submit it.

Best regards
Matthias

[soronpo]

Thank you for the proposal. Here is the feedback from the SIP Committee:

• We would like to see a different proposal that special-cases enumerations in both pattern matching and equality operations so that enumerations are considered to have a CanEqual derivation and the compiler would pick up the default equality if no explicit derives CanEqual is given. This will take care of old libraries that do not have CanEqual derivation.
• In general, we do not think its good to have a different behavior between == and pattern match equality. Do you have such a use-case?

[mberndt123]

Hey @soronpo,

I'm sorry, I'm afraid I can't do that.

Please consider the following enum:

enum Foo:
  case Bar
  case Baz(f: Int => Int)

I'm going to contend that it shouldn't ever be allowed to compare two Foo values with == because that would require us to determine whether two functions are the same, which isn't possible in a useful way.
Scala agrees with me on this already, because this type is isomorphic to Option[Int => Int], and that type can't be compared with ==, despite the fact that a CanEqual instance for Option is available by default:

scala> import scala.language.strictEquality

scala> Option.empty[Int => Int] == Option.empty[Int => Int]
-- [E172] Type Error: ----------------------------------------------------------
1 |Option.empty[Int => Int] == Option.empty[Int => Int]
  |^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |Values of types Option[Int => Int] and Option[Int => Int] cannot be compared with == or !=.

This is not a bug, this is a feature.

Now you might say, OK, then maybe we can make the CanEqual instance available only when all the fields in the enum have a CanEqual instance? I. e. make CanEqual[Foo, Foo] available only when CanEqual[Int => Int, Int => Int] is available? We can simulate this:

scala> enum Foo:
     |   case Bar
     |   case Baz(f: Int => Int)
     | object Foo:
     |   given (using CanEqual[Int => Int, Int => Int]): CanEqual[Foo, Foo] =
     |     CanEqual.derived
     | 
// defined class Foo
// defined object Foo

Alas, while this does prevent nonsensical comparisons with ==, it also fails to achieve the whole point of this proposal, which is to allow pattern matching:

scala> def bla(f: Foo) match
     |   case Foo.Bar => 0
     |   case Foo.Baz(f) => f(0)
     | 
-- [E172] Type Error: ----------------------------------------------------------
2 |  case Foo.Bar => 0
  |       ^^^^^^^
  |Values of types Foo and Foo cannot be compared with == or !=.

(insert sad trombone sound here)

Now you might ask: How come it works just right for Option? You cannot compare two Option[Int => Int] objects with ==, but it is possible to perform pattern matching on an Option[Int => Int]:

scala> Option.empty[Int => Int] match
     |   case None => 0
     |   case Some(f) => f(0)
     | 
val res0: Int = 0

That seems like the sweet spot! The reason this works is that None is of type Option[Nothing], hence this pattern match doesn't require a CanEqual[Option[Int => Int], Option[Int => Int]] but a CanEqual[Option[Int => Int], Option[Nothing]], which is available.
However this doesn't carry over to enum Foo: it doesn't have any type parameters, hence it's not possible to differentiate the empty Bar case from the non-empty Baz one on the type level.

The conclusion from all of this is that it's impossible to make this work correctly by providing a magic CanEqual instance. enum Foo is the proof for this: you either provide the instance, which makes pattern matching work but allows for == comparisons that don't make sense, or you don't, in which case you can't do pattern matching. Whatever the solution to this problem is, this is not it.

[soronpo]

Please consider the following enum:

enum Foo:
  case Bar
  case Baz(f: Int => Int)

I would not consider this to be a valid use of enum. Have you seen this type of code in the wild?

[jducoeur]

Have you seen this type of code in the wild?

Not sure offhand, but it seems like it would be unsurprising when constructing a DSL interpreter. (Where the enum is a kind of expression, and the leaf is an instantiation of one expression type.)

The interpreter for my own QL language is still on Scala 2, but I could see myself trying to build it along those lines if it was Scala 3, so I don't think that's just a strawman.

[soronpo]

Not sure offhand, but it seems like it would be unsurprising when constructing a DSL interpreter.

Still I'm not sure enum is the correct way to do it in Scala 3. It's not meant as a replacement for all kinds of case class hierarchies.

[eugengarkusha]

I would not consider this to be a valid use of enum. Have you seen this type of code in the wild?

I have seen ADTs with functions inside
(Hope its ok to consider enum as a nicer syntax for ADT)

[KristianAN]

I would not consider this to be a valid use of enum. Have you seen this type of code in the wild?

I have seen ADTs with functions inside (Hope its ok to consider enum as a nicer syntax for ADT)

From the Scala 3 book.

Algebraic Data Types (ADTs) can be created with the enum construct, so we’ll briefly review enumerations before looking at ADTs.

Personally I have almost exclusively written Scala 3 and would use enums for this kind of ADT (with a function). If that is not the correct way to do it, then the correct way must be elusive.

[jducoeur]

Yeah, agreed. My understanding of the enum feature was that it was more or less exactly to reify the pattern that the community has settled on for ADTs over the years. (With enumerations per se being sort of the degenerate case, but far from the whole story.)

Saying that only some ADTs count, and other reasonably well-formed ones don't seems kind of un-Scala to me. A key element of Scala is that functions are values; intuitively, I would expect them to work here like any other value type.

[SystemFw]

I have to agree with @jducoeur , what's the actual argument against functions in enum?
enum already goes quite some way into trying to capture some of the more complex ADT patterns that have little to do with enums, for example allowing extends to encode GADTs, having a value in there that happens to be a function seems like a lot more "normal" in comparison.

The GADT mention is not accidental, the main use case for that is encoding commands:

enum Cmd[A] { 
  case Read() extends Cmd[String]
  case Write extends Cmd[Unit]
}

and command-like GADTs will often have functions in them to allow sequencing:

enum Cmd[A] { 
  case Read() extends Cmd[String]
  case Write extends Cmd[Unit]
  case FlatMap[O, A](fa: Cmd[O], f: O => Cmd[A]) extends Cmd[A]
}

I feel like the argument that enum isn't meant for this would be a lot more stronger if extends wasn't allowed at all

[soronpo]

Let me clarify. If you want function arguments that should not be part of the pattern match, then these should come as a second argument block of the case, IMO.

enum Cmd[A]:
  case Read() extends Cmd[String]
  case Write extends Cmd[Unit]
  case FlatMap[O, A](fa: Cmd[O])(val f: O => Cmd[A]) extends Cmd[A]

If you want them to be part of the pattern match, you put them in the first block where they have same equality treatment like all the other arguments.

[JD557]

I admit that I never played with strict equality, so this might not make much sense, but:

Now you might say, OK, then maybe we can make the CanEqual instance available only when all the fields in the enum have a CanEqual instance? I. e. make CanEqual[Foo, Foo] available only when CanEqual[Int => Int, Int => Int] is available? We can simulate this:

Do we need a CanEqual[Foo, Foo] for the pattern match? Wouldn't a CanEqual[Foo, Foo.Bar.type] suffice?

Now, from what I can tell based on some quick tests, the compiler really wants a CanEqual[Foo, Foo] for pattern matching, but maybe this could be changed?

[mberndt123]

Hi @JD557,

Do we need a CanEqual[Foo, Foo] for the pattern match? Wouldn't a CanEqual[Foo, Foo.Bar.type] suffice?

Now, from what I can tell based on some quick tests, the compiler really wants a CanEqual[Foo, Foo] for pattern matching, but maybe this could be changed?

Yes, I had this in mind and I was going to propose it – you beat me to it. It can't be done with just a magic CanEqual instance, but if we slightly tweak the behaviour pattern matching under strictEquality I think it should work.

I. e.

def bla(foo: Foo) =
  foo match
    case Foo.Bar => 0 // requires CanEqual[Foo.Bar.type, Foo]
    case Foo.Baz(f) => f(0) // uses unapply

Then all we would need is a magic CanEqual instance that spawns the required CanEqual. That seems like a workable approach.

[mberndt123]

Let me clarify. If you want function arguments that should not be part of the pattern match, then these should come as a second argument block of the case, IMO.

[…]

If you want them to be part of the pattern match, you put them in the first block where they have same equality treatment like all the other arguments.

First of all, I'd like to address the question of "in the wild" examples of functions within ADTs. This is definitely something that people do, e. g.
https://github.com/typelevel/cats/blob/1cc04eca9f2bc934c869a7c5054b15f6702866fb/free/src/main/scala/cats/free/Free.scala#L219
https://github.com/typelevel/cats-effect/blob/eb918fa59f85543278eae3506fda84ccea68ad7c/core/shared/src/main/scala/cats/effect/IO.scala#L2235

I think this is perfectly good code. It should be possible to use enum for these types, and it is today – strictEquality shouldn't break that. I also see no reason to compel people to rewrite this to a style where the function goes in the second parameter list. In fact, I think it is worse that way, because == is supposed to tell you whether two things are the same – but if you just ignore the function, you're simply going to get a broken equals method that says that two things are the same even though they aren't. This is something that we shouldn't encourage. The better option is to allow the subset of equality comparisons that people are likely to need and that we know we can perform correctly – i. e. comparisons to the singleton cases – and prevent all other equality tests at compile time.

[mberndt123]

I've pushed a new revision that is based on a minor change to enum pattern matching and a magic CanEqual instance. @soronpo please let me know what you think

[soronpo]

The whole point of -language:strictEquality is to increase type safety

I think the point is for equality. Pattern matching does not fall under this category beyond reachability concerns, IMO.

[bjornregnell]

But equals is called in pattern matching with constant patterns, so it's an equality test?

[soronpo]

But equals is called in pattern matching with constant patterns, so it's an equality test?

For the record, strict equality is applied to == and not equals. I consider pattern matching to be using equals.

[bjornregnell]

Aha, thanks @soronpo for the clarification. Yes, nobody knows what equals does as it can be overridden with whatever strange behavior...

[bjornregnell]

@mberndt123 Perhaps update the proposal to upfront clarify the scope of being == and not equals ? When re-reading the intro etc I'm not sure this is absolutely clear.

[sjrd]

Pattern matching uses ==. It doesn't use equals. Proof:
https://scastie.scala-lang.org/A0eckzOaRxaygBFoCMiyWA

locally {
  val x: Any = 5L
  println(x == 5)      // true
  println(x.equals(5)) // false
  println(x match {    // yes
    case 5 => "yes"
    case _ => "nope"
  })
}

[mberndt123]

@mberndt123 Perhaps update the proposal to upfront clarify the scope of being == and not equals ? When re-reading the intro etc I'm not sure this is absolutely clear.

Hi Björn,

What @soronpo wrote there is not a clarification, it is his opinion, and it is in fact exactly where the controversy is. Currently, pattern matching against a constant (whether it's a literal or a case object or just a val) requires a CanEqual instance. @soronpo believes that this should be dropped entirely. For reasons that I've laid out in in the SIP, I don't think that this is sufficiently safe and I would rather drop the CanEqual requirement only for cases that correspond to traditional ADT pattern matching in statically typed languages like Haskell, OCaml or Rust.

[bjornregnell]

AHA!! Thanks also to @mberndt123 and @sjrd for yet more clarifications. Maybe it's just me who didn't get the actual controversy, but perhaps the intro to the SIP can be improved by referring to == versus equals and later state what is at stake in terms of compiling or not for a busy reader...

I think I'm leaning towards the stricter end of the design space of the future language - it is after all called strict equality.

[soronpo]

Pattern matching uses ==. It doesn't use equals. Proof: https://scastie.scala-lang.org/A0eckzOaRxaygBFoCMiyWA

locally {
  val x: Any = 5L
  println(x == 5)      // true
  println(x.equals(5)) // false
  println(x match {    // yes
    case 5 => "yes"
    case _ => "nope"
  })
}

What I meant is that I think it should be equals and therefore not affected by strict equality

[sjrd]

It has had the semantics of == for as long as I've been involved in Scala (around 2.9.0). Probably as far back as the introduction of pattern matching in the first place. I don't see why that should change. If it should, it would be a separate SIP with far greater compatibility issues.

IMO we should judge the present SIP with respect to the established semantics of pattern matching.

[soronpo]

It has had the semantics of == for as long as I've been involved in Scala (around 2.9.0). Probably as far back as the introduction of pattern matching in the first place. I don't see why that should change. If it should, it would be a separate SIP with far greater compatibility issues.

IMO we should judge the present SIP with respect to the established semantics of pattern matching.

I'm ok with keeping == and just inferring CanEqual under pattern matching, as I have already mentioned.

[bjornregnell]

discussed also here: https://users.scala-lang.org/t/cannot-match-scala-3-enum-with-scala-language-strictequality/8485/5

[soronpo]

This SIP was approved to move into the implementation stage. Congrats @mberndt123 !
We think that it's important for future readers to add a couple of examples of what is and isn't affected by this SIP.
My own view is that we can eliminate the need for CanEqual inside of pattern matching, but we can do this "in baby steps", and this is a step in the right direction.
Regarding the implementation, would you like to try an implement this SIP?

[SethTisue]

We think that it's important for future readers to add a couple of examples of what is and isn't affected by this SIP.

I think this would help both the committee, and the users who want to try the changes out. Users especially have trouble reading and digesting spec-ese.

It's useful not only to have examples of what the proposal does change, but what it doesn't. The full story of the proposal is: here's what already worked even before the proposal; here's examples that the proposal does fix; here's examples that don't work and it might be surprising but we're certain the behavior is correct; here's examples that don't work and we might have hoped to fix but it's out of scope for now but maybe a subsequent SIP could address it.

I know that may sound like a lot, but I'm not imagining tons of examples, just key ones that would make the SIP clear to everyone.

[mberndt123]

Thanks @soronpo and @SethTisue.

I've added a handful of examples, I hope that clarifies things

[bracevac]

Please note that the SIP template has changed:

1. The SIP number goes into a number: NN YAML header, and titles should not be prefixed with SIP NN -
2. The file name should be prefixed with the SIP number, padded to three digits.

Thanks!

[nilskp]

New to this SIP. I always thought Scala would implement strict equality akin to Javascript, with a triple equals ===. I'm sure this has been discussed, but perhaps someone can point me to where?

[soronpo]

New to this SIP. I always thought Scala would implement strict equality akin to Javascript, with a triple equals ===. I'm sure this has been discussed, but perhaps someone can point me to where?

Such discussions would usually occur on https://contributors.scala-lang.org/ but I don't remember such discussion.

[jducoeur]

New to this SIP. I always thought Scala would implement strict equality akin to Javascript, with a triple equals ===. I'm sure this has been discussed, but perhaps someone can point me to where?

Note that === is implemented in some libraries (notably Scalactic) -- for those who want to opt into that, it's long been available.

Strict equality is much further-reaching than that, though, intentionally -- it's not just opt-in on an ad hoc call-site basis, it prevents you from handling equality in an ad-hoc way, and enforces some discipline instead. (Still deciding how much I like it in practice, but I appreciate the principle.)