Merge branch 'bridge-macro' into staging/5.0

Author: j-mie6

docs/api-guide/directory.conf

@@ -2,7 +2,8 @@ laika.navigationOrder = [
     Parsley.md
     character.md
     combinator.md
-    generic.md
+    templates.md
+    macros.md
     syntax.md
     position.md
     state.md

docs/api-guide/macros.md

@@ -0,0 +1,269 @@
+{%
+laika.versioned = true
+laika.title = "`parsley.macros`"
+parsley.tabname = "Macro Bridges (parsley.macros)"
+laika.site.metadata.description = "This page describes how to use macro bridges to factor code."
+%}
+
+# Macro Bridges (`parsley.macros`, Scala 3)
+The *Parser Bridge* pattern is a technique for decoupling semantic actions from the parser itself.
+The `parsley.macros` package contains a macro that can handle generation of many kinds of bridge
+automatically to help you to get started using the technique straight away if you wish.
+
+@:callout(info)
+*The Scaladoc for this page can be found at [`parsley.macros`](@:api(parsley.macros)).*
+@:@
+
+## Motivation
+The construction of parser bridges is something that involves a certain amount of boilerplate,
+despite the benefits they provide. The [`parsley.templates`](templates.md) package demonstrates
+one way in which this can be mitigated, producing templating bridge traits which can be mixed
+into a provider of an `apply` on values in exchange for one on parsers.
+
+However, to avoid making the library too opinionated, the `templates` package only exposes
+generators for *pure* parser bridges, which are those that do not incorporate additional metadata.
+To deal with metadata, like positions, it is necessary to define your own template bridge traits.
+In addition, there is still a small amount of boilerplate to be had when using these template traits.
+
+As an example:
+
+```scala mdoc:silent
+import parsley.Parsley
+import parsley.templates.PureParserBridge2
+import parsley.position.{line, col}
+
+case class Pos(line: Int, col: Int)
+object Pos extends PureParserBridge2[Int, Int, Pos]
+
+val pos: Parsley[Pos] = Pos(line, col)
+```
+
+Here, we can see a few areas of boilerplate that are not needed:
+
+* There is a mirroring between the types of the arguments to `Pos` and the type parameters for
+  `PureParserBridge2`: this has to be kept in sync, which will be enforced by the compiler, but
+  it is annoying to change something in two places. We also have to manually ensure the bridge has
+  the correct arity, in this case `2`.
+* The strategy for the bridge, namely to use a `pure`-based semantics, is hard-coded in the
+  bridge template. If we wanted to add some metadata (more on that later), we would have to remember
+  to also adjust the `PureParserBridge2` to be something else.
+* We necessarily have to introduce the companion object, where perhaps we didn't need to before.
+  The bridge must be kept in the same file as the bridged class otherwise we will need to implement
+  the `apply` ourselves.
+
+To help mitigate this, Scala 3 `parsley` supports a macro-driven bridge generation mechanism that
+addresses the above three points, and provides other functionality on top of them.
+
+## The `bridge` Macro
+The `bridge` macro is exposed via `parsley.macros.bridge`. It takes two main forms: the first is
+`bridge[T]`, which creates a bridge for the type `T`, and `bridge[T, S >: T]`, which constructs a
+bridge for the type `T`, but upcasts the result to construct `S` instead (which is a supertype of `T`).
+To see how easy this is to use, here is the previous example again:
+
+```scala mdoc:silent
+import parsley.macros.bridge
+val Pos2 = bridge[Pos]
+
+val pos2 = Pos2(line, col)
+```
+
+Here, `Pos2` works the same way as our `Pos` bridge from before, but now all of the three painpoints
+from above have been addressed. In fact, it is now cleaner to separate the class definition and
+bridge definitions in different scopes! That is it!
+
+To be a bit more specific, `bridge` is what is called a *transparent* macro in Scala, which means
+it can return a different type depending on what the macro internally generated. In our case, the
+macro will return something derived from `bridges.ErrorBridge`, so in this case
+`Pos2: bridges.ParserBridge2[Int, Int, Pos]`. If we add or remove arguments, this will automatically
+change the type generated by the macro, so type inference will sort out the boilerplate.
+
+The `bridge` macro works on pretty much anything you can throw at it:
+
+@:todo(seems to be an mdoc bug here on the enum bridges)
+```scala
+case class A(x: Int)
+object B
+enum C {
+    case X(x: Int)
+    case Y
+}
+
+val a = bridge[A]
+val b = bridge[B.type]
+val x = bridge[C.X, C]       // the C is optional here, but I prefer to hide enum cases.
+val y = bridge[C.Y.type, C]  // the C is optional here, but I prefer to hide enum cases.
+```
+
+This also includes parameterless classes, which always produce `ParserSingletonBridge`s:
+
+```scala mdoc:silent
+class Foo()
+class Bar
+
+val foo = bridge[Foo]
+val bar = bridge[Bar]
+```
+
+Interestingly, for these, we don't treat them the same way we would `object`s, which would always
+return the same thing. Instead, we will actually create a fresh instance (with the `fresh` combinator)
+every time they are parsed:
+
+```scala mdoc
+import parsley.character.item
+import parsley.syntax.zipped.*
+(foo.from(item), foo.from(item)).zipped(_ eq _).parse("aa")
+(bar.from(item), bar.from(item)).zipped(_ eq _).parse("aa")
+```
+
+This is not something that the pure templated bridge traits can achieve.
+
+### Default Parameters
+Some classes might have default parameters that are not relevant to the parser, but that will
+be relevant for wherever that data is consumed. For example, suppose we are parsing identifiers,
+which will later be assigned a type: during parse time, there is clearly no type to be assigned,
+so it would be good if we could "forget" this for the sake of clean parsing. The `bridge` macro can
+handle this:
+
+```scala mdoc:silent
+enum Type { case NoType }
+
+case class Ident(name: String)(val ty: Type = Type.NoType)
+
+val ident: parsley.bridges.ParserBridge1[String, Ident] = bridge[Ident]
+```
+
+Here, the `ident` bridge only needs a parser for the `String`, and will place `Type.NoType` into
+the `Ident` as it constructs it. A limitation of this currently is that default parameters must not
+be in the first set of constructor parameters for the class (i.e. notice that constructor is curried).
+This can be lifted in future.
+
+### Metadata Parsing
+One area which the template traits fall short is that you need to construct new templates by hand
+when you want to do anything other than pure bridging. This is because there is a vast design space
+now possible.
+
+Let's suppose we wanted to add position information to our `Ident` class above. Do we want it
+to go as the first parameter? in the second set of parameters? At the end of the first set? So
+many choices are possible here, and so `parsley` doesn't provide any default template trait for it.
+However, the `bridge` macro does have the ability to handle this as long as the user annotates exactly
+what is metadata:
+
+```scala mdoc:nest:fail
+import parsley.macros.{bridge, isMeta}
+
+case class Ident(name: String)(@isMeta val pos: Pos, val ty: Type = Type.NoType)
+
+val ident = bridge[Ident]
+```
+
+As you can see from the above error message, when we use `@isMeta` to denote something is
+parser-produced metadata, we need to tell `parsley` how we expect it to be parsed. This is done
+via the `ParsableMetadata` typeclass. Let's create one of those:
+
+```scala mdoc:invisible
+import parsley.macros.ParsableMetadata
+
+given ParsableMetadata[Pos] with {
+    val meta = pos
+}
+```
+```scala
+import parsley.macros.ParsableMetadata
+
+given ParsableMetadata[Pos] {
+    val meta = pos
+}
+```
+
+By doing this, we provide a `meta` parser, which is then what the `bridge` macro will use to
+parse the metadata to add into the class later. By default, `parsley` provides a `ParsableMetadata[(Int, Int)]`
+instance for line and column pairs.
+
+```scala mdoc:silent
+import parsley.macros.{bridge, isMeta}
+
+case class Ident(name: String)(@isMeta val pos: Pos, val ty: Type = Type.NoType)
+
+val ident = bridge[Ident]
+```
+
+This now works. The order in which this metadata is parsed, relative to the `name` parser, will
+*always* be first. If there is more than one `@isMeta` used, they will be parsed in order of
+appearance, ahead of all other arguments. It remains future work to allow for an annotation that
+allows us to denote after which bridge parameter the metadata is parsed.
+
+### Polymorpic Bridges
+When using template bridge traits, one limitation is that if the class you want to bridge is
+polymorphic, the bridge traits can't be mixed into the companion object! Suppose you tried:
+
+```scala mdoc:fail
+class Poly[A](val x: A)
+object Poly extends parsley.templates.PureParserBridge1[A, Poly[A]]
+```
+
+Oops! Ok, so what if we just fix `A` to be a specific type?
+
+```scala mdoc:fail
+class Poly[A](val x: A)
+object Poly extends parsley.templates.PureParserBridge1[Int, Poly[Int]]
+```
+
+It wants an `apply` in the companion object that works on `Int`, but obviously the
+one it sees works generically on `A`. Basically, to use the templates here, we need to
+make it its own object, which also needs implementing `apply` by hand.
+
+Thankfully, `bridge` is undeterred:
+
+```scala mdoc:silent
+class Poly[A](val x: A)
+
+def polyA[A] = bridge[Poly[A]]
+val polyInt = bridge[Poly[Int]]
+```
+
+No problems either way. In fact, this even works with the `@isMeta` annotation:
+
+```scala mdoc:silent
+class PolyMeta[A](@isMeta val x: A)
+
+def polyMetaA[A: ParsableMetadata] = bridge[PolyMeta[A]]
+val polyMetaPos = bridge[PolyMeta[Pos]]
+```
+
+### Error Messages
+The parser bridge traits all extend an `ErrorBridge`, which allows for the integration of
+labels and reasons into bridges. The template bridge traits inherit these, so the extender of the
+trait can override `labels` and `reason` and get the right behaviours while keeping them out of the
+parser. This is nice, and the macro supports it under a slightly different mechanism.
+
+The `bridge` we've seen so far is actually not a function, but an object with an `apply[T]` and
+`apply[T, S]`: let's call this a *bridge synthesiser*. It also has two other methods: `label` and `explain`,
+which themselves return a new bridge synthesiser that can incorporate those components.
+If you use `label`, the synthesiser it returns does not itself have a `label` method, but does have
+an `explain` (and vice-versa), so you can build these up "Builder Pattern"-style. As an example:
+
+```scala mdoc:silent
+import parsley.character.digit
+case class Num(n: Int)
+
+val num = bridge.label("number").explain("numbers are made of 1 or more digits")[Num]
+```
+```scala mdoc:to-string
+num(digit.foldLeft1(0)((n, d) => n * 10 + d.asDigit)).parse("a")
+```
+
+## Limitations
+Unfortunately, the bridge macro is not perfect. There are still some areas where
+the template bridge traits manage to allow slightly more flexibility. The key is that
+the bridge macro *forces* a specific shape of bridge using all the provided arguments
+to the given types primary constructor as-is. Any modifications you might want to make to these,
+perhaps to adapt to another type, will not be picked up.
+
+For example, *disambiguation bridges* (as described in other pages) involve processing data into
+potentially different sibling types. The bridge macro can't figure this out, so that's a use-case
+that is ruled out.
+
+As another example, *validation bridges* need to inject filtering logic into the bridge, which is
+not currently supported. However, this is something that I believe can be made to work (unlike
+the disambiguation bridges above), so this may be supported by some more synthesisers at a later
+point.