From fc023e9ebfb925cc517cc1d043f5055001a68979 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:02:44 +0200 Subject: [PATCH 01/11] Add docs on debugging --- docs/docs/contributing/debugging.md | 332 ++++++++++++++++++++++++++++ 1 file changed, 332 insertions(+) create mode 100644 docs/docs/contributing/debugging.md diff --git a/docs/docs/contributing/debugging.md b/docs/docs/contributing/debugging.md new file mode 100644 index 000000000000..9fc5d8132a64 --- /dev/null +++ b/docs/docs/contributing/debugging.md @@ -0,0 +1,332 @@ +# Debugging Techniques +- [Setting up the playground](#setting-up-the-playground) +- [Show for human readable output](#show-for-human-readable-output) +- [How to disable color](#how-to-disable-color) +- [Reporting as a non-intrusive println](#reporting-as-a-non-intrusive-println) +- [Printing out trees after phases](#printing-out-trees-after-phases) +- [Configuring the printer output](#configuring-the-printer-output) +- [Figuring out an object creation site](#figuring-out-an-object-creation-site) + * [Via ID](#via-id) + * [Via tracer](#via-tracer) +- [Built-in Logging Architecture](#built-in-logging-architecture) + * [Printers](#printers) + * [Tracing](#tracing) + * [Reporter](#reporter) + +Table of contents generated with markdown-toc + +## Setting up the playground +Consider the `../issues/Playground.scala` (relative to the Dotty directory) file is: + +```scala +object Playground { + def main(args: Array[String]) = { + println("Hello World") + } +} +``` + +Then, you can debug Dotty by compiling this file via `dotc ../issues/Playground.scala` (from the SBT console) and collecting various debug output in process. This section documents techniques you can use to collect the debug info. + +[This](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/typer/Typer.scala#L2231) is the entry point to the Typer. The job of the Typer is to take an untyped tree, compute its type and turn it into a typed tree by attaching the type information to that tree (in an immutable way of course). We will use this entry point to practice debugging techniques. E.g.: + +```scala + def typed(tree: untpd.Tree, pt: Type, locked: TypeVars)(implicit ctx: Context): Tree = + trace(i"typing $tree", typr, show = true) { + println("Hello Debug!") + /*...*/ +``` + +Then: + +``` +dotc ../issues/Playground.scala +``` + +The techniques discussed below can be tried out in place of `println("Hello Debug")` in that location. They are of course applicable thoughout the codebase. + +## Show for human readable output +Many objects in the compiler have a `show` method available on them via implicit rich wrapper: + +```scala +println(tree.show) +``` + +This will output every single tree passing through the typer (or wherever else you inject it) in a human readable form. Try calling `show` on anything you want to be human-readable, and chances are it will be possible to do so. + +## How to disable color +Note that the `show` command above outputs the code in color. This is achieved by injecting special characters into the strings which terminals interpret as commands to change color of the output. This however may not be what you want, e.g. if you want to zero-in on a particular tree: + +```scala +if (tree.show == """println("Hello World")""") + println(s"${tree.show}\n${pt.show}\n${tree.uniqueId}\n===\n") +``` + +The intention above is to output an extended debug info on a tree that matches a particular human-readable representation. However, because of the color characters, the comparison will fail. + +To disable color output from `show`, run `dotc` as follows: + +`dotc -color:never ../issues/Playground.scala` + +## Reporting as a non-intrusive println +Consider you want to debug the `tree` that goes into `assertPositioned(tree)` in the `typed` method. You can do: + +```scala +println(tree.show) +assertPositioned(tree) +``` + +But you can also do: + +```scala +assertPositioned(tree.reporting(t => t.show)) +``` + +`(a: A).reporting(f: A => String): A` is defined on all types. Its job is to print a `String` produced by the argument function and return the value it is called on unchanged. + +## Printing out trees after phases +To print out the trees you are compiling after Frontend (scanner, parser, namer, typer) phase: + +``` +dotc -Xprint:frontend ../issues/Playground.scala +``` + +To print out the trees after Frontend and CollectSuperCalls phases: + +``` +dotc -Xprint:frontend,collectSuperCalls ../issues/Playground.scal +``` + +To print out the trees after all phases: + +``` +dotc -Xprint:all ../issues/Playground.scala +``` + +To find out the list of all the phases and their names, check out [this](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/Compiler.scala#L34) line in `Compiler.scala`. Each `Phase` object has `phaseName` defined on it, this is the phase name. + +## Configuring the printer output +Printing from the `show` and `-Xprint` is done from the Printers framework (discussed in more details below). The following settings influence the output of the printers: + +```scala +val printLines = BooleanSetting("-print-lines" , "Show source code line numbers.") withAbbreviation "--print-lines" +val uniqid = BooleanSetting("-uniqid" , "Uniquely tag all identifiers in debugging output.") withAbbreviation "--unique-id" +val XprintInline = BooleanSetting("-Xprint-inline" , "Show where inlined code comes from") +val XprintTypes = BooleanSetting("-Xprint-types" , "Print tree types (debugging option).") +val Ydebug = BooleanSetting("-Ydebug" , "Increase the quantity of debugging output.") +val YdebugFlags = BooleanSetting("-Ydebug-flags" , "Print all flags of definitions") +val YdebugMissingRefs = BooleanSetting("-Ydebug-missing-refs", "Print a stacktrace when a required symbol is missing") +val YdebugNames = BooleanSetting("-Ydebug-names" , "Show internal representation of names") +val YdebugPos = BooleanSetting("-Ydebug-pos" , "Show full source positions including spans") +val YdebugTrace = BooleanSetting("-Ydebug-trace" , "Trace core operations") +val YdebugTreeWithId = IntSetting ("-Ydebug-tree-with-id", "Print the stack trace when the tree with the given id is created", Int.MinValue) +val YprintDebug = BooleanSetting("-Yprint-debug" , "when printing trees, print some extra information useful for debugging.") +val YprintDebugOwners = BooleanSetting("-Yprint-debug-owners", "when printing trees, print owners of definitions.") +val YprintPos = BooleanSetting("-Yprint-pos" , "show tree positions.") +val YprintPosSyms = BooleanSetting("-Yprint-pos-syms" , "show symbol definitions positions.") +val YprintSyms = BooleanSetting("-Yprint-syms" , "when printing trees print info in symbols instead of corresponding info in trees.") +val YshowTreeIds = BooleanSetting("-Yshow-tree-ids" , "Uniquely tag all tree nodes in debugging output.") +val YshowVarBounds = BooleanSetting("-Yshow-var-bounds" , "Print type variables with their bounds") +val YtestPickler = BooleanSetting("-Ytest-pickler" , "self-test for pickling functionality; should be used with -Ystop-after:pickler") +``` + +They are defined in [ScalaSettings.scala](https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala). E.g. `YprintPos` is defined as: + +```scala +val YprintPos: Setting[Boolean] = BooleanSetting("-Yprint-pos", "show tree positions.") +``` + +And is to be used as: + +```scala +dotc -Yprint-pos ../issues/Playground.scala +``` + +If used, all the trees output with `show` or via `-Xprint:frontend` will also have positions attached to them, e.g.: + +```scala +package @ { + module object Playground { + def main( + args: + Array@[String@]@< + Playground.scala:2 + > + @ + ) = + { + println@("Hello World"@)@< + Playground.scala:3 + > + }@ + @ + }@ +}@ +@ +``` + +## Figuring out an object creation site +### Via ID +Every [Positioned](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/ast/Positioned.scala) (a trait of a `Tree`) object has `uniqueId` field. It is an integer that is unique for that tree and doesn't change from compile run to compile run. You can output these IDs from the printers (= from `show` and `-Xprint`) framework via `-Yshow-tree-ids` flag, e.g.: + +``` +dotc -Xprint:frontend -Yshow-tree-ids ../issues/Playground.scala +``` + +Gives: + +```scala +package #1047 { + final lazy module val Playground: Playground$#1049 = + new Playground$#1049#1050#1051()#1052 + #1053 + final module class Playground$() extends Object#1090#1091#1092()#1093, _root_# + 1061 + .scala#1062.Serializable#1063 { this: Playground#1054.type#1055 => + def main(args: Array#1028[String#1033]#1034#1038): Unit#1039 = + { + println#1094("Hello World"#1041)#1095 + }#1096 + #1097 + }#1099 +}#1100 +``` + +You can then use these IDs to locate the creation site of a given tree using that ID via `-Ydebug-tree-with-id`, e.g.: + +``` +dotc -Ydebug-tree-with-id 1049 ../issues/Playground.scala +``` + +When requested what you want to do: + +``` +Debug tree (id=1049) creation +Ident(Playground$) + + +a)bort, s)tack, r)esume +``` + +Input `s` for stack trace. You will get: + +``` +java.lang.Throwable + at dotty.tools.dotc.reporting.Reporter$.loop$1(Reporter.scala:55) + at dotty.tools.dotc.reporting.Reporter$.displayPrompt(Reporter.scala:63) + at dotty.tools.dotc.ast.Positioned.printTrace$1(Positioned.scala:32) + at dotty.tools.dotc.ast.Positioned.uniqueId_$eq(Positioned.scala:34) + at dotty.tools.dotc.ast.Positioned.(Positioned.scala:45) + at dotty.tools.dotc.ast.Trees$Tree.(Trees.scala:53) + at dotty.tools.dotc.ast.Trees$DenotingTree.(Trees.scala:266) + at dotty.tools.dotc.ast.Trees$NameTree.(Trees.scala:292) + at dotty.tools.dotc.ast.Trees$RefTree.(Trees.scala:298) + at dotty.tools.dotc.ast.Trees$Ident.(Trees.scala:375) + at dotty.tools.dotc.ast.untpd$.Ident(untpd.scala:301) + at dotty.tools.dotc.ast.desugar$.moduleDef(Desugar.scala:804) + at dotty.tools.dotc.ast.desugar$.defTree(Desugar.scala:1038) + at dotty.tools.dotc.typer.Namer.expand(Namer.scala:441) + at dotty.tools.dotc.typer.Namer.index$$anonfun$1(Namer.scala:722) + at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15) + at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10) + at scala.collection.immutable.List.foreach(List.scala:392) + at dotty.tools.dotc.typer.Namer.index(Namer.scala:722) + at dotty.tools.dotc.typer.Namer.recur$1(Namer.scala:484) + at dotty.tools.dotc.typer.Namer.indexExpanded(Namer.scala:501) + at dotty.tools.dotc.typer.Namer.index(Namer.scala:474) + at dotty.tools.dotc.typer.FrontEnd.enterSyms$$anonfun$1(FrontEnd.scala:69) + at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) + at dotty.tools.dotc.typer.FrontEnd.monitor(FrontEnd.scala:41) + at dotty.tools.dotc.typer.FrontEnd.enterSyms(FrontEnd.scala:71) + at dotty.tools.dotc.typer.FrontEnd.runOn(FrontEnd.scala:100) + at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:158) + at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15) + at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10) + at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) + at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) + at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) + at dotty.tools.dotc.Run.runPhases$5(Run.scala:170) + at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:178) + at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) + at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:102) + at dotty.tools.dotc.Run.compileUnits(Run.scala:185) + at dotty.tools.dotc.Run.compileSources(Run.scala:120) + at dotty.tools.dotc.Run.compile(Run.scala:104) + at dotty.tools.dotc.Driver.doCompile(Driver.scala:34) + at dotty.tools.dotc.Driver.process(Driver.scala:172) + at dotty.tools.dotc.Driver.process(Driver.scala:141) + at dotty.tools.dotc.Driver.process(Driver.scala:153) + at dotty.tools.dotc.Driver.main(Driver.scala:180) + at dotty.tools.dotc.Main.main(Main.scala) +``` + +So that tree was created at: + +``` + at dotty.tools.dotc.ast.desugar$.moduleDef(Desugar.scala:804) +``` + +Since all the stack frames above it are technical frames executing the tree creation command, and the frame in question is the location where the intent of the tree creation was expressed. + +### Via tracer +Some objects may not be `Positioned` and hence their creation site is not debuggable via the technique in the section above. Say you target a tree at `Typer`'s `typed` method as follows: + +```scala +if (tree.show == """println("Hello World")""") { + val creationSite = "" + println(creationSite) +} +``` + +In other words, you have a reference to the object and want to know were it was created. To do so, go to the class definition of that object. In our case, `tree` is a [`Tree`](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/ast/Trees.scala#L52). Now, create a new `val` member of that type: + +```scala +val tracer = Thread.currentThread.getStackTrace.mkString("\n") +``` + +Then, from the `typed` method (or wherever you want to access the trace): + +```scala +if (tree.show == """println("Hello World")""") { + val creationSite = tree.tracer + println(creationSite) +} +``` + +## Built-in Logging Architecture +Dotty has a lot of debug calls scattered throughout the code, vast majority of them disabled. At least three (possibly intertwined) architectures for logging are used for that: + +- Printer +- Tracing +- Reporter + +These do not follow any particular system and so probably it will be easier to go with `println` most of the times instead. + +### Printers +Defined in [Printers.scala](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/config/Printers.scala) as a set of variables, each responsible for its own domain. To enable them, replace `noPrinter` with `default`. [Example](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/typer/Typer.scala#L2226) from the code: + +```scala +typr.println(i"make contextual function $tree / $pt ---> $ifun") +``` + +`typr` is a printer. + +### Tracing +Defined in [trace.scala](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/reporting/trace.scala). [Example](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/typer/Typer.scala#L2232) from the code: + +```scala +trace(i"typing $tree", typr, show = true) { // ... +``` + +To enable globally, change [tracingEnabled](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/config/Config.scala#L164) to `true` (will recompile a lot of code). + +You also need to set the printer referenced in the call (in the example, `typr`) to `default` as explained in the section on printers. + +To enable for a single trace, do the following: + +```scala +trace.force(i"typing $tree", typr, show = true) { // ... +``` + +### Reporter +Defined in [Reporter.scala](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/reporting/Reporter.scala). Enables calls such as `ctx.log`, `ctx.error` etc. To enable, run dotc with `-Ylog:frontend` option. From 937ba7df3281fc37f6b4a164cb11b82451531312 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:05:01 +0200 Subject: [PATCH 02/11] Add vulpix modification docs --- .../contributing/{ => codebase}/debugging.md | 0 docs/docs/contributing/codebase/vulpix.md | 30 +++++++++++++++++++ 2 files changed, 30 insertions(+) rename docs/docs/contributing/{ => codebase}/debugging.md (100%) create mode 100644 docs/docs/contributing/codebase/vulpix.md diff --git a/docs/docs/contributing/debugging.md b/docs/docs/contributing/codebase/debugging.md similarity index 100% rename from docs/docs/contributing/debugging.md rename to docs/docs/contributing/codebase/debugging.md diff --git a/docs/docs/contributing/codebase/vulpix.md b/docs/docs/contributing/codebase/vulpix.md new file mode 100644 index 000000000000..f1cdf70e5897 --- /dev/null +++ b/docs/docs/contributing/codebase/vulpix.md @@ -0,0 +1,30 @@ +# Test Vulpix Framework +If you are modifying the Vulpix framework and need a playground with dummy tests to try out your modifications, do the following. + +Create the directory structure for the playground: + +```bash +mkdir -pv tests/playground/run tests/playground/neg +echo "stuff" > tests/playground/neg/Sample.scala +echo 'object Test { def main(args: Array[String]): Unit = {println("Hi")} }' > tests/playground/run/Sample.scala +``` + +In `CompilationTests.scala`: + +```scala + @Test def exampleNeg: Unit = { + implicit val testGroup: TestGroup = TestGroup("exampleNeg") + compileFilesInDir("tests/playground/neg", defaultOptions).checkExpectedErrors() + } + + @Test def exampleRun: Unit = { + implicit val testGroup: TestGroup = TestGroup("exampleRun") + compileFilesInDir("tests/playground/run", defaultOptions).checkRuns() + } +``` + +SBT: + +```scala +testOnly dotty.tools.dotc.CompilationTests -- *example* +``` From 5dcbd7b6c861e86625451d36bb72e973a1edae2e Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:06:29 +0200 Subject: [PATCH 03/11] Create a dedicated section for tool apps --- docs/docs/contributing/{ => tools}/eclipse.md | 0 docs/docs/contributing/{ => tools}/intellij-idea.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename docs/docs/contributing/{ => tools}/eclipse.md (100%) rename docs/docs/contributing/{ => tools}/intellij-idea.md (100%) diff --git a/docs/docs/contributing/eclipse.md b/docs/docs/contributing/tools/eclipse.md similarity index 100% rename from docs/docs/contributing/eclipse.md rename to docs/docs/contributing/tools/eclipse.md diff --git a/docs/docs/contributing/intellij-idea.md b/docs/docs/contributing/tools/intellij-idea.md similarity index 100% rename from docs/docs/contributing/intellij-idea.md rename to docs/docs/contributing/tools/intellij-idea.md From 2c5a3989fd25c3b28b69fbdbc2b8523ed2577fc7 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:14:55 +0200 Subject: [PATCH 04/11] Mill, SBT and Scalafix docs added --- docs/docs/contributing/tools/mill.md | 16 ++++++++++++++++ docs/docs/contributing/tools/sbt.md | 11 +++++++++++ docs/docs/contributing/tools/scalafix.md | 15 +++++++++++++++ 3 files changed, 42 insertions(+) create mode 100644 docs/docs/contributing/tools/mill.md create mode 100644 docs/docs/contributing/tools/sbt.md create mode 100644 docs/docs/contributing/tools/scalafix.md diff --git a/docs/docs/contributing/tools/mill.md b/docs/docs/contributing/tools/mill.md new file mode 100644 index 000000000000..a3143860be42 --- /dev/null +++ b/docs/docs/contributing/tools/mill.md @@ -0,0 +1,16 @@ +Here's an example of how to test a project that uses mill: + +```bash +mill utest.jvm[2.12.8].test +``` + +- `utest.jvm` - the name of the compiled module (obtain from `build.sc`) +- `2.12.8` – Scala cross-compile version +- `test` – task to run on the module specified with the specified Scala version + +To get mill of the most recent version, first, find it in https://github.com/lihaoyi/mill/releases (e.g. `0.4.2-1-020e28`). Copy the download link and substitute it in the following command instead of `https://github.com/lihaoyi/mill/releases/download/0.4.1/0.4.1`: + +``` +# From http://www.lihaoyi.com/mill/ +sudo sh -c '(echo "#!/usr/bin/env sh" && curl -L https://github.com/lihaoyi/mill/releases/download/0.4.1/0.4.1) > /usr/local/bin/mill && chmod +x /usr/local/bin/mill' +``` diff --git a/docs/docs/contributing/tools/sbt.md b/docs/docs/contributing/tools/sbt.md new file mode 100644 index 000000000000..6069ccf652ad --- /dev/null +++ b/docs/docs/contributing/tools/sbt.md @@ -0,0 +1,11 @@ +# SBT Commands Cheat Sheet +The basics of working with Dotty codebase are documented [here](http://dotty.epfl.ch/docs/contributing/getting-started.html) and [here](http://dotty.epfl.ch/docs/contributing/workflow.html). Below is a cheat sheet of some frequently used commands (to be used from SBT console – `sbt`). + + +| Command | Description | +|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| +| `dotc ../issues/Playground.scala` | Compile the given file – path relative to the Dotty directory. Output the compiled class files to the Dotty directory itself. | +| `dotr Playground` | Run the compiled class `Playground`. Dotty directory is on classpath by default. | +| `repl` | Start REPL | +| `testOnly dotty.tools.dotc.CompilationTests -- *pos` | Run test (method) `pos` from `CompilationTests` suite. | +| `testCompilation sample` | In all test suites, run test files containing the word `sample` in their title. | diff --git a/docs/docs/contributing/tools/scalafix.md b/docs/docs/contributing/tools/scalafix.md new file mode 100644 index 000000000000..c8d6a6f9dad3 --- /dev/null +++ b/docs/docs/contributing/tools/scalafix.md @@ -0,0 +1,15 @@ +# Working with Scalafix + +First, create a new rule as follows (command from https://scalacenter.github.io/scalafix/docs/developers/setup.html): + +```bash +sbt new scalacenter/scalafix.g8 --repo="Repository Name" +``` + +To run the rule against some codebase: + +```bash +scalafix -r file:scalafix/rules/src/main/scala/fix/YourRule.scala your/code/base/ +``` + +Where `YourRule.scala` is the rule you developed and `your/code/base` is the code base you are running the rule against. From f5f3938674f39545dd3de782679a3752e7cc93f1 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:16:15 +0200 Subject: [PATCH 05/11] Moved vulpix doc to procedures section --- docs/docs/contributing/{ => procedures}/release.md | 0 docs/docs/contributing/{codebase => procedures}/vulpix.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename docs/docs/contributing/{ => procedures}/release.md (100%) rename docs/docs/contributing/{codebase => procedures}/vulpix.md (100%) diff --git a/docs/docs/contributing/release.md b/docs/docs/contributing/procedures/release.md similarity index 100% rename from docs/docs/contributing/release.md rename to docs/docs/contributing/procedures/release.md diff --git a/docs/docs/contributing/codebase/vulpix.md b/docs/docs/contributing/procedures/vulpix.md similarity index 100% rename from docs/docs/contributing/codebase/vulpix.md rename to docs/docs/contributing/procedures/vulpix.md From 2c9f440dd45475cd075709650992861375ee5eeb Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:21:06 +0200 Subject: [PATCH 06/11] Scala 2 vs Scala 3 doc added --- docs/docs/contributing/scala2-vs-scala3.md | 38 ++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 docs/docs/contributing/scala2-vs-scala3.md diff --git a/docs/docs/contributing/scala2-vs-scala3.md b/docs/docs/contributing/scala2-vs-scala3.md new file mode 100644 index 000000000000..c1b56b1c93ab --- /dev/null +++ b/docs/docs/contributing/scala2-vs-scala3.md @@ -0,0 +1,38 @@ +# Divergences between Scala 2 and Dotty +The following issues encountered when compiling Scala 2 code as-is under Dotty: + +## Scalafix candidates +- If a method is defined `toSet()`, it cannot be called `toSet`. +- “result type of implicit definition needs to be given explicitly” +- There are no `'Symbol`s in Scala 3, you must construct symbols via `new Symbol("foo")` instead of old `'foo` + +## Trivial +- Scala 2.13 libraries cannot be used from Dotty, because the type signatures have Scala version `5.3` but `5.0` is expected. +- To use Scala 2.12 dependencies from SBT with Dotty, explicitly suffix their names with `_2.12`. +- Feature warnings about implicits `scala.language.implicitConversions` are output by default, unlike in Scala 2. This creates noise. Unclear how to turn off. + +Implicit conversions must be applied explicitly: + +```scala + implicit def IterablePath[T](s: Iterable[T])(implicit conv: T => RelPath): RelPath = { + s.foldLeft(rel){_ / conv(_)} + } +``` + +Stronger compile time guarantees on variance. Scala 2 does not assert variance on default parameters to parameters of the function value type. E.g. in geny: + +```Scala +# Dotty +def count(f: A => Boolean = (a: A) => true): Int = +| ^^^^^^^^^^^^^^ +|covariant type A occurs in contravariant position in type => A => Boolean of method count$default$1 +``` + +Fix: +```Scala +# Dotty +def count[B >: A](f: B => Boolean = (_: B) => true): Int = +``` + +## Tricky +- Scala 3 macros are completely different from Scala 2 ones, requires a migration strategy of its own From b3aa867b4da804e84b5420af947580d864f502f8 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:27:55 +0200 Subject: [PATCH 07/11] Instructions on contributing knowledge added --- README.md | 8 ++++++++ docs/docs/contributing/contribute-knowledge.md | 7 +++++++ 2 files changed, 15 insertions(+) create mode 100644 docs/docs/contributing/contribute-knowledge.md diff --git a/README.md b/README.md index 530ceb937c89..a12314687269 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,14 @@ How to Contribute * [Awesome Error Messages](http://scala-lang.org/blog/2016/10/14/dotty-errors.html) * [Issues](https://github.com/lampepfl/dotty/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) +## Contribute Internals-related Knowledge +If you know anything useful at all about Dotty, feel free to log this knowledge: + +- [📜Log the Knowledge](https://github.com/lampepfl/dotty-knowledge/issues/new) +- [🎓More about Logging the Knowledge](https://github.com/lampepfl/dotty-knowledge/blob/master/README.md) + +In short, no need to make it pretty, particularly human-readable or give it a particular structure. Just dump the knowledge you have and we'll take it from there. + License ======= Dotty is licensed under the [3-Clause BSD License](https://github.com/lampepfl/dotty/blob/master/LICENSE.md) diff --git a/docs/docs/contributing/contribute-knowledge.md b/docs/docs/contributing/contribute-knowledge.md new file mode 100644 index 000000000000..ef4452eafbb2 --- /dev/null +++ b/docs/docs/contributing/contribute-knowledge.md @@ -0,0 +1,7 @@ +# Contribute Internals-related Knowledge +If you know anything useful at all about Dotty, feel free to log this knowledge: + +- [📜Log the Knowledge](https://github.com/lampepfl/dotty-knowledge/issues/new) +- [🎓More about Logging the Knowledge](https://github.com/lampepfl/dotty-knowledge/blob/master/README.md) + +In short, no need to make it pretty, particularly human-readable or give it a particular structure. Just dump the knowledge you have and we'll take it from there. \ No newline at end of file From 8721fc01b898ca5561b6f8ebc70b826a81f0c385 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 13:34:58 +0200 Subject: [PATCH 08/11] Sidebar updated --- .../contributing/{codebase => }/debugging.md | 0 docs/sidebar.yml | 28 +++++++++++++++---- 2 files changed, 22 insertions(+), 6 deletions(-) rename docs/docs/contributing/{codebase => }/debugging.md (100%) diff --git a/docs/docs/contributing/codebase/debugging.md b/docs/docs/contributing/debugging.md similarity index 100% rename from docs/docs/contributing/codebase/debugging.md rename to docs/docs/contributing/debugging.md diff --git a/docs/sidebar.yml b/docs/sidebar.yml index 93f5f1d997bd..c78c57766068 100644 --- a/docs/sidebar.yml +++ b/docs/sidebar.yml @@ -163,18 +163,34 @@ sidebar: url: docs/reference/dropped-features/nonlocal-returns.html - title: Contributing subsection: + - title: Contribute Knowledge + url: docs/contributing/contribute-knowledge.html - title: Getting Started url: docs/contributing/getting-started.html - title: Workflow url: docs/contributing/workflow.html - - title: Eclipse - url: docs/contributing/eclipse.html - - title: IntelliJ IDEA - url: docs/contributing/intellij-idea.html - title: Testing url: docs/contributing/testing.html - - title: Release Model - url: docs/contributing/release.html + - title: Debugging + url: docs/contributing/debugging.html + - title: IDEs and Tools + subsection: + - title: Eclipse + url: docs/contributing/tools/eclipse.html + - title: IntelliJ IDEA + url: docs/contributing/tools/intellij-idea.html + - title: Mill + url: docs/contributing/tools/mill.html + - title: Sbt + url: docs/contributing/tools/sbt.html + - title: Scalafix + url: docs/contributing/tools/scalafix.html + - title: Procedures + subsection: + - title: Release Model + url: docs/contributing/procedures/release.html + - title: Modifying the Test Framework + url: docs/contributing/procedures/vulpix.html - title: Internals subsection: - title: Backend From 4297802c5c94f90ebdce94e79f3f756ce47e0fed Mon Sep 17 00:00:00 2001 From: Anatolii Date: Tue, 2 Jul 2019 14:08:33 +0200 Subject: [PATCH 09/11] Add log knowledge badge --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a12314687269..b6ab042db963 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ Dotty ===== [![Build Status](http://dotty-ci.epfl.ch/api/badges/lampepfl/dotty/status.svg)](http://dotty-ci.epfl.ch/lampepfl/dotty) [![Join the chat at https://gitter.im/lampepfl/dotty](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/lampepfl/dotty?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +[![Log Knowledge](https://img.shields.io/badge/log-knowledge-blueviolet.svg)](https://github.com/lampepfl/dotty-knowledge/issues/new/choose) * [Homepage](http://dotty.epfl.ch) * [Documentation](http://dotty.epfl.ch/docs) @@ -25,7 +26,7 @@ How to Contribute ## Contribute Internals-related Knowledge If you know anything useful at all about Dotty, feel free to log this knowledge: -- [📜Log the Knowledge](https://github.com/lampepfl/dotty-knowledge/issues/new) +- [📜Log the Knowledge](https://img.shields.io/badge/log-knowledge-blueviolet.svg)](https://github.com/lampepfl/dotty-knowledge/issues/new/choose) - [🎓More about Logging the Knowledge](https://github.com/lampepfl/dotty-knowledge/blob/master/README.md) In short, no need to make it pretty, particularly human-readable or give it a particular structure. Just dump the knowledge you have and we'll take it from there. From 035cc6e0cb462891f7be36757235eb2c0d14e309 Mon Sep 17 00:00:00 2001 From: Anatolii Kmetiuk Date: Thu, 4 Jul 2019 16:33:56 +0200 Subject: [PATCH 10/11] Minor fixes of the documentation Co-Authored-By: Guillaume Martres --- docs/docs/contributing/debugging.md | 10 +++++----- docs/docs/contributing/scala2-vs-scala3.md | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/docs/contributing/debugging.md b/docs/docs/contributing/debugging.md index 9fc5d8132a64..5bc2a7676139 100644 --- a/docs/docs/contributing/debugging.md +++ b/docs/docs/contributing/debugging.md @@ -28,7 +28,7 @@ object Playground { Then, you can debug Dotty by compiling this file via `dotc ../issues/Playground.scala` (from the SBT console) and collecting various debug output in process. This section documents techniques you can use to collect the debug info. -[This](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/typer/Typer.scala#L2231) is the entry point to the Typer. The job of the Typer is to take an untyped tree, compute its type and turn it into a typed tree by attaching the type information to that tree (in an immutable way of course). We will use this entry point to practice debugging techniques. E.g.: +[This](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/typer/Typer.scala#L2231) is the entry point to the Typer. The job of the Typer is to take an untyped tree, compute its type and turn it into a typed tree by attaching the type information to that tree. We will use this entry point to practice debugging techniques. E.g.: ```scala def typed(tree: untpd.Tree, pt: Type, locked: TypeVars)(implicit ctx: Context): Tree = @@ -167,7 +167,7 @@ package @ { ## Figuring out an object creation site ### Via ID -Every [Positioned](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/ast/Positioned.scala) (a trait of a `Tree`) object has `uniqueId` field. It is an integer that is unique for that tree and doesn't change from compile run to compile run. You can output these IDs from the printers (= from `show` and `-Xprint`) framework via `-Yshow-tree-ids` flag, e.g.: +Every [Positioned](https://github.com/lampepfl/dotty/blob/10526a7d0aa8910729b6036ee51942e05b71abf6/compiler/src/dotty/tools/dotc/ast/Positioned.scala) (a parent class of `Tree`) object has a `uniqueId` field. It is an integer that is unique for that tree and doesn't change from compile run to compile run. You can output these IDs from any printer (such as the ones used by `.show` and `-Xprint`) via `-Yshow-tree-ids` flag, e.g.: ``` dotc -Xprint:frontend -Yshow-tree-ids ../issues/Playground.scala @@ -198,7 +198,7 @@ You can then use these IDs to locate the creation site of a given tree using tha dotc -Ydebug-tree-with-id 1049 ../issues/Playground.scala ``` -When requested what you want to do: +When the tree with the correspond id is allocated, the following prompt will appear: ``` Debug tree (id=1049) creation @@ -208,7 +208,7 @@ Ident(Playground$) a)bort, s)tack, r)esume ``` -Input `s` for stack trace. You will get: +If you input `s`, you will get a stack trace like this: ``` java.lang.Throwable @@ -294,7 +294,7 @@ if (tree.show == """println("Hello World")""") { ``` ## Built-in Logging Architecture -Dotty has a lot of debug calls scattered throughout the code, vast majority of them disabled. At least three (possibly intertwined) architectures for logging are used for that: +Dotty has a lot of debug calls scattered throughout the code, most of which are disabled by default. At least three (possibly intertwined) architectures for logging are used for that: - Printer - Tracing diff --git a/docs/docs/contributing/scala2-vs-scala3.md b/docs/docs/contributing/scala2-vs-scala3.md index c1b56b1c93ab..d7f20190cf57 100644 --- a/docs/docs/contributing/scala2-vs-scala3.md +++ b/docs/docs/contributing/scala2-vs-scala3.md @@ -7,8 +7,8 @@ The following issues encountered when compiling Scala 2 code as-is under Dotty: - There are no `'Symbol`s in Scala 3, you must construct symbols via `new Symbol("foo")` instead of old `'foo` ## Trivial -- Scala 2.13 libraries cannot be used from Dotty, because the type signatures have Scala version `5.3` but `5.0` is expected. -- To use Scala 2.12 dependencies from SBT with Dotty, explicitly suffix their names with `_2.12`. +- Scala 2.13 libraries cannot be used from Dotty because the dotty-library is compiled against the 2.12 standard library which is not binary-compatible with the 2.13 one. We can't be compatible with both at the same time. +- To use Scala 2.12 dependencies from SBT with Dotty, use `withDottyCompat` as documented [here](https://github.com/lampepfl/dotty-example-project#getting-your-project-to-compile-with-dotty). - Feature warnings about implicits `scala.language.implicitConversions` are output by default, unlike in Scala 2. This creates noise. Unclear how to turn off. Implicit conversions must be applied explicitly: From 2445bb20449230d6e94c51b6b6c612971929f2d0 Mon Sep 17 00:00:00 2001 From: Anatolii Date: Fri, 5 Jul 2019 13:53:22 +0200 Subject: [PATCH 11/11] Merge workflow.md and sbt.md --- docs/docs/contributing/tools/sbt.md | 11 ----------- docs/docs/contributing/workflow.md | 12 ++++++++++++ 2 files changed, 12 insertions(+), 11 deletions(-) delete mode 100644 docs/docs/contributing/tools/sbt.md diff --git a/docs/docs/contributing/tools/sbt.md b/docs/docs/contributing/tools/sbt.md deleted file mode 100644 index 6069ccf652ad..000000000000 --- a/docs/docs/contributing/tools/sbt.md +++ /dev/null @@ -1,11 +0,0 @@ -# SBT Commands Cheat Sheet -The basics of working with Dotty codebase are documented [here](http://dotty.epfl.ch/docs/contributing/getting-started.html) and [here](http://dotty.epfl.ch/docs/contributing/workflow.html). Below is a cheat sheet of some frequently used commands (to be used from SBT console – `sbt`). - - -| Command | Description | -|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| -| `dotc ../issues/Playground.scala` | Compile the given file – path relative to the Dotty directory. Output the compiled class files to the Dotty directory itself. | -| `dotr Playground` | Run the compiled class `Playground`. Dotty directory is on classpath by default. | -| `repl` | Start REPL | -| `testOnly dotty.tools.dotc.CompilationTests -- *pos` | Run test (method) `pos` from `CompilationTests` suite. | -| `testCompilation sample` | In all test suites, run test files containing the word `sample` in their title. | diff --git a/docs/docs/contributing/workflow.md b/docs/docs/contributing/workflow.md index 1e8083355b61..870d4a56c272 100644 --- a/docs/docs/contributing/workflow.md +++ b/docs/docs/contributing/workflow.md @@ -64,3 +64,15 @@ u: dotty.tools.dotc.core.Types.Type = TypeBounds(TypeRef(ThisType(TypeRef(NoPref Many objects in the dotc compiler implement a `Showable` trait (e.g. `Tree`, `Symbol`, `Type`). These objects may be prettyprinted using the `.show` method + +## SBT Commands Cheat Sheet ## +The basics of working with Dotty codebase are documented [here](http://dotty.epfl.ch/docs/contributing/getting-started.html) and [here](http://dotty.epfl.ch/docs/contributing/workflow.html). Below is a cheat sheet of some frequently used commands (to be used from SBT console – `sbt`). + + +| Command | Description | +|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| +| `dotc ../issues/Playground.scala` | Compile the given file – path relative to the Dotty directory. Output the compiled class files to the Dotty directory itself. | +| `dotr Playground` | Run the compiled class `Playground`. Dotty directory is on classpath by default. | +| `repl` | Start REPL | +| `testOnly dotty.tools.dotc.CompilationTests -- *pos` | Run test (method) `pos` from `CompilationTests` suite. | +| `testCompilation sample` | In all test suites, run test files containing the word `sample` in their title. |