regression: some repos now fail with cross-version suffixes error on scala-xml

[SethTisue]

Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml

not sure when this started or what's going on, need to investigate. some of the failing repos are kind-projector, scalajson, akka-actor

[SethTisue]

in scalajson's case I think this might have to do with the scalajson -> scalatest -> scalactic -> scala-reflect -> scala-xml dependency chain?

oh but that isn't the chain, scala-reflect doesn't depend on scala-xml. the scala-xml dependency is scalatest's.

[SethTisue]

not sure I have a complete list of failing projects in hand, but:

I think all the failing projects are either:

• compiler plugins (kind-projector, twotails)
• ScalaTest users (the others)

[SethTisue]

from e.g. the kind-projector log,

[kind-projector] [info] 	org.scala-lang.modules#scala-xml;0.1-dbuildxa3d7c1f2fbbff31cd9ef52c50629720657fe755c from mvn-build-local in [compile, runtime, test, provided, compile-internal, runtime-internal, test-internal, scala-tool]
[kind-projector] [info] 	org.scala-lang.modules#scala-xml_2.12;1.0.6 from cache in [compile, runtime, test, provided, compile-internal, runtime-internal, test-internal]

note the missing _2.12

[SethTisue]

I wonder exactly this when regressed, there might be a clue there. but Jenkins is down

downgrading sbt-version from 0.13.17 to 0.13.16 doesn't help. running dbuild with version=2.12.4 doesn't help.

oh, now this is interesting: scala/scala-xml#181

that looks very interesting

/cc @ashawley

[SethTisue]

darnit, rewinding scala-xml to scala/scala-xml@171a199 (October 2017) doesn't make the errors go away. sorry Aaron, false alarm

[ashawley]

Yeah, I wondered what fixing the exclusion rule would actually do to do the community build, but then the Typesafe Jenkins crashed, so I never found out, until...

Ok, hope it is a false alarm.

[SethTisue]

okay, I noticed something else the failing projects have in common: they are on sbt 1.

and yeah, if I rewind kind-projector to typelevel/kind-projector@5729d05, before the sbt 1 move, the problem goes away.

so that's progress at least, this is some sbt 0.13 vs sbt 1 difference

[SethTisue]

using kind-projector as our guinea pig, on sbt 0.13.17 ++2.12.4 then show fullClasspath shows:

[info] * Attributed(/Users/tisue/.ivy2/cache/org.scala-lang/scala-compiler/jars/scala-compiler-2.12.4.jar)
[info] * Attributed(/Users/tisue/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.4.jar)
[info] * Attributed(/Users/tisue/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.12.4.jar)
[info] * Attributed(/Users/tisue/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.0.6.jar)

but on sbt 1.1.0

[info] * Attributed(/Users/tisue/.sbt/boot/scala-2.12.4/lib/scala-compiler.jar)
[info] * Attributed(/Users/tisue/.sbt/boot/scala-2.12.4/lib/scala-library.jar)
[info] * Attributed(/Users/tisue/.sbt/boot/scala-2.12.4/lib/scala-reflect.jar)
[info] * Attributed(/Users/tisue/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.0.6.jar)

is the difference significant? not sure. it sure looks relevant. also fwiw ~/.sbt/boot/scala-2.12.4/lib contains a scala-xml_2.12.jar

[SethTisue]

Scalatest's xml dependency, from its ivy-3.0.4.xml:

        <dependency org="org.scala-lang.modules" name="scala-xml_2.12" rev="1.0.5" force="true" conf="compile->compile(*),master(compile);runtime->runtime(*)"/>

[SethTisue]

I wonder if this is a bug in dbuild's "Scala" build system. org.scala-lang.modules#scala-xml;0.1-dbuildxa3d7c1f2fbbff31cd9ef52c50629720657fe755c is just wrong (missing _2.12), and it's got dbuild's fingerprints on it.

hypothesis: scala-compiler, scala-library, and scala-reflect all correctly lack the _2.12 suffix, so maybe dbuild thinks everything that's part of the Scala bootstrap is supposed to lack a binary version suffix.

that would still leave me confused about the sbt 0.13 vs sbt 1 part, though...

[SethTisue]

okay I bet we can deal with this by pulling scala-xml out of the "Scala" build system section, or maybe by not using that build system at all. which would be a nice simplification anyway actually

trying it...

[SethTisue]

thinking out loud:

• in a 2.12 context, we could say that's it, we're just not upgrading the scala-xml version used by Scaladoc anymore. so then there's no need to have scala-xml in the 2.12 community build (except perhaps as a standalone thing that isn't used by any downstream projects)
• in a 2.13 context, we plan to cut the Scaladoc tools's dependency on scala-xml anyway, so at that point scala-xml becomes just another project.

[SethTisue]

another thought would be to use conflictWarning := ConflictWarning.disable or something similar to just sweep the problem under the rug.

[SethTisue]

I bet we can deal with this by pulling scala-xml out of the "Scala" build system section

gah, that causes Scaladoc generation to fail in every project because scala-xml isn't on the classpath. we could inject scala-xml into every project, but then it's on the classpath all the time, not just for Scaladoc generation.

[SethTisue]

new hope: I think this might be addressable just using dbuild's cross-version setting appropriately. the current behavior around scala-xml isn't dbuild acting crazy, it's explicitly specified in our config file with cross-version: disabled. I never had to mess with this before or even think about it before.

[SethTisue]

-  cross-version: disabled
 
   space: scala
 
@@ -117,6 +116,7 @@ build += {
   {
     name:  "scala"
     system: assemble
+    cross-version: binary
     extra.parts.projects: [
       {
         set-version: ${vars.scala-version}
@@ -140,6 +140,10 @@ build += {
       {
         name: "scala-xml"
         uri: "https://github.com/scala/scala-xml.git"
+        cross-version: standard

this is promising, builds seem to be passing.

I'm seeing:

[kind-projector] [warn] Found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[kind-projector] [warn] 	* org.scala-lang.modules:scala-xml_2.12:1.0.6 is selected over 0.1-dbuildx172419e1480f08f2bf730d81f11ef2824641b438

which isn't ideal, we would prefer that the freshly built scala-xml be used. but it's hardly a show-stopper

[SethTisue]

@cunei reading all of this over might amuse you.

[SethTisue]

effectively, our config file was wrong before, but something changed in sbt 1 that exposed the previously latent bug.

[SethTisue]

right now I'm testing with version=2.12.4 because scala-ci.typesafe.com is down so I can't test against Scala nightlies. I'm a little afraid my fix might prove not quite right once used with a Scala version such as 2.12.5-bin-..... or 2.13.0-pre-..., but I can cross that bridge when I come to it.