Release 2.13.2

[SethTisue]

release lead: @SethTisue

SCALA_VER_BASE="2.13.2"
SCALA_VER_SUFFIX=""
SCALA_SHA=706ef1b291134a5e5bce2275df2c222261f73451
DIST_SHA=????????????????????????????????????????
SCALA_VER="$SCALA_VER_BASE$SCALA_VER_SUFFIX"

Key links:

• scala/scala milestone: https://github.com/scala/scala/milestones/2.13.2
• scala/bug milestone: https://github.com/scala/bug/milestones/2.13.2
• scala/scala-dev milestone: https://github.com/scala/scala-dev/milestones/2.13.2

N weeks before the release

• Wind down PR queue. There has to be enough time after the last (non-trivial) PR is merged and the next phase. The core of the eco-system needs time to prepare for the final!
• Triage scala/bug and scala/scala-dev tickets
• Create next scala/scala milestone, move the magical "Merge to 2.13.x" description to it (so Scabot uses it as default for new PRs)
• Close the scala/bug milestone, create next milestone, move pending issues
• Close the scala/scala-dev milestone, create next milestone, move pending issues
• Check PRs assigned to the milestone, also check WIP
• Announce expected release date and current nightly "release candidate" (nightly sha-mangled version) at https://scala-ci.typesafe.com/artifactory/scala-integration/ on https://contributors.scala-lang.org/c/announcements

Release announcement / notes

• Notify community on https://contributors.scala-lang.org/c/announcements
• Review merged PRs, make sure release-notes label is applied appropriately
• PRs with release-notes label must have excellent title & description (title will be pasted literally in release note bullet list)
  • https://github.com/scala/scala/pulls?q=milestone%3A2.13.2+is%3Aclosed+label%3Arelease-notes
• Draft release notes (to be published once GitHub tag is pushed)
  • note that GitHub release notes drafts can only be viewed by committers, so use a gist to draft the notes, so the gist can be shared with the community
• On contributors thread, link to release note gist and request feedback

N days before release

• Announce no more PRs will be merged unless last-minute regressions are found. Re-iterate current nightly sha version for testing.
• Community build
  • won't be anywhere near all green, but we should make sure some nontrivial number of projects are still green
  • run with expected-greens only: https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/?
  • also do a full run to see if there are any unexpected greens
• Run scala-collections-laws and evaluate results
  • Rex Kerr (@Ichoran) is the keeper and expert on this
• Windows Jenkins job
  • don't just trust the automated nightly runs, manually trigger a run on the exact SHA
  • https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-windows/1353
• Check any merged PRs accidentally assigned to the next milestone in this branch, and re-assign them to this milestone
• Merge in any older release branch
• Check module versioning (is everything in versions.properties up to date?)
• On major release, bump PickleFormat version

Point of no return

• Once sufficient time has passed since last merged PR (1-2 weeks depending on whether it's a maintenance or development branch), and core projects have tried out the candidate nightly, it's time to cut the release!
• Make sure there are no stray staging repos on sonatype
• Trigger a build on travis, specify SCALA_VER_BASE and SCALA_VER_SUFFIX in the custom config
  • config: before_script: export SCALA_VER_BASE=$SCALA_VER_BASE SCALA_VER_SUFFIX=$SCALA_VER_SUFFIX)
  • Check the build status on https://github.com/scala/scala/commits/2.13.x
  • Check that the scala/scala job also triggered a following scala/scala-dist job: https://travis-ci.org/scala/scala-dist/builds/678451545
  • Note the scala/scala commit (the full 40-char sha!)
  • Create the scala/scala tag locally: git tag -s -m "Scala $SCALA_VER" v$SCALA_VER $SCALA_SHA
  • Note the scala/scala-dist commit (the full 40-char sha!)
  • Create scala-dist tag locally: git tag -s -m "Scala $SCALA_VER" v$SCALA_VER $DIST_SHA
  • Note the repos to be promoted after tag is cut (see travis log)
    • https://oss.sonatype.org/content/repositories/orgscala-lang-2136
    • https://oss.sonatype.org/content/repositories/orgscala-lang-2137
    • https://oss.sonatype.org/content/repositories/orgscala-lang-2138
    • https://oss.sonatype.org/content/repositories/orgscala-lang-2139
• Sanity check jar/pom
  • https://oss.sonatype.org/content/repositories/staging/org/scala-lang/scala-compiler/$SCALA_VER/
  • in particular, if the release was staged multiple times, double check that https://oss.sonatype.org/content/repositories/staging/ has the files from the most recent build
• Check that JARs haven't mysteriously bloated — compare sizes to previous release. We have no other backstop for this.
• Remember, tags are forever, so are maven artifacts (even staged ones, as they could end up in local caches) and S3 uploads (S3 buckets can be changed, but it can takes days to become consistent)
• Push scala/scala tag: git push https://github.com/scala/scala.git v$SCALA_VER
• Add release notes to https://github.com/scala/scala/releases/tag/v$SCALA_VER
• Push scala/scala-dist tag: git push https://github.com/scala/scala-dist.git v$SCALA_VER
• Trigger two scala-dist jobs on travis (https://travis-ci.org/scala/scala-dist) with custom config. must use full-length SHAs!
  • before_script: export version=$SCALA_VER scala_sha=$SCALA_SHA mode=archives: https://travis-ci.org/github/scala/scala-dist/builds/678454894
  • before_script: export version=$SCALA_VER scala_sha=$SCALA_SHA mode=update-api: https://travis-ci.org/github/scala/scala-dist/builds/678455093
• Promote staging repos: st_stagingRepoPromote [scala-repo], st_stagingRepoPromote [modules-repo] (or use oss.sonatype.org web UI)

Check availability

• Check release on sonatype: https://oss.sonatype.org/content/repositories/releases/org/scala-lang/scala-compiler/2.13.2/
• Check the release on maven central: https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.2/
• Check the release on maven search (takes longer): http://search.maven.org/#search%7Cga%7C1%7Cg%3Aorg.scala-lang%20v%3A2.13.2

When everything is on maven central

• Prepare PR to https://github.com/scala/scala-lang/ (using scala/make-release-notes which requires a staged release and a pushed tag)
  • documentation/api.md
  • download/index.md
  • _config.yml (update devscalaversion or scalaversion)
  • release Scala 2.13.2 scala-lang#1135
• Prepare PR to https://github.com/scala/docs.scala-lang/
  • api/all.md
  • release Scala 2.13.2 docs.scala-lang#1687
• make sure the draft release notes are on GitHub tag: https://github.com/scala/scala/releases/tag/v2.13.2
• Pre-announce the release on https://contributors.scala-lang.org/c/announcements
• On 2.13.0 only: (manually) update the current symlink for the API docs
  • https://github.com/scala/scala-dist/blob/2.13.x/scripts/jobs/release/website/update-api#L15
• Check that the API docs are published
  • http://www.scala-lang.org/api/2.13.2/
  • https://docs.scala-lang.org/api/all.html ?
• Merge the scala-lang PR and the docs.scala-lang.org PR
  • wait for them to arrive on the websites and make sure they look okay

Modules

• build and release scala-collection-compat and other modules (or open tickets asking that the maintainers do so)
  • this work has moved to https://github.com/scala/make-release-notes/blob/2.13.x/projects-2.13.md
• if it's a 2.12.x release, publish macro paradise for the new version

Announcements

• Scala Users discourse https://users.scala-lang.org
• add a reply on the same https://contributors.scala-lang.org thread where you announced that the release process was starting
• Tweet from @scala_lang
• https://gitter.im/scala/scala
  • if you feel like it, say something in https://gitter.im/scala/contributors too
• ask Seth to announce on #scala IRC

Afterwards

• Create PR to add/update spec links on scala-lang.org (example: update spec links and advisements scala-lang#1050)
• Create PR to update scala-version on docs.scala-lang.org (example: bump scala-version docs.scala-lang#1294)
• Create a scala/scala PR to update versions.properties and the baseVersion in build.sbt.
  • re-STARR onto 2.13.2 scala#8910
  • If it's a major bump, also update MiMa base version, exclusion filters, and spec/_config.yml.
• If it's a major bump:
  • Update latestSpecVersion in spec/_config.yml on the old branch, so that spec is marked as no longer current
  • Ditto for the nightly build and spec links in _data/footer.yml and _data/doc-nav-header.yml on docs.scala-lang.org
• Create PR to update https://github.com/lightbend/lightbend-platform-docs/blob/master/docs/modules/getting-help/pages/build-dependencies.adoc
• Update WhiteSource

[ijuma]

Do we have a rough ETA for 2.13.2?

[SethTisue]

@ijuma I hope to build tomorrow. only a few JLine tweaks have yet to land; I expect to finish those up today

[ijuma]

Awesome, thanks!

[SethTisue]

2.13.2-bin-7d29ccc is the release candidate. If all goes well 2.13.2 itself will hit Maven Central later today

[h-vetinari]

2.13.2-bin-7d29ccc is the release candidate. If all goes well 2.13.2 itself will hit Maven Central later today

Seeing that scala/scala@7d29ccc was created 2h ago, that's a tremendously short window for a release candidate (I was, for example, hoping the spark folks would test a release candidate once it became available, with enough time to provide feedback). Why bother at all with rc's if it's <24h?

[SethTisue]

@h-vetinari Good question! I think I can explain.

Spark doesn't support 2.13 yet, except on a development branch, so until that upgrade work is farther along, I see no reason for a 2.13.x release to block waiting on Spark specifically.

Nonetheless, we welcome feedback from Spark folks at any point — the earlier the better! Note that one of the major changes in 2.13.2, scala/scala#8748, involves aligning the REPL with Spark's needs.

Also, we've already publicized several release candidates at https://contributors.scala-lang.org/t/coming-soon-scala-2-12-11-scala-2-13-2/4003/31 , the current one is merely the latest. Very little has changed in the last 4 weeks, and most of what has changed in these final weeks is in the UI code for the REPL, or in other minor features that don't affect the viability of a release.

Finally, note that the 7-month delay between 2.13.1 and 2.13.2 is an extreme outlier. 2.13.3 is likely to follow in 2–3 months, or even sooner if notable regressions turn up.

[h-vetinari]

Well, who am I to say. Just from my POV: I was waiting for the REPL work to finish, and to see an actual tagged rc under releases. Then I would have reached out to the spark people if that fixes the issues that blocked them from upgrading to 2.13, and figure out what if anything is missing before having them blocked another ~6 months.

The spark ecosystem is lagging extremely in terms of scala versions (for example, the entire cloudera stack up until the most recent version is still on scala 2.11, because spark 3.0 hasn't been released yet, much less made it into a CDH/CDP release). This creates quite an extreme stretch for some libraries in terms of widely used scala versions needing support. It's 10 months since the release of 2.13, and if there are any blockers still left in 2.13.2 as currently planned from the spark side of things, that gap will only continue to widen.

Anyway, just my 2 cents.

[smarter]

@h-vetinari There was a single issue that was known to be blocking Spark from using Scala 2.13.1: scala/bug#11840. I quickly fixed it as soon as I heard about it, and I left a comment giving detailed instructions on how to test a version of Scala with the fix: scala/bug#11840 (comment), since then, there hasn't been any feedback from Spark, so I don't really see what else you were expecting.

[SethTisue]

draft release notes, feedback welcome: https://gist.github.com/SethTisue/16a53b061754dd0f0073193263b6ba2b

[ijuma]

Then I would have reached out to the spark people if that fixes the issues that blocked them from upgrading to 2.13, and figure out what if anything is missing before having them blocked another ~6 months.

@SethTisue clarified in his message that 2.13.3 is likely to follow in 2-3 months. Given that Spark 3.0.0 RC1 is in progress[1], I don't think Scala 2.13 support will be available until Spark 3.1.0 in the best case.

[1] http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Apache-Spark-3-0-0-RC1-td29151.html

[smarter]

Yes, in fact https://issues.apache.org/jira/browse/SPARK-25075 has been listing 3.1 as target version since November.

[SethTisue]

JARs are on Maven Central, websites are updated, tweet is tweeted

[h-vetinari]

@smarter: [...] there hasn't been any feedback from Spark, so I don't really see what else you were expecting.

Considering that the spark issue for 2.13 REPL changes hasn't yet seen much action, and considering that the JLine changes came in shortly before the release, there's a non-negligible probability for another blocker, and verifying this would have IMO been beneficial. I had asked about this three weeks ago, and was then planning to ping the spark people once there was a release candidate (not realising that there would be no such tagged release candidate).

But ok, no use over-analysing this further. Let's see how things go.