chore: Upgrade build deps to ensure compatible with Scala Native (#178)

Hi, Haoyi,

This PR will be part of my work to introduce native supports to
`requests-scala` [#156].

I try to update current build dependencies to make sure that current
tool chains are compatible with Scala Native and current implementations
before introducing other breaking changes.

Here are some notable changes:

1. Scala 2.11 must be deprecated because `geny` has deprecated Scala
2.11 since 1.1.0
2. Add Java 21 LTS to GitHub CI for testing job
3. All test cases passed in my local env.
4. But mima binary checks failed. I'm not very sure if it's fine or not.

More details attached here

Local testing results after changes

```console
$ ./mill -i __.publishArtifacts + __.test
[build.mill-57/61] compile
[build.mill-57] [info] compiling 1 Scala source to /home/lqhuang/Git/requests-scala/out/mill-build/compile.dest/classes ...
[build.mill-57] [info] done compiling
[418/420] requests[2.13.15].test.test
[418]
[416/420] requests[3.6.2].test.test
[416]
[420/420] requests[2.12.20].test.test
[420]
[420/420] =============================================================== __.publishArtifacts + __.test ================================================================== 6s
```

report binary issues

```console
$ ./mill -i __.mimaReportBinaryIssues
[183/185] requests[2.13.15].mimaReportBinaryIssues
[184/185] requests[2.12.20].mimaReportBinaryIssues
...
[183] Found 52 issue when checking against com.lihaoyi:requests:0.7.0
[184] Found 52 issue when checking against com.lihaoyi:requests:0.7.0
...
[185/185] ================================================================= __.mimaReportBinaryIssues ==================================================================== 2s
3 tasks failed
requests[2.12.20].mimaReportBinaryIssues Failed binary compatibility check! Found 206 potential problems
requests[2.13.15].mimaReportBinaryIssues Failed binary compatibility check! Found 206 potential problems
requests[3.6.2].resolvedMimaPreviousArtifacts scala.MatchError: Failure(
Resolution failed for 1 modules:
--------------------------------------------
  com.lihaoyi:requests_3:0.6.7
        not found: /home/lqhuang/.ivy2/local/com.lihaoyi/requests_3/0.6.7/ivys/ivy.xml
        not found: https://repo1.maven.org/maven2/com/lihaoyi/requests_3/0.6.7/requests_3-0.6.7.pom

--------------------------------------------
```

Any feedback and suggestion is appreciated!

Regards,
Lanqing

----

Updates:

After refactoring `build.mill`, now binary check passed in my local env,
too.

```
$ ./mill clean && ./mill -i __.mimaReportBinaryIssues
[build.mill-64/68] compile
[build.mill-64] [info] compiling 1 Scala source to /home/lqhuang/Git/requests-scala/out/mill-build/compile.dest/classes ...
[build.mill-64] [info] done compiling
[1/1] clean
[1/1] ============================== clean ==============================
[197/212] mill.scalalib.ZincWorkerModule.worker
[197] Compiling compiler interface...
[200/212] requests.jvm[3.3.4].compile
[199/212] requests.jvm[2.13.15].compile
[199] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/2.13.15/compile.dest/classes ...
[200] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/3.3.4/compile.dest/classes ...
[199] [warn] 4 deprecations (since 0.9.0)
[199] [warn] 4 deprecations (since 2.13.0)
[199] [warn] 8 deprecations in total; re-run with -deprecation for details
[199] [warn] 3 feature warnings; re-run with -feature for details
[199] [warn] four warnings found
[199] [info] done compiling
[200] [warn] there were 3 feature warnings; re-run with -feature for details
[200] [warn] there were 6 deprecation warnings; re-run with -deprecation for details
[200] [warn] two warnings found
[200] [info] done compiling
[204/212] requests.jvm[2.13.15].mimaReportBinaryIssues
[208/212] requests.jvm[3.3.4].mimaReportBinaryIssues
[204] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/2.13.15/compile.dest/classes ...
[208] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/3.3.4/compile.dest/classes ...
[204] Binary compatibility check passed
[198/212] requests.jvm[2.12.20].compile
[198] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/2.12.20/compile.dest/classes ...
[208] Binary compatibility check passed
[198] [warn] 6 deprecations (since 0.9.0); re-run with -deprecation for details
[198] [warn] three feature warnings; re-run with -feature for details
[198] [warn] two warnings found
[198] [info] done compiling
[212/212] requests.jvm[2.12.20].mimaReportBinaryIssues
[212] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/2.12.20/compile.dest/classes ...
[212] Binary compatibility check passed
[212/212] ============================== __.mimaReportBinaryIssues ============================== 13s
```

And now if you access `https://self-signed.badssl.com/`, it literally
returns a "404 Not Found". Moreover, it looks like the whole site
`badssl.com` returns `404` (???? Shutdown or server error?)

I update corresponding unit tests.

```console
$ curl --insecure -vvv https://self-signed.badssl.com/
* Uses proxy env variable no_proxy == '.local,.internal,.arpa,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16'
* Host self-signed.badssl.com:443 was resolved.
* IPv6: (none)
* IPv4: 104.154.89.105
*   Trying 104.154.89.105:443...
* Connected to self-signed.badssl.com (104.154.89.105) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / prime256v1 / rsaEncryption
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=BadSSL; CN=*.badssl.com
*  start date: Dec 19 21:03:33 2024 GMT
*  expire date: Dec 19 21:03:33 2026 GMT
*  issuer: C=US; ST=California; L=San Francisco; O=BadSSL; CN=*.badssl.com
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
> GET / HTTP/1.1
> Host: self-signed.badssl.com
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: nginx/1.10.3 (Ubuntu)
< Date: Tue, 07 Jan 2025 10:41:04 GMT
< Content-Type: text/html
< Content-Length: 178
< Connection: keep-alive
<
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>
* Connection #0 to host self-signed.badssl.com left intact
```

Current notable changes:

1. Scala 2.11 must be deprecated because `geny` has deprecated Scala
2.11 since 1.1.0
2. `Option.when(VcsVersion.vcsState().commitsSinceLastTag !=
0)(VcsVersion.vcsState().lastTag).flatten` only appends `0.6.7`, so I
replaced it with concrete release versions.
3. Add Java 21 LTS to GitHub CI for testing job
4. Add Scala 3.6.2 to GitHub CI to run testing job for Scala Next
5. test cases passed in my local env.
6. binary check passed in my local env

---------

Co-authored-by: Li Haoyi <[email protected]>
Co-authored-by: Ondra Pelech <[email protected]>

Author: 3 people

.github/workflows/actions.yml

@@ -9,34 +9,40 @@ on:
   pull_request:
     branches:
       - master
+  workflow_dispatch: # be able to manually trigger a workflow run
+
+env:
+  SCALA_NEXT_VERSION: '3.6.2'
+  # TODO: perherps, it would be nice to allow failure on `SCALA_NEXT_VERSION` as a warning
+  #       to avoid too strict dev experiences in the future improvements.
 
 jobs:
   test:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        java: ['11', '17']
+        java: ['11', '17', '21']
     steps:
-      - uses: actions/checkout@v3
-      - uses: actions/setup-java@v3
+      - uses: actions/checkout@v4
+      - uses: actions/setup-java@v4
         with:
           distribution: 'temurin'
           java-version: ${{ matrix.java }}
       - name: Run tests
-        run: ./mill -i __.publishArtifacts + __.test
+        run: ./mill -i -DscalaNextVersion=${{ env.SCALA_NEXT_VERSION }} __.publishArtifacts + __.test
 
   check-binary-compatibility:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0
-      - uses: actions/setup-java@v3
+      - uses: actions/setup-java@v4
         with:
           distribution: 'temurin'
-          java-version: 11
+          java-version: 17
       - name: Check Binary Compatibility
-        run: ./mill -i __.mimaReportBinaryIssues
+        run: ./mill -i -DscalaNextVersion=${{ env.SCALA_NEXT_VERSION }} __.mimaReportBinaryIssues
 
   publish-sonatype:
     if: github.repository == 'com-lihaoyi/requests-scala' && contains(github.ref, 'refs/tags/')
@@ -47,12 +53,12 @@ jobs:
       SONATYPE_PGP_PRIVATE_KEY_PASSWORD: ${{ secrets.SONATYPE_PGP_PRIVATE_KEY_PASSWORD }}
       SONATYPE_USER: ${{ secrets.SONATYPE_USER }}
       SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
-      LANG: "en_US.UTF-8"
-      LC_MESSAGES: "en_US.UTF-8"
-      LC_ALL: "en_US.UTF-8"
+      LANG: 'en_US.UTF-8'
+      LC_MESSAGES: 'en_US.UTF-8'
+      LC_ALL: 'en_US.UTF-8'
     steps:
-      - uses: actions/checkout@v3
-      - uses: actions/setup-java@v3
+      - uses: actions/checkout@v4
+      - uses: actions/setup-java@v4
         with:
           distribution: 'temurin'
           java-version: 11
@@ -73,6 +79,7 @@ jobs:
           fi
       - name: Create GitHub Release
         id: create_gh_release
+        # WARN: `actions/create-release` is currently unmaintained. It just works now, keep it until it breaks.
         uses: actions/[email protected]
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token

.mill-version

@@ -1 +1 @@
-0.12.0
+0.12.5

build.mill

@@ -1,29 +1,32 @@
 package build
+
 import mill._
+import mill.scalalib._
 import mill.scalalib.publish.{Developer, License, PomSettings, VersionControl}
-import scalalib._
-import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.4.0`
-import $ivy.`com.github.lolgab::mill-mima::0.0.23`
+
+import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.4.1`
+import $ivy.`com.github.lolgab::mill-mima::0.0.24`
 
 import de.tobiasroeser.mill.vcs.version.VcsVersion
 import com.github.lolgab.mill.mima._
 
-val dottyVersion = sys.props.get("dottyVersion")
+val scalaNextVersion = sys.props.get("scalaNextVersion")
+val scalaVersions = List("2.12.20", "2.13.15", "3.3.4") ++ scalaNextVersion
+val scalaNativeVer = "0.5.6"
 
-val scalaVersions = List("2.12.17", "2.13.10", "2.11.12", "3.1.1") ++ dottyVersion
+trait MimaCheck extends Mima {
+   def mimaPreviousVersions = Seq("0.6.9", "0.7.0", "0.7.1", "0.8.0","0.8.2", "0.9.0").distinct
 
-object requests extends Cross[RequestsModule](scalaVersions)
-trait RequestsModule extends CrossScalaModule with PublishModule with Mima {
-  def publishVersion = VcsVersion.vcsState().format()
-  def mimaPreviousVersions =
-    (
-      Seq("0.7.0", "0.7.1", "0.8.2") ++
-      Option.when(VcsVersion.vcsState().commitsSinceLastTag != 0)(VcsVersion.vcsState().lastTag).flatten
-    ).distinct
   override def mimaBinaryIssueFilters = Seq(
     ProblemFilter.exclude[ReversedMissingMethodProblem]("requests.BaseSession.send"),
-    ProblemFilter.exclude[DirectMissingMethodProblem]("requests.Response.string")
+    ProblemFilter.exclude[DirectMissingMethodProblem]("requests.Response.string"),
   )
+}
+
+trait RequestsPublishModule extends PublishModule with MimaCheck {
+  def artifactName = "requests"
+
+  def publishVersion = VcsVersion.vcsState().format()
 
   def pomSettings = PomSettings(
     description = "Scala port of the popular Python Requests HTTP client",
@@ -32,17 +35,39 @@ trait RequestsModule extends CrossScalaModule with PublishModule with Mima {
     licenses = Seq(License.MIT),
     versionControl = VersionControl.github("com-lihaoyi", "requests-scala"),
     developers = Seq(
-      Developer("lihaoyi", "Li Haoyi","https://github.com/lihaoyi")
+      Developer("lihaoyi", "Li Haoyi", "https://github.com/lihaoyi")
     )
   )
 
-  def ivyDeps = Agg(ivy"com.lihaoyi::geny::1.0.0")
+  def ivyDeps = Agg(ivy"com.lihaoyi::geny::1.1.1")
+}
 
-  object test extends ScalaTests with TestModule.Utest {
-    def ivyDeps = Agg(
-      ivy"com.lihaoyi::utest::0.7.10",
-      ivy"com.lihaoyi::ujson::1.3.13",
-      ivy"com.dimafeng::testcontainers-scala-core:0.41.3"
-    )
+trait RequestsCrossScalaModule extends CrossScalaModule with ScalaModule {
+  def millSourcePath = build.millSourcePath / "requests"
+  def sources = T.sources(millSourcePath / "src")
+}
+
+trait RequestsTestModule extends TestModule.Utest {
+  def ivyDeps = Agg(
+    ivy"com.lihaoyi::utest::0.7.10",
+    ivy"com.lihaoyi::ujson::1.3.13",
+    ivy"com.dimafeng::testcontainers-scala-core:0.41.3"
+  )
+}
+
+object requests extends Module {
+  trait RequestsJvmModule extends RequestsCrossScalaModule with RequestsPublishModule {
+    object test extends ScalaTests with RequestsTestModule
   }
+  object jvm extends Cross[RequestsJvmModule](scalaVersions)
+
+  // trait RequestsNativeModule extends ScalaNativeModule with RequestsPublishModule {
+  //   override def scalaNativeVersion = scalaNativeVer
+  //
+  //   def ivyDeps =
+  //     super.ivyDeps() ++ Agg(ivy"com.github.lolgab::scala-native-crypto::0.1.0")
+  //
+  //   object test extends ScalaNativeTests with RequestsTestModule
+  // }
+  // object native extends Cross[RequestsNativeModule](scalaVersions)
 }

requests/test/src/requests/RequestTests.scala

@@ -271,7 +271,7 @@ object RequestTests extends HttpbinTestSuite {
     test("selfSignedCertificate"){
       val res = requests.get(
         "https://self-signed.badssl.com",
-        verifySslCerts = false
+        verifySslCerts = false,
       )
       assert(res.statusCode == 200)
     }