From 855222e92882b23491f5cf5a7e2bbd814909fdb5 Mon Sep 17 00:00:00 2001 From: Mike Bland Date: Fri, 28 Feb 2025 10:25:06 -0500 Subject: [PATCH] Bazel 8 + `rules_java` 8 updates, protoc toolchain Bumps dependencies to versions that are compatible with both Bazel 7.6.0 and 8.0.0, and adds protocol compiler toolchainization in `//protoc` for `protobuf` v29 and later. Closes #1652. Part of #1482. - ScalaPB jars: 0.11.17 => 1.0.0-alpha.1 - `rules_python`: 0.38.0 => 1.2.0 - `rules_cc`: 0.0.9 => 0.1.1 - `rules_java`: 7.12.4 => 8.11.0 - `protobuf`: 21.7 => 30.1 - `rules_proto`: 6.0.2 => 7.1.0 Bazel 6 is officially unsupported as of this change and the upcoming `rules_scala` 7.0.0 release. Updates `.bazelci/presubmit.yml` to bump the `7.x` build to `last_rc`. Registers a precompiled protocol compiler toolchain when `--incompatible_enable_proto_toolchain_resolution` is `True`. Otherwise, `register_toolchains("@rules_scala_protoc_toolchains//:all")` is a no-op, as it will be empty. `scripts/update_protoc_integrity.py` automatically updates `scala/private/protoc/protoc_integrity.bzl`. The `protobuf` patch is the `git diff` output from protocolbuffers/protobuf#19679, which also inspired the updates to `scala_proto/scala_proto_toolchain.bzl`. The `proto_lang_toolchain` call in the `BUILD` file generated by `protoc/private/protoc_toolchains.bzl` was inspired by the `README` from: - https://github.com/aspect-build/toolchains_protoc/ Loads `java_proto_library` from `com_google_protobuf`, replacing the officially deprecated version from `rules_java`. Adds the `scala` parameter to `scala_toolchains()` to control whether it instantiates the builtin Scala toolchains. Removes the `if len(toolchains) == 0` check from `_scala_toolchains_repo_impl`. The Scala version check will now happen only when both `scala` and `validate_scala_version` are `True`, which is essentially how the previous API worked. Updates to `README.md`, and updates to `WORKSPACE` and `third_party/repositories` files precipitated by the dependency updates, comprise the remainder of this change. --- We're no longer planning to support Bazel 6 in the next major release per @simuons's decision in: - https://github.com/bazelbuild/rules_scala/issues/1482#issuecomment-2690786093 The plan is now to land the Bazel 7 and 8 compatibility updates first, then land the Bzlmod change. This enables us to make only one new major version release, instead of two (whereby the first release would've continued supporting Bazel 6). It turns out the two major version plan wouldn't've been possible. Bazel 8 and `rules_java` 8 require `protobuf` >= v29, but this bump caused Windows builds to break when compiling `protoc` in #1710. `src/google/protobuf/compiler/java/java_features.pb.h`, the path specified in the error message, doesn't exist until `protobuf` v25.0. @crt-31 and I found that this was related to the Windows/MSVC 260 character file path length limit. What's more, the `protobuf` team plans to drop MSVC support specifically because of this path length limit. The protocol compiler toolchain prevents `protoc` recompilation, which fixes the Windows breakage while making all builds faster. Since Windows builds break since at least `protobuf` v25, but `protoc` toolchainization requires v29, the version bump and the `protoc` toolchain must land together. Disabling the default Scala toolchain via `scala_toolchains(scala = False)` avoids instantiating any builtin compiler JAR repos or validating the Scala version. This enables users defining custom Scala toolchains using their own JARs to still use other builtin toolchains. This was prompted by: https://github.com/bazelbuild/rules_scala/pull/1710#issuecomment-2738511164 Removing the `if len(toolchains) == 0` covers the case in the upcoming Bzlmod implementation whereby the root module may explicitly disable all builtin toolchains. This avoids potential breakage of the `register_toolchains("@rules_scala_toolchains//...:all")` call from the upcoming `MODULE.bazel` file. Removing the `scala_register_toolchains()` calls from the `dt_patches/test_dt_patches*/WORKSPACE` files proves that those calls were harmless, but ultimately unnecessary. --- I tried several things to get protocol compiler toolchainization to work with `protobuf` v28.2, described below. However, each path only led to the same suffering described in the new "Why this requires `protobuf` v29 or later" section of the README. I discovered along the way that `protobuf` v30 isn't compatible with Bazel 6.5.0 at all. I added an explanation to the "Limited Bazel 6.5.0 compatibility" section of `README.md`. --- I experimented with using `protobuf` v28.2, `rules_proto` 6.0.2, and `rules_java` 7.12.4 and 8.10.0. I updated the `protobuf` patch for v28.2 with the following statements: ```py load("//bazel/common:proto_common.bzl", "proto_common") load("@rules_proto//proto:proto_common.bzl", "toolchains") _PROTO_TOOLCHAIN = "@rules_proto//proto:toolchain_type" _PROTO_TOOLCHAIN_ATTR = "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION" _PROTOC_TOOLCHAINS = toolchains.use_toolchain(_PROTO_TOOLCHAIN) def _protoc_files_to_run(ctx): if getattr(proto_common, _PROTO_TOOLCHAIN_ATTR, False): ``` I tried using `proto_common` from `rules_proto`. I also created a `rules_proto` 6.0.2 patch for `proto_toolchain()` to fix a "no such package: //proto" breakage: ```diff diff --git i/proto/private/rules/proto_toolchain.bzl w/proto/private/rules/proto_toolchain.bzl index a091b80..def2699 100644 --- i/proto/private/rules/proto_toolchain.bzl +++ w/proto/private/rules/proto_toolchain.bzl @@ -33,7 +33,7 @@ def proto_toolchain(*, name, proto_compiler, exec_compatible_with = []): native.toolchain( name = name + "_toolchain", - toolchain_type = "//proto:toolchain_type", + toolchain_type = Label("//proto:toolchain_type"), exec_compatible_with = exec_compatible_with, target_compatible_with = [], toolchain = name, ``` I tried adding combinations of the following `--incompatible_autoload_externally` flag values to .bazelrc`: ```txt common --incompatible_autoload_externally=+@protobuf,+@rules_java ``` Nothing worked. --- After the `protobuf` v29 bump, and before the ScalaPB 1.0.0-alpha.1 bump, `scala_proto` targets would fail with the following error: ```txt ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14: ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/.../bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) --jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError: 'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)' at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329) [ ...snip... ] java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:30) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14 Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ``` --- .bazelci/presubmit.yml | 11 +- .bazelrc | 13 +- README.md | 473 ++++++++++++++++-- WORKSPACE | 36 +- docs/scala_toolchain.md | 32 +- dt_patches/test_dt_patches/WORKSPACE | 28 +- .../test_dt_patches_user_srcjar/WORKSPACE | 28 +- examples/crossbuild/WORKSPACE | 20 +- examples/overridden_artifacts/WORKSPACE | 24 +- examples/scala3/WORKSPACE | 20 +- examples/semanticdb/WORKSPACE | 20 +- .../multi_frameworks_toolchain/WORKSPACE | 20 +- .../testing/scalatest_repositories/WORKSPACE | 20 +- .../specs2_junit_repositories/WORKSPACE | 20 +- .../0001-protobuf-19679-rm-protoc-dep.patch | 91 ++++ protoc/BUILD | 9 + protoc/private/protoc_integrity.bzl | 148 ++++++ protoc/private/protoc_toolchains.bzl | 140 ++++++ protoc/private/toolchain_impl.bzl | 49 ++ protoc/toolchains.bzl | 33 ++ scala/deps.bzl | 40 +- scala/toolchains.bzl | 31 +- scala/toolchains_repo.bzl | 5 +- scala_proto/BUILD | 5 +- scala_proto/scala_proto_toolchain.bzl | 19 +- scripts/README.md | 8 + scripts/create_repository.py | 2 +- scripts/update_protoc_integrity.py | 246 +++++++++ .../io/bazel/rulesscala/scalac/reporter/BUILD | 6 +- src/protobuf/io/bazel/rules_scala/BUILD | 5 +- test_cross_build/WORKSPACE | 20 +- test_version/WORKSPACE.template | 20 +- third_party/repositories/scala_2_12.bzl | 16 +- third_party/repositories/scala_2_13.bzl | 16 +- third_party/repositories/scala_3_1.bzl | 16 +- third_party/repositories/scala_3_2.bzl | 16 +- third_party/repositories/scala_3_3.bzl | 16 +- third_party/repositories/scala_3_4.bzl | 16 +- third_party/repositories/scala_3_5.bzl | 16 +- third_party/repositories/scala_3_6.bzl | 16 +- .../test/example_external_workspace/WORKSPACE | 20 +- third_party/test/proto/WORKSPACE | 20 +- tools/bazel.rc.buildkite | 9 +- 43 files changed, 1577 insertions(+), 242 deletions(-) create mode 100644 protoc/0001-protobuf-19679-rm-protoc-dep.patch create mode 100644 protoc/BUILD create mode 100644 protoc/private/protoc_integrity.bzl create mode 100644 protoc/private/protoc_toolchains.bzl create mode 100644 protoc/private/toolchain_impl.bzl create mode 100644 protoc/toolchains.bzl create mode 100755 scripts/update_protoc_integrity.py diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 770376030..49c937177 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -29,16 +29,17 @@ tasks: # Install xmllint - sudo apt update && sudo apt install --reinstall libxml2-utils -y - "./test_rules_scala.sh" - test_rules_scala_linux_latest: - name: "./test_rules_scala (latest Bazel)" + # Switch `last_rc` to `last_green` once Bzlmod lands. + # https://github.com/bazelbuild/rules_scala/issues/1482 + test_rules_scala_linux_last_rc: + name: "./test_rules_scala (last_rc Bazel)" platform: ubuntu2004 - # Restore `bazel: latest` once Bazel 8 compatibility lands (#1625, #1652). - bazel: 7.x + bazel: last_rc shell_commands: # Install xmllint - sudo apt update && sudo apt install --reinstall libxml2-utils -y - echo "build --enable_workspace" >> .bazelrc - - "./test_rules_scala.sh || buildkite-agent annotate --style 'warning' \"Optional build with latest Bazel version failed, [see here](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) (It is not mandatory but worth checking)\"" + - "./test_rules_scala.sh || buildkite-agent annotate --style 'warning' \"Optional build with last_rc Bazel version failed, [see here](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) (It is not mandatory but worth checking)\"" test_rules_scala_macos: name: "./test_rules_scala" platform: macos diff --git a/.bazelrc b/.bazelrc index 50973fd8a..da79a94c9 100644 --- a/.bazelrc +++ b/.bazelrc @@ -1,9 +1,14 @@ +# Switch to --noenable_workspace when Bzlmod lands. +# https://github.com/bazelbuild/rules_scala/issues/1482 +common --enable_workspace --noenable_bzlmod + +# Remove once proto toolchainization becomes the default +# - https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution +# - https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit +common --incompatible_enable_proto_toolchain_resolution + build --enable_platform_specific_config #Windows needs --worker_quit_after_build due to workers not being shut down when the compiler tools need to be rebuilt (resulting in 'file in use' errors). See Bazel Issue#10498. build:windows --worker_quit_after_build --enable_runfiles - -# Switch to --noenable_workspace when Bzlmod lands. -# https://github.com/bazelbuild/rules_scala/issues/1482 -common --enable_workspace --noenable_bzlmod diff --git a/README.md b/README.md index cca4f0073..2d5ff2395 100644 --- a/README.md +++ b/README.md @@ -65,13 +65,22 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -# In `rules_scala` 7.x, `scala/deps.bzl` imports `rules_java` 7.x. This -# statement will change for `rules_scala` 8.x, which will use `rules_java` 8.x. -load( - "@rules_java//java:repositories.bzl", - "rules_java_dependencies", - "rules_java_toolchains", -) +# Only include the next two statements if using +# `--incompatible_enable_proto_toolchain_resolution`. +# See the "Using a precompiled protocol compiler" section below. +load("@platforms//host:extension.bzl", "host_platform_repo") + +# Instantiates the `@host_platform` repo to work around: +# - https://github.com/bazelbuild/bazel/issues/22558 +host_platform_repo(name = "host_platform") + +# This is optional, but register this toolchain before any others. Requires +# invoking the `scala_protoc_toolchains` repo rule, but is safe to include even +# `--incompatible_enable_proto_toolchain_resolution` is `False`. +# See the "Using a precompiled protocol compiler" section below. +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -81,13 +90,11 @@ bazel_skylib_workspace() # If you need a specific `rules_python` version, specify it here. # Otherwise you may get the version defined in the `com_google_protobuf` repo. -# We use 0.38.0 to maintain compatibility with the combination of `protobuf`, -# `rules_cc`, and related dependencies. This will change in rules_scala 7.0.0. http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -103,6 +110,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -117,6 +126,14 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +# Include this after loading `platforms`, `com_google_protobuf`, and +# `rules_proto` to enable the `//protoc` precompiled protocol compiler +# toolchains. See the "Using a precompiled protocol compiler" section below. +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +# This name can be anything, but we recommend `rules_scala_protoc_toolchains`. +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") # Stores the selected Scala version and other configuration parameters. @@ -136,16 +153,17 @@ load( "scala_toolchains", ) -# Defines a default toolchain repo for the configured Scala version that -# loads Maven deps like the Scala compiler and standard libs. On production -# projects, you may consider defining a custom toolchain to use your project's -# required dependencies instead. +# Defines a default toolchain repo for the configured Scala version that loads +# Maven deps like the Scala compiler and standard libs. Enable other builtin +# toolchains by setting their corresponding parameters to `True`. See the +# `scala_toolchains()` docstring for all builtin toolchain configuration +# options. # -# Optional builtin rules_scala toolchains may be configured by setting -# parameters as documented in the `scala_toolchains()` docstring. -scala_toolchains( - scalatest = True, -) +# On production projects, you may consider defining a custom toolchain to use +# your project's required dependencies instead. In that case, you can omit +# `scala_toolchains()` or explicitly set `scala = False` if you use it to +# instantiate other builtin toolchains. +scala_toolchains() scala_register_toolchains() ``` @@ -187,6 +205,256 @@ load( ) ``` +### Using a precompiled protocol compiler + +`rules_scala` now supports the +[`--incompatible_enable_proto_toolchain_resolution`][] flag when using +[protobuf v29 or later](#why-proto-v29). When using this flag with the +`MODULE.bazel` or `WORKSPACE` configurations below, `rules_scala` will use a +precompiled protocol compiler binary by default. + +[`--incompatible_enable_proto_toolchain_resolution`]: https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution + +__Windows builds now require using `protobuf` v29 or later with the precompiled +protocol compiler toolchain.__ See the [Windows MSVC builds of protobuf broken +by default](#protoc-msvc) section below for details. + +#### Common setup + +To set the flag in your `.bazelrc` file: + +```txt +common --incompatible_enable_proto_toolchain_resolution +``` + +In both `MODULE.bazel` and `WORKSPACE`, you must register the protocol compiler +toolchains _before_ any other toolchains. It's safe to use even when +`--incompatible_enable_proto_toolchain_resolution` is `False`. + +It is OK to call `register_toolchains` before using the `scala_protoc` extension +under Bzlmod, and before the `scala_protoc_toolchains()` repo rule under +`WORKSPACE`. + +```py +# MODULE.bazel +register_toolchains( + "@rules_scala_protoc_toolchains//...:all", + dev_dependency = True, +) + +# WORKSPACE +register_toolchains("@rules_scala_protoc_toolchains//...:all") +``` + +#### Using the `scala_protoc` module extension under Bzlmod + +The `scala_protoc` module extension instantiates the protocol compiler +toolchain under Bzlmod. It _must_ be marked as a `dev_dependency`. + +```py +# MODULE.bazel + +scala_protoc = use_extension( + "@rules_scala//scala/extensions:protoc.bzl", + "scala_protoc", + dev_dependency = True, +) +use_repo(scala_protoc, "rules_scala_protoc_toolchains") +``` + +#### Invoking the `scala_protoc_toolchains()` repo rule under `WORKSPACE` + +The `scala_protoc_toolchains` repo rule instantiates the protocol compiler +toolchain. The repo name can be anything, but we recommend +`rules_scala_protoc_toolchains`. + +```py +# WORKSPACE + +# Include this after loading `platforms`, `com_google_protobuf`, and +# `rules_proto`. +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") +``` + +#### Specifying additional `protoc` platforms + +Use the `platforms` parameter to specify additional [platforms][] if the +execution platform may differ from the host platform, as when building with +remote execution. Valid values come from the file name suffixes of +[protocolbuffers/protobuf releases][]. It's also safe to explicitly include the +host platform. + +[platforms]: https://bazel.build/extending/platforms +[protocolbuffers/protobuf releases]: https://github.com/protocolbuffers/protobuf/releases + +For example, imagine the host platform is macOS running on Apple Silicon, but +the remote execution platform is Linux running on an x86 processor. +`rules_scala` configures the `"osx-aarch_64"` platform automatically. Then in +`MODULE.bazel` you would include: + +```py +# MODULE.bazel + +scala_protoc_toolchains( + name = "rules_scala_protoc_toolchains", + platforms = ["linux-x86_64"], + dev_dependency = True, +) +``` + +In `WORKSPACE` you would include: + +```py +# WORKSPACE + +scala_protoc_toolchains( + name = "rules_scala_protoc_toolchains", + platforms = ["linux-x86_64"], +) +``` + +#### Temporary required `protobuf` patch + +At the moment, enabling protocol compiler toolchainization requires applying +[protoc/0001-protobuf-19679-rm-protoc-dep.patch][]. It is the `git diff` output +from the branch used to create protocolbuffers/protobuf#19679. Without it, a +transitive dependency on `@com_google_protobuf//:protoc` remains, causing +`protoc` to recompile even with the precompiled toolchain registered first. + +[protoc/0001-protobuf-19679-rm-protoc-dep.patch]: ./protoc/0001-protobuf-19679-rm-protoc-dep.patch + +If and when `protobuf` merges that pull request, or applies an equivalent fix, +this patch will no longer be necessary. + +#### `protobuf` patch setup under Bzlmod + +Applying the `protobuf` patch requires using [`single_version_override`][], +which also requires that the patch be a regular file in your own repo. In other +words, neither `@rules_scala//protoc:0001-protobuf-19679-rm-protoc-dep.patch` +nor an [`alias`][] to it will work. + +[`single_version_override`]: https://bazel.build/rules/lib/globals/module#single_version_override +[`alias`]: https://bazel.build/reference/be/general#alias + +Assuming you've copied the patch to a file called `protobuf.patch` in the root +package of your repository, add the following to your `MODULE.bazel`: + +```py +# MODULE.bazel + +# Required for protocol compiler toolchainization until resolution of +# protocolbuffers/protobuf#19679. +bazel_dep( + name = "protobuf", + version = "30.1", + repo_name = "com_google_protobuf", +) +single_version_override( + module_name = "protobuf", + patch_strip = 1, + patches = ["//:protobuf.patch"], + version = "30.1", +) +``` + +#### `protobuf` patch setup under `WORKSPACE` + +[`scala/deps.bzl`](./scala/deps.bzl) currently applies the `protobuf` patch to `protobuf` v30.1. + +If you need to apply the patch to a different version of `protobuf`, copy it to +your repo as described in the Bzlmod setup above. Then apply it in your own +`http_archive` call: + +```py +http_archive( + name = "com_google_protobuf", + sha256 = "1451b03faec83aed17cdc71671d1bbdfd72e54086b827f5f6fd02bf7a4041b68", + strip_prefix = "protobuf-30.1", + url = "https://github.com/protocolbuffers/protobuf/archive/refs/tags/v30.1.tar.gz", + repo_mapping = {"@com_google_absl": "@abseil-cpp"}, + patches = ["//protobuf.patch"], + patch_args = ["-p1"], +) +``` + +#### Setting up the `@host_platform` repo under `WORKSPACE` + +`WORKSPACE` must include the `host_platform_repo` snippet from +[Getting started](#getting-started) to work around bazelbuild/bazel#22558: + +```py +# WORKSPACE +load("@platforms//host:extension.bzl", "host_platform_repo") + +# Instantiates the `@host_platform` repo to work around: +# - https://github.com/bazelbuild/bazel/issues/22558 +host_platform_repo(name = "host_platform") +``` + +#### Why this requires `protobuf` v29 or later + +Using `--incompatible_enable_proto_toolchain_resolution` with versions of +`protobuf` before v29 causes build failures due to a missing internal Bazel +dependency. + +Bazel's builtin `bazel_java_proto_aspect` transitively depends on a toolchain +with a [`toolchain_type`][] of `@rules_java//java/proto:toolchain_type`. +Experimentation with `protobuf` v28.2 using both Bazel 6.5.0 and 7.5.0 led to +the following error: + +```txt +ERROR: .../external/bazel_tools/src/main/protobuf/BUILD:28:15: + in @@_builtins//:common/java/proto/java_proto_library.bzl%bazel_java_proto_aspect + aspect on proto_library rule + @@bazel_tools//src/main/protobuf:worker_protocol_proto: + +Traceback (most recent call last): + File "/virtual_builtins_bzl/common/java/proto/java_proto_library.bzl", + line 53, column 53, in _bazel_java_proto_aspect_impl + File "/virtual_builtins_bzl/common/proto/proto_common.bzl", + line 364, column 17, in _find_toolchain +Error in fail: No toolchains registered for + '@rules_java//java/proto:toolchain_type'. + +ERROR: Analysis of target + '@@bazel_tools//src/main/protobuf:worker_protocol_proto' failed +``` + +See bazelbuild/rules_scala#1710 for details of the experiment. + +For `protobuf` v29.0, protocolbuffers/protobuf#18308 added the +[`@protobuf//bazel/private/toolchains`][proto-private-tc] package and updated +`protobuf_deps()` from `@protobuf//:protobuf_deps.bzl` to register it: + +```py +native.register_toolchains("//bazel/private/toolchains:all") +``` + +[`toolchain_type`]: https://bazel.build/extending/toolchains#writing-rules-toolchains +[proto-private-tc]: https://github.com/protocolbuffers/protobuf/blob/v29.0/bazel/private/toolchains/BUILD.bazel + +protocolbuffers/protobuf#18435 then introduced +[`java_source_toolchain_bazel7`][java-proto-tc] with the required +`toolchain_type`. + +[java-proto-tc]: https://github.com/protocolbuffers/protobuf/blob/v29.0/bazel/private/toolchains/BUILD.bazel#L50-L74 + +#### More background on protocol compiler toolchainization + +- [Proto Toolchainisation Design Doc]( + https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit) + +- [bazelbuild/bazel#7095: Protobuf repo recompilation sensitivity]( + https://github.com/bazelbuild/bazel/issues/7095) + +- [bazelbuild/rules_proto#179: Implement proto toolchainisation]( + https://github.com/bazelbuild/rules_proto/issues/179) + +- [rules_proto 6.0.0 release notes mentioning Protobuf Toolchainization]( + https://github.com/bazelbuild/rules_proto/releases/tag/6.0.0) + ### Persistent workers To run with a persistent worker (much faster), add the following to @@ -284,6 +552,37 @@ https://protobuf.dev/news/v29/). For more details, see this [comment from #1710 explaining why rules_proto remains for now]( https://github.com/bazelbuild/rules_scala/pull/1710#issuecomment-2750001012). +### Using a prebuilt `@com_google_protobuf//:protoc` or C++ compiler flags + +Newer versions of `abseil-cpp`, required by newer versions of +`@com_google_protobuf//:protoc`, fail to compile under Bazel 6.5.0 and 7.6.0 by +default. [protoc will also fail to build on Windows when using +MSVC](#protoc-msvc). You will have to choose one of the following approaches to +resolve this problem. + +You may use protocol compiler toolchainization with `protobuf` v29 or later to +avoid recompiling `protoc`. You may want to enable this even if your build +doesn't break, as it saves time by avoiding frequent `protoc` recompilation. See +the [Using a precompiled protocol compiler](#protoc) section for details. + +Otherwise, if migrating to Bazel 8 isn't an immediate option, you will need to +set the following compiler flags in `.bazelrc` per bazelbuild/rules_scala#1647: + +```txt +common --enable_platform_specific_config + +common:linux --cxxopt=-std=c++17 +common:linux --host_cxxopt=-std=c++17 +common:macos --cxxopt=-std=c++17 +common:macos --host_cxxopt=-std=c++17 +common:windows --cxxopt=/std=c++17 +common:windows --host_cxxopt=/std=c++17 +``` + +Note that this example uses `common:` config settings instead of `build:`. This +seems to prevent invalidating the action cache between `bazel` runs, which +improves performance. + ## Usage with [bazel-deps](https://github.com/johnynek/bazel-deps) Bazel-deps allows you to generate bazel dependencies transitively for maven artifacts. Generally we don't want bazel-deps to fetch @@ -467,8 +766,6 @@ load( "scala_toolchains", ) -# The `scala_version` toolchain repos used by `scala_library` and `scala_binary` -# are _always_ configured, but all others are optional. scala_toolchains( scalafmt = True, scalatest = True, @@ -511,6 +808,38 @@ In `WORKSPACE`, this `register_toolchains()` call must come before calling `scala_register_toolchains()` to ensure this toolchain takes precedence. The same exact call will also work in `MODULE.bazel`. +### Disabling builtin Scala toolchains when defining custom Scala toolchains + +When [using 'setup_scala_toolchain()' with custom compiler JARs]( +docs/scala_toolchain.md#b-defining-your-own-scala_toolchain), don't use +`scala_toolchains()` if you don't need any other builtin toolchains. + +If you do need other builtin toolchains, then set `scala = False`: + +```py +# WORKSPACE +scala_toolchains( + scala = False, + # ...other toolchain parameters... +) +``` + +This avoids instantiating the default Scala toolchain and compiler JAR +repositories, and disables the corresponding Scala version check, which may +otherwise fail. This is equivalent to two ways in which the previous API avoided +the same default behavior: + +- Calling `scala_repositories(load_jar_deps = False)` would instantiate only + other `rules_scala` dependency repos (`rules_java`, `protobuf`, etc.) and + compiler source JAR repos. + +- Calling `rules_scala_setup()` directly, instead of indirectly via + `scala_repositories()`, instantiated the other dependency repositories only. + +See ["Defining your own scala_toolchain > Step 3 (optional)" from +docs/scala_toolchain.md](docs/scala_toolchain.md#step-3-optional) for futher +details. + ### Bzlmod configuration (coming soon!) The upcoming Bzlmod implementation will funnel through the `scala_toolchains()` @@ -525,14 +854,12 @@ scala_config = use_extension( "@rules_scala//scala/extensions:config.bzl", "scala_config", ) - scala_config.settings(scala_version = "2.13.16") scala_deps = use_extension( "@rules_scala//scala/extensions:deps.bzl", "scala_deps", ) - scala_deps.toolchains( scalafmt = True, scalatest = True, @@ -587,7 +914,6 @@ scala_config = use_extension( "@rules_scala//scala/extensions:config.bzl", "scala_config", ) - use_repo(scala_config, io_bazel_rules_scala_config = "rules_scala_config") ``` @@ -652,6 +978,44 @@ http_archive( ) ``` +### Windows MSVC builds of `protobuf` broken by default + +MSVC builds of recent `protobuf` versions started failing, as first noted in +bazelbuild/rules_scala#1710. On top of that, `protobuf` is planning to stop +supporting Bazel + MSVC builds per: + +- [protocolbuffers/protobuf#12947: src build on windows not working]( + https://github.com/protocolbuffers/protobuf/issues/12947) + +- [protobuf.dev News Announcements for Version 30.x:Poison MSVC + Bazel]( + https://protobuf.dev/news/v30/#poison-msvc--bazel) + +- [protocolbuffers/protobuf#20085: Breaking Change: Dropping support for + Bazel+MSVC](https://github.com/protocolbuffers/protobuf/issues/20085) + +Enable [protocol compiler toolchainization](#protoc) to fix broken Windows +builds by avoiding `@com_google_protobuf//:protoc` recompilation. + +### Minimum of `protobuf` v28 + +`rules_scala` requires at least `protobuf` v28, and at least v29 for [protocol +compiler toolchain](#protoc) support. No `ScalaPB` release supports `protobuf` +v25.6, v26, or v27. + +#### Using earlier `protobuf` versions + +If you can't update to `protobuf` v28 or later right now, build using Bazel 7 +and the following maximum versions of key dependencies. This is not officially +supported, but should work for some time. + +| Dependency | Max compatible version | Reason | +| :-: | :-: | :- | +| `protobuf` | v25.5 | Maximum version supported by `ScalaPB` 0.11.17. | +| `rules_proto` | 6.0.2 | Maximum version supporting `protobuf` v25.5 | +| `rules_java` | 7.12.4 | 8.x requires `protobuf` v27 and later. | +| `rules_cc` | 0.0.9 | 0.0.10 requires Bazel 7 to define `CcSharedLibraryHintInfo`.
0.0.13 requires at least `protobuf` v27.0. | +| `ScalaPB` | 0.11.17
(0.9.8 for Scala 2.11) | Later versions only support `protobuf` >= v28. | + ### Embedded resource paths no longer begin with `external/` [Any program compiled with an external repo asset in its 'resources' attribute @@ -774,35 +1138,43 @@ with Bazel 6.5.0 won't work at all because [Bazel 6.5.0 doesn't support https://github.com/bazelbuild/rules_scala/issues/1482#issuecomment-2515496234). At the moment, `WORKSPACE` builds mostly continue to work with Bazel 6.5.0, but -not out of the box, and may break at any time. Per bazelbuild/rules_scala#1647, -you must add the following flags to `.bazelrc`, required by the newer -`abseil-cpp` version used by `protobuf`: +not out of the box, and may break at any time. -```txt -common --enable_platform_specific_config +#### Maximum of `protobuf` v29 -common:linux --cxxopt=-std=c++17 -common:linux --host_cxxopt=-std=c++17 -common:macos --cxxopt=-std=c++17 -common:macos --host_cxxopt=-std=c++17 -common:windows --cxxopt=/std=c++17 -common:windows --host_cxxopt=/std=c++17 +You _must_ use `protobuf` v29 or earlier. `rules_scala` now uses v30 by default, +which removes `py_proto_library` and other symbols that Bazel 6.5.0 requires: + +```txt +ERROR: Traceback (most recent call last): + File ".../external/bazel_tools/src/main/protobuf/BUILD", + line 1, column 46, in + load("@com_google_protobuf//:protobuf.bzl", "py_proto_library") + +Error: file '@com_google_protobuf//:protobuf.bzl' + does not contain symbol 'py_proto_library' + +ERROR: .../src/java/io/bazel/rulesscala/worker/BUILD:3:13: + no such target '@bazel_tools//src/main/protobuf:worker_protocol_java_proto': + target 'worker_protocol_java_proto' + not declared in package 'src/main/protobuf' + defined by .../external/bazel_tools/src/main/protobuf/BUILD + (Tip: use `query "@bazel_tools//src/main/protobuf:*"` + to see all the targets in that package) + and referenced by '//src/java/io/bazel/rulesscala/worker:worker' ``` -Note that this example uses `common:` config settings instead of `build:`. This -seems to prevent invalidating the action cache between `bazel` runs, which -improves performance. +#### Configuring the protocol compiler toolchain -If you have a dependency that requires `protobuf` version before v28, use the -following maximum versions of key dependencies. Note that no `ScalaPB` release -supports `protobuf` v25.6, v26, or v27. +See [Using a prebuilt @com_google_protobuf//:protoc or C++ compiler +flags][protoc-opts] for protocol compiler configuration requirements. -| Dependency | Max compatible version | Reason | -| :-: | :-: | :- | -| `protobuf` | v25.5 | Maximum version supported by `ScalaPB` 0.11.17. | -| `rules_java` | 7.12.4 | 8.x requires `protobuf` v27 and later. | -| `rules_cc` | 0.0.9 | 0.0.10 requires Bazel 7 to define `CcSharedLibraryHintInfo`.
0.0.13 requires at least `protobuf` v27.0. | -| `ScalaPB` | 0.11.17
(0.9.8 for Scala 2.11) | Later versions only support `protobuf` >= v28. | +[protoc-opts]: #using-a-prebuilt-com_google_protobufprotoc-or-c-compiler-flags + +#### Using older versions of `protobuf` + +See [Using earlier protobuf versions](#using-earlier-protobuf-versions) for +details on using older versions of protobuf if necessary. ### `scala_proto` not supported for Scala 2.11 @@ -873,6 +1245,9 @@ scalapb_toolchain( ) ``` +Similarly, `setup_scala_proto_toolchains()` now uses a `default_gen_opts` +parameter to replace the previous boolean `enable_all_options` parameter. + ### Removal of `bind()` aliases for `twitter_scrooge` dependencies `rules_scala` 7.x removes all of the obsolete [`bind()`][] aliases under diff --git a/WORKSPACE b/WORKSPACE index ae39882d6..6f93bad43 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -5,11 +5,21 @@ load("//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load( - "@rules_java//java:repositories.bzl", - "rules_java_dependencies", - "rules_java_toolchains", -) +# Only include the next two statements if using +# `--incompatible_enable_proto_toolchain_resolution`. +load("@platforms//host:extension.bzl", "host_platform_repo") + +# Instantiates the `@host_platform` repo to work around: +# - https://github.com/bazelbuild/bazel/issues/22558 +host_platform_repo(name = "host_platform") + +# This is optional, but still safe to include even when not using +# `--incompatible_enable_proto_toolchain_resolution`. Requires invoking the +# `scala_protoc_toolchains` repo rule. Register this toolchain before any +# others. +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -19,9 +29,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -32,6 +42,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -46,6 +58,14 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +# Include this after loading `platforms`, `com_google_protobuf`, and +# `rules_proto` to enable the `//protoc` precompiled protocol compiler +# toolchains. +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +# This name can be anything, but we recommend `rules_scala_protoc_toolchains`. +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("//:scala_config.bzl", "scala_config") scala_config(enable_compiler_dependency_tracking = True) diff --git a/docs/scala_toolchain.md b/docs/scala_toolchain.md index 939a84a12..bfe1c9b3d 100644 --- a/docs/scala_toolchain.md +++ b/docs/scala_toolchain.md @@ -101,6 +101,36 @@ Register your custom toolchain: register_toolchains("//toolchains:my_scala_toolchain") ``` +#### Step 3 (optional) + +When using your own JARs for every `setup_scala_toolchain()` argument, while +using `scala_toolchains()` to instantiate other builtin toolchains, set `scala = +False`: + +```py +# WORKSPACE +scala_toolchains( + scala = False, + # ...other toolchain parameters... +) +``` + +Otherwise, `scala_toolchains()` will try to instantiate a default Scala +toolchain and its compiler JAR repositories. The build will then fail if the +configured Scala version doesn't match the `scala_version` value in the +corresponding `third_party/repositories/scala_*.bzl` file. + +If you don't specify your own jars for every `setup_scala_toolchain()` argument, +set `validate_scala_version = False` to disable the Scala version check. + +```py +# WORKSPACE +scala_toolchains( + validate_scala_version = False, + # ...other toolchain parameters... +) +``` + ## Configuration options The following attributes apply to both `scala_toolchain` and @@ -168,7 +198,7 @@ The following attributes apply to both `scala_toolchain` and

String; optional

- Enable unused dependency checking (see Unused dependency checking). + Enable unused dependency checking (see Unused dependency checking). Possible values are: off, warn and error.

diff --git a/dt_patches/test_dt_patches/WORKSPACE b/dt_patches/test_dt_patches/WORKSPACE index 9363e4857..0a232f023 100644 --- a/dt_patches/test_dt_patches/WORKSPACE +++ b/dt_patches/test_dt_patches/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(enable_compiler_dependency_tracking = True) @@ -61,11 +73,7 @@ load("@compiler_sources//:extensions.bzl", "import_compiler_source_repos") import_compiler_source_repos() -load( - "@rules_scala//scala:toolchains.bzl", - "scala_register_toolchains", - "scala_toolchains", -) +load("@rules_scala//scala:toolchains.bzl", "scala_toolchains") scala_toolchains( fetch_sources = True, @@ -73,5 +81,3 @@ scala_toolchains( ) register_toolchains(":dt_scala_toolchain") - -scala_register_toolchains() diff --git a/dt_patches/test_dt_patches_user_srcjar/WORKSPACE b/dt_patches/test_dt_patches_user_srcjar/WORKSPACE index 180f8e850..b88e6b9a0 100644 --- a/dt_patches/test_dt_patches_user_srcjar/WORKSPACE +++ b/dt_patches/test_dt_patches_user_srcjar/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(enable_compiler_dependency_tracking = True) @@ -137,11 +149,7 @@ srcjars_by_version = { }, } -load( - "@rules_scala//scala:toolchains.bzl", - "scala_register_toolchains", - "scala_toolchains", -) +load("@rules_scala//scala:toolchains.bzl", "scala_toolchains") scala_toolchains( fetch_sources = True, @@ -150,5 +158,3 @@ scala_toolchains( ) register_toolchains(":dt_scala_toolchain") - -scala_register_toolchains() diff --git a/examples/crossbuild/WORKSPACE b/examples/crossbuild/WORKSPACE index ac39245eb..b3285a23a 100644 --- a/examples/crossbuild/WORKSPACE +++ b/examples/crossbuild/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config( diff --git a/examples/overridden_artifacts/WORKSPACE b/examples/overridden_artifacts/WORKSPACE index d08cf219d..6b70cb135 100644 --- a/examples/overridden_artifacts/WORKSPACE +++ b/examples/overridden_artifacts/WORKSPACE @@ -11,11 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load( - "@rules_java//java:repositories.bzl", - "rules_java_dependencies", - "rules_java_toolchains", -) +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -25,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -38,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -52,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(scala_version = "3.3.5") diff --git a/examples/scala3/WORKSPACE b/examples/scala3/WORKSPACE index 2f5d92c4a..bead853b2 100644 --- a/examples/scala3/WORKSPACE +++ b/examples/scala3/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(scala_version = "3.6.4") diff --git a/examples/semanticdb/WORKSPACE b/examples/semanticdb/WORKSPACE index 98485cb45..f25c76b8e 100644 --- a/examples/semanticdb/WORKSPACE +++ b/examples/semanticdb/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(scala_version = "2.13.16") diff --git a/examples/testing/multi_frameworks_toolchain/WORKSPACE b/examples/testing/multi_frameworks_toolchain/WORKSPACE index c2ca66b10..38ad2cb36 100644 --- a/examples/testing/multi_frameworks_toolchain/WORKSPACE +++ b/examples/testing/multi_frameworks_toolchain/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config() diff --git a/examples/testing/scalatest_repositories/WORKSPACE b/examples/testing/scalatest_repositories/WORKSPACE index 9257c234a..53543c077 100644 --- a/examples/testing/scalatest_repositories/WORKSPACE +++ b/examples/testing/scalatest_repositories/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config() diff --git a/examples/testing/specs2_junit_repositories/WORKSPACE b/examples/testing/specs2_junit_repositories/WORKSPACE index 886f0fbd7..2e7ae21e7 100644 --- a/examples/testing/specs2_junit_repositories/WORKSPACE +++ b/examples/testing/specs2_junit_repositories/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config() diff --git a/protoc/0001-protobuf-19679-rm-protoc-dep.patch b/protoc/0001-protobuf-19679-rm-protoc-dep.patch new file mode 100644 index 000000000..86d7d1b82 --- /dev/null +++ b/protoc/0001-protobuf-19679-rm-protoc-dep.patch @@ -0,0 +1,91 @@ +diff --git a/protobuf.bzl b/protobuf.bzl +index 283c85850..ad91faba6 100644 +--- a/protobuf.bzl ++++ b/protobuf.bzl +@@ -1,7 +1,9 @@ + load("@bazel_skylib//lib:versions.bzl", "versions") + load("@rules_cc//cc:defs.bzl", "objc_library") + load("@rules_python//python:defs.bzl", "py_library") ++load("//bazel/common:proto_common.bzl", "proto_common") + load("//bazel/common:proto_info.bzl", "ProtoInfo") ++load("//bazel/private:toolchain_helpers.bzl", "toolchains") + + def _GetPath(ctx, path): + if ctx.label.workspace_root: +@@ -71,6 +73,26 @@ def _CsharpOuts(srcs): + for src in srcs + ] + ++_PROTOC_ATTRS = toolchains.if_legacy_toolchain({ ++ "_proto_compiler": attr.label( ++ cfg = "exec", ++ executable = True, ++ allow_files = True, ++ default = configuration_field("proto", "proto_compiler"), ++ ), ++}) ++_PROTOC_FRAGMENTS = ["proto"] ++_PROTOC_TOOLCHAINS = toolchains.use_toolchain(toolchains.PROTO_TOOLCHAIN) ++ ++def _protoc_files_to_run(ctx): ++ if proto_common.INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION: ++ toolchain = ctx.toolchains[toolchains.PROTO_TOOLCHAIN] ++ if not toolchain: ++ fail("Protocol compiler toolchain could not be resolved.") ++ return toolchain.proto.proto_compiler ++ else: ++ return ctx.attr._proto_compiler[DefaultInfo].files_to_run ++ + ProtoGenInfo = provider( + fields = ["srcs", "import_flags", "deps"], + ) +@@ -310,7 +332,7 @@ def _internal_gen_well_known_protos_java_impl(ctx): + args.add_all([src.path[offset:] for src in dep.direct_sources]) + + ctx.actions.run( +- executable = ctx.executable._protoc, ++ executable = _protoc_files_to_run(ctx), + inputs = descriptors, + outputs = [srcjar], + arguments = [args], +@@ -334,12 +356,9 @@ internal_gen_well_known_protos_java = rule( + "javalite": attr.bool( + default = False, + ), +- "_protoc": attr.label( +- executable = True, +- cfg = "exec", +- default = "//:protoc", +- ), +- }, ++ } | _PROTOC_ATTRS, ++ fragments = _PROTOC_FRAGMENTS, ++ toolchains = _PROTOC_TOOLCHAINS, + ) + + def _internal_gen_kt_protos(ctx): +@@ -373,7 +392,7 @@ def _internal_gen_kt_protos(ctx): + args.add_all([src.path[offset:] for src in dep.direct_sources]) + + ctx.actions.run( +- executable = ctx.executable._protoc, ++ executable = _protoc_files_to_run(ctx), + inputs = descriptors, + outputs = [srcjar], + arguments = [args], +@@ -397,12 +416,9 @@ internal_gen_kt_protos = rule( + "lite": attr.bool( + default = False, + ), +- "_protoc": attr.label( +- executable = True, +- cfg = "exec", +- default = "//:protoc", +- ), +- }, ++ } | _PROTOC_ATTRS, ++ fragments = _PROTOC_FRAGMENTS, ++ toolchains = _PROTOC_TOOLCHAINS, + ) + + def internal_objc_proto_library( diff --git a/protoc/BUILD b/protoc/BUILD new file mode 100644 index 000000000..14bec5fb2 --- /dev/null +++ b/protoc/BUILD @@ -0,0 +1,9 @@ +exports_files( + ["toolchains.bzl"], + visibility = ["//visibility:public"], +) + +toolchain_type( + name = "toolchain_type", + visibility = ["//visibility:public"], +) diff --git a/protoc/private/protoc_integrity.bzl b/protoc/private/protoc_integrity.bzl new file mode 100644 index 000000000..bfb66e4d7 --- /dev/null +++ b/protoc/private/protoc_integrity.bzl @@ -0,0 +1,148 @@ +"""Protocol compiler build and integrity metadata. + +Generated and updated by scripts/update_protoc_integrity.py. +""" + +PROTOC_RELEASES_URL = "https://github.com/protocolbuffers/protobuf/releases" +PROTOC_DOWNLOAD_URL = ( + PROTOC_RELEASES_URL + + "/download/v{version}/protoc-{version}-{platform}.zip" +) + +PROTOC_VERSIONS = [ + "30.1", + "30.0", + "29.3", + "29.2", + "29.1", + "29.0", +] + +PROTOC_BUILDS = { + "linux-aarch_64": { + "exec_compat": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + "integrity": { + "30.1": "sha256-6GbT3Ed16AMnIZFeg+P7bhq03vcZmkm0+VxNH2z0wDo=", + "30.0": "sha256-WrNHtx+4qHE5zsNqrEvQ7jrD9PKvn8aOvfVW4cCmZcY=", + "29.3": "sha256-ZCc0kUDgHwbgSecHpYcJpPIhrnOrmgQlvEoAyNDhqzI=", + "29.2": "sha256-Kc9IPi+yGCfl+sSWTjXq5HKiOOKMdi8C+xfc2T/4uJ8=", + "29.1": "sha256-H3Sj8zVd58Bma8ElYRwTUywlmPhTUh0NPmIaWwnyR5k=", + "29.0": "sha256-MF8b5a57LzlFGHCzErRcHguiaZAcg7oW2F+fnRRBs0g=", + }, + }, + "linux-ppcle_64": { + "exec_compat": [ + "@platforms//os:linux", + "@platforms//cpu:ppc64le", + ], + "integrity": { + "30.1": "sha256-QvDGG3d9y7nSMb+ty43iz/vgYl4WFH2+oOdmeFpB60Q=", + "30.0": "sha256-yWGN4tFeIPFn0yaozAed/R8ETAVeDIv7eJ428guEktM=", + "29.3": "sha256-DpiU7C45krFNGD586sFkZdam7nPh0jRpXYDm0elHAUw=", + "29.2": "sha256-uiCJWht/NKb/ql5Rw0ExbErrxMFEN3hjQITpn262f/k=", + "29.1": "sha256-B1vWZq1B60BKkjv6+pCkr6IHSiyaqLnHfERF5hbo75s=", + "29.0": "sha256-EJAnjNB1e3AsNrA+6t9KvTYCAnm7B7DfX4C9iW8+IDM=", + }, + }, + "linux-s390_64": { + "exec_compat": [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], + "integrity": { + "30.1": "sha256-orgjHBFZsAushwtu1OFs6dXSsImg4OkW/fPXCJNiC7Y=", + "30.0": "sha256-eYcU19uRFfTgQr6G3R7quXNG31Qxof4QFhXJwcHhZM8=", + "29.3": "sha256-Y3hX/bqwsTNL2ysIcz8L5JaF5pMBG2EEgJSRrGL71NU=", + "29.2": "sha256-LwpVmdprgpMqCNQz+nkTux1oWIl8zTxrwyhvAYt2xOA=", + "29.1": "sha256-J5fNVlyn/7/ALaVsygE7/0+X5oMkizuZ3PfUuxf27GE=", + "29.0": "sha256-LhXZqwaFbCXKbeYi4RR4XZGHHb25HCwQPrYouH8m3Ew=", + }, + }, + "linux-x86_32": { + "exec_compat": [ + "@platforms//os:linux", + "@platforms//cpu:x86_32", + ], + "integrity": { + "30.1": "sha256-L5oRdIK9EripNAwhJ9iHefTdo7ZLTacJJ43EmaQ6c4Y=", + "30.0": "sha256-OtMQ/NwrS/nzD1QrWbQDsrebqCss9DGb6E3poCZFrnY=", + "29.3": "sha256-VGzx5pHOc/ZuKZzCLN1aF8YaEf5LbV34UHUHg4nx/Ck=", + "29.2": "sha256-FU+NR+6YO8bxoa4tsUl+53E8Qt7vY1EXDxhmG/vNouw=", + "29.1": "sha256-nd/EAbEqC4dHHg4POg0MKpByRj8EFDUUnpSfiiCCu+s=", + "29.0": "sha256-tKyBCfKrSGLV5WuH4/cVMPug46YeyLnZQOKUdskGAQE=", + }, + }, + "linux-x86_64": { + "exec_compat": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + "integrity": { + "30.1": "sha256-VTfhWrDA5hD4CVc5SNPsfW7zh6B5keHDYaKg6MrZg+U=", + "30.0": "sha256-L7vBgYRj1+bZPBmo3qg55mPKX4V5pS73jHaIGIM1+mw=", + "29.3": "sha256-PoZmIMW+J2ZPPS+i1la18+CbUVK0Lxvtv0J7Mz6QAho=", + "29.2": "sha256-Uunn7OVcfjDn6LvSVLSyG0CKUwm8qCZ2PHEktpahMuk=", + "29.1": "sha256-AMg/6XIthelsgblBsp8Xp0SzO0zmbg8YAJ/Yk33iLGA=", + "29.0": "sha256-PFEGWvO5pgbZ4Yob9igUNzT/S55pcl1kWYV0MLp6eN8=", + }, + }, + "osx-aarch_64": { + "exec_compat": [ + "@platforms//os:osx", + "@platforms//cpu:aarch64", + ], + "integrity": { + "30.1": "sha256-A0Z8/ZZ94SphQGt0c+gCBNOuOPMPgoVTGBhtaWI347k=", + "30.0": "sha256-frW1HTe6xBC6cO+RxAT5Cx+ry4I3Ev9lZYLTSsyHynQ=", + "29.3": "sha256-K4o0A80Jf5XzumVuFLdscytrJtfxgzMLEeNu8rwCh2U=", + "29.2": "sha256-DhU6ONbaGVlMmA5/fNPqDd1SydoQaMA8DYUzNp+/6yA=", + "29.1": "sha256-uP1ZdpJhmKfE6lxutL94lZ1frtJ7/GGCVMqhBD93BEU=", + "29.0": "sha256-srWfA7AwyKdIYj1oKotbycwJnkvP0GuJZM6J7AZbMQM=", + }, + }, + "osx-x86_64": { + "exec_compat": [ + "@platforms//os:osx", + "@platforms//cpu:x86_64", + ], + "integrity": { + "30.1": "sha256-pK7v0vWczOWc+gGon+WK20C7kBD0Ot/KPE/uf9N+wsU=", + "30.0": "sha256-lr86X77v1X19wMIKLHuz8iathLeeW1CThoJDIgF7lBc=", + "29.3": "sha256-mniANtj5hU97A8MF30d3zw5U5bCB4lvxUlLah+DpCHU=", + "29.2": "sha256-uivZg7XwbsONZjtgKISll96jmQpDgD1+FT7Y98VCaeE=", + "29.1": "sha256-2wK0uG3k1MztPqmTQ0faKNyV5/OIY//EzjzCYoMCjaY=", + "29.0": "sha256-56HP/ILiHapngzARRJxw3f8eujsRWTQ4fm6BQe+rCS8=", + }, + }, + "win32": { + "exec_compat": [ + "@platforms//os:windows", + "@platforms//cpu:x86_32", + ], + "integrity": { + "30.1": "sha256-nfRHsT+ijJqqwRdPWb30Al9KG9tuS2s/HOs+5o8HEQE=", + "30.0": "sha256-g9c7ejnygRxhB/azkoWsWw8PRmNh1Gg96j796WNkMuY=", + "29.3": "sha256-x8gCjBxNgBxTYCkg8shokgVAhr2WW2sjpLqV0hHcsdQ=", + "29.2": "sha256-73CfcaUbOompsm3meCBc7zyV4h0IGLFn+/WwOackr9E=", + "29.1": "sha256-EQXg+mRFnwsa9e5NWHfauGTy4Q2K6wRhjytpxsOm7QM=", + "29.0": "sha256-154nzOTEAXRUERc8XraBFsPiBACxzEwt0stHeneUGP4=", + }, + }, + "win64": { + "exec_compat": [ + "@platforms//os:windows", + "@platforms//cpu:x86_64", + ], + "integrity": { + "30.1": "sha256-d/TgIs6eiwh8uJP1P15DNzSULRJeTNL+y/gwrHdgBFw=", + "30.0": "sha256-yEww2siMaLQKLkfFtsdi3B7Amu6zLB0efs8l153PnNo=", + "29.3": "sha256-V+pZ6fVRrY1x/6qbXPvgyh9Ocglyodt+wtEqtEv/k4M=", + "29.2": "sha256-Weph77JLnYohQXHiyj/sVcPxUX7/BnZWyHXYoc0Gzk8=", + "29.1": "sha256-fqSCJYV//BIkWIwzXCsa+deKGK+dV8BSjMoxk+M26c4=", + "29.0": "sha256-0DuSGYWLikyogGO3i/Clzec7UYCLkwxLZvBuhILDq+Y=", + }, + }, +} diff --git a/protoc/private/protoc_toolchains.bzl b/protoc/private/protoc_toolchains.bzl new file mode 100644 index 000000000..ab5e800c3 --- /dev/null +++ b/protoc/private/protoc_toolchains.bzl @@ -0,0 +1,140 @@ +"""Precompiled protoc toolchains repo rule + +Provides precompiled protocol compiler toolchains. +""" + +load(":private/protoc_integrity.bzl", "PROTOC_BUILDS", "PROTOC_DOWNLOAD_URL") +load( + ":private/toolchain_impl.bzl", + "PROTOC_TOOLCHAIN_ENABLED", + "PROTOC_TOOLCHAIN_TYPE", +) +load("@com_google_protobuf//:protobuf_version.bzl", "PROTOC_VERSION") +load("@platforms//host:constraints.bzl", "HOST_CONSTRAINTS") + +def _default_platform(): + host_platform = sorted(HOST_CONSTRAINTS) + for platform, info in PROTOC_BUILDS.items(): + if sorted(info["exec_compat"]) == host_platform: + return platform + fail( + "no protoc build found for host platform with constraints: " + + HOST_CONSTRAINTS, + ) + +def _platform_build(platform): + if platform not in PROTOC_BUILDS: + fail("no protoc build found for platform: " + platform) + + protoc_build = PROTOC_BUILDS[platform] + + if PROTOC_VERSION not in protoc_build["integrity"]: + fail( + "no protoc %s build found for platform: %s" % + (PROTOC_VERSION, platform), + ) + return protoc_build + +def _download_build(repository_ctx, platform, protoc_build): + repository_ctx.download_and_extract( + url = PROTOC_DOWNLOAD_URL.format( + platform = platform, + version = PROTOC_VERSION, + ), + output = platform, + integrity = protoc_build["integrity"][PROTOC_VERSION], + ) + +def _emit_platform_entry(platform, protoc_build): + return ' "{platform}": [\n{specs}\n ],'.format( + platform = platform, + specs = "\n".join([ + ' "%s",' % s + for s in protoc_build["exec_compat"] + ]), + ) + +def _generate_protoc_platforms(repository_ctx, builds): + content = ["PROTOC_PLATFORMS = {"] + content.extend([_emit_platform_entry(p, b) for p, b in builds]) + content.append("}\n") + + repository_ctx.file( + "platforms.bzl", + content = "\n".join(content), + executable = False, + ) + +def _scala_protoc_toolchains_impl(repository_ctx): + builds = [] + build_file_content = "" + + if PROTOC_TOOLCHAIN_ENABLED: + platforms = [_default_platform()] + platforms += repository_ctx.attr.platforms + builds = {p: _platform_build(p) for p in platforms}.items() + build_file_content = _PROTOC_TOOLCHAIN_BUILD + + for platform, build in builds: + _download_build(repository_ctx, platform, build) + + _generate_protoc_platforms(repository_ctx, builds) + + # Always generate a root package, even if it's empty, to ensure + # `register_toolchains("@rules_scala_protoc_toolchains//:all")` works. + repository_ctx.file( + "BUILD", + content = build_file_content, + executable = False, + ) + +scala_protoc_toolchains = repository_rule( + implementation = _scala_protoc_toolchains_impl, + doc = ( + "Prepares the precompiled protoc toolchain used by " + + "`--incompatible_enable_proto_toolchain_resolution`" + ), + attrs = { + "platforms": attr.string_list( + doc = ( + "Operating system and architecture identifiers for " + + "precompiled protocol compiler releases, taken from " + + "protocolbuffers/protobuf releases file name suffixes. If " + + "unspecified, will use the identifier matching the " + + "`HOST_CONSTRAINTS` from `@platforms//host:constraints.bzl`." + + " Only takes effect when" + + "`--incompatible_enable_proto_toolchain_resolution` is " + + "`True`." + ), + ), + }, +) + +_PROTOC_TOOLCHAIN_BUILD = """load(":platforms.bzl", "PROTOC_PLATFORMS") +load( + "@com_google_protobuf//bazel/toolchains:proto_lang_toolchain.bzl", + "proto_lang_toolchain", +) +load( + "@com_google_protobuf//bazel/toolchains:proto_toolchain.bzl", + "proto_toolchain", +) + +proto_lang_toolchain( + name = "protoc_scala_toolchain", + command_line = "unused-because-we-pass-protoc-to-scalapb", + toolchain_type = "{protoc_toolchain_type}", + visibility = ["//visibility:public"], +) + +[ + proto_toolchain( + name = platform, + exec_compatible_with = specs, + proto_compiler = ":%s/bin/protoc%s" % ( + platform, ".exe" if platform.startswith("win") else "" + ), + ) + for platform, specs in PROTOC_PLATFORMS.items() +] +""".format(protoc_toolchain_type = PROTOC_TOOLCHAIN_TYPE) diff --git a/protoc/private/toolchain_impl.bzl b/protoc/private/toolchain_impl.bzl new file mode 100644 index 000000000..02e544b46 --- /dev/null +++ b/protoc/private/toolchain_impl.bzl @@ -0,0 +1,49 @@ +"""Precompiled protoc toolchain utilities + +Used to implement `--incompatible_enable_proto_toolchain_resolution` +compatibility. Inspired by: +https://github.com/protocolbuffers/protobuf/pull/19679 +""" + +load("@rules_proto//proto:proto_common.bzl", "toolchains") + +PROTOC_TOOLCHAIN_ENABLED = not bool(toolchains.if_legacy_toolchain(True)) +PROTOC_TOOLCHAIN_TYPE = Label("//protoc:toolchain_type") + +PROTOC_ATTR = toolchains.if_legacy_toolchain({ + "protoc": attr.label( + allow_files = True, + cfg = "exec", + default = Label("@com_google_protobuf//:protoc"), + executable = True, + ), +}) +PROTOC_TOOLCHAINS = toolchains.use_toolchain(PROTOC_TOOLCHAIN_TYPE) + +def protoc_executable(ctx): + """Returns `protoc` executable path for rules using `PROTOC_*` symbols. + + Requires that rules use `PROTOC_ATTR` and `PROTOC_TOOLCHAINS` as follows: + + ```py + attrs = { + # ...other attrs... + } | PROTOC_ATTR, + toolchains = PROTOC_TOOLCHAINS, + ``` + + Args: + ctx: the rule context object + + Returns: + the precompiled `protoc` executable path from the precompiled toolchain + if `--incompatible_enable_proto_toolchain_resolution` is enabled, + or the path to the `protoc` compiled by `protobuf` otherwise + """ + if not PROTOC_TOOLCHAIN_ENABLED: + return ctx.attr.protoc[DefaultInfo].files_to_run.executable + + toolchain = ctx.toolchains[PROTOC_TOOLCHAIN_TYPE] + if not toolchain: + fail("Couldn't resolve protocol compiler for " + PROTOC_TOOLCHAIN_TYPE) + return toolchain.proto.proto_compiler.executable diff --git a/protoc/toolchains.bzl b/protoc/toolchains.bzl new file mode 100644 index 000000000..5429b6252 --- /dev/null +++ b/protoc/toolchains.bzl @@ -0,0 +1,33 @@ +"""Precompiled protocol compiler toolchains repository rule. + +To use this under `WORKSPACE`: + +```py +# WORKSPACE + +# Register this toolchain before any others. +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@platforms//host:extension.bzl", "host_platform_repo") + +# Instantiates the `@host_platform` repo to work around: +# - https://github.com/bazelbuild/bazel/issues/22558 +host_platform_repo(name = "host_platform") + +# ...load `com_google_protobuf`, `rules_proto`, etc... + +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +# The name can be anything, but we recommend `rules_scala_protoc_toolchains`. +# Only include `platforms` if you need additional platforms other than the +# automatically detected host platform. +scala_protoc_toolchains( + name = "rules_scala_protoc_toolchains", + platforms = ["linux-x86_64"], +) +``` +""" + +load(":private/protoc_toolchains.bzl", _toolchains = "scala_protoc_toolchains") + +scala_protoc_toolchains = _toolchains diff --git a/scala/deps.bzl b/scala/deps.bzl index 4843b4743..85fbe65e0 100644 --- a/scala/deps.bzl +++ b/scala/deps.bzl @@ -15,44 +15,56 @@ def rules_scala_dependencies(): ], ) + maybe( + http_archive, + name = "platforms", + urls = [ + "https://mirror.bazel.build/github.com/bazelbuild/platforms/releases/download/0.0.11/platforms-0.0.11.tar.gz", + "https://github.com/bazelbuild/platforms/releases/download/0.0.11/platforms-0.0.11.tar.gz", + ], + sha256 = "29742e87275809b5e598dc2f04d86960cc7a55b3067d97221c9abbc9926bff0f", + ) + maybe( http_archive, name = "rules_cc", - urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.9/rules_cc-0.0.9.tar.gz"], - sha256 = "2037875b9a4456dce4a79d112a8ae885bbc4aad968e6587dca6e64f3a0900cdf", - strip_prefix = "rules_cc-0.0.9", + urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.1/rules_cc-0.1.1.tar.gz"], + sha256 = "712d77868b3152dd618c4d64faaddefcc5965f90f5de6e6dd1d5ddcd0be82d42", + strip_prefix = "rules_cc-0.1.1", ) maybe( http_archive, name = "abseil-cpp", - sha256 = "16242f394245627e508ec6bb296b433c90f8d914f73b9c026fddb905e27276e8", - strip_prefix = "abseil-cpp-20250127.0", - url = "https://github.com/abseil/abseil-cpp/archive/refs/tags/20250127.0.tar.gz", + sha256 = "b396401fd29e2e679cace77867481d388c807671dc2acc602a0259eeb79b7811", + strip_prefix = "abseil-cpp-20250127.1", + url = "https://github.com/abseil/abseil-cpp/archive/refs/tags/20250127.1.tar.gz", ) maybe( http_archive, name = "rules_java", urls = [ - "https://github.com/bazelbuild/rules_java/releases/download/7.12.4/rules_java-7.12.4.tar.gz", + "https://github.com/bazelbuild/rules_java/releases/download/8.11.0/rules_java-8.11.0.tar.gz", ], - sha256 = "302bcd9592377bf9befc8e41aa97ec02df12813d47af9979e4764f3ffdcc5da8", + sha256 = "d31b6c69e479ffa45460b64dc9c7792a431cac721ef8d5219fc9f603fa2ff877", ) maybe( http_archive, name = "com_google_protobuf", - sha256 = "75be42bd736f4df6d702a0e4e4d30de9ee40eac024c4b845d17ae4cc831fe4ae", - strip_prefix = "protobuf-21.7", - url = "https://github.com/protocolbuffers/protobuf/archive/refs/tags/v21.7.tar.gz", + sha256 = "1451b03faec83aed17cdc71671d1bbdfd72e54086b827f5f6fd02bf7a4041b68", + strip_prefix = "protobuf-30.1", + url = "https://github.com/protocolbuffers/protobuf/archive/refs/tags/v30.1.tar.gz", repo_mapping = {"@com_google_absl": "@abseil-cpp"}, + patches = [Label("//protoc:0001-protobuf-19679-rm-protoc-dep.patch")], + patch_args = ["-p1"], ) maybe( http_archive, name = "rules_proto", - sha256 = "6fb6767d1bef535310547e03247f7518b03487740c11b6c6adb7952033fe1295", - strip_prefix = "rules_proto-6.0.2", - url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.2/rules_proto-6.0.2.tar.gz", + sha256 = "14a225870ab4e91869652cfd69ef2028277fc1dc4910d65d353b62d6e0ae21f4", + strip_prefix = "rules_proto-7.1.0", + url = "https://github.com/bazelbuild/rules_proto/releases/download/7.1.0/rules_proto-7.1.0.tar.gz", ) diff --git a/scala/toolchains.bzl b/scala/toolchains.bzl index 8d041a871..524789bbd 100644 --- a/scala/toolchains.bzl +++ b/scala/toolchains.bzl @@ -32,10 +32,10 @@ def _get_unknown_entries(entries, allowed_entries): def scala_toolchains( maven_servers = default_maven_server_urls(), overridden_artifacts = {}, - load_scala_toolchain_dependencies = True, fetch_sources = False, validate_scala_version = True, scala_compiler_srcjars = {}, + scala = True, scalatest = False, junit = False, specs2 = False, @@ -59,24 +59,30 @@ def scala_toolchains( Args: maven_servers: Maven servers used to fetch dependency jar files - overridden_artifacts: specific dependency jar files to use instead of - those from `maven_servers`, in the format: + overridden_artifacts: artifacts overriding the defaults for the + configured Scala version, in the format: ```starlark "repo_name": { "artifact": "", "sha256": "", "deps": [ - "repository_names_of_dependencies", + "repository_labels_of_dependencies", ], } ``` + The default artifacts are defined by the + `third_party/repositories/scala_*.bzl` file matching the Scala + version. fetch_sources: whether to download dependency source jars - validate_scala_version: whether to check if the configured Scala version - matches the default version supported by rules_scala + validate_scala_version: Whether to check if the configured Scala + versions matches the default versions supported by rules_scala. Only + takes effect if `scala` is `True`. scala_compiler_srcjars: optional dictionary of Scala version string to compiler srcjar metadata dictionaries containing: - exactly one "label", "url", or "urls" key - optional "integrity" or "sha256" keys + scala: whether to instantiate default Scala toolchains for configured + Scala versions scalatest: whether to instantiate the ScalaTest toolchain junit: whether to instantiate the JUnit toolchain specs2: whether to instantiate the Specs2 JUnit toolchain @@ -142,10 +148,13 @@ def scala_toolchains( }) for scala_version in SCALA_VERSIONS: - version_specific_artifact_ids = { - id: fetch_sources - for id in scala_version_artifact_ids(scala_version) - } + version_specific_artifact_ids = {} + + if scala: + version_specific_artifact_ids.update({ + id: fetch_sources + for id in scala_version_artifact_ids(scala_version) + }) if scala_proto: version_specific_artifact_ids.update({ @@ -170,7 +179,7 @@ def scala_toolchains( fetch_sources_by_id = all_artifacts, # Note the internal macro parameter misspells "overriden". overriden_artifacts = overridden_artifacts, - validate_scala_version = validate_scala_version, + validate_scala_version = (scala and validate_scala_version), ) scala_toolchains_repo( diff --git a/scala/toolchains_repo.bzl b/scala/toolchains_repo.bzl index 3fd256a56..4d00ccf1d 100644 --- a/scala/toolchains_repo.bzl +++ b/scala/toolchains_repo.bzl @@ -71,8 +71,9 @@ def _scala_toolchains_repo_impl(repository_ctx): if repo_attr.scalafmt: toolchains["scalafmt"] = _SCALAFMT_TOOLCHAIN_BUILD - if len(toolchains) == 0: - fail("no toolchains specified") + # Generate a root package so that the `register_toolchains` call in + # `MODULE.bazel` always succeeds. + repository_ctx.file("BUILD", executable = False) for pkg, build in toolchains.items(): repository_ctx.file( diff --git a/scala_proto/BUILD b/scala_proto/BUILD index e9f6dd849..1af83d89a 100644 --- a/scala_proto/BUILD +++ b/scala_proto/BUILD @@ -1,4 +1,7 @@ -load("//scala_proto/private:toolchain_deps.bzl", "export_scalapb_toolchain_deps") +load( + "//scala_proto/private:toolchain_deps.bzl", + "export_scalapb_toolchain_deps", +) toolchain_type( name = "toolchain_type", diff --git a/scala_proto/scala_proto_toolchain.bzl b/scala_proto/scala_proto_toolchain.bzl index 51bcb6e55..380a6310d 100644 --- a/scala_proto/scala_proto_toolchain.bzl +++ b/scala_proto/scala_proto_toolchain.bzl @@ -1,8 +1,15 @@ +load( + "//protoc:private/toolchain_impl.bzl", + "PROTOC_ATTR", + "PROTOC_TOOLCHAINS", + "protoc_executable", +) load("//scala:providers.bzl", "DepsInfo") load( "//scala_proto/default:default_deps.bzl", _scala_proto_deps_providers = "scala_proto_deps_providers", ) +load("@rules_java//java/common:java_info.bzl", "JavaInfo") def _generators_jars(ctx): generator_deps = ctx.attr.extra_generator_dependencies + [ @@ -19,7 +26,7 @@ def _ignored_proto_targets_by_label(ctx): def _worker_flags(ctx, generators, jars): env = dict( {"GEN_" + k: v for k, v in generators.items()}, - PROTOC = ctx.executable.protoc.path, + PROTOC = protoc_executable(ctx).path, JARS = ctx.configuration.host_path_separator.join( [f.path for f in jars.to_list()], ), @@ -36,7 +43,7 @@ def _scala_proto_toolchain_impl(ctx): generators_opts = ctx.attr.generators_opts, compile_dep_ids = compile_dep_ids, blacklisted_protos = _ignored_proto_targets_by_label(ctx), - protoc = ctx.executable.protoc, + protoc = protoc_executable(ctx), scalac = ctx.attr.scalac.files_to_run, worker = ctx.attr.code_generator.files_to_run, worker_flags = _worker_flags(ctx, generators, generators_jars), @@ -68,11 +75,6 @@ scala_proto_toolchain = rule( default = Label("//src/java/io/bazel/rulesscala/scalac"), allow_files = True, ), - "protoc": attr.label( - executable = True, - cfg = "exec", - default = Label("@com_google_protobuf//:protoc"), - ), "stamp_by_convention": attr.bool( default = False, doc = """ @@ -100,7 +102,8 @@ scala_proto_toolchain = rule( executable = False, cfg = "exec", ), - }, + } | PROTOC_ATTR, + toolchains = PROTOC_TOOLCHAINS, ) def scalapb_toolchain(name, opts = [], **kwargs): diff --git a/scripts/README.md b/scripts/README.md index e126f3741..5f44e197f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -116,3 +116,11 @@ The [bazelisk](https://github.com/bazelbuild/bazelisk) wrapper for Bazel uses `.bazelversion` files select a Bazel version. While `USE_BAZEL_VERSION` can also override the Bazel version, keeping the `.bazelversion` files synchronized helps avoid suprises when not using `USE_BAZEL_VERSION`. + +## [`update_protoc_integrity.py`](./update_protoc_integrity.py) + +Updates `protoc/private/protoc_integrity.bzl`. + +Upon a new release of +[`protocolbuffers/protobuf`](https://github.com/protocolbuffers/protobuf/releases) +add the new version to the `PROTOC_VERSIONS` at the top of this file and run it. diff --git a/scripts/create_repository.py b/scripts/create_repository.py index 5649871e2..4472afcd2 100755 --- a/scripts/create_repository.py +++ b/scripts/create_repository.py @@ -33,7 +33,7 @@ KIND_PROJECTOR_VERSION = "0.13.3" PROTOBUF_JAVA_VERSION = "4.30.1" JLINE_VERSION = '3.29.0' -SCALAPB_VERSION = '0.11.17' +SCALAPB_VERSION = '1.0.0-alpha.1' PROTOC_BRIDGE_VERSION = '0.9.8' GRPC_VERSION = '1.71.0' GRPC_COMMON_PROTOS_VERSION = '2.54.1' diff --git a/scripts/update_protoc_integrity.py b/scripts/update_protoc_integrity.py new file mode 100755 index 000000000..68bf41a70 --- /dev/null +++ b/scripts/update_protoc_integrity.py @@ -0,0 +1,246 @@ +#!/usr/bin/env python3 +"""Updates `protoc/private/protoc_integrity.bzl`. + +`protoc_integrity.bzl` contains the mapping from supported precompiled `protoc` +platforms to: + +- `exec_compatible_with` properties based on `@platforms` +- `integrity` strings for each of the supported `PROTOC_VERSIONS` + +Only computes integrity information for a `protoc` distribution if it doesn't +already exist in the integrity file. + +This borrows some code from `scripts/create_repository.py` that could probably +be extracted into a common module. Specifically, `emit_protoc_integrity_file()` +borrows heavily from `ArtifactUpdater.write_to_file()`. +""" + +from base64 import b64encode +from pathlib import Path + +import argparse +import ast +import hashlib +import json +import re +import urllib.request +import sys + +PROTOC_VERSIONS = [ + "30.1", + "30.0", + "29.3", + "29.2", + "29.1", + "29.0", +] + +PROTOC_RELEASES_URL = "https://github.com/protocolbuffers/protobuf/releases" +PROTOC_DOWNLOAD_SUFFIX = "/download/v{version}/protoc-{version}-{platform}.zip" +PROTOC_DOWNLOAD_URL = PROTOC_RELEASES_URL + PROTOC_DOWNLOAD_SUFFIX + +PROTOC_BUILDS = { + "linux-aarch_64": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + "linux-ppcle_64": [ + "@platforms//os:linux", + "@platforms//cpu:ppc64le", + ], + "linux-s390_64": [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], + "linux-x86_32": [ + "@platforms//os:linux", + "@platforms//cpu:x86_32" + ], + "linux-x86_64": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64" + ], + "osx-aarch_64": [ + "@platforms//os:osx", + "@platforms//cpu:aarch64", + ], + "osx-x86_64": [ + "@platforms//os:osx", + "@platforms//cpu:x86_64" + ], + "win32": [ + "@platforms//os:windows", + "@platforms//cpu:x86_32" + ], + "win64": [ + "@platforms//os:windows", + "@platforms//cpu:x86_64" + ], +} + +THIS_FILE = Path(__file__) +REPO_ROOT = THIS_FILE.parent.parent +INTEGRITY_FILE = REPO_ROOT / 'protoc/private/protoc_integrity.bzl' +INTEGRITY_FILE_HEADER = f'''"""Protocol compiler build and integrity metadata. + +Generated and updated by {THIS_FILE.relative_to(REPO_ROOT)}. +""" + +PROTOC_RELEASES_URL = "{PROTOC_RELEASES_URL}" +PROTOC_DOWNLOAD_URL = ( + PROTOC_RELEASES_URL + + "{PROTOC_DOWNLOAD_SUFFIX}" +) + +''' + + +class UpdateProtocIntegrityError(Exception): + """Errors raised explicitly by this module.""" + + +def get_protoc_integrity(platform, version): + """Emits the integrity string for the specified `protoc` distribution. + + This will download the distribution specified by applying `platform` and + `version` to `PROTOC_DOWNLOAD_URL`. + + Args: + platform: a platform key from `PROTOC_BUILDS` + version: a valid `protobuf` version specifier + + Returns: + a string starting with `sha256-` and ending with the base 64 encoded + sha256 checksum of the `protoc` distribution file + + Raises: + `UpdateProtocIntegrityError` if downloading or checksumming fails + """ + url = PROTOC_DOWNLOAD_URL.format(version = version, platform = platform) + print(f'Updating protoc {version} for {platform}:\n {url}') + + try: + with urllib.request.urlopen(url) as data: + body = data.read() + + sha256 = hashlib.sha256(body).digest() + return f'sha256-{b64encode(sha256).decode('utf-8')}' + + except Exception as err: + msg = f'while processing {url}: {err}' + raise UpdateProtocIntegrityError(msg) from err + + +def add_build_data(platform, exec_compat, existing_build): + """Adds `protoc` integrity data to `existing_build` for new protoc versions. + + Args: + platform: a platform key from `PROTOC_BUILDS` + exec_compat: compatibility specifier values from `PROTOC_BUILDS` + existing_build: an existing `PROTOC_BUILDS` output value for `platform`, + or `{}` if it doesn't yet exist + + Returns: + a new dictionary to emit as a `PROTOC_BUILDS` entry in the output file + """ + integrity = dict(existing_build.get("integrity", {})) + + for version in PROTOC_VERSIONS: + if version not in integrity: + integrity[version] = get_protoc_integrity(platform, version) + + return { + "exec_compat": exec_compat, + "integrity": dict(sorted(integrity.items(), reverse=True)), + } + + +def stringify_object(data): + """Pretty prints `data` as a Starlark object to emit into the output file. + + Args: + data: a Python list or dict + + Returns: + a pretty-printed string version of `data` to represent a valid Starlark + object in the output file + """ + result = ( + json.dumps(data, indent=4) + .replace('true', 'True') + .replace('false', 'False') + ) + # Add trailing commas. + return re.sub(r'([]}"])\n', r'\1,\n', result) + '\n' + + +def emit_protoc_integrity_file(output_file, integrity_data): + """Writes the updated `protoc` integrity data to the `output_file`. + + Args: + output_file: path to the updated `protoc` integrity file + integrity_data: `protoc` integrity data to emit into `output_file` + """ + with output_file.open('w', encoding = 'utf-8') as data: + data.write(INTEGRITY_FILE_HEADER) + data.write("PROTOC_VERSIONS = ") + data.write(stringify_object(PROTOC_VERSIONS)) + data.write("\nPROTOC_BUILDS = ") + data.write(stringify_object(dict(sorted(integrity_data.items())))) + + +def load_existing_data(existing_file): + """Loads existing `protoc` integrity data from `existing_file`. + + This enables the script to avoid redownloading `protoc` distribution files + when the integrity information already exists. + + Args: + existing_file: path to the existing integrity file + + Returns: + the existing `PROTOC_BUILDS` integrity data from `existing_file`, + or `{}` if the file does not exist + """ + if not existing_file.exists(): + return {} + + with existing_file.open('r', encoding='utf-8') as f: + data = f.read() + + marker = 'PROTOC_BUILDS = ' + start = data.find(marker) + + if start == -1: + msg = f'"{marker}" not found in {existing_file}' + raise UpdateProtocIntegrityError(msg) + + return ast.literal_eval(data[start + len(marker):]) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description = "Updates precompiled `protoc` distribution information.", + ) + + parser.add_argument( + '--integrity_file', + type=str, + default=str(INTEGRITY_FILE), + help=f'`protoc` integrity file path (default: {INTEGRITY_FILE})', + ) + + args = parser.parse_args() + integrity_file = Path(args.integrity_file) + + try: + existing_data = load_existing_data(integrity_file) + updated_data = { + k: add_build_data(k, v, existing_data.get(k, {})) + for k, v in PROTOC_BUILDS.items() + } + emit_protoc_integrity_file(integrity_file, updated_data) + + except UpdateProtocIntegrityError as err: + print(f'Failed to update {integrity_file}: {err}', file=sys.stderr) + sys.exit(1) diff --git a/src/java/io/bazel/rulesscala/scalac/reporter/BUILD b/src/java/io/bazel/rulesscala/scalac/reporter/BUILD index 210b6c5d4..814a183d9 100644 --- a/src/java/io/bazel/rulesscala/scalac/reporter/BUILD +++ b/src/java/io/bazel/rulesscala/scalac/reporter/BUILD @@ -1,4 +1,8 @@ -load("@rules_java//java:defs.bzl", "java_library", "java_proto_library") +load( + "@com_google_protobuf//bazel:java_proto_library.bzl", + "java_proto_library", +) +load("@rules_java//java:java_library.bzl", "java_library") load("@rules_proto//proto:defs.bzl", "proto_library") load("//scala:scala_cross_version_select.bzl", "select_for_scala_version") diff --git a/src/protobuf/io/bazel/rules_scala/BUILD b/src/protobuf/io/bazel/rules_scala/BUILD index 438db80a9..4bd043883 100644 --- a/src/protobuf/io/bazel/rules_scala/BUILD +++ b/src/protobuf/io/bazel/rules_scala/BUILD @@ -1,5 +1,8 @@ +load( + "@com_google_protobuf//bazel:java_proto_library.bzl", + "java_proto_library", +) load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_java//java:defs.bzl", "java_proto_library") proto_library( name = "diagnostics_proto", diff --git a/test_cross_build/WORKSPACE b/test_cross_build/WORKSPACE index a5c273b2e..fe0feabae 100644 --- a/test_cross_build/WORKSPACE +++ b/test_cross_build/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config( diff --git a/test_version/WORKSPACE.template b/test_version/WORKSPACE.template index e367fc3aa..a39fa8609 100644 --- a/test_version/WORKSPACE.template +++ b/test_version/WORKSPACE.template @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(enable_compiler_dependency_tracking = True) diff --git a/third_party/repositories/scala_2_12.bzl b/third_party/repositories/scala_2_12.bzl index dac30629b..704040773 100644 --- a/third_party/repositories/scala_2_12.bzl +++ b/third_party/repositories/scala_2_12.bzl @@ -819,8 +819,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_2.12:0.11.17", - "sha256": "a9dc6cc0dbe6ff53a7c914433d5a19711018217b432b385c97778cd4050210d0", + "artifact": "com.thesamet.scalapb:compilerplugin_2.12:1.0.0-alpha.1", + "sha256": "8c771e34186ed2e382a3f4634ec424379d69deab1df2fce4b171cb7cf54e5a5f", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -829,8 +829,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_2.12:0.11.17", - "sha256": "c984f7695e9a5034afbf725b7eab919fc00bb24dc30c8f6f923d6d32096a1fa0", + "artifact": "com.thesamet.scalapb:lenses_2.12:1.0.0-alpha.1", + "sha256": "c1ba1e1bfe373f5e984c9c7464a9554d16f5c0025820ddbcdb2370878bb72bb9", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -853,8 +853,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_2.12:0.11.17", - "sha256": "6624beb8e47c11de33262f867dd86d25e66ddce5507c9c13bfd7cc2f2e7652fe", + "artifact": "com.thesamet.scalapb:scalapb-runtime_2.12:1.0.0-alpha.1", + "sha256": "d103300acea36d2ef0fd42fc4b1882144e1761a52b87eae94d3581482582a116", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -863,8 +863,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.12:0.11.17", - "sha256": "7919fbb62f3ae9de9eec3a102b24dc1ef570ff098d1e41e464cf2ac7398cff5f", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.12:1.0.0-alpha.1", + "sha256": "029260ac16b227b6a037c72ca89c6b3eae0efe5f368ba53f52c9671b9ece9fbc", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_2_13.bzl b/third_party/repositories/scala_2_13.bzl index d5552ab97..4a5b33f4c 100644 --- a/third_party/repositories/scala_2_13.bzl +++ b/third_party/repositories/scala_2_13.bzl @@ -843,8 +843,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_2.13:0.11.17", - "sha256": "d36b84059289c7aa2f2bf08eeab7e85084fcf72bf58b337edf167c73218880d7", + "artifact": "com.thesamet.scalapb:compilerplugin_2.13:1.0.0-alpha.1", + "sha256": "218640423ba8156f994d6d700ef960d65025f79a5918070c0898213f4384df1f", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -853,8 +853,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_2.13:0.11.17", - "sha256": "4abe3fe573b8505a633414b0fbbcae4240250690ba48a9d4a6eeb3dfc3302ddf", + "artifact": "com.thesamet.scalapb:lenses_2.13:1.0.0-alpha.1", + "sha256": "46902feb0fd848fce92e234514254dc43b3cde5f6e10e88ae6eec52f4c016fbc", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -877,8 +877,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:0.11.17", - "sha256": "fe91faf58bccef68be348e76cab339a5fe2c215e48f7bd8f836190449ed94077", + "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:1.0.0-alpha.1", + "sha256": "0ceaaf48bc3fa41419fcb8830d21685aea8b7a5e403b90b3246124d9f4b6d087", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -887,8 +887,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:0.11.17", - "sha256": "c03687c038f2a45bb413551519542069a59faf322de29fd1f9e06f2dd65003d0", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:1.0.0-alpha.1", + "sha256": "75eb71fea9509308070812b8bcf1eec90c065be3e9d8c60b12098f206db6c581", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_1.bzl b/third_party/repositories/scala_3_1.bzl index b824bd5fd..71535c66d 100644 --- a/third_party/repositories/scala_3_1.bzl +++ b/third_party/repositories/scala_3_1.bzl @@ -889,8 +889,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_2.13:0.11.17", - "sha256": "d36b84059289c7aa2f2bf08eeab7e85084fcf72bf58b337edf167c73218880d7", + "artifact": "com.thesamet.scalapb:compilerplugin_2.13:1.0.0-alpha.1", + "sha256": "218640423ba8156f994d6d700ef960d65025f79a5918070c0898213f4384df1f", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library_2", @@ -899,8 +899,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_2.13:0.11.17", - "sha256": "4abe3fe573b8505a633414b0fbbcae4240250690ba48a9d4a6eeb3dfc3302ddf", + "artifact": "com.thesamet.scalapb:lenses_2.13:1.0.0-alpha.1", + "sha256": "46902feb0fd848fce92e234514254dc43b3cde5f6e10e88ae6eec52f4c016fbc", "deps": [ "@io_bazel_rules_scala_scala_library_2", "@org_scala_lang_modules_scala_collection_compat", @@ -923,8 +923,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:0.11.17", - "sha256": "fe91faf58bccef68be348e76cab339a5fe2c215e48f7bd8f836190449ed94077", + "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:1.0.0-alpha.1", + "sha256": "0ceaaf48bc3fa41419fcb8830d21685aea8b7a5e403b90b3246124d9f4b6d087", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library_2", @@ -933,8 +933,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:0.11.17", - "sha256": "c03687c038f2a45bb413551519542069a59faf322de29fd1f9e06f2dd65003d0", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:1.0.0-alpha.1", + "sha256": "75eb71fea9509308070812b8bcf1eec90c065be3e9d8c60b12098f206db6c581", "deps": [ "@io_bazel_rules_scala_scala_library_2", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_2.bzl b/third_party/repositories/scala_3_2.bzl index 29e153185..6ccc2dab7 100644 --- a/third_party/repositories/scala_3_2.bzl +++ b/third_party/repositories/scala_3_2.bzl @@ -889,8 +889,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_2.13:0.11.17", - "sha256": "d36b84059289c7aa2f2bf08eeab7e85084fcf72bf58b337edf167c73218880d7", + "artifact": "com.thesamet.scalapb:compilerplugin_2.13:1.0.0-alpha.1", + "sha256": "218640423ba8156f994d6d700ef960d65025f79a5918070c0898213f4384df1f", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library_2", @@ -899,8 +899,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_2.13:0.11.17", - "sha256": "4abe3fe573b8505a633414b0fbbcae4240250690ba48a9d4a6eeb3dfc3302ddf", + "artifact": "com.thesamet.scalapb:lenses_2.13:1.0.0-alpha.1", + "sha256": "46902feb0fd848fce92e234514254dc43b3cde5f6e10e88ae6eec52f4c016fbc", "deps": [ "@io_bazel_rules_scala_scala_library_2", "@org_scala_lang_modules_scala_collection_compat", @@ -923,8 +923,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:0.11.17", - "sha256": "fe91faf58bccef68be348e76cab339a5fe2c215e48f7bd8f836190449ed94077", + "artifact": "com.thesamet.scalapb:scalapb-runtime_2.13:1.0.0-alpha.1", + "sha256": "0ceaaf48bc3fa41419fcb8830d21685aea8b7a5e403b90b3246124d9f4b6d087", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library_2", @@ -933,8 +933,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:0.11.17", - "sha256": "c03687c038f2a45bb413551519542069a59faf322de29fd1f9e06f2dd65003d0", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_2.13:1.0.0-alpha.1", + "sha256": "75eb71fea9509308070812b8bcf1eec90c065be3e9d8c60b12098f206db6c581", "deps": [ "@io_bazel_rules_scala_scala_library_2", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_3.bzl b/third_party/repositories/scala_3_3.bzl index 0bea57d43..922c271a3 100644 --- a/third_party/repositories/scala_3_3.bzl +++ b/third_party/repositories/scala_3_3.bzl @@ -905,8 +905,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_3:0.11.17", - "sha256": "ed057718a1d2352288dfa01b643a9f03b7cc3e141939c43f39a9025c21685e92", + "artifact": "com.thesamet.scalapb:compilerplugin_3:1.0.0-alpha.1", + "sha256": "e7d7156269fc23cbb539eea60f07c3230aa05a726434fc942b040495567f0a2d", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -915,8 +915,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_3:0.11.17", - "sha256": "e887e38b7d0f713fc849a40c33cbeee088bf28a43a35b6d58292b5e0d9889366", + "artifact": "com.thesamet.scalapb:lenses_3:1.0.0-alpha.1", + "sha256": "63fdffc573947402c526c49cf6ee92990ede88d55eb56af5123dfd247b365185", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -939,8 +939,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_3:0.11.17", - "sha256": "8253f9ad2c1fa908f8eb4a90ed0980c81c950142d15433d770ebcc36ea17447a", + "artifact": "com.thesamet.scalapb:scalapb-runtime_3:1.0.0-alpha.1", + "sha256": "37ec7d72d56f58e3adb78e385e39ecb927a5097e290f4e51332bbd55fc534a65", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -949,8 +949,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:0.11.17", - "sha256": "218bd015f897391d6c6d34b96c808d4fc7c732c1674157698a2574e32a7e4a7e", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:1.0.0-alpha.1", + "sha256": "0c8574f91693cb08795ed16a601bcf6d5ba46ba8dbd71792910b706cce995c7a", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_4.bzl b/third_party/repositories/scala_3_4.bzl index 45caf68dc..524107dbd 100644 --- a/third_party/repositories/scala_3_4.bzl +++ b/third_party/repositories/scala_3_4.bzl @@ -896,8 +896,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_3:0.11.17", - "sha256": "ed057718a1d2352288dfa01b643a9f03b7cc3e141939c43f39a9025c21685e92", + "artifact": "com.thesamet.scalapb:compilerplugin_3:1.0.0-alpha.1", + "sha256": "e7d7156269fc23cbb539eea60f07c3230aa05a726434fc942b040495567f0a2d", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -906,8 +906,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_3:0.11.17", - "sha256": "e887e38b7d0f713fc849a40c33cbeee088bf28a43a35b6d58292b5e0d9889366", + "artifact": "com.thesamet.scalapb:lenses_3:1.0.0-alpha.1", + "sha256": "63fdffc573947402c526c49cf6ee92990ede88d55eb56af5123dfd247b365185", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -930,8 +930,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_3:0.11.17", - "sha256": "8253f9ad2c1fa908f8eb4a90ed0980c81c950142d15433d770ebcc36ea17447a", + "artifact": "com.thesamet.scalapb:scalapb-runtime_3:1.0.0-alpha.1", + "sha256": "37ec7d72d56f58e3adb78e385e39ecb927a5097e290f4e51332bbd55fc534a65", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -940,8 +940,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:0.11.17", - "sha256": "218bd015f897391d6c6d34b96c808d4fc7c732c1674157698a2574e32a7e4a7e", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:1.0.0-alpha.1", + "sha256": "0c8574f91693cb08795ed16a601bcf6d5ba46ba8dbd71792910b706cce995c7a", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_5.bzl b/third_party/repositories/scala_3_5.bzl index 1b494d9e6..3abbad2ae 100644 --- a/third_party/repositories/scala_3_5.bzl +++ b/third_party/repositories/scala_3_5.bzl @@ -896,8 +896,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_3:0.11.17", - "sha256": "ed057718a1d2352288dfa01b643a9f03b7cc3e141939c43f39a9025c21685e92", + "artifact": "com.thesamet.scalapb:compilerplugin_3:1.0.0-alpha.1", + "sha256": "e7d7156269fc23cbb539eea60f07c3230aa05a726434fc942b040495567f0a2d", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -906,8 +906,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_3:0.11.17", - "sha256": "e887e38b7d0f713fc849a40c33cbeee088bf28a43a35b6d58292b5e0d9889366", + "artifact": "com.thesamet.scalapb:lenses_3:1.0.0-alpha.1", + "sha256": "63fdffc573947402c526c49cf6ee92990ede88d55eb56af5123dfd247b365185", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -930,8 +930,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_3:0.11.17", - "sha256": "8253f9ad2c1fa908f8eb4a90ed0980c81c950142d15433d770ebcc36ea17447a", + "artifact": "com.thesamet.scalapb:scalapb-runtime_3:1.0.0-alpha.1", + "sha256": "37ec7d72d56f58e3adb78e385e39ecb927a5097e290f4e51332bbd55fc534a65", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -940,8 +940,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:0.11.17", - "sha256": "218bd015f897391d6c6d34b96c808d4fc7c732c1674157698a2574e32a7e4a7e", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:1.0.0-alpha.1", + "sha256": "0c8574f91693cb08795ed16a601bcf6d5ba46ba8dbd71792910b706cce995c7a", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/repositories/scala_3_6.bzl b/third_party/repositories/scala_3_6.bzl index 2ec4d4ebf..fc9893527 100644 --- a/third_party/repositories/scala_3_6.bzl +++ b/third_party/repositories/scala_3_6.bzl @@ -905,8 +905,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_compilerplugin": { - "artifact": "com.thesamet.scalapb:compilerplugin_3:0.11.17", - "sha256": "ed057718a1d2352288dfa01b643a9f03b7cc3e141939c43f39a9025c21685e92", + "artifact": "com.thesamet.scalapb:compilerplugin_3:1.0.0-alpha.1", + "sha256": "e7d7156269fc23cbb539eea60f07c3230aa05a726434fc942b040495567f0a2d", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -915,8 +915,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_lenses": { - "artifact": "com.thesamet.scalapb:lenses_3:0.11.17", - "sha256": "e887e38b7d0f713fc849a40c33cbeee088bf28a43a35b6d58292b5e0d9889366", + "artifact": "com.thesamet.scalapb:lenses_3:1.0.0-alpha.1", + "sha256": "63fdffc573947402c526c49cf6ee92990ede88d55eb56af5123dfd247b365185", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", @@ -939,8 +939,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime": { - "artifact": "com.thesamet.scalapb:scalapb-runtime_3:0.11.17", - "sha256": "8253f9ad2c1fa908f8eb4a90ed0980c81c950142d15433d770ebcc36ea17447a", + "artifact": "com.thesamet.scalapb:scalapb-runtime_3:1.0.0-alpha.1", + "sha256": "37ec7d72d56f58e3adb78e385e39ecb927a5097e290f4e51332bbd55fc534a65", "deps": [ "@com_google_protobuf_protobuf_java", "@io_bazel_rules_scala_scala_library", @@ -949,8 +949,8 @@ artifacts = { ], }, "scala_proto_rules_scalapb_runtime_grpc": { - "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:0.11.17", - "sha256": "218bd015f897391d6c6d34b96c808d4fc7c732c1674157698a2574e32a7e4a7e", + "artifact": "com.thesamet.scalapb:scalapb-runtime-grpc_3:1.0.0-alpha.1", + "sha256": "0c8574f91693cb08795ed16a601bcf6d5ba46ba8dbd71792910b706cce995c7a", "deps": [ "@io_bazel_rules_scala_scala_library", "@org_scala_lang_modules_scala_collection_compat", diff --git a/third_party/test/example_external_workspace/WORKSPACE b/third_party/test/example_external_workspace/WORKSPACE index 91f5c1d59..20b554747 100644 --- a/third_party/test/example_external_workspace/WORKSPACE +++ b/third_party/test/example_external_workspace/WORKSPACE @@ -11,19 +11,25 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config() diff --git a/third_party/test/proto/WORKSPACE b/third_party/test/proto/WORKSPACE index 7092bb8e7..cf9d11fa7 100644 --- a/third_party/test/proto/WORKSPACE +++ b/third_party/test/proto/WORKSPACE @@ -11,7 +11,13 @@ load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") rules_scala_dependencies() -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") +load("@platforms//host:extension.bzl", "host_platform_repo") + +host_platform_repo(name = "host_platform") + +register_toolchains("@rules_scala_protoc_toolchains//...:all") + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") rules_java_dependencies() @@ -21,9 +27,9 @@ bazel_skylib_workspace() http_archive( name = "rules_python", - sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", - strip_prefix = "rules_python-0.38.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", + sha256 = "2ef40fdcd797e07f0b6abda446d1d84e2d9570d234fddf8fcd2aa262da852d1c", + strip_prefix = "rules_python-1.2.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/1.2.0/rules_python-1.2.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -34,6 +40,8 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + rules_java_toolchains() load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") @@ -48,6 +56,10 @@ load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") rules_proto_toolchains() +load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains") + +scala_protoc_toolchains(name = "rules_scala_protoc_toolchains") + load("@rules_scala//:scala_config.bzl", "scala_config") scala_config() diff --git a/tools/bazel.rc.buildkite b/tools/bazel.rc.buildkite index 6a2495d05..479197f37 100644 --- a/tools/bazel.rc.buildkite +++ b/tools/bazel.rc.buildkite @@ -1,5 +1,10 @@ -build --strategy=Scalac=worker --strategy=ScroogeRule=worker --worker_max_instances=3 - # Switch to --noenable_workspace when Bzlmod lands. # https://github.com/bazelbuild/rules_scala/issues/1482 common --enable_workspace --noenable_bzlmod + +# Remove once proto toolchainization becomes the default +# - https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution +# - https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit +common --incompatible_enable_proto_toolchain_resolution + +build --strategy=Scalac=worker --strategy=ScroogeRule=worker --worker_max_instances=3