Skip to content

refactor!(toolchain): remove uname dep in the repository_rule stage #2406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,39 @@ Unreleased changes template.

{#v0-0-0-changed}
### Changed

**Breaking**:
* (toolchains) stop exposing config settings in python toolchain alias repos.
Please consider depending on the flags defined in
`//python/config_setting/...` and the `@platforms` package instead.
* (toolchains) consumers who were depending on the `MACOS_NAME` and the `arch`
attribute in the `PLATFORMS` list, please update your code to respect the new
values. The values now correspond to the values available in the
`@platforms//` package constraint values.
* (toolchains) `host_platform` and `interpreter` constants are no longer created
in the `toolchain` generated alias `.bzl` files. If you need to access the
host interpreter during the `repository_rule` evaluation, please use the
`@python_{version}_host//:python` targets created by
{bzl:obj}`python_register_toolchains` and
{bzl:obj}`python_register_multi_toolchains` macros or the {bzl:obj}`python`
bzlmod extension.

Other changes:
* (python_repository) Start honoring the `strip_prefix` field for `zstd` archives.

{#v0-0-0-fixed}
### Fixed
* Nothing fixed.
* (toolchains) stop depending on `uname` to get the value of the host platform.

{#v0-0-0-added}
### Added
* (gazelle): Parser failures will now be logged to the terminal. Additional
details can be logged by setting `GAZELLE_VERBOSE=1`.
* (toolchains) allow users to select which variant of the support host toolchain
they would like to use through
`RULES_PYTHON_REPO_TOOLCHAIN_{VERSION}_{OS}_{ARCH}` env variable setting. For
example, this allows one to use `freethreaded` python interpreter in the
`repository_rule` to build a wheel from `sdist`.

{#v0-0-0-removed}
### Removed
Expand Down
2 changes: 1 addition & 1 deletion WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps
_py_gazelle_deps()

# This interpreter is used for various rules_python dev-time tools
load("@python//3.11.9:defs.bzl", "interpreter")
interpreter = "@python_3_11_9_host//:python"

#####################
# Install twine for our own runfiles wheel publishing.
Expand Down
10 changes: 10 additions & 0 deletions docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ Determines the verbosity of logging output for repo rules. Valid values:
* `TRACE`
:::

:::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH

Determines the python interpreter platform to be used for a particular
interpreter `(version, os, arch)` triple to be used in repository rules.
Replace the `VERSION_OS_ARCH` part with actual values when using, e.g.
`3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the
os, arch values are the same as the ones mentioned in the
`//python:versions.bzl` file.
:::

:::{envvar} RULES_PYTHON_PIP_ISOLATED

Determines if `--isolated` is used with pip.
Expand Down
4 changes: 2 additions & 2 deletions examples/bzlmod/MODULE.bazel.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion python/private/python_register_toolchains.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,11 @@ def python_register_toolchains(
platform = platform,
))

host_toolchain(name = name + "_host")
host_toolchain(
name = name + "_host",
platforms = loaded_platforms,
python_version = python_version,
)

toolchain_aliases(
name = name,
Expand Down
6 changes: 5 additions & 1 deletion python/private/repo_utils.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ def _logger(mrctx, name = None):

Returns:
A struct with attributes logging: trace, debug, info, warn, fail.
Please use `return logger.fail` when using the `fail` method, because
it makes `buildifier` happy and ensures that other implementation of
the logger injected into the function work as expected by terminating
on the given line.
"""
if _is_repo_debug_enabled(mrctx):
verbosity_level = "DEBUG"
Expand Down Expand Up @@ -140,7 +144,7 @@ def _execute_internal(
result = mrctx.execute(arguments, environment = environment, **kwargs)

if fail_on_error and result.return_code != 0:
logger.fail((
return logger.fail((
"repo.execute: {op}: end: failure:\n" +
" command: {cmd}\n" +
" return code: {return_code}\n" +
Expand Down
74 changes: 74 additions & 0 deletions python/private/toolchain_aliases.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Create toolchain alias targets."""

load("@rules_python//python:versions.bzl", "PLATFORMS")

def toolchain_aliases(*, name, platforms, visibility = None, native = native):
"""Create toolchain aliases for the python toolchains.

Args:
name: {type}`str` The name of the current repository.
platforms: {type}`platforms` The list of platforms that are supported
for the current toolchain repository.
visibility: {type}`list[Target] | None` The visibility of the aliases.
native: The native struct used in the macro, useful for testing.
"""
for platform in PLATFORMS.keys():
if platform not in platforms:
continue

native.config_setting(
name = platform,
flag_values = PLATFORMS[platform].flag_values,
constraint_values = PLATFORMS[platform].compatible_with,
visibility = ["//visibility:private"],
)

prefix = name
for name in [
"files",
"includes",
"libpython",
"py3_runtime",
"python_headers",
"python_runtimes",
]:
native.alias(
name = name,
actual = select({
":" + platform: "@{}_{}//:{}".format(prefix, platform, name)
for platform in platforms
}),
visibility = visibility,
)

native.alias(
name = "python3",
actual = select({
":" + platform: "@{}_{}//:{}".format(prefix, platform, "python.exe" if "windows" in platform else "bin/python3")
for platform in platforms
}),
visibility = visibility,
)
native.alias(
name = "pip",
actual = select({
":" + platform: "@{}_{}//:python_runtimes".format(prefix, platform)
for platform in platforms
if "windows" not in platform
}),
visibility = visibility,
)
Loading