User site-packages directory should be ignored

[siddharthab]

🚀 feature request

Relevant Rules

py_{library,binary,test}

Description

When running builds and tests, the user site-packages directory is not ignored, which makes the build less hermetic.

Describe the solution you'd like

The registered toolchains could set an env var `PYTHONNOUSERSITE=1` every time they are run. This env var is already set for gazelle runs. More generally, the toolchains could also allow the user to set other environment variables for the toolchain.

Describe alternatives you've considered

I have currently set `--action_env= PYTHONNOUSERSITE=1` to all bazel build steps, but it is less clean because actions that have nothing to do with Python don't need this env var.

[rickeylev]

More generally, the toolchains could also allow the user to set other environment variables for the toolchain.

Can you expand a bit on the rationale/motivation for this idea? It's utility isn't obvious to me, though it's still an interesting idea.

[siddharthab]

Can you expand a bit on the rationale/motivation for this idea?

General configurability of the toolchain. For example, this issue itself. Other examples could include PYTHONHASHSEED, PYTHONWARNINGS, PYTHONCOERCECLOCALE, etc.

Obviously, these could also be set for all actions through --action_env, but setting them just for the toolchain is cleaner.

[dflems]

This one bit me recently. A user had some packages installed in their user site packages dir (~/Library/Python/X.Y/lib/python/site-packages on macOS) and those would always be selected first when importing, no matter what order was specified in the python path. Unfortunately --action_env=PYTHONNOUSERSITE=1 didn't help either in this case.

edit: I can reliably reproduce this with the protobuf package after installing it in the user site packages without installing from wheels: python3 -m pip install --user protobuf --no-binary :all: (using Python 3.9)

[siddharthab]

Interesting. For us, the following in our .bazelrc shields us just fine. I verified by deleting some deps from my test target. The tests pass if I don't set PYTHONNOUSERSITE, or if I restore the deps.

build --action_env=PYTHONNOUSERSITE=1

Note that your Python version needs to support the env var.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[siddharthab]

I think this issue is still relevant.

[alexeagle]

I think this is already fixed in https://github.com/aspect-build/rules_py because we run the interpreter in isolated mode.

[siddharthab]

I was not able to get rules_py to work because of aspect-build/rules_py#159.

[aignas]

@siddharthab, I think some parts of the isolation are present when running under Python 3.11 or newer. Is your issue related to #382, or is it that you need the hermeticity to be a part of build actions?

[siddharthab]

The latter. I want bazel tests to not consider the packages the user has installed in their home directory. The other issue you reference is about PYTHONSAFEPATH env var. This issue is about a different env var.

A general solution to have env vars be part of the toolchain configuration will be helpful in many contexts.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[github-actions]

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"

[FrankPortman]

We have been using rules_python in a relatively large group for years, so I am surprised we have not hit this yet. I found this thread while trying to chase down something else related to our upgrade to Python 3.11 (or perhaps, it is the same issue in disguise). Is the behavior being described in this thread still expected to be present in recent releases of rules_python coupled with Python 3.11 interpreter? It's unclear to me what is "more isolated in Python 3.11 or newer" per @aignas comment.