Skip to content

Disable assertion rewriting external modules #13421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fb29583
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
14b0382
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 13, 2025
10a2709
13403: Disable assertion rewriting for external modules - move root p…
Tusenka May 13, 2025
fc0c87c
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
3918365
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 15, 2025
62fe69d
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
8671103
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
3beb48e
13403: Disable assertion rewriting for external modules - add tests
Tusenka May 18, 2025
0015d0a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
d5eb2a6
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 19, 2025
392a01d
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 22, 2025
ccb0dca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 22, 2025
e0bbaa3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
dbc5a59
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 24, 2025
23e0d70
13403: Disable assertion rewriting for external modules - eliminate o…
Tusenka May 25, 2025
559a5c0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 27, 2025
a3cacb8
13403: Disable assertion rewriting for external modules - fix ruff
Tusenka Jun 1, 2025
3358751
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 1, 2025
6ec5c30
Update testing/test_assertrewrite.py
Tusenka Jun 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/13403.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Disable assertion rewriting of external modules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we elaborate a bit on "external modules" here? This should be written for users scanning the changelog.

6 changes: 6 additions & 0 deletions src/_pytest/assertion/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,14 @@ class AssertionState:
def __init__(self, config: Config, mode) -> None:
self.mode = mode
self.trace = config.trace.root.get("assertion")
self.config = config
self.hook: rewrite.AssertionRewritingHook | None = None

@property
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this must mot use getwd instead use the invoction params

Copy link
Author

@Tusenka Tusenka May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then we cannot change it on runtime as far as invocation param for pytester changes rootpath

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the type - this code shoud use the rootdir or the invocationdir from the invocation params of the config
see https://docs.pytest.org/en/stable/reference/reference.html#pytest.Config.invocation_params as well as the config rootdir

Copy link
Author

@Tusenka Tusenka May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My error, I mean as far as Invocation Param is frozen - it can't be changed on runtime. Pytester starts after the config has been loaded. So to change the rootpath for pytester I need to rewrite the rootpath in someway. I could mock all invocation params or I could use getcwd alongside with config rootdir for the testing purpose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The complete object can be replaced with a changed one

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. would be done)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But is config.invocation_params.dir the correct thing to use? If I execute the tests using pytest src/tests/foo.py, I expect pytest to continue to rewrite the same files as if I execute just pytest.

I feel we should use config.rootpath here? What do you think @RonnyPfannschmidt ?

def rootpath(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have rootpath in Config, but this property returns something different so this is a bit misleading. I suggest we rename it to something else to avoid this confusion, how about:

Suggested change
def rootpath(self):
def invocation_path(self):

"""Get current root path (current working dir)"""
return str(self.config.invocation_params.dir)


def install_importhook(config: Config) -> rewrite.AssertionRewritingHook:
"""Try to install the rewrite hook, raise SystemError if it fails."""
Expand Down
2 changes: 1 addition & 1 deletion src/_pytest/assertion/rewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ def _should_rewrite(self, name: str, fn: str, state: AssertionState) -> bool:
# rewritten if they match the naming convention for test files
fn_path = PurePath(fn)
for pat in self.fnpats:
if fnmatch_ex(pat, fn_path):
if fnmatch_ex(pat, fn_path) and fn_path.is_relative_to(state.rootpath):
state.trace(f"matched test file {fn!r}")
return True

Expand Down
9 changes: 9 additions & 0 deletions src/_pytest/pytester.py
Original file line number Diff line number Diff line change
Expand Up @@ -749,6 +749,15 @@ def chdir(self) -> None:
This is done automatically upon instantiation.
"""
self._monkeypatch.chdir(self.path)
self._monkeypatch.setattr(
self._request.config,
"invocation_params",
Config.InvocationParams(
args=self._request.config.invocation_params.args,
plugins=self._request.config.invocation_params.plugins,
dir=Path(self._path),
),
Comment on lines +755 to +759
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Config.InvocationParams(
args=self._request.config.invocation_params.args,
plugins=self._request.config.invocation_params.plugins,
dir=Path(self._path),
),
dataclasses.replace(self._request.config.invocation_params, dir=Path(self._path))

)

def _makefile(
self,
Expand Down
120 changes: 120 additions & 0 deletions testing/test_assertrewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import inspect
import marshal
import os
from os import mkdir
from pathlib import Path
import py_compile
import re
Expand All @@ -35,6 +36,7 @@
from _pytest.assertion.rewrite import rewrite_asserts
from _pytest.config import Config
from _pytest.config import ExitCode
from _pytest.monkeypatch import MonkeyPatch
from _pytest.pathlib import make_numbered_dir
from _pytest.pytester import Pytester
import pytest
Expand Down Expand Up @@ -370,6 +372,7 @@ def test_rewrites_plugin_as_a_package(self, pytester: Pytester) -> None:
pytester.makeconftest('pytest_plugins = ["plugin"]')
pytester.makepyfile("def test(special_asserter): special_asserter(1, 2)\n")
result = pytester.runpytest()

result.stdout.fnmatch_lines(["*assert 1 == 2*"])

def test_honors_pep_235(self, pytester: Pytester, monkeypatch) -> None:
Expand Down Expand Up @@ -1294,6 +1297,34 @@ def test_meta_path():
)
assert pytester.runpytest().ret == 0

def test_rootpath_base(self, pytester: Pytester, monkeypatch: MonkeyPatch) -> None:
"""Base cases for get rootpath from AssertionState"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test name and docstring are a bit confusing to me, why "base cases"? Isn't this test just testing the AssertState.rootpath property?

from _pytest.assertion import AssertionState

config = pytester.parseconfig()
state = AssertionState(config, "rewrite")
assert state.rootpath == str(config.invocation_params.dir)
new_rootpath = str(pytester.path / "test")
if not os.path.exists(new_rootpath):
os.mkdir(new_rootpath)
monkeypatch.setattr(
config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(new_rootpath),
),
)
state = AssertionState(config, "rewrite")
assert state.rootpath == new_rootpath

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
@pytest.mark.skipif(
sys.platform.startswith("sunos5"), reason="cannot remove cwd on Solaris"
)
Comment on lines +1322 to +1327
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these skipif marks still necessary?

def test_write_pyc(self, pytester: Pytester, tmp_path) -> None:
from _pytest.assertion import AssertionState
from _pytest.assertion.rewrite import _write_pyc
Expand Down Expand Up @@ -1971,6 +2002,95 @@ def test_simple_failure():
assert hook.find_spec("file") is not None
assert self.find_spec_calls == ["file"]

def test_assert_rewrites_only_rootpath(
self, pytester: Pytester, hook: AssertionRewritingHook, monkeypatch
) -> None:
"""Do not rewrite assertions in tests outside `AssertState.rootpath` (#13403)."""
pytester.makepyfile(
Copy link
Member

@nicoddemus nicoddemus Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to avoid mocking here, instead using a real scenario to ensure Python files outside of the rootpath do not have their assertions rewritten.

Please create a simple but real-world project using pytester, something like:

myproject/
   /venv
     /lib
        /external_lib.py
     pyvenv.cfg  

   /src
      main.py
   /tests
      conftest.py
      test_main.py
   pytest.ini
outside.py

pytest.ini should have these contents:

[pytest]
python_files = *.py

Then execute pytest.runpytest(), which will execute from the root of the above tree.

If I understand the objective of the issue correctly:

  • myproject/venv/lib/external_lib.py: should not have assertions rewritten -- the file is inside a virtual environment and is not part of a pytest plugin. The pyvenv.cfg file is how pytest detects virtual environments.
  • myproject/src/main.py: rewritten -- inside rootpath and matches python_files.
  • myproject/tests/conftest.py: rewritten -- conftest.py files are always rewritten.
  • myproject/tests/test_main.py: rewritten -- test files are always rewritten.
  • outside.py: should not have assertions rewritten -- outside the rootpath.

I think the scenario above should cover what needs to be tested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to include an external plugin here too, to test the same scenario as in test_assert_rewrite_correct_for_plugins.

**{
"file.py": """\
def test_simple_failure():
assert 1 + 1 == 3
"""
}
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is not None

rootpath = f"{os.getcwd()}/tests"
if not os.path.exists(rootpath):
mkdir(rootpath)
monkeypatch.setattr(
pytester._request.config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(rootpath),
),
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is None

def test_assert_rewrite_correct_for_conftfest(
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mamy name it in another way - test_assert_rewrite_for_conftfest

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My previous suggestion about using a real-case scenario makes this test unnecessary.

self, pytester: Pytester, hook: AssertionRewritingHook, monkeypatch
) -> None:
"""Conftest is always rewritten regardless of the root dir"""
pytester.makeconftest(
"""
import pytest
@pytest.fixture
def fix(): return 1
"""
)

rootpath = f"{os.getcwd()}/tests"
if not os.path.exists(rootpath):
mkdir(rootpath)
monkeypatch.setattr(
pytester._request.config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(rootpath),
),
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("conftest") is not None

def test_assert_rewrite_correct_for_plugins(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My previous suggestion about using a real-case scenario makes this test unnecessary.

self, pytester: Pytester, hook: AssertionRewritingHook, monkeypatch
) -> None:
"""
Plugins has always been rewritten regardless of the root dir
"""
pkgdir = pytester.mkpydir("plugin")
pkgdir.joinpath("__init__.py").write_text(
"import pytest\n"
"@pytest.fixture\n"
"def special_asserter():\n"
" def special_assert(x, y):\n"
" assert x == y\n"
" return special_assert\n",
encoding="utf-8",
)
hook.mark_rewrite("plugin")
rootpath = f"{os.getcwd()}/tests"
if not os.path.exists(rootpath):
mkdir(rootpath)
monkeypatch.setattr(
pytester._request.config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(rootpath),
),
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("plugin") is not None

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
Expand Down