Fix incremental issue with namespace packages (option 1) #18907
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes python#12664 A root cause is there is this stateful `_update_ns_ancestors` thing in `modulefinder`, so if things get called in the wrong order, you can get incorrect results. See also the logic in `all_imported_modules_in_file` where we've fixed several bugs like this previously, like python#13124 and python#10937 As a result of (seemingly accidentally) reusing imports across modules, we can end up in a situation where the namespace gets added as a dependency to other modules and so on the cached run we attempt to find namespace before package, which does not work I am not sure this `imports` code path is even needed, so I will open an alternate PR. I can't write a good test for this because it requires something in site_packages, but here's a minimal repro: ``` set -eux rm -rf repro mkdir repro cd repro SITEPACK=env/site-packages mkdir -p $SITEPACK mkdir $SITEPACK/ruamel mkdir $SITEPACK/ruamel/yaml printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py printf '' > $SITEPACK/ruamel/yaml/py.typed printf 'import ruamel.yaml' > a.py printf 'import a' > main.py rm -rf .mypy_cache PYTHONPATH=$SITEPACK mypy main.py PYTHONPATH=$SITEPACK mypy main.py ```
hauntsaninja
added a commit
to hauntsaninja/mypy
that referenced
this pull request
Apr 11, 2025
Fixes python#12664 See python#18907 which is another possible fix I'm not sure why this logic is actually needed given that we should record the dependency from import analysis in fastparse... maybe just because it adjusts priority? I can't write a good test for this because it requires something in site_packages, but here's a minimal repro: ``` set -eux rm -rf repro mkdir repro cd repro SITEPACK=env/site-packages mkdir -p $SITEPACK mkdir $SITEPACK/ruamel mkdir $SITEPACK/ruamel/yaml printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py printf '' > $SITEPACK/ruamel/yaml/py.typed printf 'import ruamel.yaml' > a.py printf 'import a' > main.py rm -rf .mypy_cache PYTHONPATH=$SITEPACK mypy main.py PYTHONPATH=$SITEPACK mypy main.py ```
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
hauntsaninja
changed the title
ilevkivskyi
approved these changes
Apr 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #12664
A root cause is there is this stateful
_update_ns_ancestorsthing inmodulefinder, so if things get called in the wrong order, you can get incorrect results.See also the logic in
all_imported_modules_in_filewhere we've fixed several bugs like this previously, like #13124 and #10937As a result of (seemingly accidentally) reusing imports across modules, we can end up in a situation where the namespace gets added as a dependency to all other modules and so on the cached run we attempt to find namespace before package, which does not work
I am not sure this
importscode path is even needed, so I will open an alternate PR, see #18908.Relevant history:
I can't write a good test for this because it requires something in site_packages, but here's a minimal repro: