Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a LocalScope for CollectionComprehensions and generate a Declaration #2019

Draft
wants to merge 54 commits into
base: main
Choose a base branch
from

Conversation

KuechA
Copy link
Contributor

@KuechA KuechA commented Jan 31, 2025

Fixes #2005 by

  • Entering a LocalScope for list, set, dict comprehensions and generators. This somewhat simulates PEP 709 where the loop variable is available only inside the loop.
  • Generating a VariableDeclaration inside this scope in the PythonAddDeclarationsPass for the variables.
  • Adds test that the local variables are not accessible outside the comprehension and that the refersTo edges are set
  • For AssignExpressions with the := operator inside a list/set/dict comprehension, the variable is looked up or created outside of the comprehension's LocalScope to address PEP 572

Copy link

codecov bot commented Jan 31, 2025

Codecov Report

Attention: Patch coverage is 90.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 78.29%. Comparing base (541dea1) to head (1ebbc6f).

Files with missing lines Patch % Lines
.../main/kotlin/de/fraunhofer/aisec/cpg/graph/Node.kt 60.00% 0 Missing and 2 partials ⚠️
...s/kotlin/de/fraunhofer/aisec/cpg/test/TestUtils.kt 50.00% 2 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
...ain/kotlin/de/fraunhofer/aisec/cpg/ScopeManager.kt 82.70% <100.00%> (+0.05%) ⬆️
...otlin/de/fraunhofer/aisec/cpg/graph/TypeBuilder.kt 56.41% <100.00%> (+2.56%) ⬆️
...er/aisec/cpg/frontends/python/ExpressionHandler.kt 83.04% <100.00%> (ø)
...ofer/aisec/cpg/passes/PythonAddDeclarationsPass.kt 92.78% <100.00%> (+2.11%) ⬆️
.../main/kotlin/de/fraunhofer/aisec/cpg/graph/Node.kt 86.17% <60.00%> (-1.48%) ⬇️
...s/kotlin/de/fraunhofer/aisec/cpg/test/TestUtils.kt 71.27% <50.00%> (-1.55%) ⬇️

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@oxisto oxisto added the python label Jan 31, 2025
Copy link
Contributor

@maximiliankaul maximiliankaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general. However, there is still an issue with AssignExpressions leaking to the containing scope. See the added tests. I'm in favor of this PR once these issues are resolved.

@oxisto oxisto requested a review from maximiliankaul February 6, 2025 20:05
bDeclaration,
"We expect that the reference \"b\" in the tuple refers to the VariableDeclaration of \"b\" which is added outside the list comprehension (in statement 0).",
)
assertRefersTo(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should discuss what we expect in this case. My version of Python3.13 crashes with the error "UnboundLocalError: cannot access local variable 'a' where it is not associated with a value".

We may interpret this by setting the refersTo to same as the reference a in the second item and later, an analysis must figure out that this variable is not initialized yet (we should check the EOG for this too).

We may also interpret this that refersTo should be null

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that refersTo should be null. I think we did that in other cases where we have an unbound error as well

Copy link
Contributor Author

@KuechA KuechA Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, since the SymbolResolver is in charge of this, I added issue #2035 which blocks this PR. I also added the asserts again.

@KuechA KuechA added the blocked Blocked by an external factor label Feb 7, 2025
@KuechA KuechA marked this pull request as draft February 7, 2025 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Blocked by an external factor python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python comprehensions should open a local scope and set/infer VariableDeclarations for the variables
3 participants