Skip to content

feat: conservatively abstract unsupported Python constructs#566

Open
MikaelMayer wants to merge 8 commits intomainfrom
feat/conservative-python-translation
Open

feat: conservatively abstract unsupported Python constructs#566
MikaelMayer wants to merge 8 commits intomainfrom
feat/conservative-python-translation

Conversation

@MikaelMayer
Copy link
Contributor

@MikaelMayer MikaelMayer commented Mar 12, 2026

What

Instead of failing on unsupported Python constructs, translate them to sound over-approximations so the verifier can still run.

How

Construct Abstraction
Unsupported expressions Hole (fresh uninterpreted value)
Unsupported binary operators Hole
Compound LHS assignment (a[i]=v, (a,b)=v) Havoc all in-scope variables (aliasing may affect any of them)
Tuple unpacking in for (for a,b in ...) Havoc all target names
with statement Body translated; as-targets declared as Hole
Unsupported statements Havoc all in-scope variables + assert *

Unsupported statements and compound LHS assignments are recorded in TranslationContext.abstractedStatements and emitted as warnings by pyAnalyzeLaurel.

Tests

Added test_with_statement.py.

@MikaelMayer MikaelMayer requested a review from a team March 12, 2026 21:00
@MikaelMayer MikaelMayer force-pushed the feat/conservative-python-translation branch 3 times, most recently from 3b6064f to 6c73095 Compare March 12, 2026 21:13
@MikaelMayer MikaelMayer marked this pull request as draft March 12, 2026 21:13
@MikaelMayer MikaelMayer force-pushed the feat/conservative-python-translation branch 4 times, most recently from 360dfb9 to a376cca Compare March 12, 2026 21:47
Copy link
Contributor

@olivier-aws olivier-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general comment, I'm not a big fan of hiding away what we don't support. There are many risks of unsoundness with this. At the very least, we need a message in the log that the translation encountered an unsupported expression/statement and over-approximated it.

Comment on lines +510 to +512
-- Compound LHS (e.g. a[i] = v, (a, b) = v) — havoc all Name targets
-- This is a sound over-approximation: we lose the assignment but keep
-- the program structure.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is sound in Python, due to aliasing. This Python script prints 999:

a = [1, 2, 3]
b = a
b[0] = 999
print(a[0])

Here you would only havoc b, not a

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed good catch. The answer there is that we should havoc not just all name targets but all names in scope.

I hadn't reviewed that part yet but I'm happy you did!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a second thought, if we stored dictionaries via an indirection to the heap, then no target would be havoc-ed, we would here only havoc the heap to indicate something changed, so we would still be able to prove a[0] == b[0]
What is the status of the heap in Laurel? Is it fully supported?

Copy link
Contributor

@ssomayyajula ssomayyajula Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK Laurel does allow arbitrary StmtExprs to occur on the LHS of an assignment, it's just that Laurel->Core only supports assignmets to identifiers https://github.com/strata-org/Strata/blob/main/Strata/Languages/Laurel/LaurelToCoreTranslator.lean#L362 (that is, Laurel supports heap-parameterized procedures + the creation of objects of composite type on the heap, but not assigning to their fields after the fact)

Copy link
Contributor

@keyboardDrummer keyboardDrummer Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(that is, Laurel supports heap-parameterized procedures + the creation of objects of composite type on the heap, but not assigning to their fields after the fact)

What? You can assign to fields whenever you like: https://github.com/strata-org/Strata/blob/main/StrataTest/Languages/Laurel/Examples/Objects/T1_MutableFields.lean#L40

I would prefer to change the field selector syntax from # to . but I ran into some technical issues. Will resolve them at some point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The unsupported statement catch-all now records the abstraction in ctx.abstractedStatements and emits a warning via pyAnalyzeLaurel. Unsupported expressions also record warnings via the ExprM monad.

@MikaelMayer MikaelMayer force-pushed the feat/conservative-python-translation branch 17 times, most recently from 2a4ed7d to 8074972 Compare March 17, 2026 20:15
@MikaelMayer MikaelMayer force-pushed the feat/conservative-python-translation branch 2 times, most recently from 677106f to 5da2e01 Compare March 17, 2026 20:43
| some _ => throw (.internalError s!"Keyword arg should be a Dict")
| none =>
let exprtranslateExpr ctx expr
let (expr, _) ← (translateExpr ctx expr).run'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this dropping "warnings"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and it should not. Thanks for spotting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: translateDictKWords, translateVarKwargs, translateKwargs, and translateCall now use ExprM so warnings from nested expression translations are propagated rather than dropped. translateStmt also uses translateExprW for condition/value expressions to merge warnings into ctx.abstractedStatements.

- abstractedStatements field in TranslationContext
- ExprM monad for expression translation with warning accumulation
- havocAllVars helper (reusable across conservative abstraction)
- Unsupported binary operators → Hole + warning
- Unsupported expressions → Hole + warning
- Unsupported statements → havocAllVars + assert * + record abstraction
- Emit abstraction warnings from pyAnalyzeLaurel pipeline
@MikaelMayer MikaelMayer force-pushed the feat/conservative-python-translation branch from 5da2e01 to 4e4238b Compare March 19, 2026 16:12
- translateDictKWords, translateVarKwargs, translateKwargs, translateCall
  now use ExprM so warnings from nested expressions are not dropped
- translateStmt uses translateExprW for condition/value expressions
  to merge warnings into ctx.abstractedStatements
- With statement expression warnings propagated via currentCtx mutation
@MikaelMayer MikaelMayer marked this pull request as ready for review March 19, 2026 18:34
Copy link
Contributor Author

@MikaelMayer MikaelMayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core idea of abstracting unsupported constructs to sound over-approximations instead of failing is good. The ExprM monad for accumulating warnings and the havoc-all-vars + assert-star pattern for unsupported statements are well-designed.

However, there is a bug that makes the warning mechanism non-functional for the main body: at line 1758 of PythonToLaurel.lean, pythonToLaurel' returns ctx instead of finalCtx, so all abstractedStatements accumulated during translateStmtList are lost. The pyAnalyzeLaurel caller will never see any warnings. Fix: return (program, finalCtx). I verified this compiles cleanly.

Additionally, translateAssign uses translateExprVal which silently drops expression warnings, creating an inconsistency with how translateStmt handles them.

The PR description mentions compound LHS assignment and tuple unpacking in for as abstracted constructs, but neither is implemented — both still throw unsupportedConstruct. The catch-all at the end of translateStmt only catches unmatched statement types, not sub-cases within already-matched statements like For or Assign. The description also claims test_with_statement.py was added, but it's not in the diff (it was added in PR #580).

I'd like the bug fixed and the PR description corrected before approving.

- translateCall: use translateExpr directly (already in ExprM context)
- translateAssign: use translateExprW to propagate RHS/target warnings
- ExprM.warn: prepend instead of append (O(1)), reverse at run' boundary
- Statement catch-all: pattern-match constructors instead of repr
# Conflicts:
#	Strata/Languages/Python/PythonToLaurel.lean
# Conflicts:
#	Strata/Languages/Python/PythonToLaurel.lean
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants