Skip to content

Fix Python translation pipeline: InstanceCall, control flow, and construct support#613

Draft
joehendrix wants to merge 14 commits intomainfrom
jhx/fix_dispatch
Draft

Fix Python translation pipeline: InstanceCall, control flow, and construct support#613
joehendrix wants to merge 14 commits intomainfrom
jhx/fix_dispatch

Conversation

@joehendrix
Copy link
Contributor

@joehendrix joehendrix commented Mar 19, 2026

Fix end-to-end translation of target.method(args) instance calls, control flow constructs (return, break/continue, while loops, nested try/except), and 10 previously unsupported Python constructs that caused the pipeline to fail with translation or type errors.

  • Emit InstanceCall from PythonToLaurel for target.method(args) patterns and translate through Laurel resolution and Core translator with self-stripping, since PySpec already strips self from method signatures.
  • PySpec procedures now declare Any-typed parameters with preconditions (requires Any..isfrom_string(x)), keeping the Core translator language-agnostic.
  • Handle void procedures (0 outputs) when their result is assigned to a variable — emit call without assignment target to avoid Core arity mismatch.
  • Translate return as Exit "$body" instead of assigning to the first output parameter, which after heap parameterization could clobber $heap. Generate unique labels for nested try/except blocks. Hoist while loop variables to function scope. Translate break/continue via labeled Exit.
  • Translate augmented assignment (+=, -=, *=), ** operator, f-strings, ternary expressions, slicing, tuple for-targets, and import tracking. Emit Hole for dict/set comprehensions and generators. Type exception variables as Any.
  • Add regression tests exercising the fixes through the dispatch/heap-parameterization path:
    • test_instance_call_result.py — assign result of client.get_item() to a variable and return it (the core InstanceCall bug this PR fixes).
    • test_while_var_scope.py — variables declared inside a while loop body must be accessible after the loop exits; exercises while-loop hoisting.
    • test_augassign.py — augmented assignment (count += 1) after a service call.
    • test_pow_operator.py** power operator (base ** exp).
    • test_fstring.py — f-string interpolation (f"{name}_bucket") used as a service call argument.
    • test_ternary.py — ternary/conditional expression ("a" if flag else "b") used as a service call argument.
    • test_slice.py — list slicing (items[1:]) on a service call result.
    • test_tuple_for.py — tuple unpacking in for-loop targets (for k, v in pairs).
    • test_import_usage.py — referencing an imported module (sys.argv[1]) as a service call argument; exercises moduleAliases tracking.
    • test_except_var_usage.py — using the exception variable (as e) inside an except block; exercises exception variable typing.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@joehendrix joehendrix changed the title Add regression tests for Python translation and type checking gaps Fix Python translation pipeline: InstanceCall, control flow, and construct support Mar 19, 2026
joehendrix and others added 13 commits March 19, 2026 23:45
Return statements in PythonToLaurel now generate explicit
`LaurelResult := v; exit $body` instead of `Return (some v)`, which
was assigning the return value to `$heap` after heap parameterization
prepended it as the first output parameter.

Implement the Exit statement in LaurelToCoreTranslator (was a TODO
no-op). Preserve labeled Blocks during Laurel-to-Core translation so
that exit targets (e.g. from try/except) resolve correctly. Fix
try/except structure so the body is wrapped in the handler-labeled
block, letting `exit "exception_handlers"` target an enclosing block.

Add Core type checking to AnalyzeLaurelTest to catch type errors like
Heap vs Any. Add test_heap_return and test_list_str regression tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
test_nested_try.py: nested try/except generates duplicate "try_end"
block labels, which Core rejects as shadowing.

test_try_scope.py: variables declared inside try body are scoped to
the block, but Python expects them visible after the try/except.

Both are marked as expected failures with their current error messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These tests now pass after #572 merged, which fixed nested try/except
labels and variable scoping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 8 expected-failure tests covering remaining translation and type
checking gaps:
- Slice expressions, ternary (IfExp), tuple for-loop targets
- Augmented assignment (+=), power operator (**)
- F-strings starting with variable interpolation (FormattedValue)
- Undeclared variables from unmodeled imports (e.g. sys.argv)
- Exception handler variable typed as Composite where Any expected

Also add failPrefix matching to Expected so translation errors can
match on the error type without being fragile to AST byte offsets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- AugAssign: desugar `x += y` to `x = x + y` for arithmetic operators
- Pow: add `**` operator to binary op translation table
- FormattedValue: handle f-string interpolation by translating inner expr
- IfExp/Slice: translate ternary and slice expressions to Hole
- Tuple for-target: destructure `for k, v in ...` into individual vars
- PythonError: type exception variables as Any instead of Composite
- While hoisting: hoist variable declarations from while body to outer scope

Flips 8 regression tests from expected failure to success. Import handling
(test_import_usage) remains as expected failure — needs design work to
distinguish dispatch imports from external module imports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add moduleAliases field to TranslationContext mapping Python identifiers
  to module names, supporting import aliasing (e.g. `import X as Y`)
- Register imports early in pythonToLaurel' so aliases flow to functions
- Module attribute access on unmodeled modules produces Hole gracefully
- Fix isPackage to recognize module aliases as package references
- Initialize hoisted while-loop variables with AnyNone instead of Hole
  to avoid heap parameterization inferring Composite types
- Mark test_while_var_scope as expected failure (root cause: InstanceCall
  emitting Hole needs proper implementation per pyspec_methods.md plan)
- Flip test_import_usage to success

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implement InstanceCall in LaurelToCoreTranslator at all 5 positions
  (expression, initializer, assignment, parallel assignment, statement)
  as generic Core call statements with no language-specific logic
- Change PySpec procedures to use Any-typed parameters and return values,
  with type constraints expressed as preconditions (e.g. isfrom_string)
- Emit InstanceCall from PythonToLaurel instead of Hole for instance
  method calls on typed variables
- Fix while loop variable hoisting: emit hoisted declarations as separate
  statements in parent scope instead of wrapping in a Block, preventing
  variables from going out of scope before return
- Handle InstanceCall in computeExprType (LaurelTypes) and isFunction
  (Resolution) to support instance procedure nodes
- Add test_instance_call_result.py; flip test_while_var_scope to success
- Mark test_slice, test_tuple_for, test_augassign as expected failures
  (void list_items return assigned to variable causes arity mismatch)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Handle void procedures (0 outputs) in Core translator: emit call
  without assignment target when assigning result of a void procedure
- Fix HighType.Top → HighType.Unknown for InstanceCall fallback
- Fix translateCombinedLaurel API usage (returns tuple, not Except)
- Reorganize test expectations: all 26 tests now pass with no
  expected failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…atures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace custom fmtHighType/fmtParam with Laurel.formatParameter and
Laurel.formatStmtExprVal from LaurelFormat.lean. Update guard_msgs
expectations to validate preconditions (e.g. requires Any..isfrom_string(x))
so the type-to-precondition mapping is tested end-to-end.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace anyTesterForType (string list) with anyPreconditionForParam
(monadic, returns StmtExpr). Optional types now generate disjunctions:
  requires Any..isfrom_none(x) | Any..isfrom_string(x)
Add coverage for TReal (isfrom_float), UserDefined (isfrom_ClassInstance),
ListStr/ListAny. Unrecognized types report a warning via reportError
instead of being silently discarded.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Process each arg in one for-loop: build Any-typed input and precondition
together using Array.push (O(n)) instead of intermediate lists with
filterMap and append (O(n²)). Remove unused argToParameter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 8 test files that reproduce known type errors as expected failures:
- Cat 1: __init__ not found (class constructors with/without kwargs)
- Cat 2: Any vs Composite type mismatch (str(e) on exception var)
- Cat 3: Box..Any undefined type (class with Any-typed field)
- Cat 4: __enter__ not found (with open statement)
- Cat 5: Unlifted holes (Hole nested in kwarg argument)
- Cat 6: wrapFieldInAny gap (Any-typed class field access)
- Cat 7: Non-modeled function arity mismatch (datetime.now)

Also add DumpCoreTest debugging tool for inspecting Core IR output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants