Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve compatibility for python syntax in AST rewrite #249

Merged
merged 12 commits into from
Feb 11, 2025

Conversation

vineetg3
Copy link
Contributor

@vineetg3 vineetg3 commented Feb 6, 2025

Resolve #222

@vineetg3 vineetg3 self-assigned this Feb 10, 2025
@@ -69,7 +102,7 @@ def visit_FunctionDef(self, node):
name=node.name,
args=node.args,
body=prefix + node.body,
decorator_list=node.decorator_list,
decorator_list=decorator_list,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug fix: Since we need to remove init_pydough_context, the updated deco list needs to be sent.

@vineetg3
Copy link
Contributor Author

Resolve #222

This PR modifies the AST visitor to support more python syntax. Here are examples of syntax that is supported now:
Here’s the updated list with meaningful headings instead of the test function names:

  1. Variable-Length Arguments with *args and **kwargs: Handling variable-length arguments and keyword arguments in function definitions.
    Ex:

    def impl(*args, **kwargs):
        terms = {}
  2. Tuple Unpacking: Assigning multiple values from a tuple in a single statement.
    Ex:

    start, end = (1992, 1994)
  3. Unpacking Nested Structures: Unpacking complex nested structures like lists or tuples.
    Ex:

    a, (b, c) = ["GERMANY", ["FRANCE", "ARGENTINA"]]
  4. Dynamic Pairing with zip: Unpacking paired values dynamically within an iterable using zip.
    Ex:

    for i, j in zip(range(5), range(1992, 1997)):
        terms[f"c{i}"] = COUNT(orders.WHERE(YEAR(order_date) == j))
  5. Context Managers with with: Managing multiple resources using a with block and importing modules with aliases.
    Ex:

    with tf.TemporaryFile() as tf_handle1, tf.TemporaryFile() as tf_handle2:
        tf_handle1.write(b"Canada")
  6. Exception Handling: Using try, except, and finally blocks to handle exceptions.
    Ex:

    try:
        raise Exception("Canada")
  7. Object-Oriented Programming: Using classes to encapsulate logic and data.
    Ex:

    class Customer:
        def __init__(self, countries):
            self._countries = countries
  8. Dynamic Term Generation with Loops: Using a loop and dictionary to dynamically generate terms.
    Ex:

    for i in range(3):
        terms[f"interval_{i}"] = COUNT(
            customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000))
        )
  9. Reusable Logic with Functions: Defining reusable logic to generate terms using regular functions.
    Ex:

    def interval_n(n):
        return COUNT(customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000)))
  10. Shadowing Variables in Functions: Using function arguments that overlap with collection fields.
    Ex:

    def interval_n(n, name="test"):
        return COUNT(customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000)))
  11. Lambda Functions for Concise Logic: Using a lambda function for one-line anonymous logic.
    Ex:

    interval_n = lambda n: COUNT(
        customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000))
    )
  12. Dictionary Comprehensions: Dynamically populating dictionaries using comprehensions.
    Ex:

    terms.update(
        {
            f"interval_{i}": COUNT(
                customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000))
            )
            for i in range(3)
        }
    )
  13. List Comprehensions: Generating and extending lists using comprehensions.
    Ex:

    terms.extend(
        [
            COUNT(customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000)))
            for i in range(3)
        ]
    )
  14. Set Comprehensions for Unique Terms: Generating unique terms with a set comprehension and sorting.
    Ex:

    terms.extend(
        set(
            {
                COUNT(customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000)))
                for i in range(3)
            }
        )
    )

@vineetg3 vineetg3 marked this pull request as ready for review February 10, 2025 20:28
Copy link
Contributor

@knassre-bodo knassre-bodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fantastic @vineetg3 !!! Just a few things before I'll approve.

tests/test_unqualified_node.py Outdated Show resolved Hide resolved
pydough/unqualified/unqualified_transform.py Outdated Show resolved Hide resolved
pydough/unqualified/unqualified_transform.py Show resolved Hide resolved
pydough/unqualified/unqualified_transform.py Show resolved Hide resolved
@knassre-bodo
Copy link
Contributor

Resolve #222

This PR modifies the AST visitor to support more python syntax. Here are examples of syntax that is supported now: Here’s the updated list with meaningful headings instead of the test function names:

  1. Variable-Length Arguments with *args and **kwargs: Handling variable-length arguments and keyword arguments in function definitions.
    Ex:
    def impl(*args, **kwargs):
        terms = {}
  2. Tuple Unpacking: Assigning multiple values from a tuple in a single statement.
    Ex:
    start, end = (1992, 1994)
  3. Unpacking Nested Structures: Unpacking complex nested structures like lists or tuples.
    Ex:
    a, (b, c) = ["GERMANY", ["FRANCE", "ARGENTINA"]]
  4. Dynamic Pairing with zip: Unpacking paired values dynamically within an iterable using zip.
    Ex:
    for i, j in zip(range(5), range(1992, 1997)):
        terms[f"c{i}"] = COUNT(orders.WHERE(YEAR(order_date) == j))
  5. Context Managers with with: Managing multiple resources using a with block and importing modules with aliases.
    Ex:
    with tf.TemporaryFile() as tf_handle1, tf.TemporaryFile() as tf_handle2:
        tf_handle1.write(b"Canada")
  6. Exception Handling: Using try, except, and finally blocks to handle exceptions.
    Ex:
    try:
        raise Exception("Canada")
  7. Object-Oriented Programming: Using classes to encapsulate logic and data.
    Ex:
    class Customer:
        def __init__(self, countries):
            self._countries = countries
  8. Dynamic Term Generation with Loops: Using a loop and dictionary to dynamically generate terms.
    Ex:
    for i in range(3):
        terms[f"interval_{i}"] = COUNT(
            customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000))
        )
  9. Reusable Logic with Functions: Defining reusable logic to generate terms using regular functions.
    Ex:
    def interval_n(n):
        return COUNT(customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000)))
  10. Shadowing Variables in Functions: Using function arguments that overlap with collection fields.
    Ex:
    def interval_n(n, name="test"):
        return COUNT(customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000)))
  11. Lambda Functions for Concise Logic: Using a lambda function for one-line anonymous logic.
    Ex:
    interval_n = lambda n: COUNT(
        customers.WHERE(MONOTONIC(n * 1000, acctbal, (n + 1) * 1000))
    )
  12. Dictionary Comprehensions: Dynamically populating dictionaries using comprehensions.
    Ex:
    terms.update(
        {
            f"interval_{i}": COUNT(
                customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000))
            )
            for i in range(3)
        }
    )
  13. List Comprehensions: Generating and extending lists using comprehensions.
    Ex:
    terms.extend(
        [
            COUNT(customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000)))
            for i in range(3)
        ]
    )
  14. Set Comprehensions for Unique Terms: Generating unique terms with a set comprehension and sorting.
    Ex:
    terms.extend(
        set(
            {
                COUNT(customers.WHERE(MONOTONIC(i * 1000, acctbal, (i + 1) * 1000)))
                for i in range(3)
            }
        )
    )

Btw, make sure to include this comment of yours in the merge commit. These are FANTASTIC details to include.

Copy link
Contributor

@knassre-bodo knassre-bodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vineetg3 vineetg3 merged commit 621ceb4 into main Feb 11, 2025
5 checks passed
@vineetg3 vineetg3 deleted the vineet/python_syntax branch February 11, 2025 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure PyDough code is compatible with full Python syntax including function definitions/lambdas
2 participants