Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow matmul sweeps to be run locally as well #18800

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

nsorabaTT
Copy link

Ticket

Link to Github Issue

Problem description

Currently, matmul test suites can only be launched via sweeps_parameter_generator that uses custom harness including elastic cloud. This PR extends the change in previous PR that added the ability to locally run the tests using pytest.

What's changed

  1. Create test_ functions that pytest can recognize and execute automatically.
  2. Tie test_ to parameters declared as a dictionary
  3. Keep a run function that passes all arguments to retain compatibility with existing testing infra.

Checks

  1. pytest generated sweeps correlates with tests generated by sweep framework.

Known issue

Pytest handles all None values to be unique. This results in 1% additional duplicate tests in pytest. This is to be addressed in a separate PR as a low priority item.

Checklist

@nsorabaTT nsorabaTT marked this pull request as ready for review March 7, 2025 20:42
@nsorabaTT nsorabaTT self-assigned this Mar 7, 2025
@nsorabaTT nsorabaTT linked an issue Mar 7, 2025 that may be closed by this pull request
Copy link
Contributor

@bbradelTT bbradelTT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the description:

  1. For Checks, please provide numbers
  2. For the None issue, please elaborate with a brief example.

The values of the inner dictionaries are lists of parameter values to be used for sweep.
Returns:
A dictionary of pytest parametrize arguments.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please provide a short example of input and output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create ability to run all matmul sweeps locally
2 participants