Cortex-M backend: Make pytest collection seeding deterministic#20623
Cortex-M backend: Make pytest collection seeding deterministic#20623zingo wants to merge 1 commit into
Conversation
Seed the Cortex-M test RNG during pytest configuration in OSS/local test runs so module-level random inputs are deterministic before collection imports test modules. Use TEST_SEED for the Cortex-M test session seed, matching the Arm backend test behavior. Keep packaged non-OSS Buck behavior unchanged by skipping that early torch import there, while preserving the per-test seeding fixture. Signed-off-by: Zingo Andersen <Zingo.Andersen@arm.com> Change-Id: I69a8b6757b1c6960f470861df1c02fc6292f7770
zingo
added
help wanted
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20623
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New FailuresAs of commit 2ece063 with merge base 0567b0a ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
|
Hi @Gasoonjia @digantdesai @rascani Here is a new version of a patch that got reverted (but for Arm backend) due to torch was used to early in your buck2 setup. This version try to guard this with the runtime.is_oss so hopefully this works both on GutHub and internally, but internally you will not get the same nice locked down randomness in the tests but it will work the same way as today. |
There was a problem hiding this comment.
Pull request overview
This PR makes Cortex-M backend pytest runs more deterministic by seeding RNGs during pytest configuration, so module-level random inputs created at import/collection time become reproducible. It also aligns the Cortex-M test seed configuration with the TEST_SEED environment variable and preserves internal/non-OSS Buck behavior by avoiding an early torch import there.
Changes:
- Seed
random+torchRNGs duringpytest_configure(OSS/local pytest runs) and surface the chosen seed in the pytest report header. - Add an autouse fixture that re-seeds per test using the session seed derived from
TEST_SEED(includingTEST_SEED=RANDOM). - Set a Buck test env var to skip early seeding (and the early
torchimport) in packaged non-OSS Buck runs.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| backends/cortex_m/test/TARGETS | Sets an env var for the Buck test target to control whether early RNG seeding (and early torch import) should run. |
| backends/cortex_m/test/conftest.py | Adds session seed setup in pytest_configure, per-test reseeding fixture, and includes seed info in pytest header output. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def pytest_report_header(config): | ||
| return f"cortex-m op-test targets: {', '.join(_selected_targets(config))}" | ||
| return ( | ||
| f"cortex-m op-test targets: {', '.join(_selected_targets(config))} " | ||
| f"{config._test_seed_label}" | ||
| ) |
| def pytest_configure(config): | ||
| seed, seed_label = _setup_random_seed() | ||
| config._test_seed = seed | ||
| config._test_seed_label = seed_label | ||
|
|
| @pytest.fixture(autouse=True) | ||
| def set_random_seed(request): | ||
| """Control random numbers in the Cortex-M test suite. | ||
|
|
||
| By default this uses a fixed seed (0) for reproducible tests. Use | ||
| TEST_SEED to set a custom session seed, or set it to RANDOM to choose a | ||
| random session seed. | ||
| """ | ||
| _set_random_seed(request.config._test_seed) | ||
|
|
Summary
Seed the Cortex-M test RNG during pytest configuration in OSS/local test runs so module-level random inputs are deterministic before collection imports test modules.
Use TEST_SEED for the Cortex-M test session seed, matching the Arm backend test behavior.
Keep packaged non-OSS Buck behavior unchanged by skipping that early torch import there, while preserving the per-test seeding fixture.
cc @digantdesai @freddan80 @per @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani @psiddh @AdrianLundell