Skip to content

Use SciMLTesting v1.2 (folder-based run_tests)#44

Merged
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Jun 14, 2026
Merged

Use SciMLTesting v1.2 (folder-based run_tests)#44
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Converts ConcreteStructs.jl's test suite to the SciMLTesting v1.2 folder-discovery model.

What changed

  • runtests.jl becomes using SciMLTesting; run_tests() (bare folder-discovery).
  • Core split from the monolithic core.jl into self-contained top-level files — unit_tests.jl, end_to_end_tests.jl, issue_3.jl, commutation_tests.jl — each run isolated in its own @safetestset by folder discovery.
  • The previously unconditional "Invalid usage error path" testset becomes a self-contained Core file invalid_usage_tests.jl. It is a @concrete macro-correctness unit test and belongs to Core; CI continues to exercise it on the Core lane.
  • QA stays at test/qa/ with its existing qa.jl + Project.toml sub-env.
  • SciMLTesting + SafeTestsets added to the main test env and the QA sub-env; Pkg dropped from the test env (no test file uses it; the harness owns all Pkg ops). test_groups.toml unchanged.
  • Removed the checked-in test/Manifest.toml (it is gitignored and was stale after the dep change; Pkg re-resolves it fresh).

Verification

  • TOML.parsefile on every Project.toml + test_groups.toml: OK.
  • Meta.parseall on runtests.jl and every moved/new test file: OK.
  • Core's assertion set is preserved exactly: 42 @test/@test_throws before and after (including the moved invalid-usage check).
  • Drove the real SciMLTesting v1.2.0 run_tests folder-discovery dispatcher over every GROUP value: Core runs the 5 top-level files isolated; QA activates the qa/ sub-env and runs qa.jl; All runs Core (QA is its own CI lane per the v1.2 folder semantics). CI dispatches Core and QA as separate matrix lanes from test_groups.toml.
  • All authored/changed Julia files are Runic-clean.

This supersedes the v1.0.0 harness form previously on this branch and the canonicalize-safetestset PR.

Ignore until reviewed by @ChrisRackauckas.

Convert the test suite to the SciMLTesting v1.2 folder-discovery model:

- runtests.jl becomes `using SciMLTesting; run_tests()`.
- Core: the monolithic core.jl is split into self-contained top-level files
  (unit_tests.jl, end_to_end_tests.jl, issue_3.jl, commutation_tests.jl), each
  run isolated in its own @safetestset by folder discovery. The previously
  unconditional "Invalid usage error path" testset becomes a self-contained
  Core file (invalid_usage_tests.jl) — it is a macro-correctness unit test that
  belongs to Core; CI still exercises it on the Core lane.
- QA stays at test/qa/ with its existing qa.jl + Project.toml sub-env.
- SciMLTesting + SafeTestsets added to the main test env and the QA sub-env;
  Pkg dropped from the test env (no test file uses it; the harness owns Pkg
  ops). test_groups.toml unchanged.
- Removed the checked-in test/Manifest.toml: it is gitignored and was stale
  after the dependency change, so Pkg re-resolves it fresh.

Core's assertion set is preserved exactly (42 @test/@test_throws before and
after, including the moved invalid-usage check). Per the v1.2 folder semantics
the QA group is its own CI lane and is not part of the local `All` default.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude ChrisRackauckas-Claude changed the title Use SciMLTesting v1.0.0 (run_tests harness) Use SciMLTesting v1.2 (folder-based run_tests) Jun 14, 2026
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 14, 2026 23:54
@ChrisRackauckas ChrisRackauckas merged commit 480e52f into SciML:main Jun 14, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants