Add Core/QA grouped-tests CI structure#40
Merged
ChrisRackauckas merged 2 commits intoJun 8, 2026
Conversation
Convert the test suite to the canonical SciML grouped-tests layout: - runtests.jl becomes a GROUP dispatcher. The existing tests are moved verbatim into test/core.jl and run under GROUP=Core (and the default GROUP=All). - New QA group: test/qa/Project.toml ([deps] Aqua + JET, [sources] ConcreteStructs path="../..") and test/qa/qa.jl running Aqua.test_all + JET.test_package(target_defined_modules=true), dispatched under GROUP=QA. - Root test/test_groups.toml encodes the matrix: Core on [lts, 1, pre], QA on [lts, 1]. - .github/workflows/ci.yml: the matrix test job is replaced by a thin caller of SciML/.github grouped-tests.yml@v1. The docs job is kept (no standalone Documentation workflow exists). The redundant runic/spellcheck/downgrade jobs are dropped because standalone FormatCheck.yml, SpellCheck.yml, and Downgrade.yml already provide them. Core group verified locally (julia 1.11): 39 tests pass. The QA group surfaces pre-existing package findings (Aqua project/compat metadata and a JET-detected `nerror` typo in src); these are reported for triage, not silenced. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aqua.test_all surfaced two legitimate metadata findings on the root Project.toml: - deps_compat (extras): `Test` was listed in [extras] without a [compat] entry. Add `Test = "1"`. - project_extras: the root declared `julia = "1.1"`, which makes Aqua enforce root/test project consistency for packages supporting Julia <= 1.1, so it flagged `LinearAlgebra` + `Suppressor` (test-only deps in test/Project.toml) as missing from the root [extras]. Bump the root [compat] julia floor to "1.10" (the LTS / SciML org minimum), which both conforms to convention and lifts the <= 1.1 consistency check. The test-only deps are intentionally NOT added to the root [extras]. Verified locally on Julia 1.11 (test/qa env): Aqua.test_all(ConcreteStructs) now reports 11/11 passing (was 9/2). JET still fails on this branch due to the latent `nerror` typo; that is fixed separately in the fix-nerror-typo PR. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 9, 2026
ChrisRackauckas
added a commit
that referenced
this pull request
Jun 16, 2026
The QA group's JET linting (added in #40) reports four "local variable may be undefined" errors that fail CI on main: - `_parse_head(head::Expr)`: the `if/elseif` over `head.head` had no fallthrough, so `struct_name`, `type_params`, and `super` were undefined for any other head. Add an `else` that errors, matching the existing invalid-usage handling elsewhere in the file. This makes the function total and removes three of the JET findings. - `_parse_line(line::Expr)`: `val` was bound only inside `if assignment` but its binding is visible at the later `if assignment` read site, so JET cannot prove it defined. Initialize `val = nothing` alongside the existing `annotation = nothing`; behavior is unchanged since `val` is only read when `assignment` is true. Verified locally on Julia 1.12 (CI's `julia 1`): the QA group now passes (Aqua 11/11, JET 1/1, zero findings). Core group still passes 40/40 on Julia 1.10, including the invalid-usage assertion. Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Converts ConcreteStructs.jl to the canonical SciML Core/QA grouped-tests structure. This repo had no QA group; this adds one from scratch and moves the existing CI test matrix into a
test/test_groups.toml-driven thin caller.What changed
test/runtests.jl. The existing test suite is moved verbatim intotest/core.jl(byte-identical to the oldruntests.jl) and runs underGROUP=Coreand the defaultGROUP=All.runtests.jlis now a small dispatcher.test/qa/Project.toml([deps]Aqua + JET,[sources] ConcreteStructs = {path = "../.."}) andtest/qa/qa.jlrunningAqua.test_all(ConcreteStructs)+JET.test_package(ConcreteStructs; target_defined_modules = true), dispatched underGROUP=QA. The dispatcher activates/instantiatestest/qaandPkg.develops the repo path so the QA group tests the PR branch on Julia 1.10 too ([sources]is ignored pre-1.11).test/test_groups.tomlencoding the matrix:[Core]on["lts", "1", "pre"],[QA]on["lts", "1"]..github/workflows/ci.yml— the hand-maintained version matrix test job is replaced by a thin caller ofSciML/.github/.github/workflows/grouped-tests.yml@v1. Standardconcurrencyblock +paths-ignore: ['docs/**']added; existingmainbranch + tag triggers preserved.docsjob is kept (there is no standalone Documentation workflow in this repo).runic/spellcheck/downgradejobs insideci.ymlwere dropped because standaloneFormatCheck.yml,SpellCheck.yml, andDowngrade.ymlalready provide those checks — no coverage is lost, the duplicate runs are removed. No other workflow files were touched.Local QA run (julia 1.11) — reported honestly
Core group: PASS — 39 tests (Unit 23, End-to-end 11, Issue #3 1, Commutation 4), all green via the new dispatcher.
QA group: surfaces 3 genuine pre-existing findings (not silenced, not worked around). These are package issues, not artifacts of this PR — reporting for triage rather than fixing here:
Aqua.test_all — 9 passed, 2 failed:
Compare Project.toml and test/Project.toml(project_extras): the rootProject.tomldeclaresjulia = "1.1", so Aqua requires root/test project consistency;test/Project.tomlcarriesLinearAlgebra+Suppressortest deps absent from the root[extras].Compat bounds → extras: rootProject.tomllistsTestin[extras]but declares no[compat]entry forTest.These stem from the repo carrying both a
[extras]/[targets]block in the rootProject.tomland a separatetest/Project.toml.JET.test_package — 1 possible error:
src/ConcreteStructs.jl:254returnsnerror("Invalid usage of @concrete.")—nerroris undefined and is almost certainly a typo forerror(...). It only fires on the invalid-input error path, so the existing tests never hit it. This is a real (latent) source bug; left for @ChrisRackauckas to decide on the fix.🤖 Generated with Claude Code