Merge pull request #55 from astrofrog/generalize-core-package

Generalize package: configurable core package and flat column list

Author: astrofrog

.github/workflows/integration.yml

@@ -17,14 +17,40 @@ concurrency:
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
+  # Read the `columns` list from packages.yaml and emit it as the job
+  # matrix, so the (python, variant) combinations have a single source
+  # of truth and never need hand-syncing into this workflow.
+  setup:
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    outputs:
+      matrix: ${{ steps.columns.outputs.matrix }}
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
+
+      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405  # v6.2.0
+        with:
+          python-version: '3.12'
+
+      - name: Build matrix from packages.yaml
+        id: columns
+        run: |
+          pip install pyyaml
+          python -c '
+          import json, yaml
+          cols = yaml.safe_load(open("packages.yaml"))["columns"]
+          matrix = [{"python": str(c["python"]), "variant": c["variant"]} for c in cols]
+          print("matrix=" + json.dumps(matrix))
+          ' >> "$GITHUB_OUTPUT"
+
   variant:
+    needs: setup
     strategy:
       fail-fast: false
       matrix:
-        # Keep in sync with `python_versions` in packages.yaml.
-        # (free-threaded builds: use the "t" suffix, e.g. "3.14t".)
-        python: ["3.12", "3.14"]
-        variant: [stable, pre, dev]
+        # Generated by the `setup` job from `columns:` in packages.yaml;
+        # each entry is one {python, variant} object.
+        include: ${{ fromJson(needs.setup.outputs.matrix) }}
     runs-on: ubuntu-latest
     timeout-minutes: 240
     steps:

README.md

@@ -12,43 +12,54 @@ The dashboard is published to
 [astropy.github.io/astropy-integration-testing](https://astropy.github.io/astropy-integration-testing/)
 after each scheduled run.
 
+The harness itself is not astropy-specific: `core_package` in
+`packages.yaml` defines the ecosystem's core package, so the same code
+can drive integration testing for another ecosystem (e.g. sunpy) just
+by pointing the config at a different core package and package list.
+
 How it works
 ------------
 
 The `variant` job runs on a schedule (and on `workflow_dispatch`) as a
-matrix over astropy variant and Python version. The three variants are:
-
-| Variant  | Astropy                                             | Each package                           |
-|----------|-----------------------------------------------------|----------------------------------------|
-| `stable` | Latest non-pre-release on PyPI                      | Latest non-pre-release on PyPI         |
-| `pre`    | Latest including pre-releases (`--prerelease=allow`)| Latest including pre-releases          |
-| `dev`    | Latest dev wheel from the astropy/simple channel    | `git+<repo_url>` (HEAD of main branch) |
-
-Within each matrix job, a single shared venv is built and packages are
-installed one at a time in a deterministic order (coordinated first,
-alphabetical within each tier). If a package can't be installed
-alongside the existing venv (e.g., it pins `astropy<7` but we already
-installed astropy 8), it's skipped and recorded; the rest of the venv
-is untouched. After installs, `pytest --pyargs <module>` runs for each
-package that installed successfully.
-
-The `dashboard` job then downloads the per-matrix-job result JSONs and
-publishes the dashboard to `gh-pages`.
+matrix over the *columns* configured in `packages.yaml`. Each column is
+an independent (Python version, variant) pair — the two axes are not a
+cross-product, so you can test e.g. `3.12 + stable`, `3.13 + pre` and
+`3.14 + dev` if you want. The three variants are:
+
+| Variant  | Core package                                          | Each package                           |
+|----------|-------------------------------------------------------|----------------------------------------|
+| `stable` | Latest non-pre-release on PyPI                        | Latest non-pre-release on PyPI         |
+| `pre`    | Latest including pre-releases (`--prerelease=allow`)  | Latest including pre-releases          |
+| `dev`    | Latest dev wheel from `core_package.dev_index_urls` (or `git+repo_url` if unset) | `git+<repo_url>` (HEAD of main branch) |
+
+Within each matrix job, a single shared venv is built: the core package
+is installed first, then each package one at a time in a deterministic
+order (coordinated first, alphabetical within each tier). If a package
+can't be installed alongside the existing venv (e.g., it pins
+`astropy<7` but we already installed astropy 8), it's skipped and
+recorded; the rest of the venv is untouched. After installs, `pytest
+--pyargs <module>` runs for each package that installed successfully.
+
+A small `setup` job parses `columns:` from `packages.yaml` and emits it
+as the workflow matrix, so the column list has a single source of
+truth. The `dashboard` job then downloads the per-matrix-job result
+JSONs and publishes the dashboard to `gh-pages`.
 
 What's in the repo
 ------------------
 
 | File                                  | Purpose                                              |
 |---------------------------------------|------------------------------------------------------|
-| `packages.yaml`                       | The list of packages tested + `python_versions` to test against. |
-| `astropy_integration/run.py`          | Runs one or more (variant, python) combos: resolve specs, install, test, write `results/<variant>__<python>.json`. |
+| `packages.yaml`                       | The config: `core_package`, the `columns` to test, and the `packages` list. |
+| `astropy_integration/config.py`       | Loads and validates `packages.yaml` (shared by `run` and `dashboard`). |
+| `astropy_integration/run.py`          | Runs one or more columns: resolve specs, install, test, write `results/<variant>__<python>.json`. |
 | `astropy_integration/dashboard.py`    | Reads `results/*.json`, renders `site/index.html` (single self-contained page). |
 | `astropy_integration/cli.py`          | Console entry point that dispatches the `run` and `dashboard` subcommands. |
 | `astropy_integration/status.py`       | Shared status vocabulary (used by both `run` and `dashboard`). |
 | `astropy_integration/templates/`      | HTML/CSS for the dashboard (loaded as package data). |
 | `pyproject.toml`                      | Package metadata; declares the `astropy-integration` console script. |
 | `conftest.py`                         | Repo-root pytest plugin that caps each package to the first `PYTEST_LIMIT_N` tests for PR previews. |
-| `.github/workflows/integration.yml`   | The matrix workflow (variant x python + dashboard).  |
+| `.github/workflows/integration.yml`   | The matrix workflow (`setup` builds the matrix, `variant` runs each column, `dashboard` publishes). |
 | `.github/workflows/preview-link.yml`  | Companion that posts the "View dashboard preview" status check on PRs. |
 | `sunpy_pytest.ini`                    | Custom pytest config referenced by sunpy's `pytest_args` (sunpy's own config requires plugins we don't install). |
 
@@ -78,23 +89,49 @@ python -m http.server -d site 8000
 Results land in `results/<variant>__<python>.json`; the dashboard in
 `site/`. Both directories are gitignored.
 
-Python versions
----------------
+Core package
+------------
+
+`packages.yaml` has a top-level `core_package` block — the package
+installed into the shared venv before everything else:
+
+```yaml
+core_package:
+  pypi_name: astropy
+  module: astropy
+  repo_url: https://github.com/astropy/astropy.git
+  # dev variant: install nightly wheels from these indexes.
+  # omit to install the dev version from git+repo_url instead.
+  dev_index_urls:
+    - https://pypi.anaconda.org/astropy/simple
+    - https://pypi.anaconda.org/liberfa/simple
+```
+
+To retarget the harness at a different ecosystem, point `core_package`
+at that ecosystem's core package and replace the `packages` list.
+`dashboard_title` (also top-level) sets the dashboard's heading.
+
+Columns
+-------
 
-`packages.yaml` has a top-level `python_versions` list (uv notation,
-so `"3.14t"` means the free-threaded 3.14 build):
+`packages.yaml` has a top-level `columns` list. Each column is one
+(Python version, variant) pair and becomes one dashboard column;
+Python version and variant are independent, so list whatever
+combinations you want (uv notation for Python, so `"3.14t"` is the
+free-threaded 3.14 build):
 
 ```yaml
-python_versions:
-  - "3.12"
-  - "3.14t"
+columns:
+  - {python: "3.12", variant: stable}
+  - {python: "3.13", variant: pre}
+  - {python: "3.14t", variant: dev}
 ```
 
-The runner tests every (variant x python_version) combination. The
-dashboard renders Python versions as grouped header columns above the
-three variants. **Keep the `matrix.python` list in
-`.github/workflows/integration.yml` in sync** with this — the CI uses
-its own matrix because GitHub Actions can't read it from YAML directly.
+The runner tests every column; `--variant` / `--python` narrow that to
+a subset. The dashboard groups consecutive columns that share a Python
+version under a spanning header, so a classic `python x variant`
+layout still renders as grouped columns. The CI matrix is generated
+from this list by the `setup` job, so there's nothing to keep in sync.
 
 Adding or disabling a package
 -----------------------------
@@ -118,15 +155,15 @@ Triggering a run from GitHub
 
 1. Actions tab -> `integration-matrix` workflow.
 2. "Run workflow" dropdown -> green button.
-3. The matrix expands to `len(variants) x len(python_versions)`
-   parallel jobs; the `dashboard` job waits for them and publishes
-   to `gh-pages`.
+3. The `setup` job reads `columns:` from `packages.yaml` and the
+   matrix expands to one parallel `variant` job per column; the
+   `dashboard` job waits for them and publishes to `gh-pages`.
 
 PR previews
 -----------
 
-`integration-matrix` also runs on pull requests. Same three-variant
-matrix as the scheduled run, just with a different final step: the
+`integration-matrix` also runs on pull requests. Same column matrix
+as the scheduled run, just with a different final step: the
 `dashboard` job uploads `site/index.html` as a non-zipped artifact
 (`actions/upload-artifact@v7` with `archive: false`) instead of
 publishing to gh-pages. The companion `preview-link` workflow

astropy_integration/config.py

@@ -0,0 +1,69 @@
+"""Load and validate the integration-testing config (``packages.yaml``).
+
+The config has three top-level keys:
+
+  - ``core_package``: the ecosystem's core package (astropy here, but
+    swap it to retarget the whole harness at another ecosystem). It is
+    installed into the shared venv before anything else.
+  - ``columns``: a flat list of ``{python, variant}`` pairs. Each one
+    is a single run and a single dashboard column; Python version and
+    variant are fully decoupled.
+  - ``packages``: the per-package list (one entry = one dashboard row).
+
+Both ``run`` and ``dashboard`` read this file, so the parsing and
+validation live here in one place.
+"""
+
+from pathlib import Path
+
+import yaml
+
+from . import status
+
+
+def _load(path):
+    return yaml.safe_load(Path(path).read_text()) or {}
+
+
+def load_packages(path):
+    return list(_load(path).get("packages", []))
+
+
+def load_core_package(path):
+    """Return the validated ``core_package`` block.
+
+    ``module`` defaults to ``pypi_name`` when not given.
+    """
+    core = _load(path).get("core_package") or {}
+    if not core.get("pypi_name"):
+        raise ValueError(f"{path}: 'core_package.pypi_name' is required")
+    core.setdefault("module", core["pypi_name"])
+    return core
+
+
+def load_columns(path):
+    """Return the validated ``columns`` list as ``[{python, variant}, ...]``.
+
+    Order is preserved: it drives both the run order and the dashboard
+    column order.
+    """
+    raw = _load(path).get("columns") or []
+    columns = []
+    for i, col in enumerate(raw):
+        python = col.get("python")
+        variant = col.get("variant")
+        if not python or not variant:
+            raise ValueError(f"{path}: columns[{i}] needs both 'python' and 'variant'")
+        if variant not in status.VARIANTS:
+            raise ValueError(
+                f"{path}: columns[{i}] has variant '{variant}'; "
+                f"expected one of {', '.join(status.VARIANTS)}"
+            )
+        columns.append({"python": str(python), "variant": variant})
+    if not columns:
+        raise ValueError(f"{path}: at least one entry under 'columns' is required")
+    return columns
+
+
+def load_dashboard_title(path):
+    return _load(path).get("dashboard_title") or "Ecosystem integration matrix"