Update CI to run a11y tests on more browser / OS combos #2095
Labels
tag: accessibility
Issues related to accessibility issues or efforts
tag: CI
Pull requests that update GitHub Actions code
Comments
|
We could expand the matrices for the a11y tests. Still, IIRC, when we first introduced them, we decided to keep them as minimal as possible to avoid significant disruptions to the contributor workflow, especially as we worked on accessibility remediation. Separately, it seems that in the PR linked, there are indeed two tests failing.
see https://github.com/pydata/pydata-sphinx-theme/pull/2093/checks#step:4:1550 |
|
🤔 ignore what I said about the failing tests (I will follow up on the corresponding PR). In the meantime, since I already have a WIP PR for CI improvements, I expanded the a11y testing matrix. However, I separated these from the overall CI as we already have a lot of tests going on there. If you are interested, these are already working at https://github.com/pydata/pydata-sphinx-theme/actions/runs/12790504035 (though now some tests are failing in Firefox, so I need to look into those) |
Yes I was going to say "pretty sure I saw those tests fail on Thank you expanding the a11y test matrix. It's a very compelling aspect of the value-add of pydata-sphinx-theme |
trallard
added
tag: accessibility
3 tasks
drammock
added a commit
that referenced
this issue
Mar 4, 2025
I thought I would do some winter/summer cleaning (depending on your location) ⛄🌞. This PR adds several improvements/updates to our CI with a focus on improving the contributor experience and security Details below: ## 🔒 Security focused - Use SHA for third-party actions and our internal action for development setup - Replace potentially [dangerous trigger `workflow_run`](https://woodruffw.github.io/zizmor/audits/#dangerous-triggers) for `workflow_call` and use as a reusable workflow - Create and use a dedicated environment for releases: - [x] Created a `pst-release` environment in the repo (restricted to `main` only) - [x] Use `pst-release` for our `release-PST` step in `publish.yml` - [ ] Add `pst-release` as the default env in PyPI - Add a `zizmor.yml` workflow to run static analysis on our GH workflows - Add explicit `persist-credentials: false` to relevant actions (where we do not need further git operations) ## 👩🏽🎤 Contributor experience - Prevent the `pre-release.yml` workflow from running in repos not under the `pydata` org (forks) - Our CI workflow has grown significantly with the various tests and checks. This PR splits it into: - `CI.yml`: pytest, a11y-tests, profiling, coverage - `docs.yml`: docs-related checks like building across OSes and Python and Sphinx versions, check for broken links (new, note that I had to fix some broken links to get this in 🙈 and there seem to be still some others to fix) - Add `tox run -e docs-linkcheck` to check for broken links in our docs - Add workflow to add a link to the RTD docs preview in PRs. ## 🧰 Maintenance - Adds Python `3.13` to our testing matrices (`3.12` is left as the `default` until we are confident all is ok with `3.13`) - Add an explicit `ubuntu-22.04` target as `ubuntu-latest` will soon be `24.04` (being rolled out right now) -> I think I might actually have explicit versions on both and only change to latest (or not) when the rollout is completed ---- ## Questions / notes - @drammock, we have `"sphinx-theme-builder @ https://github.com/pradyunsg/sphinx-theme-builder/archive/87214d0671c943992c05e3db01dca997e156e8d6.zip",` in our `project. tool` and `tox.ini`. I do not believe this pin is needed anymore, so I would like to remove it, too. WDYT? - @drammock did you create the token for Anaconda.org? I would like to make this an environment secret (vs a repository secret as it is right now) - Also, while adding a new environment, I noticed a `github-pages` environment that I do not think we are using, so I'd like to delete it. - Finally, I deleted a leftover `PYPI_TOKEN,` which should have been removed when we changed to trusted publishers. Closes #2095 --------- Co-authored-by: Daniel McCloy <[email protected]>
gabalafou
pushed a commit
to gabalafou/pydata-sphinx-theme
that referenced
this issue
Mar 10, 2025
I thought I would do some winter/summer cleaning (depending on your location) ⛄🌞. This PR adds several improvements/updates to our CI with a focus on improving the contributor experience and security Details below: ## 🔒 Security focused - Use SHA for third-party actions and our internal action for development setup - Replace potentially [dangerous trigger `workflow_run`](https://woodruffw.github.io/zizmor/audits/#dangerous-triggers) for `workflow_call` and use as a reusable workflow - Create and use a dedicated environment for releases: - [x] Created a `pst-release` environment in the repo (restricted to `main` only) - [x] Use `pst-release` for our `release-PST` step in `publish.yml` - [ ] Add `pst-release` as the default env in PyPI - Add a `zizmor.yml` workflow to run static analysis on our GH workflows - Add explicit `persist-credentials: false` to relevant actions (where we do not need further git operations) ## 👩🏽🎤 Contributor experience - Prevent the `pre-release.yml` workflow from running in repos not under the `pydata` org (forks) - Our CI workflow has grown significantly with the various tests and checks. This PR splits it into: - `CI.yml`: pytest, a11y-tests, profiling, coverage - `docs.yml`: docs-related checks like building across OSes and Python and Sphinx versions, check for broken links (new, note that I had to fix some broken links to get this in 🙈 and there seem to be still some others to fix) - Add `tox run -e docs-linkcheck` to check for broken links in our docs - Add workflow to add a link to the RTD docs preview in PRs. ## 🧰 Maintenance - Adds Python `3.13` to our testing matrices (`3.12` is left as the `default` until we are confident all is ok with `3.13`) - Add an explicit `ubuntu-22.04` target as `ubuntu-latest` will soon be `24.04` (being rolled out right now) -> I think I might actually have explicit versions on both and only change to latest (or not) when the rollout is completed ---- ## Questions / notes - @drammock, we have `"sphinx-theme-builder @ https://github.com/pradyunsg/sphinx-theme-builder/archive/87214d0671c943992c05e3db01dca997e156e8d6.zip",` in our `project. tool` and `tox.ini`. I do not believe this pin is needed anymore, so I would like to remove it, too. WDYT? - @drammock did you create the token for Anaconda.org? I would like to make this an environment secret (vs a repository secret as it is right now) - Also, while adding a new environment, I noticed a `github-pages` environment that I do not think we are using, so I'd like to delete it. - Finally, I deleted a leftover `PYPI_TOKEN,` which should have been removed when we changed to trusted publishers. Closes pydata#2095 --------- Co-authored-by: Daniel McCloy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
During review of #2093 we discovered an a11y test that should have failed in CI but did not: #2093 (comment)
The problem is that CI only runs the a11y tests on Ubuntu. If possible, they should run on more OS / browser combos.
The text was updated successfully, but these errors were encountered: