-
Notifications
You must be signed in to change notification settings - Fork 679
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: Generating the e2e test matrix #5763
Conversation
interesting! have you considred nextest's filtersets here as well? we'd still need a matrix, but the pattern matching might be easier than what you're doing here |
After looking into it, I don't think it would help that much. The filterset language doesn't support matching "ignored" only (which is how we indicate these tests), so it would only be helpful to match the binary |
ef5045d
to
4d96cf9
Compare
b187229
to
0bdcd92
Compare
* use nextest list to generate the set of e2e tests for bitcoin-tests (this switches the workflow from being an "inclusion list" to an "exclusion list"). * this changeset matches the current set of tests run in `develop`. it does not try to enable tests that were perhaps overlooked.
0bdcd92
to
b6e5b91
Compare
this is neat! the only thing we should be aware of is it'll be easy to overwhelm the amount of tests in the matrix, so it's something we'll want to keep an eye on over time. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM and opened #5802 for future enhancement.
This PR is a first pass at trying to generate the test matrix automatically. Right now, I'm just trying to make sure that it gets a single test to run using the excludes functions. Once I'm convinced that works, I'll update it to match the current exclusion set.