Add automatic benchmarking #1062
Conversation
benchmarks/data/input_full.yaml
Outdated
There was a problem hiding this comment.
Is it necessary to add another input file?
There was a problem hiding this comment.
We have a small chicken-egg thing with default, plan it to move to that when it catches up to main
.github/workflows/benchmark.yaml
Outdated
There was a problem hiding this comment.
Since this would ultimately go into the docs, you might consider combining this whole job in the deploy-pages workflow
There was a problem hiding this comment.
You know, that would work! Nice idea, @misi9170 ok with you?
Yes, it makes a nice interactive chart, I think we have to do this first merge for it to exist right, then I can do a second PR linking it into the docs |
|
Is it intentional to merge into |
If you have it locally, a screenshot would be nice. It's just tough to review not knowing what it produces. |
Intentional, it'll be a process pointed at main, which now reminds me why I can't combine it with the publish pages, which points at develop. It has to do with the fact that it can run from a fork, so only pushes to main work as a trigger |
|
|
@rafmudaf , ok @misi9170 and I talked and made some changes based on your comments and discussing here:
|
| destination_dir: docs # Publishes to the docs folder | ||
|
|
||
|
|
||
| jobs: |
There was a problem hiding this comment.
Does this work? I would have guessed you can't have two "job" blocks
There was a problem hiding this comment.
|
@paulf81 Some things that would help tidy this up and make a review easier are:
|
Add automatic benchmarking to FLORIS
The PR will supercede #1060 (which itself superceded #992) and seeks to avoid issues running some of the github actions from a fork and further makes the changes direct to main to make sure the changes to gh-pages work together. This shouldn't be an issue going forward since the action is specified to run on develop at a specific time and is not connected to pulls or pushes.
Adds automatic code benchmarking to FLORIS. Proposed solution is to use pytest-benchmark to implement set timing tests:
https://pytest-benchmark.readthedocs.io/en/latest/
https://github.com/ionelmc/pytest-benchmark
And then try to schedule some semi-daily execution of these tests with logged performance checks so we can track changes over time. Here focused on:
https://github.com/benchmark-action/github-action-benchmark
To this end I added a first test to the tests folder including benchmarking to the
tests/folder and confirm the command line:pytest floris_benchmark_test.pyThis first version includes only a few tests with the idea with once the framework is up and running more possible to add future improvements. This PR is a bit in the way of other PRs so descoping a bit to get it done.