-
Notifications
You must be signed in to change notification settings - Fork 326
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move benchmark download tool and visualization to the CI #9075
Merged
Merged
Changes from all commits
Commits
Show all changes
94 commits
Select commit
Hold shift + click to select a range
1d31527
modulearize bench_download tool
Akirathan 261a1cf
Move python version control at the beginning
Akirathan 49223b2
Improve error message for dependencies miss
Akirathan d6bc3a4
Extract dataclasses into __init__
Akirathan 33575cf
Add gh.py for GH query handling
Akirathan 05c5591
Do not fail if there are more than 1 artifact in a bench report
Akirathan a1c21c7
Add jinja template functionality to template_render.py
Akirathan 1f6c079
Improve logging
Akirathan fb02b4a
Add simple test case
Akirathan 25063ce
Implement fetch_file from GH
Akirathan 3c37a48
Add cache skeleton
Akirathan ad89214
Add git.py
Akirathan 3f49ea1
Fix Jinja template render
Akirathan c4d1718
gh.fetch_file does not fail
Akirathan 0cf7abd
gh.invoke_gh_api does not fail, and takes method arg
Akirathan 94db479
Add test case to test_gh
Akirathan 07bca0b
Implement remote_cache.py
Akirathan 41f479e
Use readonly remote cache in bench_download script
Akirathan e8c7f71
Fix path in template rendering
Akirathan 9cb2a26
Remove unused imports
Akirathan bc1f372
Remove unused Cache classes
Akirathan 923d870
Move some commont functionality into bench_results and utils
Akirathan 6d3ee25
Improve logging
Akirathan 58cdf01
Add skeleton of website_regen.py
Akirathan 54a9ce2
Add some tests on bench_results
Akirathan fcf1be1
Search bench report artifact by name
Akirathan c50aac7
Add utils.parse_commit_timestamp
Akirathan 8e3ff5c
Optimize imports
Akirathan 762a256
Redirect output of git.clone
Akirathan 1e8caff
Test SyncRemoteCache
Akirathan d401604
Move more common functionality into utils and bench_results
Akirathan 555666b
Add git.pull
Akirathan 96c1cc4
RemoteCache does not have initialize method
Akirathan 1e7c761
Move generate_bench_website to utils
Akirathan 51e1b09
SyncRemoteCache can be initialized with local repo
Akirathan ffe2fab
SyncRemoteCache has html files fields
Akirathan 4ffdadf
Add test_sync_remote_cache_from_local_repo
Akirathan faa9b41
Finish implementation of website_regen
Akirathan 98f0175
generate_website is in its own module
Akirathan f8e1848
Fix tests
Akirathan 220ed1c
Add test_website_regen
Akirathan 36dfbd9
Merge branch 'develop' into wip/akirathan/8857-bench-on-ci
Akirathan 4f7f164
Remove deprecated methods
Akirathan 4c9f463
Metnion how to run tests in the README
Akirathan 0de278c
Skip remote repo cloning test
Akirathan 0868f9a
Move some common functionality into utils
Akirathan 7493c2b
Test fetching stdlib job report
Akirathan 9f737d1
bench_report is searched by all the artifact names
Akirathan 52e6c31
website_regen has dry_run option
Akirathan 6793a4f
Remove deprecated method
Akirathan 313c90a
JobRun and JobReport have json encoding
Akirathan 0a7f4d2
Json files are saved in pretty form
Akirathan 2633e5c
Add Benchmarks upload workflow file
Akirathan de89366
Fix input name in workflow file
Akirathan 7dc9ed7
[WIP] temporarily enable the workflow on all pull requests
Akirathan 7195ffb
Fix repository arg
Akirathan 46ef8ef
Add dry-run step
Akirathan b650ac6
Fix cd
Akirathan c53f854
Update deps
Akirathan 4d7c694
Fix path to engine-benchmark-results repo
Akirathan b4e52bb
Use GH_TOKEn secret
Akirathan 1278ad3
Use GH_TOKEn secret even in dry-run
Akirathan d4b3259
Better error message in git.commit
Akirathan dc1f2dd
Add setup git step
Akirathan 9c23c85
Add even more error messages
Akirathan d640a40
Do not do --dry-run on the CI
Akirathan 6e4c9f0
Run the job every day at 5:30 AM
Akirathan 99286f4
Set the git email to actions-at-github.org
Akirathan e1ee00d
Use the correct token for bench uploadeds
Akirathan 56c04cb
Set the remote url to enable push to engine-benchmark-results repo
Akirathan 6ecbbff
Add optional fields param to gh.invoke_api
Akirathan 51e9cce
Add git.head_commit function
Akirathan ce319dc
Push via GH API, and not via `git push`
Akirathan 01faa53
[WIP] Temporarily disable the Upload benchmarks job
Akirathan d6aa6ca
[WIP] Do plain `git push` in the action
Akirathan 0506edb
Set token in checkout action
Akirathan 3f8b290
Unset extra authentication header
Akirathan 7af7146
Fix field body in gh.invoke_gh_api
Akirathan c081488
experiment
mwu-tow 37473f9
experiment
mwu-tow f52c058
experiment
mwu-tow 05d9083
experiment
mwu-tow aa1d283
Use workflow_run trigger instead of schedule
Akirathan 970e3da
Fix "Set up git" step
Akirathan 010843d
Enable the Upload benchmarks step
Akirathan 7da61c7
Add anchor to the HTML template
Akirathan 9128d33
Revert "Push via GH API, and not via `git push`"
Akirathan d7283ae
Implement backoff in gh api
Akirathan 8753c7c
Add time when the website was generated
Akirathan 3e765d1
Relax the assertion in git.commit
Akirathan 45d68fc
Add more reasonable commit message
Akirathan 76802d3
Fix git.status
Akirathan 9cefc5f
Remove the `on: pull_request` trigger
Akirathan 846e02e
fmt
Akirathan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# This file is manually managed. It is used to upload benchmarks to to the | ||
# https://github.com/enso-org/engine-benchmark-results repository. | ||
|
||
name: Benchmarks upload | ||
on: | ||
workflow_run: | ||
workflows: ["Benchmark Engine", "Benchmark Standard Libraries"] | ||
types: | ||
- completed | ||
jobs: | ||
upload-benchmarks: | ||
name: Upload benchmarks | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout enso repository | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: enso-org/enso | ||
path: enso | ||
- name: Checkout engine-benchmark-results repository | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: enso-org/engine-benchmark-results | ||
path: engine-benchmark-results | ||
token: ${{ secrets.ENSO_BENCHMARK_RESULTS_TOKEN }} | ||
- name: Install dependencies | ||
run: | | ||
sudo apt-get update | ||
sudo apt-get install -y \ | ||
python3 \ | ||
python3-jinja2 \ | ||
python3-numpy \ | ||
python3-pandas | ||
sudo apt-get install -y gh | ||
- name: Set up git | ||
run: | | ||
git config --global user.email "[email protected]" | ||
git config --global user.name "Enso CI Bot" | ||
- name: Upload benchmarks | ||
run: | | ||
cd enso/tools/performance/engine-benchmarks | ||
python3 website_regen.py \ | ||
-v \ | ||
--local-repo ${{ github.workspace }}/engine-benchmark-results | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.ENSO_BENCHMARK_RESULTS_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to point out that the
engine-benchmarks
name is no longer fully correct. The tool supports-s engine
as well as-s stdlib
- e.g. it is not only for engine benchmarks.