Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Docker optimisations for CI #247

Open
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Jan 20, 2025

Building on #233 by using the ghcr.io for caching. Will likely need permissions updates to get working properly, plus a few rounds of debugging in the CI

@ml-evs ml-evs force-pushed the ml-evs/docker-optimisations branch from 2982d3a to fe86383 Compare January 20, 2025 15:03
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.21%. Comparing base (89e3d7c) to head (f9b960a).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #247   +/-   ##
========================================
  Coverage    71.21%   71.21%           
========================================
  Files           50       50           
  Lines         6749     6749           
  Branches      1083     1083           
========================================
  Hits          4806     4806           
  Misses        1576     1576           
  Partials       367      367           

@ml-evs
Copy link
Member Author

ml-evs commented Jan 20, 2025

Hi @gpetretto, this PR seems to be working ok and cuts out a chunk of a build time. I pushed my local build and cache to the registry, so they should be usable by everyone locally too, and the CI now makes much better use of the GHA cache. I'm still not entirely sure whether its actually caching between the two different build jobs, or if they now just have their own cache, so we might also be able to remove the build-test-bench job at some point. I'll wait until the other PR is merged to tidy this one up!

@ml-evs ml-evs changed the title [WIP] docker optimisations [WIP] Docker optimisations for CI Jan 20, 2025
@gpetretto
Copy link
Contributor

Hi @ml-evs, this looks great! Thanks!

I have tested this locally and it works nicely, except for the issue that I have mentioned in the other PR #233 (comment): Slurm and SGE work only if I emulate am64 with Rosetta and PBS works only if I emulate with Docker VMM. Anyway, it seems this is just an issue with my setup. If I can't find a better solution, I will probably introduce an option for pytest to set the WORKERS variable in the integration tests to make my life easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants