Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase docker caching in workflows #224

Open
1 task
ericbuckley opened this issue Feb 25, 2025 · 0 comments
Open
1 task

increase docker caching in workflows #224

ericbuckley opened this issue Feb 25, 2025 · 0 comments
Labels
automation Automation of processes

Comments

@ericbuckley
Copy link
Collaborator

ericbuckley commented Feb 25, 2025

Summary

On Apr 1, Docker will implement a change to reduce unauthenticated requests to just 10 pulls an hour. We need to be sure that our workflows are caching docker images as much as possible to reduce/prevent our chances of running into 429 failures.

Acceptance Criteria

  • Adjustments to the github action workflows to improve docker cache hits

Details / Tasks

Areas to consider for improvements

  • The smoke test job uses the docker build ... command to create an image to run. This likely hits Docker hub to retrieve the python:3.12-slim image every time. Consider using docker/build-push-action to make use of caching.
  • Our testing workflows depend on database services that use Docker hub images. It's likely that Github is caching some of these images, but in the case of an image like ....-latest it's probably having to check Docker hub every time to see if the image has changed. Maybe pinning these services to tags that don't change will prevent that (eg postgres:17 -> postgres:17.4-bookwarm)? Another alternative is to use a github mirror, that is copy the images we want into a repo on ghrc.io/cdcgov/recordlinkage, then have our workflow services reference those images.
  • Also, as part of an optimization, investigate whether the buildx step is necessary in the docs job or the release workflow.
@ericbuckley ericbuckley added the automation Automation of processes label Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation Automation of processes
Projects
None yet
Development

No branches or pull requests

1 participant