Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement orchestration management of lab runners #395

Open
ProjectsByJackHe opened this issue Oct 26, 2024 · 0 comments
Open

Implement orchestration management of lab runners #395

ProjectsByJackHe opened this issue Oct 26, 2024 · 0 comments
Labels
lab Specific to lab environment P0 Windows_ES

Comments

@ProjectsByJackHe
Copy link
Collaborator

ProjectsByJackHe commented Oct 26, 2024

Today, our lab uses hyper-V machine checkpoints in order to revert the VM back to a previously known good state.

This is how we are achieving (semi) stateless runners. We are doing this hyper-V revert on a schedule: once every 3 hours.

At the very least, if a PR causes a crash/bugcheck, we can be confident the system will heal without manual intervention.

Still, a 3 hour wait may be sub-optimal. We would like to do a revert prior to every run. That means we need some sort of orchestration system so that concurrent jobs do not step on each other's toes, and we may need to accept some trade-offs in terms of complexity / accept that not every job will have a fresh state if we have multiple concurrent jobs.

@ProjectsByJackHe ProjectsByJackHe added lab Specific to lab environment P0 Windows_ES labels Oct 26, 2024
@ProjectsByJackHe ProjectsByJackHe changed the title Add lifecycle management for lab VMs. Implement orchestration management of lab runners Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lab Specific to lab environment P0 Windows_ES
Projects
Status: No status
Development

No branches or pull requests

1 participant