Epic: Productionize manual live migrations

[stradig]

Motivation

We want to be able drain nodes without service interruption in order to shrink Kubernetes clusters to save cost.

DoD

Admins can manually live migrate computes without breaking connections.

Implementation ideas

Investigate if #828 can be helpful.

Tasks

Tasks

Preview Give feedback

1. https://github.com/neondatabase/cloud/issues/22506
2. Audit VMMigration controller code #1233
   a/tech_debt c/autoscaling/neonvm +2
3. Do semi-automated VM migration tests on staging #1234
   +0
4. Feature: Auto-deletion of VirtualMachineMigrations #504
   c/autoscaling/neonvm t/feature +2
5. Implement VirtualMachineMigration webhook validators #719
   c/autoscaling/neonvm t/bug +2

Other related tasks, Epics, and links

• https://github.com/neondatabase/cloud/issues/5820
• Next step: Epic: Productionize automated live migration #834

[Omrigan]

Done:

• https://github.com/neondatabase/cloud/issues/22506
• neonvm-controller: add some migration unit tests #1269
• Fix routing for the overlay network

In progress:

• neonvm-controller: make IPAM code production-ready #1278
• Audit VMMigration controller code #1233

Planned:

• https://github.com/neondatabase/cloud/issues/24831
• Implement VirtualMachineMigration webhook validators #719
• Do semi-automated VM migration tests on staging #1234

What the next milestone?

Get overlay network to work, and verify it does not regress performance or stability.

When do we expect to finish?

At this point, the optimistic estimate is mid March, but a lot of it is “try it and see where it breaks, and then fix it”, and it is hard to predict when this process converges.