Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ADR on virtualisation #18

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/hardware_ready/ADRs/virtualisation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: "Virtualisation"
date: "2025-03-17"
---


| status: | date: | decision-makers: |
| --- | --- | --- |
| proposed | 2025-03-17 | Alexandra Aldershaab |


## Context and Problem Statement

One important aspect is to determine whether the clusters should run on an OS directly on the machines, or if it makes sense to add a virtualisation layer.

Running directly on the hardware gives you a 1-1 relationship between the machines and the nodes. This is not always advised if the machines are particularly beefy. Running directly on the hardware will of course have lower latency than when adding a virtualisation layer.

A virtualisation layer can benefit via abstracting the actual hardware, and enable simple zero downtime hardware maintenance.

For larger companies, it is usually possible to provision VM's from the IT department, and in that case the clear recommendation would alway be to just use what is available.

## Considered Options

* KubeVirt
* Incus
* Proxmox

## Decision Outcome

Chosen option: "{title of option 1}", because {justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force {force} | … | comes out best (see below)}.

<!-- This is an optional element. Feel free to remove. -->
### Consequences

* Good, because {positive consequence, e.g., improvement of one or more desired qualities, …}
* Bad, because {negative consequence, e.g., compromising one or more desired qualities, …}
* … <!-- numbers of consequences can vary -->
10 changes: 1 addition & 9 deletions docs/hardware_ready/_index.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,11 @@
---
title: Getting your hardware ready
---
## Virtualisation or bare metal
One important aspect is to determine whether the clusters should run on an OS directly on the machines, or if it makes sense to add a virtualisation layer.

Running directly on the hardware gives you a 1-1 relationship between the machines and the nodes. This is not always advised if the machines are particularly beefy. Running directly on the hardware will of course have lower latency than when adding a virtualisation layer.

A virtualisation layer can benefit via abstracting the actual hardware, and enable simple zero downtime hardware maintenance.

In case virtualisation is chosen, the below recommendations are what you would run in your VM. For setting up your VM’s we recommend Talos with KubeVirt.

## Decision Matrix
| Problem domain | Description | Reason for importance | Tool recommendation |
|:---:|:---:|:---:|:---:|
| Kubernetes Node Operating System | The Operating System running on each of the hosts that will be part of your Kubernetes cluster | Choosing the right OS will be the foundation for building a production-grade Kubernetes cluster | [Talos OS](hardware_ready/ADRs/talos_as_os.md) |
| Storage solution | The underlying storage capabilities which Kubernetes will leverage to provide persistence for stateful workloads | Choosing the right storage solution for your clusters needs is important as there is a lot of balance tradeoffs associated with it, e.g redundancy vs. complexity | |
| Container Runtime (CRI) | The software that is responsible for running containers | You need a working container runtime on each node in your cluster, so that the kubelet can launch pods and their containers | |
| Network plugin (CNI) | Plugin used for cluster networking | A CNI plugin is required to implement the Kubernetes network model | [Cilium](Cilium_as_network_plugin.md) |
| Virtualisation | An optional layer between your hardware and your Kubernetes tech stack | In some scenarioes it might be benefitial to abstract the underlying hardeware away, and have everything running in virtual machines | [KubeVirt??](virtualisation.md) |