Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow vmtemplate selection based on tags #343

Merged
merged 17 commits into from
Feb 5, 2025
Merged

feat: allow vmtemplate selection based on tags #343

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f60de48
feat: allow template selection based on tags
pborn-ionos Nov 28, 2024
3baa094
revert webhook changes
pborn-ionos Feb 4, 2025
c382058
introduce TemplateSource
pborn-ionos Feb 4, 2025
71428a0
comment on TemplateSource
pborn-ionos Feb 4, 2025
ff449bd
add testcase for VMTemplateNotFound
pborn-ionos Feb 4, 2025
cf1bc58
add test for SourceNode, TemplateID and TemplateSelector being unset
pborn-ionos Feb 4, 2025
003fea5
revert GetNode() changes
pborn-ionos Feb 4, 2025
9c738c7
remove redundant TemplateSelector check
pborn-ionos Feb 4, 2025
8ae6460
update TemplateSelector description
pborn-ionos Feb 4, 2025
78d81b3
introduce ErrTemplateNotFound error
pborn-ionos Feb 5, 2025
8a83be3
add FindVMTemplateByTags test-case with nil vmTags
pborn-ionos Feb 5, 2025
1623b9b
Update api/v1alpha1/proxmoxmachine_types_test.go
pborn-ionos Feb 5, 2025
8844532
make the linter happy
pborn-ionos Feb 5, 2025
0bfb76b
Update proxmoxmachine_types.go
pborn-ionos Feb 5, 2025
b758aba
Update advanced-setups.md
pborn-ionos Feb 5, 2025
f7b3d89
Update proxmoxmachine_types.go
pborn-ionos Feb 5, 2025
6687247
refaormat
mcbenjemaa Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion api/v1alpha1/proxmoxcluster_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,9 @@ func defaultCluster() *ProxmoxCluster {
ProxmoxMachineSpec: map[string]ProxmoxMachineSpec{
"controlPlane": {
VirtualMachineCloneSpec: VirtualMachineCloneSpec{
SourceNode: "pve1",
TemplateSource: TemplateSource{
SourceNode: "pve1",
},
},
},
},
Expand Down
36 changes: 33 additions & 3 deletions api/v1alpha1/proxmoxmachine_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,8 +155,8 @@ const (
TargetStorageFormatVmdk TargetFileStorageFormat = "vmdk"
)

// VirtualMachineCloneSpec is information used to clone a virtual machine.
type VirtualMachineCloneSpec struct {
// TemplateSource defines the source of the template VM.
type TemplateSource struct {
// SourceNode is the initially selected proxmox node.
// This node will be used to locate the template VM, which will
// be used for cloning operations.
Expand All @@ -173,12 +173,22 @@ type VirtualMachineCloneSpec struct {
// will be cloned onto the same node as SourceNode.
//
// +kubebuilder:validation:MinLength=1
SourceNode string `json:"sourceNode"`
// +optional
SourceNode string `json:"sourceNode,omitempty"`

// TemplateID the vm_template vmid used for cloning a new VM.
// +optional
TemplateID *int32 `json:"templateID,omitempty"`

// TemplateSelector defines MatchTags for looking up VM templates.
// +optional
pborn-ionos marked this conversation as resolved.
Show resolved Hide resolved
TemplateSelector *TemplateSelector `json:"templateSelector,omitempty"`
pborn-ionos marked this conversation as resolved.
Show resolved Hide resolved
}

// VirtualMachineCloneSpec is information used to clone a virtual machine.
type VirtualMachineCloneSpec struct {
TemplateSource `json:",inline"`

// Description for the new VM.
// +optional
Description *string `json:"description,omitempty"`
Expand Down Expand Up @@ -213,6 +223,16 @@ type VirtualMachineCloneSpec struct {
Target *string `json:"target,omitempty"`
}

// TemplateSelector defines MatchTags for looking up VM templates.
type TemplateSelector struct {
// Specifies all tags to look for, when looking up the VM template.
// Passed tags must be an exact 1:1 match with the tags on the template you want to use.
// If multiple VM templates with the same set of tags are found, provisioning will fail.
//
// +kubebuilder:validation:MinItems=1
65278 marked this conversation as resolved.
Show resolved Hide resolved
MatchTags []string `json:"matchTags"`
}

// NetworkSpec defines the virtual machine's network configuration.
type NetworkSpec struct {
// Default is the default network device,
Expand Down Expand Up @@ -526,6 +546,8 @@ type ProxmoxMachine struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

// +kubebuilder:validation:XValidation:rule="[has(self.sourceNode), has(self.templateSelector)].exists_one(c, c)",message="must define either SourceNode with TemplateID, OR TemplateSelector"
// +kubebuilder:validation:XValidation:rule="[has(self.templateID), has(self.templateSelector)].exists_one(c, c)",message="must define either SourceNode with TemplateID, OR TemplateSelector."
// +kubebuilder:validation:XValidation:rule="self.full && self.format != ''",message="Must set full=true when specifying format"
Spec ProxmoxMachineSpec `json:"spec,omitempty"`
Status ProxmoxMachineStatus `json:"status,omitempty"`
Expand Down Expand Up @@ -566,6 +588,14 @@ func (r *ProxmoxMachine) GetTemplateID() int32 {
return -1
}

// GetTemplateSelectorTags get the tags, the desired vm template should have.
func (r *ProxmoxMachine) GetTemplateSelectorTags() []string {
if r.Spec.TemplateSelector != nil {
return r.Spec.TemplateSelector.MatchTags
}
return nil
}

// GetNode get the Proxmox node used to provision this machine.
func (r *ProxmoxMachine) GetNode() string {
return r.Spec.SourceNode
Expand Down
35 changes: 26 additions & 9 deletions api/v1alpha1/proxmoxmachine_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,14 @@ func defaultMachine() *ProxmoxMachine {
Namespace: metav1.NamespaceDefault,
},
Spec: ProxmoxMachineSpec{
VirtualMachineCloneSpec: VirtualMachineCloneSpec{
SourceNode: "pve1",
},
ProviderID: ptr.To("proxmox://abcdef"),
VirtualMachineID: ptr.To[int64](100),
VirtualMachineCloneSpec: VirtualMachineCloneSpec{
TemplateSource: TemplateSource{
SourceNode: "pve1",
TemplateID: ptr.To[int32](100),
},
},
Disks: &Storage{
BootVolume: &DiskSize{
Disk: "scsi0",
Expand All @@ -56,18 +59,32 @@ var _ = Describe("ProxmoxMachine Test", func() {
})

Context("VirtualMachineCloneSpec", func() {
It("Should not allow empty source node", func() {
It("Should not allow specifying format if full clone is disabled", func() {
dm := defaultMachine()
dm.Spec.SourceNode = ""
dm.Spec.Full = ptr.To(false)

Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("should be at least 1 chars long")))
Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("Must set full=true when specifying format")))
})

It("Should not allow specifying format if full clone is disabled", func() {
It("Should disallow absence of SourceNode, TemplateID and TemplateSelector", func() {
dm := defaultMachine()
dm.Spec.Full = ptr.To(false)
dm.Spec.TemplateSource.SourceNode = ""
dm.Spec.TemplateSource.TemplateID = nil
dm.Spec.TemplateSelector = nil
Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("must define either SourceNode with TemplateID, OR TemplateSelector")))
})

Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("Must set full=true when specifying format")))
It("Should not allow specifying TemplateSelector together with SourceNode and/or TemplateID", func() {
dm := defaultMachine()
dm.Spec.TemplateSelector = &TemplateSelector{MatchTags: []string{"test"}}
Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("must define either SourceNode with TemplateID, OR TemplateSelector")))
})

It("Should not allow specifying TemplateSelector with empty MatchTags", func() {
dm := defaultMachine()
dm.Spec.TemplateSelector = &TemplateSelector{MatchTags: []string{}}

Expect(k8sClient.Create(context.Background(), dm)).Should(MatchError(ContainSubstring("should have at least 1 items")))
pborn-ionos marked this conversation as resolved.
Show resolved Hide resolved
})
})

Expand Down
51 changes: 46 additions & 5 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 16 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_proxmoxclusters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,22 @@ spec:
a new VM.
format: int32
type: integer
templateSelector:
description: TemplateSelector defines MatchTags for looking
up VM templates.
properties:
matchTags:
description: |-
Specifies all tags to look for, when looking up the VM template.
Passed tags must be an exact 1:1 match with the tags on the template you want to use.
If multiple VM templates with the same set of tags are found, provisioning will fail.
items:
type: string
minItems: 1
type: array
required:
- matchTags
type: object
virtualMachineID:
description: VirtualMachineID is the Proxmox identifier
for the ProxmoxMachine VM.
Expand Down Expand Up @@ -590,8 +606,6 @@ spec:
x-kubernetes-validations:
- message: end should be greater than or equal to start
rule: self.end >= self.start
required:
- sourceNode
type: object
type: object
x-kubernetes-validations:
Expand Down
18 changes: 16 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_proxmoxclustertemplates.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -600,6 +600,22 @@ spec:
for cloning a new VM.
format: int32
type: integer
templateSelector:
description: TemplateSelector defines MatchTags
for looking up VM templates.
properties:
matchTags:
description: |-
Specifies all tags to look for, when looking up the VM template.
Passed tags must be an exact 1:1 match with the tags on the template you want to use.
If multiple VM templates with the same set of tags are found, provisioning will fail.
items:
type: string
minItems: 1
type: array
required:
- matchTags
type: object
virtualMachineID:
description: VirtualMachineID is the Proxmox identifier
for the ProxmoxMachine VM.
Expand Down Expand Up @@ -632,8 +648,6 @@ spec:
- message: end should be greater than or equal to
start
rule: self.end >= self.start
required:
- sourceNode
type: object
type: object
x-kubernetes-validations:
Expand Down
24 changes: 22 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_proxmoxmachines.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,6 +527,22 @@ spec:
VM.
format: int32
type: integer
templateSelector:
description: TemplateSelector defines MatchTags for looking up VM
templates.
properties:
matchTags:
description: |-
Specifies all tags to look for, when looking up the VM template.
Passed tags must be an exact 1:1 match with the tags on the template you want to use.
If multiple VM templates with the same set of tags are found, provisioning will fail.
items:
type: string
minItems: 1
type: array
required:
- matchTags
type: object
virtualMachineID:
description: VirtualMachineID is the Proxmox identifier for the ProxmoxMachine
VM.
Expand Down Expand Up @@ -557,10 +573,14 @@ spec:
x-kubernetes-validations:
- message: end should be greater than or equal to start
rule: self.end >= self.start
required:
- sourceNode
type: object
x-kubernetes-validations:
- message: must define either SourceNode with TemplateID, OR TemplateSelector
rule: '[has(self.sourceNode), has(self.templateSelector)].exists_one(c,
c)'
- message: must define either SourceNode with TemplateID, OR TemplateSelector.
rule: '[has(self.templateID), has(self.templateSelector)].exists_one(c,
c)'
- message: Must set full=true when specifying format
rule: self.full && self.format != ''
status:
Expand Down
18 changes: 16 additions & 2 deletions config/crd/bases/infrastructure.cluster.x-k8s.io_proxmoxmachinetemplates.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,22 @@ spec:
a new VM.
format: int32
type: integer
templateSelector:
description: TemplateSelector defines MatchTags for looking
up VM templates.
properties:
matchTags:
description: |-
Specifies all tags to look for, when looking up the VM template.
Passed tags must be an exact 1:1 match with the tags on the template you want to use.
If multiple VM templates with the same set of tags are found, provisioning will fail.
items:
type: string
minItems: 1
type: array
required:
- matchTags
type: object
virtualMachineID:
description: VirtualMachineID is the Proxmox identifier for
the ProxmoxMachine VM.
Expand Down Expand Up @@ -589,8 +605,6 @@ spec:
x-kubernetes-validations:
- message: end should be greater than or equal to start
rule: self.end >= self.start
required:
- sourceNode
type: object
required:
- spec
Expand Down
8 changes: 8 additions & 0 deletions docs/advanced-setups.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,14 @@ This behaviour can be configured in the `ProxmoxCluster` CR through the field `.

For example, setting it to `0` (zero), entirely disables scheduling based on memory. Alternatively, if you set it to any value greater than `0`, the scheduler will treat your host as it would have `${value}%` of memory. In real numbers that would mean, if you have a host with 64GB of memory and set the number to `300`, the scheduler would allow you to provision guests with a total of 192GB memory and therefore overprovision the host. (Use with caution! It's strongly suggested to have memory ballooning configured everywhere.). Or, if you were to set it to `95` for example, it would treat your host as it would only have 60,8GB of memory, and leave the remaining 3,2GB for the host.

## Template lookup based on Proxmox tags

Our provider is able to look up templates based on their attached tags, for `ProxmoxMachine` resources, that make use of an tag selector.

For example, you can set the `TEMPLATE_TAGS="tag1,tag2"` environment variable. Your custom image will then be used when using the [auto-image](https://github.com/ionos-cloud/cluster-api-provider-ionoscloud/blob/main/templates/cluster-template-auto-image.yaml) template.

Please note: Passed tags must be an exact 1:1 match with the tags on the template you want to use. The matched result must be unique. If multiple templates are found, provisioning will fail.

## Proxmox RBAC with least privileges

For the Proxmox API user/token you create for CAPMOX, these are the minimum required permissions.
Expand Down
1 change: 1 addition & 0 deletions envfile.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ export PROXMOX_TOKEN=""
export PROXMOX_SECRET=""
export PROXMOX_SOURCENODE="pve"
export TEMPLATE_VMID=100
export TEMPLATE_TAGS="tag1,tag2"
export VM_SSH_KEYS="ssh-ed25519 ..., ssh-ed25519 ..."
export KUBERNETES_VERSION="1.25.1"
export CONTROL_PLANE_ENDPOINT_IP=10.10.10.4
Expand Down
6 changes: 4 additions & 2 deletions internal/service/vmservice/helpers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,10 @@ func setupReconcilerTest(t *testing.T) (*scope.MachineScope, *proxmoxtest.MockCl
},
Spec: infrav1alpha1.ProxmoxMachineSpec{
VirtualMachineCloneSpec: infrav1alpha1.VirtualMachineCloneSpec{
SourceNode: "node1",
TemplateID: ptr.To[int32](123),
TemplateSource: infrav1alpha1.TemplateSource{
SourceNode: "node1",
TemplateID: ptr.To[int32](123),
},
},
},
}
Expand Down
Loading
Loading