Opt-in for network-policies provided by hco-bundle for CNAO and its components pods (#2341)

* manifests: Add opt-in labels for hco-bundle network-policies

The hco-bundle should provide network-policies allow operators
work as expected under network restrictions (global deny-all
network-policy).
The provided NPs affects pods who has the appropriate labels.

Opt-in for hco-bundle NPs by adding the following labels:
1. hco.kubevirt.io/allow-access-cluster-services
   Allow CNAO pods egress access the cluster API and DNS.
2. hco.kubevirt.io/allow-prometheus-access
   Allow prometheus pods ingress the metrics endpoint

With this change the manifest-templator & csv-gen should produce
the project Deployment and CSV with the above labels.
Allowing CNAO to operate under network restrictions when installed
by HCO.

Signed-off-by: Or Mergi <[email protected]>

* KMP: Add opt-in labels for hco-bundle network-policies

The hco-bundle should provide network-policies allow operators
work as expected under network restrictions (global deny-all
network-policy).
The provided NPs affects pods who have the appropriate labels.

Opt-in for hco-bundle NPs by adding the following labels:
1. hco.kubevirt.io/allow-access-cluster-services
   Allow KMP pods egress access the cluster API and DNS.
2. hco.kubevirt.io/allow-prometheus-access
   Allow prometheus pods ingress the controller-manager metrics endpoint

With this change CNAO should generate and install kubemacpool
manifests with the above labels.
Allowing kubemacpool operate under network restrictions when
installed by CNAO and HCO.

Signed-off-by: Or Mergi <[email protected]>

* KSD: Add opt-in labels for hco-bundle network-policies

The hco-bundle should provide network-policies allow operators
work as expected under network restrictions (global deny-all
network-policy).
The provided NPs affects pods who have the appropriate labels.

Opt-in for hco-bundle NPs by adding the "hco.kubevirt.io/allow-access-cluster-services" label.
It allows kube-secondary-dns pods access the cluster API and DNS.

With this change CNAO should generate and install kube-secondary-
dns manifests with the above labels.
Allowing ube-secondary-dns operate under network restrictions
when installed by CNAO and HCO.

Signed-off-by: Or Mergi <[email protected]>

* ipam-ext: Add opt-in labels for hco-bundle network-policies

The hco-bundle should provide network-policies allow operators
work as expected under network restrictions (global deny-all
network-policy).
The provided NPs affects pods who have the appropriate labels.

Opt-in for hco-bundle NPs by adding the "hco.kubevirt.io/allow-access-cluster-services" label.
It allows kubevirt-ipam-contoller pods access the cluster
API and DNS.

With this change CNAO should generate and install
kubevirt-ipam-contoller manifests with the above labels.
Allowing kubevirt-ipam-contoller operate under network
restrictions when installed by CNAO and HCO.

Signed-off-by: Or Mergi <[email protected]>

---------

Signed-off-by: Or Mergi <[email protected]>

Author: ormergi

data/kube-secondary-dns/secondarydns.yaml

@@ -73,6 +73,7 @@ spec:
     metadata:
       labels:
         k8s-app: secondary-dns
+        hco.kubevirt.io/allow-access-cluster-services: ""
       annotations:
         kubectl.kubernetes.io/default-container: status-monitor
         openshift.io/required-scc: "restricted-v2"

data/kubemacpool/kubemacpool.yaml

@@ -153,6 +153,7 @@ spec:
         app: kubemacpool
         control-plane: cert-manager
         controller-tools.k8s.io: "1.0"
+        hco.kubevirt.io/allow-access-cluster-services: ""
     spec:
       affinity: {{ toYaml .Placement.Affinity | nindent 8 }}
       containers:
@@ -245,6 +246,8 @@ spec:
         app: kubemacpool
         control-plane: mac-controller-manager
         controller-tools.k8s.io: "1.0"
+        hco.kubevirt.io/allow-access-cluster-services: ""
+        hco.kubevirt.io/allow-prometheus-access: ""
     spec:
       affinity: {{ toYaml .Placement.Affinity | nindent 8 }}
       containers:

data/kubevirt-ipam-controller/001-kubevirtipamcontroller.yaml

@@ -178,6 +178,7 @@ spec:
       labels:
         app: ipam-virt-workloads
         control-plane: manager
+        hco.kubevirt.io/allow-access-cluster-services: ""
     spec:
       containers:
         - args:

hack/components/bump-kube-secondary-dns.sh

@@ -35,6 +35,7 @@ function __parametize_by_object() {
         yaml-utils::set_param ${f} spec.template.spec.affinity '{{ toYaml .Placement.Affinity | nindent 8 }}'
         yaml-utils::set_param ${f} spec.template.spec.tolerations '{{ toYaml .Placement.Tolerations | nindent 8 }}'
         yaml-utils::set_param ${f} 'spec.template.metadata.annotations."openshift.io/required-scc"' '"restricted-v2"'
+        yaml-utils::set_param ${f} 'spec.template.metadata.labels."hco.kubevirt.io/allow-access-cluster-services"' '""'
         yaml-utils::remove_single_quotes_from_yaml ${f}
         ;;
       ./ServiceAccount_secondary.yaml)

hack/components/bump-kubemacpool.sh

@@ -59,6 +59,15 @@ patches:
   target:
     version: v1
     kind: Namespace
+- path: add-pod-template-label-allow-access-cluster-services_patch.yaml
+  target:
+    version: v1
+    kind: Deployment
+- path: add-pod-template-label-allow-prometheus-access_patch.yaml
+  target:
+    version: v1
+    kind: Deployment
+    name: mac-controller-manager
 EOF
 
     cat <<EOF > config/cnao/cnao_kubemacpool_manager_patch.yaml
@@ -145,6 +154,16 @@ EOF
   path: /metadata/labels
 EOF
 
+    cat <<EOF > config/cnao/add-pod-template-label-allow-access-cluster-services_patch.yaml
+- op: add
+  path: /spec/template/metadata/labels/hco.kubevirt.io~1allow-access-cluster-services
+  value: ""
+EOF
+    cat <<EOF > config/cnao/add-pod-template-label-allow-prometheus-access_patch.yaml
+- op: add
+  path: /spec/template/metadata/labels/hco.kubevirt.io~1allow-prometheus-access
+  value: ""
+EOF
 
     (
         cd config/cnao

hack/components/bump-kubevirt-ipam-controller.sh

@@ -29,6 +29,7 @@ function __parametize_by_object() {
         yaml-utils::set_param ${f} spec.template.spec.nodeSelector '{{ toYaml .Placement.NodeSelector | nindent 8 }}'
         yaml-utils::set_param ${f} spec.template.spec.affinity '{{ toYaml .Placement.Affinity | nindent 8 }}'
         yaml-utils::set_param ${f} spec.template.spec.tolerations '{{ toYaml .Placement.Tolerations | nindent 8 }}'
+        yaml-utils::set_param ${f} 'spec.template.metadata.labels."hco.kubevirt.io/allow-access-cluster-services"' '""'
         yaml-utils::remove_single_quotes_from_yaml ${f}
         ;;
       ./Service_kubevirt-ipam-controller-webhook-service.yaml)

pkg/components/components.go

@@ -181,6 +181,10 @@ func GetDeployment(version string, operatorVersion string, namespace string, rep
 					Labels: map[string]string{
 						"name":                   Name,
 						names.PrometheusLabelKey: names.PrometheusLabelValue,
+						// opt-in to hco-bundle network-policy allowing egress to cluster services
+						"hco.kubevirt.io/allow-access-cluster-services": "",
+						// opt-in to hco-bundle network-policy allowing ingress to the metrics endpoint
+						"hco.kubevirt.io/allow-prometheus-access": "",
 					},
 					Annotations: map[string]string{
 						"description": "cluster-network-addons-operator manages the lifecycle of different Kubernetes network components on top of Kubernetes cluster",