Deploy multi-tenant type site with sub-cluster in STL labs

[eak13]

Building on #129, we want to define a multi-tenant type reference site which includes a sub-cluster deployment in the STL labs. This is will include, but may not be limited to the following:

• All of the SIP/ViNO integration into Treasuremap - Create sub-cluster networking catalog for multi-tenant type #103, Deploy Virtual Node Operator (ViNO) via Airship Phase in multi-tenant type #104, Deliver Support Infra Provider (SIP) Cluster CRD/Operator via Airship Phase in multi-tenant Type #105, Deliver ViNO CR in multi-tenant type Airship Phase #106, Deploy Synclabeller Operator via Airship Phase in multi-tenant type #123 - completed
• Keepalived VRRP for K8s Ingress over host management/OAM interface #94 - completed
• Create Sub-Cluster Manifest Type/Functions #82 - completed
• Provision target baremetal nodes with 'vm-infra-bridge' #122 - complete
• Deploying sample (WordPress) sub-cluster - Create multi-tenant Type Sample Sub-cluster  #107 - completed
• Deploying LMA sub-cluster - Deploy Kibana Dashboards in airship-core/multi-tenant types #150 (in progress) & LMA sub-cluster for multi-tenant type #118 (complete)
• Calico Policies - Define phase to apply Calico network policies #119 to define phase for delivering Calico policies - completed & Create network policies (N/S and E/W) for infrastructure and sub-clusters #131 network policies for infra & sub-clusters - completed
• Create phase plans to deploy LMA & Wordpress sub-clusters #139
• Create multi-tenant type scripts to deliver SIP/ViNO phases and sub-cluster phase plans #140
• Sub-clusters Deployment: Add Multi-Tenant Cluster Control Plane Nodes to ViNO VM Network #177
• Match SIP Labels for CAPI Provisioning of Sub-cluster Nodes #172
• Provide way to define/override Vino sub-cluster VM flavor definitions & apply to STL3 #170
• Create delivery mechanism for sub-cluster QCOW images #169
• Align Reference site disk layout for sub-cluster support #168
• Move lma-stack contents to the workload phase #160

Resolve any bugs encountered during implementation:

• Sub-clusters Deployment: ViNO VMs Provisioning Issue #189
• Sub-clusters Deployment: Fix Server/Rack Label Configuration for Multi-Tenant #185
• Sub-clusters Deployment: Need SSH Keys Populated to KubeadmControlPlane  #183
• Sub-clusters Deployment: Remove VRRP from Sub-Cluster KCP #182
• Sub-clusters Deployment: Sync Sub-Cluster API Server VIP/Port Between SIP CR and Metal3Cluster  #181
• Sub-clusters Deployment: Limit vino network cloud-init types to ipv4 #180
• Failed Applying provide-infra-lma Phase in Multi-Tenant Reference Site #178

Dex (not critical path for sub-cluster delivery)

• Provide Encryption for External Password for LDAP Backend for Dex #128
• Add Dex for Sub-Cluster Authentication #134
• Enhance Dex Deployment to Pull Configuration from Catalogues for Target Cluster #135
• Enhance Sub-Cluster Dex Deployment to Pull Configuration from Catalogues #136

Semi-related, needed for gating & stability

• Use airshipctl plan run in gates #166
• Implement phase/plan validation changes in Treasuremap manifests #176

[lb4368]

Some of the tasks for the sub-cluster deployments are:

1. Create phase plans to deploy the LMA and WordPress sub-clusters
2. Create scripts to deliver SIP/ViNO phases and subcluster phase plans
3. Update SIP to the latest version and configure the recently added infrastructure services.

Below are patchsets from @drewwalters96 that may help someone to complete these tasks.

https://review.opendev.org/c/airship/treasuremap/+/788042
https://review.opendev.org/c/airship/treasuremap/+/788080
https://review.opendev.org/c/airship/treasuremap/+/788029/3

[lb4368]

Closing this issue as sub-cluster delivery intent has been met. Residual items open for possible future enhancement