ansible role for deploy capi mgmt cluster for magnum capi driver (rackerlabs#1465)

* add ansible automation for deploying capi mgmt cluster on genestack cloud

* update lb pool deletion task to a handler and use command module instead of shell module

* update the playbook to load the variables from a vars file instead of defining them from the main playbook

* move the vars from ubuntu.yml to main.yml

Author: puni4220

ansible/playbooks/capi-mgmt-cluster-main.yaml

@@ -0,0 +1,155 @@
+- name: Play to bootstrap project and user for capi mgmt cluster
+  hosts: localhost
+  gather_facts: true
+  vars_files:
+    - vars/main.yml
+    - vars/{{ ansible_distribution | lower }}.yml
+  environment: "{{ os_admin_env | default({}) }}"
+  pre_tasks:
+    - name: check that the admin password is provided
+      ansible.builtin.assert:
+        that:
+          - os_admin_password is defined
+          - os_admin_password | length > 0
+        fail_msg: >
+          password for the admin user is required
+          provide the admin password with -e os_admin_password=<passwd>
+    - name: check that the password capi_mgmt_user is provided
+      ansible.builtin.assert:
+        that:
+          - os_user_password is defined
+          - os_user_password | length > 0
+        fail_msg: >
+          password for the capi mgmt user is required
+          provide the password for capi mgmt user with -e os_user_password=<passwd>
+          this will be password for creating the capi mgmt user
+  tasks:
+    - name: Create the project for capi mgmt cluster
+      openstack.cloud.project:
+        name: "{{ capi_mgmt_project_name }}"
+        domain: "{{ capi_mgmt_project_domain }}"
+        description: "capi mgmt cluster project"
+        is_enabled: True
+        state: present
+
+    - name: Create the user for the capi mgmt cluster
+      openstack.cloud.identity_user:
+        name: "{{ capi_mgmt_user_name }}"
+        domain: "{{ capi_mgmt_user_domain }}"
+        description: "capi mgmt cluster user"
+        state: present
+        password: "{{ os_user_password }}"
+        default_project: "{{ capi_mgmt_project_name }}"
+
+    - name: Assign the admin role to the capi mgmt user
+      openstack.cloud.role_assignment:
+        user: "{{ capi_mgmt_user_name }}"
+        role: admin
+        project: "{{ capi_mgmt_project_name }}"
+        domain: service
+
+    - name: Modify the quotas for the capi mgmt cluster project
+      openstack.cloud.quota:
+        name: "{{ capi_mgmt_project_name }}"
+        cores: 30
+        volumes: 20
+        snapshots: 20
+        instances: 20
+
+    - name: Persist the capi mgmt user creds
+      ansible.builtin.set_fact:
+        capi_mgmt_os_env: "{{ capi_mgmt_env }}"
+
+- name: Playbook to create capi mgmt cluster infra
+  hosts: localhost
+  gather_facts: true
+  environment: "{{ capi_mgmt_os_env | default({}) }}"
+  roles:
+    - capi_cluster
+
+- name: Playbook to deploy kubernetes cluster on capi mgmt cluster vms
+  hosts: capi_mgmt_bootstrp_vm
+  gather_facts: true
+  become: true
+  tasks:
+    - name: ping the capi mgmt cluster bootstrp vm fip
+      ansible.builtin.ping:
+
+    - name: copy the ssh key to the bootstrp vm
+      ansible.builtin.copy:
+        src: "{{ ansible_env.HOME }}/capi_mgmt_private_key"
+        dest: /home/ubuntu/capi_mgmt_cluster_keypair
+        mode: 0400
+
+    - name: copy the capi mgmt cluster inventory to the bootstrp vm
+      ansible.builtin.copy:
+        src: /var/tmp/capi-mgmt-inventory.ini
+        dest: /home/ubuntu/capi-mgmt-inventory.ini
+
+    - name: copy the capi mgmt cluster vars to the bootstrp vm
+      ansible.builtin.copy:
+        src: /var/tmp/capi-cluster-vars.yml
+        dest: /home/ubuntu/capi-cluster-vars.yml
+
+    - name: check if dns overrides exists on the localhost
+      ansible.builtin.stat:
+        path: /var/tmp/capi-cluster-dns-vars.yml
+      delegate_to: localhost
+      register: _capi_dns_vars_file
+
+    - name: copy the dns vars to the bootstrp vm
+      ansible.builtin.copy:
+        src: /var/tmp/capi-cluster-dns-vars.yml
+        dest: /home/ubuntu/capi-cluster-dns-vars.yml
+      when: _capi_dns_vars_file.stat.exists
+
+    - name: copy the shell script to the install the cluster on the bootstrp vm
+      ansible.builtin.copy:
+        src: /var/tmp/capi-cluster-install.sh
+        dest: /home/ubuntu/capi-cluster-install.sh
+        mode: u+x
+
+    - name: copy the etcd backup script to the bootstrp vm
+      ansible.builtin.copy:
+        src: /var/tmp/etcd-backup.sh
+        dest: /usr/local/bin/etcd-backup.sh
+        mode: u+x
+
+    - name: Notify the user about the script logs
+      ansible.builtin.debug:
+        msg: >
+          "installing capi-mgmt-cluster; check /var/log/capi-mgmt-cluster-install.log on
+          {{ inventory_hostname }} for further details"
+
+    - name: run the script to install the capi-mgmt-cluster
+      ansible.builtin.command:
+        cmd: /home/ubuntu/capi-cluster-install.sh
+      async: 1800
+      poll: 0
+      register: _capi_cluster_install_script
+
+    - name: check the status of the script until script is running
+      ansible.builtin.async_status:
+        jid: "{{ _capi_cluster_install_script.ansible_job_id }}"
+      register: _capi_cluster_install_script_result
+      until: _capi_cluster_install_script_result is finished
+      retries: 180
+      delay: 10
+
+    - name: print message if the script runs without any issue
+      ansible.builtin.debug:
+        msg: >
+         capi cluster install script went without any issues; the management
+         cluster has been installed
+      when:
+        - _capi_cluster_install_script_result.rc is defined
+        - _capi_cluster_install_script_result.rc == 0
+
+    - name: copy the kubeconfig from the capi mgmt cluster
+      ansible.builtin.fetch:
+        src: /etc/kubernetes/admin.conf
+        dest: /var/tmp/capi_mgmt_cluster.kubeconfig
+        flat: yes
+      when:
+        - _capi_cluster_install_script_result.rc is defined
+        - _capi_cluster_install_script_result.rc == 0

ansible/playbooks/vars/main.yml

@@ -0,0 +1,52 @@
+---
+os_endpoint_type: public
+os_interface: "{{ os_endpoint_type }}"
+os_username: admin
+os_project_name: admin
+os_tenant_name: '{{ os_project_name }}'
+os_auth_type: password
+os_admin_password: ''
+os_auth_url: http://keystone.pikachu.jabronis.dev/v3
+os_user_domain_name: default
+os_project_domain_name: default
+os_region_name: RegionOne
+os_identity_api_version: 3
+os_auth_version: 3
+nova_endpoint_type: "{{ os_endpoint_type }}"
+capi_mgmt_project_name: capi-mgmt-cluster-project
+capi_mgmt_user_name: capi-mgmt-user
+capi_mgmt_project_domain: service
+capi_mgmt_user_domain: service
+os_user_password: ''
+
+os_admin_env:
+  OS_ENDPOINT_TYPE: "{{ os_endpoint_type }}"
+  OS_INTERFACE: "{{ os_interface }}"
+  OS_USERNAME: "{{ os_username }}"
+  OS_PASSWORD: "{{ os_admin_password }}"
+  OS_PROJECT_NAME: "{{ os_project_name }}"
+  OS_TENANT_NAME: 'admin'
+  OS_AUTH_TYPE: password
+  OS_AUTH_URL: "{{ os_auth_url }}"
+  OS_USER_DOMAIN_NAME: "{{ os_user_domain_name }}"
+  OS_PROJECT_DOMAIN_NAME: "{{ os_project_domain_name }}"
+  OS_REGION_NAME: "{{ os_region_name }}"
+  OS_IDENTITY_API_VERSION: "{{ os_identity_api_version }}"
+  OS_AUTH_VERSION: "{{ os_auth_version }}"
+  NOVA_ENDPOINT_TYPE: "{{ nova_endpoint_type }}"
+
+capi_mgmt_env:
+  OS_ENDPOINT_TYPE: "{{ os_endpoint_type | default('public') }}"
+  OS_INTERFACE: "{{ os_interface | default('public') }}"
+  OS_AUTH_URL: http://keystone.pikachu.jabronis.dev/v3
+  OS_USERNAME: "{{ capi_mgmt_user_name }}"
+  OS_PASSWORD: "{{ os_user_password }}"
+  OS_PROJECT_NAME: "{{ capi_mgmt_project_name }}"
+  OS_USER_DOMAIN_NAME: service
+  OS_PROJECT_DOMAIN_NAME: service
+  OS_AUTH_TYPE: password
+  OS_REGION_NAME: "{{ os_region_name | default('RegionOne') }}"
+  OS_IDENTITY_API_VERSION: "{{ os_identity_api_version }}"
+  OS_AUTH_VERSION: "{{ os_auth_version }}"
+  NOVA_ENDPOINT_TYPE: "{{ nova_endpoint_type }}"
+  KUBECONFIG: "{{ ansible_env.HOME }}/.kube/config"

ansible/playbooks/vars/ubuntu.yml

@@ -0,0 +1,2 @@
+---
+# provide variables specific to ubuntu distro

ansible/roles/capi_cluster/README.md

@@ -0,0 +1,66 @@
+CAPI CLUSTER
+=========
+
+This is the ansible role for creating the infra for installing capi mgmt cluster on a genestack cloud
+
+Requirements
+------------
+
+These are the infra level requirements for the role to create the required infra for the capi mgmt cluster:
+* cinder-volume service enabled: Currently the role only supports bfv instances; so cinder-volume should be enabled
+* external network: A pre-created shared external neutron network (either flat or vlan) to be used for the role
+* keystone admin credentials: when running the role; keystone admin user credentials would be required
+* octavia service enabled: role creates loadbalancer for the capi mgmt cluster; so octavia should be enabled
+
+These are the network level requirements for the role to create the capi mgmt cluster:
+* dns server: Atleast one dns server to be defined:
+  - dns server should be reachable from the external neutron network used with the role
+  - dns server should be able resolve external endpoints for the openstack services on genestack cloud
+* external network reachability:
+  - the external network should be reachable from the ansible control node from where the playbook is run
+  - the external network should be able reach the public endpoints for openstack services
+
+There are a few other requirements for this role as well:
+* openstack sdk installed: On the ansible control node openstack-sdk should be installed (generally genestack venv is sufficient)
+* sufficient storage space on glance: the role will upload a ubuntu-24 image; should the backend for glance should have sufficient space
+* currently there is no provision to include custom tls certs; so if the openstack endpoints are behind a tls gateway then the certs should be
+  signed by a well-known CA like verisign etc; self signed certs will cause failures with magnum after the mgmt cluster is deployed
+
+Role Variables
+--------------
+
+These are the role variables:
+* required role variables:
+  - ext_net_id: external neutron network ID (should already exist)
+  - os_admin_password: keystone admin password
+  - os_user_password: password for the new user to be created for capi-mgmt-cluster-project
+  - capi_mgmt_dns_servers: dns server to be used for the capi mgmt cluster
+  - capi_boot_from_volume: should be set to true (default)
+
+* other important role variables:
+  - capi_mgmt_cluster_flavor: flavor the capi mgmt cluster vms (flavor name and specs)
+  - capi_mgmt_cluster_volume_type: volume type for the capi mgmt cluster (defaults to lvmdriver-1)
+  - capi_mgmt_cluster_volumes: can be used to define the size of the volumes for capi mgmt cluster vms
+  - capi_mgmt_etcd_backup_volume: can be used to define the size of the etcd backup volume and volume type
+
+Refer to defaults/main.yml with the role directory for more details on the variables
+
+Dependencies
+------------
+
+There are no external dependencies for the role; basic genestack venv should be sufficient
+
+Example Playbook
+----------------
+
+The playbook which includes the role for capi mgmt cluster infra should be used as below:
+
+```
+ansible-playbook capi-mgmt-main.yaml -e os_admin_password=<keystone_admin_passwd> -e os_user_password=rack1234 -e ext_net_id='7f84bb82-996e-4520-a2f0-50a9602de363'
+```
+
+Author Information
+------------------
+
+Name: Punit Shankar Kundal\
+Email: punitshankar.kundal@rackspace.com