(PE-39412) PEADM should be able to add multiple compilers at once wit…

…hout requiring service restarts on Primary (#512) * feat(plans): add_compilers plan created - Added `peadm::add_compilers` to support adding multiple compilers. - Updated parameter `compiler_host` to `compiler_hosts` to accept an array of compiler targets. - Modified `dns_alt_names` parameter to accept an array of strings, each representing a comma-separated list of DNS alt names. - Updated all references in documentation and tests to reflect the new plan name. - Enhanced the plan to handle multiple compilers in parallel, including updating PostgreSQL configurations and running necessary tasks on each compiler. - Introduced a new plan `peadm::add_compiler` as a proxy for `peadm::add_compilers`.
puppetlabs · Oct 14, 2024 · 315d4ea · 315d4ea
1 parent 4b139a9
commit 315d4ea
Show file tree

Hide file tree

Showing 10 changed files with 357 additions and 232 deletions.
diff --git a/.github/workflows/test-add-compiler-matrix.yml b/.github/workflows/test-add-compiler-matrix.yml
@@ -84,10 +84,10 @@ jobs:
             architecture=${{ matrix.architecture }} \
             console_password=${{ secrets.CONSOLE_PASSWORD }} \
             version=${{ matrix.version }}
-      - name: Run add_compiler plan
+      - name: Run add_compilers plan
         timeout-minutes: 50
         run: |
-          bundle exec bolt plan run peadm_spec::add_compiler -v \
+          bundle exec bolt plan run peadm_spec::add_compilers -v \
             --inventoryfile spec/fixtures/litmus_inventory.yaml \
             --modulepath spec/fixtures/modules \
             architecture=${{ matrix.architecture }}

diff --git a/.github/workflows/test-add-compiler.yaml b/.github/workflows/test-add-compiler.yaml
@@ -85,10 +85,10 @@ jobs:
             architecture=${{ matrix.architecture }} \
             version=${{ matrix.version }} \
             console_password=${{ secrets.CONSOLE_PASSWORD }
-      - name: Run add_compiler plan
+      - name: Run add_compilers plan
         timeout-minutes: 50
         run: |
-          bundle exec bolt plan run peadm_spec::add_compiler -v \
+          bundle exec bolt plan run peadm_spec::add_compilers -v \
             --inventoryfile spec/fixtures/litmus_inventory.yaml \
             --modulepath spec/fixtures/modules \
             architecture=${{ matrix.architecture }} \

diff --git a/REFERENCE.md b/REFERENCE.md
@@ -94,7 +94,8 @@
 
 #### Public Plans
 
-* [`peadm::add_compiler`](#peadm--add_compiler): Add a new compiler to a PE architecture or replace an existing one with new configuration.
+* [`peadm::add_compiler`](#peadm--add_compiler): Proxy plan for peadm::add_compilers.
+* [`peadm::add_compilers`](#peadm--add_compilers): Add new compilers to a PE architecture or replace an existing with new configuration.
 * [`peadm::add_database`](#peadm--add_database)
 * [`peadm::add_replica`](#peadm--add_replica): Add or replace a replica host.
 Supported use cases:
@@ -1610,7 +1611,7 @@ Which port to query the status API on
 
 ### <a name="peadm--add_compiler"></a>`peadm::add_compiler`
 
-Add a new compiler to a PE architecture or replace an existing one with new configuration.
+Proxy plan for peadm::add_compilers.
 
 #### Parameters
 
@@ -1626,7 +1627,7 @@ The following parameters are available in the `peadm::add_compiler` plan:
 
 Data type: `Enum['A', 'B']`
 
-_ Either A or B; whichever of the two letter designations the compiler is being assigned to
+_ Either A or B; whichever of the two letter designations the compiler are being assigned to
 
 Default value: `'A'`
 
@@ -1640,7 +1641,7 @@ _ The hostname and certname of the new compiler
 
 Data type: `Optional[String[1]]`
 
-_ A comma_separated list of DNS alt names for the compiler
+_ A comma-separated list of DNS alt names for the compiler.
 
 Default value: `undef`
 
@@ -1658,6 +1659,56 @@ _ The hostname and certname of the PE-PostgreSQL server with availability group
 
 Default value: `undef`
 
+### <a name="peadm--add_compilers"></a>`peadm::add_compilers`
+
+Add new compilers to a PE architecture or replace an existing with new configuration.
+
+#### Parameters
+
+The following parameters are available in the `peadm::add_compilers` plan:
+
+* [`avail_group_letter`](#-peadm--add_compilers--avail_group_letter)
+* [`compiler_hosts`](#-peadm--add_compilers--compiler_hosts)
+* [`dns_alt_names`](#-peadm--add_compilers--dns_alt_names)
+* [`primary_host`](#-peadm--add_compilers--primary_host)
+* [`primary_postgresql_host`](#-peadm--add_compilers--primary_postgresql_host)
+
+##### <a name="-peadm--add_compilers--avail_group_letter"></a>`avail_group_letter`
+
+Data type: `Enum['A', 'B']`
+
+_ Either A or B; whichever of the two letter designations the compilers are being assigned to
+
+Default value: `'A'`
+
+##### <a name="-peadm--add_compilers--compiler_hosts"></a>`compiler_hosts`
+
+Data type: `TargetSpec`
+
+_ The hostnames and certnames of the new compilers
+
+##### <a name="-peadm--add_compilers--dns_alt_names"></a>`dns_alt_names`
+
+Data type: `Optional[Array[String[1]]]`
+
+_ An array of strings, where each string is a comma-separated list of DNS alt names for the compilers. Order matters; if a compiler doesn't need dns_alt_names, use "undef" as string.
+
+Default value: `undef`
+
+##### <a name="-peadm--add_compilers--primary_host"></a>`primary_host`
+
+Data type: `Peadm::SingleTargetSpec`
+
+_ The hostname and certname of the primary Puppet server
+
+##### <a name="-peadm--add_compilers--primary_postgresql_host"></a>`primary_postgresql_host`
+
+Data type: `Optional[Peadm::SingleTargetSpec]`
+
+_ The hostname and certname of the PE-PostgreSQL server with availability group $avail_group_letter
+
+Default value: `undef`
+
 ### <a name="peadm--add_database"></a>`peadm::add_database`
 
 The peadm::add_database class.

diff --git a/documentation/add_compiler.md b/documentation/add_compiler.md
diff --git a/documentation/add_compilers.md b/documentation/add_compilers.md
@@ -0,0 +1,93 @@
+# Add Compilers
+
+- [Add Compilers](#Add-Compilers)
+  - [Introduction](#Introduction)
+  - [Add compilers to an existing PE cluster](#Add-compilers-to-an-existing-PE-cluster)
+  - [Running the `add_compilers` plan](#running-the-add_compilers-plan)
+  - [Optional Parameters](#optional-parameters)
+
+## Introduction
+
+The `peadm::add_compilers` plan can be used to add new compilers to a PE cluster or replace an existing with new configuration.
+
+## Add compilers to an existing PE cluster
+
+As seen in the example below, this is the minimal parameters required to add a compiler to an existing PE cluster. These can be passed as command line variables to the plan, or in this case added to a params.json file.
+
+```json
+{
+  "compiler_hosts": "pe-xl-compiler-0.lab1.puppet.vm",
+  "primary_host": "pe-xl-core-0.lab1.puppet.vm"
+}
+```
+
+And for multiple compilers, this is the minimal parameters required.
+
+```json
+{
+  "compiler_hosts": [
+    "pe-xl-compiler-0.lab1.puppet.vm",
+    "pe-xl-compiler-1.lab1.puppet.vm"
+  ],
+  "primary_host": "pe-xl-core-0.lab1.puppet.vm"
+}
+```
+
+## Running the `add_compiler` plan
+
+```
+bolt plan run peadm::add_compilers --params @params.json
+```
+
+This command will retrieve the current PEADM configuration to determine the setup rules needed for a compiler's secondary PuppetDB instances. The plan will configure the primary with appropriate rules for allowing access from the new compiler. On the primary, the `puppet` service is stopped and the `pe-postgresql` service is reloaded. If required, a puppet agent will be installed on the new compiler host. The compiler agent's certificate is regenerated to include data required by the `peadm::subplans::component_install` plan. A subsequent Puppet agent run will happen on the following components.
+
+- _\<compiler-host\>_
+- _\<primary_postgresql_host\>_
+- _\<replica host\>_
+- _\<primary_postgresql_host\>_
+
+The `puppet` service is then restarted.
+
+## Optional Parameters
+
+As well as `compiler_hosts` and `primary_host`, the `add_compiler` plan has a number of optional parameters. These can be viewed in the following params example.
+
+```json
+{
+  "avail_group_letter": "B",
+  "compiler_hosts": "pe-xl-compiler-0.lab1.puppet.vm",
+  "dns_alt_names": ["puppet,puppet.lab1.puppet.vm"],
+  "primary_host": "pe-xl-core-0.lab1.puppet.vm",
+  "primary_postgresql_host": "pe-psql-6251cd-0.us-west1-a.c.slice-cody.internal"
+}
+```
+
+for multiple compilers.
+
+```json
+{
+  "avail_group_letter": "B",
+  "compiler_hosts": [
+    "pe-xl-compiler-0.lab1.puppet.vm",
+    "pe-xl-compiler-1.lab1.puppet.vm"
+  ],
+  "dns_alt_names": [
+    "puppet,puppet.lab1.puppet.vm",
+    "puppet2,puppet.lab2.puppet.vm"
+  ],
+  "primary_host": "pe-xl-core-0.lab1.puppet.vm",
+  "primary_postgresql_host": "pe-psql-6251cd-0.us-west1-a.c.slice-cody.internal"
+}
+```
+
+Please note, the optional parameters and values of the plan are as follows.
+
+<!-- table -->
+
+| Parameter                 | Default value | Description                                                                                                                    |
+| ------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------ |
+| `avail_group_letter`      | `A`           | By default, each compiler will be added to the primary group A.                                                                |
+| `dns_alt_names`           | `undef`       |                                                                                                                                |
+| `primary_postgresql_host` | `undef`       | By default, this will pre-populate to the required value depending if your architecture contains HA and or external databases. |
+
+For more information around adding compilers to your infrastructure [Expanding Your Deployment](expanding.md#adding-compilers-with-peadmadd_compiler)