From 146b1ad3fd4669c30560988fd7dc04a040cad7cd Mon Sep 17 00:00:00 2001 From: petergmurphy Date: Wed, 15 Jan 2025 15:12:11 +0000 Subject: [PATCH] (PE-39577) Optimise legacy compiler support This commit: - Adds the `node_group_unpin` task. - `node_group_unpin` task is called in the convert plan to remove legacy compilers from from the PE Master node group. - Legacy compilers `pp_auth_role` changed to `pe_compiler_legacy`. - Changes the PEADM config to use the PE Certificate Authority node group. - Removes peadm_legacy_compiler OID extension. - Adds a task to update PE Master rules to support 'pe_compiler_legacy' as a pp_auth_role. - Adds a task to check if PE Master rules have been updated to support the new method of legacy compiler identification. - This task is being used in the `add_compilers` and `convert_compiler_to_legacy` plans to check if the infrastructure has been converted to support this new method of legacy compiler identification. If the infrastructure has not been converted, it will fail each plan and tell the user to run the Convert plan. --- REFERENCE.md | 36 ++++- manifests/setup/legacy_compiler_group.pp | 53 ++++---- manifests/setup/node_manager.pp | 67 +++++----- plans/add_compilers.pp | 6 + plans/convert.pp | 61 ++++++++- plans/convert_compiler_to_legacy.pp | 14 +- plans/subplans/component_install.pp | 4 +- plans/subplans/configure.pp | 4 + plans/subplans/install.pp | 8 +- plans/update_compiler_extensions.pp | 25 ---- plans/upgrade.pp | 25 +--- spec/plans/add_compilers_spec.rb | 13 ++ spec/plans/convert_spec.rb | 6 +- spec/plans/upgrade_spec.rb | 18 ++- tasks/check_pe_master_rules.json | 10 ++ tasks/check_pe_master_rules.rb | 163 +++++++++++++++++++++++ tasks/get_peadm_config.rb | 8 +- tasks/node_group_unpin.json | 17 +++ tasks/node_group_unpin.rb | 118 ++++++++++++++++ tasks/update_pe_master_rules.json | 8 ++ tasks/update_pe_master_rules.rb | 119 +++++++++++++++++ 21 files changed, 648 insertions(+), 135 deletions(-) delete mode 100644 plans/update_compiler_extensions.pp create mode 100644 tasks/check_pe_master_rules.json create mode 100755 tasks/check_pe_master_rules.rb create mode 100644 tasks/node_group_unpin.json create mode 100755 tasks/node_group_unpin.rb create mode 100644 tasks/update_pe_master_rules.json create mode 100755 tasks/update_pe_master_rules.rb diff --git a/REFERENCE.md b/REFERENCE.md index 6cae62a6..9f897504 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -60,6 +60,7 @@ * [`backup_classification`](#backup_classification): A task to call the classification api and write to file * [`cert_data`](#cert_data): Return certificate data related to the Puppet agent * [`cert_valid_status`](#cert_valid_status): Check primary for valid state of a certificate +* [`check_pe_master_rules`](#check_pe_master_rules): Checks if the PE Master group rules have already been updated to support 'pe_compiler_legacy' as a pp_auth_role * [`classify_compilers`](#classify_compilers): Classify compilers as legacy or non-legacy * [`code_manager`](#code_manager): Perform various code manager actions * [`code_manager_enabled`](#code_manager_enabled): Run on a PE primary node to check if Code Manager is enabled. @@ -74,6 +75,7 @@ * [`infrastatus`](#infrastatus): Runs puppet infra status and returns the output * [`mkdir_p_file`](#mkdir_p_file): Create a file with the specified content at the specified location * [`mv`](#mv): Wrapper task for mv command +* [`node_group_unpin`](#node_group_unpin): Unpins nodes from a specified PE node group * [`os_identification`](#os_identification): Return the operating system runnin gon the target as a string * [`pe_install`](#pe_install): Install Puppet Enterprise from a tarball * [`pe_ldap_config`](#pe_ldap_config): Set the ldap config in the PE console @@ -90,6 +92,7 @@ * [`ssl_clean`](#ssl_clean): Clean an agent's certificate * [`submit_csr`](#submit_csr): Submit a certificate signing request * [`transform_classification_groups`](#transform_classification_groups): Transform the user groups from a source backup to a list of groups on the target server +* [`update_pe_master_rules`](#update_pe_master_rules): Updates the PE Master group rules to support 'pe_compiler_legacy' as a pp_auth_role * [`validate_rbac_token`](#validate_rbac_token): Check an RBAC token stored in a file is valid * [`wait_until_service_ready`](#wait_until_service_ready): Return when the orchestrator service is healthy, or timeout after 15 seconds @@ -129,7 +132,6 @@ Supported use cases: * `peadm::subplans::modify_certificate` * `peadm::subplans::prepare_agent` * `peadm::uninstall`: Single-entry-point plan for uninstalling Puppet Enterprise -* `peadm::update_compiler_extensions` * `peadm::util::code_sync_status` * `peadm::util::copy_file` * `peadm::util::db_disable_pglogical` @@ -1110,6 +1112,12 @@ Data type: `String` The certifcate name to check validation of +### `check_pe_master_rules` + +Checks if the PE Master group rules have already been updated to support 'pe_compiler_legacy' as a pp_auth_role + +**Supports noop?** false + ### `classify_compilers` Classify compilers as legacy or non-legacy @@ -1326,6 +1334,26 @@ Data type: `String` New path of file +### `node_group_unpin` + +Unpins nodes from a specified PE node group + +**Supports noop?** false + +#### Parameters + +##### `node_certnames` + +Data type: `Array[String]` + +The certnames of the nodes to unpin + +##### `group_name` + +Data type: `String` + +The name of the node group to unpin the nodes from + ### `os_identification` Return the operating system runnin gon the target as a string @@ -1622,6 +1650,12 @@ Data type: `String` Location of target node group yaml file and where to create the transformed file +### `update_pe_master_rules` + +Updates the PE Master group rules to support 'pe_compiler_legacy' as a pp_auth_role + +**Supports noop?** false + ### `validate_rbac_token` Check an RBAC token stored in a file is valid diff --git a/manifests/setup/legacy_compiler_group.pp b/manifests/setup/legacy_compiler_group.pp index 0fb161e4..4eff95b6 100644 --- a/manifests/setup/legacy_compiler_group.pp +++ b/manifests/setup/legacy_compiler_group.pp @@ -1,20 +1,19 @@ # @api private class peadm::setup::legacy_compiler_group ( String[1] $primary_host, - Optional[String] $internal_compiler_a_pool_address = undef, - Optional[String] $internal_compiler_b_pool_address = undef, + Optional[String] $internal_compiler_a_pool_address = undef, + Optional[String] $internal_compiler_b_pool_address = undef, ) { Node_group { purge_behavior => none, } node_group { 'PE Legacy Compiler': - parent => 'PE Master', - rule => ['and', - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], - ], - classes => { + ensure => 'present', + parent => 'PE Master', + purge_behavior => 'rule', + rule => ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], + classes => { 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_a_pool_address, $internal_compiler_b_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], @@ -23,21 +22,20 @@ } node_group { 'PE Legacy Compiler Group A': - ensure => 'present', - parent => 'PE Legacy Compiler', - rule => ['and', - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], + ensure => 'present', + parent => 'PE Legacy Compiler', + purge_behavior => 'rule', + rule => ['and', + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'A'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], ], - classes => { - 'puppet_enterprise::profile::master' => { + classes => { + 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_b_pool_address, $internal_compiler_a_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], }, }, - data => { - # Workaround for GH-118 + data => { 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], }, @@ -45,21 +43,20 @@ } node_group { 'PE Legacy Compiler Group B': - ensure => 'present', - parent => 'PE Legacy Compiler', - rule => ['and', - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], + ensure => 'present', + parent => 'PE Legacy Compiler', + purge_behavior => 'rule', + rule => ['and', + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'B'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], ], - classes => { - 'puppet_enterprise::profile::master' => { + classes => { + 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_a_pool_address, $internal_compiler_b_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], }, }, - data => { - # Workaround for GH-118 + data => { 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], }, @@ -67,6 +64,8 @@ } node_group { 'PE Compiler': - rule => ['and', ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'false']], + parent => 'PE Master', + purge_behavior => 'rule', + rule => ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], } } diff --git a/manifests/setup/node_manager.pp b/manifests/setup/node_manager.pp index f74cb217..9fcc336b 100644 --- a/manifests/setup/node_manager.pp +++ b/manifests/setup/node_manager.pp @@ -81,8 +81,9 @@ # PE Compiler group comes from default PE and already has the pe compiler role node_group { 'PE Compiler': - parent => 'PE Master', - rule => ['and', ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'false']], + parent => 'PE Master', + purge_behavior => 'rule', + rule => ['and', ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler']], } # This group should pin the primary, and also map to any pe-postgresql nodes @@ -116,14 +117,14 @@ # Configure the A pool for compilers. There are up to two pools for DR, each # having an affinity for one "availability zone" or the other. node_group { 'PE Compiler Group A': - ensure => 'present', - parent => 'PE Compiler', - rule => ['and', + ensure => 'present', + purge_behavior => 'rule', + parent => 'PE Compiler', + rule => ['and', ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'A'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'false'], ], - classes => { + classes => { 'puppet_enterprise::profile::puppetdb' => { 'database_host' => pick($postgresql_a_host, $notconf), }, @@ -134,7 +135,7 @@ 'puppetdb_port' => [8081], }, }, - data => { + data => { # Workaround for GH-118 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], @@ -175,14 +176,14 @@ } node_group { 'PE Compiler Group B': - ensure => 'present', - parent => 'PE Compiler', - rule => ['and', + ensure => 'present', + purge_behavior => 'rule', + parent => 'PE Compiler', + rule => ['and', ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'B'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'false'], ], - classes => { + classes => { 'puppet_enterprise::profile::puppetdb' => { 'database_host' => pick($postgresql_b_host, $notconf), }, @@ -193,7 +194,7 @@ 'puppetdb_port' => [8081], }, }, - data => { + data => { # Workaround for GH-118 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], @@ -202,12 +203,10 @@ } node_group { 'PE Legacy Compiler': - parent => 'PE Master', - rule => ['and', - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], - ], - classes => { + parent => 'PE Master', + purge_behavior => 'rule', + rule => ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], + classes => { 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_a_pool_address, $internal_compiler_b_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], @@ -218,20 +217,20 @@ # Configure the A pool for legacy compilers. There are up to two pools for DR, each # having an affinity for one "availability zone" or the other. node_group { 'PE Legacy Compiler Group A': - ensure => 'present', - parent => 'PE Legacy Compiler', - rule => ['and', - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], + ensure => 'present', + parent => 'PE Legacy Compiler', + purge_behavior => 'rule', + rule => ['and', + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'A'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], ], - classes => { + classes => { 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_b_pool_address, $internal_compiler_a_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], }, }, - data => { + data => { # Workaround for GH-118 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], @@ -242,20 +241,20 @@ # Configure the B pool for legacy compilers. There are up to two pools for DR, each # having an affinity for one "availability zone" or the other. node_group { 'PE Legacy Compiler Group B': - ensure => 'present', - parent => 'PE Legacy Compiler', - rule => ['and', - ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], + ensure => 'present', + parent => 'PE Legacy Compiler', + purge_behavior => 'rule', + rule => ['and', + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], ['=', ['trusted', 'extensions', peadm::oid('peadm_availability_group')], 'B'], - ['=', ['trusted', 'extensions', peadm::oid('peadm_legacy_compiler')], 'true'], ], - classes => { + classes => { 'puppet_enterprise::profile::master' => { 'puppetdb_host' => [$internal_compiler_a_pool_address, $internal_compiler_a_pool_address].filter |$_| { $_ }, 'puppetdb_port' => [8081], }, }, - data => { + data => { # Workaround for GH-118 'puppet_enterprise::profile::master::puppetdb' => { 'ha_enabled_replicas' => [], diff --git a/plans/add_compilers.pp b/plans/add_compilers.pp index 879d9fb2..a57e3c39 100644 --- a/plans/add_compilers.pp +++ b/plans/add_compilers.pp @@ -16,6 +16,12 @@ $compiler_targets = peadm::get_targets($compiler_hosts) $primary_target = peadm::get_targets($primary_host, 1) + # Check if PE Master rules have been updated to support pe_compiler_legacy + $rules_check = run_task('peadm::check_pe_master_rules', $primary_host).first.value + unless $rules_check['updated'] { + fail_plan('Please run the Convert plan to convert your Puppet infrastructure to be managed by this version of PEADM.') + } + # Get current peadm config to determine where to setup additional rules for # compiler's secondary PuppetDB instances $peadm_config = run_task('peadm::get_peadm_config', $primary_target).first.value diff --git a/plans/convert.pp b/plans/convert.pp index 1ff1771a..585575b1 100644 --- a/plans/convert.pp +++ b/plans/convert.pp @@ -60,6 +60,48 @@ out::message('# Gathering information') + $cert_extensions_temp = run_task('peadm::cert_data', $all_targets).reduce({}) |$memo,$result| { + $memo + { $result.target.peadm::certname() => $result['extensions'] } + } + + # Add legacy compiler role to compilers that are missing it + $compilers_with_legacy_compiler_flag = $cert_extensions_temp.filter |$name, $exts| { + ($name in $compiler_targets.map |$t| { $t.name } or $name in $legacy_compiler_targets.map |$t| { $t.name }) and + $exts and $exts[peadm::oid('peadm_legacy_compiler')] != undef + } + + if $compilers_with_legacy_compiler_flag.size > 0 { + $legacy_compilers_with_flag = $compilers_with_legacy_compiler_flag.filter |$name,$exts| { + $exts[peadm::oid('peadm_legacy_compiler')] == 'true' + }.keys + + $modern_compilers_with_flag = $compilers_with_legacy_compiler_flag.filter |$name,$exts| { + $exts[peadm::oid('peadm_legacy_compiler')] == 'false' + }.keys + + if $modern_compilers_with_flag.size > 0 { + run_plan('peadm::modify_certificate', $modern_compilers_with_flag, + primary_host => $primary_target, + remove_extensions => [peadm::oid('peadm_legacy_compiler')], + ) + } + + if $legacy_compilers_with_flag.size > 0 { + run_plan('peadm::modify_certificate', $legacy_compilers_with_flag, + primary_host => $primary_target, + add_extensions => { + 'pp_auth_role' => 'pe_compiler_legacy', + }, + remove_extensions => [peadm::oid('peadm_legacy_compiler'), peadm::oid('pp_auth_role')], + ) + } + + run_task('peadm::puppet_runonce', peadm::flatten_compact([ + $compiler_targets, + $legacy_compiler_targets, + ])) + } + # Get trusted fact information for all compilers. Use peadm::certname() as # the hash key because the apply block below will break trying to parse the # $compiler_extensions variable if it has Target-type hash keys. @@ -214,7 +256,6 @@ add_extensions => { peadm::oid('pp_auth_role') => 'pe_compiler', peadm::oid('peadm_availability_group') => 'A', - peadm::oid('peadm_legacy_compiler') => 'false', }, ) }, @@ -224,7 +265,6 @@ add_extensions => { peadm::oid('pp_auth_role') => 'pe_compiler', peadm::oid('peadm_availability_group') => 'B', - peadm::oid('peadm_legacy_compiler') => 'false', }, ) }, @@ -232,9 +272,8 @@ run_plan('peadm::modify_certificate', $legacy_compiler_a_targets, primary_host => $primary_target, add_extensions => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'A', - peadm::oid('peadm_legacy_compiler') => 'true', }, ) }, @@ -242,9 +281,8 @@ run_plan('peadm::modify_certificate', $legacy_compiler_b_targets, primary_host => $primary_target, add_extensions => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'B', - peadm::oid('peadm_legacy_compiler') => 'true', }, ) }, @@ -283,6 +321,14 @@ include peadm::setup::convert_node_manager } + + # Unpin legacy compilers from PE Master group + if $legacy_compiler_targets { + run_task('peadm::node_group_unpin', $primary_target, + node_certnames => $legacy_compiler_targets.map |$target| { $target.peadm::certname() }, + group_name => 'PE Master', + ) + } } else { # lint:ignore:strict_indent @@ -314,6 +360,9 @@ run_command('systemctl restart pe-puppetserver.service pe-puppetdb.service', $compiler_targets) } + # Update PE Master rules to support legacy compilers + run_task('peadm::update_pe_master_rules', $primary_target) + # Run puppet on all targets again to ensure everything is fully up-to-date run_task('peadm::puppet_runonce', $all_targets) } diff --git a/plans/convert_compiler_to_legacy.pp b/plans/convert_compiler_to_legacy.pp index c75924bd..c612684b 100644 --- a/plans/convert_compiler_to_legacy.pp +++ b/plans/convert_compiler_to_legacy.pp @@ -7,6 +7,12 @@ $primary_target = peadm::get_targets($primary_host, 1) $convert_legacy_compiler_targets = peadm::get_targets($legacy_hosts) + # Check if PE Master rules have been updated to support pe_compiler_legacy + $rules_check = run_task('peadm::check_pe_master_rules', $primary_target).first.value + unless $rules_check['updated'] { + fail_plan('Please run the Convert plan to convert your Puppet infrastructure to be managed by this version of PEADM.') + } + $cluster = run_task('peadm::get_peadm_config', $primary_host).first.value $error = getvar('cluster.error') if $error { @@ -102,7 +108,7 @@ run_plan('peadm::modify_certificate', $compiler_targets, primary_host => $primary_target, add_extensions => { - peadm::oid('peadm_legacy_compiler') => 'false', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', }, ) }, @@ -110,9 +116,8 @@ run_plan('peadm::modify_certificate', $legacy_compiler_a_targets, primary_host => $primary_target, add_extensions => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'A', - peadm::oid('peadm_legacy_compiler') => 'true', }, ) }, @@ -120,9 +125,8 @@ run_plan('peadm::modify_certificate', $legacy_compiler_b_targets, primary_host => $primary_target, add_extensions => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'B', - peadm::oid('peadm_legacy_compiler') => 'true', }, ) }, diff --git a/plans/subplans/component_install.pp b/plans/subplans/component_install.pp index c117ccb6..2fad74e8 100644 --- a/plans/subplans/component_install.pp +++ b/plans/subplans/component_install.pp @@ -21,13 +21,11 @@ $certificate_extensions = { peadm::oid('pp_auth_role') => 'pe_compiler', peadm::oid('peadm_availability_group') => $avail_group_letter, - peadm::oid('peadm_legacy_compiler') => false, } } elsif $role == 'pe_compiler_legacy' { $certificate_extensions = { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => $avail_group_letter, - peadm::oid('peadm_legacy_compiler') => true, } } else { $certificate_extensions = { diff --git a/plans/subplans/configure.pp b/plans/subplans/configure.pp index f2ebada6..6cc6e1d7 100644 --- a/plans/subplans/configure.pp +++ b/plans/subplans/configure.pp @@ -174,5 +174,9 @@ $legacy_compiler_targets, ])) + # Update PE Master rules to support legacy compilers + run_task('peadm::update_pe_master_rules', $primary_host) + run_task('peadm::puppet_runonce', $legacy_compiler_targets) + return("Configuration of Puppet Enterprise ${arch['architecture']} succeeded.") } diff --git a/plans/subplans/install.pp b/plans/subplans/install.pp index 693c056c..96af47f0 100644 --- a/plans/subplans/install.pp +++ b/plans/subplans/install.pp @@ -287,7 +287,6 @@ extension_requests => { peadm::oid('pp_auth_role') => 'pe_compiler', peadm::oid('peadm_availability_group') => 'A', - peadm::oid('peadm_legacy_compiler') => 'false', } ) }, @@ -296,25 +295,22 @@ extension_requests => { peadm::oid('pp_auth_role') => 'pe_compiler', peadm::oid('peadm_availability_group') => 'B', - peadm::oid('peadm_legacy_compiler') => 'false', } ) }, background('compiler-a-csr.yaml') || { run_plan('peadm::util::insert_csr_extension_requests', $legacy_a_targets, extension_requests => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'A', - peadm::oid('peadm_legacy_compiler') => 'true', } ) }, background('compiler-b-csr.yaml') || { run_plan('peadm::util::insert_csr_extension_requests', $legacy_b_targets, extension_requests => { - peadm::oid('pp_auth_role') => 'pe_compiler', + peadm::oid('pp_auth_role') => 'pe_compiler_legacy', peadm::oid('peadm_availability_group') => 'B', - peadm::oid('peadm_legacy_compiler') => 'true', } ) }, diff --git a/plans/update_compiler_extensions.pp b/plans/update_compiler_extensions.pp deleted file mode 100644 index 784f919e..00000000 --- a/plans/update_compiler_extensions.pp +++ /dev/null @@ -1,25 +0,0 @@ -# @api private -plan peadm::update_compiler_extensions ( - TargetSpec $compiler_hosts, - Peadm::SingleTargetSpec $primary_host, - Boolean $legacy = false, -) { - $primary_target = peadm::get_targets($primary_host, 1) - $host_targets = peadm::get_targets($compiler_hosts) - - run_plan('peadm::modify_certificate', $host_targets, - primary_host => $primary_target, - add_extensions => { peadm::oid('peadm_legacy_compiler') => String($legacy) }, - ) - - run_task('peadm::puppet_runonce', $primary_target) - run_task('peadm::puppet_runonce', $host_targets) - - if $legacy { - run_command('systemctl restart pe-puppetserver.service', $host_targets) - } else { - run_command('systemctl restart pe-puppetserver.service pe-puppetdb.service', $host_targets) - } - - return("Added legacy cert with value ${legacy} to compiler hosts ${compiler_hosts}") -} diff --git a/plans/upgrade.pp b/plans/upgrade.pp index 9b8a4116..5fe36cc4 100644 --- a/plans/upgrade.pp +++ b/plans/upgrade.pp @@ -135,22 +135,11 @@ peadm::assert_supported_pe_version($_version, $permit_unsafe_versions) - # Gather certificate extension information from all systems - $cert_extensions_temp = run_task('peadm::cert_data', $all_targets).reduce({}) |$memo,$result| { - $memo + { $result.target.peadm::certname => $result['extensions'] } + $rules_check = run_task('peadm::check_pe_master_rules', $primary_target).first.value + unless $rules_check['updated'] { + fail_plan('Please run the Convert plan to convert your Puppet infrastructure to be managed by this version of PEADM.') } - $compiler_missing_legacy_targets = $cert_extensions_temp.filter |$name,$exts| { - ($name in $compiler_targets.map |$t| { $t.name }) and (peadm::oid('peadm_legacy_compiler') in $exts and $exts[peadm::oid('peadm_legacy_compiler')] == undef) - }.keys - - run_plan('peadm::modify_certificate', $compiler_missing_legacy_targets, - primary_host => $primary_target, - add_extensions => { - peadm::oid('peadm_legacy_compiler') => 'false', - }, - ) - # Gather certificate extension information from all systems $cert_extensions = run_task('peadm::cert_data', $all_targets).reduce({}) |$memo,$result| { $memo + { $result.target.peadm::certname => $result['extensions'] } @@ -188,8 +177,8 @@ $compiler_m1_nonlegacy_targets = $compiler_targets.filter |$target| { ($cert_extensions.dig($target.peadm::certname, peadm::oid('peadm_availability_group')) == $cert_extensions.dig($primary_target[0].peadm::certname, peadm::oid('peadm_availability_group'))) and - ($cert_extensions.dig($target.peadm::certname, peadm::oid('peadm_legacy_compiler')) - == 'false') + ($cert_extensions.dig($target.peadm::certname, peadm::oid('pp_auth_role')) + == 'pe_compiler') } $compiler_m2_targets = $compiler_targets.filter |$target| { @@ -200,8 +189,8 @@ $compiler_m2_nonlegacy_targets = $compiler_targets.filter |$target| { ($cert_extensions.dig($target.peadm::certname, peadm::oid('peadm_availability_group')) == $cert_extensions.dig($replica_target[0].peadm::certname, peadm::oid('peadm_availability_group'))) and - ($cert_extensions.dig($target.peadm::certname, peadm::oid('peadm_legacy_compiler')) - == 'false') + ($cert_extensions.dig($target.peadm::certname, peadm::oid('pp_auth_role')) + == 'pe_compiler') } peadm::plan_step('preparation') || { diff --git a/spec/plans/add_compilers_spec.rb b/spec/plans/add_compilers_spec.rb index 9dbb4f94..76acfa1f 100644 --- a/spec/plans/add_compilers_spec.rb +++ b/spec/plans/add_compilers_spec.rb @@ -47,10 +47,18 @@ def allow_standard_non_returning_calls } end + let(:pe_rule_check) do + { + 'updated' => 'true', + 'message' => 'a message' + } + end + it 'runs successfully when no alt-names are specified' do allow_standard_non_returning_calls expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['server_a']) expect_plan('peadm::subplans::component_install') @@ -65,6 +73,7 @@ def allow_standard_non_returning_calls cfg['role-letter']['server']['B'] = 'server_b' expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['server_b']) expect_plan('peadm::subplans::component_install') @@ -79,6 +88,7 @@ def allow_standard_non_returning_calls allow_standard_non_returning_calls expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['custom_postgresql']) expect_plan('peadm::subplans::component_install') @@ -94,6 +104,7 @@ def allow_standard_non_returning_calls cfg['params']['replica_postgresql_host'] = 'external_postgresql' expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['external_postgresql']) expect_plan('peadm::subplans::component_install') @@ -109,6 +120,7 @@ def allow_standard_non_returning_calls cfg['role-letter']['server']['B'] = 'replica' expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['external_postgresql']) expect_plan('peadm::subplans::component_install') @@ -124,6 +136,7 @@ def allow_standard_non_returning_calls cfg['params']['replica_postgresql_host'] = 'replica_external_postgresql' expect_task('peadm::get_peadm_config').always_return(cfg) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect_task('peadm::get_psql_version').with_targets(['replica_external_postgresql']) expect_plan('peadm::subplans::component_install') diff --git a/spec/plans/convert_spec.rb b/spec/plans/convert_spec.rb index 39ec7367..99e453a4 100644 --- a/spec/plans/convert_spec.rb +++ b/spec/plans/convert_spec.rb @@ -9,7 +9,7 @@ end let(:params) do - { 'primary_host' => 'primary' } + { 'primary_host' => 'primary', 'legacy_compilers' => ['pe_compiler_legacy'] } end it 'single primary no dr valid' do @@ -18,9 +18,11 @@ allow_any_task allow_apply - expect_task('peadm::cert_data').return_for_targets('primary' => trustedjson) + expect_task('peadm::cert_data').return_for_targets('primary' => trustedjson).be_called_times(2) expect_task('peadm::read_file').always_return({ 'content' => '2021.7.9' }) expect_task('peadm::get_group_rules').return_for_targets('primary' => { '_output' => '{"rules": []}' }) + expect_task('peadm::node_group_unpin').with_targets('primary').with_params({ 'node_certnames' => ['pe_compiler_legacy'], 'group_name' => 'PE Master' }) + expect_task('peadm::check_legacy_compilers').with_targets('primary').with_params({ 'legacy_compilers' => 'pe_compiler_legacy' }).return_for_targets('primary' => { '_output' => '' }) # For some reason, expect_plan() was not working?? allow_plan('peadm::modify_certificate').always_return({}) diff --git a/spec/plans/upgrade_spec.rb b/spec/plans/upgrade_spec.rb index 41852f01..b92b828f 100644 --- a/spec/plans/upgrade_spec.rb +++ b/spec/plans/upgrade_spec.rb @@ -20,6 +20,13 @@ def allow_standard_non_returning_calls JSON.parse File.read(File.expand_path(File.join(fixtures, 'plans', 'trusted-compiler.json'))) end + let(:pe_rule_check) do + { + 'updated' => 'true', + 'message' => 'a message' + } + end + it 'minimum variables to run' do allow_standard_non_returning_calls expect_task('peadm::get_group_rules').return_for_targets('primary' => { '_output' => '{"rules": []}' }) @@ -28,7 +35,8 @@ def allow_standard_non_returning_calls .with_params('path' => '/opt/puppetlabs/server/pe_build') .always_return({ 'content' => '2021.7.3' }) - expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(2) + expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(1) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect(run_plan('peadm::upgrade', 'primary_host' => 'primary', @@ -44,7 +52,8 @@ def allow_standard_non_returning_calls .always_return({ 'content' => '2021.7.3' }) expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary, - 'compiler' => trusted_compiler).be_called_times(2) + 'compiler' => trusted_compiler).be_called_times(1) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check).be_called_times(1) expect(run_plan('peadm::upgrade', 'primary_host' => 'primary', @@ -93,7 +102,7 @@ def allow_standard_non_returning_calls .with_params('path' => '/opt/puppetlabs/server/pe_build') .always_return({ 'content' => installed_version }) - expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(2) + expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(1) expect_task('peadm::get_group_rules').return_for_targets('primary' => { '_output' => '{"rules": []}' }) end @@ -109,6 +118,7 @@ def allow_standard_non_returning_calls # uploading runs afoul of the fact that write_file creates a source tempfile, # and we can't expect_upload() because we don't have the tempfile name. allow_any_upload + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect(run_plan('peadm::upgrade', 'primary_host' => 'primary', @@ -120,6 +130,7 @@ def allow_standard_non_returning_calls it 'warns if upgrading to 2023.3+ from 2023.0- without r10k_known_hosts set' do # This is fairly horrible, but expect_out_message doesn't take a regex. expect_out_message.with_params(r10k_warning) + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect(run_plan('peadm::upgrade', 'primary_host' => 'primary', @@ -132,6 +143,7 @@ def allow_standard_non_returning_calls it 'does not warn if r10k_known_hosts is not set' do expect_out_message.with_params(r10k_warning).not_be_called + expect_task('peadm::check_pe_master_rules').always_return(pe_rule_check) expect(run_plan('peadm::upgrade', 'primary_host' => 'primary', diff --git a/tasks/check_pe_master_rules.json b/tasks/check_pe_master_rules.json new file mode 100644 index 00000000..7eb2f3b8 --- /dev/null +++ b/tasks/check_pe_master_rules.json @@ -0,0 +1,10 @@ +{ + "description": "Checks if the PE Master group rules have already been updated to support 'pe_compiler_legacy' as a pp_auth_role", + "input_method": "stdin", + "private": true, + "implementations": [ + {"name": "check_pe_master_rules.rb"} + ], + "parameters": {}, + "supports_noop": false +} \ No newline at end of file diff --git a/tasks/check_pe_master_rules.rb b/tasks/check_pe_master_rules.rb new file mode 100755 index 00000000..bed87f8b --- /dev/null +++ b/tasks/check_pe_master_rules.rb @@ -0,0 +1,163 @@ +#!/opt/puppetlabs/puppet/bin/ruby +# frozen_string_literal: true + +require 'json' +require 'net/https' +require 'puppet' + +# CheckPeMasterRules task class +class CheckPeMasterRules + def initialize(params) + @params = params + end + + def https_client + client = Net::HTTP.new(Puppet.settings[:certname], 4433) + client.use_ssl = true + client.cert = @cert ||= OpenSSL::X509::Certificate.new(File.read(Puppet.settings[:hostcert])) + client.key = @key ||= OpenSSL::PKey::RSA.new(File.read(Puppet.settings[:hostprivkey])) + client.verify_mode = OpenSSL::SSL::VERIFY_PEER + client.ca_file = Puppet.settings[:localcacert] + client + end + + def get_pe_master_group_id + net = https_client + res = net.get('/classifier-api/v1/groups') + + unless res.code == '200' + raise "Failed to fetch groups: HTTP #{res.code} - #{res.body}" + end + + groups = JSON.parse(res.body) + pe_master_group = groups.find { |group| group['name'] == 'PE Master' } + + raise 'Could not find PE Master group' unless pe_master_group + pe_master_group['id'] + rescue JSON::ParserError => e + raise "Invalid JSON response from server: #{e.message}" + rescue StandardError => e + raise "Error fetching PE Master group ID: #{e.message}" + end + + def get_current_rules(group_id) + net = https_client + url = "/classifier-api/v1/groups/#{group_id}/rules" + req = Net::HTTP::Get.new(url) + res = net.request(req) + + unless res.code == '200' + raise "Failed to fetch rules: HTTP #{res.code} - #{res.body}" + end + + JSON.parse(res.body)['rule'] + rescue JSON::ParserError => e + raise "Invalid JSON response from server: #{e.message}" + rescue StandardError => e + raise "Error fetching rules: #{e.message}" + end + + def check_rules_updated(rules) + # If not an array, return false + return false unless rules.is_a?(Array) + + # Check if there is at least 2 elements + if rules.length > 1 + # Check if the first element is an 'or' rule for pe_compiler and pe_compiler_legacy + if rules[1].is_a?(Array) && rules[1][0] == 'or' + # Look for the pe_compiler and pe_compiler_legacy rules + pe_compiler_found = false + pe_compiler_legacy_found = false + + rules[1][1..-1].each do |rule| + next unless rule.is_a?(Array) && + rule[0] == '=' && + rule[1].is_a?(Array) && + rule[1] == ['trusted', 'extensions', 'pp_auth_role'] + + pe_compiler_found = true if rule[2] == 'pe_compiler' + pe_compiler_legacy_found = true if rule[2] == 'pe_compiler_legacy' + end + + return pe_compiler_found && pe_compiler_legacy_found + end + end + + false + end + + def https_pdb_client(port = 8081) + client = Net::HTTP.new(Puppet.settings[:certname], port) + client.use_ssl = true + client.cert = @cert ||= OpenSSL::X509::Certificate.new(File.read(Puppet.settings[:hostcert])) + client.key = @key ||= OpenSSL::PKey::RSA.new(File.read(Puppet.settings[:hostprivkey])) + client.verify_mode = OpenSSL::SSL::VERIFY_PEER + client.ca_file = Puppet.settings[:localcacert] + client + end + + def check_nodes_with_legacy_compiler_oid + pdb = https_pdb_client + pdb_request = Net::HTTP::Get.new('/pdb/query/v4') + pdb_request.set_form_data({ + 'query' => 'inventory[certname,trusted.extensions] { + trusted.extensions."1.3.6.1.4.1.34380.1.1.9814" is not null + }' + }) + + response = pdb.request(pdb_request) + + unless response.code == '200' + raise "Failed to query PuppetDB: HTTP #{response.code} - #{response.body}" + end + + nodes = JSON.parse(response.body) + + { + 'nodes_found' => !nodes.empty?, + 'count' => nodes.size, + 'nodes' => nodes.map { |n| n['certname'] } + } + rescue JSON::ParserError => e + raise "Invalid JSON response from PuppetDB: #{e.message}" + rescue StandardError => e + raise "Error checking for legacy compiler OID: #{e.message}" + end + + def execute! + group_id = get_pe_master_group_id + current_rules = get_current_rules(group_id) + + rules_updated = check_rules_updated(current_rules) + legacy_compiler_nodes = check_nodes_with_legacy_compiler_oid + + # Overall status is updated only if rules are updated AND no nodes have legacy compiler OID + is_updated = rules_updated && !legacy_compiler_nodes['nodes_found'] + + message = if !rules_updated + 'PE Master rules need to be updated to support pe_compiler_legacy' + elsif legacy_compiler_nodes['nodes_found'] + 'PE Master rules are updated, but nodes with legacy compiler OID still exist' + else + 'PE Master rules have been updated with pe_compiler_legacy support and no legacy compiler OIDs found' + end + + result = { + 'updated' => is_updated, + 'message' => message, + 'legacy_compiler_oid' => legacy_compiler_nodes + } + + puts result.to_json + rescue StandardError => e + puts({ 'error' => e.message, 'updated' => false }.to_json) + exit 1 + end +end + +# Run the task unless an environment flag has been set +unless ENV['RSPEC_UNIT_TEST_MODE'] + Puppet.initialize_settings + task = CheckPeMasterRules.new(JSON.parse(STDIN.read)) + task.execute! +end diff --git a/tasks/get_peadm_config.rb b/tasks/get_peadm_config.rb index 9eb3aa02..8d24117f 100755 --- a/tasks/get_peadm_config.rb +++ b/tasks/get_peadm_config.rb @@ -22,7 +22,7 @@ def execute! def config # Compute values - primary = groups.pinned('PE Master') + primary = groups.pinned('PE Certificate Authority') replica = groups.pinned('PE HA Replica') server_a = server('puppet/server', 'A', [primary, replica].compact) server_b = server('puppet/server', 'B', [primary, replica].compact) @@ -94,8 +94,7 @@ def groups def compilers @compilers ||= pdb_query('inventory[certname,trusted.extensions] { - trusted.extensions.pp_auth_role = "pe_compiler" and - trusted.extensions."1.3.6.1.4.1.34380.1.1.9814" = "false" + trusted.extensions.pp_auth_role = "pe_compiler" }').map do |c| { 'certname' => c['certname'], @@ -108,8 +107,7 @@ def compilers def legacy_compilers @legacy_compilers ||= pdb_query('inventory[certname,trusted.extensions] { - trusted.extensions.pp_auth_role = "pe_compiler" and - trusted.extensions."1.3.6.1.4.1.34380.1.1.9814" = "true" + trusted.extensions.pp_auth_role = "pe_compiler_legacy" }').map do |c| { 'certname' => c['certname'], diff --git a/tasks/node_group_unpin.json b/tasks/node_group_unpin.json new file mode 100644 index 00000000..c94f3654 --- /dev/null +++ b/tasks/node_group_unpin.json @@ -0,0 +1,17 @@ +{ + "description": "Unpins nodes from a specified PE node group", + "parameters": { + "node_certnames": { + "type": "Array[String]", + "description": "The certnames of the nodes to unpin" + }, + "group_name": { + "type": "String", + "description": "The name of the node group to unpin the nodes from" + } + }, + "input_method": "stdin", + "implementations": [ + {"name": "node_group_unpin.rb"} + ] +} \ No newline at end of file diff --git a/tasks/node_group_unpin.rb b/tasks/node_group_unpin.rb new file mode 100755 index 00000000..2a2e5637 --- /dev/null +++ b/tasks/node_group_unpin.rb @@ -0,0 +1,118 @@ +#!/opt/puppetlabs/puppet/bin/ruby +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'net/https' +require 'puppet' + +# NodeGroupUnpin task class +class NodeGroupUnpin + def initialize(params) + @params = params + raise "Missing required parameter 'node_certnames'" unless @params['node_certnames'] + raise "'node_certnames' must be an array" unless @params['node_certnames'].is_a?(Array) + raise "Missing required parameter 'group_name'" unless @params['group_name'] + @auth = YAML.load_file('/etc/puppetlabs/puppet/classifier.yaml') + rescue Errno::ENOENT + raise 'Could not find classifier.yaml at /etc/puppetlabs/puppet/classifier.yaml' + end + + def https_client + client = Net::HTTP.new(Puppet.settings[:certname], 4433) + client.use_ssl = true + client.cert = @cert ||= OpenSSL::X509::Certificate.new(File.read(Puppet.settings[:hostcert])) + client.key = @key ||= OpenSSL::PKey::RSA.new(File.read(Puppet.settings[:hostprivkey])) + client.verify_mode = OpenSSL::SSL::VERIFY_PEER + client.ca_file = Puppet.settings[:localcacert] + client + end + + def groups + @groups ||= begin + net = https_client + res = net.get('/classifier-api/v1/groups') + + unless res.code == '200' + raise "Failed to fetch groups: HTTP #{res.code} - #{res.body}" + end + + NodeGroup.new(JSON.parse(res.body)) + rescue JSON::ParserError => e + raise "Invalid JSON response from server: #{e.message}" + rescue StandardError => e + raise "Error fetching groups: #{e.message}" + end + end + + def unpin_node(group, nodes) + raise 'Invalid group object' unless group.is_a?(Hash) && group['id'] && group['name'] + + net = https_client + begin + data = { "nodes": nodes }.to_json + url = "/classifier-api/v1/groups/#{group['id']}/unpin" + + req = Net::HTTP::Post.new(url) + req['Content-Type'] = 'application/json' + req.body = data + + res = net.request(req) + + case res.code + when '204' + puts "Successfully unpinned nodes '#{nodes.join(', ')}' from group '#{group['name']}'" + else + begin + error_body = JSON.parse(res.body.to_s) + raise "Failed to unpin nodes: #{error_body['kind'] || error_body}" + rescue JSON::ParserError + raise "Invalid response from server (status #{res.code}): #{res.body}" + end + end + rescue StandardError => e + raise "Error during unpin request: #{e.message}" + end + end + + # Utility class to aid in retrieving useful information from the node group + # data + class NodeGroup + attr_reader :data + + def initialize(data) + @data = data + end + + # Aids in digging into node groups by name, rather than UUID + def dig(name, *args) + group = @data.find { |obj| obj['name'] == name } + if group.nil? + nil + elsif args.empty? + group + else + group.dig(*args) + end + end + end + + def execute! + group_name = @params['group_name'] + node_certnames = @params['node_certnames'] + group = groups.dig(group_name) + if group + unpin_node(group, node_certnames) + puts "Unpinned #{node_certnames.join(', ')} from #{group_name}" + else + puts "Group #{group_name} not found" + end + end +end + +# Run the task unless an environment flag has been set +unless ENV['RSPEC_UNIT_TEST_MODE'] + Puppet.initialize_settings + task = NodeGroupUnpin.new(JSON.parse(STDIN.read)) + task.execute! +end diff --git a/tasks/update_pe_master_rules.json b/tasks/update_pe_master_rules.json new file mode 100644 index 00000000..e64663e9 --- /dev/null +++ b/tasks/update_pe_master_rules.json @@ -0,0 +1,8 @@ +{ + "description": "Updates the PE Master group rules to support 'pe_compiler_legacy' as a pp_auth_role", + "input_method": "stdin", + "private": true, + "implementations": [ + {"name": "update_pe_master_rules.rb"} + ] +} \ No newline at end of file diff --git a/tasks/update_pe_master_rules.rb b/tasks/update_pe_master_rules.rb new file mode 100755 index 00000000..947b187a --- /dev/null +++ b/tasks/update_pe_master_rules.rb @@ -0,0 +1,119 @@ +#!/opt/puppetlabs/puppet/bin/ruby +# frozen_string_literal: true + +require 'json' +require 'net/https' +require 'puppet' + +# UpdatePeMasterRules task class +class UpdatePeMasterRules + def initialize(params) + @params = params + end + + def https_client + client = Net::HTTP.new(Puppet.settings[:certname], 4433) + client.use_ssl = true + client.cert = @cert ||= OpenSSL::X509::Certificate.new(File.read(Puppet.settings[:hostcert])) + client.key = @key ||= OpenSSL::PKey::RSA.new(File.read(Puppet.settings[:hostprivkey])) + client.verify_mode = OpenSSL::SSL::VERIFY_PEER + client.ca_file = Puppet.settings[:localcacert] + client + end + + def get_pe_master_group_id + net = https_client + res = net.get('/classifier-api/v1/groups') + + unless res.code == '200' + raise "Failed to fetch groups: HTTP #{res.code} - #{res.body}" + end + + groups = JSON.parse(res.body) + pe_master_group = groups.find { |group| group['name'] == 'PE Master' } + + raise 'Could not find PE Master group' unless pe_master_group + pe_master_group['id'] + rescue JSON::ParserError => e + raise "Invalid JSON response from server: #{e.message}" + rescue StandardError => e + raise "Error fetching PE Master group ID: #{e.message}" + end + + def get_current_rules(group_id) + net = https_client + url = "/classifier-api/v1/groups/#{group_id}/rules" + req = Net::HTTP::Get.new(url) + res = net.request(req) + + unless res.code == '200' + raise "Failed to fetch rules: HTTP #{res.code} - #{res.body}" + end + + JSON.parse(res.body)['rule'] + rescue JSON::ParserError => e + raise "Invalid JSON response from server: #{e.message}" + rescue StandardError => e + raise "Error fetching rules: #{e.message}" + end + + def modify_pe_master_rules(rules) + # If not an array, return as is + return rules unless rules.is_a?(Array) + + # Make a copy of the rules to avoid modifying the original + result = rules.dup + + result[1] = [ + 'or', + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler'], + ['=', ['trusted', 'extensions', 'pp_auth_role'], 'pe_compiler_legacy'], + ] + + result + end + + def update_rules(group_id) + net = https_client + begin + current_rules = get_current_rules(group_id) + + # Transform rules recursively to handle nested structures + new_rules = modify_pe_master_rules(current_rules) + + # Update the group with the modified rules + url = "/classifier-api/v1/groups/#{group_id}" + req = Net::HTTP::Post.new(url) + req['Content-Type'] = 'application/json' + req.body = { rule: new_rules }.to_json + + res = net.request(req) + + case res.code + when '200', '201', '204' + puts "Successfully transformed pe_compiler rule to use regex match for *_compiler roles in group #{group_id}" + else + begin + error_body = JSON.parse(res.body.to_s) + raise "Failed to update rules: #{error_body['kind'] || error_body}" + rescue JSON::ParserError + raise "Invalid response from server (status #{res.code}): #{res.body}" + end + end + rescue StandardError => e + raise "Error during rules update: #{e.message}" + end + end + + def execute! + group_id = get_pe_master_group_id + update_rules(group_id) + end +end + +# Run the task unless an environment flag has been set +unless ENV['RSPEC_UNIT_TEST_MODE'] + Puppet.initialize_settings + task = UpdatePeMasterRules.new(JSON.parse(STDIN.read)) + task.execute! +end