Skip to content

Commit f5cef7a

Browse files
Solarch-564 adding plan to backup data (#226)
* (SOLARCH-564) first shot at returning node classification * (SOLARCH-564) mistake on default string * (SOLARCH-564) extra bracket * (SOLARCH-564) cat walked over keyboard and added S to line * (SOLARCH-564) missed out the unless which actually runs it * (SOLARCH-564) missing an end * (SOLARCH-564) adjusting message string to sub in variavble correctly * (SOLARCH-564) changing variable choice to directory and not file name for simplicity * (SOLARCH-564) correcting the string * (SOLARCH-564) correcting variable name * (SOLARCH-564) correcting variable * (SOLARCH-564) correcting string * (SOLARCH-564) correcting comment * (SOLARCH-564) changing to a patter to check it starts and ends with a / * (SOLARCH-564) first draft of plan * SOLARCH-564 test running simply classification and cert backup * SOLARCH-564 removing over complication of type * SOLARCH-564 correcting linting error and variable error * SOLARCH-564 testing database backup command * (SOLARCH-564) missing a target * (SOLARCH-564) testing a lamda of database selection and names * SOLARCH-564 corrected database to backup array * SOLARCH-564 correcting type * SOLARCH-564 changing to allow backup on postgres external db * SOLARCH-564 correcting primary host * (SOLARCH-564) changing directory to not end with a slash * (SOLARCH-564) changing path to absolutepath to check its a valid directory * (SOLARCH-564) wasn't thinking just keep it as a string * (SOLARCH-564) adding basic plan testing and correcting a lint error * (SOLARCH-716) updated default version to latest LTS .8 * (SOLARCH-716) updated peadm to be compatible with 2.x puppetlabs/service * Fix failing table output tests The gem was updated and the output format changed slightly. These tests should probably just be removed; they are too exact and finicky. * Update plans/backup.pp Change backup to output directory for better naming Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * (SOLARCH-564) first shot at returning node classification * (SOLARCH-564) mistake on default string * (SOLARCH-564) extra bracket * (SOLARCH-564) cat walked over keyboard and added S to line * (SOLARCH-564) missed out the unless which actually runs it * (SOLARCH-564) missing an end * (SOLARCH-564) adjusting message string to sub in variavble correctly * (SOLARCH-564) changing variable choice to directory and not file name for simplicity * (SOLARCH-564) correcting the string * (SOLARCH-564) correcting variable name * (SOLARCH-564) correcting variable * (SOLARCH-564) correcting string * (SOLARCH-564) correcting comment * (SOLARCH-564) changing to a patter to check it starts and ends with a / * (SOLARCH-564) first draft of plan * SOLARCH-564 test running simply classification and cert backup * SOLARCH-564 removing over complication of type * SOLARCH-564 correcting linting error and variable error * SOLARCH-564 testing database backup command * (SOLARCH-564) missing a target * (SOLARCH-564) testing a lamda of database selection and names * SOLARCH-564 corrected database to backup array * SOLARCH-564 correcting type * SOLARCH-564 changing to allow backup on postgres external db * SOLARCH-564 correcting primary host * (SOLARCH-564) changing directory to not end with a slash * (SOLARCH-564) changing path to absolutepath to check its a valid directory * (SOLARCH-564) wasn't thinking just keep it as a string * (SOLARCH-564) adding basic plan testing and correcting a lint error * Update plans/backup.pp Change backup to output directory for better naming Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * Update plans/backup.pp Changing variable name Co-authored-by: Reid Vandewiele <[email protected]> * (SOLARCH-564) adding in creation of backup directory via apply * (SOLARCH-564) updating for all backups to go to backup directory * (SOLARCH-564) add time to allow multiple backups on a day * (SOLARCH-564) making dates consistent * (SOLARCH-564) updated with secret keys for ldap and orchestrator and fixed remote puppetdb backup * (SOLARCH-564) updating to output if test failed * (SOLARCH-564) removing needless comma * (SOLARCH-564) dropped part of key names in error * (SOLARCH-564) inserting deliberate error to test exit code * (SOLARCH-564) changing approach so it exits properly * (SOLARCH-564) test succesful for failure correcting to correct certificate * (SOLARCH-564) fixing plan spec with limitiation of timestamps * (SOLARCH-564) backup classification was in error using server status API * (solarch-564) removing parsing Co-authored-by: Reid Vandewiele <[email protected]>
1 parent 130948d commit f5cef7a

File tree

4 files changed

+165
-0
lines changed

4 files changed

+165
-0
lines changed

Diff for: plans/backup.pp

+87
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# @summary Backup the core user settings for puppet infrastructure
2+
#
3+
# This plan can backup data as outlined at insert doc
4+
#
5+
plan peadm::backup (
6+
# Standard
7+
Peadm::SingleTargetSpec $primary_host,
8+
Optional[Peadm::SingleTargetSpec] $replica_host = undef,
9+
10+
# Large
11+
Optional[TargetSpec] $compiler_hosts = undef,
12+
13+
# Extra Large
14+
Optional[Peadm::SingleTargetSpec] $primary_postgresql_host = undef,
15+
Optional[Peadm::SingleTargetSpec] $replica_postgresql_host = undef,
16+
17+
# Which data to backup
18+
Boolean $backup_orchestrator = true,
19+
Boolean $backup_rbac = true,
20+
Boolean $backup_activity = true,
21+
Boolean $backup_ca_ssl = true,
22+
Boolean $backup_puppetdb = false,
23+
Boolean $backup_classification = true,
24+
String $output_directory = '/tmp',
25+
){
26+
27+
$timestamp = Timestamp.new().strftime('%F_%T')
28+
$backup_directory = "${output_directory}/pe-backup-${timestamp}"
29+
# Create backup folder
30+
apply_prep($primary_host)
31+
apply($primary_host){
32+
file { $backup_directory :
33+
ensure => 'directory',
34+
owner => 'root',
35+
group => 'pe-postgres',
36+
mode => '0770'
37+
}
38+
}
39+
# Create an array of the names of databases and whether they have to be backed up to use in a lambda later
40+
$database_to_backup = [ $backup_orchestrator, $backup_activity, $backup_rbac, $backup_puppetdb]
41+
$database_names = [ 'pe-orchestrator' , 'pe-activity' , 'pe-rbac' , 'pe-puppetdb' ]
42+
43+
peadm::assert_supported_bolt_version()
44+
45+
# Ensure input valid for a supported architecture
46+
$arch = peadm::assert_supported_architecture(
47+
$primary_host,
48+
$replica_host,
49+
$primary_postgresql_host,
50+
$replica_postgresql_host,
51+
$compiler_hosts,
52+
)
53+
54+
if $backup_classification {
55+
out::message('# Backing up classification')
56+
run_task('peadm::backup_classification', $primary_host,
57+
directory => $backup_directory,
58+
)
59+
}
60+
61+
if $backup_ca_ssl {
62+
out::message('# Backing up ca and ssl certificates')
63+
run_command("/opt/puppetlabs/bin/puppet-backup create --dir=${backup_directory} --scope=certs", $primary_host)
64+
}
65+
66+
# Check if /etc/puppetlabs/console-services/conf.d/secrets/keys.json exists and if so back it up
67+
out::message('# Backing up ldap secret key if it exists')
68+
run_command("test -f /etc/puppetlabs/console-services/conf.d/secrets/keys.json && cp -rp /etc/puppetlabs/console-services/conf.d/secrets/keys.json ${backup_directory} || echo secret ldap key doesnt exist" , $primary_host) # lint:ignore:140chars
69+
70+
# IF backing up orchestrator back up the secrets too /etc/puppetlabs/orchestration-services/conf.d/secrets/
71+
if $backup_orchestrator {
72+
out::message('# Backing up orchestrator secret keys')
73+
run_command("cp -rp /etc/puppetlabs/orchestration-services/conf.d/secrets ${backup_directory}/", $primary_host)
74+
}
75+
76+
$database_to_backup.each |Integer $index, Boolean $value | {
77+
if $value {
78+
out::message("# Backing up database ${database_names[$index]}")
79+
# If the primary postgresql host is set then pe-puppetdb needs to be remotely backed up to primary.
80+
if $database_names[$index] == 'pe-puppetdb' and $primary_postgresql_host {
81+
run_command("sudo -u pe-puppetdb /opt/puppetlabs/server/bin/pg_dump \"sslmode=verify-ca host=${primary_postgresql_host} sslcert=/etc/puppetlabs/puppetdb/ssl/${primary_host}.cert.pem sslkey=/etc/puppetlabs/puppetdb/ssl/${primary_host}.private_key.pem sslrootcert=/etc/puppetlabs/puppet/ssl/certs/ca.pem dbname=pe-puppetdb\" -f /tmp/puppetdb_$(date +%F_%T).bin" , $primary_host) # lint:ignore:140chars
82+
} else {
83+
run_command("sudo -u pe-postgres /opt/puppetlabs/server/bin/pg_dump -Fc \"${database_names[$index]}\" -f \"${backup_directory}/${database_names[$index]}_$(date +%F_%T).bin\"" , $primary_host) # lint:ignore:140chars
84+
}
85+
}
86+
}
87+
}

Diff for: spec/plans/backup_spec.rb

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
require 'spec_helper'
2+
3+
describe 'peadm::backup' do
4+
include BoltSpec::Plans
5+
let(:params) { { 'primary_host' => 'primary' } }
6+
7+
it 'runs with default params' do
8+
allow_apply_prep
9+
allow_apply
10+
expect_out_message.with_params('# Backing up ca and ssl certificates')
11+
# The commands all have a timestamp in them and frankly its prooved to hard with bolt spec to work this out
12+
allow_any_command
13+
expect_out_message.with_params('# Backing up database pe-orchestrator')
14+
expect_out_message.with_params('# Backing up database pe-activity')
15+
expect_out_message.with_params('# Backing up database pe-rbac')
16+
expect_out_message.with_params('# Backing up classification')
17+
expect_task('peadm::backup_classification')
18+
expect(run_plan('peadm::backup', params)).to be_ok
19+
end
20+
end

Diff for: tasks/backup_classification.json

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"puppet_task_version": 1,
3+
"supports_noop": false,
4+
"description": "A task to call the classification api and write to file",
5+
"parameters": {
6+
"directory": {
7+
"type": "String",
8+
"description": "The directory to write the classification output to. Directory must exist",
9+
"default": "/tmp"
10+
}
11+
},
12+
"input_method": "stdin"
13+
}

Diff for: tasks/backup_classification.rb

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
#!/opt/puppetlabs/puppet/bin/ruby
2+
3+
# Puppet Task Name: backup_classification
4+
require 'net/https'
5+
require 'uri'
6+
require 'json'
7+
require 'puppet'
8+
9+
# BackupClassiciation task class
10+
class BackupClassification
11+
def initialize(params)
12+
@params = params
13+
end
14+
15+
def execute!
16+
File.write("#{@params['directory']}/classification_backup.json", return_classification)
17+
puts "Classification written to #{@params['directory']}/classification_backup.json"
18+
end
19+
20+
private
21+
22+
def https_client
23+
client = Net::HTTP.new('localhost', '4433')
24+
client.use_ssl = true
25+
client.cert = @cert ||= OpenSSL::X509::Certificate.new(File.read(Puppet.settings[:hostcert]))
26+
client.key = @key ||= OpenSSL::PKey::RSA.new(File.read(Puppet.settings[:hostprivkey]))
27+
client.verify_mode = OpenSSL::SSL::VERIFY_NONE
28+
client
29+
end
30+
31+
def return_classification
32+
classification = https_client
33+
classification_request = Net::HTTP::Get.new('/classifier-api/v1/groups')
34+
35+
classification.request(classification_request).body
36+
end
37+
end
38+
# Run the task unless an environment flag has been set, signaling not to. The
39+
# environment flag is used to disable auto-execution and enable Ruby unit
40+
# testing of this task.
41+
unless ENV['RSPEC_UNIT_TEST_MODE']
42+
Puppet.initialize_settings
43+
task = BackupClassification.new(JSON.parse(STDIN.read))
44+
task.execute!
45+
end

0 commit comments

Comments
 (0)