Skip to content

Commit 564cafe

Browse files
timidritimidri
committed
Re-add backup/restore plans
1 parent 0aac78b commit 564cafe

File tree

6 files changed

+527
-0
lines changed

6 files changed

+527
-0
lines changed

‎documentation/backup.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Backup Puppet Enterprise using the PEADM module
2+
3+
## What is being backed up?
4+
5+
By default the `peadm::backup` plan will backup the following items:
6+
7+
1. Orchestrator
8+
2. PuppetDB
9+
3. RBAC
10+
- Also, It copies the LDAP secret key if it present
11+
4. Activity
12+
5. Classification
13+
14+
Optionally you can also backup:
15+
16+
1. CA (CA and SSL certificates)
17+
18+
----
19+
20+
Most of the backups will be a direct copy of the databases with the exception of:
21+
22+
- Classification
23+
- The backup is done using an API call
24+
- CA
25+
- The certificate files will be copy via the puppet-backup script.
26+
27+
28+
**Note:**
29+
30+
It is important to highlight that the `peadm::backup` plan's output is different than the one you will get when you backup manually using [the `puppet-backup create` command.](https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html#back_up_pe_infrastructure).
31+
32+
The differences between these two backup tools are:
33+
34+
1. The structure of the backup file, since this plan `peadm:backup` uses a combination of scripts, API calls, and DB backups, you will not be able to restore it using the traditional `sudo puppet-backup restore <backup-filename>` command. [To read more about the difference between the options (flags), please read the official PE doc for backing up & restoring.](https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html#back_up_pe_infrastructure)
35+
36+
1. The core idea of the `peadm:backup` plan is a focus on being able to separate customers data from the infrastructure data. This contrasts with the PE backup script which is more focused on a traditional backup and restore after a system failure where will be restoring the same infrastructure. It, therefore, targets things like backing up PostgreSQL rather than the actual data you want.
37+
- This can be seen when restoring node groups with `peadm:backup` only restores the non-PEinfrastructure source groups and restores them to target.
38+
39+
- [PE backup](https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html#back_up_pe_infrastructure) is not capable of restoring to a DR setup and requires you to provision a new DR setup whereas [`peadm:restore`](restore.md) will reinitialise replicas as part of the process
40+
41+
1. One of the other key differences with the `peadm:backup` is it is compatible with the extra-large (XL) architectures as it can communicate with the remote PostgreSQL server using temporary escalated privileges.
42+
43+
## How can I customize my backup?
44+
45+
We need to pass the `backup` parameter to the `peadm::backup` plan.
46+
47+
**Example**
48+
49+
**Note:** The `peadm::backup` plan can only be executed from the PE primary server.
50+
51+
Let's Backup _only_ RBAC
52+
53+
```
54+
# backup_params.json
55+
56+
{
57+
"backup": {
58+
"orchestrator": false,
59+
"puppetdb": false,
60+
"rbac": true,
61+
"activity": false,
62+
"ca": false,
63+
"classifier": false
64+
},
65+
"output_directory": "/tmp"
66+
}
67+
```
68+
69+
We selected our backup options and the `output_directory` (default `/tmp`).
70+
71+
To run the backup plan with our custom parameters:
72+
73+
bolt plan run peadm::backup --params @backup_params.json
74+

‎documentation/restore.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Restore Puppet Enterprise using the PEADM module
2+
3+
Once you have a [backup](backup.md) you can restore a PE primary server. It is important to highlight that you can not use the `peadm::restore` plan with a backup that was not created with the `peadm::backup` plan.
4+
5+
**Things to consider**
6+
7+
There is some downtime involved when the `peadm:restore` plan is executed. The following services will be restarted:
8+
9+
1. pe-console-services
10+
2. pe-nginx
11+
3. pxp-agent
12+
4. pe-puppetserver
13+
5. pe-orchestration-services
14+
6. puppet
15+
7. pe-puppetdb
16+
17+
There is also a procedure related to the restoration of the databases where `peadm::restore` will temporarily set privileges (permissions) to a DB user with the only purpose of restoring the database. These temporary privileges are removed right after the PostgreSQL restore command finishes.
18+
19+
Also, this plan uses internal calls to the `peadm::get_targets` function, this means the plan expects the services to be up and running when you start either a restore or a backup.
20+
21+
## How to use the restore plan?
22+
23+
As in the `peadm::backup` plan, you can choose what you want to restore by specifying the parameter `restore`. The `input_file` parameter refers to the location of the backup tarball.
24+
25+
Example:
26+
27+
**Note:** The `peadm::restore` plan can only be executed from the PE primary server.
28+
29+
```
30+
# restore_params.json
31+
32+
{
33+
"restore": {
34+
"orchestrator": false,
35+
"puppetdb": false,
36+
"rbac": true,
37+
"activity": false,
38+
"ca": false,
39+
"classifier": false,
40+
},
41+
"input_file": "/tmp/path/backup_tarball.tgz"
42+
}
43+
```
44+
To run the `peadm::restore` plan with our custom parameters file, we can do:
45+
46+
bolt plan run peadm::restore --params @restore_params.json

‎plans/backup.pp

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# @api private
2+
# @summary Backup the core user settings for puppet infrastructure
3+
#
4+
# This plan can backup data as outlined at insert doc
5+
#
6+
plan peadm::backup (
7+
# This plan should be run on the primary server
8+
Peadm::SingleTargetSpec $targets,
9+
10+
# Which data to backup
11+
Peadm::Recovery_opts $backup = {},
12+
13+
# Where to put the backup folder
14+
String $output_directory = '/tmp',
15+
) {
16+
peadm::assert_supported_bolt_version()
17+
18+
$recovery_opts = (peadm::recovery_opts_default() + $backup)
19+
$cluster = run_task('peadm::get_peadm_config', $targets).first.value
20+
$arch = peadm::assert_supported_architecture(
21+
getvar('cluster.params.primary_host'),
22+
getvar('cluster.params.replica_host'),
23+
getvar('cluster.params.primary_postgresql_host'),
24+
getvar('cluster.params.replica_postgresql_host'),
25+
getvar('cluster.params.compiler_hosts'),
26+
)
27+
28+
$timestamp = Timestamp.new().strftime('%Y-%m-%dT%H%M%SZ')
29+
$backup_directory = "${output_directory}/pe-backup-${timestamp}"
30+
31+
$primary_target = getvar('cluster.params.primary_host')
32+
$puppetdb_postgresql_target = getvar('cluster.params.primary_postgresql_host') ? {
33+
undef => getvar('cluster.params.primary_host'),
34+
default => getvar('cluster.params.primary_postgresql_host'),
35+
}
36+
37+
$backup_databases = {
38+
'orchestrator' => $primary_target,
39+
'activity' => $primary_target,
40+
'rbac' => $primary_target,
41+
'puppetdb' => $puppetdb_postgresql_target,
42+
}.filter |$key,$_| {
43+
$recovery_opts[$key] == true
44+
}
45+
46+
# Create backup folders
47+
apply($primary_target) {
48+
file { $backup_directory :
49+
ensure => 'directory',
50+
owner => 'root',
51+
group => 'root',
52+
mode => '0700',
53+
}
54+
55+
# Create a subdir for each backup type selected
56+
$recovery_opts.filter |$_,$val| { $val == true }.each |$dir,$_| {
57+
file { "${backup_directory}/${dir}":
58+
ensure => 'directory',
59+
owner => 'root',
60+
group => 'root',
61+
mode => '0700',
62+
}
63+
}
64+
}
65+
66+
if getvar('recovery_opts.classifier') {
67+
out::message('# Backing up classification')
68+
run_task('peadm::backup_classification', $primary_target,
69+
directory => "${backup_directory}/classifier",
70+
)
71+
}
72+
73+
if getvar('recovery_opts.ca') {
74+
out::message('# Backing up ca and ssl certificates')
75+
# lint:ignore:strict_indent
76+
run_command(@("CMD"), $primary_target)
77+
/opt/puppetlabs/bin/puppet-backup create --dir=${shellquote($backup_directory)}/ca --scope=certs
78+
| CMD
79+
}
80+
81+
# Check if /etc/puppetlabs/console-services/conf.d/secrets/keys.json exists and if so back it up
82+
if getvar('recovery_opts.rbac') {
83+
out::message('# Backing up ldap secret key if it exists')
84+
# lint:ignore:140chars
85+
run_command(@("CMD"/L), $primary_target)
86+
test -f /etc/puppetlabs/console-services/conf.d/secrets/keys.json \
87+
&& cp -rp /etc/puppetlabs/console-services/conf.d/secrets ${shellquote($backup_directory)}/rbac/ \
88+
|| echo secret ldap key doesnt exist
89+
| CMD
90+
# lint:endignore
91+
}
92+
# lint:ignore:140chars
93+
# IF backing up orchestrator back up the secrets too /etc/puppetlabs/orchestration-services/conf.d/secrets/
94+
if getvar('recovery_opts.orchestrator') {
95+
out::message('# Backing up orchestrator secret keys')
96+
run_command(@("CMD"), $primary_target)
97+
cp -rp /etc/puppetlabs/orchestration-services/conf.d/secrets ${shellquote($backup_directory)}/orchestrator/
98+
| CMD
99+
}
100+
# lint:endignore
101+
$backup_databases.each |$name,$database_target| {
102+
run_command(@("CMD"/L), $primary_target)
103+
/opt/puppetlabs/server/bin/pg_dump -Fd -Z3 -j4 \
104+
-f ${shellquote($backup_directory)}/${shellquote($name)}/pe-${shellquote($name)}.dump.d \
105+
"sslmode=verify-ca \
106+
host=${shellquote($database_target.peadm::certname())} \
107+
user=pe-${shellquote($name)} \
108+
sslcert=/etc/puppetlabs/puppetdb/ssl/${shellquote($primary_target.peadm::certname())}.cert.pem \
109+
sslkey=/etc/puppetlabs/puppetdb/ssl/${shellquote($primary_target.peadm::certname())}.private_key.pem \
110+
sslrootcert=/etc/puppetlabs/puppet/ssl/certs/ca.pem \
111+
dbname=pe-${shellquote($name)}"
112+
| CMD
113+
}
114+
115+
run_command(@("CMD"/L), $primary_target)
116+
umask 0077 \
117+
&& cd ${shellquote(dirname($backup_directory))} \
118+
&& tar -czf ${shellquote($backup_directory)}.tar.gz ${shellquote(basename($backup_directory))} \
119+
&& rm -rf ${shellquote($backup_directory)}
120+
| CMD
121+
# lint:endignore
122+
return({ 'path' => "${backup_directory}.tar.gz" })
123+
}

0 commit comments

Comments
 (0)