Skip to content

Commit 5fab565

Browse files
committed
Fail peadm::util::retrieve_and_upload on PCP
The `peadm::util::retrieve_and_upload` plan downloads a PE installer tarball and then uploads it to a set of hosts that need it (Primary and Database nodes). When the PCP transport is in use, this upload is done via the `bolt_shim::upload` task, which essentially reads the file into memory, encodes it, and then ships it over Orchestrator as a task argument. This method of file transfer is utterly inadequate for shipping files as large as the PE installer which is over half of a gigabyte in size. Use of "bolt_shim::upload" will cause either a Linux kernel OOM, an Orchestrator OOM, or slam into the `max-message-size` allowed by the Orchestrator. This commit updates `peadm::util::retrieve_and_upload` to fail with a descriptive error message if any of the target nodes is using the PCP transport.
1 parent 31cfdea commit 5fab565

File tree

2 files changed

+35
-0
lines changed

2 files changed

+35
-0
lines changed

plans/util/retrieve_and_upload.pp

+22
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,28 @@
55
String[1] $local_path,
66
String[1] $upload_path,
77
) {
8+
$nodes.peadm::fail_on_transport('pcp', @(HEREDOC/n))
9+
\nThe "pcp" transport is not available for uploading PE installers as
10+
the ".tar.gz" file is too large to send over the PE Orchestrator
11+
as an argument to the "bolt_shim::upload" task.
12+
13+
To upgrade PE XL database nodes via PCP, use "download_mode = direct".
14+
If Puppet download servers are not reachable over the internet,
15+
upload the ".tar.gz" to an internal fileserver and use the
16+
"pe_installer_source" parameter to retrieve it.
17+
18+
For information on configuring plan parameters, see:
19+
20+
https://forge.puppet.com/modules/puppetlabs/peadm/plans
21+
22+
Or, use the "ssh" transport for database nodes so that the
23+
installer can be transferred via SCP.
24+
25+
For information on configuring transports, see:
26+
27+
https://www.puppet.com/docs/bolt/latest/bolt_transports_reference.html
28+
|-HEREDOC
29+
830
$exists = without_default_logging() || {
931
run_command("test -e '${local_path}'", 'local://localhost',
1032
_catch_errors => true,

spec/plans/util/retrieve_and_upload_spec.rb

+13
Original file line numberDiff line numberDiff line change
@@ -41,4 +41,17 @@
4141

4242
expect(run_plan('peadm::util::retrieve_and_upload', 'nodes' => 'primary', 'source' => '/tmp/source', 'upload_path' => '/tmp/upload', 'local_path' => '/tmp/download')).to be_ok
4343
end
44+
45+
it 'fails when nodes are configured to use the pcp transport' do
46+
result = run_plan('peadm::util::retrieve_and_upload',
47+
{'nodes' => ['pcp://node.example'],
48+
'source' => '/tmp/source',
49+
'upload_path' => '/tmp/upload',
50+
'local_path' => '/tmp/download'})
51+
52+
53+
expect(result).not_to be_ok
54+
expect(result.value.kind).to eq('unexpected-transport')
55+
expect(result.value.msg).to match(%r{The "pcp" transport is not available for uploading PE installers})
56+
end
4457
end

0 commit comments

Comments
 (0)