(MODULES-10023) Fix multiple xtrabackup regressions #1245

fraenki · 2019-10-16T12:29:40Z

https://tickets.puppetlabs.com/browse/MODULES-10023

1. Regression: xtrabackup nonfunctional due to backup dir change

Currently the xtrabackup provider is nonfunctional when $incremental_backups is set to false. Performing the backup will produce an error starting with the 2nd run (the very first backup is successful):

# /usr/local/sbin/xtrabackup.sh --target-dir=/data/mysql-backup --backup
[...]
xtrabackup: Can't create/write to file '/data/mysql-backup/xtrabackup_logfile' (Errcode: 17 - File exists)
xtrabackup: error: failed to open the target stream for 'xtrabackup_logfile'.

This regression was introduced in #1188 with commit 155f8a1. The value of --target-dir was changed from

${backupdir}/`date +%F_%H-%M-%S`

to just ${backupdir}.

As a result, every backup is written to the exact same target directory, which is not supported by xtrabackup and results in the above error message. So after performing the first backup (which succeeds because the target directory is still empty), all following backup jobs will fail.

Furthermore, this change renders the $backuprotate parameter nonfunctional, because only one backup can be kept this way.

I think this was an oversight and should be seen as a regression, because it effectively breaks xtrabackup for full backups. That's why I propose to revert --target-dir back to the old (working) value.

2. Regression: backup rotation nonfunctional due to incompatible find

While working on the first regression, I've noticed a second regression that is directly related. #1176 added backup rotation for xtrabackup. However, the find that was added in commit 9834d9a is incompatible:

find "${DIR}/" -maxdepth 1 -type f -name "${PREFIX}*.sql*" -mtime +${ROTATE} -print0 | xargs -0 -r rm -f

Issues:

First of all, the file name pattern is invalid, because xtrabackup does not use the .sql file extension, but instead multiple other file types.
Besides that the ${PREFIX} is nowhere to be found.
Finally, -maxdepth 1 is incompatible, because a backup may contain several (sub-) directories – and every full backup should be stored in it's own directory (see previous regression).

It looks like this find was copied from another backup provider, so I propose to change it in a way that deals with xtrabackup's uniqueness. The new approach assumes that every backup (incremental or full) resides in it's own sub-directory, so it's pretty easy to remove an obsolete backup by simply removing the whole directory.

Small quality of life improvements

While working on fixing the regressions I've implemented small changes that improve the xtrabackup experience:

For incremental backups:
- Add a warning if backup rotation is set to less than 7 days, because this would make incremental backups impossible. (A full backup must exist when trying to create an incremental backup, but a full backup is only performed by the weekly cronjob every sunday.)
- Change the path for the weekly full backup. This is required in order for the backup rotation to work as expected and to make it possible to keep more than one full backup (that is more than 7 days of backups).
Deal with the GNU/BSD date difference.
Add more comments explaining what this code (or my change) does and why it's important.
Adjust unit tests accordingly.

fraenki · 2019-10-17T09:28:31Z

I've successfully verified that both full and incremental backups are working perfectly fine after applying the proposed fixes.

Verified: Full backups

The daily cron job works as expected...

/usr/local/sbin/xtrabackup.sh --target-dir=/data/mysql-backup/$(date +%F_%H-%M-%S) --backup
[...]
191017 11:13:57 completed OK!

...and produces the following directory structure (depends on the time when the cron job ran):

# ls -l /data/mysql-backup/
total 8
drwxr-x--- 9 root root 4096 Oct 17 10:59 2019-10-17_10-59-12/
drwxr-x--- 9 root root 4096 Oct 17 11:13 2019-10-17_11-13-52/

Note: I ran the cron job twice manually for this test case.

Verified: Incremental backups

The weekly cronjob creates the full backup just fine...

/usr/local/sbin/xtrabackup.sh --target-dir=/data/mysql-backup/$(date +%F)_full --backup
[...]
191017 11:10:54 completed OK!

...and the daily cron job is able to perform incremental backups based on the weekly full backup;

/usr/local/sbin/xtrabackup.sh --incremental-basedir=/data/mysql-backup/$(date -d "last sunday" +%F)_full --target-dir=/data/mysql-backup/$(date +%F_%H-\%M-%S) --backup
[...]
incremental backup from 86330169 is enabled.
[...]
xtrabackup: using the full scan for incremental backup
[...]
xtrabackup: The latest check point (for incremental): '86330169'
[...]
xtrabackup: Transaction log of lsn (86330169) to (86330178) was copied.
191017 11:11:50 completed OK!

...and it produces the following directory structure (depends on the time when the cron job ran):

# ls -l /data/mysql-backup/
total 12
drwxr-x--- 9 root root 4096 Oct 17 11:10 2019-10-13_full/
drwxr-x--- 9 root root 4096 Oct 17 11:11 2019-10-17_11-11-46/
drwxr-x--- 9 root root 4096 Oct 17 11:13 2019-10-17_11-13-16/

Note: I had to rename the manually created first full backup to match last sunday's date to make this manual test case work. But I'm confident that the cron job works perfectly fine now, I just did not want to wait until sunday :)

sanfrancrisko · 2019-10-22T15:52:02Z

@fraenki Thanks for the PR, the comprehensive descriptions and test coverage you've added. Looking through the change with my colleagues and we'll get back to you a.s.a.p.

sanfrancrisko · 2019-10-28T15:16:31Z

@fraenki After some discussion internally, we were thinking it might be good to add an acceptance test for the xtrabackup functionality to ensure we don't run in to this scenario again. Do you feel that's something you can add? We can help / support if needs be.

fraenki · 2019-11-03T23:41:47Z

@cmccrisken-puppet Sure, I'll have a look. Are there any existing acceptance tests from Puppetlabs that I should use as a reference?

lionce · 2019-11-08T10:04:42Z

Hello @fraenki ,

You can check module's tests for reference. For running acceptance tests we're using puppet_litmus. To run the acceptance tests follow the instructions from this point here. Please let us know if you need help/support :)

Cheers!

fraenki · 2019-11-12T21:26:44Z

You can check module's tests for reference.

@lionce Before I dive into this: are the module's tests expected to run without failure? Because they ALL fail for me:

$ bundle exec rake litmus:acceptance:parallel
┌ [✖] Running against 1 targets.
└── [✖] localhost:2222, centos:7
================
localhost:2222, centos:7
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

...

Finished in 1 minute 34.35 seconds (files took 2.05 seconds to load)
47 examples, 47 failures

lionce · 2019-11-13T13:11:53Z

Hey @fraenki ,

The tests should be successfully ran.
Successful on 1 nodes: ["eloquent_gauss, waffleimage/centos7"]
Can you please provide me the errors? Also your set-up steps please? maybe the platform is not completely configured.

e.g. you can check all the steps and the results here on travis report

26.64s$ bundle exec rake 'litmus:provision_list[travis_el7]'
26.15s$ bundle exec rake 'litmus:install_agent[puppet6]'
13.59s$ bundle exec rake litmus:install_module
873.26s$ bundle exec rake litmus:acceptance:parallel

travis_el7:
  provisioner: docker_exp
  images: ['waffleimage/centos7', 'waffleimage/oraclelinux7', 'waffleimage/scientificlinux7']

If you want to provision just one platform, you should use:
bundle exec rake 'litmus:provision[docker_exp, waffleimage/centos7]'

Cheers!

fraenki · 2020-01-06T21:36:06Z

@cmccrisken-puppet @lionce

I've added the following to this PR:

acceptance tests for xtrabackup
merged (MODULES-10023) expose $incremental_backups parameter #1244 into this PR (in order to be able to add complete acceptance tests)
fixed default of $xtrabackup_package_name on CentOS 6 (which was revealed by new acceptance tests)
added xtrabackup information to the README

One step closer, only two CI failures that I can't explain:

PLATFORMS=deb_puppet5 reports an issue that does not seem to be related to my changes.
PLATFORMS=el6_puppet5 fails to fetch the Percona repository metadata.

Any advice on these two issues? :)

sheenaajay · 2020-01-17T13:49:38Z

@fraenki Reopening the PR to rerun the Travis jobs. Thank you.

carabasdaniel · 2020-02-06T13:16:07Z

Hi @fraenki,

Both these failures seem to be related to the installation of the percona package and its dependencies:

For Debian: The following packages have unmet dependencies: percona-release : Depends: gnupg2 but it is not going to be installed E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
For the PLATFORMS=el6_puppet5 it seems to be problems with the metadata to install the percona package for some of the OSes tested

I recommend you try to run the installation of the percona package manually from inside the containers to debug these issues a bit easier. Using Litmus you can provision the images on your machine using:
bundle exec rake litmus:provision['docker_exp','waffleimage/debian8']
In order to run the tests install the puppet agent and module using bundle exec rake litmus:install_agent[puppet5] and bundle exec rake litmus:install_module

In your inventory.yaml (will be created after the provisioning) file you will find the necessary details to ssh into the container and run manually from there.

…fic 6

fraenki · 2020-02-16T22:01:17Z

@carabasdaniel Thanks for the hints. All tests are working now. Ready for merge :)

(Fixes in 3e0e11a: On Debian and older versions of Ubuntu the xtrabackup package name was wrong. Debian also requires the gnupg2 package. On Scientific Linux 6 the value of $releasever in YUM differs from RedHat/CentOS, making the Percona repo incompatible (an upstream issue, nothing that should be handled in this module); as a workaround $releasever is set to the same value that works on RedHat/CentOS.)

carabasdaniel · 2020-02-19T09:38:26Z

Hi @fraenki

Thanks again for submitting this PR. Looks really great now.

In order to get this PR merged we have to ensure that this feature works on all the OSs this module supports.

Running the full suite of acceptance tests seems to point out there are still failures on installing percona-xtrabackup on a few machines:

├── [✖] stepwise-octet.delivery.puppetlabs.net, redhat-5-x86_64
├── [✔] hectic-impulse.delivery.puppetlabs.net, redhat-6-x86_64
├── [✔] sylvan-inflow.delivery.puppetlabs.net, redhat-7-x86_64
├── [✖] barefooted-hex.delivery.puppetlabs.net, redhat-8-x86_64
├── [✖] amazing-stench.delivery.puppetlabs.net, centos-5-x86_64
├── [✔] galactic-misery.delivery.puppetlabs.net, centos-6-x86_64
├── [✔] amber-longhand.delivery.puppetlabs.net, centos-7-x86_64
├── [✖] token-trapper.delivery.puppetlabs.net, oracle-5-x86_64
├── [✖] master-cement.delivery.puppetlabs.net, oracle-6-x86_64
├── [✔] holy-initiation.delivery.puppetlabs.net, oracle-7-x86_64
├── [✔] dread-pulsation.delivery.puppetlabs.net, scientific-6-x86_64
├── [✔] pleasant-ileum.delivery.puppetlabs.net, scientific-7-x86_64
├── [✔] uncanny-drywall.delivery.puppetlabs.net, debian-8-x86_64
├── [✔] central-analyst.delivery.puppetlabs.net, debian-9-x86_64
├── [✔] snappy-hairpin.delivery.puppetlabs.net, debian-10-x86_64
├── [✖] fool-pedal.delivery.puppetlabs.net, sles-11-x86_64
├── [✔] granular-sponge.delivery.puppetlabs.net, ubuntu-1404-x86_64
├── [✔] sappy-homestead.delivery.puppetlabs.net, ubuntu-1604-x86_64
└── [✔] nocturnal-logic.delivery.puppetlabs.net, ubuntu-1804-x86_64```

On RedHat-8: ``` Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install percona-xtrabackup' returned 1: Error: Unable to find a match```
On SLES: ``` Error: Execution of '/usr/bin/zypper --quiet install --auto-agree-with-licenses --no-confirm percona-xtrabackup' returned 104```
On CentOS-5 and RedHat-5: ```Error: Execution of '/bin/rpm -i http://repo.percona.com/yum/percona-release-latest.noarch.rpm' returned 1: warning: /var/tmp/rpm-xfer.hzr0u8: Header V4 DSA signature: NOKEY, key ID cd2efd2a
       error: Failed dependencies:
       	rpmlib(FileDigests) <= 4.6.0-1 is needed by percona-release-1.0-14.noarch
       	rpmlib(PayloadIsXz) <= 5.2-1 is needed by percona-release-1.0-14.noarch```
 
If you need help in getting this tests working, please let us know.

fraenki · 2020-02-27T20:46:33Z

@carabasdaniel I've addressed these issues in 7d43a58. How can I run this "full suite of acceptance tests"?

Besides that after the latest commit the RHEL6/7 tests fail on travis, although they work perfectly fine locally. No idea how this travis "timeout" error should be useful to anyone. :-/ Maybe you have access to logs or debug output.

┌ [✔] Running against 3 targets.
├── [✔] a7fd595f70f6, waffleimage/centos7
├── [✔] 4a786dcbdda7, waffleimage/oraclelinux7
└── [✔] cc31598fc975, waffleimage/scientificlinux7
================
4a786dcbdda7, waffleimage/oraclelinux7
.....................................................................................................

Finished in 6 minutes 2 seconds (files took 38.73 seconds to load)
101 examples, 0 failures

Libhoney::Client: no writekey configured, disabling sending events
pid 11620 exit 0
================
a7fd595f70f6, waffleimage/centos7
.....................................................................................................

Finished in 6 minutes 6 seconds (files took 1 minute 11.37 seconds to load)
101 examples, 0 failures

Libhoney::Client: no writekey configured, disabling sending events
pid 11616 exit 0
================
cc31598fc975, waffleimage/scientificlinux7
.....................................................................................................

Finished in 7 minutes 47 seconds (files took 1 minute 38.01 seconds to load)
101 examples, 0 failures

┌ [✔] Running against 2 targets.
├── [✔] 111f52cba808, waffleimage/centos6
└── [✔] 1a1c889a9c3a, waffleimage/scientificlinux6
================
111f52cba808, waffleimage/centos6
............................................................................................

Finished in 5 minutes 52 seconds (files took 33.14 seconds to load)
92 examples, 0 failures

Libhoney::Client: no writekey configured, disabling sending events
pid 19358 exit 0
================
1a1c889a9c3a, waffleimage/scientificlinux6
............................................................................................

Finished in 6 minutes 51 seconds (files took 57.87 seconds to load)
92 examples, 0 failures

sheenaajay · 2020-02-28T13:32:26Z

@fraenki Thank you for incorporating the comments. We use vmpoolers to run the full acceptance tests. As of now we don't have an option to run full acceptance tests outside the puppet . It is failing with following error
Error: Execution of '/bin/rpm -i http://repo.percona.com/yum/release/5/os/noarch/percona-release-0.1-5.noarch.rpm' returned 1: warning: /var/tmp/rpm-xfer.gdrIa5: Header V4 DSA signature: NOKEY, key ID cd2efd2a error: Failed dependencies: rpmlib(FileDigests) <= 4.6.0-1 is needed by percona-release-0.1-5.noarch rpmlib(PayloadIsXz) <= 5.2-1 is needed by percona-release-0.1-5.noarch Error: /Stage[main]/Main/Package[percona-release]/ensure: change from 'absent' to 'present' failed: Execution of '/bin/rpm -i http://repo.percona.com/yum/release/5/os/noarch/percona-release-0.1-5.noarch.rpm' returned 1: warning: /var/tmp/rpm-xfer.gdrIa5: Header V4 DSA signature: NOKEY, key ID cd2efd2a error: Failed dependencies: rpmlib(FileDigests) <= 4.6.0-1 is needed by percona-release-0.1-5.noarch rpmlib(PayloadIsXz) <= 5.2-1 is needed by percona-release-0.1-5.noarch Error: Execution of '/bin/rpm -i https://download.fedoraproject.org/pub/epel/epel-release-latest-5.noarch.rpm' returned 1: error: skipping https://download.fedoraproject.org/pub/epel/epel-release-latest-5.noarch.rpm - transfer failed - Unknown or unexpected error Error: /Stage[main]/Main/Package[epel-release]/ensure: change from 'absent' to 'present' failed: Execution of '/bin/rpm -i https://download.fedoraproject.org/pub/epel/epel-release-latest-5.noarch.rpm' returned 1: error: skipping https://download.fedoraproject.org/pub/epel/epel-release-latest-5.noarch.rpm - transfer failed - Unknown or unexpected error Error: Could not find package percona-xtrabackup-20 Error: /Stage[main]/Mysql::Backup::Xtrabackup/Package[percona-xtrabackup-20]/ensure: change from 'purged' to 'present' failed: Could not find package percona-xtrabackup-20

On the following platforms
Failed on 6 nodes: ["logical-handbag.delivery.puppetlabs.net, sles-11-x86_64", "boxy-attendance.delivery.puppetlabs.net, oracle-6-x86_64", "bacterial-spurt.delivery.puppetlabs.net, redhat-6-x86_64", "lacy-horseflesh.delivery.puppetlabs.net, redhat-5-x86_64", "sly-letterhead.delivery.puppetlabs.net, oracle-5-x86_64", "fool-whirling.delivery.puppetlabs.net, centos-5-x86_64"]

Please let us know if you need more information. Thank you

fraenki · 2020-02-29T21:41:38Z

@david22swan Thanks for fixing the travis tests! 👍

@sheenaajay I've addressed the CentOS/RHEL 5 errors in 3efa123. However, RHEL5 is EOL so the available docker images are pretty useless and couldn't run all tests. I couldn't find docker images for SLES11 so I can't test it at all. Regarding the EL6 failures: Why don't these failures show up in travis, or: why does Puppetlabs use different images for the final tests? Since I can't trigger these failures, I have to ask you for the full error messages for all EL6-like operating systems.

sheenaajay · 2020-03-04T13:05:14Z

@fraenki

Thank you for incorporating the comments. For PR level we perform testing on the following docker images which are available (debian8 debian9 debian10 ubuntu14.04 ubuntu16.04 ubuntu18.04 centos6 scientificlinux6 centos7 oraclelinux7 scientificlinux7)

But during bug fix or release testing, we cover most of the remaining supported platforms during the testing. Will send you the error information for the failing platform.

sheenaajay · 2020-03-10T06:43:07Z

Hi @fraenki ,Please find the failures below. Let us know if you need more information.

sles-11-x86_64

`Failures:

with xtrabackup enabled should work with no errors when configuring mysql backup
On host advisable-pad.delivery.puppetlabs.net' Failure/Error: idempotent_apply(pp) RuntimeError: apply manifest expected no changes puppet apply /tmp/manifest_20200309_18204_161fzrt.pp --detailed-exitcodes`
====== Start output of Puppet apply with unexpected changes ======
Error: Execution of '/usr/bin/zypper --quiet install --auto-agree-with-licenses --no-confirm xtrabackup' returned 104: Problem retrieving the repository index file for service 'spacewalk':
[|] This client is not registered to any spacewalk server.

No provider of 'xtrabackup' found.
Error: /Stage[main]/Mysql::Backup::Xtrabackup/Package[xtrabackup]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/zypper --quiet install --auto-agree-with-licenses --no-confirm xtrabackup' returned 104: Problem retrieving the repository index file for service 'spacewalk':
[|] This client is not registered to any spacewalk server.

No provider of 'xtrabackup' found.
Warning: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-weekly]: Skipping because of failed dependencies
Warning: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-daily]: Skipping because of failed dependencies
Notice: Compiled catalog for advisable-pad.delivery.puppetlabs.net in environment production in 0.43 seconds
Notice: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-weekly]: Dependency Package[xtrabackup] has failures: true
Notice: Applied catalog in 1.02 seconds

====== End output of Puppet apply with unexpected changes ======

./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:317:in report_puppet_apply_change' ./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:77:in apply_manifest'
./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:13:in idempotent_apply' ./spec/acceptance/mysql_backup_spec.rb:207:in block (3 levels) in <top (required)>'`

oracle-5-x86_64
redhat-7-x86_64
centos-5-x86_64
oracle-6-x86_64

`Failures:

with xtrabackup enabled should work with no errors when configuring mysql backup
On host citrated-dive.delivery.puppetlabs.net' Failure/Error: idempotent_apply(pp) RuntimeError: apply manifest expected no changes puppet apply /tmp/manifest_20200309_18202_2bhhwo.pp --detailed-exitcodes`
====== Start output of Puppet apply with unexpected changes ======
Error: Execution of '/bin/rpm -i https://archives.fedoraproject.org/pub/archive/epel/epel-release-latest-5.noarch.rpm' returned 1: error: skipping https://archives.fedoraproject.org/pub/archive/epel/epel-release-latest-5.noarch.rpm - transfer failed - Unknown or unexpected error
Error: /Stage[main]/Main/Package[epel-release]/ensure: change from 'absent' to 'present' failed: Execution of '/bin/rpm -i https://archives.fedoraproject.org/pub/archive/epel/epel-release-latest-5.noarch.rpm' returned 1: error: skipping https://archives.fedoraproject.org/pub/archive/epel/epel-release-latest-5.noarch.rpm - transfer failed - Unknown or unexpected error
Error: Execution of '/usr/bin/yum -e 0 -y install percona-xtrabackup-20' returned 1: Error: failure: repodata/0382344d75a4d35e9bdbb2d4e0bda5fe7a755e07f8c8b1346a88ba3e5a8e64f5-primary.sqlite.bz2 from percona-release-noarch: [Errno 256] No more mirrors to try.
Error: /Stage[main]/Mysql::Backup::Xtrabackup/Package[percona-xtrabackup-20]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/yum -e 0 -y install percona-xtrabackup-20' returned 1: Error: failure: repodata/0382344d75a4d35e9bdbb2d4e0bda5fe7a755e07f8c8b1346a88ba3e5a8e64f5-primary.sqlite.bz2 from percona-release-noarch: [Errno 256] No more mirrors to try.
Warning: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-weekly]: Skipping because of failed dependencies
Warning: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-daily]: Skipping because of failed dependencies
Notice: Compiled catalog for citrated-dive.delivery.puppetlabs.net in environment production in 0.42 seconds
Notice: /Stage[main]/Mysql::Backup::Xtrabackup/Cron[xtrabackup-weekly]: Dependency Package[percona-xtrabackup-20] has failures: true
Notice: Applied catalog in 1.42 seconds

====== End output of Puppet apply with unexpected changes ======

./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:317:in report_puppet_apply_change' ./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:77:in apply_manifest'
./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:13:in idempotent_apply' ./spec/acceptance/mysql_backup_spec.rb:207:in block (3 levels) in <top (required)>'`

redhat-8-x86_64

`Failures:

with xtrabackup enabled should work with no errors when configuring mysql backup
On host hurt-abjection.delivery.puppetlabs.net' Failure/Error: idempotent_apply(pp) RuntimeError: apply manifest expected no changes puppet apply /tmp/manifest_20200309_18189_u4ene2.pp --detailed-exitcodes`
====== Start output of Puppet apply with unexpected changes ======
Notice: Compiled catalog for hurt-abjection.delivery.puppetlabs.net in environment production in 0.33 seconds
Notice: /Stage[main]/Main/Package[epel-release]/ensure: created
Notice: Applied catalog in 1.46 seconds

====== End output of Puppet apply with unexpected changes ======

./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:317:in report_puppet_apply_change' ./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:77:in apply_manifest'
./.bundle/gems/ruby/2.5.0/gems/puppet_litmus-0.15.0/lib/puppet_litmus/puppet_helpers.rb:13:in idempotent_apply' ./spec/acceptance/mysql_backup_spec.rb:207:in block (3 levels) in <top (required)>'
`

fraenki · 2020-03-29T22:29:41Z

@sheenaajay Commit 1f8eefc disables the new xtrabackup tests on all EOL operating systems, so it should finally pass your internal tests.

NOTE: Other tests started failing, because the tests are now run on Puppet 6.14 – this Puppet Agent bug is to blame. No failures when running these tests on Puppet 6.13 or Puppet 6 nightly (aka 6.15 prerelease).

@sheenaajay Please, if there's yet another minor issue that could easily be fixed, may I ask you to just merge this PR and commit the fix afterwards? It's been 5 months already, and this is starting to get really frustrating.

sheenaajay · 2020-03-31T09:48:13Z

@fraenki Sorry for the delay. Will be talking to the team on priority to review this PR. We can fi the minor problems too. Thanks alot for your contribution. Much appreciated your help.

sheenaajay · 2020-04-03T08:11:41Z

@fraenki The PR is top of the list to get it merged. There is small change to exclude it from running on Oracle 6 box the (os[:family] is Redhat instead of redhat). We will do the changes now.
All the tests are running clean on puppet agent 5. As you have mentioned there is a defect on puppet agent 6.14 version and the agent team will be doing a release in April.
If we accept this PR the tests will be failing on PR level and on release checks.
We will merge this PR as soon as the release from the agent team is available. Hope that's ok. Thanks again for a great PR.

sheenaajay · 2020-05-12T12:42:21Z

fix xtrabackup target directory and related regressions

4f0ea63

fraenki requested a review from a team as a code owner October 16, 2019 12:29

fix handling of incremental backups with xtrabackup

99ecc8c

fraenki changed the title ~~[WIP] Fix multiple xtrabackup regressions~~ Fix multiple xtrabackup regressions (+JIRA ticket) Oct 16, 2019

sanfrancrisko added the bugfix label Oct 22, 2019

fraenki added 4 commits November 23, 2019 23:32

MODULES-10023 add parameter $incremental_backups to parent class

815d79d

MODULES-10023 add acceptance tests for xtrabackup

7730e27

MODULES-10023 add xtrabackup documentation

be838eb

MODULES-10023 fix acceptance tests

0765d15

fraenki mentioned this pull request Jan 6, 2020

(MODULES-10023) expose $incremental_backups parameter #1244

Closed

MODULES-10023 fix xtrabackup tests on CentOS 6

dc43dd9

sheenaajay closed this Jan 17, 2020

sheenaajay reopened this Jan 17, 2020

MODULES-10023 fix xtrabackup tests on Debian 8, Ubuntu 14.04, Scienti…

3e0e11a

…fic 6

fraenki requested review from sanfrancrisko and carabasdaniel and removed request for a team February 18, 2020 17:07

fraenki and others added 2 commits February 27, 2020 21:52

MODULES-10023 fix xtrabackup tests on SLES, RHEL 8 and RHEL/CentOS 5

5abb54d

Merge branch 'master' into xtrabackup_rotate

0659b80

MODULES-10023 fix xtrabackup tests on RHEL/CentOS 5

3efa123

fraenki changed the title ~~Fix multiple xtrabackup regressions (+JIRA ticket)~~ (MODULES-10023) Fix multiple xtrabackup regressions Mar 4, 2020

MODULES-10023 disable new xtrabackup tests on EOL operating systems

1f8eefc

(maint) exclude oracle6/centos6 platforms on running the feature tests

1472392

sanfrancrisko approved these changes May 12, 2020

View reviewed changes

sanfrancrisko merged commit db30274 into puppetlabs:master May 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(MODULES-10023) Fix multiple xtrabackup regressions #1245

(MODULES-10023) Fix multiple xtrabackup regressions #1245

fraenki commented Oct 16, 2019 •

edited

Loading

fraenki commented Oct 17, 2019 •

edited

Loading

sanfrancrisko commented Oct 22, 2019

sanfrancrisko commented Oct 28, 2019

fraenki commented Nov 3, 2019 •

edited

Loading

lionce commented Nov 8, 2019

fraenki commented Nov 12, 2019 •

edited

Loading

lionce commented Nov 13, 2019 •

edited

Loading

fraenki commented Jan 6, 2020 •

edited

Loading

sheenaajay commented Jan 17, 2020

carabasdaniel commented Feb 6, 2020

fraenki commented Feb 16, 2020 •

edited

Loading

carabasdaniel commented Feb 19, 2020

fraenki commented Feb 27, 2020 •

edited

Loading

sheenaajay commented Feb 28, 2020 •

edited

Loading

fraenki commented Feb 29, 2020 •

edited

Loading

sheenaajay commented Mar 4, 2020

sheenaajay commented Mar 10, 2020

fraenki commented Mar 29, 2020 •

edited

Loading

sheenaajay commented Mar 31, 2020

sheenaajay commented Apr 3, 2020

sheenaajay commented May 12, 2020

(MODULES-10023) Fix multiple xtrabackup regressions #1245

(MODULES-10023) Fix multiple xtrabackup regressions #1245

Conversation

fraenki commented Oct 16, 2019 • edited Loading

1. Regression: xtrabackup nonfunctional due to backup dir change

2. Regression: backup rotation nonfunctional due to incompatible find

Small quality of life improvements

fraenki commented Oct 17, 2019 • edited Loading

Verified: Full backups

Verified: Incremental backups

sanfrancrisko commented Oct 22, 2019

sanfrancrisko commented Oct 28, 2019

fraenki commented Nov 3, 2019 • edited Loading

lionce commented Nov 8, 2019

fraenki commented Nov 12, 2019 • edited Loading

lionce commented Nov 13, 2019 • edited Loading

fraenki commented Jan 6, 2020 • edited Loading

sheenaajay commented Jan 17, 2020

carabasdaniel commented Feb 6, 2020

fraenki commented Feb 16, 2020 • edited Loading

carabasdaniel commented Feb 19, 2020

fraenki commented Feb 27, 2020 • edited Loading

sheenaajay commented Feb 28, 2020 • edited Loading

fraenki commented Feb 29, 2020 • edited Loading

sheenaajay commented Mar 4, 2020

sheenaajay commented Mar 10, 2020

fraenki commented Mar 29, 2020 • edited Loading

sheenaajay commented Mar 31, 2020

sheenaajay commented Apr 3, 2020

sheenaajay commented May 12, 2020

fraenki commented Oct 16, 2019 •

edited

Loading

fraenki commented Oct 17, 2019 •

edited

Loading

fraenki commented Nov 3, 2019 •

edited

Loading

fraenki commented Nov 12, 2019 •

edited

Loading

lionce commented Nov 13, 2019 •

edited

Loading

fraenki commented Jan 6, 2020 •

edited

Loading

fraenki commented Feb 16, 2020 •

edited

Loading

fraenki commented Feb 27, 2020 •

edited

Loading

sheenaajay commented Feb 28, 2020 •

edited

Loading

fraenki commented Feb 29, 2020 •

edited

Loading

fraenki commented Mar 29, 2020 •

edited

Loading