Add MariaDB dumpfile fetch and import.

[ghost]

Draft request at this stage as preview hoping for comments, on style and substance, and expecting more integration work to be needed.

Closes #7 #5 (for MariaDB only)

But does not yet address #6

[gregcorbett]

What's the output like when the logging level is error? The current monitoring assumes no output on a successful run.

[ghost]

My reading is that the current monitoring requires "completed ok" as the final line of output from a successful run. Any other condition is a failure.
I am re-working the logging to allow specification of a log file to which info/debug/error output is written while "completed ok" or an error message goes to stdout.

[gregcorbett]

so "completed ok" currently goes to the syslog via the logger command, whereas the monitoring checks the log produced by the cron job that runs the import process hourly and assumes the underlying 1_runDbUpdate.sh produces no output to stdout or stderr on a successful import.

You could have the logging report at the debug level rather than info - so if something is going wrong with the process one can switch the more verbose mode on.

[ghost]

:lightbulb: 1_runDbUpdate.sh has an internal bash function 'logger' which (appears to me) is hardwired to output to a local file, so not to syslog. How does the output end up in syslog?

In any case, I've changed it so that stdout should be empty and "completed ok", or error message, will go in the specified log file, with return status as appropriate.

[gregcorbett]

ah yes, you're right - I mistook logger for the pre defined function and not one that was defined in the script itself. And I've only just discovered updateLog.txt is even a thing...

error messages should go to stderr as well, I'm not convinced of the need to have two log files - one for the script and one for the cron.

[gregcorbett]

Is dbImport.pwd intended to be used for the configuration of other mysql options? or solely for the password?

It's referenced as mysqlOptionsPath in the code, but the templates name suggests its only for the password.

[gregcorbett]

codacy has highlighted somethings as well (though the README issues probably keep the file self consistent, so if you wanted to fix them, please do it in a standalone PR.

[ghost]

Is dbImport.pwd intended to be used for the configuration of other mysql options? or solely for the password?

It's referenced as mysqlOptionsPath in the code, but the templates name suggests its only for the password.

Yes, that's a bit confusing. There's no intent, but it is possible and there seems no reason to restrict it. I have renamed to a .cnf_template suffix and included the user so that all mysql options arrive through that route. And expanded the comments in config.ini.

[ghost]

After discussion:

1. Add ISO 8601 timestamp to the logged output
2. Implement failure retry with delay backoff

[ghost]

Closing for now as I'll be force pushing some intermediate/incomplete/untested updates.