Merge pull request #2 from GOCDB/ssh

Use ssh instead of wget to fetch the dmp file

Author: gregcorbett

README.md

@@ -7,7 +7,7 @@ This repo contains the service and cron scripts used to run a failover gocdb ins
 * autoEngageFailover/
   * Contians a Service script (```gocdb-autofailover.sh```) and child scripts that monitors the main production instance. If a prolonged outage is detected, the GOCDB top DNS alias 'goc.egi.eu' is swtiched from the production instance to the failover instance. This switch can also be performed manually when needed. 
 * importDBdmpFile/
-  * Contains a sript that should be invoked by cron hourly (```1_runDbUpdate.sh```) to download and install a .dmp of the production DB into the local failover DB. This runs separtely from the autoEngageFailover process. 
+  * Contains a script that should be invoked by cron hourly (```1_runDbUpdate.sh```) to fetch and install a .dmp of the production DB into the local failover DB. This runs separtely from the autoEngageFailover process. 
 * nsupdate_goc/
   * Scripts for switching the DNS to/from the production/failover instance. 
 * archiveDmpDownload/
@@ -21,14 +21,13 @@ This repo contains the service and cron scripts used to run a failover gocdb ins
       |_ gocdb-autofailover.sh# MAIN SERVICE SCRIPT to mon production instance
       |_ engageFailover.sh    #   Child script, run if prolonged outage is detected
       
-  importDBdmpFile/            # Scripts download/install a .dmp of the prod data
+  importDBdmpFile/            # Scripts fetch/install a .dmp of the prod data
       |_ 1_runDbUpdate.sh     # MAIN SCRIPT that can be called from cron, invokes child scripts below 
       |_ ora11gEnvVars.sh     #   Setup oracle env
-      |_ getDump.sh           #   Download a .dmp of the production data 
+      |_ getDump.sh           #   Fetch a .dmp of the production data 
       |_ dropGocdbUser.sh     #   Drops the current DB schema
-      |_ loadData.sh          #   Load the last successfully downloaded DB dmp into the RDBMS
+      |_ loadData.sh          #   Load the last successfully fetched DB dmp into the RDBMS
       |_ gatherStats.sh       #   Oracle gathers stats to re-index
-      |_ pass_wgetrc_exemplar.txt #   Sample pwd file for getDump.sh (rename to pass_wgetrc) 
       |_ pass_file_exemplar.txt   #   Sample pwd file for DB (rename to pass_file)
 
   nsupdate_goc/              # Scripts for switching the DNS to the failover
@@ -51,15 +50,15 @@ following:
 * the gocdb admins are emailed, 
 * the age of the last successfully imported dmp file is
   checked to see that it is current, 
-* the hourly cron that downloads the dmp file is stopped (see
+* the hourly cron that fetches the dmp file is stopped (see
   importDBdmpFile below), 
 * <strike>symbolic links to the server cert/key are updated so they
   point to the 'goc.egi.eu' cert/key</strike> (note, no longer needed as cert contains dual SAN) 
 * the dnscripts are invoked to change the dns (see
   nsupdate_goc below).  
 
 ## /root/importDBdmpFile/ 
-Contains scripts that download the .dmp file and install this
+Contains scripts that fetches the .dmp file and install this
 dmp file into the local Oracle XE instance. The master script
 is '1_runDbUpdate.sh' which needs to be invoked from an hourly
 cron:   
@@ -71,10 +70,11 @@ cron:
 /root/importDBdmpFile/1_runDbUpdate.sh
 ```
 
-You will also need to modify the two password files to specify
-your own pw ('pass_wgetrc' and 'pass_file'). These contain the
-pw for the secure download of dmp file and the pw of the DB
-system user. 
+You will also need to: 
+* generate a public/private key pair using `ssh-keygen` and ensure the public
+key is present on the host with the database dmp file.
+* populate `importDBdmpFile/failover_TEMPLATE.sh` with
+appropriate values and copy it to `/etc/gocdb/failover.sh`
 
 ## /root/nsupdate_goc/
 Contains the nsupdate keys and nsupdate scripts for switching
@@ -138,7 +138,7 @@ failover so the dns points back to the production instance
 and restore/restart the failover process. This includes:   
 * <strike>restore the symlinks to the goc.dl.ac.uk server cert and key
   (see details below)</strike> (no longer needed as cert contains dual SAN) 
-* restore the hourly cron to download the dmp of the DB
+* restore the hourly cron to fetch the dmp of the DB
 * run nsupdate procedure to repoint 'goc.egi.eu' back to
   'gocdb-base.esc.rl.ac.uk'
   MUST read /root/nsupdate_goc/nsupdateReadme.txt. 

importDBdmpFile/failover_TEMPLATE.sh

@@ -0,0 +1,8 @@
+# This file is a template for /etc/gocdb/failover.sh.
+# Copy this file to that location and set the variables as appropriate.
+
+# Set the user@host:path to copy the DB dump file from.
+DB_DUMP_FROM=user@hostname:/path/to/db/dump
+
+# Set the local location to copy the DB dump file to.
+DB_DUMP_TO=/tmp

importDBdmpFile/getDump.sh

@@ -1,19 +1,16 @@
 #!/bin/bash
 
-# seems that curl don't return a 0 exit code when it fails to download successfully, see: 
-# http://superuser.com/questions/590099/can-i-make-curl-fail-with-an-exitcode-different-than-0-if-the-http-status-code-i
-# 
-#curl --capath /etc/grid-security/certificates -u failover:kugA7Rer https://goc.egi.eu/dbDump/goc5dump.dmp -o /tmp/goc5dump.dmp
+# Set some things that cause this script to exit on a failure,
+# rather than carry on blindly.
+# -e Exit on any error
+# -u Classify unset variables as errors
+set -eu
 
-# therefore use wget which does return 0 exit code if downloaded ok
-# /usr/bin/wget -O /tmp/goc5dump.dmp --user failover --password u_LK28_B2fv_dm --no-check-certificate https://goc.egi.eu/dbDump/goc5dump.dmp
+# Get useful variables to refer to later in this script.
+source /etc/gocdb/failover.sh
 
-DUMPDIR=/tmp
-BASEDIR=/root/importDBdmpFile
+# Copy the DB dump file.
+/usr/bin/scp $DB_DUMP_FROM $DB_DUMP_TO
 
-export WGETRC=$BASEDIR/pass_wgetrc
-
-/usr/bin/wget --no-check-certificate \
-	--ca-directory /etc/grid-security/certificates \
-	-O $DUMPDIR/goc5dump.dmp \
-	https://goc.egi.eu/dbDump/goc5dump.dmp
+# unset things to not affect the rest of the Failover process.
+set +eu