HEP Batch System

Here at the Imperial College HEP group we are fortunate enough to have access to a very powerful resource - the Batch System.

In short it is a collection of processing nodes with 352 cores available to use, with processing time limits of 1 hour, 6 hours or 72 hours. Anyone in the group, with access to the machines lx04, lx05 or lx06 can use it. For more information please see the details available here.

This page will not echo available information, instead we will add useful tips and tricks that people come up with.

The Quick Sub Script

By Christopher Hunt

I developed this little tool after constantly running little scripts and programs that turned out to be a lot more processor and time consuming than I initially anticipated. As we have regulations regarding the correct usage of processing power on the lx- machines, it is only courteous that we run resource hungry programs in the environment that was designed for it - The Batch System. This tool turns the process of writing a script and submitting it to the system into a one-line command.

There are two parts to this tool. The executable script and the hidden directory.

The Script

This should be located in a directory designed for executable files, typically this would be ~/bin/ for most users. It performs some checks, sets some environment variables, and submits a single batch job. It is designed to be used as so:

sub <QUEUE> '<COMMANDS-TO-EXECUTE>'

A recent example for me would be:

sub long '. env.sh && python bin/simulate_mice.py --configuration_file Conf/Conf_test.py'

Note that it is designed such that all commands are enclosed with quotation marks (either single or double depending on the easiest way to do so). I would encourage people to read up on how BASH treats quotations if things don't immediately work (I know I can never remember!). In the above example I source a local environment script, before running my chosen command - safe in the knowledge that the script knows the current working directory.

The Hidden Directory

This is the other half of the tool, and should be located as "/.sub_script", although it is not difficult to change the name. Making use of the fact that the command qsub can include the names of local variables to be passed into the batch system environment, the command you would like to execute is stored in variable called "SUB_SCRIPT_COMMAND" and executed by a script called "/.sub_script/skeleton.sh".

All outputs from this tool are stored in the directory called "~/.sub_script/outputs/". I personally don't like them cluttering up the home directory - however you will need to remember to delete them after you review them to save one precious memory!

Getting the Tool

I have put a tar file containing all the files available to download at my home page: sub_script_v1.tar.gz

There is a little readme included for extra information, but for the quick installation:

1. Download "sub_script_v1.tar.gz" to your home directory "~/"

2. Un-tar the file in place. tar -vzxf sub_script_v1.tar.gz

3. Read the readme if you get bored. nano SUB_SCRIPT_README.txt

4. Copy or link the executable to your local executable directory. cp ~/.sub_script/sub ~/bin/ or ln -s ~/.sub_script/sub ~/bin/sub

5. If at first you don't succeed - check its executable chmod +x ~/bin/sub or ask someone intelligent...

Let me know if you have any comments or suggestions for improvements.