Merge pull request #92 from UCL-ARC/removescratch

Remove mentions to /scratch/scratch; remove old Singularity section

Author: heatherkellyucl

mkdocs-project-dir/docs/Background/Data_Storage.md

@@ -28,13 +28,15 @@ Many programs will write hidden config files in here, with names beginning with
 
 ### Scratch
 
-Every user also has a Scratch directory. It is intended that this is a larger space to keep your working
-files as you do your research, but should not be relied on for secure long-term permanent storage.
+Every user also has a Scratch directory. On Myriad and Kathleen this exists for backwards 
+compatibility and all of your home space should be considered scratch.
+
+It is intended that this is a larger space to keep your working files as you do your 
+research. It should not be relied on for secure long-term permanent storage.
 
 Important data should be regularly backed up to another location.
 
-- Location: `/scratch/scratch/<username>`
-- Also at: `/home/<username>/Scratch` (a shortcut or symbolic link to the first location).
+- Location: `/home/<username>/Scratch` (either a directory or a shortcut or symbolic link to a different location).
 
 ### Temporary storage for jobs (TMPDIR)
 
@@ -62,8 +64,7 @@ The ACFS is available read-write on the login nodes but read-only on the compute
 that your jobs can read from it, but not write to it, and it is intended that you copy data onto
 it after deciding which outputs from your jobs are important to keep.
 
-Initially rolled out on Kathleen, you will later be able to access it from Myriad too and see
-the same files from both clusters.
+Available on Myriad and Kathleen, you will be able to see the same files from both clusters.
 
 - Location: `/acfs/users/<username>`
 - Also at: `/home/<username>/ACFS` (a shortcut or symbolic link to the first location).

mkdocs-project-dir/docs/Clusters/Myriad.md

@@ -49,25 +49,25 @@ Please refer to the page on [How do I transfer data onto the system?](../howto.m
 
 ## Quotas
 
-The default quotas on Myriad are 150GB for home and 1TB for Scratch.
+The default quota on Myriad is 1TB for home (which is also considered scratch).
 
-These are hard quotas: once you reach them, you will no longer be able
-to write more data. Keep an eye on them, as this will cause jobs to fail
+The "hard limit" number means that once you reach it, you will no longer be able
+to write more data. Keep an eye on your quota, as this will cause jobs to fail
 if they cannot create their .o or .e files at the start, or their output
 files partway through.
 
-You can check both quotas on Myriad by running: 
+You can check your quota on Myriad by running: 
 
 ```
-lquota
+gquota
 ``` 
 
 which will give you output similar to this:
 
 ```
-     Storage        Used        Quota   % Used   Path
-        home  721.68 MiB   150.00 GiB       0%   /home/uccaxxx
-     scratch   52.09 MiB     1.00 TiB       0%   /scratch/scratch/uccaxxx
+Current Usage: 108.7GiB
+Soft Limit: 1024GiB
+Hard Limit: 1024GiB
 ```
 
 You can apply for quota increases using the form at [Additional Resource Requests](../Additional_Resource_Requests.md).

mkdocs-project-dir/docs/Software_Guides/AlphaFold3.md

@@ -25,10 +25,10 @@ Having added the model weights, you will also need to create input and output di
 You should now have three locations which are unique to you. For ease of reference  in the job script I'm going to set environment variables to them so that the command-line is consistent and so you can change them without messing with the command line options.
 
 ```
-export AF3_INPUT=/scratch/scratch/uccaoke/af3_input # Replace with your input folder
+export AF3_INPUT=/home/${USER}/Scratch/af3_input # Replace with your input folder
 export AF3_INPUT_FILE=fold_input.json # Replace with a file in your input folder
-export AF3_OUTPUT=/scratch/scratch/uccaoke/af3_output # Replace with your output folder
-export AF3_WEIGHTS=/scratch/scratch/uccaoke/weights # Replace with the folder you put the weights in
+export AF3_OUTPUT=/home/${USER}/Scratch/af3_output # Replace with your output folder
+export AF3_WEIGHTS=/home/${USER}/Scratch/weights # Replace with the folder you put the weights in
 ```
 
 ## Running AlphaFold3
@@ -59,10 +59,10 @@ Write a job script that requests GPU nodes:
 # Set the working directory to the current working directory.
 #$ -cwd
 
-export AF3_INPUT=/scratch/scratch/uccaoke/af3_input # Replace with your input folder
+export AF3_INPUT=/home/${USER}/Scratch/af3_input # Replace with your input folder
 export AF3_INPUT_FILE=fold_input.json # Replace with a file in your input folder
-export AF3_OUTPUT=/scratch/scratch/uccaoke/af3_output # Replace with your output folder
-export AF3_WEIGHTS=/scratch/scratch/uccaoke/weights # Replace with the folder you put the weights in
+export AF3_OUTPUT=/home/${USER}/Scratch/af3_output # Replace with your output folder
+export AF3_WEIGHTS=/home/${USER}/Scratch/weights # Replace with the folder you put the weights in
 
 apptainer exec --nv --bind ${AF3_INPUT}:/root/af_input --bind ${AF3_OUTPUT}:/root/af_output --bind ${AF3_WEIGHTS}:/root/models --bind /shared/ucl/apps/AlphaFold3_db:/root/public_databases --no-home --no-mount bind-paths  /shared/ucl/apps/AlphaFold3/alphafold3.sif sh -c "XLA_FLAGS='--xla_disable_hlo_passes=custom-kernel-fusion-rewriter' python3 /app/alphafold/run_alphafold.py --json_path=/root/af_input/${AF3_INPUT_FILE} --model_dir=/root/models --db_dir=/root/public_databases --output_dir=/root/af_output --flash_attention_implementation=xla"
 ```
@@ -95,12 +95,12 @@ If you wish to queue for an A100, this job should work:
 # Set the working directory to the current working directory.
 #$ -cwd
 
-export AF3_INPUT=/scratch/scratch/uccaoke/af3_input # Replace with your input folder
+export AF3_INPUT=/home/${USER}/Scratch/af3_input # Replace with your input folder
 export AF3_INPUT_FILE=fold_input.json # Replace with a file in your input folder
-export AF3_OUTPUT=/scratch/scratch/uccaoke/af3_output # Replace with your output folder
-export AF3_WEIGHTS=/scratch/scratch/uccaoke/weights # Replace with the folder you put the weights in
+export AF3_OUTPUT=/home/${USER}/Scratch/af3_output # Replace with your output folder
+export AF3_WEIGHTS=/home/${USER}/Scratch/weights # Replace with the folder you put the weights in
 
 apptainer exec --nv --bind ${AF3_INPUT}:/root/af_input --bind ${AF3_OUTPUT}:/root/af_output --bind ${AF3_WEIGHTS}:/root/models --bind /shared/ucl/apps/AlphaFold3_db:/root/public_databases --no-home --no-mount bind-paths  /shared/ucl/apps/AlphaFold3/alphafold3.sif sh -c "python3 /app/alphafold/run_alphafold.py --json_path=/root/af_input/${AF3_INPUT_FILE} --model_dir=/root/models --db_dir=/root/public_databases --output_dir=/root/af_output"
 ```
 
-For other options you can pass to AlphaFold 3, please consult [DeepMind's documentation](https://github.com/google-deepmind/alphafold3/tree/main/docs).
+For other options you can pass to AlphaFold 3, please consult [DeepMind's documentation](https://github.com/google-deepmind/alphafold3/tree/main/docs).

mkdocs-project-dir/docs/Software_Guides/Singularity.md

@@ -78,77 +78,10 @@ as long as they use a local filesystem and not home or Scratch.
 Our default setup uses `$XDG_RUNTIME_DIR` on the local disk of the login nodes, or `$TMPDIR` on a 
 compute node (local disk on the node, on clusters that are not diskless).
 
-If you try to build a container on a parallel filesystem, it will fail with a number of
+If you try to build a container on a parallel filesystem, it may fail with a number of
 permissions errors.
 
 
-## Singularity
-
-Run `singularity --version` to see which version we currently have installed.
-
-
-!!! important "Singularity update to Apptainer"
-    On Myriad, we are updating to Singularity to Apptainer. This update will occur on 14th 
-    November during a [planned outage](../Planned_Outages.md)
-
-    This update may affect any containers that are currently downloaded, so users will have to test
-    them to check their workflow still functions correctly after the update. We expect most to work 
-    as before, but cannot confirm this.
-
-    A Singularity command that will no longer be available in Apptainer is 
-    `singularity build --remote`. If any of you have workflows that depend on this, 
-    please email [email protected]. We are currently looking into how we would provide 
-    equivalent functionality.
-
-    Updates to the other clusters will follow, dates tbc.
-
-
-### Set up cache locations and bind directories
-
-The cache directories should be set to somewhere in your space so they don't fill up `/tmp` on 
-the login nodes.
-
-The bindpath mentioned below specifies what directories are made available inside the container - 
-only your home is bound by default so you need to add Scratch.
-
-You can either use the `singularity-env` environment module for this, or run the commands manually.
-
-```
-module load singularity-env
-```
-
-or:
-
-```
-# Create a .singularity directory in your Scratch
-mkdir $HOME/Scratch/.singularity
-
-# Create cache subdirectories we will use / export
-mkdir $HOME/Scratch/.singularity/tmp
-mkdir $HOME/Scratch/.singularity/localcache
-mkdir $HOME/Scratch/.singularity/pull
-
-# Set all the Singularity cache dirs to Scratch
-export SINGULARITY_CACHEDIR=$HOME/Scratch/.singularity
-export SINGULARITY_TMPDIR=$SINGULARITY_CACHEDIR/tmp
-export SINGULARITY_LOCALCACHEDIR=$SINGULARITY_CACHEDIR/localcache
-export SINGULARITY_PULLFOLDER=$SINGULARITY_CACHEDIR/pull
-
-# Bind your Scratch directory so it is accessible from inside the container
-#      and the temporary storage jobs are allocated
-export SINGULARITY_BINDPATH=/scratch/scratch/$USER,/tmpdir
-```
-
-Different subdirectories are being set for each cache so you can tell which files came from where.
-
-You probably want to add those `export` statements to your `.bashrc` under `# User specific aliases and functions` so those environment variables are always set when you log in.
-
-For more information on these options, have a look at the Singularity documentation:
-
-* [Singularity user guide](https://sylabs.io/guides/3.5/user-guide/index.html)
-* [Singularity Bind Paths and Mounts](https://sylabs.io/guides/3.5/user-guide/bind_paths_and_mounts.html)
-* [Singularity Build Environment](https://sylabs.io/guides/3.5/user-guide/build_env.html)
-
 ## Downloading and running a container
 
 Assuming you want to run an existing container, first you need to pull it from somewhere online that

mkdocs-project-dir/docs/Supplementary/Troubleshooting.md

@@ -49,23 +49,26 @@ see here it's trying to parse `^M` as an option.
 
 ### I think I deleted my Scratch space, how do I restore it?
 
-You may have accidentally deleted or replaced the link to your Scratch
-space. Do an `ls -al` in your home - if set up correctly, it should look
-like this:
+If you are on a cluster (not Myriad) where Scratch is a symbolic link (shortcut) you may have 
+accidentally deleted or replaced the link to your Scratch space but not removed its contents. 
+Do an `ls -al` in your home - if set up correctly, it should look like this:
 
 ```
 lrwxrwxrwx   1 username group       24 Apr 14  2022 Scratch -> /scratch/scratch/username
 ```
 
 where `username` is your UCL user ID and `group` is your primary group. 
 
-If this link is not present, you can recreate it with
+If this link is not present, it will be automatically recreated when you log back in, or you 
+can recreate it with
 
 ```
 ln -s /scratch/scratch/$(whoami) Scratch
 ```
 
-If you have actually deleted the files stored in your Scratch space, there is unfortunately no way to restore them.
+If you are on a cluster like Myriad where Scratch is a normal directory in your home or you 
+have actually deleted the files stored in your Scratch space, there is unfortunately no way 
+to restore them.
 
 ### Which MKL library files should I use to build my application?