Clean-up hh5 references, and other logkeeping/documentation fixes (#275)

* Remove hh5 references from docs, build scripts

* Updated datastores quickstart notebook

* Note openmpi requirement for builds

* Update CHANGELOG to v0.1.5 inclusive

* Re-run of docs/usage/quickstart.ipynb

* Correct build script module loads

Author: marc-white

.gitignore

@@ -133,4 +133,7 @@ sandpit.ipynb
 bin/build_all.sh.o*
 
 # Vs Code 
-.vscode/
+.vscode/
+
+# User-made files
+bin/build_all_test.sh

CHANGELOG.rst

@@ -4,6 +4,29 @@ Changelog
 This CHANGELOG documents only key changes between versions. For a full description 
 of all changes see https://github.com/ACCESS-NRI/access-nri-intake-catalog/releases
 
+v0.1.5
+------
+
+21/11/2024
+
+- Switch to a date-based catalog versioning schema
+- Add support for BARPA data
+- Allow for coordinate searching
+- Translators rationalisation
+- metadata-validate provides complete report
+- EraiTranslator and catalogs removed
+- Add support for ccam data
+- Support for Python 3.12
+
+v0.1.4
+------
+
+03/10/2024
+
+- Added archive/reference copies of existing catalog source metadata
+- Store experiment and metadata schemas internally
+- Restucture Builders
+
 v0.1.3
 ------
 

bin/build_all.sh

@@ -1,7 +1,7 @@
 #!/bin/bash -l
 
 #PBS -P iq82
-#PBS -l storage=gdata/xp65+gdata/ik11+gdata/cj50+gdata/hh5+gdata/p73+gdata/dk92+gdata/al33+gdata/rr3+gdata/fs38+gdata/oi10
+#PBS -l storage=gdata/xp65+gdata/ik11+gdata/cj50+gdata/p73+gdata/dk92+gdata/al33+gdata/rr3+gdata/fs38+gdata/oi10
 #PBS -q normal
 #PBS -W block=true
 #PBS -l walltime=03:00:00
@@ -26,7 +26,8 @@ if [ ! $# -eq 0 ]; then
 fi
 
 module use /g/data/xp65/public/modules
-module load conda/access-med-0.6
+module load conda/analysis3-24.12
+module load openmpi
 
 OUTPUT_BASE_PATH=/g/data/xp65/public/apps/access-nri-intake-catalog
 CONFIG_DIR=/g/data/xp65/admin/access-nri-intake-catalog/config

bin/test_end_to_end.sh

@@ -1,7 +1,7 @@
 #!/bin/bash -l
 
 #PBS -P iq82
-#PBS -l storage=gdata/xp65+gdata/ik11+gdata/cj50+gdata/hh5+gdata/p73+gdata/dk92+gdata/al33+gdata/rr3+gdata/fs38+gdata/oi10
+#PBS -l storage=gdata/xp65+gdata/ik11+gdata/cj50+gdata/p73+gdata/dk92+gdata/al33+gdata/rr3+gdata/fs38+gdata/oi10
 #PBS -q normal
 #PBS -W block=true
 #PBS -l walltime=00:30:00

docs/datastores/builders.rst

@@ -8,7 +8,7 @@ set of Intake-ESM datastore Builders for different ACCESS model outputs. In gene
 datastore for your ACCESS model output should be as simple as passing your output base directory to an 
 appropriate Builder.
 
-The access-nri-intake package is installed in the :code:`hh5` and :code:`xp65` analysis environments, or 
+The access-nri-intake package is installed in the :code:`xp65` analysis environment, or 
 users can install it into their own environment (see :ref:`installation` for details). The Builders can be 
 imported from the :code:`access_nri_intake.source.builders` submodule.
 

docs/datastores/quickstart.ipynb

@@ -34,23 +34,18 @@ In order to use the catalog, you will need to have the following:
 
       Catalog users will only be able to load data from projects that they have access to.
 
-#. **An installation of the catalog**: the catalog is pre-installed in the 
-   `CLEX CMS "analysis3" conda environment <http://climate-cms.wikis.unsw.edu.au/Conda>`_ (>=23.04) and 
-   the ACCESS-NRI "access-med" conda environment (all versions). Users are encouraged to use one of these
-   environments to use the catalog. Alternatively, you can install the catalog into your own environment 
+#. **An installation of the catalog**: the catalog is pre-installed in 
+   the ACCESS-NRI "access-med" conda environment (all versions). Users are encouraged to use this
+   environment to use the catalog. Alternatively, you can install the catalog into your own environment 
    as described below.
 
-   .. attention::
-      In order to use the CLEX CMS conda environments, you will need to also be a member of the 
-      :code:`hh5` project
-
 .. _installation:
 
 Installing the catalog
 ^^^^^^^^^^^^^^^^^^^^^^
 
 Most users will not need to install the catalog themselves and will instead use the catalog through one 
-of the public analysis environments provided in either :code:`hh5` or :code:`xp65` (see below).
+of the public analysis environments provided in :code:`xp65` (see below).
 
 Advanced users that want to install the catalog into their own environment can do so in three ways:
 
@@ -95,16 +90,16 @@ data it references are available from your session. In particular:
 
   .. include:: ../storage_flags.rst
 
-  If you want to use the :code:`hh5` analysis environment, you'll also need to add :code:`gdata/hh5`.
-
   .. attention::
      You need to be a member of all projects you enter here. You can see what projects you are part of 
      at `https://my.nci.org.au/mancini <https://my.nci.org.au/mancini>`_.
 
 * **Setting the environment**: you need to make sure that the catalog is installed in your JupyterLab 
-  session. As mentioned above, the easiest way to do this is to use either the :code:`hh5` or 
-  :code:`xp65` public analysis environments. You can activate the :code:`hh5` environment within your 
+  session. As mentioned above, the easiest way to do this is to use the
+  :code:`xp65` public analysis environment. You can activate the :code:`xp65` environment within your 
   JupyterLab session using the "Advanced options" to set the "Module directories" to 
-  :code:`/g/data/hh5/public/modules` and "Modules" to :code:`conda/analysis3-unstable`. Similarly, to use 
-  the :code:`xp65` environment, set "Module directories" to :code:`/g/data/xp65/public/modules` and 
-  "Modules" to :code:`conda/access-med`.
+  :code:`/g/data/xp65/public/modules` and "Modules" to :code:`conda/access-med`.
+
+  .. note::
+   If you are intending to do a multi-threaded catalog build, you will also need to explicitly load
+   the :code:`openmpi` module in "Modules".