Create catalog on the basis of the output on non NCI cluster

[VanuatuN]

I've just got an output from the ACCESS-OM2 RYF run on Leonardo supercomputer. Wondering is there any easy way to replace location of the output, so that I can use the recipes to plot the data model outputs?

[marc-white]

Hi @VanuatuN , thanks for getting in touch!

What exactly are you looking to do? Are you:

1. Looking to get your data onto the Gadi supercomputer in Australia, and have it added to the access-nri-intake-catalog so other users can access it? Or,
2. Are you asking how to access your data on Leonardo via recipes (which I presume is the COSIMA Cookbook (https://github.com/COSIMA/cosima-recipes/), which typically uses access-nri-intake-catalog to load data.

I presume you're after number 2...

[VanuatuN]

Hi @marc-white, thank you for your reply!

I've just finished porting ACCES OM2 on Leonardo supercomputer under the slurm scheduler, all works
and saves output files locally. So yes, the option number 2. Would be grateful for any hint on how to adapt
existing tools for the output analysis to another supercomputer. Do you think the "intake.cat.access_nri" in the recipes
can be adjusted so that it takes data from the local machine?

Thanks!
Natalia

[marc-white]

Hi @VanuatuN , yes, that should be very doable.

These lines in the recipes:

cat = intake.cat.access_nri
experiment = cat['<experiment name>'] # -> intake_esm catalog

return an Intake-ESM catalog. Therefore, for compatibility with the recipes, you'll simply need to build your data into an Intake-ESM catalog. For example, see: https://intake-esm.readthedocs.io/en/stable/how-to/build-a-catalog-from-timeseries-files.html .

You'll notice that access-nri-intake-catalog already includes a custom builder for ACCESS-OM2. Therefore, you could go directly to using that Builder to manually create yourself an Intake-ESM catalog, rather than trying to massage the standard Intake-ESM builder into shape.

[VanuatuN]

@marc-white, thank you very much for the hint, otherwise it would take me much more time
to sort it out.
Sounds very doable, indeed! I'll work on that in the coming days.
Natalia