CTSM 5.3: New surface datasets, git-fleximod intro, Leung dust, explicit AC, Matrix-CN, FATES transient LU, time/history metadata changes

Purpose and description of changes since CTSM 5.2 (tag ctsm5.2.005)

New features

• manage_externals replaced by git-fleximod. (PR #2559)
• No longer runs the 0th time step in first segment of startup and hybrid runs; branch and continue runs never had this 0th time step. (PR #2084)
• New CN Matrix method speeds up spinup for the BGC model. (PR #640; Liao et al. 2023).
• New Leung_2023 dust emissions. (PR #1897; Leung et al 2023, Leung et al. 2024)
• Explicit air conditioning for the urban model. (PR #2275; Li et al. 2024)
• FATES compsets can now be run with transient land use; off by default. (PR #2507)
• Ability to handle CAM7 in LND_TUNING_MODE. (PR #2632)

Answer changes

Changes to defaults for clm6_0 physics:

• Urban explicit A/C turned on (links above).
• Snow thermal conductivity method is now Sturm1997. (PR #2348; see also discussion #1960)
• New initial conditions files for f09 ("1-degree" 1850, 2000), f19 (“2-degree” 1850), and ne30 (1850, 1979, 2000) resolutions.
• New crop calendars. (PR #2664; informed by Rabin et al., 2023)
• Dust emissions method is now Leung_2023 (links above).
• Excess ice is turned on. (PR #1787)
• Updates to MEGAN for BVOCs. (PR #2588)
• New BGC fire method li2024crujra: Avoid crop fires during growing season; allow lightning ignitions in tropical closed forests; add effect of landscape fragmentation on ignitions and duration; recalibrate against GFED5 burned area and with CRU-JRA climate. (PR #2684, PR #2711, PR #2715)

Changes for all physics versions:

• Parameters updated for CRU-JRA forcing: PPE-based modifications were made to the parameters leafcn, slatop, froot_leaf, medlynslope, and kmax. Lowers LAI and biomass in boreal and tropical forests, without reducing latent heat in the tropics. Affected PFTs: NET temperate, NET boreal, BET tropical, BDS boreal, C3 arctic grass. (PR #2500)
• FATES parameter file updated (see section below).
• Pass active glacier (CISM) runoff directly to river model (MOSART) (MOSART PR #94)
• New surface datasets and landuse timeseries files (see section below).
• CNMatrix is new default spinup method (links above).

Heads up

• Small glacier changes mean that you can’t use a 5.3 surface dataset with pre-5.3 code and vice versa anymore. (Merged with PR #2500)
• Updates the definition of history variable “time” from end of time_bounds to middle of time_bounds. (PR #2838; see section below)
• Standardizes history variable attributes and a history dimension name. (PR #2052; see section below)

Additional detail

Changes related to time and history files

(Note that the same information in this section applies to MOSART and RTM.)

Startup and hybrid runs no longer run the 0th time step, consistent with the same change in CAM. (Branch and continue runs never had this 0th time step.) This means you will not get an extraneous initial history file anymore. In some circumstances this may also affect the names of history files.

In most cases, the history time variable is now defined as the middle of a history file’s time_bounds instead of the end, for consistency with the same change in CAM. The exception is if you specify hist_avgflag_pertape = 'I' for that file, in which case it will be treated as an “instantaneous” file. Instantaneous history files (a) have their time coordinate set to the end of the last timestep (as did all history files before this tag) and (b) do not include time_bounds.

The history dimension name hist_interval (of output variable time_bounds) is standardized to be nbnd. History variables time_bounds, mcdate, mcsec, mdcur, and mscur are standardized to include the calendar attribute.

New surface datasets and landuse timeseries files (PR #2500)

• Transient landuse timeseries files going back to 1700 now possible (and made for f09).
• Fix an important bug on soil fields that was there since ctsm5.2.0. This has the side effect of mksurfdata_esmf now giving identical answers with a change in number of processors, as it should. (Issue #2744)
• Surface datasets now provided for the ne0np4.POLARCAP.ne30x4 grid. (PR #2716, issue #2720)
• Surface datasets now have their version number embedded to prevent mismatch of surface dataset and CTSM version. (Issue #2723)
• Remove outdated hydrology VIC (Variable Infiltration Capacity Hydrology model) fields from surface datasets.
• Updates to input datasets:
  • PFT/LAI/soil-color raw datasets; now from the TRENDY2024 timeseries that ends in 2023. (Issues #2570 and #2452)
  • Two fire datasets: crop fire peak month and peatland fraction. (Issue #2618)
  • Glacier behavior dataset (related to how non-Greenland glaciers are handled). (Issue #423)

Changes to FATES parameter file

• PR #2507 (ctsm5.2.013) / FATES PR #1116 (sci.1.77.1_api.36.0.0)
  • Adds new parameters for new land use harvest mode and land use by PFT capabilities
• PR #2700 (ctsm5.3.003) / FATES PR #1255 (sci.1.78.3_api.36.1.0)
  • Adds two arctic shrub PFTs, increasing the number of default PFTs to 14 and update arctic grass parameters (FATES PR#1236)
  • Splits fates_turnover_leaf parameter into canopy and understory specific turnover rates and provides new values for both (FATES PR#1136)
  • Updates grass allometry parameters (FATES PR#1206)
  • Changes the prescribe nutrient uptake defaults from 1 to 0 for all PFTs (only relevant to ELM-FATES)

Changes to rpointer files

The rpointer files are simple text files that CESM uses to keep track of how far simulations have progressed, pointing to the filename of the latest restart file for that component. There is one such file for each component, so for CTSM I cases that's lnd, cpl, and atm (and rof if it's active). Normally, when the user is just extending the length of simulations, there’s no need to worry about these files.

However, if there was a problem when a simulation shut down, it's possible that different components will have mismatched restarts and rpointer files. In the past, this meant figuring out what restart file should be pointed to in each component rpointer file and correcting it by hand in an editor. There was only the final set of rpointer files that was kept for a case.

Now, with this update, the lnd, cpl, and atm rpointer files have the simulation date in the filenames, so it's easy to spot if the restarts are mismatched for one of the components. Also, since there are matching rpointer files for each time restarts are created, it's now easier to (a) make sure restarts and rpointer files are all correctly matched and (b) for a user to take a set of restarts and matching rpointer files to start up from for any part of an existing simulation. This means you don't have to hand-edit the rpointer files, making sure you don't make a mistake when you do.

Old rpointer filenames:

• rpointer.atm
• rpointer.cpl
• rpointer.lnd

New names:

• rpointer.atm.YYYY-MM-DD-SSSSS
• rpointer.cpl.YYYY-MM-DD-SSSSS
• rpointer.lnd.YYYY-MM-DD-SSSSS

Where YYYY-MM-DD-SSSSS is the year, month, day, and seconds into the day for the simulation timestamp. For example, rpointer.lnd.2000-01-01-00000 for a rpointer file for starting at midnight (beginning of the day) on January 1st, 2000.

Note that this is backwards-compatible, so for all the components you can use either the new or old format for the rpointer filenames. Thus, if you are restarting from an existing case before ctsm5.3.016, you CAN use the rpointer filenames that don't have the timestamps in the name.

Simulations supportin...

CTSM 5.2: New surface datasets and mksurfdata_esmf tool to create them

Summary

• All new surface datasets, with updated input datasets.
• New mksurfdata_esmf tool to make global surface datasets.
• New tools to create inputs for regional simulations.
• ne0np4 grid: New 1979 surface dataset and 1979-2026 land use.
• Transient urban and lake by default turned on for transient cases.
• Ocean is run as baresoil rather than wetland (for clm6_0).
• The urban streams file was also updated.
• Update the README files.
• New FATES parameter file: Tree PFT allometry, allometric mode options, leaf maintenance scaling coefficients.

New surface datasets

The new surface datasets are incompatible with previous versions (for example the ctsm5.1 series)—ctsm5.2.0 and following versions can NOT use the previous ctsm5.1 datasets, and vice versa.

See the section below about the new datasets used in their creation. Improvements in how landunits on coastal areas were also made.

Fields added to the surface datasets in ctsm5.2:

• ORGC, BULK, CFRAG, PHAQ (soil data) (currently NOT used by CTSM)
• mapunits (map units from the soil dataset)
• LANDFRAC_MKSURFDATA (for reference NOT used by CTSM)
• PCT_OCEAN (previously PCT_WETLAND was used)

Fields removed from the surface datasets in ctsm5.2:

• AREA
• PFTDATA_MASK

New input data used for making surface datasets

• New soil dataset: ISRIC/WISE dataset (Batjes, 2016; https://doi.org/10.1016/j.geoderma.2016.01.034)
• New PFT, soil-color, LAI datasets: Created by Peter J. Lawrence (2022)
• New Glacier datasets: Glacier outlines from RGI version 6 (Arendt et al., 2017).
  • vector data for GrIS and AIS retrieved from BedMachine version 4 and version 2 (Morlighem et al., 2017, 2020), respectively.
  • 30-arcsec topography/land mask retrieved GMTED2010 (Danielson and Gesch, 2011).
• New urban datasets: Gao and O'Neill (2021) and Gao and Pesaresi (2022), Oleson and Feddema (2020)
• New lake datasets: HydroLake: Messager et. al. (2016)

New mksurfdata_esmf Tool

mksurfdata_esmf is a parallel version of mksurfdata that uses ESMF regridding directly so that offline mapping files don't have to be created as a separate step. This allows surface datasets to be created at much higher resolutions.

The build for the tool is based on the CESM/CIME build system and uses cmake. This allows the build to be kept up with changes in CESM. Currently it's only setup and working on Derecho, but this design will enable it to be built and run on any CESM-supported machine (or a machine that a user ports to).

Any input grid from ccs_config can be used, or the user can supply their own mesh file to define the output grid. The user no longer has to add to the list of valid resolutions (as in the now-deprecated mksurfdata_map).

Creation of supported single point datasets. These datasets are created through the use of subset_data.

Test datasets for dynUrban, dynLake, and dynPFT is done with a simple NCO script.

All datasets can be easily made by running make all in the tools/mksurfdata_esmf/ directory.

For detailed instructions, see tools/mksurfdata_esmf/README.md.

release-clm5.0.37

Synopsis

Changes needed to run on Derecho. Flip the test list from Cheyenne to Derecho.
Add working PE layouts. Remove tools and tools testing. Update readme files.

What's Changed

• add smyle L83 compset and SSP3-7.0 extension by @jedwards4b in #1614
• Fix issues #1769, #1607, #1389 in release documentation by @olyson in #1781
• Fix issue #1864 in release documentation by @olyson in #1916

release-cesm2.2.04: Update PElayout and tests to work on Derecho

Get working PE-layouts for Derecho. Move test list over to Derecho from
Cheyenne.
Remove all of the CTSM tools, users should use the latest.
Some fixes to various tests to get them working.

NEON release: Some NEON updates fixing AG sites, update MOSART, small fixes

release tag for use with NEON:

Minor changes to python scripts and usermod_dirs for NEON cases. Also update the lightning mesh file so that it goes with the
smaller lightning file. Have NEON use new use-cases for 2018 and 2018-PD conditions for CLM. Have NEON
Agricultural sites run with prognostic crop. Simple fix for warning about NaN's in import/export data from/to coupler.

Get NEON tests working on izumi, add --inputdata-dir to subset_data and modify_singlept_site_neon.py so they aren't tied
to only running on cheyenne.

Also update MOSART with fixed for direct_to_outlet option.

Add error checking in ParitionWoodFluxes. Fix value of albgrd_col in SurfaceAlbefdoType.F90.
Previously, the wrong value (albgri_col) was being set in InitHistory.

What's Changed

• Neon updates by @wwieder in #1872
• Fix albgrd_col value in SurfaceAlbedoType by @ka7eh in #1810
• Add error checking in ParitionWoodFluxes by @ka7eh in #1814

New Contributors

• @ka7eh made their first contribution in #1810

Full Changelog: ctsm5.1.dev113...ctsm5.1.dev114

Add SSP3-7.0 extension, and SMYLE 1850 and transient, compsets and use-cases

Update externals for cism and cime to cesm2.1.4-rc-10. cime update was needed to run on izumi.
iAdds a compset for the smyle L83 model (both 1850 and transient). BWsc1850smyle (jedwards4b)
Add SSP3-7.0 extension compset and use-case, that runs for fixed 2100 conditions with 2100 landuse
and 2100 harvest and fertilizer.

Update Nitrogen-deposition datasets with files created from WACCM simulations for SSP scenarios (SSP3-7.0,
SSP5-3.4, and SSP5-8.5).

Fix several issues.

Add LILAC

Add LILAC: The Lightweight Infrastructure for Land-Atmosphere Coupling. This
infrastructure consists of two major pieces:

(1) A lightweight coupling infrastructure built on top of ESMF that makes it easier for
atmosphere models to call CTSM directly, rather than using the hub-and-spoke architecture
that is used by CESM.

(2) A set of python-based tools for building CTSM and creating its runtime inputs when
running in an atmosphere model via LILAC. Although these tools are built on top of cime,
details of the create_newcase / case.setup / case.build process are hidden from the user,
because many of the aspects of this workflow don't make sense in the LILAC context.

So far we have used LILAC to couple CTSM to WRF. There are plans to use the same
infrastructure to couple CTSM to other regional atmosphere models.

Documentation of LILAC is provided in
https://escomp.github.io/ctsm-docs/versions/master/html/lilac/index.html (though there are
still some missing sections), as well as in various presentations on the wiki
(https://github.com/ESCOMP/CTSM/wiki/Presentations).

There have been many contributors to the development, testing and documentation of LILAC;
chief among them being Mariana Vertenstein, Negin Sobhani, Bill Sacks, Joe Hamman, Sam
Levis, Mike Barlage and Dave Lawrence.

Update documentation for release-clm5.0 branch, and fix issues with no-anthro surface dataset creation

Fix the issues with the no-anthro surface dataset creation (the PtVg simulation year). This
makes sure GDP and urban is set to zero and abm to missing for this case. Also bring in the python3
fix that was intended for the last release tag.

Turn irrigation on for 2300 SSP extensions

Irrigation was turned off for the 2300 extensions. This turns it on. Also add SSP2-4.5 to the
anomaly forcing script.

Extensions to 2300 for SSP5-8.5,SSP5-3.4, and SSP1-2.6

Add 2100 surface datasets, landuse.timeseries files for 2100-2300 for SSP5-3.4, SSP5-8.5, and SSP1-2.6.
Also add compsets to run them and the build-namelist use-cases for them. Only for f09 resolution with crop on.

Updates cime as a cime update was needed to run the extensions in I compsets (for presaero and CO2).
Add a test for it. Add -ignore_warnings to cmip6_waccm_deck user-mod which is required for the
fully coupled compset for BWSSP534osext as the ndep file doesn't exist for it, but WACCM is
providing ndep. Get more of the build-namelist unit tester tests working, which previously weren't running.