Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for C1152 ATM-Aero cycled DA to run on WCOSS2 #3309

Open
wants to merge 19 commits into
base: develop
Choose a base branch
from

Conversation

CoryMartin-NOAA
Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA commented Feb 7, 2025

Description

This PR enables the ability to cycle C1152 atm+aerosol on WCOSS2 to support GFSv17.
Wallclock times were extended because 1) model execution is now slower with GOCART (this should probably be configurable in the config.resources depending on APP but that is outside the scope of my PR) 2) the aerosol DA variational solver needs more time at this resolution and 3) it takes FOREVER to copy C1152 restart files with the GOCART aerosol species in them.

Also, the prep job runs out of memory so this spreads it out over more nodes. There are also misc. things that were added to get things to run on WCOSS2 that may need altered/removed post the most recent RFC.

Finally, GDAS hash is updated.

Resolves #3275

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES
    • EMC verif-global
    • GDAS already merged, updating to develop here
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

  • Clone and build on WCOSS
  • Cycled test on WCOSS

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

env/WCOSS2.env Outdated Show resolved Hide resolved
walltime="00:10:00"
walltime="00:30:00"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should walltime be CASE-specific?

Comment on lines +457 to +459
walltime="00:45:00"
ntasks=$(( layout_x * layout_y * 6 ))
threads_per_task=1
threads_per_task=5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, should threads and walltime be specific to the resolution?

ush/module-setup.sh Outdated Show resolved Hide resolved
@RussTreadon-NOAA
Copy link
Contributor

@CoryMartin-NOAA , when was the last time you ran build_gdas.sh on WCOSS2 for this PR?

Following RFC 13644 I found it necessary to add the line

export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/opt/cray/pe/mpich/8.1.19/ofi/intel/19.0/lib"

to sorc/gdas.cd/build.sh on WCOSS2.

I did so in a working copy of build.sh as

@@ -87,6 +87,10 @@ case ${BUILD_TARGET} in
     ;;
 esac
 
+if [[ $BUILD_TARGET == "wcoss2" ]]; then
+  export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/opt/cray/pe/mpich/8.1.19/ofi/intel/19.0/lib"
+fi
+
 CMAKE_OPTS+=" -DCLONE_JCSDADATA=$CLONE_JCSDADATA -DMACHINE=$BUILD_TARGET"

@CoryMartin-NOAA
Copy link
Contributor Author

@RussTreadon-NOAA it was Monday on Cactus. I should try now on Dogwood. Let me try it this afternoon.

@CoryMartin-NOAA
Copy link
Contributor Author

@CatherineThomas-NOAA I hope this is sufficient to meet the v17 deadline, and will address the comments "soon"

@RussTreadon-NOAA RussTreadon-NOAA added the JEDI Feature development to support JEDI-based DA label Feb 8, 2025
@RussTreadon-NOAA RussTreadon-NOAA added this to the GFS v17 milestone Feb 8, 2025
@RussTreadon-NOAA
Copy link
Contributor

Sorry, it appears g-w practice is to only assign issues to milestones.

@RussTreadon-NOAA RussTreadon-NOAA removed this from the GFS v17 milestone Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
JEDI Feature development to support JEDI-based DA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update defaults to support C1152 aerosol DA for GFSv17
5 participants