GSI changes to enable the use of historical or reprocessed SATWND and other observations for Reanalysis

[jack-woollen]

Description

This PR mainly addresses an issue regarding reanalysis satellite wind data assimilation in the GSI. In particular, the read_satwnd subroutine has been extensively refactored for 1) to expand the gateway function of controlling which particular types of satwnd data can be processed to include all versions of historical satwnd data dating from 1979, and 2) to optimize operation of the routine by replacing several hundred lines of if tests with a table lookup function to more precisely specify satwnd objects with much less overhead. The read_satwnd refactoring is focusing on utilizing a lookup table which is a function of three variables, BUFR dumpfile subset, satellite id, and satwnd computation method, which replaces many if tests in the current code used to define QC and filtering functions of this subroutine with a simple single lookup based on data input. The table is created by one call to an initialization routine. Introducing this table also greatly simplify the process of updating read_satwnd parameters such as adding or modifying specific types of satwnd processing.

Several other subroutines are have small changes in this PR for the general purpose of enabling the GSI to process older and/or reprocessed versions of observations for reanalysis which are not currently present in operational DA processing.

This PR is designed to address some of the subjects discussed in issue #820, in general including various shortcomings in the GSI code regarding the ability to assimilate historical and reprocessed datasets for reanalysis purposes.

Resolves #820

Type of change

Please delete options that are not relevant.

• [x ] Bug fix (non-breaking change which fixes an issue)
• [x ] New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

These changes have been tested on Hercules and WCOSS2 to verify that the changes enable the effective use of the reanalysis datasets not usable by the current GSI code. These include testing the use of historical TOVS, ATOVS, SATWND, and reprocessed NetCDF ozone datasets created at NASA/GMAO, all of which have been designated for reanalysis. tests were also run to verify identical operation of current operations given the changes made in this PR. Examples of the datasets which are enabled for use by this PR are found in the NNJA AWS cloud based reanalysis database. Details are available by request. No special configuration is required once the GSI is amended by these changes.

Checklist

• [ x] My code follows the style guidelines of this project
• [ x] I have performed a self-review of my own code
• [ x] I have commented my code, particularly in hard-to-understand areas
• [ x] New and existing tests pass with my changes
• Any dependent changes have been merged and published

[RussTreadon-NOAA]

@jack-woollen : who would you like to invite to be peer reviewers of these changes?

[RussTreadon-NOAA]

Hercules CI
Install NOAA-EMC:develop at 92165a4 and jack-woollen:develop at 4f98338 on Hercules. Run ctests with the following results

Test project /work/noaa/da/rtreadon/git/gsi/pr837/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  484.73 sec
2/6 Test #6: global_enkf ......................   Passed  725.47 sec
3/6 Test #2: rtma .............................   Passed  964.90 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1094.47 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1158.18 sec
6/6 Test #1: global_4denvar ...................   Passed  1681.28 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1681.28 sec

[jack-woollen]

@BrettHoover-NOAA Can you review the read_satwnd changes proposed? Thanks.

[RussTreadon-NOAA]

Orion CI

Install NOAA-EMC:develop at 92165a4 and jack-woollen:develop at 4f98338 on Orion. Run ctests with the following results

Test project /work2/noaa/da/rtreadon/git/gsi/pr837/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #6: global_enkf ......................   Passed  728.53 sec
2/6 Test #3: rrfs_3denvar_rdasens .............   Passed  966.73 sec
3/6 Test #2: rtma .............................   Passed  1687.64 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  2785.32 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  3024.16 sec
6/6 Test #1: global_4denvar ...................   Passed  3602.33 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 3602.35 sec

[RussTreadon-NOAA]

WCOSS2 CI

Install NOAA-EMC:develop at 92165a4 and jack-woollen:develop at 4f98338 on Cactus. Run ctests with the following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/pr837/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  730.84 sec
2/6 Test #6: global_enkf ......................   Passed  855.68 sec
3/6 Test #2: rtma .............................   Passed  980.89 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1220.93 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1221.08 sec
6/6 Test #1: global_4denvar ...................   Passed  1745.20 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1745.25 sec

[BrettHoover-NOAA]

Everything looks good to me, thanks @jack-woollen!

[RussTreadon-NOAA]

Hera CI

Install NOAA-EMC:develop at 92165a4 and jack-woollen:develop at 98debf7 on Hera. Run ctests with the following results

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/gsi/pr837/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  492.34 sec
2/6 Test #6: global_enkf ......................   Passed  766.09 sec
3/6 Test #2: rtma .............................   Passed  1029.07 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1249.09 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1472.41 sec
6/6 Test #1: global_4denvar ...................   Passed  2045.15 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 2045.16 sec

[RussTreadon-NOAA]

@HaixiaLiu-NOAA , @emilyhcliu , & @ADCollard : @jack-woollen made minor changes to

• src/gsi/gsi_obOperTypeManager.F90 - add ompsnmnc and ompsnpnc
• src/gsi/radinfo.f90 - varx initialization
• src/gsi/read_bufrtovs.f90 - add bfr2bhdr(1) to logical test

Would you please look over these changes as peer reviewers?

Thank you!

[RussTreadon-NOAA]

Looks good. Only minor comments.

[RussTreadon-NOAA]

Thank you @jack-woollen for making these cosmetic changes. Looks good to me.

Approve.

[RussTreadon-NOAA]

@jack-woollen : This PR can move forward after we get at least one more peer review & approval.

@HaixiaLiu-NOAA , @emilyhcliu , @ADCollard - can you take a quick look at Jack's changes and, if acceptable, approve?