Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix marine vrfy job #829

Closed
CoryMartin-NOAA opened this issue Jan 3, 2024 · 6 comments · Fixed by #856
Closed

Fix marine vrfy job #829

CoryMartin-NOAA opened this issue Jan 3, 2024 · 6 comments · Fixed by #856
Assignees
@apchoiCMD
Labels
bug Something isn't working soca

Comments

@CoryMartin-NOAA
Copy link
Contributor

EVA yamls need to be updated, and probably $PYTHONPATH needs to be revised. See #828 for more information
@guillaumevernieres guillaumevernieres added bug Something isn't working soca labels Jan 5, 2024
@apchoiCMD
Copy link
Collaborator

Hera ctests

(gdasapp) [Mindo.Choi@hfe07 soca]$ ctest
Test project /scratch1/NCEPDEV/da/Mindo.Choi/workflow_01162024/global-workflow/sorc/gdas.cd/build/gdas/test/soca
      Start  1: test_gdasapp_convert_bufr_temp_dbuoy
 1/23 Test  #1: test_gdasapp_convert_bufr_temp_dbuoy ..................   Passed    3.18 sec
      Start  2: test_gdasapp_convert_bufr_salt_dbuoy
 2/23 Test  #2: test_gdasapp_convert_bufr_salt_dbuoy ..................   Passed    0.55 sec
      Start  3: test_gdasapp_convert_bufr_temp_mbuoyb
 3/23 Test  #3: test_gdasapp_convert_bufr_temp_mbuoyb .................   Passed    0.19 sec
      Start  4: test_gdasapp_convert_bufr_salt_mbuoyb
 4/23 Test  #4: test_gdasapp_convert_bufr_salt_mbuoyb .................   Passed    0.19 sec
      Start  5: test_gdasapp_convert_bufr_tesacprof
 5/23 Test  #5: test_gdasapp_convert_bufr_tesacprof ...................   Passed    0.21 sec
      Start  6: test_gdasapp_convert_bufr_trkobprof
 6/23 Test  #6: test_gdasapp_convert_bufr_trkobprof ...................   Passed    0.21 sec
      Start  7: test_gdasapp_convert_bufr_sfcships
 7/23 Test  #7: test_gdasapp_convert_bufr_sfcships ....................   Passed    0.20 sec
      Start  8: test_gdasapp_convert_bufr_sfcshipsu
 8/23 Test  #8: test_gdasapp_convert_bufr_sfcshipsu ...................   Passed    0.19 sec
      Start  9: test_gdasapp_soca_nsst_increment_to_mom6
 9/23 Test  #9: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed    8.82 sec
      Start 10: test_gdasapp_soca_prep
10/23 Test #10: test_gdasapp_soca_prep ................................   Passed    3.74 sec
      Start 11: test_gdasapp_soca_run_clean
11/23 Test #11: test_gdasapp_soca_run_clean ...........................   Passed    1.06 sec
      Start 12: test_gdasapp_soca_setup_obsproc
12/23 Test #12: test_gdasapp_soca_setup_obsproc .......................   Passed    0.65 sec
      Start 13: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
13/23 Test #13: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed  108.01 sec
      Start 14: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
14/23 Test #14: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed   44.69 sec
      Start 15: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
15/23 Test #15: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed   74.49 sec
      Start 16: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
16/23 Test #16: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed  106.40 sec
      Start 17: test_gdasapp_soca_copy_scratch
17/23 Test #17: test_gdasapp_soca_copy_scratch ........................   Passed    0.98 sec
      Start 18: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
18/23 Test #18: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed   42.41 sec
      Start 19: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
19/23 Test #19: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ....   Passed   11.26 sec
      Start 20: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY
20/23 Test #20: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY ....   Passed  333.53 sec
      Start 21: test_gdasapp_soca_socahybridweights
21/23 Test #21: test_gdasapp_soca_socahybridweights ...................   Passed  108.01 sec
      Start 22: test_gdasapp_soca_incr_handler
22/23 Test #22: test_gdasapp_soca_incr_handler ........................   Passed   42.60 sec
      Start 23: test_gdasapp_soca_ens_handler
23/23 Test #23: test_gdasapp_soca_ens_handler .........................   Passed   75.48 sec

100% tests passed, 0 tests failed out of 23

Total Test time (real) = 969.14 sec

@ShastriPaturi
Copy link
Collaborator

ShastriPaturi commented Jan 17, 2024 •
edited
Loading

vrfy ctest fails on orion.

INFO EvaSetup: Running figure driver
Traceback (most recent call last):
File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/eva/bin/eva", line 8, in
sys.exit(main())
File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/eva/lib/python3.9/site-packages/eva/eva_driver.py", line 120, in main
eva(config_file)
File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/eva/lib/python3.9/site-packages/eva/eva_driver.py", line 85, in eva
figure_driver(eva_dict, data_collections, timing, logger)
File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/eva/lib/python3.9/site-packages/eva/plotting/batch/base/plot_tools/figure_driver.py", line 43, in figure_driver
graphics = graphics_section.get('figure_list')
AttributeError: 'list' object has no attribute 'get'
running eva on evayamls/eva_sst_metopb_l3u_2018041512.yaml
Traceback (most recent call last):
File "/work2/noaa/da/spaturi/sandbox/CDA_testbed/20240116/global-workflow/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_vrfy.py", line 230, in
subprocess.run(['eva', infile], check=True)
File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/eva/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['eva', 'evayamls/eva_sst_metopb_l3u_2018041512.yaml']' returned non-zero exit status 1.

  • JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY[1]: postamble JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY 1705509773 1

@apchoiCMD
Copy link
Collaborator

Traceback (most recent call last):

I am going to test it on Orion once build is successful. Thanks for the information.

@guillaumevernieres
Copy link
Contributor

Thanks for the update @apchoiCMD . You can push a draft pr if you want us to start reviewing your changes.

@apchoiCMD
Copy link
Collaborator

Thanks for the update @apchoiCMD . You can push a draft pr if you want us to start reviewing your changes.

Oh, good idea and I will push it soon.

@apchoiCMD apchoiCMD mentioned this issue Jan 17, 2024
Merged
@apchoiCMD
Copy link
Collaborator

Orion ctests

(gdasapp) Orion-login-3[87] mindoc$ ctest
Test project /work2/noaa/da/mchoi3/workflow_01162024/global-workflow/sorc/gdas.cd/build/gdas/test/soca
      Start  1: test_gdasapp_convert_bufr_temp_dbuoy
 1/23 Test  #1: test_gdasapp_convert_bufr_temp_dbuoy ..................   Passed    0.27 sec
      Start  2: test_gdasapp_convert_bufr_salt_dbuoy
 2/23 Test  #2: test_gdasapp_convert_bufr_salt_dbuoy ..................   Passed    0.28 sec
      Start  3: test_gdasapp_convert_bufr_temp_mbuoyb
 3/23 Test  #3: test_gdasapp_convert_bufr_temp_mbuoyb .................   Passed    0.27 sec
      Start  4: test_gdasapp_convert_bufr_salt_mbuoyb
 4/23 Test  #4: test_gdasapp_convert_bufr_salt_mbuoyb .................   Passed    0.26 sec
      Start  5: test_gdasapp_convert_bufr_tesacprof
 5/23 Test  #5: test_gdasapp_convert_bufr_tesacprof ...................   Passed    0.29 sec
      Start  6: test_gdasapp_convert_bufr_trkobprof
 6/23 Test  #6: test_gdasapp_convert_bufr_trkobprof ...................   Passed    0.27 sec
      Start  7: test_gdasapp_convert_bufr_sfcships
 7/23 Test  #7: test_gdasapp_convert_bufr_sfcships ....................   Passed    0.29 sec
      Start  8: test_gdasapp_convert_bufr_sfcshipsu
 8/23 Test  #8: test_gdasapp_convert_bufr_sfcshipsu ...................   Passed    0.26 sec
      Start  9: test_gdasapp_soca_nsst_increment_to_mom6
 9/23 Test  #9: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed   32.51 sec
      Start 10: test_gdasapp_soca_prep
10/23 Test #10: test_gdasapp_soca_prep ................................   Passed    5.24 sec
      Start 11: test_gdasapp_soca_run_clean
11/23 Test #11: test_gdasapp_soca_run_clean ...........................   Passed    0.03 sec
      Start 12: test_gdasapp_soca_setup_obsproc
12/23 Test #12: test_gdasapp_soca_setup_obsproc .......................   Passed    0.81 sec
      Start 13: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
13/23 Test #13: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed  267.76 sec
      Start 14: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
14/23 Test #14: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed  362.72 sec
      Start 15: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
15/23 Test #15: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed  396.04 sec
      Start 16: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
16/23 Test #16: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed  298.49 sec
      Start 17: test_gdasapp_soca_copy_scratch
17/23 Test #17: test_gdasapp_soca_copy_scratch ........................   Passed    1.20 sec
      Start 18: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
18/23 Test #18: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed  684.37 sec
      Start 19: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
19/23 Test #19: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ....   Passed  268.18 sec
      Start 20: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY
20/23 Test #20: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY ....   Passed  682.90 sec
      Start 21: test_gdasapp_soca_socahybridweights
21/23 Test #21: test_gdasapp_soca_socahybridweights ...................   Passed  298.45 sec
      Start 22: test_gdasapp_soca_incr_handler
22/23 Test #22: test_gdasapp_soca_incr_handler ........................   Passed  301.11 sec
      Start 23: test_gdasapp_soca_ens_handler
23/23 Test #23: test_gdasapp_soca_ens_handler .........................   Passed  333.97 sec

100% tests passed, 0 tests failed out of 23

guillaumevernieres pushed a commit that referenced this issue Jan 17, 2024
@apchoiCMD
Fix Marine VRFY Task in the SOCA ctests (#856)
f63fe7e 
#### What this PR includes:

- Fix test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY

by updating `marine_gdas_plots.yaml` format and it related
`marine_eva_post.py`

- <del>As of now, ctests passed on `Hera Machine` and `Orion Machine` is
still needed to check</del>
- ctests passed on both `Hera Machine` and `Orion Machine`, see
#829

Fixes #829

Thanks to @CoryMartin-NOAA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@apchoiCMD apchoiCMD

Labels
bug Something isn't working soca
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Marine VRFY Task in the SOCA ctests NOAA-EMC/GDASApp
5 participants
@CoryMartin-NOAA @guillaumevernieres @ShastriPaturi @AndrewEichmann-NOAA @apchoiCMD