Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make x0050 default experiment for geos_atmosphere #501

Merged

Conversation

Dooruk
Copy link
Collaborator

@Dooruk Dooruk commented Feb 10, 2025

This PR combined with #478 makes x0050 as default for 3dvar_atmos and 3dfgat_atmos while increasing the horizontal resolution to c180.

Currently, x0050 observations were converted only for October 10th, 2023.

FYI @gmao-wgu @gmao-yzhu

@Dooruk Dooruk requested a review from rtodling February 10, 2025 15:46
@Dooruk
Copy link
Collaborator Author

Dooruk commented Feb 10, 2025

@rtodling 3dfgat doesn't have enough resources for C180. Would decreasing iteration be sufficient or should we modify npx_proc and npy_proc?

@Dooruk Dooruk changed the title Feature/aircraft bias lastmile Make x0050 default experiment for geos_atmosphere Feb 10, 2025
@metdyn
Copy link
Collaborator

metdyn commented Feb 10, 2025

This PR combined with #478 makes x0050 as default for 3dvar_atmos and 3dfgat_atmos while increasing the horizontal resolution to c180.

Currently, x0050 observations were converted only for October 10th, 2023.

Testing.. @rtodling
Isn't it faster to keep c90 in tier-1 run: asking Ricardo to rerun x50 with c90.

@Dooruk Dooruk marked this pull request as ready for review February 10, 2025 21:51
@Dooruk
Copy link
Collaborator Author

Dooruk commented Feb 10, 2025

@rtodling @metdyn please take another look.

ufo_testing fails because there is no geovals to compare against for x0050 but other tests pass.

@Dooruk
Copy link
Collaborator Author

Dooruk commented Feb 11, 2025

@rtodling there may be issues with the x0050-geovals you created, ufo_testing fails at airs_aqua:

Error: Variable skin_temperature_at_surface_where_sea not found in .. airs_aqua

@metdyn
Copy link
Collaborator

metdyn commented Feb 11, 2025

@Dooruk, about the x50_ufo, when I manually swell create, swell launch the ufo, I got both all nc4 and eva figures.
-rw-r--r-- 1 yyu11 s2839 120M Feb 10 19:45 x50_ufo.airs_aqua.20211211T210000Z.nc4
and the cylc did not show an error:

OOPS_STATS Run end - Runtime: 242.57 sec, Memory: total: 130.30 Gb, per task: min = 3.59 Gb, max = 3.82 Gb
Run: Finishing oops::Test running test::ObsFilters with status = 0
OOPS Ending 2025-02-11 00:46:26 (UTC+0000)

@Dooruk
Copy link
Collaborator Author

Dooruk commented Feb 11, 2025

@Dooruk, about the x50_ufo, when I manually swell create, swell launch the ufo, I got both all nc4 and eva figures. -rw-r--r-- 1 yyu11 s2839 120M Feb 10 19:45 x50_ufo.airs_aqua.20211211T210000Z.nc4 and the cylc did not show an error:

OOPS_STATS Run end - Runtime: 242.57 sec, Memory: total: 130.30 Gb, per task: min = 3.59 Gb, max = 3.82 Gb Run: Finishing oops::Test running test::ObsFilters with status = 0 OOPS Ending 2025-02-11 00:46:26 (UTC+0000)

With this branch? Weird.. Let me trigger the actions then

@metdyn
Copy link
Collaborator

metdyn commented Feb 11, 2025

I didnot use tier-1 test auto script, I manually swell create ufo; swell lanch; that is probably the difference from your triggered test flow.

With this branch? Weird.. Let me trigger the actions then

@metdyn
Copy link
Collaborator

metdyn commented Feb 11, 2025

I realized 3dvar-tier1.yaml contains only
model_components: ['geos_ocean']
Hence, I could not find the atmosphere directory. We can modify to
model_components: ['geos_atmosphere', 'geos_ocean']

@Dooruk
Copy link
Collaborator Author

Dooruk commented Feb 11, 2025

@metdyn it fails (I tested both locally and gmao_ci), see below:

https://github.com/GEOS-ESM/swell/actions/runs/13268623192

@Dooruk Dooruk merged commit fe3ad63 into feature/msienkie/aircraft_bias_NC4 Feb 11, 2025
13 of 15 checks passed
@Dooruk Dooruk deleted the feature/aircraft_bias_lastmile branch February 11, 2025 22:10
@metdyn
Copy link
Collaborator

metdyn commented Feb 11, 2025

Thanks. I reproduced your results:
Error: Variable skin_temperature_at_surface_where_sea not found in run/20231010T000000Z/geos_atmosphere/airs_aqua_geovals.20231009T210000Z.nc4

I find my previous run get the wrong results, because I source a wrong module that set the $PATH for swell to another branch.

@metdyn it fails (I tested both locally and gmao_ci), see below:

https://github.com/GEOS-ESM/swell/actions/runs/13268623192

Dooruk added a commit that referenced this pull request Feb 12, 2025
…e of nc4 files to use with updated bias correction (#478)

* new configuration with NC4 bias file for aircraft bias coefficients - modified aircraft.yaml
to use the new file, and modified gsi_bc_to_ioda.py to generate the NC4 coeff. file
Working on get_observations.py to get the NC4 aircraft bias coefficient file instead of csv

* change indentation of 'obs bias' section
(it's not a part of obs space)

* some modifications - not yet running

* Try 'satbias' type in get_observations.py (?)
additions for gsi_bc_to_ioda.py - use an aircraft 'sensor'

* remove old 'csv' processing that was left in the missing_obs_test file
change aircraftAscentRate to instantaneousAltitudeRate to conform with
recently changed nomenclature

* Added missing line for appending info on output to yaml
Fixed typo: add 'predictors' not 'default predictors' to output

* Needed to update the predictor coefficient name to match the new
naming convention.  Use 'instantaneousAltitudeRate' and
'instantaneousAltitudeRate_order_2'

* Remove reference to a non-existent file that was pasted into the bias section

* Corrections for python coding norms (indentation, long lines, too
many blank lines)

* Adjust indentation

* Add indentation on two lines for coding standard.  (Hope this works)

* try increase indent again

* tweak indentation again

* new aircraft bias format and extension

* fixes to provide _cov file separately for aircraft

* fix test

* reset location of alternative - bkg - files

* split aircraft T/W and adjust yamls  - including commenting filter not working at this time

* needs these too - sorry

* add temp and wind for aircraft but intentinally left aircraft alone

* eva seems to work now

* output files for aircraft varBC

* minor fix

* updates for handling 20231010 00z case

* Make x0050 default experiment for `geos_atmosphere` (#501)

* fix tier1 overrides

* aircraft is two separate entries now

* codestyle

* arrange defaults and overrides for tier1 tests

* make ufo_testing work for x0050

* deactivate ocean for hofx temporarily

* c91 as tier1 test, adapt to geos experiment folder style

* hofx has to use x0050 data

* add geovals for ufo_testing

---------

Co-authored-by: gmao-msienkie <[email protected]>
Co-authored-by: dooruk <[email protected]>
Co-authored-by: Doruk Ardağ <[email protected]>
Co-authored-by: Ricardo Todling <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants