tagged Release/2021.5.0 (#78)

* remove Model class, no longer needed (#1)

* remove makeobs (#2)

* remove makeobs.x, no longer needed

* remove Model class, no longer needed (#1)

* Implement Geometry class (#3)

* ctest passed for Geometry class

* Modified test CMakeLists.txt

* Travis's suggested changes completed by Ligang

* Travis's suggested changes completed by Ligang

* add dummy serializeable interfaces (#5)

* add serializable interfaces to State (#8)

* replace boost::shared_ptr with std::shared_ptr (#9)

* change jcsda repo locations (#10)

* Implement state class (#11)

* Temporary implememting class State for Travis to check.

* Updated State.cc

* Updating class State

* Updating class State.

* Updating class State.

* Updating class State.

* Finished class State, passed all test.

* Passed all test for class State.

* fix binary path

Co-authored-by: Travis Sluka <[email protected]>

* Finished class Increment.  (#17)

* Finished class Increment.

* Finished class increment.

* Updated increment.yml for class increment.

* Updated for class Increment.

* add test observations (#22)

Merged.

* Changed float to double for JEDI, read/write netCDF with float for consistency. (#24)

* Create Fields base class for Increment/State (#25)

* add MPI support (#26)

* add missing ModelAux stubs (#27)

* Feature/rename (#28)

* rename to umdsst

* switch to public jedi repo locations

* fix travisci

* Implement GetValues (#29)

* start GetValues

* add level(1) to fields

* getvalues working?

* Merge class LinearGetValues (#31)

* Implemented class LinearGetValues, passed all tests.

* minor change of a comment.

* add hofx_nomodel test (#32)

* add hofx_nomodel test

* remove use of external OOPS_TRAPFPE env var, deal with that later

* Finished bugfix/ConertToCelsius. (#41)

* Finished feature class Covariance. (#42)

* fix print in getvalues/lineargetvalues (#45)

* Finished Feature/dirac.x (#43)

* Implementing dirac.x.

* Finished feature dirac.x.

* Finishing feature/dirac.x.

Co-authored-by: Travis Sluka <[email protected]>

* Fixed conversion between Celsius and Kelvin. (#46)

* Fixed conversion between Celsius and Kelvin.

* Finished bugfix/C_K_conversion.

* update for ufo::locations change (#47)

* Finished Feature/staticbinit.x (#48)

* Implementing staticbinit.x

* Finished feature/staticbinit.x

* Finished feature/staticbinit.x 2nd Time.

* keep up with JEDI (#49)

* Finished Feature/var.x (#50)

* Implementing feature/var.x.

* Implementing feature/var.x.

* Implementing var.x

* Finished feature/var.x.

* Finished feature/var.x.

* Feature/add mask field (#51)

* added landmask.nc

* Finished feature/addMaskField.

* Finished feature/addMaskField.

* Finished feature/addMaskField.

* Remove hardcoded 180 and 360.

* Finished feature/useLandmask. (#55)

* Finished feature/useLandmask.

* Finished feature/useLandmask.

* Finished feature/useLandmask.

* add eckit::Configuration to LinearGetValues constructor (#59)

* Add support for horizontally varying correlation lengths (#58)

* clean old bump files on test run

* manually specify correlation lengths

* fix coding norms

* fix dependencies of errorcovariance test

* add StdDev variable change (#60)

* feature/addQC finished (#62)

* Finished AddQC, Solved problems with passing vector of pointers of FieldImpl from C to Fortran.

* Finishing feature/addQC.

* Feature/better landmask (#63)

* Test improved landmask.

* Finished feature/betterLandmask.

* Finished feature/betterLandmask.

* update for oops hofx3d (#64)

* Misc updates (#65)

* convert corr length from gaussian to GC

* add "default" var change

* update reference answers

* fix docker on travisci (#66)

* remove GeneralizedDepartures (#67)

Approved.

* Bugfix/grid issues (#68)

* invert lat on netcdf I/O

* landmask is explicitly applied to background

* flood  the test background file

* fix landmask/grid in yamls

* update reference answers

* add cycling script (#69)

* Create Model2GeoVaLs class (#70)

* fields working for any number of fields

* implement Model2GeoVaLs

* remove ggmask from Geometry

* cleanup

* Rossby radius based horizontal correlation lengths (#73)

* rossby radius based corr loc, using dummy RR values

* kd interpolation in geometry

* using rossby radius in covariance

* fix tests and coding norms

* document interptogeom

* add .dat to lfs

* fix compile error

* support for modules with new ecbuild (#76)

* support for module with new ecbuild

* use public atlas/fckit

* Feature/experiment - Added some initial background of 0.25x0.25 grid. (#77)

* Added some 0.25x0.25 files.

* Added new initial background for 20120101.

* Updated for some initial background of 0.25x025 grid.

* Updated initial background of 0.25x0.25 grid.

* Updated yml files with correct renamed file names.

* Updated to have correct file names.

* update most recent stable repo tags

Co-authored-by: Logan (Ligang) Chen <[email protected]>

.gitattributes

@@ -1,2 +1,4 @@
+*.dat   filter=lfs diff=lfs merge=lfs -text
 *.nc    filter=lfs diff=lfs merge=lfs -text
-*.nc4   filter=lfs diff=lfs merge=lfs -text
+*.nc4   filter=lfs diff=lfs merge=lfs -text
+*.dat filter=lfs diff=lfs merge=lfs -text

.github/Dockerfile

@@ -1,17 +1,14 @@
-
 FROM  jcsda/docker-gnu-openmpi-dev:latest
 
-RUN groupadd jcsda -g 9999 \
-    && adduser jcsdauser \
-    && mkdir -p /jcsda \
-    && chown -R jcsdauser:jcsda /jcsda \
+RUN  mkdir -p /jcsda \
+    && chown -R jedi:jedi /jcsda \
     && chmod 6755 /jcsda
 
-USER jcsdauser
+USER jedi
 
 COPY default-mca-params.conf /jcsda/mca-params.conf
 
 RUN mkdir -p ~/.openmpi \
     && cp /jcsda/mca-params.conf ~/.openmpi/
 
-CMD ["/bin/bash"]
+CMD ["/bin/bash"]

.travis.yml

@@ -30,10 +30,15 @@ before_install:
 
 script:
 - |
-  # build/test code
-  docker exec jcsda_container bash \
-   -c 'cd /jcsda/build && ecbuild /jcsda/src/bundle && cd oisst && make -j 4'
-  docker exec jcsda_container bash \
-   -c 'cd /jcsda/build/oisst && ctest'
-  
-after_success: 
+  # build code
+  docker exec -it jcsda_container bash \
+   -c 'cd /jcsda/build && ecbuild -DBUILD_FCKIT=ON -DBUILD_ATLAS=ON /jcsda/src/bundle && cd umdsst && make -j 4; exit $?' \
+   || travis_terminate 1
+
+- |
+  # test code
+  docker exec -it jcsda_container bash \
+   -c 'cd /jcsda/build/umdsst && ctest --output-on-failure' \
+   || travis_terminate 1
+
+after_success:

CMakeLists.txt

@@ -4,122 +4,46 @@
 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.
 
 ################################################################################
-# OISST
+# UMDSST
 ################################################################################
+cmake_minimum_required( VERSION 3.12 )
+project( umdsst VERSION 2021.5.0 LANGUAGES C CXX Fortran)
 
-cmake_minimum_required( VERSION 3.8 FATAL_ERROR )
-
-project( oisst C CXX Fortran)
-
-set( CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake;${CMAKE_MODULE_PATH})
-
-set( ENABLE_OS_TESTS           OFF CACHE BOOL "Disable OS tests" FORCE )
-set( ENABLE_LARGE_FILE_SUPPORT OFF CACHE BOOL "Disable testing of large file support" FORCE )
-set( ENABLE_MPI                ON CACHE BOOL "Compile with MPI" )
-
+find_package(ecbuild 3.3.2 QUIET)
 include( ecbuild_system NO_POLICY_SCOPE )
-
-ecbuild_requires_macro_version( 2.5 )
-
-
-################################################################################
-# Project
-################################################################################
-
 ecbuild_declare_project()
 
-ecbuild_enable_fortran( REQUIRED )
-ecbuild_add_cxx11_flags()
-
-set( OISST_LINKER_LANGUAGE CXX )
+list( APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake)
+include( umdsst_compiler_flags )
 
 
 ################################################################################
 # Dependencies
 ################################################################################
-# Boost Headers
-set( Boost_MINIMUM_VERSION "1.47" )
-include_directories( ${Boost_INCLUDE_DIR} )
-
-# NetCDF
 find_package( NetCDF REQUIRED COMPONENTS Fortran )
-include_directories( ${NETCDF_INCLUDE_DIRS} )
-
-# eckit
-ecbuild_use_package( PROJECT eckit VERSION 1.1.0 REQUIRED )
-include_directories( ${ECKIT_INCLUDE_DIRS} )
-link_libraries( ${ECKIT_LIBRARIES} )
-
-# fckit
-ecbuild_use_package( PROJECT fckit VERSION 0.4.1 REQUIRED )
-include_directories( ${FCKIT_INCLUDE_DIRS} )
-
-# atlas
-ecbuild_use_package( PROJECT atlas VERSION 0.18.1 REQUIRED )
-include_directories( ${ATLAS_INCLUDE_DIRS} )
-
-# oops
-ecbuild_use_package( PROJECT oops VERSION 0.1.0 REQUIRED )
-include_directories( ${OOPS_INCLUDE_DIRS} )
-add_definitions(-DATLASIFIED=1)
-
-# saber
-ecbuild_use_package( PROJECT saber VERSION 0.0.1 REQUIRED )
-include_directories( ${SABER_INCLUDE_DIRS} )
-
-# ioda
-ecbuild_use_package( PROJECT ioda VERSION 0.1.0 REQUIRED )
-include_directories( ${IODA_INCLUDE_DIRS} )
-
-# ufo
-ecbuild_use_package( PROJECT ufo VERSION 0.1.0 REQUIRED )
-include_directories( ${UFO_INCLUDE_DIRS} )
-
-# MPI
-ecbuild_find_mpi( COMPONENTS CXX Fortran REQUIRED )
-ecbuild_include_mpi()
-link_libraries(${MPI_CXX_LIBRARIES} ${MPI_Fortran_LIBRARIES})
-
-
-################################################################################
-# Export package info
-################################################################################
-list( APPEND OISST_TPLS atlas atlas_f LAPACK MPI NetCDF )
-
-set( OISST_INCLUDE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/src
-                       ${CMAKE_CURRENT_BINARY_DIR}/src
-                       ${CMAKE_Fortran_MODULE_DIRECTORY} )
-set( OISST_LIBRARIES oisst )
-
-get_directory_property( OISST_DEFINITIONS COMPILE_DEFINITIONS )
-
-foreach( _tpl ${OISST_TPLS} )
-string( TOUPPER ${_tpl} TPL )
-list( APPEND OISST_EXTRA_DEFINITIONS   ${${TPL}_DEFINITIONS}  ${${TPL}_TPL_DEFINITIONS}  )
-list( APPEND OISST_EXTRA_INCLUDE_DIRS  ${${TPL}_INCLUDE_DIRS} ${${TPL}_TPL_INCLUDE_DIRS} )
-list( APPEND OISST_EXTRA_LIBRARIES     ${${TPL}_LIBRARIES}    ${${TPL}_TPL_LIBRARIES}    )
-endforeach()
+find_package( eckit  1.11.6   REQUIRED)
+find_package( fckit  0.7.0    REQUIRED)
+find_package( atlas  0.20.2   REQUIRED)
+find_package( oops   1.0.0    REQUIRED)
+find_package( saber  1.0.0    REQUIRED)
+find_package( ioda   1.0.0    REQUIRED)
+find_package( ufo    1.0.0    REQUIRED)
 
 
 ################################################################################
 # Sources
 ################################################################################
-include( oisst_compiler_flags )
-include_directories( ${OISST_INCLUDE_DIRS} ${OISST_EXTRA_INCLUDE_DIRS} )
-
+set( UMDSST_LINKER_LANGUAGE CXX )
 add_subdirectory( src )
 add_subdirectory( test )
 
-if(ECBUILD_INSTALL_FORTRAN_MODULES)
-  install(DIRECTORY ${CMAKE_Fortran_MODULE_DIRECTORY}/${CMAKE_CFG_INTDIR} DESTINATION ${INSTALL_INCLUDE_DIR} )
-endif()
 
 ################################################################################
 # Finalise configuration
 ################################################################################
 
 # prepares a tar.gz of the sources and/or binaries
-ecbuild_install_project( NAME oisst )
+ecbuild_install_project( NAME umdsst )
 
 # print the summary of the configuration
 ecbuild_print_summary()

README.md

@@ -1,23 +1,21 @@
-[![Build Status](https://travis-ci.org/UMD-AOSC/OISSTv3.svg?branch=develop)](https://travis-ci.org/UMD-AOSC/OISSTv3)
+[![Build Status](https://travis-ci.org/UMD-AOSC/UMD-SST.svg?branch=develop)](https://travis-ci.org/UMD-AOSC/UMD-SST)
 
-# OISSTv3
+# UMD-SST
 
-Prototype for the Optimal Interpolation Sea Surface Temperature v3 (OISSTv3)
+Testbed sea surface temperature analysis system.
 
-A variational data assmilation system built on the JCSDA Joint Effort for Data Assimilation Integration (JEDI). This project is just beginning, and further documentation will be created as the project proresses.
+A variational data assmilation system built on the JCSDA Joint Effort for Data Assimilation Integration (JEDI). This project is just beginning, and further documentation will be created as the project progresses.
 
 ## Getting Started
 The following steps assume you already have one of the JEDI containers setup and working on your computer. See the [JEDI Documentation](https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/) for additional details on how to do this.
 
-> ***NOTE: until JEDI is publicly released, anyone trying to build OISSTv3 must have access to the private JEDI repositories!***
-
-To build OISSTv3 and run the continuous integration tests:
+To build UMD-SST and run the continuous integration tests:
 ```
-git clone https://github.com/UMD-AOSC/OISSTv3.git oisst.src
-mkdir -p oisst.build
-cd oisst.build
-ecbuild ../oisst.src/bundle
-cd oisst
+git clone https://github.com/UMD-AOSC/UMD-SST.git umdsst.src
+mkdir -p umdsst.build
+cd umdsst.build
+ecbuild ../umdsst.src/bundle
+cd umdsst
 make -j <nprocs>
 ctest
 ```

bundle/.gitignore

@@ -6,4 +6,4 @@ ioda/
 oops/
 saber/
 ufo/
-oisst
+umdsst

bundle/CMakeLists.txt

@@ -4,34 +4,47 @@
 # WHICH CAN BE OBTAINED AT HTTP://WWW.APACHE.ORG/LICENSES/LICENSE-2.0.
 
 #
-# OISST-BUNDLE
+# UMDSST-BUNDLE
 #
 
-project( OISST-bundle C CXX Fortran )
+cmake_minimum_required( VERSION 3.12 )
+project( umdsst-bundle VERSION 2021.5.0 LANGUAGES C CXX Fortran )
 
-cmake_minimum_required( VERSION 3.8 FATAL_ERROR )
+find_package( ecbuild 3.3.2 QUIET)
 include( ecbuild_bundle )
-
-set( ECBUILD_DEFAULT_BUILD_TYPE RelWithDebInfo )
-set( ENABLE_MPI ON CACHE BOOL "Compile with MPI" )
-
 ecbuild_bundle_initialize()
-ecbuild_requires_macro_version( 2.7 )
 
-# Optional repositories
-option( BUILD_ECKIT "download and build eckit (not needed if in a jedi container)" )
+
+#===================================================================================================
+# ECMWF repos that you probably dont need to build yourself because they should be in the
+# jedi-stack and containers
+#===================================================================================================
+option(BUILD_ECKIT "download and build eckit (not needed if in a jedi container)")
 if ( BUILD_ECKIT )
-  ecbuild_bundle( PROJECT eckit         GIT "https://github.com/JCSDA/eckit.git"           UPDATE BRANCH release-stable )
+  ecbuild_bundle( PROJECT eckit  GIT "https://github.com/ecmwf/eckit.git"  UPDATE TAG b3adb28 )
 endif ()
 
+option(BUILD_FCKIT "download and build fckit (not needed if in a jedi container)")
+if (BUILD_FCKIT)
+  ecbuild_bundle( PROJECT fckit  GIT "https://github.com/jcsda/fckit.git"  UPDATE TAG 8502aae )
+  # TODO: replace with https://github.com/ecmwf/fckit.git TAG 0.9.2
+endif()
+
+option(BUILD_ATLAS "download and build atlas (not needed if in a jedi container)")
+if (BUILD_ATLAS)
+  ecbuild_bundle( PROJECT atlas  GIT "https://github.com/jcsda/atlas.git"  UPDATE TAG e0b7709 )
+  # TODO: repalce with https://github.com/ecmwf/atlas.git TAG 0.24.1
+endif()
+
+
+#===================================================================================================
 # required repositories
-ecbuild_bundle( PROJECT fckit           GIT "https://github.com/JCSDA/fckit.git"           UPDATE BRANCH release-stable )
-ecbuild_bundle( PROJECT atlas           GIT "https://github.com/JCSDA/atlas.git"           UPDATE BRANCH release-stable )
-ecbuild_bundle( PROJECT oops            GIT "https://github.com/JCSDA/oops.git"            UPDATE BRANCH develop )
-ecbuild_bundle( PROJECT saber           GIT "https://github.com/JCSDA/saber.git"           UPDATE BRANCH develop )
-ecbuild_bundle( PROJECT ioda            GIT "https://github.com/JCSDA/ioda.git"            UPDATE BRANCH develop )
-ecbuild_bundle( PROJECT ioda-converters GIT "https://github.com/JCSDA/ioda-converters.git" UPDATE BRANCH develop )
-ecbuild_bundle( PROJECT ufo             GIT "https://github.com/JCSDA/ufo.git"             UPDATE BRANCH develop )
-ecbuild_bundle( PROJECT oisst         SOURCE ../)
+#===================================================================================================
+ecbuild_bundle( PROJECT oops            GIT "https://github.com/jcsda/oops.git"            UPDATE TAG 8f517dd )
+ecbuild_bundle( PROJECT saber           GIT "https://github.com/jcsda/saber.git"           UPDATE TAG 0422a16 )
+ecbuild_bundle( PROJECT ioda            GIT "https://github.com/jcsda/ioda.git"            UPDATE TAG 04880ef )
+ecbuild_bundle( PROJECT ioda-converters GIT "https://github.com/jcsda/ioda-converters.git" UPDATE TAG ab0a195 )
+ecbuild_bundle( PROJECT ufo             GIT "https://github.com/jcsda/ufo.git"             UPDATE TAG eb2e70f )
+ecbuild_bundle( PROJECT umdsst          SOURCE ../)
 
 ecbuild_bundle_finalize()

cycling/config/staticbinit.yaml

@@ -0,0 +1,40 @@
+geometry:
+  grid:
+    name: S360x180
+    domain:
+      type: global
+      west: -180
+  landmask:
+    filename: landmask.nc
+  rossby radius file: rossby_radius.dat
+
+analysis variables: &vars [sea_surface_temperature]
+
+background:
+  state variables: *vars
+  date: 2018-04-15T00:00:00Z # date doesn't matter here, just pick something
+  filename: bkg.nc
+
+background error:
+  covariance model: umdsstCovar
+  bump:
+    verbosity: main
+    datadir: bump
+    prefix: bump_sst
+    method: cor
+    strategy: specific_univariate
+    mask_check: 1
+    network: 1
+    new_nicas: 1
+    ntry: 3
+    resol: 6.0
+    mpicom: 2
+    nc1max: 100000
+
+  correlation lengths:
+    base value: 0.0
+    rossby mult: 1.0
+    min grid mult: 1.5
+    #min value: 25.0e3
+    #max value:
+