(Notes from Steve V)
2021-09-09
Yannick started the meeting and did not have any general announcements.
We started with JEDI updates
JEDI1
JEDI 1 update from Mark
Maryam and Mark have been helping out with the JEDI-MPAS release. They are building new containers with an upgrade to gcc/gfortran 10.3 and AWS upgraded CodeBuild support.
Steve H is working on a few bugs/issues that have arisen in ioda and ioda-converters. He has PRs in for properly reading the netcdf/hdf fill values and deleting variables. There is also a PR & discussion on whether to build the python bindings by default in ioda-converters. These can be a bit sensitive to users' python environments, leading to test failures.
Steve is also working on the Fortran API for ioda-engines, refactoring some code from Ryan in the old ioda-engines repo. As part of this effort, we intend to start using the new jckit repo in bundles. jckit is intended for general-purpose utilities useful across repos. Steve is using it now for passing string vectors between Fortran and C++ but the plan is to move much of the utilities in the oops/src/util directory to jckit. Yannick added that one of the motivations behind jckit is to remove the oops dependency from repos like ioda.
Rick has a new set of modules on S4 based on the jedipara role account. There is a PR to jedi-docs with instructions on how to use them. Please give them a try. When we verify that they are working correctly for everyone, these will replace the current modules hosted by Mark M.
Rick has also built new modules for the jedipara role account that was recently implemented on Orion. There is not yet a PR for this because he's seeing some tests hanging for the intel modules and he's not sure why. Executing the tests individually seems to work fine. If anyone wants to try the new modules and/or has an idea on why the tests might be hanging, please let us know.
Ben J asked if the jedipara role account is there to replace the mmiesch modules. Yes, the idea is to eventually migrate everyone to the jedipara account and obsolete the mmiesch modules. The mmiesch modules will remain during this transition to facilitate coordination of the migration.
JEDI2
JEDI 2 update from Dan
Jedi 2.2 - Generalized locations (Francois, Emily)
• Understanding of the GSI FOV calculation
Jedi 2.3 - Use of NUOPC driver with FV3-JEDI (Mark Potts)
• Working on updating FMS on the UFS side to be compatible with the version of FMS now needed by JEDI. Making a PR in the GFDL_atmos_cubed_sphere repo to take only the changes needed by JEDI that aren’t part of the build system. Some issues still exist where UFS setup is looking for the wrong namelist file in the Geometry. Discussed with Mark Miesch creating a container with UFS tag and adding a new CI build with UFS. Could be useful for Academies and training too.
Jedi 2.7 - Background error model validation (Benjamin, Dan, Cathy, Ricardo)
• 3DVar static B benchmark is ready to run. All the scripts were put into fv3-jedi-tools. Produced analysis with 3DVar.
• Cathy has a standalone GSI setup on Orion.
• BJ found that the new Static B vertical balance is very similar to what we had before. Some differences in the diagnosed length scales. A little shorter than expected.
• Opened a new PR in Saber with bugfix to subsampling. There could be some small changes to the test results. Would like to merge tonight.
Jedi 2.8 - Regional DA (Ting, Ming)
• Continuing to work on the prescreening of observations. Adding checks to make sure observations are available and then modifying the yaml file on the fly. Using the identity observation operator. Issue with Id obs operator where the top of the atmosphere field is used instead of the lowest level.
Jedi 2.9 - MPAS general updates (JJ, BJ)
• Working on tutorials and documentation of the interface.
• BJ running extensive tests of new static B. Cycling tests done with new static B. Still need to do some refinement of parameters.
Jedi 2.12 - Neptune general updates (Sarah, Nancy)
• Revisiting how the hydrostatic DA system can provide non-hydrostatic. MPAS group looking at a similar issue. BJ trying to add hydrostatic balance. Analysis variable is surface pressure. After being analysed add hydrostatic balance before adding to the model. Has a big impact near the model top. Want to try an incremental form of the hydrostatic balance.
Jedi 2.15 - VADER (Steve V, Dan)
• PR in OOPS is nearly ready. Will turn attention to corresponding PR in UM-JEDI.
Jedi 2.18 - Localizations refactoring (Dan, Ryan)
• Localizations class refactored to use the ObsGroup instead of vectors.
JEDI3
JEDI 3 update from Anna
Yannick Tremolet issued a bugfix for zero-observation Variational runs: minimization is now exited when the initial gradient is too small: oops#1361.
Benjamin Menetrier issued a bugfix for empty B halo in BUMP saber#127.
Benjamin also issued an update to the atlas interface part of the Increment test oops#1366. Benjamin's PR for improving subsampling in BUMP for regional models with small domains was merged (saber#120)
Clementine Gas added serialize/deserialize implementations for ObsBiasIncrement: ufo#1495.
Anna Shlyaeva updated the ufo ObsError code to improve I/O for observation errors with cross-variable correlations, and created a simple converter from GSI covariance files to JEDI ioda-v2 files: ufo#1480 and ioda-converters#612.
There is an upcoming change in oops related to AnalyticInit refactoring, that will require minor changes in the model interfaces. Look for announcements on JEDI-models github page soon.
JEDI4
Brief JEDI 4 update from Yannick
Work is being done to ensure that R2D2 has all the backgrounds and observations required for the desired tests.
OBS3
OBS3 update from Ryan
Many people have been out on PTO, myself included, so minimal updates this week. The next OBS3 meeting is next Wednesday.
The IODA conventions are almost ready for release. We need advice from the core team regarding how the bias correction and ObsError-related groups should be named and described. Beyond that, reviewers have flagged a few outstanding comments in the convention tables. Once resolved, we plan to host the document and tables on Google Drive, but with public links on our ReadTheDocs site.
As Ben mentioned, the OBS team will meet to discuss PRs in UFO. UFO is a high volume repository - we average around 60 PRs per month. In the short term, we want to speed up reviews. We also want to make reviewer guidelines and distribute the workload so that we can better review both the science and coding practices. We need to ensure that UFO remains collaborative, with different agencies building off of each others' capabilities, and having us regularly review the pull request queue should help. As we set up these meetings, we plan to invite a progressively broader group. You'll hear more about it as it develops.
OBS2
OBS 2 update from Francois
-Rachel and Eric continue the work on re-writing the diagnostics scripts from the Notebooks into object oriented Python (#4).
-Rachel added utilities to generate plots from MPAS output for the Joint Testbed App framework
-Fabio is working on a linear forecast application needed for the forecast sensitivity calculations in FSOI.
-Hailing is working on fixing the ROPP ctest that recently broke (#266).
-Haling and Francois had a working meeting with Ben Ruston at NRL to help add the (Eumetsat-licensed) ROPP GNSSRO UFO in the Neptune bundle.
CRTM
CRTM update from Ben
Ben had a few topics which were briefly discussed:
The current JEDI implementation requires frequent “flipping” of the z orientation. After some discussion a decision was reached to make sure that the geovals are getting data passed in the correct orientation. A test should be added to ensure this. Then all “flipping” code can be removed from UFO. Then there was discussion of which orientation geovals should use. It was decided that geovals would use the same orientation as CRTM, since the models did not seem to have a majority with a favored orientation.
Also discussed using radiance as a separate obs type, in addition to brightness temperature. This will be explored.
Also discussed the JEDI transition to CRTM 2.4 and when that should occur. Probably in spring 2022, since that is when the change is happening operationally.
SOCA
SOCA update from Guillaume
- Work is being done to move data to R2D2 and the hope is to transition to using that soon.
- Hernan is making good progress on the JEDI interface to ROMS
- The SOCA team would like to get some help with configuring CI/CD
- Hamideh is woking on coupled GEOS, fixing issues
- Progress is being made with LETKF, GETKF development with EMC
Ming reported making progress with FV3 testing/verification using observations/backgrounds from the Dec 15, 2020 test set