Yannick announced that a marine code sprint is underway in Boulder. Then we collected updates from everyone.
Boulder
Guillaume announced that during the marine code sprint, the team will be converting obs from various tanks into the IODA format and then using these obs to perform DA cycling. The goal is to have prototype DA cycling (3DVar) working by the end of the code sprint. The obs types include SST, surface salinity, altimeters (5 or 6), GODAE profiles, and others. Some of the obs data sets are pre-qc'd such as the ones from the GMAO tank.
Xin is working on a variational bias correction framework. The work is broken up into two stages. Stage 1 is to read the bias parameters and predictors from a GSI file, stage 2 is to calculate the predictors online. When finished, only the parameters will be read in from a file. Xin will submit a pull request soon for the stage 1 piece of the project.
Ming has variational DA running using WRF. He needs to fix an issue with field input before continuing, and is using the MPAS interface as a guide to help with this task.
Chris H has the toy shallow water (SW) model running forecasts now. He is currently working on a MakeObs function so that he can generate obs values for assimilation. After some research, he decided to use the MakeObs scheme in UFO/IODA. Chris' next step will be to get interpolation (GetValues) working. The SW model uses a Cartesian grid (as opposed to Lat, Lon) which will add some work, but it was felt that this is a solvable problem. Chris is doing the work in a repo outside of the JCSDA github set so some discussion ensued about what is the best way to introduce the SW model work into the JCSDA repos. This hasn't been decided yet, but the group was leaning toward creating a new SW model repo in the JCSDA github set.
Hailing is working out the details of a 2D Locations object for the 2D GNSSRO obs operators. She is working with Anna to see if the 2D Locations object can be made generic. Hailing currently has 2 points working in 3DVar using FV3, and will try 4 points next.
Steve H reported that a pull request for the ObsSpace refactoring work has been submitted. He is resolving some issues that came up during the review and hopes to have this merged in within the next couple of days. The next step will be to build the IodaIO interfaces (Python, Fortran, C) which can be utilized by the ioda-converters scripts. Once this is done and the scripts ported over to using the IodaIO interfaces, then the underlying file that IodaIO uses can be modified transparently. For example, if we decide to go with ODB2 for our file format, then this can be switched in without the converter scripts needing any changes.
Dan is cleaning up the surface obs/processing in CRTM. Dan also is working with Benjamin to improve the use of BUMP in his flows.
Yannick announced that lots of merges (replace BOOST based tests with ECKIT based tests, 4D BUMP features, and others) have occurred over the past week. Don't hesitate to call the core team for help if you run into issues.
Mark has CharlieCloud working on Cheyenne for ufo-bundle, but he is running into issues with mounting the scratch disk for mpas- and fv3-bundles. He is working with CISL to help get this resolved. Benjamin had run into similar issues with mounting the scratch disk on S4 at the U. of Wisconsin.
Steve V has a IODA converter from MetOffice ATOV AMSU-A ODB2 files working. JJ is testing the files by attempting to assimilate the obs. They have found some differences between the GSI AMSU-A files and the MetOffice ATOV files.
- There are a lot more locations in the ATOV files. This is to be expected because the GSI files are dumped out after a DA run and GSI is thinning the obs.
- Channel 8 on the NOAA 19 (n19) satellite has questionable values and differences from the GSI data. Anna noted that channel 8 on n19 is noisy and others are dealing with this by simply not selecting channel 8 for assimilation. Yannick mentioned that removing channel 8 is a typical methodology, and mentioned that EMC does this. Anna added that GSI (as well as JEDI) has a mechanism for selecting channels. Steve V asked if we should devise a means for filtering out channels from the IODA file, and Yannick responded that we want to keep everything in IODA and then filter during the DA run.
- The MetOffice files have a system of QC flags consisting of 8 flags, but IODA has just one holder (the PreQC group). Discussion ensued with the points that every center handles this differently making it difficult to generalize, there is general agreement to make a zero value represent "good", and there is a general desire to have non-zero values represent the reasons why these obs values are not in the "good" category. Yannick added that we don't have this much sophistication yet in IODA, and if anyone wants to help develop a standard they are welcome to work on this.
JJ along with Junmei have written a set of plotting scripts for post-DA diagnostics. These scripts read the files that IODA writes at the end of the DA run. Examples include O-A and O-F stats in forms such as scatter plots and profiles. JJ showed a few sample plots, which were well received. Guillaume added that the marine groups has their own set of plots for similar purposes. It was discussed if these scripts (both from JJ and Guillaume) should be placed in the IODA repo or if a new repo should be created for diagnostic tools. No conclusion yet for this however. Yannick mentioned that sometime later this year, he is thinking of having a code sprint for developing diagnostic tools. For more information and plots see these two pull requests:
https://github.com/JCSDA/mpas/pull/78
https://github.com/JCSDA/mpas/pull/86
Maryam announced that the ECKIT based testing pull request has been merged into develop. All tests in OOPS, UFO, and IODA have been switched from BOOST to ECKIT. If you have a test that does not use the oops/src/test/interface include files, you will need to switch it from BOOST to ECKIT since we are moving to eliminate our dependence on boost libraries (we can still use boost header only packages). Maryam noted that if you have tests that you do switch to ECKIT, be sure to adjust any tolerance specs accordingly (see this github discussion for details: https://github.com/orgs/JCSDA/teams/jedi/discussions/22).
Yannick added that once BOOST testing is totally eliminated, that should enable us to remove the BOOST libraries.
Junmei is working with Hailing and JJ to be able to assimilate GNSSRO obs using MPAS. She has refractivity working and merged into develop, and a pull request will be submitted soon for bending angle. Chris S asked how one gets access to the ROPP library. It was mentioned that one person from each "lab" needs to obtain a license and then the entire lab can have access to the ROPP library. Junmei has a ROPP license so NCAR should be good to go.
Jim announced that the fix for the MPAS H(x) test failure has been merged into develop. The feature for parallel processing of profiles (OpenMP mods) is ready and Jim will submit a pull request soon.
Met Office
Marek and Steve S are working in the integration and testing of an RTTOV obs operator. They have run into an issue where if there are more process elements than obs (ie, some PEs get zero obs), JEDI sometimes crashes. Yannick acknowledged that this hasn't been addressed yet, and mentioned that the following two tests need to added: A test where one obs is in the input file during a run with multiple process elements, and another where DA is run with zero obs (which should complete and match the result from running a stand-alone forecast.
EMC
Andrew is updating ctests with the new obs types that they have introduced (e.g., sbuv, 2m temperature). Steve H added that it won't be necessary to add every new obs type to the IODA tests. Other parts of the system (obs operators, DA flows) will eventually test all of the obs types, and it is redundant to put all of the obs types in the IODA tests. Yannick added that we do however need to test all of the major obs types (Radiosonde, Radiance, etc.) in the IODA tests.
NRL
Sarah reported that they have switched over to using BUMP and testing is in progress.