Yannick opened the meeting by mentioning his travels over the last few weeks, which include visits to NRL and the UK Met Office.
Maryam then presented some slides (below) on the new unit testing framework she implemented in JEDI by means of a current pull request on OOPS.
The new framework replace the BOOST test library with equivalent functionality in eckit. After this is implemented, building the JEDI environment will no longer require us to compile Boost, which can sometimes be a hassle on some systems and compilers. We will still use some boost functionality, so you may still notice some Boost items in the code, but these will only make use of the Boost header files, which can just be copied to an appropriate include directory without the need for compilation.
The new unit testing framework will require changes in all JEDI code repositories that include unit tests.
After this pull request is merged, OOPS will no longer recognize Boost, so if you do not make these changes, then your compilation will likely fail
The required code modifications are as follows (see slides for further details)
- Remove the BOOST keyword from all your ecbuild_add_test() function calls
These calls are located in the test/CMakeLists.txt files - Change the tolerances in your config (yaml/json) files
Tolerances for the BOOST tests are interpreted as percentages whereas tolerances in eckit are expressed as fractions. So, a tolerance of, say 1.e-6 in BOOST would correspond to a value of 1.e-8 in eckit. - Change your custom boost tests, if any
Most users/developers will not need to worry about this because most unit tests in, e.g. the model repos, are specific instantiations of the generic test interfaces in oops. However, if you do have any locally-defined Boost tests you will have to convert them to eckit. This is a straightforward replacement of the Boost headers and commands such as register_tests(), BOOST_CHECK_CLOSE(), etc. with corresponding eckit calls. See Maryam's slides for an example. She will post more examples on the JEDI Team page on GitHub.
After Maryam's presentation we went around the table in Boulder for updates.
Steve H is continuing to refactor the ObsSpace class in ioda. This should be ready soon (ufo-bundle tests are now passing) but while working on this he ran into several issues. First, he noticed that some of the netcdf files have the wrong data types. For example, the PreQC variables should be integers but they are sometimes rendered as doubles or floats in the netcdf files. Steve plans to implement a conversion where needed but when this is needed he will also arrange for a warning message to be printed out so we are aware that these files should be corrected. This ensures that tests and applications still work while also enabling us to update the files and make them consistent over time. A second issue Steve noticed had to do with similar data representation for the obsvector. Some calls to obsvector don't match the native data types (e.g. double data and integer QC). A similar strategy will be employed here, with a conversion and a warning message where needed, allowing us to identify and fix these problems over time.
Ming is continuing to work on the WRF-JEDI interface. He has been focusing on the 3DVar application and specifically the covariance code. He will then move on to the increment implementation.
Junmei has been working with Hailing and BJ on the GNSSRO interface for MPAS. They have the refractivity implemented and they plan to merge soon. Then they went on to test the bending angle and noticed some issues with the QC flags.
Hailing is running 3DVar with FV3. She's seeing problems with the ROPP operator that may be related to recent changes in FV3.
Steve V is updating his import script for ODB2 files. Now he makes use of a yaml config file to manage the conversion. He has it working for conventional observations. Next he will turn to ATOVS files.
JJ is working on assimilating CRTM radiance observations into MPAS, assessing their impact through O-B and O-A comparisons. When he moved to higher resolution he started running into problems using multiple cpus and nclouds > 0. For now he's setting nclouds=0 and has 3DVar partially working, though there is inconsistent behavior with high-res DA of amsua.
Mark has just been cleaning up and merging a few minor code changes required to get all ufo-bundle tests to pass with intel compilers on AWS and Cheyenne. All tests now pass for intel v17.0.1 in both Debug and Release mode. However, he's still seeing an internal compiler error when he used intel v 19.0 on AWS. He also had a tip when using intel compilers - they tend to make substantial use of the stack, especially at high levels of optimization. So, if you see test failures when you build in Release mode, try increasing the stack size on your system.
Jim is working on several bugs that have been reported for CRTM.
Xin continues to implement variational bias correction - now working on amsua. He has a prototype working and will work with Anna toward a goal of reproducing the GSI bias correction.
The floor then turned to the UK Met Office. Marek is working on creating an increment in C++ using an identity localization matrix. Yannick mentioned that Benjamin M is now here visiting Boulder, making changes to some 4D functionality in JEDI and encouraged Marek to talk to him. Marek said they have run 4Densvar. And, they have also run 3DVar with RTTOV. They are now planning to implement RTTOV into their 4DensVar and they do not expect any problems.
Chris H has the forecast application working for his toy shallow-water module, including the full parallel functionality and parallel io. His next step is to create a MakeObs application so he can assimilate observations. He was trying to decide whether to use the instantiation of MakeObs that is in ufo & ioda, as is done with other models such as fv3, or to create a customized implementation as is done for QG. After consulting with Anna, Steve, Yannick, and others he decided that he will exploit the existing functionality in ufo & ioda and adapt it to the toy model. Yannick mentioned that he will have to write a GetValues() routine to run MakeObs. This can be a simple bilinear interpolation, as is done in the QG model. He will work with the JEDI team to implement this.
The floor then turned to EMC.
Hui and Andrew reported that the mini-code sprint at EMC ended yesterday and was very productive. The main goal was to implement several GSI operators into JEDI and to train some new JEDI users. They tested these with MSF 
data.
Andrew asked a question about the future of ncdiag files. The current format is inconvenient for their analysis tools - is it worth the effort to re-organize them?
Yannick responded that it is probably not worth the effort. In the next few months we will be testing and evaluating ODB/ODC as a possible primary data format for JEDI. If this goes well, we will likely switch to this as a preferred file format.
Andrew was happy with that, adding that the ODB/ODC format works well with their analysis tools
Shastri has been working on a converter for satellite SST data in preparation for the marine code sprint. It's almost ready.
Then the floor turned to NRL: Sarah mentioned they are setting up their repos and training 2 new JEDI users/developers.
Then the floor turned to GMAO. Dan is working on implementing surface calculations into FV3 in a way that mimics GSI. He pointed out that the field of view calculation for surface variables requires the model to obtain information from the observation operators from ufo. And, he asked how we should handle this - should it be handled through the Locations class?
Yannick confirmed that the Locations class is where this should go and added that we need to work on a generic implementation of this.
Chris S then discussed continuing work from Yali on implementing surface pressure into MPAS DA. And, he asked how we can use JEDI to interpolate a 2D field - currently they are representing the 2D surface pressure as a 3D field and using bump to do the interpolation. Yannick asked if this was related to the field of view variables that Dan just mentioned. Chris agreed that it is related, but simpler. Marek then said that he is under the impression that that functionality is already in JEDI. He needs to check, but he thought that the LFRic group is already using bump to interpolate 2D fields. He and Chris agreed to discuss this offline.
Meeting adjourned