Yannick opened the meeting by announcing that JJ and Junmei have started a collection of diagnostic scripts for plotting results from DA runs. He then turned it over to JJ to present more details about this effort.
JJ showed us the following slides
Highlights from the presentation are:
Some discussion ensued about the generality of the scripts. The file, directory and variable naming conventions are specific to mpas-bundle so this will need to be addressed. In particular, if you use these scripts, make sure you modify these items as needed: line 23 in slide 4 and lines 40-50 in slide 5. Note also the plotting group definitions in lines 29-30 of slide 4. However, the script that shows the development of the cost function during the DA run is already generic.
Everyone is encouraged to contribute to the DAdiagnostics repository by either putting in new capabilities or helping with making the existing script more generic.
For now this DAdiagnostics repository will be managed collectively. In the .future, we may designate someone to oversee its development. And, we plan to discuss the future development of diagnostics at future meetings and potentially code sprints.
A code sprint in College Park is underway this week, and Andrew updated us on this effort. There are about a dozen people from EMC, GMAO and the JEDI core team participating with the goal of improving and expanding the UFO QC capability. Lots of progress has been made with QC checks for radiance and conventional obs data, and with the identification of how to organize the code to facilitate future QC efforts.
Mark attended the UCAR Software Engineering Assembly (SEA) on Improving Scientific Software in Boulder last week. (Maryam and Steve H also attended.) The SEA is an annual event that UCAR sponsors. The two primary subjects of the conference this year were: Machine Learning and Containers. Mark found the container sessions very useful and will be applying his learning to improving the JEDI containers over the next few weeks. His plan is to develop a unified, module-based build process that can be used for containers, the cloud (e.g. AWS), and environment modules on HPC systems.
Hailing is working on the obs operator for GNSSRO bending angle. This is work in progress which includes updating variable names to the netcdf-CF conventions. She intends to make the GPS implementation more generic.
Maryam attended the SEA conference last week and found the sessions on software testing useful. Maryam announced that the "is_close" function (which is used in the ECKIT test framework but does not require a modification/fork of ecmwf::eckit) has been enhanced to report the data values that are being compared. This is being done to assist the developer with debugging and development of tests. This capability has been merged into develop so it is available now.
Travis has rewritten SOCA tests (ctest) and has reduced the test suite execution time from 15 minutes to under 1 minute. He accomplished this primarily by reducing the ocean domain size. Travis is also developing marine tests for the ioda-converters repository. He should have a pull request later today for this work.
Steve H is refactoring the NcWriter class in the ioda-converters which will help with performance issues and facilitate the calling scripts that are appending supplemental meta data to the output netcdf files. He has the ODB2, GSI ncdiag, and 4 of the marine converter scripts working with the refactored NcWriter class. There are a number of marine scripts left to go before the pull request can be submitted.
Steve H reported that he is seeing two test failures (test_qg_parameters_bump_lct and test_qg_hofx3d) in the develop branches for ufo-bundle using GNU 7.4.0 compilers. He has submitted a ZenHub issue in OOPS (#210) with details and discussion about these failures. Anyone with knowledge about these tests is encouraged to look at the issue and help get these resolved. Yannick added that the test_qg_hofx3d failure has been around for a while now and work is in progress in getting this fixed.
Steve V has placed a couple enhancements into the ODB2 ioda converter scripts of which have been merged into develop.
Xin is making good progress with the UFO bias correction scheme. He needed to refactor some code in OOPS to get this accomplished and will be submitting a pull request soon.
Junmei is testing a one-month DA cycle using MPAS.
Yali (Wu) is developing the surface pressure obs operator in UFO. She has a new QC check which is modeled after a similar check in WRF-DA. This code has been pushed to a feature branch and is work in progress.
Chris H has submitted a pull request for the shallow water toy model forecast capability. He discovered a GNU compiler issue with some Fortran 2003 code, and is working on getting this resolved. Chris will let the reviewers know when it's okay to start again.
Steve S is working on enabling split MPI communicators for the IO server in LFRic. This work will enable LFRic to run ensembles on its own. He has the LFRic implemenation running and now needs to interface it with oops.
Marek is currently out of the office which is why he hasn't responded to a request from Anna to review her geovals refactoring pull request. Steve S has picked up this work, which requires modifications in the LFRic interface, and he has everything working except hofx. Steve S will keep Anna updated as he makes progress.
Yaswant is getting up to speed on C++ and the LFRic model. He has written some LFRic documentation and pushed this work to the github repository.
Sarah reported that they have finished up their BUMP interpolation work, and they have model refactoring work in progress.