The meeting opened with a round-robin report from Boulder.
Steve H continues to work on fixing the missing value implementation in ioda (see last week's notes). He thanked everyone who has helped add features to the ioda_converters repository in recent weeks.
Maryam has implemented Travis-CI automated testing in the oops repository. Her changes have been merged into develop so whenever there is a pull request into develop, Travis CI will run tests. Users can see the results of these tests by accessing the pull request on GitHub and selecting the green check mark (passed) or red X (failed) and following the links where that leads. Maryam will give a demonstration at the JEDI meeting next week but feel free to explore this beforehand if you wish.
Travis registered a complaint that pull requests into oops, ioda, and ufo can often break tests in soca and this seems to be becoming more common. The team often discovers multiple failures in tests that had been passing the previous day and this is commonly due to code changes that change the output of tests. Sometimes these changes are minor, but enough to break tests. Furthermore, some of these code changes are incorporated into large pull requests that are difficult to sort through to find the relevant code changes. A recent example is the large PR merged this week into oops that refactored the qg and l95 model implementations. This PR included some changes in bump that changed soca test results.
So, the appeal from Travis and Yannick an the rest of the JEDI team:
Please split large code changes into a series of smaller pull requests
This makes code review easier and it facilitates debugging when problems arise.
Travis also suggested that we need to develop a better workflow for larger test configurations. Mark suggested that periodic tests can be triggered on a weekly or monthly basis using Amazon's batch service, which can be set up much like chron jobs. David Hahn (JCSDA) is already using this for IOS. Yannick mentioned that this is part of what new software engineer Mark Olah (JCSDA) will be working on. Yannick also mentioned that bump will be moved to its own repo (saber) soon, so changes here will be decoupled from other parts of the code in the future.
Xin is completing his work on the ObsAux class in oops, as a first stage in the implementation of variational bias correction. This is taking a bit longer than anticipated but he is now finalizing the comments arising from the code reviews and expects to merge this week. Then he will being the next stage of changes, which focus on ioda.
Steve V has been modifying the mpas-bundle build script to be compatible with the most recent containers and environment modules on Cheyenne. He has also been working on an ODB2 converter to ingest data from the ECMWF amsua reports into ioda.
Travis attended the GODAE OceanView Symposium that was held last week in Halifax, Nova Scotia. He said that there was a great deal of interest in JEDI from the ocean DA community. He mentioned an ocean DA team at the UK Met Office in particular who are very interested in working with JEDI and he plans to strengthen ties with them.
Travis also mentioned that go got JEDI+SOCA working in the Charliecloud container on Gaea.
Mark has been building new modules on AWS and Cheyenne with the jedi-stack build system. We are now able to test JEDI on AWS with clang as well as gnu and intel compilers. He has also been preparing AMIs for the upcoming JEDI academy.
Jumei is testing data from GSI/NOAA, for use in MPAS applications and tests.
Hailing continues to work on the bending angle operator
Ming continues to work on the variational part of the WRF-JEDI interface
Anna is refactoring how the observation operators are handled in oops, allowing for different locations in different operators
JJ noticed an error in the DA Diagnostics repository - the scripts are plotting model-observation instead of observation-model. He plans to fix this soon. He also asked Xin when the new bias correction implementation will be available. They are working on cycling runs with satellite data and he wants to know if he should use the new bias correction or handle it another way. Xin responded that the first PR should be merged into oops in the next day or two. The next step is to read the bias correction from the GSI data file. Steve H and Cory offered to help with this (they will create an issue on ZenHub). The hope is to get this done before the JCSDA workshop May 28-31.
BJ has been working with the MPAS mesh implementation.
Mark O has been assessing options for a workflow language to be used to build, test, and run applications, eventually using a web-based GUI. Several of these use the Common Workflow Language.
The floor then passed to EMC and GMAO
Cory offered to help with any work that has to do with GSI diag data - if you're working on something related to this, please create a ZenHub issue and add Cory to it. He has also been working on ufo obs operators for surface measurements (e.g., 2m temperature) - he is close to issuing a PR.
Dan has an open PR in fv3-jedi in which he exploits his recent multi-resolution capability to greatly improve the efficiency of tests. On discover, the time for the fv3-jedi test suite decreased from 11 min to 2 min and the size of the data files needed from git-LFS decreases by an order of magnitude, from 3GB to 0.3GB. However, the 4DVar tests are not converging. He suspects that this has to do with the variable transforms in TL/AD.
Andrew has added ctests to ufo and has a current PR in ufo for subsetting radiance channels with crtm.
Guillame is working on the change of variables for ocean DA.
Sylianos continues to work with ocean observations and model interfaces. He announced that the system administrators have agreed to install Singularity on Theia.
Hamideh has updated GMI observation and geovals data, and the GMI CRTM obs operator. Pull requests are under review and will be merged in soon.
Rahul is working on improving the SOCA workflow for cycling runs. He's using new functionality in the ioda_converters repo to ingest raw observations.
Marek has a PR in ufo and ioda that enhances the RTTOV interface, with tests. Marek opened a discussion concerning Locations and GeoVaLs in regard to a DA flow he is working on. Yannick, Anna and Marek talked about several options and tradeoffs, and then agreed to continue the discussion offline.