After Yannick opened the meeting, the floor went to Dan and colleagues at GMAO.
Dan has been working with Jun (EMC) on interfacing FV3-GFS and FV3-GEOS with JEDI. Progress has been good and these interfaces now compile. Linking FV3-GFS-cap with JEDI has been particularly promising. Dan has also been working on an implementation of FSOI that is within core memory, minimizing IO.
Guillaume has made progress on bringing the QC with ocean observations up to date with recent developments in ioda and ufo.
Andrew mentioned that he is identifying people at EMC who may be interested in working more closely with JEDI. He may organize a mini code sprint to help get new users up to speed. He intends to build off of Anna's recent work with GeoVaLs and made plans to maintain contact with the JEDI core team to coordinate efforts.
UK Met office: Marek announced a recent pull request on LFRic with new model developments, including aquaplanet and an improved capability to run with more than 6 MPI tasks. He also generated a pseudo obs file for radiosonde observations and reported that it is working with 3DVar. Currently only temperature is implemented; the file is yet not set up for winds.
Marek had a question about the netcdf files - he noticed that they do not currently include units in the metadata. He also asked why the time is expressed in hours. Steve H and Yannick agreed the obs file metadata should include units and added this to the work that needs to be done. Steve H also mentioned that work is planned for how the DateTime is handled; we intend to implement a string representation of the absolute time in the netcdf files that can more readily be read into JEDI.
Marek then elaborated on the improved multi-processor functionality of LFRic and mentioned that he had an improved version of the compare script (used for integration testing) that can better handle multiple MPI tasks. He reported progress on setting up the LFRic test suite using the Rose stem software package developed at UKMO, which can now call ctest.
Yannick mentioned that we will step up our work on improving the JEDI test suite in January, when a new software engineer will join the JEDI core team.
Then the floor moved to the Boulder group. Ming reported continuing progress with the integration of WRF into JEDI. He is currently working on writing a GetValues() function for WRF and in particular the bump implementation.
Hailing reported that she pushed several developments this week that enhanced the functionality of ioda and ufo, particularly with regard to GNSSRO. This week she is working with Yannick on the QC handling for these and other obs.
Steve V reported that he is working closely with Steve H on ioda design, particularly with regard to how to store obs data and how to effectively and efficiently read and write obs data. He also mentioned a python script he is developing that will import Met Office ODB radiosonde data into JEDI.
JJ has been working with Anna on moving the Locations class from ioda to ufo. They added an (optional) capability for each obs operator to do its own conversion from obspace metadata to the Locations object and then to GeoVals. This enhanced functionality is intended for more complex observations; there is a default Locations description available that will be sufficient for most observations.
JJ also reported an error with 3DVar in MPAS. When they increased the radiosonde observation number from 600 to over 1000 the cost function increased by an order of magnitude. He asked if anyone had any insight into why this may have happed. A few possibilities were offered. Marek mentioned that the background check has not yet been switched on. Yannick agreed that this may have something to do with it and said that we will be working on that today. Steve H mentioned that just before and after the Academy there had been some work done on how missing values are handled. JJ confirmed that this error started right after the Academy, suggesting this as a possibility.
Yannick then introduced Clementine, an intern who is working with the JEDI core team on ensemble DA in oops. She is currently working on adding parallel ensemble DA (including pre-conditioning) to the Lorentz and QG models and will move on to the more complex models. This involves developing the capability for one group of MPI tasks to serialize data and send it to another group where it is deserialized.
Anna mentioned that she has been working on several ufo updates that have been pushed and merged this past week. They are mostly refactoring so they should not affect most users. She has prepared netcdf geovals files for a variety of radiance data on the April 15 date to be used for testing. There are issues that still need to be worked out with the aircraft and radiosonde data and she may want to work on this with people at EMC.
Andrew asked if ozone has been implemented yet. Anna responded not yet, but this is planned.
Xin issued a pull request this week that contains the new C++ implementation of the ioda ObsSpace. This pull request is still being reviewed. When merged, it should be transparent to most users since it will not change the interfaces that were implemented last month. He has tested this new implementation with both GNU compilers and clang. He's getting only one test failure in each and is now investigating them. He has not yet tested ODB functionality of the new ioda implementation but Steve H said he could help with that.
Steve H said that much of what he's been working on for the past week has already been mentioned by others. He has been involved in the code reviews of the ioda and ufo work that has been merged this week and he has been working with Steve V on enhanced conversion scripts into a format that is easily readable by ioda. Steve asked for feedback on how these obs-to-JEDI conversion scripts should be constructed - this is the time for input.
Mark announced a new container option for JEDI: Charliecloud. This is an alternative to Singularity that has the advantage that you do not need root privileges to install and run Charliecloud containers. This is particularly useful for HPC systems but it can also be used for linux pcs, vagrant virtual machines, and cloud computing. For example, Mark tested this on Cheyenne yesterday and it worked where Singularity does not. He installed Charliecloud in his home directory and successfully built and tested ufo-bundle within the Charliecloud container with no need to contact the system administrators.
The JEDI Documentation will be updated later today with instructions on how to install Charliecloud, how to download the JEDI container, and how to use it (in the JEDI Environment section of the Docs). Keep an eye out for this.
Mark also mentioned that the JEDI team had a meeting to discuss how to handle build scripts for those who work outside the containers out of either preference or necessity. It was decided that we will provide modules on selected systems that will be available to be loaded by JEDI users. Our initial list of supported platforms is Cheyenne, Theia, and Discover. Instructions on how to use these modules will also be posted in the JEDI Documentation in the coming days (also in the JEDI Environment section). So, check back frequently for this content as well. Contact Mark if you have any questions.
The floor then turned to NRL where Sarah reported that work is continuing to integrate NEPTUNE into JEDI.