Yannick opened the meeting by announcing that the JEDI core team, along with several other members of the JCSDA Boulder branch, has now moved from NOAA DSRC to UCAR FL4. Beginning next week, many of us will connect to the weekly JEDI meeting from FL4. However, those at NOAA are still encouraged to meet in 2B-407 to connect to the google hangouts. Anna will take over as the NOAA DSRC contact for the weekly JEDI meetings.
Yannick also announced that he has been going through a large number of code developments and pull requests yesterday and today, after coming back from his travels and handling other administrative matters. So, you can expect changes to the develop branches of many JEDI repos this week. Keep an eye out for any problems that may arise.
Then the discussion turned to NASA GMAO. Guillaume reported that he was still waiting for the new version of ioda to be ready for integrating marine obs operators (see below for an update from Steve H). There were also several questions about QG filters in ufo and ioda and how to handle Observation errors. The current functionality for here is limited but we plan to address the Observation error access with the new version of ioda. Yannick intends to write up an example of how to use the current implementation of QC filters.
Guillaume and Rahul have been working with Anna on an application that would allow a JEDI DA run to dump out geovals for testing. The need for this application is to generate UFO tests for observation types that were not assimilated in the April 15th GSI run. Rahul brought up an issue regarding the ordering of variables in GeoVaLs and how to ensure that this is the same as the ordering in the observations. He asked if there was any strategy here. Yannick responded that we should discuss how to proceed and mentioned that this may only be an issue for testing.
Dan reported on an outstanding pull request for oops and ufo that involves the integration of radiance observations. This involved an allocone() method that was introduced to allocate a GeoVaLs of length one. This has now been eliminated in favor of a parameter specification in the config file and the GeoVaLs constructors. However, this caused problems in the tests for QG and L95, which do not use ufo. There was some discussion on whether or not it is worth implementing something analogous in QG and L95 just to pass the tests. This requires further discussion - for now it was suggested that we just turn off the QG and L95 tests in question.
Dan also reported that he is almost finished with the FSOI application he has been working on. He will add an equivalent test for QG and then issue a pull request soon.
Rahul has been working on forking the FMS repo directly from GFDL and arranging it so the fv3 and soca implementations in JEDI both use this common fork.
Then discussion turned to NCAR. BJ reported that they fixed a bug with regard to the GNSSRO bending angle and is working on another fix to sort our some ambiguity between geometric height vs geopotential height.
Steve V reported that an initial implementation of OBD read functions for the new version of ioda is complete and a pull request is now being reviewed.
Ming reported that he has been working on the JEDI WRF interface and wanted to talk with Yannick on how to proceed.
Discussion then turned to the Met Office.
Steve S reported that the aquaplanet implementation for LFRIc is nearly done. He also reported some issues with Singularity. A member of the team installed version 3.0 of Singularity but had trouble pulling and launching the JEDI image file. This was solved by reverting to version 2.5 of Singularity. This is a useful warning to others who may be having problems with Singularity version 3.
There was also some discussion about the name of the Singularity image file that appears when pulling from the Singularity hub as follows:
singularity pull shub://JCSDA/singularity
In the JEDI documentation, the name of this file is JCSDA-singularity-master-latest.simg. However, users may see other names. There was some question as to whether this is the latest version. Mark responded that users should not need to worry about this. The default singularity image file for JCSDA at shub is generated automatically whenever a push is made to the master branch in the JCSDA/Singularity repository on GitHub. So, the pull command above should retrieve the latest version of the image, even if the name is different from that noted above. The JEDI team will update the documentation to make this more clear.
Then Marek reported progress on getting H(x) to pass and asked about the new 4DVar implementation that Anna and Dan have been working on that handles temporal binning of observations in the Locations class. Yannick responded that this implementation is nearing completion. Dan will be visiting Boulder next week and we we have a min-code-sprint that will focus on finalizing this 4DVar implementation and on the new ioda ObsSpace implementation.
The issue of data filters came up again and Yannick mentioned that he intends to write up an example of how to use the current implementation.
Then Steve S asked about the new ioda implementation and where developers can participate in the discussion. Steve H responded that the most up-to-date discussion of this is occurring in the ioda team discussion board on GitHub:
https://github.com/orgs/JCSDA/teams/ioda
There is also some information on the JEDI wiki:
IODA Design and Implementation Ideas
Please note that the information in the GitHub discussion group is more up-to-date that the information on the JEDI wiki.
Steve H also mentioned that there is a pull request now being reviewed for ioda that is designed to facilitate the progressive migration of observation operators from the old (fortran-based) ioda ObsSpace to the new (C++-based) ObsSpace implementation. This includes a higher-level factor for ObsSpace objects that allows one to specify the old or the new interface. This will allow user/developers to implement the new interfaces, using the radiosonde implementation as an example.
Xin and Steve plan to present an overview of the new ioda ObSpace implementation during next week's JEDI Weekly Meeting (Oct 25), along with instructions on how user/developers can proceed to migrate their own observation operators to the new system.
Mark then announced progress in our continuing efforts to benchmark FV3-GFS on the Amazon cloud. Jim Rosinski was recently able to get a version of FV3 (provided by Dom Heinzeller) to run across multiple AWS nodes. Mariusz asked about the hardware used by AWS. Mark responded that the chip hardware (based on intel Xeon processors) is comparable to HPC systems. However, the interconnect between nodes and the disk access are not as efficient as in HPC systems (e.g. infiniband) so it remains to be seen how big of a performance hit this is. We are seeking to benchmark FV3-GFS on AWS in order to assess the feasibility of running NWP applications in the cloud.
Then discussion turned to Ben (J), who is in Boulder this week and next week. He has been working on adding aerosols back into CRTM and will work with Dan next week on testing.