Note: There was a problem this week with the email meeting announcements
Two email announcements were sent out yesterday and one today through Google Calendar as in previous weeks but nobody received them. There appears to be a problem - we will look into it.
The main agenda item for today was just a round-table update and discussion of progress and potential problems, particularly in light of recent code changes such as git-lfs-enabled versions of ufo and ioda. Also, there will be no meeting next week so this is a chance to discuss issues before then.
We started with an update from the Met Office. Steve (H.) is now visiting to learn about ODB. He successfully installed Singularity on a WIndows laptop and ran ufo-bundle. Since this is the first time (to our knowledge) that JEDI has been run on a Windows platform, Steve plans to write up his experience on ReadtheDocs. He also installed ODB and started working with David's code.
Steve asked about how Locations will be handled for more complex observation types such as satellite observations where a simple lat, lon is insufficient. Yannick responded that (lat,lon) was only a simple beginning to the Locations class - this will continue to evolve to include more sophisticated subclasses to handle satellite, radar, and other data that require more information than just lat,lon. Initial efforts to expand the scope of the Locations class is now underway with GNSRO data and future developments are planned. Anna will give an overview of these efforts at our next JEDI meeting on July 12.
Steve mentioned that this may impact what the Observation database looks like and intends to discuss the issue with colleagues at the Met Office this week.
Andrew (C.) offered to organize a breakout meeting of those who work with satellite data to discuss how best to define the Locations class. This would take place sometime after July 9. If you are interested in participating, contact him (andrew.collard@noaa.gov).
LFRic update
- keeping up to date with recent changes
- currently working on MPI implementation
Marek suggested that it would be beneficial to have a document outlining steps that participants should take in preparing for the B-Matrix code sprint in August. Chris (S.) and Yannick agreed. The plan is to set up a google doc that participants can edit collaboratively. Stay tuned for further information.
Yannick mentioned that the main objective of the B Matrix code sprint is to have a multivariate B matrix with balance operators implemented. This would hopefully be generic as possible, with several specific model instances. The code sprint will focus on using bump. The question was raised as to whether JEDi is committed to bump in the future. Yannick responded that it is a useful general-purpose tool for now but we are open to other possibilities if and when they arise.
The discussion then came back to ODB and the status of ODB1 in particular: Is it still proprietary? The short answer is Yes.
Yannick mentioned that ODB2 is distributed as an open-source tar file from ECMWF and that ODB1 is in fact included in that distribution. However, it is included without a license so it's currently impractical to use. There is some interest among researchers/developers at ECMWF to make ODB1 open source but making this happen politically is another question.
MPAS update
BJ says they are now working on getting MPAS to run on multiple cores/threads. They were able to run successfully with 2 cores in the Singularity container but CRTM is giving slightly different results than the single-processor version. Ben mentioned that there is no parallelization in CRTM and the MPI messages might be getting mixed up. There appears to be a bug in the ufo/CRTM interface for multiple processors. Those concerned agreed to work on this.
Chris asked about progress on Cheyenne and said that they were having issues getting the MPAS bundle to work. Yannick responded that ufo-bundle is known to work on Cheyenne but, to our knowledge, no other bundles have been implemented.
update from Dan:
Benjamin (M) pushed some substantial changes to oops this morning that have to do with bump and there seem to be some bugs to work out. This could effect anyone who is using bump. In particular, Dan and others have been implementing multiple bump instances and this could be part of the problem.
After the meeting, Dan created a new develop-bump branch of oops to replace the feature/bump branch.
Other progress with fv3-jedi includes the implementation of GEOS background states and the full implementation of git-lfs, with the removal of previous netCDF files. As with ufo and ioda previously, this involved creating a new fv3-jedi repo (the old one is now in fv3-jedi-old). Anyone using fv3-jedi should delete their local copy and re-clone from GitHub to ensure that your copy is consistent with these changes.
Update from Xin:
Obs operators for aircraft have now been implemented and are now being tested. The initial list of variables include u, v, and T but the full list will be implemented soon.
Update from Ben
TL/AD has now been implemented into CRTM and is now being tested/debugged. It was noted that some variables are not well suited for linearization (e.g. integers). Anna suggested possibly using different variables for the trajectory and the TL/AD. Another option is to zero out some variable perturbations.
update from Rahul:
GSWE is now optional in ufo (pull request yesterday). However, if you do a pull request on develop please keep the marine operators in.
Request: Please put Guillame, Rahul, and Anna as reviewers for any pull request on ufo or ioda that has to do with the marine data/models.
Update from Mark:
We are now looking into running JEDI and other JCSDA applications in the cloud, through Amazon Web Services (AWS). AWS is a promising platform for testing (continuous delivery) and potentially for the distribution of data, code, and computing resources in the future. Stay tuned for updates.