(Note change in title naming convention for Meeting Notes, so they are sorted more intuitively in the JEDI Wiki)
Yannick opened the meeting by noting that the marine code sprint is still going on through tomorrow (so many JEDIs are still in Boulder this week) and asked for a report from Guillaume.
Guillaume reported that the sprint has been productive - they have integrated 12 satellites into the marine ioda/ufo that include data for SST, salinity, and ice fraction, and these data have been successfully assimilated into their cycling system. They have successfully run a 30-day 3DVar application that looks promising.
Then we proceeded to go around the table in Boulder.
Hailing reported that her implementation of 2D DA in ROPP has been completed and she issued a PR yesterday that is currently being reviewed. The 2D DA is now fully functional with input from GeoVaLs.
Junmei has been looking into using the cycl workflow software to manage cycling jobs with MPAS
Yannick advised not to spend too much time on this because the JEDI project is currently interviewing candidates for a new Software Engineer position and the primary responsibility of this new hire will be to develop workflow procedures, tools, and interfaces that incorporate much of the functionality of cycl.
BJ and the MPAS team have been working with Benjamin on the use of BUMP in MPAS 4DVar applications. In particular, they are continuing to work on assimilating surface pressure. Currently they still assimilate it as a 3D field, only building the first level, but they are working with Benjamin on a more efficient 2D implementation.
Benjamin then gave an update of recent progress with BUMP and OOPS. Pull requests have been merged in the past week into OOPS and the JCSDA fork of fckit that replace some of the intrinsic functionality of BUMP (such as the kd tree) with equivalent functionality in fckit/eckit. This reduces duplication of code. But Benjamin warned that test results can be slightly different due to, for example, slight differences on the way the kd tree selects among neighboring points with similar distances from the target point. This is of no scientific significance but it may require some adjustment of unit tests.
Mark will install this new develop version of JCSDA::fckit into the container today or tomorrow: when the new containers are ready he will announce it on the JEDI GitHub Team Page.
Benjamin has also been working with the MPAS and SOCA teams on implementing 2D variables such as surface pressure as separate from the way 3D variables are handled, in order to improve efficiency. It's now working with SOCA. Yannick asked if this required changes from the various models and Benjamin responded that the old way should still work in principle but the new implementation saves memory so it would be beneficial to implement at some point. Benjamin is now working with Dan to implement it into f3-jedi.
Benjamin also announced a plan to separate BUMP from OOPS. Soon it will be placed in its own repository, with its own suite of tests, and it will be built as a library. This will require a minimal change to the model repositories when it is implemented, namely the addition of one line to the CMakeLists.txt file. When this has been done, it will be announced on the JEDI GitHub Team Page, with instructions on how to implement it.
Yannick mentioned that one of the motivations for this change is to allow for alternative B-matrix implementations in the future.
Steve V has been working with the MPAS group toward the goal of cycling for 1 month, so he has been assembling the required obs data files.
Yannick asked him if he has heard any update with ODC but nobody has.
Steve H has been responding to an ongoing review of his recent pull request for ioda on refactoring the ObsSpace class. This should be merged soon. Next he will be working on the interface to ioda to enable sharing of conversion scripts. He has been participating in the marine code sprint and this has inspired him to use nc-writer to help define this common interface. He estimates this work will take 1-2 weeks.
Mark did not have much to report - he has been working on assessing the overhead of the Charliecloud container on Cheyenne and on the fv3-gfs implementation on Amazon.
Maryam have modified the fv3-bundle so that it no longer requires that BOOST be compiled. Soon we plan to remove the compiled BOOST libraries from the container so let us know if you'd like for them to be retained in the short term. The BOOST header files are still used in JEDI and will still be included in the container. These do not need to be compiled - only copied to an appropriate include directory.
Ming has continued to work on the WRF interface. He's completed the implementation of the Field and State objects and has verified that the H(x) works. But, he's still checking the 3DVar - he suspects something is still wrong there.
Steve S introduced two additions to the JEDI team at the UKMO: Yaswant Pradhan and Michael Cook. Both are from the satellite applications team and will work on obs operators and QC obs filters in ufo.
Steve S has been working on how MPI communicators are passed between JEDI and LFRic.
Yannick mentioned that Chris Harrop's Shallow Water model is almost ready to be merged into JEDI and this will provide an illustrative example of how to handle the MPI interfaces between JEDI and models.
We then heard from Corey Martin of EMC, who is working on a python converter script that will facilitate the importing of GSI diag files into ioda and ufo. Steve H asked if this built off of Anna's existing python scripts. Corey confirmed that it does, but it implements them in a more user-friendly was as a python module that can be imported. He also confirmed that it builds off of Steve's nc-writer python class.
Anna has been refactoring ufo to be more user-friendly. Part of this is to move the definition and storage of variables to Fortran. She has also been working on implementing the ObsVector as a 2D array, with the first dimension variables (such as channels in the case of radiance) and the second dimension locations. She has implemented this for the radiance, vertical profile, and identity obs operators in ufo.
Anna is also working on updating the JEDI documentation to include a how-to page on adding an observation operator. She is also contemplating changes to the GeoVaLs class. The plan is to discuss her thought on this in one of the upcoming JEDI Weekly Meetings.
Chris H has made progress on his new Shallow Water model. He has create a GitHub repo and an associated JEDI bundle repo and he is about to issue a pull request with the latest updates. The model can run independently now but he is still working on the JEDI interface. In particular, he is still working on the multi-processor implementation of the interpolation in getValues() - he needs to update halo points so the interpolation can proceed across MPI tasks.
Yannick asked if the JEDI interface components other than GetValues() works - in particular, can you run a forecast from within JEDI? Chris H confirmed that yes, this can be done. Yannick responded that this would be good to merge as soon as possible, even if the GetValues() and DA is not working yet. This will provide a useful tool to demonstrate how models parallelized with MPI can be incorporated into the JEDI MPI framework and it will allow us to play with and further develop the parallel functionality and infrastructure.
Chris also mentioned that he is currently using the application nccmp to compare netcdf files. He intends to eliminate this in order to avoid adding new dependencies to JEDI. However, Benjamin mentioned that he would also find something like this useful and Yannick said that if it is useful, it might be ok to add this to the JEDI list of dependencies. At the very least, we can plan to include it in the container in the near future.
Dan introduced a new JEDI developer from NOAA EMC: Lou Kouvaris. Dan has been working with Benjamin on the use of BUMP in FV3-jedi and on implementing the fckit/eckit kd tree. Lou will be working on adding a ctest to ufo for ozone instruments.
NRL announced a new JEDI developer: Patrick Flynn. Sarah then reported that she's currently working on runtime errors with variable initialization that she expects to sort out today.
Tom then announced that, as of Monday (April 1), JCSDA has entered a new annual operational cycle. Tom will review the 2019 operation plan at an all hands meeting this afternoon at 1pm MDT. All JCSDA personnel are expected to attend. There will be hubs set up at JCSDA/UCAR (Boulder), NOAA DSRC (Boulder), NRL (Monterey), College Park (EMC), and Greenbelt (GMAO).