Yannick opened the meeting. There were no special announcements so we went around and collected updates from everyone.
Dan H has FV3-GEOS in-core execution working in JEDI. The GEOS model code is now building with CMake which facilitates including it in the JEDI build environment. The FV3-GEOS model is ready to be added to the NRT flow. Dan also has in-core GFS working with the FV3 regional model. For the next few weeks, Dan will be performing clean up on the models.
Mark M reported on a new system we have in place for tracking ECMWF updates. We've created forked repos in JCSDA of the following ECMWF repos: ecbuild, eckit, fckit, atlas, odc, and pyodc. We will sync the master and develop branches in our forks on a regular (nightly) basis. Each forked repository has a "release-stable" branch which contains the latest known good version for JEDI. The release-stable branch will trail closely behind the develop branches, and provide a buffer from updates that come in from ECMWF. This should allow for us to stay in sync with ECMWF without thrashing every time ECMWF merges in their PRs. We will keep an eye on the process and make modifications as the needs come up. See this JEDI team github discussion for more details, and note that the release-stable branches of the forked ECMWF repos should be used in all the bundle repos (ecbuild_bundle commands).
Mark M has a workaround for the issue where the eckit release-stable branch failing to build on intel compilers with versions greater than 19.0.5. This branch builds and tests successfully with Intel 19.*, and the issue was brought to light when trying to build on the new Orion HPC system. ECMWF is working on a fix and has approved us skipping the test as a short-term workaround (the code is okay, only the test is broken). Mark will work today to get the JCSDA eckit release-stable branch updated with the workaround (update: the working version is now tagged as eckit/1.10.1.jcsda2). He took this opportunity to note that in our scheme to stay in sync with ECMWF, we will adopt the git flow hotfix and bugfix naming convention for bug fixes, with the twist where the hotfix branches will be merged directly into the JCSDA release-stable branch and the bugfix branches will be merged into the upstream ECMWF develop branch. One can use the "git cherry-pick ..." command to select the changed that need to be merged to
Yannick requested that the tracking scheme with our forked ECMWF repositories be documented, perhaps in the ReadTheDocs website. Mark is working on this - expect this to appear next week.
Steve S asked when we should be switching bundle repos to the new release-stable branches. Mark M responded that some coordination needs to be done first between new containers and the CI testing mechanism, and there might be a brief period today when the CI testing is broken. Mark noted that the new GNU containers will include an upgrade from 7.4 to 9.3.
Steve S asked about the status of the fix for an issue in saber related to CMake not being able to find boost. Two things for Steve to check came up. First, jedi-stack is using the "boost_ROOT" environment variable method to help CMake find boost, and this might help address Steve's issue. Second, check to make sure that in the bundle CMakeLists.txt file that the ecbuild_bundle command for oops comes before saber. The dependency order between oops and saber recently changed, and now that oops comes first it might resolve the finding of boost for saber. If these suggestions don't pan out, then Steve will open a github JEDI team discussion to help get this resolved.
Steve H reported that he and Ryan are making good progress with the new ioda interface and refactoring effort. We are currently working on integrating the new interface into the ioda repository. We are starting with ObsSpace and should have an initial version working in the next few days. Once the ObsSpace is converted fully to the new interface, the next step will be to migrate the ioda converters to the new interface. Some discussion ensued about moving UFO and the converters to the new ioda interface. We are planning on making incremental changes to avoid thrashing everything at once. For example the Fortran interface to ObsSpace that UFO is using will initially remain the same and underneath calls will be made to the new interface. Then more of the interface will be exposed in subsequent updates. The idea with the ioda converters is to have the converters work at the ObsSpace level and let the ObsSpace deal with reading and writing the ioda files. This is why the ObsSpace conversion needs to occur before the converters are migrated. Rahul mentioned that he has a few new people ramping up to help with the converter migration, so Rahul will send a the names of these people to Steve so that they can participate in the planning of the converter migration effort.
JJ asked about the status of EDA and LETKF. He will be working on these for MPAS. EDA is working in FV3. It should be a fairly simple change and FV3 can be used as a guide. The main thing is that separate MPI communicator groups are used for each member, and the model interface needs to be updated to handle the communicator groups properly. LETKF is running in FV3 and SOCA, and the results are being checked for scientific validity. These two model interfaces can be used as guide for MPAS.
Sergey asked if there is an easy way to monitor JEDI development. We are using ZenHub for tracking development, and training sessions will be coming soon. Be on the lookout for invitations to the ZenHub training from Phil.
Sergey then asked about plans for all-sky DA. Greg Thompson, new hire in the OBS project, will be working on this, and Jake mentioned that NCAR has begun testing microwave and IR all-sky assimilation. Yannick added that the development of all-sky DA should be ramping up over the next month.
Yannick announced that he has done a clean up of the YAML file organization in preparation for this summer's JEDI release. He has a draft PR in oops for this change and he will likely remove the draft status later today or tomorrow. While there are extensive changes to the YAML, there are just a few code changes in this PR. This PR will impact all the models, so there will be a coordinated effort to update all the models along with this PR.
Chris S asked about the status of the oops PR to make the CMake configuration compatible with ecbuild version 3. Mark O is the author of this PR and responded that the PR is close to being ready to merge and all that's left is to submit a companion PR in saber. Mark has assigned many reviewers, noting that Steve Vahl got added recently, and recommends everyone to give this a try now.
Mark O announced that the near real time (NRT) website (which is powered by jedi-rapids) has been useful in flushing out bugs in ioda obs files. A positive result of this is that he was able to add fault-tolerance to jedi-rapids in that it will now skip over bad obs files and complete using the remaining good files. Next steps for the NRT website are to add in VIIRS AOD obs and GEOS forecasts. Mark is working with Dan to make the GEOS backgrounds and compute resources (Discover) available to jedi-rapids.
Guillaume asked about the status of refactoring GeoVaLs, and Yannick responded that not much has been done yet and this effort will ramp up over the next month.
Yannick closed the meeting at this point with the announcement that next week we will have a focused discussion on Static-B.