Yannick was absent today so Mark M ran the meeting and opened by announcing that we have a few new people, who have joined our github JEDI team, from last week's Academy. Then we collected updates from everyone.
Boulder
Hailing is with working with Francois to go through the flow starting with GNSSRO obs conversion to IODA format, and ending with H(x) calculation. This is being run successfully on Cheyenne.
Mark M announced that instructions will be coming shortly for getting started on AWS.
Clementine got an assimilation working with FV3 in her ensemble/block flow, and is currently verifying the results.
Mark O created a new repository, JCSDA/bufrlib, which contains the latest version of the NCEP Fortran BUFR file reader/writer code. This has CMake control to build and install the a library file which can be linked into Fortran programs that convert BUFR to IODA format. The plan is to use the JCSDA/bufrlib repo to build and install the bufrlib library in the jedi-stack process.
On the topic of NCEP provided code, Mark M asked if anyone is using the nceplibs package, which is currently being built and installed in the containers and AWS. Discussion followed and it was determined that FV3 is probably using nceplibs, but that would be for FV3 outside of JEDI. Also, inside their firewall, NCEP has github repositories for these packages (bufrlib and nceplibs) and it would be good for us to get contacts at NCEP to coordinate enhancements and fixes we make. Rahul will get some names for us.
Anna mentioned that Cory's PR containing updates of the GSI ncdiag obs and geoval files has been merged into develop. Steve H thanked Cory for his work in this PR to get the obs files into compliance with the IODA requirements (no double data types, using Netcdf fill values for missing marks, using integers for QC marks). This narrows the set of files out of compliance, that are being used in unit tests, down to a handful of files. Steve H will submit ZenHub issues for the remaining non-compliant files.
Steve H announced that a PR containing a configuration to build and install the JEDI stack software onto Mac OSX has been merged into develop in the jedi-stack repository. This uses Clang for C and C++ and gfortran for Fortran. This update allows one to run natively on the Mac outside the container, and uses Lua modules to setup your environment. Check out the README.md file in the jedi-stack repository (https://github.com/JCSDA/jedi-stack) for instructions. It should also be straightforward to use the jedi-stack tool to build an environment using only the gnu compilers (gcc/g++/gfortran) if you prefer.
Steve V has been investigating software development tools on Cheyenne. He is looking at DDT (debugger) and MAP (profiler), and using the MPAS flow to try these out. He just got these working so we will hear more about this later. Mark M pointed out that the JEDI core team is currently looking at DDT versus TotalView as the debugger tool we will purchase for use on AWS. There is a ZenHub issue in the jedi-stack repository (https://github.com/JCSDA/jedi-stack/issues/22) where we are holding a discussion, and anyone who has any preference for TotalView or DDT is encouraged to share your perspectives.
Ming has the WRF based 3DVar flow working. This includes TL/AD using a single obs type and single variable, radiosonde(?) temperature, and he is currently extending this to use multiple variables from multiple obs types.
JJ has created a draft PR in UFO (https://github.com/JCSDA/ufo/pull/379) containing work on the CRTM Jacobians. This is incomplete, thus the "Draft" status of the PR, but JJ asked those who are interested to take a look and give feedback.
Maryam is working to get Travis-CI running on AWS. She is developing ways to run higher level (ie, more expensive runtime) tests beyond unit testing. In addition to using AWS nodes for these tests, another possibility is to use the NCAR classroom workstations (which essentially sit idle at night).
EMC
Andrew C has a PR under review in UFO (https://github.com/JCSDA/ufo/pull/316) that contains a fix for microwave channel subsetting. Anna asked if we could get this PR merged in soon, so Andrew will work on the revisions that need to be made to address reviewer feedback.
Cory has a PR under review in UFO (https://github.com/JCSDA/ufo/pull/334) that contains new obs operators for surface obs (2m temp, 10m winds).
Jong attended last week's Academy, is understanding the CICE6 source code, and is ready to start on the CICE6 JEDI interface.
Hamideh asked if the RTTOV repository is ready, along with what is the process for running RTTOV tests. Anna and others currently use an RTTOV 12.2 library that can be obtained as a tar ball from David Rundle (UKMO) for those who have obtained an RTTOV license. David is now working on a cmake/ecbuild implementation of RTTOV 12.3 that will be made available as a JCSDA GitHub repository for licensed users. This should be ready soon, possibly next week. In order to use the RTTOV software (either the 12.2 tar ball or the 12.3 GitHub repo), each individual (as opposed to one person per lab) needs to register for a license. Contact Mark M (miesch@ucar.edu), Marek (marek.wlasak@metoffice.gov.uk) or David Rundle (david.rundle@metoffice.gov.uk) for the instructions for registering.
UKMO
Marek reported getting a git-lfs SMUDGE error when attempting to update to the latest IODA develop branch (lots of git-lfs obs files got updated with Cory's recent merge). This appears to be an issue with the git-LFS servers on Amazon S3. Mark M advised anyone who runs into this problem to continue forcing the update until it completes by repeating the following command as many times as necessary:
git checkout -f HEAD
NRL
Sarah reported that they are rapidly getting caught up on their PRs regarding their NEPTUNE refactoring (for JEDI) effort.