Yannick was out of the office today so Mark M ran the meeting. Mark opened by announcing next week's meeting would be a focused topic discussion. We don't have the topic selected yet, but will announce that topic before next week's meeting. One possibility would be to continue the C++ tutorials from the February 13th meeting. Please submit ideas for focused topics by contacting a core team member or entering an issue on the discussion topic ZenHub board.
We then collected update from everyone:
Boulder
Steve V reported an internal compiler error (ICE) when building on a Mac in the docker container. This occurred using GNU with openMPI while compiling the lorenz95 EDA code. This had a familiar ring to it, but no one could recall the specifics of what happened before. If anyone has a fix for this ICE in their notes, please let Steve V know. A suggestion that was offered is to shut off optimization on the compiler (ecbuild --build=Debug ...). Another suggestion is to turn on verbosity during make (make -j4 VERBOSE=1) so the specific command issuing the ICE can be revealed and then different compiler options can be tried manually to further debug. (Update: one fix for this, that we noticed in the past, is to do a serial make; make instead of make -j4; this suggests that there is a dependency somewhere that isn't properly being taken into account)
Steve V is also developing an ecbuild/cmake configuration for the MPAS code.
Xin is refactoring the ObsBias classes to eliminate duplicated code. He is also helping with JEDI WRF development.
Sarah L attended last week's academy in Monterey and is working on getting AOD observations assimilated. She is currently looking at NOAA-20 VIIRS, GOES-16 ABI and MODIS AOD observations.
Steve H helped run last week's academy. He is working on the ObsSpace refactoring, and is assisting several people with new IODA converters including Wei Han (GIIRS LW radiance) and Sarah Lu (AOD from NOAA-20, GEOS-16 and MODIS).
Mark M helped run last week's academy and has created a post-academy survey. For those that attended, you should have received an email with a link to the survey, and please fill out the survey.
Mark M is migrating a new jedi-stack build on Discover to SLES 12. New modules should be available soon after the build is migrated, and instructions will be sent out. (Update: jedi gnu and intel modules are now available on Discover SLES12; all fv3-bundle tests pass with gnu but there are some test failures with intel that are still being investigated; contact Mark M for details).
Mark M has a draft PR in OOPS containing a new interpolation interface.
Ming is working on JEDI WRF. He is testing forecasting now, and will move on to hofx next. Once hofx is working, he will start performing runtime benchmarking.
Jim has a working auto profiling capability in his GPTL library. He will add a configuration so that CMake can find the installed GPTL library for linking in to client code. A draft PR will be submitted soon. Jim mentioned that it will work to mix auto-profiling (specified by adding compiler options) and manual profiling (specified by the user by adding control statements to code).
Travis is working on cleanup in SOCA and mentioned that SOCA broke with the recent SABER PR merge.
Travis attended the recent AGU Ocean Sciences Meeting, and he noted a significant interest in JEDI from other attendees.
Cheng is working on understanding CRTM lookup coefficients, from an aerosol perspective, to help develop an offline simulator that will generate CRMT lookup tables.
Clementine is running EDA with 80 members using FV3 while ramping up the number of assimilated observations. She is currently validating results.
EMC/GMAO
Guillaume and Hamideh reported issues with modules on Discover when building/testing fv3-bundle. There was some discussion about the ecbuild version, and Mark M reported that despite the name "module list" reports (jcdsa-bugfix-old-linker) ecbuild is updated to the correct version. Due to a number of recent CMake configuration changes, it was suggested to wipe out the current build area and start over from scratch.
Emily is making good progress on QC filters for radiances. Expect to see several new PR's soon.
Emily introduced two new EMC in-kind contributors, Dagmar and Iliana. Dagmar will be working on filters for radiosonde and aircraft, and Iliana will be working on satwinds. Emily asked if the new people need to acquire AWS accounts. Mark M responded that they do not need accounts. The data that they would need from S3 are in public directories, and although CodeBuild test results are not viewable without AWS accounts, we will soon add in a feature that will email results of CodeBuild testing to the owner of the PR being tested.
Ryan is updating his lecture and practical documents according to feedback from last week's academy to prepare for the next academy coming up in June.
Ryan announced that Hera has been updated with supporting packages for his jedi-build system, and he will start testing it out.
Rahul asked Mark O if jedi-rapids is ready for a demo with EMC/GMAO folks. Mark O responded that there need to be a few more updates before it is ready. After some discussion it was decided to try to have the demo take place next week. Mark mentioned that participants need access to either Discover or S4, and that the number of people needs to be limited to 6.
Guillaume asked Mark O for an update on the coordination of jedi-rapids with the workflow Travis put together for SOCA. Mark O responded that he has been working with Travis to handle the marine obs data and to handle the full cycling capability of the current SOCA workflow. Jedi-rapids will need a few more features to cover the SOCA workflow. Mark O added that currently jedi-rapids can run GEOS for real-time hofx and can run 3DVar.
Rahul then mentioned for jedi-rapids, he is interested in getting support for FV3-GFS. Rahul also asked about support for static netcdf libraries since only the static libraries are available on the HPC systems. Mark O responded that he is working on that and the work entails updates to the findNetcdf script in ecbuild, and updating modules (of which Mark M will help with). Mark M added that support is now in place for GNU 9.1 with openMPI on Discover, and recommends moving to that configuration.
Met Office
Marek reported that good progress is being made with interpolation updates.
NRL
Sarah mentioned that Intel 19 compilers are working for them, and they are getting caught up with several model changes.
Mark concluded the meeting with a reminder to send ideas for the upcoming Thursday focused discussions, and thanking everyone for participating today.