OBS general discussion:
In preparation for the AOP planning meeting, have been reviewing the OBS issues and have cleared up the remaining AOP20 icebox issues, and now moving through the "new issues" and either retiring them or moving them up in priority to either Backlog or Icebox. Also the MetOffice has an ongoing sprint this week dealing with the top-down ordering or the model fields which will help to resolve a handful of issues.
OBS1:
Most UFO work now focuses on data feed and conversion at this point. OBS team is working on WMO bufr conversion using different approaches to ensure the readiness of data feed for different applications. This is also the focus area of the in-kind efforts lately. UKMet is working on vadiating JEDI data processing in comparison of their operational capabilities (radiance, conventional). EMC is mostly working on NCEP bufr converters (will be reported by OBS3), focusing on aircraft data types at the moment. EMC is also working on the AOD operators. NRL finished their IODA_v2 upgrade. In addition, JCSDA continue to work with UCAR regarding the obs error studies for GNSSRO.
OBS2:
- Hailing has made progress deploying JEDI EDA for impact assessment. She will be presenting her results at the AMS next week.
- Rachel has set-up a web application for the visualization of the diagnostics using Voila dashboarding tools.
- Fabio has been working on adding the LinearForecast application PR (#1407).
- Francois has been working on the UM-JEDI HoFX application with Air Force's GALWEM forecast files for selected instruments (RAOB, GNSSRO and ATMS).
- AWS sandbox accounts have been created for testing the suite of diagnostics tools (BESPIN/PADME).
OBS3:
- First, Ron gave a presentation about the status of his bufr2ioda code. Most of his recent work has focused on extending NCEPLIB-bufr with a new functions to enable arbitrary queries on bufr data. This new query interface is then used within ioda-converters. Ron's presentation showed everyone how BUFR files are structured and how the new query functionality works.
- At this point, basically every data field in every kind of BUFR file should be readable. Examples already exist for mhs, mtiasi, adpsfc, and adpupa. However, all of this work is in the feature/query branch of ioda-converters and is somewhat blocked from moving to develop until NCEPLIB-bufr accepts the new features. Once this is addressed, then we have plans to port it to ioda as a new backend. This way we can read BUFR in memory and convert it into the IODA format without a temporary file touching the disk. We already do this with the ODC reader, and we do not anticipate significant problems.
- For the second half of the meeting, I talked about the new JEDI Data Conventions. We merged three pull requests to develop yesterday in ioda, ioda-converters, and ufo-data. These PRs introduce a new program, ioda-validate, that tells users if an obs file respects the data conventions. Any issues with the data are reported back to the user. Now that the PRs are completed, we now have automated reports for all of our converters and for all ufo-data files. Next, we plan on looking at these reports and consolidating common issues, and we will write a program to automatically fix most common issues. The remaining work after that will be addressed in a code sprint.
CRTM:
- New coefficients in develop, finalizing IASI-NG coefficients, delivery from STAR, also for the GeoXO sounder. Article proposal for BAMS (update of the CRTM reference article of 2006).
- Question of Chris Snyder: slow I/O for IASI in MPAS => Yannick answers: working on that with parallel I/O.
SOCA:
Problematic container update that broke SOCA CI without warning. How should people be warned about these updates? Maybe starting with a GitHub discussion group and then moving to an automated email system?
LAND:
No update
JEDI1:
- Work on spack stack continues, fv3-bundle builds and runs on macOS with apple-clang + gfortran, all tests pass; cheyenne: ioda/ufo/oops tests pass, but fv3-bundle tests crash (with Intel 2021.2.0); orion: currently building
- First meeting on speeding up I/O in IODA - Steve and Ryan will be working on this
- Started working on option to turn off OpenMP completely (changes needed in many of our repositories, and some of the dependencies)
- New CI tests for UFO data, uses Ryan’s IODA validator (for now with clang)
- Updating UK Metoffice container (based on our GNU container)
- jedi-stack: update of ECMWF packages (eckit, fckit, atlas, odc), all dev containers updated, all CI containers updated, work in progress to update all HPC platforms
- jedi-stack: moved json-schema-validator from core to compiler dependent modules, work in progress to update all platforms.
JEDI2/3:
JEDI4:
Tidying up and finishing PR for FV3-JEDI (2 PR away from cycling... should be ready soon!)