Yannick
- note no meeting next week: Thanksgiving
JEDI 1, Steve H
Maryam:
- created CodeBuild projects for crtm, shallow-water, and wrf_hydro repositories and removed Travis-CI
- working on creating CodeBuild nightly build/test for fv3-bundle
Rick:
- jedi stack on hera is updated
- now uses the most recent version of intelpython, which is part of the Intel OneAPI compiler / tools suite.
- Intelpython utilizes conda and a Intel-specific conda channel to keep the conda environment updated, and allows for choosing different versions of python packages that the jedi stack may find useful.
Steve:
- PRs under review for the new epoch style DateTime representation
- Ioda reader will accept the string DateTime representation for backward compatibility
- The old representation using a “date_time” global attribute with a time offset will be obsoleted after these PRs get merged in
- Initial testing indicates at least an order of magnitude speed up for ioda read and write operations
- Next I will work on completing the ioda Fortran API which is targeted for:
- UFO operators/filters written in Fortran
- Handful of ioda converters that are written Fortran
- Intel compiler group
- No bug-fix support for 18.x and older (21.4 is latest version)
- oneAPI now provides
- Classic (proprietary) compilers: icc, ifort
- LLVM based (open source) compilers: icx, ifx
- LLVM based compilers (icx, ifx ) are the direction Intel is going
JEDI 2 and JEDI 3 updates, Anna
Upcoming changes in the generic layer affecting model interfaces:
- Anna Shlyaeva introduced a minor change in one section for the State test yaml: https://github.com/orgs/JCSDA-internal/teams/jedi-models/discussions/54, will be merged November 29.
- Sergey Frolov and Wei Huang introduced a change in GeometryIterator (allowing for 3D obs-space localization): https://github.com/orgs/JCSDA-internal/teams/jedi-models/discussions/55, tentatively will be merged December 6.
- Coming up in the next month: change related to the variable and linear variable change (Steve Vahl), more information later.
- Coming up in the next month: changes related to GetValues refactoring. Yannick Tremolet will discuss this in the JEDI2 meeting on November 22.
- For both of the latter (large) changes we are planning mini-code sprints on Thursdays, tentatively starting December 2, with the developers of the changes available to assist/answer questions on the required model interface changes.
Other updates:
- Jo Waller’s PR on adding preconditioning for VarBC (https://github.com/JCSDA-internal/oops/pull/1364) is being tested with MPAS-jedi cycling experiments by Junmei Ban.
- Progress on updating oops documentation on L95 toy models, and on oops applications (see PRs in jedi-docs)
- Anna Shlyaeva opened a PR on adding implementations of oops interface classes for a generic two-model coupled model (initial development by Yannick Tremolet): https://github.com/JCSDA-internal/oops/pull/1538
This week we had a focused discussion on the current state of background error covariance and Var development.
Benjamin Menetrier presented details on the update in SABER (https://github.com/JCSDA-internal/saber/pull/145) that introduces “saber blocks” and improves separation of concerns within saber, as well as future plans on the saber development. One of the future developments is generalizing saber to estimate covariance functions other than GC99, including peaked functions, and functions with negative lobes. (20211115_Benjamin_saber.pdf)
Marek Wlasak presented um-jedi static B status, and raised questions on the file format for background error covariance statistics, and workflow for the background error covariance estimation. (20211115_umjedi_staticB_status2.pdf).
Dan Holdaway presented fv3-jedi static B status. (2021-11-15 JEDI Algo Meeting_DanHoldaway.pdf)
BJ Jung presented mpas-jedi static B status. (20211117_staticB_mpasjedi.pdf)
Guillaume Vernieres presented soca static B status. (jedi2-soca-static-B.pdf)
JEDI 4, Yannick
Testing some new ideas for EWOK in QG, L95
CRTM, Ben J
- Patrick working on PR adding Parameters subclasses for CRTM AOD, improved interpolation in coefficient files, working on tutorial for generating transmission coefficients
- Cheng working on optimizing number of streams used, especially near clouds
- Ben working on CMake build without ecbuild/autotools
- Isaac extending hydro model for non-spherical particles -- more cloud tables with different particle shapes. For microwave and IR. Will help for polarization in CRTM3
- Work on UV transmittance capability in support of CRTM 3
- Work on simobs application via UFO
OBS 1, Ben R
- PR in for conventional obs error inflation
- Work on buffer decoder for GNSSRO
- Work on three-cornered hat obs error estimates
OBS 2, Francois V
- Travis has written a draft of the specifications for the binning of diagnostics files (BESPIN). The document is under review. As a demonstration, Travis is planning to run BESPIN on AWS to process the HofX files generated during the JEDI-GDAS experiments earlier this year.
- Voila was deployed on a EC2 t2.micro instance to create a prototype web site application for JEDI diagnostics. The website is currently using the jupyter notebook that directly accesses HofX output files from a JCSDA S3 bucket. Rachel is now working on sprucing the website.
- The PR #1407 on the addition of the LinearForecast application in oops is under review.
OBS 3, Ryan
- Greg made significant progress on buffer to IODA v2 converters
- Ryan progress on IODA conventions document and tables
- will be posted on website
- Working on request to clarify UFO filter order of execution. Considering specifying pre vs post filters more clearly, applying filters in yaml order.
- New repo jckit will be added to bundles soon, to contain various pieces of utility code
LAND, Andrew
- Still seeing some unexpected behaviour with the FV3-JEDI CI - one of the builds was failing, unrelated to anything we’ve been doing - which was updating a test (PR#285).
- On the WRF-Hydro side, Everything is up-to-date with the latest OOPS changes, and are waiting on the SABER Blocks PR to merge in that.
- Thanks to Maryam we’ve been switched from Travis CI to CodeBuild, and that PR is merged.
- Soren has managed to get a 80% speed up in CONUS H(x) experiments, so calling that good for now and moving on to some other developments
- Had a good meeting (11/8/21) with Sujay Kumar (NASA Goddard) and Jerry Weigel (USAF), updating them on what we’ve been doing, and discussing some outstanding tasks related to developing a converter for the new USAF-Snow and Ice product, and some preliminary soil moisture obs operator work
CONSTITUENTS, Jerome
- Work on static B for aerosol by Andy Tangborn, waiting for PR soon
- Work on Aeronet AOD PR to update converters
- Work on strategy for providing a vertical coordinate for the averaging kernel operator
SOCA, Guillaume
- Public release (Nov 4)
- Diagnostics: bespin development (Travis)
- Finishing up the refactoring of the marine ioda converters (Hamideh, Kriti)
- Investigating the use of a 6 hour DA window for the ocean (All)
- Reviving the coupling repo (G)