Infrastructure (Dom Heinzeller)
R2D2:
- First beta testers (Clementine, Yannick) will be starting today/tomorrow
- Draft PRs for R2D2, EWOK, fv3-jedi in place
- Eric and Dom will work on a timeline for transitioning, will be communicated early next week
IODA:
- Removal of obsframe, required for parallel reader, is underway
- This should solve the (majority of the) memory issues we've seen with IODA
- IODA developers forum (bi-weekly meeting) started Tuesday this week
spack-stack:
- Preparing for next release in early March
- Use unified environment on HPCs and AWS Parallelcluster, possibly also on macOS/Linux (alternatively skylab)
- Improvements to CI testing and Narwhal+GNU port are coming
- Build error on Hera when building crtm in bundle as opposed to taking it from spack-stack (linker step fails with missing link to math library)
Misc:
- Ashley started last week and is currently ramping up
- As part of her induction, she will provide a first write-up of the "Onboarding process WIKI" (in jedi-docs repo -> Wiki)
- New set of organization-wide Github labels were rolled out yesterday (Feb 15), these can be seen at https://docs.google.com/spreadsheets/d/1uips6rDepQowmeNlPIxuozX42qdbGpQ0cvj-yGy_z0w/edit#gid=1548731625 or by going to any JCSDA-internal repo (except archived) or JCSDA/ufs-bundle
- But: old labels still in Zenhub - will look into it
QA:
Travis noted that some old labels are still showing up, it might be caused by archived repos and Dom will look into it.
Algo ( Anna Shylyaeva)
- Variational bias correction discussion meeting on Feb 13: Junmei Ban (NCAR), Bill Campbell (NRL), Hailing Zhang (JCSDA) presented on their results with cycling with VarBC (mpas, neptune, fv3-jedi). Two bugfixes were issued in the last month, and tested: bias correction coefficient error covariance output (https://github.com/JCSDA-internal/ufo/pull/2616) andcomputing/applying covariances and preconditioner at first outer loop (https://github.com/JCSDA-internal/oops/pull/2038).
- Integrating fv3-jedi + ufs-atm forecast task in ewok, PRs in review. (https://github.com/JCSDA-internal/ewok/pull/562)
- Update to ewok yamls/suites to specify variables in top-level experiment yaml to simplify maintenance of different fv3-jedi-based experiments. (https://github.com/JCSDA-internal/ewok/pull/553)
- An option to generate ensemble for HTLM (from B matrix) as part of application (instead of reading from file) (https://github.com/JCSDA-internal/oops/pull/2052)
- Christian and Benjamin working on comparing B matrix re-trained for fv3-jedi for BUMP with new features and using GSI B statistics in BUMP.
- An option for BUMP to use GSI covariance parameters. (https://github.com/JCSDA-internal/saber/pull/439)
- New application to output fields to lat/lon grid using a generic postprocessor. (https://github.com/JCSDA-internal/oops/pull/2056)
- Benjamin and Andrea Piacentini from CERFACS added: different communications options for 4DEnVar (some experiments show significant speed-up) (https://github.com/JCSDA-internal/oops/pull/2033)
- Benjamin and Andrea Piacentini from CERFACS added: an option for time localization in 4DEnVar (note: only works for the oops-level ensemble localization, not the saber-block ensemble localization). (https://github.com/JCSDA-internal/oops/pull/2060)
- Discussions in preparation for B matrix training code sprint on generalizing the ErrorCovarianceTraining application for different error covariance models.
- Saber interpolation block progress; remaining discussions on the halo exchanges. (https://github.com/JCSDA-internal/saber/pull/408)
- Bugfix to random fieldset generation (different across different MPI tasks); variable number of levels in the quench fields
- Steve V working on model naming convention adoption process, and passing the model constants to vader
- Vader cookbook passed as an argument to vader ctor and fully controlled by model interfaces. (https://github.com/JCSDA-internal/vader/pull/140)
- JJ Guerrette (NCAR) added an option to LETKF to specify increment variables that could be different than background variables (https://github.com/JCSDA-internal/oops/pull/2064)
- Merged the first version of particle flow filter (developed by Femi Kolade, Peter Jan van Leeuwen, Clementine Gas). (https://github.com/JCSDA-internal/oops/pull/1859)
- First JEDI-EDU tutorials plan to be beta-tested at CSU class (Peter Jan van Leeuwen).
Q/A
Ron asked about testing VarBC in the context of aircraft observations and how much work might be needed to accomplish that.
Hui chimed in that that was on their to do list and that there is a directory in UFO with predictors that might serve as an example of the work required.
Dom also mentioned that cycling with UFS atmo is progressing and some code changes might be needed and he will be reaching out to the code owners.
SOCA (Travis Sluka)
* ocean color forward operator (OASIM) has been merged into UFO. This is just a "plumbing" accomplishment, we still need to make sure it is actually working (it probably isn't)
* we are still recovering from ioda v3 changes, but should be back to running science experiments for hybrid-envar soon.
Q/A
Anna mentioned that Skylab-Marine was having some failures and Travis said he would take a look.
Compo (Jerome Barre)
Progress has been made on non-cycled 3dvar aerosol experiments.
There have been some bugfixes
Q/A
Tom and Jerome will discuss what is needed for cycling with aerosols offline.
OBS (Hui Shao)
(1) Building generic vertinterp: given some discussions through github and side meetings with some of the developers, core team and in-kinds agree to build a more generic vertical interpolation operator to incorporate needs of different applications. NOAA PSL is currently merging a land vertinterp with atmverinterp. EMC Cory agrees to lead a followup work, probably a code sprint in the new future to merge all other interp routines: marine, atmvertinperplay, and snow (to be added). OBS will reach out to help coordinate this code sprint.
(2) VarBC: a few PR were merged to fix the issues identified for using variational BC. OBS will continue to validate VarBC preconditioning, focusing on NOAA method. We'll start with the test data previously provided by Haixia Liu (EMC), generated via GSI. Then move on to more generous testing. Also OBS is working with JEDI to add VarBC cycling into ewok workflow.
(3) OBS set up a meeting with Cory (EMC) and Dan (GMAO) to coordinate obs monitoring efforts. Will start with inventorying the existing capabilities
(4) Start working with CRTM team to test CRTMv3 in terms of UFO. We are trying to build compatibility so that we can continue to support CRTMv2.4, while bringing the new capabilities from v3.
CRTM (Ben Johnson)
Working on V3 in UFO a bit of manual merging but is close.
There are some tolerance differences and a manual coefficient change will add space based radar
v3 not fully ready wait until internal testing is complete.
Should be a microphysics update to address issues with a sea mask variable and working on code to get particle radius distribution . We are seeing reasonable radiances over land with this progress
Visible not tested yet and writing a CRTM overview paper. Should be out soon for citation. Cheng is working on Texas A&M aerosol for dust and looking at snow emissivity model from nick to do a better job with snow emissivity .
Emily noticed that emc coeffs produce radiances too large switch to boxcar coeffs and working to figure out what is going on.
Patrick is chasing down inconsistencies in legacy coeffs and comparing to new coeffs.
There is a unit inconsistency in some files and need to carefully step through to check that they are correct.
Cheng mentioned that in v 2.4.1 there will be a minor code struct change in relative humidity
Coordination with obs team about which units should be used
EMC:
Working through own UFO evaluation and making progress and wants to work on aircraft bias correction soon.
Ron mentioned planning on reaching out to coordinate with EMC on this and reach out to JCSDA too