2024-02-15 JEDI Weekly Notes

Opening announcements (Yannick)

• Noticed that running JEDI on Intel it takes 3x more memory than GNU
• Discussion on the coordinated merge - proposed to have someone with merge permissions across all the repos (Dom/Yannick) to do the merge at the same time

INFRA (Dom)

• IODA - bufr backend in work, standalone app is in progress, and truncate obs space
• EWOK/Skylab - observation ingest suite, MPAS end to end forecast implemented 
• CI - instructions provided to set up our CI test environment 
• Spack-stack - installing an environment on discover and pcluster to support GEOS, testing spack-stack on discover milan, issue with latest version of hdf5 and working on that fix

ALGO (Anna)

• Clementine, Christian, and Nate presented at AMS
• Merged PRs that update bump to add check for resolution
• Work on background error covariance training suite in EWOK
• Conversations happened on using vader for changes in background error covariance
• Discussion with MET Office with handling halos in direct test
• EWOK/Skylab cleanup and merges
• Resurrect model variable convention effort to implement across the code base in the next couple of months 
• Work on snow 2dvar 

OBS (Francois)

• VarBC sprint is being finished and adding generic methods to be demonstrated on aircraft
• IODA convention for VarBC predictors are being sorted/finalized
• Bias coefficient file format will update to IODA obs format
• Ocean surface winds work - single GNSSR converter used working with multiple sources: CYGNSS and private vendors
• Developing scalar wind operator: GNSS-R and passive microwave (SSMIS, AMSU2, SMAP). We tagged our MetOffice and NOAA partners
• Read the docs convention page is up to date with the current obs space files
• GOES-R JEDI GLM operator for lightening is being worked on and using a more generic operator - follow up used in storms, pollution, graupel, etc  
• The OBS Team Contributed to the ObsSpace reduction in IODA: a new function, ObsSpace::reduce, that will remove locations indicated by a vector and then shrink the memory footprint of the ObsSpace accordingly.

CRTM (Ben J)

• CRTM v3.1.0 is released with space based optic radar and CMAKE build system
• CRTM log size is reduced and includes quiet options, still work in progress. Please keep reporting log issues
• Fixed CRTM 2.4.1 coefficient download in UFO with Fabio
• Prepare AOD code sprint w/ COMPO team
• Visible reflectance work/addition to CRTM and UFO capabilities
• Generic optical properties user mode/ interface for CRTM instead of using provided tables (providing optical properties but not jacobian work.. yet). Will be ready to merge this week/next and this feature will be transparent to black-box users.
• CRTM AI simulations for IR under evaluation (Lucas Howard) -- generating radiance and jacobians, with decent fit, will train on additional data. The goal for a CRTM AI frontend for quick operations in a couple of years
• Thread safety discussion and tech debt reduction will look at this in CRMT
• Continue support on CRTM r v2.3, v2.4, v3, v3.1

COMPO (Jerome)

• We noticed a compiler dependent bug with the minimization that still needs attention. GNU is fine but not intel
• We are working towards cycling our skylab experiments
• Working on having a high resolution minimization c360 with the skylab trace gas TEMPO demonstration
• I have made good progress on BUMP nicas global and local file generation for composition thanks to the help of Benjamin M, BJ, Travis and Anna. I realized recently that nicas global are resolution independent and potentially grid independent with model level conversions. We could share one nicas file from one model to another and this also could facilitate implementations of outer loops in the future.
• I am working closely with Hongli Wang at NOAA to implement aerosol variable transforms in Vader. This opens up possibilities for simplification for the control vector.
• Planning a week of effort with EMC and CRTM team to clean up the AOD operators and update examples for Bias corrections and QC.

SOCA (Travis)

• Kriti is working on adding UFS marine forecast to skylab, which was working but is broken with the latest version of UFS. More discussion is needed for UFS tags and versions
• Cleaning up of non-saber items (ie linear variable changes) to work with SOCA then move to saber outer blocks (diffusion saber central block, parametric ocean standard deviation block)
• SOCA staff will be out next week at AGU ocean science meeting
• SOCA tutorial is in the works for a soca workflow

EMC (Cory)

• Continuing progress on snow DA and global aerosol DA
• Gullaume is testing SOCA operators and planning on cycling for a month or two. Looking at adding Saber block with torch library (see PR in SOCA)

GMAO (Ron)

• Completed 3rd round of observer station ??
• Need to sort out some consistencies between GSI and JEDI
• Cycled system work being done by Ricardo 

Met Office (Phil)

• More than 3/4s done with UK obs porting, effort to increase obs coverage for 3dvar
• ODB write functionality is complete
• Phil will be coming to JCSDA working with Steve H on refactoring of the obs space for a couple of months

NRL (Sarah)

• Looking into various linear solvers - settled on DRPCG
• Paused at Skylab 6 and using repo checkouts from that release without any changes
• Model upgrades will be coming

MMM (Jake)

• MPAS used to use DRIPCG, but switched to DRPCG too and noticed faster convergence
• Work on adding VarBC predictor for cloud impact parameter 
• Found a problem (maybe related to CRTM?) using all sky radiance DA, needs brighten temperature parameter assuming clearsky, reported bug to Ben and cloud fraction discussion
• NOAA hyperspectral IR sensor work