JEDI1 and JEDI4 update (Dom Heinzeller):

  • Spack-stack work continues, https://github.com/noaa-emc/spack-stack 
       - Currently building on macos, orion, cheyenne, gaea
       - Challenges:
           - Compiler toolchains, especially with ECMWF software
           - Need to bring bug fixes back to shumlib
           - Air-gapped systems
           - The sheer number of packages and their dependencies
       - Solved:
           - Choice between lua or tcl modulefiles
  • Update C++ standard from 14 to 17?
       - https://en.cppreference.com/w/cpp/compiler_support
       - https://github.com/JCSDA-internal/jedi-discussions/discussions/30
       - Old cray at Met Office is a problem, but the machine will be replaced around the end of 2022
       - Need to follow up with Navy about Narwhal
  • Negotiations with AWS: implementing CI/CD pipelines to test code changes across repositories (aka “don’t break soca again”)
  • Current jedi-stack:
       - On discover, fv3-jedi crashes with intel-2021.3.0 - but works on hera
       - Recent update to eckit: https://github.com/ecmwf/eckit/pull/42 , we should try this - if it works, ECMWF can tag it
  • IODA
       - Fixed issue with ioda reading a file with zero obs (structure is valid, but nlocs == 0) and crashing.
       - Added warning if file contains no obs
       - Ongoing work: Removed the framing mechanism from ioda writer. This is a step toward completion of the writer improvements project. Anticipating PR this week for the writer improvement project.
  • EWOK/R2D2
       - Updated EWOK2 installation on Orion, installing EWOK2 on S4
       - Updated r2d2-data/R2D2_EWOK2 database in jcsda-internal
       - Ongoing: Update new R2D2 and EWOK documentation and tutorial
       - Ongoing: Work with BenR to select the new EWOK2 demo date range, then download c768 GFS backgrounds and process into R2D2 on Orion.
  • Testing
       - Tier 2 tests off for fv3-jedi Intel CodeBuild. We run them in nightly runs now - https://github.com/JCSDA-internal/fv3-jedi/pull/406 
       - New CDASH server using AWS x2large instance
       - Added Python libraries to jopa Singularity container
       - Jcsda-bot credential (GH token) is now used in CodeBuild and AWS instead of Maryam’s
       - Ongoing: Issues building dependencies (rebuild container should solve this)

JEDI2 and JEDI3 update (Yannick Tremolet):

  • Update on git value of observations. Will meet with people working on interfaces later today. Organized it in a way so that we will not have to touch the model interfaces again. Initial merge does not have all of the optimization. Only the interfaces that are changed now. 

OBS1 (Ben Ruston):

  • AOP planning, we took the opportunity in the JCSDA internal OBS team meeting to go through the ZenHub board and now have cleaned out all the "New Issues" all being assigned to in progress, backlog, awaiting feedback etc.   This gave us a good idea of what is being actively worked on and what is being planned.
  • Have been examining new datatypes, in particular PlanetiQ GNSS-RO data and have established a near real-time feed of TROPICS data which we are pursuing to begin preliminary evaluation.
  • The aircraft data, we have established a new data feed as the NOMADS "aircar" files had stopped 06UTC 21Feb2022 and continues to be unavailable.  The new data feed from LDM is "raw" data and does not contain pre-QC which is done upstream currently of what is distributed by NOMADS or used in the prepBUFR for instance so should be a good example to help verify quality checks in UFO.
  • Lastly, a number of users have reported the UFO output is quite noisy.  We need to re-evaluate this and more actively use the DEBUG, TRACE, etc flags for output and minimize this to more critical information.  Related to this a succinct statistical summary of the UFO operation performed should be added as well to the output.
  • Discussion: Issue with verbose stream for log files is a good idea. Debug should not stay when they merge the code. We should have this information in normal output and debug is for debugging. No debug in develop. (Yannick Tremolet)

OBS2 (Francois Vandenberghe):

  • We are conducting an evaluation of the GNSSRO observations from the commercial vendor PlanetIQ. The stats can be seen in near real time on the JCSDA demo web site https://demo.jcsda.org.
  • Scripts for aircrafts, radiosondes and GNSSRO on a map have been updated to work with IODA2. Scripts for plotting vertical profiles of GNSSRO occultations have also been developed.

OBS3 (Ryan Honeyager):

  • We have a IODA-converters code sprint next week on the afternoons of March 8 - 11. Thanks in advance to everyone who has volunteered. You will receive calendar invites soon.
  • I gave a presentation outlining the sprint yesterday at the OBS3 meeting. In this sprint, we will change variable names and units to match the JEDI Data Conventions.
  • Additionally, we want to switch over to a better representation of dates and times in our files. We are focusing only on ioda-converters for now. Changes to UFO and other repositories will occur later.

CRTM (Benjamin Johnson):

  • Patrick has fixed a number of bugs in the CRTM OpenMP-over-channels parallelization that occur for edge cases, like computing only a single channel for an instrument.
  • The rest of his time went into IASI-NG validation calculations for including heavy molecules in the absorber set. Benjamin Johnson has created a preliminary
  • Cheng has been working on adding RTTOV aerosol optical properties to the CRTM
    and preliminary work on adding netCDF I/O to the CRTM surface structure.
  • Discussion: CRTM bandwidth issues with GIT-LFS. (Benjamin Johnson, Maryam Abdi-Oskouei, Ryan Honeyager, Yannick Tremolet)

COMPO (Jerome Barre):

  • Working last week on FV3-JEDI interface for GEOS-CF. After that I am working running HOFX on GEOS-CF. Running into problems. The way the operator is designed, there will be changes needed on that operator. Maybe a week of work. The problem is that operator requires more variables than necessary.
  • Andy is working on bias correction.

SOCA (Travis Sluka):

  • No update

LAND (Andy Fox):

Lots of progress over the last few weeks towards putting together a cycling system for WRF-Hydro

  • A main component has been porting over the snow DA algorithm/approach from FV3 to this interface. 

  • Small amount of work to add a method to Geometry iterator to return model surface elevation, and was then able to bring across the elevation obs localization from FV3 pretty much as is. 

  • Potentially need to tune some of the length scales used in the two localizations, but it's already giving acceptable results at full resolution over CONUS using global model values. So a good example of “Joint-ness” I guess.

  • Other work has involved assembling a workflow using wrfyhydropy API to control model, and like everyone else, some YAML manipulation

  • Also had a PR for the basic IODA-converter we use, with a minor bugfix to deal with metadata datatypes

  • Also taken some code from the EMC land group for snowpack layering updates, and that only required minor modifications for our model geometry

  • Now, working on updates for the GetValues refactor

On the GFS side, meetings have fallen on holidays, so I’m a bit behind on current status. Expect a larger update on that next time

Questions and discussions:

  • Discussion: Super Obbing in UFO and SOCA. (Travis Sluka, Jerome Barre, Yannick Tremolet, JJ. Guerrette, Ben Johnson, Joanna Waller)
  • Discussion: PyGrip and eccodes. (Cory Martin, Travis Sluka, Dom Heinzeller)
  • Discussion: NetCDF C++ interface. (Steve Herbener, Ryan Honeyager, Yannick Tremolet, Dom Heinzeller, Tom Aulinge)



  • No labels