JEDI infrastructure

  • Spack-stack 1.9 was target for this month; still waiting on some upstream package issues specially ESMF is one of the big block which is needed for UFS and Neptune. Spack-stack 1.9 will be tested with JEDI inside and outside JEDI to make sure the support and functionality is full there and ready for everyone.
  • Also working on CI test. Submitted a couple changes to the CI system. Device out of storage issue is still coming up, hopeful less. Please feel free to report in the infrastructure channel.
  • Continued work on JEDI workflow and SkyLab; testing R2D2.

 

 

JEDI Algorithm

  • Yannick: There is PR in oops from Benjamin, which is a quick fix to what broke saber.
  • There was a SABER PR related to background variable conversions and linear variable transformations
  • Christian: finalizing changes needed to enable the shifting of the end of the DA window between outer loops (for continuous DA) with a PR to follow soon
  • Clementine: has GEOS cycling now thanks to Ricardo's help, which will enable cycling EDA experiments.
  • Nate: PRs for doing multi-scale spectral localization have gone through a first round of reviews and are ready to a second



JEDI Interfaces and Optimization

  • Steve V. implemented get_db logic for the FrameCols OSDF ObsSpace container.
  • Steve H. was working to provide contrived observation receipt times for the continuous DA application. The idea is to fabricate receipt times from existing ioda obs files for the purpose of simulating the arrival of obs at different times for demonstrating and testing the continuous DA capability.
  • Steve V. is working on design changes for Linear Variable Changes.
  • François H. was testing improvements to the interpolator used as part of GSI-B in saber, for getting the increment to/from the GSI grid. Goal is to remove yet another copy of the old/slow/inaccurate oops interpolator in favor of something more scalable and accurate, testing is underway to verify behavior. Bonus: should enable tlnmc balances in GSI, will test this with Ricardo Todling.
  • François H. was looking into several bug reports from EMC and MMM regarding the behavior of the code in regional applications.

 

 

JEDI OBS


  • Jianjun (EMC)'s PR on cloud fraction over land is being reviewed. Mike (UKMO) and Jianjun have been working diligently to make this code generic. They have been making a good progress
  • EMC reported an issue related to the NBAM observation error inflation issue (not the generic error inflation). It had an issue during the EMC phase-2 test. Hailing has been working with Xuanli (EMC) and had a PR to fix the bug related to the inflation filter variable. Now this PR is being tested by EMC.
  • Sam continues working on GNSS-R ocean surface wind. She went to AGU and met data providers. OBS will continue to explore the use of GNSS-R measurements, potentially other observables as well
  • Ben and others from OBS had a meeting with io, planning cal/val activities related to its upcoming launch. We have a tropics converter in JEDI, compatible with tomorrow.ioobservations.
  • Also the ground TEC converter is ready after sorting out some decoding issues. Thanks for NRL to provide the descriptions of the data format
  • Last is about the ROPP source license. People can't obtain source code without a license, which can be obtained by going to the ROM/SAF website. Therefore, OBS has restricted access to the ROPP-test repo. 


 

JEDI CRTM

  • Working on CRTM 3.2 release. Hopefully get it done by the end of this mongth. Working on testing.
  • Met with tomorrow io, along with OBS team. Discussed details of radiances we are seeing from theiur sensos initially we were hoping to make tropics and some of their sounder similarly in terms of how we treat those. One of the key differences was how they treat the viewing angle. There are some inconsistencies. Will working with them to make sure get everything. Will have the latest data files to test.
  • Focusing on getting ready for AMS.



JEDI COMPO

  • Tuning the no2 DA system to solve the UTLS NO2 DA problems. Tropopause fix on UFO and localization tunning to avoid propagation of tropospheric increments in strat.
  • Looking into DA results in detail allows us to make some validation assessments of the TEMPO NO2 products: problems with the slant path in the morning and likely dependence with the solar zenith angle - this has been confirmed with colleagues at GMAO with their TEMPO validation work.
  • Found a bug in lat/lon values in analysis files from SkyLab experiments. This happens in a more recent build.
  • Start to test level 1 reflectance DA. Temporarily tested with simple background error, will update to EnVar with 40 members provided by in-kind. QC will be tested.
  • Freeing space on discover r2d2 and moving things to the css partition
  • Updated documentation on UFO column retrieval operator.



EMC

  • Tuning GFS V17; making runs and adjusting; tuning in the background; model of aerosals and marine; [laying around with lots of QCs. Some development for letkf for the ocean.
  • Had a couple of PRs in UFO; Jianjun contributed some filters for all-sky and precipitation.
  • For regional, finding bugs to do domain check.
  • Trying to clean up something in fv3-jedi; adopting standardized variable names and cleaning up technical debts; all variable names should be long names in yaml and you would not see all the different versions of names.
  • Jerome: can you check your analysis files for the coorinates? Maybe one of your PR messed up the lon/lat in the analysis files?
  • Ricardo: The 3d pressure is not updated correctly.


Met Office Michael Cooke

  • A very good meeting last week with Yannick and Anna to establish a development plan to improve EnKF capabilities – details will be presented in early February. 
  • We will start work to improve the OOPS I/O used by the covariance and the HTLM.  Use of NETCDF parallel I/O. 


Hernan A:

  • Experimented with global files in BUMP_NICAS to avoid recomputing the costly background error covariance training everything that the number of processes is changed for a particular high-resolution application.  Unfortunately, there is a parallel bug in the single global file approach.  The gathering (tiled-to-global array during writing) and scattering (global-to-tiled arrays during reading) of data require a particular packing with the appropriate MPI collective communications (i.e., MPI_GATHERV and MPI_SCATTERV) to reorganize the 2D/3D data in continuous memory (Fortran or C). As far as I can see, all the collective communications in BUMP are done with MPI_ALLREDUCE as a summation on the global vector.  The problem is that MPI_ALLREDUCE does not have an inverse like MPI_GATHERV is for MPI_SCATTERV.  The usage of these two routines is very tricky since, by default, it requires that all parallel tiles must have the same number of points, which is rarely the case.  Things can get very complicated with 3D slabs of data that are not continuous in memory.  I will post an issue and a pseudo-code for how to use MPI_GATHERV and MPI_SCATTERV.   The major problem is that BUMP is quite complex to modify, and any modification may break other capabilities.  I hope that Benjamin can use the pseudo-code to provide a solution.
  • Anna Shlyaeva: we've successfully used BUMP global files for snow 2DVar (with the caveat that it's 1 variable/ 1 level, and correlations are very simple). Benjamin has been improving on global files too in the past year
  • Hernan G Arango: Good to know. Yes, 2D is simpler to reorganize the global vector in continuous memory.  3D arrays are much harder, but I figure out a way to do it outside of the MPI library calls by packing the data correctly for MPI_SCATTERV and MPI_GATHERV.


  • I will continue to work on ROMS-JEDI linear variable changes. For flexibility, I convinced myself that we need to clean up and unify our metadataapproach and use the standard name (long name) everywhere, including YAML files: this is what SOCA is doing and the plan for FV3-JEDI. The advantage is that fields in the state and analysis vectors are more specific with standard/long names (as opposed to short names), and one can append additional attribute suffices for units, etc., which may be necessary for specific variable changes.
  • Specifying the tolerance for testLinearVariableChangeAdjoint() in the YAML file is desirable. Currently, it is hardwired to tol = 1e-10.


NRL Sara

  • Working on ozone assimilation
  • Working on forecast model update; going to catchup develop for JEDI; updating workflow to cilc
  • Will have a poster with progress on Falcon Neptune in AMS


NRL space weather: John Haiducek

  • Working on IODA converter to output IODA files; currently getting it with C++ codes; encountered a challenge with the geovals&latitides about the missing values. Hui wrote a script to remove fill values and truncates the array and got it digested in UFO.
  • Having trouble with Pyiri about the states. Need to make some changes to geometry.


NOAA boulder Ming Hu

  • Building MPAS-JEDI workflow
  • Testing regional domain
  • No labels