2022-03-17

Working on rewriting the UFO operator for the trace gases. This is a complete new redesign as the current coding is not correct scientifically and doesn't fit the recent plans for IODA evolution.
A series of PRs are planned. Currently working on a correct partial column calculation.

Had a meeting with Colorado University. They agree to provide background GEOS-Chem fields for our future EWOK experimentations.

Attending the CEOS meeting this week.

- soca is wrapping up an initial superobbing tool for our current needs (it turns out 1 Billion viirs sst obs a day are a little too much). Current limitations are that it doesn't work with multichannel data, or observations with a vertical component. We will schedule follow up discussions on how to make this tool generic for everyone, and where to move it.

UFO output cleanup is coming into focus

• Too much verbosity in log files
  • Neill B started work to put in verbosity controls in the oops logging system
• Need to add summary tables at the end of the operator
• Jo W's work on flow of variables through the system
  • UFO PR 1896

Skylab experiments that are under way

• Hybrid 3DVar
  • Have seen issues with ensembles, but have some options now to try
• Looking at performance
• NCEP BUFR converters for cycling experiments

UFO development

• In-kinds are focused on data processing (conventional and radiance)
• Yung Chu (NASA) is working on bias correction

• There is a new release of NRT Observation Impact (FSOI) website: https://ios.jcsda.org/
  • Includes a total of 7 centers, and 58 platforms
• Work is in progress to add:
  • ozone mixing layer
  • MLS platform
• Work is in progress to make available in Skylab a weather app for NRT processing developed by Eric L

We held a code sprint last week to apply the JEDI Data Conventions. Many thanks to everyone who attended. This is part of a series of sprints targeting ioda, ioda-converters, and ufo. We aim to apply the UFO changes sometime in summer. Until then, our results will reside in a long-lived feature branch. We have an automatic process that detects when new converter development will introduce conflicts into the sprint branch, so the impact should be small.

This sprint was somewhat tricky because we are applying changes to all of our converters and need to coordinate updates to around 80 test files in our repository. We had to made changes to both individual converters, to ioda, and to the converter Python library simultaneously. As of yesterday, we added twenty variables to the JEDI Data Conventions, we merged 18 PRs in ioda-converters, and 17 in IODA. 14 converter PRs remain in review. For anyone listed as a reviewer on the pull requests, please review since we have a backlog. I am aiming for the reviews to be done throughout this week and next.

As a byproduct of the sprint, a small group will meet some time next week to gauge feasibility of the ioda-Fortran API. This is needed for several Fortran UFO operators and one converter.

Outside of the sprint, yesterday we had the general OBS3 meeting. Not much to report there, except a discussion originating from Neill Bowler about how we need to improve our logging capabilities in JEDI. He made pull requests in oops, eckit, and fckit adding a "verbose" output stream. The eckit developers wrote back asking us to separate logging channels from severity levels. Updates to the logging system could be an early implementation goal for jckit. Referred to JEDI1 for further consideration.

JEDI 1:
- spack-stack updates:
    - Expecting to be rolled out by the end of this month
    - Builds on gaea, macos, orion, cheyenne, linux
    - Compiler toolchain problems fixed, requires a few bug fixes to ECMWF software (fckit, atlas)
    - Need to bring bug fixes back to shumlib - https://github.com/metomi/shumlib/pull/1
    - Hera and jet: spack mirror whitelist request approved and implemented
    - Public versions of transi and fiat now available, will replace faux and trans
    - Next to test on S4 and Discover
- No news on “Update C++ standard from 14 to 17?”
    - https://en.cppreference.com/w/cpp/compiler_support 
    - https://github.com/JCSDA-internal/jedi-discussions/discussions/30 
- Originally reported by Ryan for the obs team:
    - Issue with logging in oops: it uses eckit but not all of its functionality
    - We need to revise how we handle logging in general
    - Neill's originall PR to eckit https://github.com/ecmwf/eckit/pull/43
    - https://github.com/JCSDA-internal/oops/pull/1673
    - Initial PR for jckit by Ryan in Nov ‘21 - good place to start: https://github.com/ecmwf/fckit/pull/19
- UKMO will be working on updating the atlas feature/um branch and merging it into develop
    - Then can use the same version of atlas across all bundles
    - Hopefully solves issue with atlas in jopa container (undefined references)
- crtm develop branch is now using code build instead of Travis CI
- crtm jedi release branch and develop branch differ in their cmake target definitions
- crtm error:
    - SpcCoeff_ReadFile(Binary)(FAILURE) : Error reading channel data. I/O past end of record on unformatted file
    - CRTM_SpcCoeff_Load(FAILURE) : Error reading SpcCoeff file #1, /glade/work/guerrett/pandac/fixed_input/crtm_bin/amsua_metop-b.SpcCoeff.bin
- Ioda writer improvements are work in progress, test with 8 MPI tasks, need to scale out
    - Hope to have PR this week

JEDI 4
- Began work on enhancing the JCSDA USAF demo web app by integrating R2D2 fetch of images from S3
- Adding modifications to EWOK/FV3-JEDI to allow some tasks to execute on the log in nodes (no network connection on the compute nodes)
- Solving issue where a seg fault occurs during plot rendering using matpotlib on Orion in EWOK
- Set up temporary jedipara venv on Cheyenne for utilization by BenR
- Currently processing the month of Aug 2021 of radiosonde raob data from BenR on Orion into R2D2.  

JEDI2/3 update:
- All fv3-jedi were updated to use 127 level backgrounds (Clementine). If you work with develop in fv3-jedi, make sure to update both fv3-jedi and fv3-jedi-data repositories.

- A SABER PR https://github.com/JCSDA-internal/saber/pull/185 (Benjamin) was merged as mentioned in https://github.com/orgs/JCSDA-internal/teams/jedi-models/discussions/63. The regeneration of NICAS files is needed, but it is also possible to update NICAS files with a python script attached to the discussion thread.
- The multi-component BUMP (Benjamin) is ready. It can estimate and apply "peaked" functions. The main PR and the companion PRs are all opened (except LFRIC). BJ is testing the branch with MPAS at full resolution, Benjamin plans to test it with FV3-JEDI at full resolution too.
- Upgrade of the quench pseudo-model to use the "ErrorCovarianceTraining" and "Dirac" applications (Benjamin). Will be useful for the spectral and GSI SABER blocks.
- GetValues refactoring aimed at speeding up the interpolation for H(x) is nearing completion (Yannick, Francois H, all model interface teams). See PRs: https://github.com/JCSDA-internal/oops/pull/1680, https://github.com/JCSDA-internal/saber/pull/193, https://github.com/JCSDA-internal/ufo/pull/1930.
- Work on porting GSI covariances to SABER (Dan): an interpolation SABER block is added, as well as the block that will use GSI covariances (linked as a library): https://github.com/JCSDA-internal/saber/pull/184.

• Updates for coefficients have been made for:
  • Tropics polarization
  • GOES18
  • JPSS2
• Cheng will have a PR soon containing IR emissivity coefficients
• Working on proposal to the ISSI in regard to coordinating innovative methods to assimilate visible radiance data more efficiently
  • This forum has a very broad audience (Europe, US, etc)
  • Proposal contains lots of development work of forward operators
• A code sprint is being planned to address issues with hard coded metadata in Fortran
  • Land surface type encoding is an example