Andrew Fox, Two items for land:
Have the August 2021 in-situ snow and SMAP snow moisture observations in R2D2 thanks to Eric. There has been some back and forth with NOAA colleagues about how to bin the SMAP observations to ensure we have everything in the 6hr files, but I think we have it sorted now. Have tested out H(x) yamls with the c768 backgrounds - so ready to test with EWOK. As there are so few none-zero in-situ snow observations we may want to bring in the IMS snow cover observations - but this requires a lot of preprocessing so not sure it is worth it at this stage.
We actually have WRF-Hydro working with the develop branches of OOPS/IODA/UFO for the first time since the getvalues refactor started. We did see some change in behavior with the background check filter - which probably means we weren’t using it right before, as I don’t think there has been any changes in that filter in the last couple of months? Translating observation names to variable names when using identity operators seems to be more complex than it should be. I think that there has been some talk of making this more straightforward with Vader? Is that right? I think it will be required before we bring in the standardized observation names.
Ryan Honeyager
OBS3 notes:
BUFR
A new version of NCEPLIB-bufr has been released. This unblocks Ron's longstanding feature/query branch in ioda-converters. Ron's PR is large, so we plan on holding a review meeting. We want to try this as a strategy for speeding up reviews of large contributions. Once approved, we also need new software stacks and containers with the new BUFR library, but we expect the infrastructure group will have addressed this by then.
Chris question on YAMLs
We merged in a PR from Chris Thomas on Monday that changes the behavior of VertInterp. Users can now specify whether they want linear or log-linear interpolation. Chris and Anna asked if this option should eventually be made mandatory, and after discussion the OBS group agreed. We will post to the model discussion board in advance of making that change. However, since we expect that downstream YAMLs would need only minor changes, we wondered if there was good CI support for building and testing *all* of the models? If not, then is adding this capability important for the broader group?
Test file cleanup
The obs team has started planning an internal review sprint where we will go through ufo and ioda-converters to clean up our test data and YAML configurations. We will remove duplicate tests, reorganize our test data folders, and will push a few tests to higher test tiers. Next week we will meet with Dom and Maryam to discuss how the test tiering can better work with our automated build pipelines. The internal sprint is scheduled for the week of June 8.
Ben Ruston
notes from the OBS team and OBS1:
- UFO/IODA-converter cleaning sprint (week of 06Jun). Address ctests remove deprecated, identify largest files help to reduce size.
- have been able to use prepBUFR for radiosonde and are beginning to compare this with LDM (GTS proxy) feed for these vital observations
- continue to work with 3 cornered hat (3CH) observation error estimates for GNSS-RO focusing on COSMIC-2 at present
Dom, JEDI1
1. spack-stack development
- Expecting code freeze for first release by Jun 3, 2022, then testing and finalizing documentation
- Will give a demo and/or tutorial in June on how to install and use spack-stack
2. jedi-stack updates
- Added atlas-0.29.0 and installed on all HPCs and containers
- Exception: Discover (already using spack prototype)
- fv3-bundle builds and runs with new version of atlas on Hera and Discover
- mpas-bundle builds and runs when using atlas-0.29.0 with eckit-1.16.x and fckit-0.9.2
- mpas-bundle builds and crashes when using atlas-0.29.0 with eckit-1.18.2 and fckit-0.9.5
- Reason to be investigated, Dom has seen similar crashes in the eckit MPI init on other platforms in the past with eckit-1.18.2 (e.g. Discover, therefore the move to spack-stack)
- See https://github.com/JCSDA-internal/mpas-jedi/issues/730
JEDI 2/3
Want to draw people's attention to the upcoming changes to the Atlas interface methods as it will affect all models:
https://github.com/orgs/JCSDA-internal/teams/jedi-models/discussions/71
Hope to merge this change the week of June 13, giving models a little more than two weeks to adapt.
Jerome Barre, Composition
Progress on getting aerosol observations into R2D2
CRTM -
Had meeting to discuss new release. Discovered some bugs and fixed them.
Better snow emmisivity table being worked on. Should be released with v3.0.
Miscellaneous
Dom Introduced new JED1 team member Anubhav to the group.
David Simonin warned that he found a situation where the obs bias correction was getting lost. Will enter an issue.
Chris Snyder introduced a discussion topic about certain obs types that consume a lot of memory, causing his team to try tricks to preserve memory. Nancy mentioned she is running into this as well.
Ben said an R2D2 query is want we want to have. (Planned, but not available yet.) This would allow a reduced set of obs in the file.
Yannick suggested possibly using more nodes, because that’s what operational systems do.
Jake Liu wants to know if we can remove obs from memory that have been thinned. Yannick said that’s not that easy. Would rather wait for R2D2 to be ready. But there seems to be some interest in finding a temporary solution.
Ryan suggested that IODA functionality could maybe be leveraged better to help with this.
Ben is going to put together a meeting on this to discuss further.
Guillaume mentioned that interpolation is not working for SOCA recently.