Yannick opened the meeting announcing the focused topic of the day, namely the new GetValues classes in oops introduced by Anna. He suspected that this may not last the full hour and we would have time at the end for other questions and updates (it turned out to last nearly the full hour, about 50 min).
The pull request for oops that will be referred to throughout the presentation is oops#514. Dan has not yet issued a pull request for his changes to fv3 but they can be found in the feature/getvalues branch.
Then Anna presented the following slides.
What follows is a summary - see the slides for further details.
Anna opened by announcing that the changes that she will describe will require refactoring of all model interfaces.
She then proceeded to review the purpose and the design philosophy of GetValues, which focuses on using GeoVaLs as an intermediary layer to keep the forward operators model-agnostic. Previously, GetValues was a method in the State class and its linear counterparts (GetValuesTL and GetValuesAD) were methods in the Increment class. This redesign will replace those methods with the independent oops::GetValues and oops:LinearGetValues classes. Anna emphasized that this approach offers a design symmetry between the GetValues (State → GeoVaLs) and ObsOperator (GeoVaLs → ObsVector) classes.
The new approach also offers more flexibility: see slides 6-7. In particular, it will allow for the possibility of using different GetValues subclasses with different Obs Operators. For example, the CRTM operator could use a GetValues class that works with SimpleLocations or one that works with SlantPathLocations. Or, different GetValues subclasses could handle vertical and temporal interpolation differently.
Rahul asked if the call would be the same for different GetValues classes. Anna said yes - you could specify what subclass(es) you want in the yaml and an ObserverFactory would allow you to dynamically create the combination you want to use of GetValues, ObsOperator, ObsFilters, etc.
Steve S asked if the new GetValues classes would be generically implemented in JEDI or would models still be able to define their own custom GetValues classes. This prompted a discussion about the tradeoffs between a generic implementation and a custom implementation. We plan to implement both in the future. We plan to move more of the GetValues computation into C++ and make it more generic with the introduction of standardized interfaces and data structures. But, we plan to allow for modelers to override the generic implementation with their own custom implementation, if desired, in order to improve efficiency.
Jake then asked if the SlantPath option for Locations is available now. Anna responded no - that's still in development.
Anna then compared the new GetValues interfaces to the old interfaces and discussed what needs to be changed on the model side (slides 8-11). One difference that she highlighted (and that Dan later elaborated on) is that the new class approach creates a GetValues object for the entire time window requested, as opposed to the previous approach of creating Interpolator trajectory objects for each subwindow (see Dan's slide 2). This can improve efficiency since the interpolation weights can be calculated just once in the GetValues constructor and then applied over the entire time window.
Another difference is that the Variables class is no longer explicitly passed to GetValues. Instead, they are obtained from the GeoVaLs class that is passed to the compute() method, to minimize the chance of inconsistency (variables passed not matching GeoVaLs). Also, the InterpolatorTraj class has been removed - this information is now contained within the LinearGetValues class.
Chris H asked how close this is to being finalized - can models start to implement it immediately? Anna and Dan said the interface is not likely to change so yes, modelers can begin to implement it immediately.
Then Dan presented the following slides - again, consult the slides for details:
Dan started with an overview of what the corresponding Fortran types look like in fv3-jedi, to accommodate the new C++ classes. Then he discussed the two interpolation options that are currently implemented for fv3-jedi, namely bump and unstructured interpolation (unsinterp), and how the new time mask in ufo::Locations can be applied.
Dan then discussed the Variable change, which has been removed and isolated from the new GetValues classes. Now the change of variables can be called first, operating from model grid to model grid. Then the desired variables can be interpolated onto the observation grid with a call to GeoVaLs::compute (or fillvalues - the names are still being discussed). Dan noted that the new GetValues class is now generic with respect to the variables. He also highlighted the new FieldsMetaData class, which is used to define and managed the large number of fields that are now supported by UFO, and that UFO might potentially request from the model.
Marek asked whether the locations can be accessed from C++ through GetValues. Anna responded yes. Marek added that some ufo components are currently accessible from Fortran but not from C++ - more interfaces are needed. Yannick responded that the plan is to completely refactor GeoVaLs so that more of data is stored in C++ and accessed from Fortran. That would solve that problem. Anna added that we'd like to do the same with Locations.
Jake asked Dan to clarify how the new approach minimized redundancy, using crtm as an example. Dan answered that the separation of variable transforms from interpolation (GetValues) allows you to obtain any variables you need just once and then apply the interpolation to different observational grids as needed.
Steve S asked if the 4D state is now stored in the Variable change objects. Dan said no - only a 4D variable change object is created. This may contain and store some 4D fields, but not at the full model resolution.
After Dan's presentation and the subsequent questions finished, Yannick asked if there were any other issues people would like to discuss, either relating to JEDI or to the logistics of working from home.
Dan warned model groups about recent test failures in fv3-jedi that were traced to changes in some ioda observation files. All modelers should check that their tests still pass and, if not, you may need to update your reference files.
Steve S asked when the GeoVaLs changes will be going in. The JEDI core team responded that we would like to implement as soon as possible but not before models are ready. Anna added that the code is already mature but we were waiting to announce it until we had working examples (now implemented in qg, l95, and fv3-jedi). Anna suggested that maybe we could do a remote code sprint - pick a week when model teams can work together on this implementation and the JEDI core team will be available to help. Jake said that Steve Vahl will work on this for MPAS.
Yannick asked if the meeting format was working out well for everyone (in the context of stay-at-home teleworking), and there were a number of positive responses. Today we had 47 people connected and the technology seemed to handle it reasonably well. If anyone is experiencing issues with connectivity, sound, video, etc. please let the core team know so we can address them.
Marek closed the meeting by thanking Anna and Dan for their work on this - it looks like important added functionality.