...
- We are looking to understand why the latest VarBC changes introduce unexpected differences in the results of running the var; these are more roundoff that would be caused by the precision of coefficients in the old and new formats not been preserved.
- We are working to do a very close comparison between GSI and JEDI geovals and clear out differences that seem more than acceptable.
MetOffice:
Philip Underwood reported an issue Chris Thomas (in copy) has identified with the MPI distribution of global aircraft observations. When looking at this obtype in isolation, it’s been identified that some MPI communicators are handling several million locations, whereas others may only be handling tens of thousands (see attached image). Interpolating these to generate the GeoVals is taking a long time as the round-robin distribution means that many of the locations are in geographical regions that are not covered by the model region their MPI communicator is handling. As a result, there is a significant amount of MPI communication taking place for interpolation.
Chris has made three suggestions to explore and potentially solve this issue:
- Exclude locations that failed thinning prior to calling GetValues
- Separate pre-filter step
- Create an obs domain decomposition that matches the model one
The following discussion led to the first question; is this problem related to 3D or 4D? Apparently, these issues are less pronounced with 4D. Francois Hebert said that changing the obs distribution would not fix the issue, and that isolating and looking at a single ob type enhances this problem. Hernan G Arango described a more technical approach to MPI distribution, that to be honest I didn’t understand.
(plot was sent after the JEDI meeting for a reference)
NCAR:
reported an issue when we run mpas-jedi in 4DEnVar mode: it will fail if we turn on the output of geovals and ydiags files. Will create an issue for this and seek the solution.
...