Craig Bishop from the University of Melbourne gave a presentation on his research into building and maintaining TLM Adjoints. Craig presented slides, however he asked not to post these in a public area so I have not attached them to these notes. Please contact Yannick or Craig if you want to view the slides.
Summary of Craig's presentation.
Craig started with motivation for this research. He argued that machine learning (ML) has started to produce impressive results with weather prediction, but these techniques rely heavily on reanalysis data. High quality reanalysis data relies on numerical weather prediction (NWP) techniques, and it remains questionable if ML techniques could do better. Given this, the primary purpose of DA may tend to focus on reanalysis and if this actually happens it may become important to develop ensemble and TLM Adjoint techniques.
Craig characterized linearization as an "art form" to emphasize that it can be tricky to come up with effective parameterizations. The challenge then becomes to create an easier, more accurate way to build and maintain TLM Adjoints.
Craig continued with two primary techniques he is developing with his research. The first is called "Local Ensemble TLM" (LETLM) and the second is called "Implicit Ensemble TLM" (IETLM). LETLM uses ensemble perturbations and EOF analysis to attempt to more directly find optimal solutions compared with a more empirical style approach of running a large number of ensemble members and trying to find the optimal solution within those results. He described a modified LETLM method called "reduced LETLM" (LETLM-R) that was successful at reducing the number of ensemble members, compared to the empirical approach, needed to get the desired accuracy.
LETLM-R is meant for use with models that use explicit time stepping, and this is where the second technique IETLM comes in. IETLM is for use with models that use implicit time stepping. This technique uses an analytical approach to reduce the Adjoint to a smaller problem that allowed for the technique to work with a much smaller amount of ensemble members.
Craig showed a slide depicting a use case that benefited greatly from these techniques. It showed the following reduction:
- With no reduction, 29 ensemble members were required to get acceptable accuracy
- With LETLM-R, the same accuracy was attained with 20 ensemble members
- With IETLM, the same accuracy was attained with 5 ensemble members
Discussion
Hernan asked what makes a "good" ensemble? Craig responded that since we are wanting to build a TLM Adjoint we only need the gradients (not the "true" forecast). We want the perturbations to be in the linear regime. When the perturbations start to become non-linear then you need to "reboot" the ensemble and continue to keep the perturbations in the linear regime. Note that the reduction techniques in the presentation depend on the perturbations remaining in the linear regime.
Jerome asked if two ensembles are required - one to carry forward with the forecast/analysis, and a second keeping perturbations small for the reduction techniques. I'm not sure I captured this correctly, but I think Craig said you only need one ensemble and what you do is select from the members that stayed close (ie, small perturbations) to the forecast for use with the reduction techniques.
Hernan asked how to deal with the impacts of observations on the reduction techniques. Craig responded that an FSOI can be run to work out how to deal with the observations impacts. He noted that he has a published paper that discusses this method (of using FSOI to deal with observations impacts).
Yannick asked if the reduction techniques can be used for Hybrid TLM. Craig mentioned that Tim Payne at the Met Office has been experimenting with running a traditional TLM, followed by running a vertical TLM and applying the reduction techniques. Tim is trying to reduce the vertical TLM problem because gravity wave effects can be realized across a long distance (eg, surface to stratosphere). Without the reduction, Tim needed to run with 100 members to capture the large scale effects accurately, and by using the reduction technique was able to get good results using much less members. Craig noted that convective parameterization adds to the difficulty of the reduction techniques.
Tom asked, in the context of AI, what are the different philosophies. Craig currently prefers analytic techniques to reduce the TLM and maintain accuracy. He mentioned that ML techniques are being looked at, and interesting results are emerging. The idea is to bombard the system with many cases to allow the system to learn to recognize patterns. However, there remain some accuracy challenges with ML techniques such as maintaining geostrophic balance (since the concept of geostrophic balance isn't inherent in the ML techniques). Craig also mentioned that you need gradient values for the TLM Adjoint reduction and ML tends to hide these (again since ML is learning to recognize patterns).
Greg T commented that there exists a recent paper that shows microphysics parameterizations don't get much benefit from stochastic methods. But, ML techniques might have more promise. Craig acknowledged this point and mentioned that one of the biggest sources of uncertainty is impacts of clouds and precipitation. You need to get radiative properties, etc. correct, and the only way to do that so far is with physics based parameterizations. This is particularly challenging for climate models since radiative properties of clouds are so important (and the same for aerosol impacts). Greg noted that CRTM development is moving forward with providing accurate cloud impacts (radiative and aerosol).