Stochastic Physics Testing and Evaluation

Overview

It is known that global and regional numerical weather prediction (NWP) ensemble systems are under-dispersive, producing unreliable and overconfident ensemble forecasts. With growing evidence that initial-condition uncertainties are not sufficient to entirely explain forecast uncertainty, the role of model uncertainty is receiving increasing attention.   In the last decade, a number of different strategies have been proposed to represent uncertainty arising from model formulation. Typical approaches to alleviate this problem include the use of multiple dynamic cores, multiple physics suite configurations, or a combination of the two. While these approaches may produce desirable results, they have practical and theoretical deficiencies and are more difficult and costly to maintain.

In the multi-physics approach, each ensemble member uses a different set of physics parameterizations to represent parameterized processes like convection, boundary layer, and land surface effects. While it can be challenging to find different sets of physics parameterizations that are physically consistent with each other (for example, a land surface model that is consistent with the planetary boundary layer parameterization), multiple physics schemes introduce large diversity among the ensemble members, leading to improved forecast skill. While characterized by good performance, multi-physics schemes have several theoretical and practical disadvantages. For each physical process, several different parameterizations need to be developed and maintained, which is resource intensive. More importantly, and from a statistical perspective, multi-physics ensembles do not form consistent distributions, since some parameterization schemes are more closely related than others. Statistical post-processing generally assumes independent and identically distributed random variables,  a requirement that is not met by multi-physics ensembles. Finally, each ensemble member has a different climatology and mean error. The fact that different members have different biases is one of the reasons why the multi-physics approach improves spread, but this result conflicts with the fundamental purpose of forecast uncertainty, which aims at representing the random - and not the systematic - component of forecast error.

An area of active research is to introduce ensemble spread by perturbing ensemble simulations stochastically. This method leads to statistically consistent ensemble distributions and has been successfully implemented in a number of operational weather models. The two most commonly used stochastic parameterization schemes - he Stochastic-Kinetic Energy Backscatter scheme (SKEB) and the Stochastic Perturbations of Physics Tendencies (SPPT) scheme - are formulated to represent unresolved subgrid-scale processes as well as sample the distribution of the subgrid physics tendencies.

While the performance of stochastic parameterization schemes is very good, they have been criticized  because they are added a posteriori to NWP models that have been independently developed and tuned. Ideally, stochastic perturbations should represent model uncertainty where it occurs and should be  developed alongside physical parameterizations. Therefore, the multi-parameter approach, addresses parameterization uncertainty at its source by perturbing the parameters in the physics parameterizations.  This approach, in RE task studies, is labeled as Stochastic Parameter Perturbation (SPP). SPP in combination with SKEB and SPPT has been tested with both RAP (13-km grid spacing) and HRRR (3-km grid spacing, convection allowing) based ensembles with the goal of getting this approach ready for transition to operation of the next generation regional ensemble.