Progress in forecast skill
at three leading global operational NWP centers during 2015-2017 as
seen in Summary Assessment Metrics (SAMs)
Ross Hoffman
AOML
7 May, Noon, in 2155
Abstract:
The summary assessment metric (SAM) method is applied to an array of
primary assessment metrics (PAMs) for the deterministic forecasts of
some leading numerical weather prediction (NWP) centers for the years
2015-2017. The PAMs include anomaly correlation, RMSE, and absolute
mean error (i.e., the absolute value of bias) for different forecast
times, vertical levels, geographic domains, and variables. SAMs
indicate that in terms of forecast skill ECMWF is better than NCEP,
which is better than but approximately the same as UKMO. The use of
SAMs allows a number of interesting features of the evolution of
forecast skill to be observed. All centers improve over the three year
period. NCEP short-term forecast skill substantially increases during
the period. Quantitatively, the effect of the 2016 May 11 NCEP upgrade
to the 4D-ensemble variational (4DEnVar) system is the largest SAM
impact during the study period. However, the observed impacts are
within the context of slowly improving forecast skill for operation
global NWP as compared to earlier years. Clearly the systems lagging
ECMWF can improve, and there is evidence from SAMs in addition to the
4DEnVar example that improvements in forecast and data assimilation
systems are still leading to forecast skill improvements.