This study provides the first comprehensive verification of extratropical cyclones around North America and the adjacent oceans within the NCEP models in several years. The sea-level pressure (SLP) errors are quantified for cyclones within the NCEP GFS and NAM for the 2002-2007 cool seasons (October to March), as well as the NCEP Short-Range Ensemble Forecast (SREF) system for the 2004-2007 cool seasons. The GFS analyses were used as truth for the central pressure, since verification of the GFS, NAM, and NARR analyses using surface observations show that the GFS-analyzed cyclone pressures are more accurate than the NAM and NARR on average. The model and observed data were interpolated to a common 80 km grid, and the cyclones were identified using an automated cyclone finding and tracking system originally developed by NCEP. The observed and simulated cyclones were matched using the closest pair within 800 km of each other. This approach was ~90% effective in identifying cyclone events and matches in our dataset.
The results suggest that the NCEP GFS is more skillful than the NAM over many regions, especially over the eastern Pacific, where the NAM has a large positive bias in cyclone central pressure. The cyclone errors are larger over the Pacific than other regions for short-term (0-60 h) forecasts. However, the errors for relatively deep cyclones over the eastern U.S. and western Atlantic are larger than the eastern Pacific by hour 72, since the cyclones over the western Atlantic are underpredicted (too weak) on average in the extended range. This suggests that there is still a need to dramatically improve the > day 3 cyclone forecasts over the eastern U.S. We also show that many large cyclone errors over the eastern U.S. in the 72-96 h range can be traced back to the errors over the eastern Pacific 2-3 days earlier. Verification also reveals that there is little added skill from the 06/18 UTC model cycle as compared to the 00/12 UTC on average for the cyclone forecasts in many regions. There has also been little overall cyclone forecast improvements during the past 5 years over the eastern U.S. in the day 2 forecast.
The NCEP SREF was also verified for the 2004-2007 cool seasons.
The SREF is more skillful than the operational NAM in many regions, but
not the operational GFS. The SREF suffers from overdispersion and
positive biases in many regions early in the forecast, which hurts its
performance. Also, not all members are equally skillful. Future work
will identify those synoptic patterns associated with the best and
worst SREF forecasts.