Routine evaluation of forecast performance provides essential feedback to both users and model developers on the quality of the forecasting system. ECMWF maintains a comprehensive range of verification statistics to evaluate the accuracy of its forecasts. Regular monitoring of the forecast skill is complemented with more in-depth diagnosis to provide a comprehensive understanding of strengths and weaknesses of the forecasting system.
ECMWF continues to develop evaluation methods. Appropriate scores for verification of severe weather events are being developed and assessed, and access to relevant observations beyond what is available via global exchange (the GTS), such as high‐resolution in‐situ data from national networks is being investigated. Flow‐dependent aspects of forecast performance at both the medium and extended ranges are being evaluated through diagnostic studies and the development of appropriate verification metrics. Satellite observations are being exploited to provide additional verification of radiation and cloudiness.
Operational forecast evaluation
Model climate evaluation
Evaluation of precipitation (SEEPS)
ECMWF developed a new verification score, SEEPS (Stable Equitable Error in Probability Space) to monitor the long-term trend in performance for forecasting precipitation. Forecast precipitation accumulated over 24 hours is evaluated against observed precipitation amounts reported from SYNOP stations. At each observation location, the weather is partitioned into three categories: “dry”, “light precipitation” and “heavy precipitation”. The boundary between “light” and “heavy” is determined by the station climatology so that SEEPS assesses salient features of the local weather and accounts for climate differences between stations. The SEEPS score evaluates the performance of the forecast across all three categories.
More information about the SEEPS score and its use for monitoring developments in forecasting precipitation can be found in the following papers:
- Haiden, T., M. Rodwell, D. Richardson, A. Okagaki, T. Robinson, and T. Hewson, 2012: Intercomparison of global model precipitation forecast skill in 2010/11 using the SEEPS score. Mon. Wea. Rev., 140, 2720-2733. (Also available as ECMWF Technical Memorandum 665)
- Rodwell, M. J., Richardson, D.S., Hewson, T.D. and Haiden, T., 2010. A new equitable score suitable for verifying precipitation in numerical weather prediction. Q.J.R.Meteorol. Soc, 136, 1344-1363. (Also available as ECMWF Technical Memorandum 615)