A new equitable score suitable for verifying precipitation in NWP

Title
A new equitable score suitable for verifying precipitation in NWP
Technical memorandum
Date Published
02/2010
Secondary Title
ECMWF Technical Memoranda
Number
615
Author
Mark Rodwell
David Richardson
Tim Hewson
Publisher
ECMWF
Abstract A new equitable score is developed for monitoring precipitation forecasts and for guiding forecast system development. To accommodate the difficult distribution of precipitation, the score measures error in ‘probability space' through use of the climatological cumulative distribution function. For sufficiently skillful forecasting systems, the new score is less sensitive to sampling uncertainty than other established scores. It is therefore called here ‘Stable Equitable Error in Probability Space' (SEEPS). Weather is partitioned into three categories: ‘dry', ‘light precipitation' and ‘heavy precipitation'. SEEPS adapts to the climate of the region in question so that it assesses the salient aspects of the local weather, encouraging ‘refinement' and ‘discrimination' and discouraging ‘hedging'. To permit continuous monitoring of a system whose resolution is increasing with time, forecasts are verified against point observations. With some careful choices, observation error and lack of representativeness of model grid-box averages are found to have minimal impact. SEEPS can identify key forecasting errors including the over-prediction of drizzle, failure to predict heavy large-scale precipitation, and incorrectly locating convective cells. Area-averages are calculated taking into account the observation density, so that all sub-regions are treated more equally. A gain of ∼2 days, at lead-times 3–9 days, over the last 14 years is found in extratropical scores of forecasts made at ECMWF. This gain is due to system improvements, not the increased amount of data assimilated. SEEPS may also be applicable for verifying other quantities that suffer from difficult spatio-temporal distributions.
URL https://www.ecmwf.int/en/elibrary/76205-new-equitable-score-suitable-verifying-precipitation-nwp
DOI 10.21957/h886wj14
Download citation