Use and verification of ECMWF products in Member and Co-operating States (2016)

Each summer ECMWF invites Member States and Co-operating States to submit updated reports on the application and verification of ECMWF’s forecast products. This report summarises the findings from the 2016 reports, in combination with findings from official visits made to certain member and co-operating states during the preceding 12 months, and in combination with feedback provided during the “Using ECMWF Forecasts” (UEF) workshop held in June 2016. As such this report provides a snapshot, for 2016, of the way in which ECMWF forecasts are being used, of how these forecasts are performing relative to forecasts from other models, notably Limited Area Models (LAMs), and of aspects of ECMWF output that users would particularly like to see improved. The report does not refer to ECMWF’s in-house verification of its own forecasts.

Key findings regarding usage are as follows. At short lead times, up to about day 3, ECMWF IFS (Integrated Forecast System) output provides boundary conditions required for most operational LAM runs in Europe, with forecasts from those LAMs typically being used alongside the IFS forecasts themselves. From day 3 onwards direct IFS output is the primary source of forecast information, and for lead times of 10-30 days, and beyond, it is the only source for most users. Interest in monthly and seasonal time ranges continues to grow, fuelled in particular by requirements of the energy sector.

Regarding verification, most reports focused on comparing HRES (ECMWF’s high resolution forecast) with LAMs, for sensible weather parameters. Relative performance varies, but overall HRES performance is as good as LAM performance. However LAMs have better precipitation biases than HRES, versus point totals, and also usually perform better over complex topography. Some improvement in HRES output was noted, as expected, when resolution increased from 16 to 9km in March 2016, though only limited evidence was available at the time of the reports.

Model problems highlighted by users include ongoing issues with incorrect sea ice cover, and, partly related to this, near-coast sea surface temperature errors. As in previous years ECMWF also received reports of problems with inversions and low cloud cover, notably in winter. There was also reference to underestimation of peak totals in orographic precipitation events, and to poor handling of summer-time convection in some areas near the Alps. There was positive feedback regarding improvements to near-coast 2m temperatures following the change ECMWF made to the radiation scheme in spring 2016.