ECMWF Lead Scientist Roberto Buizza chaired the opening session of the Annual Seminar 2017.
Between 11 and 14 September more than 100 people from across the world joined ECMWF scientists and 28 speakers at the Annual Seminar on ‘Ensemble prediction: past, present and future’ to discuss how to further improve the accuracy and reliability of ensemble-based forecasts.
Thanks to advances in models, data assimilation schemes and the methods used to simulate initial and model uncertainties, today ensembles are widely used to provide a reliable estimate of possible future scenarios. This is expressed for example in terms of probabilities of weather events or of risk indices.
Increasingly ensembles are routinely used to provide forecasters and users with the range of weather scenarios that could happen in the future. An example is the ECMWF ensemble-based strike probability for Hurricane Irma, issued by ECMWF on 5 September.
The chart shows the ECMWF ensemble-based probability that Hurricane Irma would pass within a 120 km radius during the next 10 days, according to the forecast issued at 00 UTC on 5 September. Both ECMWF’s high-resolution forecast (solid line) and the ensemble mean (dotted line) turned out to be very close to the track as subsequently observed (crosses). However, the ensemble as a whole gave vital information on the range of possible tracks, something which a single forecast cannot provide.
Avenues of progress
Speakers at the Annual Seminar reviewed the fundamental ideas on which operational ensembles are based and set out how the original ensembles implemented in the early 1990s were successively modified and improved.
They discussed the different methods developed to simulate initial and model uncertainties in ensembles of analyses and forecasts. They then highlighted how errors propagate across scales and how ensembles of forecasts and re-forecasts can be combined to extract predictable signals and design valuable products for a range of applications.
They also presented verification metrics and diagnostic tools that can be used to assess the quality of an ensemble of analyses and forecasts, and they discussed how much improvement we can expect in the future.
Looking to the future, speakers gave a clear indication that models need to improve, for example in the way they simulate wave propagation between the tropics and the extratropics. Another area that if tackled should lead to improved predictions is the simulation of model uncertainties: the schemes used today are simulating only some of the known model uncertainties.
Eugenia Kalnay from the University of Maryland gave a talk entitled 'Historical perspective: earlier ensembles and forecasting forecast skill'.
On the issue of ensemble size, there were indications that the current size of about 25–50 is too small to provide reliable probabilistic forecasts, especially in the tails of the distributions (the ranges that users with low cost/loss ratios would be more interested in).
Similarly, results from Canada indicated that having a large ensemble size is even more important if one wants to provide a reliable analysis error estimate (Canada today uses an Ensemble Kalman Filter with 256 members).
These results imply that care must be taken in trading ensemble size for resolution, and that different classes of users could benefit from different ensemble configurations in terms of size and resolution.
Poster session
For the first time, the Annual Seminar included a poster session and a panel discussion. The poster session, which included about 20 contributions, provided the participants with an overview of the main characteristics of some of the world’s best ensembles of analyses and forecasts.
Laure Raynaud presents her poster on the two ensemble systems at Météo-France.
Sarah-Jane Lock explains her poster on the main characteristics of ECMWF ensemble forecasts.
Panel discussion
The panel discussion provided an opportunity to explore and discuss in more detail some of the fundamental questions we are currently tackling, such as:
- Should we be moving to small ensembles at high resolution or large ensembles at more moderate resolution?
- If the most cost-effective ensemble structure changes with lead time, should our ensemble be built so as to give a resolution and ensemble size that changes with lead time?
- If an ideal ensemble consists of a set of equally likely members, is there a role for an unperturbed/central forecast?
- What do we expect from the future in terms of our ability to represent model error in ensemble systems, and the representation of perturbations more generally?
A panel discussion provided an opportunity to explore some fundamental issues in relation to ensemble prediction.
During the lively panel discussion a number of interesting points were made:
- Some users would react even at small probabilities: they would be the ones to benefit more from an increase in size.
- Size is very important both for the extended/long ranges and for high-resolution ensembles, to be able to capture the fine-scale details.
- ECMWF’s ensemble size of 50 seems about right considering its range of users
- The Centre should work with users to develop calibration methods and understand whether the balance between ensemble size and resolution should be revisited once calibration methods are more widely used.
- We need more observation-based diagnostics to understand model error, and design better schemes.
These issues are highly relevant to the future of ensemble forecasting at ECMWF as the Centre implements its 2016–2025 Strategy, which calls for a global 5 km ensemble by 2025. This goal is at the limit of what is possible scientifically and in terms of computing power.
The Seminar confirmed once again that the continuous exchange of knowledge and ideas is key to our ability to advance the science of weather prediction and the design of operational ensembles that can deliver timely, reliable and accurate forecasts. It also showed that events such as the ECMWF Annual Seminar provide great opportunities for these exchanges to happen.
For the first time the Annual Seminar included a poster session.
All presentations and recordings are available on the Annual Seminar 2017 web page.