Almost 200 experts from around the world joined a virtual workshop on the treatment of random and systematic errors in satellite data assimilation for numerical weather prediction (NWP), organised by ECMWF and EUMETSAT’s NWP SAF (Satellite Application Facility) from 2 to 5 November 2020. Dealing with random and systematic errors in observations and models is at the heart of making optimal use of the wealth of information from satellite data to initialise forecasts or to create reanalyses. Errors and uncertainties arise from many areas, such as in the forecast model, the observations, or the ‘observation operators’ used to convert model fields to observation equivalents. The challenge is to separate the different errors and to deal with them adequately during the assimilation. The meeting continued the strong tradition of workshops organised jointly with the NWP SAF, fostering further exploitation of satellite data.
One of the aims of the workshop was to connect activities in different communities, spanning data providers, operational forecasting centres and academia, and to identify where NWP can make use of advances made in other fields in the characterisation of observation-related uncertainties. Reanalysis featured as a strong component, where treating biases in observations and models poses its own challenges.
The twenty-three speakers reviewed the strong progress in recent years in several areas. There were lively interactions between participants around the globe. This was in no small part thanks to the dedication shown by the participants, some of whom turned their nights into days to join their European colleagues live, in rich exchanges through online chats and video-conferencing rooms. Poster sessions, panel discussions and working groups provided further opportunities for discussion.
Characterisation of uncertainties
Several presentations covered the characterisation of observational and model uncertainties which informs our treatment of these in the assimilation. On the topic of instrumental biases, talks covered the latest developments in the on-orbit characterisation of the CrIS hyperspectral infrared instruments and the Aeolus Doppler wind lidar, as well as the ongoing activities of the Global Space-based Inter-Calibration System (GSICS, a World Meteorological Organization/Coordination Group for Meteorological Satellites initiative). The development of metrology-inspired approaches to the characterisation of observational uncertainties featured prominently in two of the talks, and the topic was revisited frequently in the panel discussion on biases and in working group discussions.
Representation error, resulting from a mismatch between the model representation of the state and observations, was explored in two of the talks. Promising approaches were described involving high-resolution model runs and observations. A talk on the characterisation of uncertainties in historic observations illustrated the value of the long-term perspectives brought by reanalyses.
Correction of observation and model biases
Correcting observational biases is essential for the successful assimilation of many satellite observations, and separating observation and model biases continues to be challenging. Adaptive bias correction methods are now commonly used within the assimilation system to correct for observation biases, and some schemes are developed to address model bias. Several talks discussed the best way to design models of observation bias, with the aim to avoid absorbing model error in the observational bias correction. The concept of scale separation also featured prominently. This applies where model error contains identifiable large-scale structures, which opens a new perspective in the quest to attribute the correct source of biases. Machine learning methods trained on analysis increments or anchor observations also showed some potential to learn the correct structure of model biases.
All the talks illustrated the importance of anchor observations, and more specifically GNSS-RO (Global Navigation Satellite System radio occultation), to estimate the different types of biases. In this context, reanalysis is facing a challenge, with highly variable observational coverage that becomes sparse going further back in time.
Representation of observation errors
The random error characteristics assigned to observations in the assimilation play a key role in determining the weighting of observations in the assimilation. Several talks reviewed the state of the art of modelling observation errors at various centres. Error correlations and situation-dependence of observation errors are increasingly being taken into account in operational systems. The required observation error modelling mostly relies on diagnostics based on differences between observations and model equivalents from short-range forecasts or analyses. Several speakers highlighted the potential, but also pitfalls of these approaches.
To represent situation-dependence, results from the diagnostics are used to determine simple parametrized models that capture main variations in error contributions, for instance from relatively poor surface or cloud modelling contributing to representation error. Concepts for modelling situation-dependent inter-channel error correlations are being developed. A better understanding and treatment of spatially correlated observation errors is also emerging, although algorithmic challenges remain, especially for variational data assimilation systems.
Looking to the future
A growing and increasingly diverse observing system will pose additional challenges for the treatment of systematic and random errors. Hyperspectral sounding data with unprecedented temporal resolution will be available from geostationary satellites, observations from some smaller satellites may have less well-characterised uncertainties, and an increasing number of observations will be driving multiple Earth system components with different model error and levels of maturity. All of these aspects will require continued development of the methods and capabilities used, including for the NWP-independent characterisation of all error sources.
Further information
Recordings of all presentations and panel discussions, all posters and a full workshop report are available from https://events.ecmwf.int/event/170/.