Dr Alan Geer, ECMWF Principal Scientist, all-sky assimilation
There is a great wealth of observations sensitive to cloud and precipitation coming from space- and ground-based instruments. In the past these were too difficult to assimilate in weather forecasting systems, but now we are already benefitting from all-sky microwave assimilation, and the way is open to using new observation types, including infrared and visible cloud imagery, and ground and space-based radar.
Figure 1: Meteosat visible channel image showing evening convection over central Africa at 16:00 UTC on 7 May 2016. (Image: © EUMETSAT 2017)
We are all familiar with visible-wavelength imagery of the earth, and the finely detailed pictures of clouds that it provides. For example, EUMETSAT's SEVIRI instrument scans the earth every 15 minutes at a horizontal resolution up to 1km, capturing images such as this evening convection over central Africa (Figure 1). The low sun is casting long shadows that pick out detailed 3D structures in these mesoscale convection systems. A movie would show how these systems started, just a few hours ago, as tiny isolated cells amongst the afternoon cumulus clouds.
While visible-wavelength imagery is used widely for other purposes, and it does give important information via cloud-track wind vectors, it provides no other quantitative contribution to the initialisation of operational weather forecasts: visible wavelength radiances are not assimilated. This is because:
- forecast models still use simplified representations of cloud and convection
- the radiative transfer models required to simulate such 3D structures accurately would be far too expensive for operational use, and
- the predictability of cloud and precipitation tails off in a matter of hours.
In the past these problems have made it difficult to use such observations in a data assimilation system, but now we are starting to understand how to deal with these problems.
We have already started to use observations sensitive to cloud and precipitation at microwave wavelengths, with great benefits. In the last five years these observations have gone from providing around 8% to around 20% of all observational impact on the quality of short-range forecasts, and this beneficial impact persists into the medium range. That success means we can now consider the assimilation of many new categories of observations sensitive to cloud and precipitation.
Figure 2: Schematic representation of the information on cloud and precipitation microphysical properties and sub-grid heterogeneity. This information is available from different observation types, both space-based passive instrumentation in the microwave, sub-mm, infrared, solar and broadband wavelengths, lightning detectors, and active instrumentation such as lidars and radars, and ground-based gauges and radars.
There is a vast and growing array of sensors available that can give information on cloud and precipitation, including its microphysical details, from visible-wavelength cloud imagery to ground-based rain radar and rain gauges (Figure 2). There are also an increasing number of satellites providing this information, launched by space agencies from Europe, the USA, Japan, China and India.
With help from scientists across the areas of physical modelling, observations and data assimilation, I was asked to coordinate a Science Advisory Committee special topic paper looking at our current capabilities and what we can do to assimilate all this new data. This is a rapidly developing new area, with implications for the way ground and satellite observations will be used in the future, and for the way we develop our numerical weather prediction (NWP) systems. In the past, NWP centres mainly expected to initialise their forecasts using more direct observations of wind, temperature and humidity. Now we are increasingly seeing that cloud and precipitation observations are useful too.
Figure 3: The mechanism of generalised 4D-Var tracing illustrated through the assimilation of a single observation sensitive to snowfall along a front in the north Pacific.
All-sky microwave radiances and how they benefit weather forecasts
The microwave radiances that currently provide benefit to forecasts are assimilated in "all-sky" (clear-sky, cloudy and precipitating) conditions. Most of the wavelengths used for all-sky assimilation are sensitive to water vapour in clear-skies. Hence a lot of the benefit comes from tracing features in the water vapour fields, whether or not clouds are present. However, useful information also comes directly from the cloud and precipitation features in these radiances. This is obtained through the generalised tracing effect, i.e. the ability of a 4D-variational (4D-Var) assimilation system to infer increments to dynamical fields (like wind and temperature) from observations of cloud and precipitation.
For example, we've seen how our system can fit observations of frontal snowfall, not by changing the amount of snow directly, or the relative humidity of the airmass that generates that snow, but by adjusting the strength of a developing low pressure system that started 11 hours before, 1000 km upstream (Figure 3).
These advances in our assimilation system have taken sustained investment over several decades, in a tricky and initially unpromising new area. Many developments have been necessary to achieve this, but the work would have been impossible without the tangent-linear and adjoint models, including cloud and precipitation physics, that are needed for the generalised 4D-Var tracing effect. The history and current impact of all-sky microwave has been documented in a paper recently accepted by QJ. In the future, the involvement of a growing global community of scientists working on all-sky radiance assimilation will really help move things forward.
The importance of microphysical and sub-grid scale details
One theme running through ECMWF's internal review was the importance of cloud and precipitation microphysics and sub-grid heterogeneity. Figure 1 illustrates the sensitivity of different observations to microphysical details (like particle size distributions, or the shape of frozen particles) and sub-grid scale heterogeneity (such as cloud overlap and 3D structure). These sensitivities are both ambiguous and overlapping across the different instruments, but within an NWP system, data assimilation can start to constrain these properties in the forecast model. The task we have is to start linking microphysical and sub-grid details in the forecast model directly to the observations, through the full chain of modelling and data assimilation. At the moment, only the bulk properties of cloud and precipitation are represented in the model, so this may require new developments, for example introducing more detailed microphysical schemes.
We also need to start using more consistent assumptions within the weather forecast model. For example, cloud overlap is represented separately, and in different ways, in the model radiation scheme, the convection scheme, the large-scale cloud scheme, and in several different all-sky observation operators.
A possibly unattainable but ideal goal could be called 'microphysical closure' - that when we are assimilating enough different observations of cloud and precipitation, the microphysical and sub-grid assumptions in the forecast model will be completely constrained. There would be no possibility of making incorrect physical assumptions because they would show up as poorer fits to some of the cloud- and precipitation- sensitive observations.
The broad conclusions of our review
Over the next few years, we expect observations sensitive to cloud and precipitation to play an increasing part in the initialisation of our weather forecasts. ECMWF will aim to assimilate the growing variety of observations related to cloud and precipitation, including new developments like lightning sensors. We also hope to make quantitative use of visible wavelength images through direct all-sky radiance assimilation. Even if these observations cannot all provide benefits initially, they can be monitored to guide developments in cloud and precipitation modelling. When our scientific understanding has developed, these observations can be activated in the same way as all-sky microwave data assimilation, with progressive advances that slowly build up to a big impact.
These observations will become increasingly important to weather forecasting and this has implications in the wider NWP community, particularly for planning and developing the global observing system. When designing new satellite missions and instruments, space agencies look to how observations will be used in future decades. They should expect that weather forecasting centres will be assimilating the radiances directly, in all-sky conditions, even at wavelengths such as visible. There are also existing observation types that could give much benefit to weather forecasting, if they were available in near-real-time, with good quality and calibration. The existing regional and national networks of rain radars could contribute a lot if they were made available for global forecasting, for example.
In the next few decades, all forecasting centres will be assimilating observations sensitive to cloud and precipitation as a routine and essential part of their activities.