ECMWF Director of Computing Martin Palkovič opened the ‘1st Artificial Intelligence for Copernicus Workshop’ with a talk on ECMWF’s vision for big data, AI and cloud computing.
Sixty-nine scientists came together at ECMWF from 5 to 7 November 2019 to explore the scope for widening the use of artificial intelligence (AI) in Earth system applications.
The event was organised by the two EU-funded Copernicus Earth observation services implemented by ECMWF: the Copernicus Atmosphere Monitoring Service (CAMS) and the Copernicus Climate Change Service (C3S).
“The workshop was a great success, with lots of real-life examples of the use of different forms of AI in the Earth sciences and beyond,” says Director of CAMS Vincent Henri-Peuch, one of the organisers of the event. “It confirmed that the bubbling up of ideas in this field has reached a certain maturity.”
Director of CAMS Vincent-Henri Peuch was one of the organisers of the event.
The talks by 28 speakers showed that there is considerable scope for the use of AI not just for Copernicus but also for ECMWF’s core mission of numerical weather prediction.
Indeed, might all environmental and weather forecasts be produced by AI techniques one day instead of by today’s physically based but computationally costly numerical algorithms?
For Peter Düben, who is working on machine learning at ECMWF, this is an open question. “First tests of using neural networks to predict aspects of the weather have shown encouraging results, but there are still many obstacles to overcome,” he says.
What is AI anyway?
Some key terms used in this debate are artificial intelligence, machine learning and deep learning.
Artificial intelligence broadly refers to intelligence demonstrated by machines.
Forms of artificial intelligence being explored in Earth system science include machine learning (in which computers do not receive specific instructions but rely on patterns and inference to perform tasks); data mining; and data fusion.
Finally, deep learning is a form of machine learning based on artificial neural networks. Such networks are characterised by the presence of ‘hidden layers’: elements of the algorithm that can identify patterns in data without being explicitly programmed.
“Artificial neural networks require training on input and output datasets before they can start to produce outputs based on new inputs,” says Peter, who co-organised a workshop on machine learning for weather and climate modelling in Oxford in the first week of September.
Peter Düben was one of the speakers at the workshop.
Use of AI in Copernicus
In the Copernicus Earth observation programme, AI is already used in several ways.
“Examples are interpreting satellite images, identifying changes in the Earth’s surface over time, and downscaling air quality forecasts to finer resolutions,” says Vincent-Henri.
He emphasises that more and more users are interested in applying AI to Copernicus data. “We need to anticipate this demand and adapt our data to make them easily useable in AI applications. We will also have to investigate making AI tools available in the C3S Climate Data Store and the future CAMS Atmosphere Data Store.”
Philippe-Pierre Mathieu from the European Space Agency (ESA) gave a talk on ‘The rise of artificial intelligence for Earth observation’.
Vincent-Henri believes that Earth observation and weather prediction can learn from other areas of science, such as high-energy physics, that are more advanced in their use of AI.
“That is why we invited a few speakers from different fields to the workshop, who gave us some fascinating insights into what is being done in their areas.”
Use of AI in weather prediction
There are also plenty of use cases for machine learning in numerical weather prediction, and the question of how to make the most of AI techniques is an active area of research at ECMWF and beyond.
Samantha Adams from the UK Met Office gave a talk on using self-organising maps to understand non-linear cloud-circulation couplings.
“Potential areas of use include data quality control; bias correction in data assimilation; emulating model components; quantifying uncertainty; and many more,” Peter says.
He is more cautious on the question of whether ultimately the whole forecasting system could run on AI rather than today’s physically based algorithms.
“Although we have a lot of data to train AI systems on, it may not be nearly enough to train them to make predictions for the whole Earth system with all its complex interactions,” he says.
“On top of that, we are living in a changing climate with many extreme and unprecedented events, which AI may find difficult to handle.”
Nevertheless, experiments carried out at ECMWF illustrate that even today properly trained neural networks can make short-range predictions of surprising accuracy.
The plots show the difference in geopotential at 500 hPa (in m2/s2) between 00 UTC on 1 March and 00 UTC on 2 March 2017 according to the analysis (our best estimates of the state of the atmosphere at those times – left) and according to a 24-hour neural network forecast starting from the analysis at 00 UTC on 1 March (right). For further details, see the article by Peter Düben and Peter Bauer published by Geoscientific Model Development.
“There is no doubt that this is an exciting area of research, especially since much of today’s high-performance computing research is geared towards AI applications,” Peter says.
“But only the future will tell just how far AI can be taken in the context of numerical weather prediction over the years and decades to come.”
Next steps
One of the outcomes of the workshop is the realisation that there is a lack of suitable training datasets for scientists working on some kinds of AI problems.
One suggestion is for ECMWF to partner with other organisations to generate a repository of labels to attach to its datasets for common machine learning problems.
“This would speed up research as well as innovation, as users could focus on the development of AI applications rather than pre-processing the data,” says ECMWF scientist Claudia Vitolo, who co-organised the workshop.
She adds that, more broadly, there is little doubt that the role of AI at ECMWF and in Copernicus will continue to grow over the years to come.
“Artificial intelligence is under discussion as a theme for ECMWF’s next ten-year Strategy, and it is certain that this first workshop will be followed by others.”
Some of the 69 scientists who attended the first Copernicus AI workshop at ECMWF. The workshop was organised by ECMWF Director of Copernicus Services Jean-Noël Thépaut, Director of CAMS Vincent-Henri Peuch, and ECMWF scientists Peter Düben, Baudouin Raoult, Stephan Siemen and Claudia Vitolo.
More information
For more information on the workshop, including presentation slides and recordings of the talks, visit the workshop page on the C3S website or on the CAMS website.