Peter Dueben, ECMWF scientist working on numerical methods
I was one of the organisers of the workshop on Machine Learning for Weather and Climate Modelling that took place at Corpus Christi College in Oxford in the first week of September. The four-day workshop discussed how machine learning techniques could be used to improve weather and climate models in the future.
What is machine learning?
In general, machine learning tools can be used to develop a computer model that is able to represent a complex “system” from a large set of “input” and “output” data pairs. A “system” could be a human who is looking at a picture (input) and understand what is shown (output); a chess player who considers the position of chess pieces on the board (input) and decides the next move (output); or the driver of a car who is estimating the position and speed of a bike (input) and decides whether he will overtake (output). If there is a sufficient number of input/output pairs for training, machine learning can be used to develop numerical tools that can mimic the decision of a rather complex system.
How could machine learning be used in numerical weather prediction?
There are a large number of possible application areas for machine learning throughout the workflow of numerical weather prediction. Here, the “system” could be a tool to pick and bias-correct observations that are used within data assimilation; a tool to project real-world observations into a representation of variables as they are used within models; or a part of the numerical forecast model – such as the radiation scheme which predicts the heating of the atmosphere given surface temperature, cloud patterns and incoming radiation from the sun. It could also be a tool that is used in post-processing of the output of a weather forecast model, for example to detect extreme weather events in forecasts or to bias-correct forecasts for a specific location.
In fact, the workshop presentations covered a large number of application areas within the weather and climate modelling workflow. It was really inspiring to learn about the areas in which machine learning tools have already been used successfully and those areas where more work will be required for machine learning to become as good as conventional tools.
The use of machine learning methods is not fundamentally new to weather and climate modelling and it can be argued that machine learning is standard within many areas of research already, for example when calculating the correlation of two variables within forecasts. It can also be argued that the process of data assimilation, as it is used at ECMWF to generate initial conditions for numerical weather forecasts, can be viewed as a machine learning application. However, the development of new tools – such as deep neural networks with a huge number of degrees-of-freedom – allows us to approach systems with machine learning that are much more complex than conventional application areas. Furthermore, with today’s computing infrastructure, much more data for input and output pairs can be stored and processed and this makes machine learning tools much more powerful than in the past.
In fact, machine learning is strongly influencing the development of future supercomputers. This means that the community of weather and climate modelling will need to learn in the near future how to make efficient use of such hardware optimised for very specific applications (requiring the use of dense linear algebra at very low numerical precision).
Progress and challenges
During the workshop more than 100 scientists from around the world discussed progress and challenges of machine learning in weather and climate modelling via talks, poster presentations, and breakout groups. The workshop brought together researchers from meteorology and Earth sciences, computing, applied mathematics, and statistics as well as scientists from universities, weather and climate research centres and industry. Four members of staff from ECMWF attended the workshop, with three presenting talks.
The contributions have shown – once again – that machine learning is very powerful and that the scope of machine learning applications has increased significantly over the last couple of years. In fact, it is difficult to keep up-to-date with developments at the moment as the research is progressing at a breath-taking pace – one of the reasons why the workshop was so important and timely.
Participants at the Machine Learning for Weather and Climate Modelling workshop. (Photo: Sam Hatfield)
During the workshop we also discussed a number of challenges for the field of weather and climate modelling when using machine learning tools, that need to be addressed by the community. For example, it is difficult to utilise existing knowledge about a physical system to improve machine learning tools, because such tools learn from data and are used as a “black-box” application. So, it is difficult for these tools to represent conservation properties, and there is a risk that data-trained models fail if a new input is used which was not represented by the set of data used during training. It is also challenging to link machine learning software with conventional models when running on supercomputers.
The “system” that is learned by machine learning tools could also be the real-world “weather”. The tool would then use information about the state of the atmosphere at a certain time (input) to predict the state at a later time (output). Once trained, such a tool can be used to predict into the future and can actually produce a weather prediction in the same way as a conventional weather prediction model. There were a couple of contributions to the workshop that have tried to do exactly this (including my own paper).
From all possible applications of machine learning in numerical weather predictions, this application can be considered as being the most challenging one. By the end of the workshop, having reviewed and discussed a wide range of possibilities and options, there was no agreement between participants about whether such a machine learning model would be capable of beating a conventional weather prediction model at some point in the future. Opinions ranged from: “certainly-yes” to “I-will-bet-my-house-against-it”. I do not know yet whether this will become possible and it would certainly require a lot of very challenging (and therefore interesting) research within weather and climate science, but also deep learning and computer science. I look forward to contributing!
Acknowledgements
The scientific organisation of the workshop was done by Matthew Chantry, Hannah Christensen, ECMWF Fellow Tim Palmer (all from the University of Oxford) and myself. We have received crucial support for the general organisation from Philippa Towler and the conference was possible thanks to the financial support from Amazon, the Copernicus Atmosphere Monitoring Service (CAMS), the Copernicus Climate Change Service (C3S), the ESiWACE Centre of Excellence, NVIDIA, the Office of Naval Research and Vulcan.
Banner image: monsitj/iStock/Getty Images Plus