ECMWF’s 2019 Summer of Weather Code finished on a high with a showcase day for the seven successful projects on 20 September.
Faithful to its spirit of innovation, the second edition of ECMWF’s Summer of Weather Code (ESoWC) had invited proposals for 12 open-source projects addressing weather-related software challenges.
Developers from around the world submitted proposals and eight were selected for coding. From May to the end of August, ECMWF mentors and participants joined forces to work on projects related to machine learning, data visualisation, software development, Jupyter notebooks and functionalities of open-source geographic information systems.
The seven ESoWC coding teams and their ECMWF mentors came together at the Centre on 20 September.
Outcomes
All the teams used the online software development platform GitHub to track the progress of the projects and to share their knowledge with the wider open-source community.
Seven teams successfully completed the coding period to develop:
- Python code to obtain online aircraft metadata
- new calibration software called ecPoint-Calibrate
- a Jupyter notebook for OpenIFS
- scripts to generate Earth animations with the Blender Visualization Toolkit
- GIS-based dissolve functionalities using GRASS GIS
- machine learning techniques to better predict and understand drought
- machine learning techniques for high-impact weather
Their results offer benefits for forecasting models, users of forecast information, and training.
Obtaining online aircraft metadata
Aircraft meteorological reports provide valuable input for numerical weather prediction. ECMWF receives over 800,000 aircraft reports per day, which are mainly in Aircraft Meteorological Data Relay (AMDAR) format. However, AMDAR reports do not provide the aircraft type, which can be a vital piece of information for evaluating temperature biases and wind errors.
This ESoWC aircraft metadata team developed Python code to compare AMDAR flights with those from websites such as flightaware.com and flightradar24.com to identify aircraft types. The matching rate varies greatly among countries because there are many parameters to consider, such as the distance between airports and the number of aircrafts reported in AMDAR.
The availability of aircraft type would improve the use of aircraft meteorological observations within the forecasting process by targeting data exclusions and temperature bias correction.
New calibration software: ecPoint-Calibrate
The team working on this challenge developed new software that uses conditional verification concepts to compare numerical weather prediction model output with point observations in different weather situations, and thereby anticipate sub-grid variability and grid-scale bias.
ecPoint-Calibrate provides a dynamic, user-friendly environment for post-processing model parameters to deliver better probabilistic forecasts for point locations.
Users can select from five modules: 1) Activate data analysis, 2) Generate a new point data table, 3) Upload an existing point data table, 4) Construct a new decision tree, and 5) Test an existing decision tree.
Jupyter notebook for OpenIFS
The project aimed to develop an interactive Jupyter environment for OpenIFS, an easy-to-use version of ECMWF’s Integrated Forecasting System for external academic users.
The resulting notebook is simpler, modern and compatible with the Fortran interface of the IFS and overall enhances the scientific productivity of OpenIFS. One benefit is that the Python-based interface can be used for physical parametrization training sessions.
Blender Visualization Toolkit
The challenge was to develop a set of scripts that can generate Earth animations to help the general public interpret the data produced by ECMWF.
The solution uses the Blender Visualization Toolkit (BVTK), an add-on for the Blender 3D graphics software package to make it possible to use the VTK visualization library inside Blender through a graphical interface, based on node trees. The VTK objects can be used to create an image or an animation, taking advantage of the powerful instruments of 3D graphics software.
3D animations of the kind developed with the Blender Visualization Toolkit can be shown on the interactive globe at ECMWF to help people better understand atmospheric data.
GIS-based dissolve functionalities using GRASS GIS
The challenge was to address workflow problems within the CEMS-EWS European Flood Awareness System (EFAS) when a new partner is added to the EFAS domain.
The team that took on this challenge developed tools to add a new partner region, update river basins with new partner region IDs, and update grid points in river basins with new partner region IDs.
The open-source geographic information system GRASS GIS was used to develop a tool that supports dissolve functionalities and that resolves overlaps and gaps between adjacent polygons, thus automating a very manual process.
Machine learning to better predict and understand drought
The goal was to predict pixel-wise values of vegetation health in Kenya using machine learning (ML) techniques.
The project developed a pipeline of different processing components, written in Python, that can be used to apply the implemented approach on a variety of different data and regions.
MAchine learning TEchniques for High-Impact Weather (MATEHIW)
The goal for this project was to compare different ML techniques applied to the prediction of flooding events, based on ERA5 data and river discharge data from the CEMS-EWS Global Flood Awareness System (GloFAS).
The team developed a machine learning model architecture that splits the flood forecast model into two parts: a transport model that accounts for advection of water from upstream river to downstream river gridpoints, and a local model that accounts for the difference to observed values.
The two explorative projects applying machine-learning algorithms on ECMWF and Copernicus open data were supported by the EU-funded Copernicus services operated by ECMWF: the Copernicus Climate Change Service (C3S) and the Copernicus Atmosphere Monitoring Service (CAMS). The projects explored the great potential to apply ML techniques in environmental applications and have prepared the ground for further research in this field.
Thomas Lees and Gabriel Tseng teamed up with ECMWF mentors to explore machine learning techniques for predicting droughts in Kenya.
“The second edition of the ECMWF Summer of Weather Code resulted in outcomes useful for ECMWF and the broader open-source community,” said Julia Wagemann, one of the programme’s organisers. “The high quality of the results of ESoWC 2019 is testimony to the commitment and dedication of everyone involved in the programme: the mentors and the participants. We will take ESoWC’s 2019 success as a baseline to make ESoWC 2020 even better.”
More information