First Scalability Day charts way forward

The first ECMWF Scalability Day was held on 24 May 2016. It served to chart the way forward for the Scalability Programme, which aims to prepare the Centre for the exascale era of supercomputing. The event provided a forum for staff directly involved in the programme to explain their work, and it gave all staff an opportunity to get an update on the challenges and potential solutions across data handling, data assimilation, modelling, the software stack and hardware. It was the first in a series of similar events intended to foster discussions on how best to manage different tasks across ECMWF.

The 23 presentations that were given demonstrated that, two years after it was established, the Scalability Programme has already achieved a lot, in particular:

• The provision of a workflow for observational data pre-processing that optimises data handling for both operational suites and research experiments. The framework relies heavily on the Observational Database (ODB), whose efficiency has recently been substantially increased, producing much better scalability of 4DVar.
• The maturing of the object-oriented data assimilation framework, which will introduce a more flexible and scalable approach to producing initial conditions. This project has received significant resources over the past two years and we expect its first demonstration in 2016.
• The introduction of a grid-point module complementing the spectral dynamical model core based on the same octahedral cubic grid. In addition, an efficient way of representing and managing data structures supporting algorithmic flexibility and the parallelised handling of model fields has been developed. This will also benefit data assimilation and post-processing.
• The revision of model output data handling to reduce data movement and introduce flexible interfaces to distribute workloads on demand to potentially heterogeneous computer architectures.
• Significant progress on code optimisation, employing a variety of programming models and investigating possible benefits of new processor types. This is supported by the on-site availability of a new GPU cluster and the inclusion of accelerators in the Cray Phase 2 upgrade.
• The complementary support of ECMWF projects by external funding (through the European Commission’s Horizon 2020 framework) and bilateral projects with hardware vendors, but also with the strong involvement of ECMWF Member States across the Scalability Programme.

The success of the Scalability Programme hinges on ECMWF-wide collaboration. It requires the realisation that all aspects of the forecasting system need to be included and that obtaining efficiency gains will require some scientific rethinking. ECMWF’s new ten-year Strategy defines ambitious goals and – given limited high-performance computing resources – relies on achieving substantial efficiency gains across the forecast generation and product dissemination chain. A roadmap for obtaining such efficiency gains in the forecasting system is being drawn up. The Scalability Programme will contribute to this roadmap as its outcomes take shape. The Scalability Days will help to coordinate shorter-term and longer-term activities and to facilitate cross-departmental collaboration.