The multi-model Sub-seasonal to Seasonal (S2S) prediction database operated at ECMWF since May 2015 has successfully been synchronised with a second S2S database run by the China Meteorological Administration (CMA).
Facts and figures (June 2016)
- Total size of the database today: 40 terabytes, 1 billion meteorological fields
- Real-time forecasts: 6 terabytes
- Re-forecasts: 34 terabytes
- Real-time forecasts currently grow by about 750 gigabytes/ month, while re-forecasts grow by about 1.5 terabytes/month.
- Number of users: 498 from 68 countries.
To achieve the timely synchronisation of the two databases, Dr Chaoyang Sun from CMA spent one year at ECMWF as a visiting analyst to design and implement an appropriate synchronisation system. Chaoyang participated in all decisions related to the development and operation of the S2S datasets. He developed a system to transmit to CMA not only routinely acquired near-real-time data or on-the-fly re-forecasts, but also the re-forecasts from fixed configurations which were ingested at ECMWF before the database was opened.
Access to ECMWF’s S2S database
The S2S database can be accessed using two methods:
- A web interface (http://apps.ecmwf.int/datasets/data/s2s/), where users can navigate through the S2S dataset in a dynamic and user-friendly way. After each selection, the web page is updated automatically in order to reflect the availability of data. The purpose of this interface is to help users navigate the content of the database, get familiar with it, extract sample data interactively and/or obtain the script to download data in batch. An additional tool, the S2S history page (http://apps.ecmwf.int/datasets/history/s2s-prod/), shows a summary of available dates by data provider and cycle.
- A batch interface using the ECMWF Web API (https://software.ecmwf.int/wiki/display/WEBAPI/What+is+ECMWF+WebAPI), the preferred way for data download via scripting languages such as Python.
One of the main challenges of the S2S database is to provide easy access to re-forecast data. With the different configurations of the various models, it can be quite difficult for users to grasp what is available, let alone write the extraction scripts to retrieve the data in an efficient way. To facilitate this, documentation is available at https://software.ecmwf.int/wiki/display/WEBAPI/S2S+reforecasts+retrieval+efficiency. This shows users how to access re-forecasts efficiently from the archive. Note that this page and related S2S pages can be ‘watched’. This is an extremely useful feature if users want to be notified of changes. Suggested areas to watch are the ‘News’ section (https://software.ecmwf.int/wiki/display/S2S/News) and the section on ‘Issues with data’ (https://software.ecmwf.int/wiki/display/S2S/Issues+with+data).
A mechanism was also built in to enable the retransmission of old cycles. If there were any outage in routine transmission, it would thus be possible to resend the data at a later time. CMA is using ECMWF’s MARS system to support its S2S database, and Chaoyang’s stay at ECMWF helped to configure CMA’s MARS instance. Chaoyang returned to CMA at the end of June 2016. He will continue to be involved in the operation of the S2S database at CMA.
The database works by transferring data from each data provider using ECMWF’s data acquisition system. The data is then pushed to CMA via ECMWF’s Product Delivery System (ECPDS). ECMWF’s contribution is handled like that of any other data provider, so the synchronisation of ECMWF’s contribution to CMA happens automatically.
The S2S database is part of the S2S project, a joint research initiative launched in 2013 by the World Weather Research Program (WWRP) and World Climate Research Program (WCRP). The project’s main goal is to improve forecast skill and the understanding of the sub-seasonal to seasonal timescale, and to promote its uptake by operational centres and its exploitation by the application communities. The database currently includes ten models. A further model, that of the Korean Meteorological Administration (KMA), is due to be ingested upon the completion of tests.
Web links
S2S Project website: http://www.s2sprediction.net/
S2S at ECMWF: http://s2s.ecmwf.int
S2S at CMA: http://s2s.cma.cn
S2S model descriptions: https://software.ecmwf.int/wiki/display/S2S/Models