ECMWF is participating in a major effort by the World Meteorological Organization (WMO) to review and modernise the monitoring of the conventional components of the Global Observing System (GOS). At the moment, WMO monitoring of conventional observations is based on monthly reports produced by leading Centres. The aim is to move towards a near-real-time (e.g. daily) monitoring of the GOS in terms of data availability and data quality. This would help the WMO to report back to data providers quickly so that problems can be fixed in a timely manner.
Pilot project
Two WIGOS (WMO Integrated GOS) workshops on Quality Monitoring and Incident Management, held in December 2014 and December 2015, developed plans for a WIGOS Data Quality Monitoring System (WDQMS) comprising three components: Quality Monitoring, Quality Evaluation and Incident Management. ECMWF, the US National Centers for Environmental Prediction (NCEP) and the Japan Meteorological Agency (JMA) have agreed to participate in a Data Quality Monitoring pilot project and to provide quality information based on feedback from their data assimilation systems daily. It was decided to start with surface (SYNOP) observations and to extend the project to other observation types later. These quality monitoring reports will provide the input for the Evaluation function. This function will combine information from the quality monitoring reports with metadata about the observing stations from the WMO Observing System Capability Analysis and Review Tool (OSCAR) to generate routine performance reports. It will then be down to the WIGOS Incident Management function to assess the issues considered by the Evaluation function and to take the necessary action.
The second workshop led to the creation of a Task Team on the WIGOS Data Quality Monitoring System (TT-WDQMS) in order to test and consolidate the concept of the WDQMS. It consolidated plans for a Demonstration Project to be implemented in WMO Region I to test the end-to-end functionality and outputs of each of the three components. The Demonstration Project will run for eight months from the beginning of July 2016.
Inter-centre comparisons
This initiative represents not only a major step forward in the WMO monitoring of the GOS, it also opens new doors to exchanging information on NWP observational data quality and usage. Comparisons of the availability and use of SYNOP observations in the data assimilation (DA) systems of the three participating NWP centres are already possible in near real-time. They suggest that, when the same data is available to different centres, the statistics of ‘used’ and ‘not used’ are similar, with only a small proportion of discarded observations. For observations which are not shared between different centres, the proportion of ‘used’ data is much lower and there is a big difference in the amount of data seen by the two centres (totals of 11,688 and 621 for ECMWF and NCEP, respectively, in the example shown in the figure). In fact, this is a consequence of the different characteristics of the two DA systems. NCEP shows only the data that pass a quality control step prior to DA, therefore part of the data available that is deemed to be of poor quality or duplicate is filtered out and is not available to the DA, whereas ECMWF shows even the data that is deemed to be of poor quality and is rejected and/or blacklisted. This example shows that the procedures used at the different NWP centres must be taken into account in such comparisons.
More details on the pilot project are available here: https://software.ecmwf.int/wiki/display/WIGOS/WIGOS+pilot+project+on+data+quality+monitoring.