There were six working groups covering five topics. Working groups 1 and 2 covered the same topic.
Working Group 1: TIGGE-S2S Challenge: user oriented variables
Chair: Mark Rodwell
S2S/TIGGE challenge
Questions for the TIGGE/S2S Challenges: user oriented variables
One of the aims of the World Meteorological Organisation (WMO) is to encourage better communication between forecasters and users. This communication will clearly be facilitated if it can be focused around user-oriented forecast variables. Simple examples are windspeed-cubed for energy production, and the multi-variate “Discomfort Index” (which combines temperature and humidity) for the health sector. The aim of this breakout group is to discuss this strategy. A few salient questions to consider are the following:
1. Can we define simple, elegant, easily implementable user-relevant variable for each sector (power, health, agriculture, transport, finance)?
2. What are the key issues for the user (key event definitions, costs and losses)?
3. What is the current predictive skill in terms of proper scores, reliability, refinement, sharpness?
4. How do users calibrate user-oriented variables? How much extra skill do they obtain, and from which issue (bias, spread etc)?
5. What can forecast system developers learn from the calibration applied? What are the key issues for NWP development (large biases, too confident, “complete misses”, very different predictability from more standard scores)?
6. How well do models and calibration techniques preserve multi-variate dependencies, spatial and temporal relationships between variables, and plausible “trajectories” over the course of the forecast?
7. Can the desirable user-oriented variables be derived from the TIGGE and S2S database, or do we need to archive additional variables?
Working Group 2: S2S and TIGGE databases: technical aspects
Chair: Manuel Fuentes
Rapporteur: Richard Mladek
S2S and TIGGE databases: technical aspects
1. What was the biggest technical challenges to be able to participate in S2S/TIGGE projects at the beginning? What could improve next time?
2. What are the biggest challenges in the current production phase to ensure long-term minimal effort operations? How could the data archive centres support you more in that role?
3. How could we improve cooperation between data providers and archiving centres (how to communicate/automate tasks/implement checking tools etc)?
4. What are good/bad features in the design of S2S/TIGGE databases (data format/structure/encoding/compression etc)?
5. What are good/bad features in the interfaces for getting S2S/TIGGE data (web portals/Web API/direct MARS access etc)?
6. What would be the most welcomed technical update of S2S/TIGGE databases from user point of view (data formats/data access/new products like time-series etc)?
7. Do you know other data archives from similar projects which could be inspiring next time (information exchange/interoperability/data standards etc)?
8. Are you generally satisfied with the way the S2S/TIGGE databases have been created and supported until now? What would you like to change or improve the most?
Working Group 3: Processes/Forecasts
Chair: Laura Ferranti
Rapporteur: Christian Grams
Questions for Processes/Forecasts
1. How can we improve the TIGGE and S2S databases for more process diagnostics (list of variables/ resolution, frequency, reforecast configuration…)
2. Process studies/diagnostics: how to better coordinate these studies across the community (code sharing, postprocessed data…).
3. Process studies/diagnostics from TIGGE and S2S data: which areas need more attention?
4. Processes/forecasts: how can we bridge the gap between medium-range (TIGGE database with uncalibrated products) and extended-range (S2S database with calibrated products)?
5. How can we harmonize more the TIGGE and S2S databases? (pre-calibrated variables?)
6. Re-forecasts: What is the impact of ignoring the bias at medium range? What would be the optimal design for medium-range and extended range?
7. How important is it for research to have access to forecasts closer to real-time
8. How important is to diagnose processes associated with flow dependent predictability from medium to extended range?
9. Ensemble generation: how optimal is it currently for seamless prediction? How useful are the TIGGE and S2S databases for intercomparing initialization strategies.
10. What is more important for ensemble prediction: forecast frequency/ensemble size/resolution/complexity?
11. TIGGE and S2S Forecast websites (e.g. “TIGGE museum”, “S2S museum”, “ECMWF S2S products”): how useful are these products. How could they be improved?
Working Group 4: Processes/Forecasts
Chair: John Methven
Questions for Processes/Forecasts
1. How can we improve the TIGGE and S2S databases for more process diagnostics (list of variables/ resolution, frequency, reforecast configuration…)
2. Process studies/diagnostics: how to better coordinate these studies across the community (code sharing, postprocessed data…).
3. Process studies/diagnostics from TIGGE and S2S data: which areas need more attention?
4. Processes/forecasts: how can we bridge the gap between medium-range (TIGGE database with uncalibrated products) and extended-range (S2S database with calibrated products)?
5. How can we harmonize more the TIGGE and S2S databases? (pre-calibrated variables?)
6. Re-forecasts: What is the impact of ignoring the bias at medium range? What would be the optimal design for medium-range and extended range?
7. How important is it for research to have access to forecasts closer to real-time
8. How important is to diagnose processes associated with flow dependent predictability from medium to extended range?
9. Ensemble generation: how optimal is it currently for seamless prediction? How useful are the TIGGE and S2S databases for intercomparing initialization strategies.
10. What is more important for ensemble prediction: forecast frequency/ensemble size/resolution/complexity?
11. TIGGE and S2S Forecast websites (e.g. “TIGGE museum”, “S2S museum”, “ECMWF S2S products”): how useful are these products. How could they be improved?
Working Group 5: Verification/Calibration
Chair: Caio Coehlo
Rapporteur: Zied Ben Bouallegue
Questions for Verification/Calibration
1. User-oriented v process-oriented verification – what are the differences in approach? Are different approaches/methodologies essential? Are there methodologies that can be relevant to both?
2. Are there things we want to verify but can’t? Why not? (lack of observations? No obvious methodology? Lack of forecast data? (eg extreme events, multi-variate applications)
3. Observation/analysis data for verification/calibration – are the required datasets readily available? (standard datasets, gaps, specialised datasets)
4. How important is observation (or analysis) uncertainty in verification? How should we take account of these uncertainties?
5. What are the requirements for training data for statistical post-processing? How dependent are these on the parameter, forecast time-range, geographical region?
6. How are reforecasts used for calibration and verification? what is the optimal reforecasting strategy for both applications (frequency, ensemble size, …)?
7. How important is calibration of forecast products (for bench forecasters, end users, public)? Is there a minimum level of calibration that should be recommended for any public products?
8. What calibration methods are practical for centres with limited computational resource (and limited bandwidth for retrieving training data)?
9. Seamless: How consistent are the calibration/verification methods for medium-range (TIGGE) and subseasonal (S2S)? Can these be made more seamless? What are the barriers to doing this? (lack of reforecasts for TIGGE, different time/spatial scales, different levels of underlying predictability (signal/noise)
10. Inter-comparison of forecasting systems – are verification procedures/guidelines sufficient? Is the required data available? Are there any needs for standardisation (cf WMO standard verification procedures for operational medium-range (and seasonal) forecast) but not yet for sub-seasonal
11. Methodologies for calibration: type of technique(s) used for postprocessing and blending
12. Calibration verification for specific applications (renewable energy, agriculture) – general or tailored approach
13. Machine learning – prospects for using machine learning with TIGGE/S2S for calibration?
Working Group 6: Multi-model
Chair: Craig Bishop
Questions to facilitate Multi-Model ensemble forecasting
1. To what extent can multi-model ensemble forecasts be superior to single model ensemble forecasts?
2. What are the reasons for the superiority (or inferiority) of multi-model ensemble forecasts?
3. Individual model performance is constantly changing so model weights derived from last years performance may be non-optimal for this year’s performance. What are judicious statistical post-processing approaches in such circumstances?
4. How much more powerful would multi-model statistical post-processing be if centres made more of the historical performance of their currently operational system available?
5. Given that (i) error independence lies at the heart of multi-model mean superiority, and (ii) many satellite observations are deliberately thinned before assimilation to address correlated error and computational efficiency issues, should operational centres collaborate to try and ensure that different centres assimilate different subsets of equally valuable observations?
6. To what extent would individual model bias correction improve the utility of multi-model ensemble forecasting?
7. Multi-model forecasting has worked particularly well for atmospheric features that can be described with a handful of parameters such as the position, width and intensity of Tropical Cyclones. Other high impact weather features such as fronts, fog location/duration/extent or extra-tropical cyclones could be similarly described with a handful parameters. Would, for example, multi-model means of such parameters be more useful than the approach of taking the multi-model means of the raw model fields and then assessing the features?