Simplified relations between near surface model variables and tunable parameters of the transfer scheme in the new PBL parameterization scheme.

abstract of the talk:

Recognising that most of the systematic errors of the near surface variables (T2m, Td2m and V10m) for clear sky conditions are related to specifications of the transfer scheme (describing the transport of quantities between the rigid surface of the earth and the lowest model layer), it is important to know the special relationships between those variables and uncertain formulations, definitions and parameter values of that scheme in order to deduce how to change specification there. Important aspects are, besides others:

• Crucial assumptions of the scheme.
• Definitions of the 2m- and 10m-values (especially in mountainous regions)
• Crucial parameters and their influence on the daily cycle of near surface parameters.
• The influence of the soil moisture analysis (SMA).
• The main differences to the old transfer scheme will also be discussed

Open discussion about:

• Selection of representative weather stations (SYNOP) to be used for near surface meteorological parameters verification
• Best use of near surface parameters

Marco Arpagaus/Mathias Raschendorfer introduce the problem coming from WG3 experience
Ulrich Damrath, M. Ferri, Carlo Cacciamani, F. Schubiger, G. Galanis can briefly summarise ideas on this topic coming from WG5 experience and can drive an open discussion.

WG3 viewpoint (M. Arpagaus):

Matthias Raschendorfer focused the representation of SYNOP stations by considering the definition of near surface parameters (e.g. 10m wind) within the model and the WMO rules for the measurement of near surface parameters and will then discuss how well these two "values" can possibly match (e.g. for mountainous regions). It may be worthwhile to then discuss possibilities to select representative stations by e.g., considering the environment of a station (spatial variability within the model grid box), comparing super-observations with larger model domains, using objective algorithms which select representative stations as a function of parameter, meteorological situation and "representativity- radius" (e.g. 7km at the moment), etc.

WG5 viewpoint (U. Damrath/F. Schubiger)

One should be careful concerning the choice of point for surface parameter verification. Stations for verification should be representative at least for the scale of the model. Therefore stations in deep values and on the peaks of the mountains are only suitable to declare that LM forecasts are not useful for such points. On the other hand any user may be interested in LM forecasts over complex terrain. In this case one has two opportunities:

1. Say to the user: Use the forecast with care!
2. Interpret forecasts using statistical methods

Problems:
Representativity of stations;
Conditional verification: precipitation only for strong winds; 2m temperature and dew point only for clear sky

Verification with "feedback" on the physics parameterizations in the case of the transfer scheme in the new PBL parameterization scheme.

abstract

Apart from documenting the model quality, the verification should have a "feedback" on the model design. Therefore methods have to be developed to minimize model uncertainties. Of course, a kind of conditional verification, where dependencies of model errors on special situations (cloudiness, external parameter values, etc.) can be demonstrated would be a good start. If the model errors can be restricted mainly to wrong values of uncertain model parameters, the desired "feedback" could be achieved by determining the optimal values of those parameters. This can be done e.g. with a variational approach or analytically by the help of simplified model relations. Both possibilities will be explained using the example of two parameters (one of the transfer scheme and one of the soil model) to be tuned with or without a running soil moisture analysis (SMA).

Verification indices and assessment of statistical significance

abstract

Some indices widely used for the verification of deterministic (threat score, bias, false alarm rate) and probabilistic forecast systems (ROC area, Brier Skill Score, Cost-loss Analysis) are presented. The problem of the statistical significance of a difference in the scores obtained by two different forecast is also discussed and are presented some methodologies of hypothesis testing (e.g. resampling technique).

Application of a Statistical Methodology for Limited Area Model Intercomparison using a Bootstrap Technique

abstract

Within the European project "INTERREG II C" it has been applied a numerically based statistical method designed to discriminate significative differences between precipitation skill scores associated to different model forecasts. A Bootstrap resampling technique is used as hypothesis test. Three different operational Limited Area Models (LAMs) are evaluated over the Piedmont and Liguria Regions from 1st October 2000 to 31st May 2001 as a test of the statistical method.

Short presentations of last "standard" verification results in the COSMO countries (10 min. each):

1. DWD (ex.: Current results of operational verification using radiosonde data; last results on verification of surface weather elements, U. Damrath)
2. Meteoswiss (ex.: Precip verification LM-DWD vs aLMo; F. Schubiger)
3. UGM (verification of LM in Italy: M. Ferri)
4. Greece (verification of LM in Greece: G. Galanis)

Progresses on the precipitation verification process in Piedmont and Northern Italy (PowerPoint presentation, 0,3Mb)

abstract

The results of two years of precipitation verification over the Piedmont basins will be presented, including some climatological aspects and, moreover, some result obtained with the data shared among other italian regions.

New results of radar verification

abstract

The talk will present results about verification of precipitation using radar data that use "pattern recognition" technique in order to investigate the realism of the LM simulation of precipitation pattern.

Discussion

The problem:
Precipitation is a highly variable quantity. Observation networks are often not enough dense to investigate the small scale features of the precipitation fields. LM is able to produce such a small scale structures.
How can we make a "sound" verification of these small scale features?
Do we probably have to move the interest towards new quantity to verify like averaged values (in space and time), integrated quantities, frequency of occurrence of intense events in selected areas, precipitation over a defined threshold, in a selected sub-area of the integration domain instead that the grid point verification ? Are the common scores used (BIAS, TS etc..) adequate to do this job ? How can we correct them ? How can we use the data ? Up-scaling of LM QPF to the resolution of the GTS observing network ? Definition of probability functions of events predicted by LM in the area where the up-scaling is achieved ?

Need to investigate more clever methods to study the realism of the LM forecast!
Investigation of the spatial structure of the simulated/observed precipitation fields:
Evaluation of space correlation lagged in space and time between observed and simulated fields;
Pattern recognition of observed/simulated mesoscale meteorological structures and pattern correlation between observed and simulated fields. What kind of data we need to achieve these new methods ? Are they available ? If yes: how can we obtain them ?….

Discussion with chairman: Carlo Cacciamani (ARPA-SMR)

Actions and efforts must be done to start an exchange of non-GTS and radar data among the COSMO partners. The goal is to constitute a data base of these type of data to allow a sound verification of LM; first priority: daily values of accumulated precipitation (from 6 to 6 GMT if possible). Second priority: radar data. 1 2d parameter (precipitation or reflectivity)

Discussion with chairman: Ulrich Damrath (DWD)

Proposal of Periods for the "test" cases at ECMWF:
first period : MAP period: SEP-NOV 1999
second period (summer 2001 ?)
Needs of input from COSMO WG3;
Case studies: 15 IOPs of the MAP field phase + other interesting case proposed by the Greece
colleagues + other interesting case occurred recently (ex: adriatic "bomb")

Discussion with chairman: Massimo Capaldo (UGM): Setup of a common verification package at ECMWF

We need to propose a person (called "NN3" hereafter) inside our WG5. A "solid" software must be installed at ECMWF.
Parameters to verify in the global domain at ECMWF: T2m, TD2m, Wind, SLP, vertical profiles; more detail defined during the 3th WG5 meeting that will be held in Italy (Torino) next year (it may be around January 2002).
Concerning the verification procedure at ECWMF, discussion on the following items:

1. Scientific requirements (scores, methods, weather elements)
2. Practical requirements (access to observations, access to forecasts)
3. Software requirements (How to use the verification script(s), production of verification metafiles, production of summaries and so on)
4. N3 requirements
5. If any user want to verify a single forecast or a couple of forecasts, how should it be done? (delegation to N3 or using a script written by N3(a,b,..))
6. If ther is no N3, what can we do?

Discussion with chairman: Carlo Cacciamani & Marco Arpagaus

General discussion about WG3 and WG5 activities. Production of a short report with some recommendations which have been came out during the meeting