Last updated: 7 Sep 2020
Project duration: Sept. 2015 – August 2020
FTEs (planned / used) 8.20 / 6.50 in COSMO year 2015-2016
7.12 / 7.12 (no kidding or tuning, even though a perfect shot here seems as improbable as a hole-in-one in golf sports) in COSMO year 2016-2017
6.85 / 7.96 in COSMO year 2017-2018
7.50 / 7.20 in COSMO year 2018-2019
7.15 / 8.86 in COSMO year 2019-2020
36.8 / 37.6 in total
The aim of the project is to further improve and extend the data assimilation system and use of observations in the framework of the KENDA-LETKF in view of better convective-scale deterministic and ensemble forecasts, particularly of quantities related to cloud and precipitation. With this, the project shall also pave the way towards the use of NWP for nowcasting, as far as data assimilation capabilities are concerned. For this purpose, a main focus will be to include the use of high-resolution frequent observations in KENDA, particularly remote sensing data and observations related to the boundary layer, humidity, cloud and precipitation, as well as the surface.
In the precedent project KENDA (Km-scale ENsemble-based Data Assimilation), a prototype 4-dimensional Local Ensemble Transform Kalman Filter (4D-LETKF) scheme following Hunt et al. (2007) has been developed for the COSMO model. The main purpose of this data assimilation (DA) scheme is to provide the initial conditions both for deterministic and ensemble forecasting on the convective scale (i.e. with 1 – 3 km model mesh size). Foci of interest include improved forecasts e.g. in convective situations or low stratus conditions, and especially in the very short time range. Latent heat nudging (LHN) of radar precipitation has been integrated into the KENDA-LETKF system and found in first tests to have a positive impact on precipitation scores, without any obvious negative impact on the LETKF performance for other variables. In summer test periods performed at DWD (over up to now 5 weeks) using conventional data, LETKF plus LHN outperforms the current operational scheme based on nudging plus LHN in terms of deterministic forecast quality. Similar tests at MeteoSwiss have been altogether neutral. Tests are to be continued to confirm that the break-even point of the KENDA system relative to the operational nudging has been reached or even passed which is the most important criterion for operationability of KENDA. Furthermore, even though the Ensemble Kalman Filter (EnKF) method assumes and works best for Gaussian distributions of the model forecast and observation errors, results have been rather promising e.g. for the assimilation of cloud top height data despite very non-Gaussian observation increments.
A major strategic decision has been taken and formulated in the COSMO Science Plan to further develop the ensemble-based data assimilation based on the 4D-LETKF. Scientific reasons include its capability to use truly 4-dimensional flow-dependent background error covariances at full temporal resolution. Practical reasons include its comparatively low computational costs compared to hybrid EnVar approaches such as the scheme under development for the global model ICON at DWD, which combines LETKF with variational data assimilation. The results obtained (so far) in the predecessor project KENDA project support this strategic decision. Recent developments suggest that the LETKF also offers the possibility to be combined relatively easily with particle filter (PF) techniques if the latter operate in ensemble space. This may further enhance the capability of the system to cope with non-Gaussian probability density distributions.
The focus of the KENDA project has been on the algorithmic development of the LETKF and the use of a similar set of conventional observations that are used operationally in the observation nudging scheme. Even though implementing the direct use of several high-resolution remote-sensing data type has started, these developments are by far not mature enough yet to have quasi-operational status. Therefore, a focus of work in the next years is to continue these efforts and attain operational applicability for as many data types as possible. Algorithmic research will also be required to make use of these data and enhance forecast improvements.
Many of the major tasks of this project are ongoing activities that have already been started in the precedent KENDA project. The main reasons to organize these activities in a priority project are to foster coordination, collaboration, exchange of information, to attract additional resources, and to reflect the high relevance of this development for COSMO and its strategic goals as described in the science plan. The project is scheduled for 5 years. The main deliverable is to develop the ability of using the observations types listed in Task 2 operationally with benefit. However for many of them, the work will continue thereafter in order to achieve further improvements.
The data assimilation system and use of observations will be further developed, improved and extended in the framework of the KENDA-LETKF.
A main focus is on the use remote sensing data and observations related to the boundary layer, humidity, cloud and precipitation, as well as the surface. This includes radar data, GNSS slant path delay, cloud information and water vapour from SEVIRI satellite radiances, ground-based remote sensing data related to the planetary boundary layer, Mode-S aircraft data, and screen-level observations.
To make best use of these data may require algorithmic developments and extensions of the LETKF scheme, e.g. multi-scale multi-step approaches or specific modules to correct phase errors. High-resolution data dealt with in this project are often related to nonlinear processes and non-Gaussian errors, possibly with non-zero temporal correlations. Therefore, the question of the optimal analysis update frequency has to be studied carefully, and more exploratory research towards hybrid extensions of the EnKF, e.g. using particle filter approaches, should address issues such as non-Gaussianity or rank deficiency.
To improve the initial and boundary conditions for locally influenced weather situations close to the surface, the use of satellite data for defining the soil variables will also be explored. Furthermore, the system has to be ported from COSMO to ICON.
Even though the focus of development clearly lies on the convective scale, the system will also be maintained and slightly extended for larger-scale applications. Here, one task addresses the use of polar-orbiting satellite radiances in clear sky or above clouds.
A crucial part of the project from about mid 2017 is the porting of the KENDA system from the COSMO model to the limited-area mode of ICON (ICON-LAM).
See summary above.
Here, the individual tasks are only listed. Detailed information is available in the private pages 'Annual Tasks'.
1. Further development of LETKF scheme
Optimization, refinement and further development of the LETKF system, introduction of additive covariance inflation and/or physics perturbations, etc.
2. Extended use of observations
3. Soil moisture analysis using satellite soil moisture data
4. Adaptation to ICON-regional, hybrid methods, non-Gaussianity
There is a general scientific risk that the gain in forecast quality from the use of each of the additional observation types may be difficult to prove and does not meet the expectations. It should be noted that the use of data types such as 3-d reflectivity, GNSS slant path delay, or satellite radiances related to cloud information in a LETKF are quite novel fields of research. There is rather little experience from other research groups that could help to estimate how serious certain scientific issues really are, how they can be tackled, and how much work this will require. These scientific issues include:
In the LETKF, the available number of degrees freedom to fit the observations within the localisation scale does not exceed the number of ensemble members. Given the limited ensemble size, this rank deficiency problem poses a more severe limitation for the use of the high-resolution data sets that are in the focus of this project than for the conventional in-situ data used for the core development in the precedent KENDA project. Enhancing the effective degrees of freedom in the analysis by reducing the localisation scale may lead to imbalances and incomplete consideration of real background error correlations.
Model bias is difficult to account for in data assimilation, and current operational schemes do not do this (except that bias corrections can be applied to observations in order to unbias the latter with respect to the model). In particular, the ensemble Kalman filter (EnKF) is blind to systematic errors. In the presence of significant systematic model errors, analysis increments tend to be inferred to make up for these errors at the observation locations (and for the observed variables). However, such increments are done for the wrong reason, and usually result in erroneous corrections to the initial state elsewhere. For the assimilation of data related to cloud and precipitation the corresponding model variables are relevant, and these are often affected by model bias. The benefit from the new data types will depend significantly on the model quality.
It is noted therefore that improving the model quality is important too.
Without appropriate bias correction, model biases can have a similar effect as (typically large-scale) correlated representation errors of observations. Unaccounted observation error correlations lead to an excessive influence of the observations and too large analysis increments. This can severely compromise the benefit from spatially and temporally dense data sets.
The LETKF makes the Gaussian assumption. The probability density distributions and errors, however, are more non-Gaussian and multi-modal in the convective scale and for weather related variables such as cloud and precipitation than for variables like pressure, temperature, or wind. Thus, the issue of non-Gaussianity is more relevant for the high-resolution data types dealt in the current project. Note that if the analysis system is able to closely follow the true (observed) trajectory by assimilating frequent observations in a very rapid update cycle, it is required to treat only the part of the attractor that is close to the observations, and the dynamics may not have enough time to become strongly non-linear and non-Gaussian. On the other hand, this may imply having too little ensemble spread, in particular if the model error growth is not taken into account correctly.
Germany / DWD: Christoph Schraff, Hendrik Reich, Andreas Rhodin, Roland Potthast, Klaus Stephan, Harald Anlauf, Ulrich Blahak, Christian Welzbacher, Michael Bender, Elisabeth Bauernschubert, Kobra Khosravian, Axel Hutt, Christine Sgoff, etc.
Switzerland / MeteoSwiss: Daniel Leuenberger, Claire Merker, Alexander Haefele; Sylvain Robert (ETHZ)
Italy: Lucio Torrisi, Francesca Marcucci (CNMCA), Valerio Cardinali, Paride Ferrante (CNMCA / Eumetsat); Chiara Marsigli, Virginia Poli, Thomas Gastaldo, Tiziana Paccagnella, Mario Corbani (ARPAE-SIMC)
Russia / Roshydromet: Mikhail Tsyrulnikov, Dmitriy Gayfullin
Bick T, Simmer C, Trömel S, Wapler K, Hendricks Franssen H-J, Stephan K, Blahak U, Schraff C, Reich H, Zeng Y, Potthast R, 2016: Assimilation of 3D radar reflectivities with an Ensemble Kalman Filter on the convective scale. Q. J. R. Meteorol. Soc., 142, 1490-1504, doi:10.1002/qj.2751.
Harnisch F, Keil C. 2015. Initial conditions for convective-scale ensemble forecasting provided by ensemble data assimilation. Mon. Wea. Rev. 143: 1583–1600, doi:10.1175/MWR-D-14-00209.1.
Hunt BR, Kostelich EJ, and Szunyogh I, 2007: Efficient data assimilation for spatiotemporal chaos: a Local Ensemble Transform Kalman Filter. Physica D 230, 112 – 126.
Lange H, Craig GC. 2014. The Impact of Data Assimilation Length Scales on Analysis and Prediction of Convective Storms. Mon. Wea. Rev. 142: 3781-3808, doi:10.1175/MWR-D-13-00304.1.
Lange H, Janjic T. 2016. Assimilation of Mode-S aircraft observations in COSMO-KENDA. Mon. Wea. Rev. 144: 1697-1711, doi:http://dx.doi.org/10.1175/MWR-D-15-0112.1.
Periáñez A, Reich H, Potthast R. 2014: Optimal localization for Ensemble Kalman Filter systems. J. Meteor. Soc. Japan 92: 585–597, doi:10.2151/ jmsj.2014-605.
Schomburg A, Schraff C, Potthast R. 2015. A concept for the assimilation of satellite cloud information in an ensemble Kalman filter: single-observation experiments. Q. J. R. Meteorol. Soc. 141(688): 893–908, doi:10.1002/qj.2407.
Schraff C, Reich H, Rhodin A, Schomburg A, Stephan K, Periáñez A, Potthast R, 2016: Kilometre-scale ensemble data assimilation for the COSMO model (KENDA). Q. J. R. Meteorol. Soc., 142: 1453-1472, doi:10.1002/qj.2748.
Sommer M, Weissmann M. 2014. Observation impact in a convectivescale localized ensemble transform Kalman filter. Q. J. R. Meteorol. Soc. 140(685): 2672–2679, doi:10.1002/qj.2343.
Stephan K, Klink S, Schraff C. 2008. Assimilation of radar-derived rain rates into the convective-scale model COSMO-DE at DWD. Q. J. R. Meteorol. Soc. 134: 1315–1326, doi:10.1002/qj.269.
Zeng Y. 2013. Efficient Radar Forward Operator for Operational Data Assimilation within the COSMO-model. Dissertation, IMK-TRO, Department of Physics, Karlsruhe Institute of Technology, doi:10.5445/KSP/1000036921.
Zeng Y, Blahak U, Jerger D. 2016. An efficient modular volume scanning radar forward operator for NWP-models: Description and coupling to the COSMO-model. Q. J. R. Meteorol. Soc., doi:10.1002/qj.2904.
Zeng Y, Blahak U, Neuper M, Epperlein D. 2014. Radar beam tracing methods based on atmospheric refractive index. J. Atmos. Oceanic. Technol. 31: 2650–2670, doi:10.1175/JTECH-D-13-00152.1.