11.07.2013
This version contains the interfaces to the ECMWF grib library, the grib_api, to read and
write GRIB data in GRIB1 and GRIB2. Besides that, the nudging code has been revised and
a modified quality control for surface pressure observations has been implemented.
- Implementation of grib_api (GRIB1/2)
- Bug Fixes in src_runge_kutta.f90
- Revision of the Nudging Code and
Extension of Quality Control for Surface Pressure Observations
- Changes to the asynchronous GRIB-IO: mpe_io2.f90
- Changes to the Latent Heat Nudging
- Adapted COSMO-ART to new tracer version
- Technical Changes and (minor) Bug Fixes
- Changes to the Namelists
- Changes of Results
(by Ulrich Schättler)
This version of the COSMO-Model now is able to write GRIB data with the grib_api from
ECMWF. If GRIB2 is chosen as writing format, the multi-level atmospheric variables are
written using the new general vertical coordinate (with level type 150).
Note that at least version 1.11.0 from grib_api is necessary to compile
the COSMO-Model with grib_api, because of the use of the new general vertical coordinate.
The usage of GRIB2, the new general vertical coordinate (and most important
the local use sections of GRIB2) will be documented on the COSMO-Web Page:
grib
This page is still under construction. We hope to extend it until
end of July 2013 significantly.
Basic changes to implement grib_api
The basic changes have been in the modules:
Two new modules have been implemented:
- io_metadata.f90
to set most GRIB metadata.
- vgrid_refatm_utils.f90
this is not really related to GRIB, but to get a consistent treatment of
all vertical coordinate and reference atmosphere parameters.
Implemented new GRIB2 leveltype 'generalVertical' and
'generalVerticalLayer'
The main difference to the other vertical coordinate types 'hybrid' and
'hybridLayer' is, that now there is no computational algorithm how the
vertical layers are constructed, but a full 3D file (called a HHL-file for the
height of half levels in COSMO) is provided, that has to be processed together
with other 3D variables defined on these levels.
With this new vertical level type, also some new keys were introduced for
the WMO GRIB2 standard, namely:
- nlev (alias for numberOfVerticalCoordinateValues)
nlev is ke+1 in COSMO, because the number of half
levels are specified here.
- numberOfVGridUsed
represents the vertical coordinate type ivctype.
- uuidOfVGrid
is a unique identifier for a special HHL-file.
HHL-files can only be created by the INT2LM, not by the COSMO-Model.
These keys replace the vertical coordinate parameters, which are present in
data with level types 'hybrid' or 'hybridLayer'.
The new general vertical coordinate and its usage in the COSMO-Model system will
be documented and explained in detail on the COSMO web page (coming soon).
Changes in the COSMO-Model to implement the new vertical coordinate
New module vgrid_refatm_utils.f90
This new module contains data structures, variables and routines necessary
for dealing with the vertical grid and the reference atmosphere. The new
data structures are:
- refatm_type: which contains variables for the reference atmosphere
- vcoord_type: which contains variables for the vertical grid
This module also contains data and routines, which have been in other modules before:
- routines from meteo_utilities to compute the different reference
atmospheres (reference_atmosphere_xx
- data for the vertical coordinate parameters from data_grid_lm
and data_grid_in, resp.
The module vgrid_refatm_utils also contains subroutines to create a UUID and
to compare different UUIDs.
Read HHL from an extra file
Implemented possibility to read HHL from an extra file with full precision
(but only when reading or writing GRIB2).
For that, 2 new Namelist switches have been implemented in group /GRIBIN/:
| Group |
Name |
Meaning |
Default |
| /GRIBIN/ |
ydirhhl |
directory which contains the HHL-file |
' ' |
| ynamhhl |
file name for the HHL-file |
' ' |
If a HHL-file has to be read, the readloop in src_input is invoked again after
reading the laf-file, to also read the HHL-file.
Transferring the vertical coordinate and reference atmosphere parameters
In GRIB1 (and NetCDF) the vertical coordinate and reference atmosphere parameters
are transferred from INT2LM to the COSMO-Model (or from COSMO-Nudging to COSMO-forecast)
via the meta data. This is no more possible in GRIB2 because of the new general vertical
coordinate. But these informations, which are set in the INT2LM, are still needed in
the COSMO-Model to compute the proper reference state for the model.
In order to provide the correct information in both programs, a few sets of default
parameters have been implemented in the new module vgrid_refatm_utils.f90.
These sets are numbered consecutively and the corresponding number of a special set
is written into the localDefinitionNumber of the fields P or PP for
reference atmosphere sets and in HHL for the vertical coordinate parameters.
In this way, the different programs can determine the correct set of parameters when
processing the corresponding variables.
Defining a new set of parameters for either the reference atmosphere or for the
vertical coordinate parameters therefore is now more complicated, because it requires
to change the source code of the programs. But this is only a first implementation to
start the tests with GRIB2. We can think of a better solution for that problem soon.
Up to now only few set of parameters are implemented. We will extend these sets in
the near future with all important settings that are in use.
Some more organizational data structures
Introduced arrays to convert some GRIB1 codings to grib_api strings
- GRIB1 level types: ylevltypes_in, ylevltypes_out
- GRIB1 time range indicators: ysteptypes
- GRIB2 scaleFactors / scaledValues: scalefactors
Back to Contents
(by Michael Baldauf)
- Use of time level nnow (instead of nnew) for the moisture
variables in the water loading contribution of the buoyancy.
In the Runge-Kutta scheme, nnew for the moisture variables actually means
the old time level. Therefore, using nnow leads to a better
coupling of the dynamics to the moisture physics (concerning this water
loading contribution).
This correction has the potential to improve the precipitation forecast in strongly
convective situations.
This measure is a bug fix and will change the results if itype_fast_waves=2
is used.
- Exchange of wadvt in the case ldyn_bbc=.TRUE.:
This bug fix guarantees reproducibility for different processor configurations
in the case itype_fast_waves=2 and ldyn_bbc=.TRUE. .
In this case, results are changed.
Back to Contents
(by Christoph Schraff)
- New option for quality control (QC) of surface pressure observations against
the fields that provide the lateral boundary conditions ('LBC-QC').
This includes a threshold QC for each individual pressure observation as well
as a spatial consistency check.
- New option to read observations from NetCDF feedobs (or feedback) files
('fof' files).
- As an option, the simulated observations from the feedobs files can be
deployed as observation values, so that they may serve in an OSSE setup as
synthetic observations derived from a nature run.
- As an additional option, the (simulated or original) observations can be
randomly perturbed.
Observations can be read both from ('cdfin') NetCDF observation input files
and from 'fof' files in the same model run.
- More relaxed conditions on variable (names) required to exist in ('cdfin')
NetCDF observation input files (in order to better cope with historic cases);
the relaxed conditions relate to:
- variable name for station ID;
- variable names for main observables in single-level reports;
- variable name(s) for vertical coordinate in aircraft reports;
- variable for roll angle in aircraft reports.
-
Restructuring of the code so as to have one module for the observation
operators (src_obs_operator_conv.f90) and another one for the threshold
quality control (src_obs_qc_conv.f90) for conventional in-situ and profiling
data.
Both modules and their routines have clean interfaces which consist only of
subroutine arguments and of fortran parameters (plus station ID) from a
limited set of data modules that can easily be used by other programs.
This allows these two modules to be shared now with the 3DVAR-LETKF package
of DWD.
Note that the spatial consistency checks (for surface pressure and integrated
water vapour) are not part of the QC module and hence the shared code.
In the new version, the observation operators and the QC (which are both used
for the nudging, verification, and LETKF), and the computation of other local
information used purely for the nudging, are much more disentangled.
- bit reproducibility irrespective of the domain decomposition re-established,
due to a correction of a minor bug introduced in V4_22 at the reduction of the
geostrophic wind increments depending on the geographic latitude.
- other minor bug fixes, e.g.:
- QC threshold for passive humidity obs from upper-air single-level reports
corrected (before, they were zero when using the operational settings);
- correction of a bug because of which pressure increments derived from
multi-level reports and flagged passive in the individual threshold QC
could never be accepted in the spatial consistency check;
- surface pressure increment in height and thickness check taken into account
now even if surface pressure has been rejected by threshold QC;
- the lowest height observation increment of a multi-level report, that is
written to the SOR and used for the LETKF, is now computed in a way (in
routine 'ps_obs_operator_mult' instead of 'mult_obs_operator(_z)') which
is consistent with the surface pressure increment used in the nudging;
- GPS obs type set correctly in data structure 'cma'.
- minor bug fixes affecting verification and other diagnostic output, e.g.:
- spatial consistency check also performed for observations rejected in the
observation pre-processing;
- surface pressure increments derived from multi-level reports written to
the VOF also if rejected by QC;
- upper-air height obs, even from radiosondes, now flagged to passive,
except for the lowest radiosonde height obs (to be used in the LETKF);
- removal of QC (flags) for aircraft 'height' (because this is not an
observation);
- diagnostic recording of rejection of screen-level obs (on file YUSTATS)
corrected.
- changes to the VOF file YUVERIF:
- the condition to write a radiosonde height increment is now the existence
of the height obs instead of the temperature obs, i.e. height increments
are more rare now;
- simulated observations of zenith total path delay (ZPD) also written to VOF
(no new VOF version is defined - the program 'lmstat' will not read and
write simulated ZPD).
- changes to the NetCDF feedobs file (NEFF) output:
- surface pressure obs error converted from height to pressure units (not only
in feedobs file output, but generally within COSMO (the ODR));
- flight phase now set in the report header only, not any more in report body;
- GPS station height filled into height value in report body, to avoid
problems in LETKF;
- simulated values of GPS zenith total path delay (ZPD), cloud base height,
mid-level and high cloud cover additionally written;
- bug fix: flag of lapse rate check also written.
- general technical changes:
- all direct calls of MPI routines in the data assimilation code replaced
by calls of routines from parallel_utilities.f90; for this purpose,
new routines introduced to module parallel_utilities.f90
(see below);
- all statement functions replaced by elemental or intrinsic functions;
- change of data structure: simulated observations instead of observation
increments stored in 'SOR' arrays (see data_obs_record.f90);
- special comments introduced to indicate common usage of modules by the COSMO
and the 3DVAR-LETKF packages, with indication of current code owners for
each of the packages.
- (technical) changes related to the writing of feedobs files:
- added routines for writing volumn radar data to the NEFF file (to prepare
the processing and use of 3D radar data in the LETKF, by Ulrich Blahak).
- common modules ('mo_*') from 3DVAR-LETKF package updated to 3DVAR-V1_23,
svn cycle 9070; e.g. new variables defined for writing cloud top height
and cloud type; module 'mo_fdbk_io.f90' added.
New modules in the nudging code:
- src_obs_operator_conv.f90: observation operators for conventional data
- rc_obs_qc_conv.f90: quality control (QC) for conventional data
- data_obs_qc_limits.f90: QC thresholds (moved from 'data_nudge_local.f90')
- src_obs_fdbk_in.f90: reading obs from feedobs (feedback) files
- mo_fdbk_io.f90: routines and derived type for I/O of feedback files
- mo_random.f90: random number generator
Back to Contents
(by Ulrich Schättler)
Besides the implementation of grib_api interfaces, some more changes have been
implemented in mpe_io2.f90:
- Because of problems with array-bound checking:
- Inlined subroutine buffer_list_probe_head into subroutine
buffer_list_write_action
- subroutine buffer_list_finalize_write: Removed the last call to
subroutine buffer_list_write_action
(because of problems on IBM pwr7 when turning on array bound checking)
- Implemented different strategies for gathering the data from the compute tasks
to the I/O-tasks:
- slow and sorted:
the different records are sent in an ordered manner to the I/O tasks, as is
done in synchronous mode. The records in the grib file appear in the same order
as in a pure sequential program.
This is the way it always used to be.
- buffered:
records are received from any task, but are eventually buffered and written
in a sorted manner. Again, the records in the grib file appear in the same
order as in a pure sequential program
But this can lead to memory problems because of exhausting buffering!
- fast and unsorted:
records are received from any task and are directly written to the file
without buffering. Therefore records can occur in a different order.
To control the way how the I/O communication is done, the 3rd character of the
Namelist variable ymode_write is used. Up to now, only the first character of
this 3-character string has been really used (the second character is used for
model-internal purposes).
Re-interpretation of the 3 characters in ymode_write
- 1st: 'r', 'w' for reading / writing (used up to now)
- 2nd: for library used: 'g','1','2' for DWD griblib,
grib_api/Grib1, grib_api/Grib2 (can only be used model-internal)
- 3rd: for speed of I/O communication and writing:
- 's' or ' ': slow and sorted (as it was up to now;
the ' ' is for backward compatibility
- 'b': for buffered communication and sorted output
- 'f': for non-buffered, but fast communication and un-sorted output
Back to Contents
(by Klaus Stephan)
Revision of data reading procedure, to avoid the former rewind of the radar data files.
Now, the data of one file are read only once, shortly after opening the data file.
The time assignment is done in a next step, as well as the identification as precipitation or
quality data. The change also affects the subroutine lhn_sumrad, where no reading has to be
performed anymore. In addition this subroutine has been changed a bit, which is
slightly changing the results.
There is a small inconsistency between former and new subroutine considering the precision of
KIND=ireals and KIND=irealgrib. In the former version a observation point was added
if ds_rad(ind) > 0.0_irealgrib.
In the new version the observation is already converted to ireals (SR lhn_obs_read), therefore the
condition is changed to datafield_all(i,j,n) ≥ 0.0_ireals.
The former normalisation term ndata is now space dependend, which better accounted
for data outage of single radar stations (ie. datafield_all(i,j,n) < 0.0_ireals).
All input procedures have been extended to work also with grib_api (GRIB1)
Back to Contents
(by KIT)
All COSMO-ART and POLLEN tracers are now handled by the new tracer module.
Therefore, the corresponding code could be eliminated again in most COSMO-modules.
Back to Contents
- data_io.f90:
- changes to the data type pp_nl:
added:
- igribapi_id: to store the handler for the grib sample for every output group
- lsfc_ana: to indicate whether a surface analysis field will be written
removed:
- n_num: for current grid number, when nesting (which was never done)
- ysystem: only 'file' accepted anyhow
- ydbid: database handling has been eliminated
- ydbtype: database handling has been eliminated
- ydbpw: database handling has been eliminated
- New Namelist variables to set additional GRIB2 meta data
- nlocaldefnr: local definition number for GRIB2 local section
- nsubcenter: originating sub-center identification
- data_modelconfig.f90:
- Introduced endlon_tot, endlat_tot as global variables
(are computed in src_setup.f90)
- Removed vertical grid and reference atmosphere variables and put them
to vgrid_refatm_utils.f90
- data_satellites.f90:
- Extensions to provide GRIB2 shortnames and additional meta data
ychan_name: Names of channels used (IRx.y, WVx.y)
- Renamed nlist_chan to nchan_list (for consistent naming)
- fast_waves_sc.f90:
- Improved 3D divergence damping by correct calling of subroutine
l_calc_lhs_at_1st_RKstep and setting appropriate rhs (dependent on
switches l_no_div_damping_in_1st_step, ...
- Adaptation of the dynamical bottom boundary condition to the newer formulation
of the buoyancy term and small bug fix in the use of dw_dt_slow.
(changes in results only, if ldyn_bbc=.TRUE. or 3D div.-damping is used,
which must be activated by an internal switch l_3D_div_damping=.TRUE.)
- mpe_io2.f90:
testing with array-bound checking on the IBM pwr7 showed two problems:
- subroutine buffer_list_write_action: call to subroutine
buffer_list_probe_head could be eliminated by inlining this
subroutine (?? problems when passing non-associated variables?)
- subroutine buffer_list_finalize_write:
just commented the last call to buffer_list_write_action
(which had a comment: "paranoia" anyhow)
- organize_data.f90:
- Removed Nest-handling from this subroutine
- Initialize variables ylevltypes, ysteptypes, rscalefac
- organize_dynamics.f90:
Conditional namelist setting of lw_freeslip (Uli Blahak).
(no change of results; prevents model breaks in idealised tests)
- organize_physics.f90:
Compute depth of soil half layers (czhls) in all subdomains (US)
- organize_satellites.f90:
Extensions to provide GRIB2 shortnames and additional meta data
- parallel_utilities.f90:
- new routines reports_all2all, scatterv_values, and gatherv_values;
- if(n)def NOMPI: compiling option introduced to make this shared module
compatible with the 3DVAR-LETKF package (so that binaries of the latter
can also be made without using the MPI library).
- src_artifdata.f90:
- Clarified example given for usage of COSMO-Tracer construct
- Use subroutines and variables for vertical grid and reference atmospheres
from module vgrid_refatm_utils.f90
- src_input.f90:
- reorganized check_required, fill_gribarray and
get_vertcoord slightly to adapt to grib_api meta data
usage and handling
- extended the read_loop to also read HHL-files
- src_output.f90:
Moved subroutines for setting I/O meta data to new module io_metadata.f90
- src_runge_kutta.f90:
- Proper calling of subroutine l_calc_lhs_at_1st_RKstep in the case of
3D divergence damping; (changes only, if 3D div.-damping is used)
- Changes to adapt COSMO-ART to new tracer module: some dependencies to
COSMOART and POLLEN deleted, because this is now handled by the tracer module
- src_setup_vartab.f90:
Added new fiels SP_10M for output of near-surface analysis fields
- src_tracer.f90:
Correction of a tracer-index
- turbulence_interface.f90:
Removed memory leak: if h_ice_p is allocated, it has to be deallocated again
Back to Contents
There were new Namelist variables in the following groups:
| Group |
Name |
Meaning |
Default |
| /RUNCTL/ |
lroutine |
NEW |
to indicate an operational run (if .TRUE.).
This variable is used to set GRIB2 meta data. |
.FALSE. |
| /IOCTL/ |
nsubcenter |
NEW |
originating sub-center identification for GRIB |
255 |
| nlocaldefnr |
NEW |
to specify local definition number for GRIB
local sections. The default value of -1 means,that no local section is
present. |
-1 |
| yform_read |
new values |
To specify the format of the input files.
New value apix is now accepted to read GRIB data with ECMWF's
grib_api. |
-1 |
| /GRIBIN/ |
ydirhhl |
NEW |
directory that contains the HHL-file
(which is necessary for reading / writing the GRIB2 general vertical
coordinate). |
' ' |
| ynamhhl |
NEW |
name of the file that contains the HHL field
(which is necessary for reading / writing the GRIB2 general vertical
coordinate). |
' ' |
| /GRIBOUT/ |
yform_write |
new values |
to specify the format of output files
New values are now accepted:
- api1 to write GRIB1 data with ECMWF's grib_api.
- api2 to write GRIB2 data with ECMWF's grib_api.
|
'grb1' |
| ysystem |
DELETED |
to specify a data handling system.
But only file was used up to now. |
|
| ydbtype |
DELETED |
to specify the data base type.
But this option is not supported any more. |
|
| n_num |
DELETED |
to specify the nest number
But the nesting was never finally implemented and has been removed now. |
|
| /NUDGING/ |
qcflbcp |
NEW |
enhancement factor to the threshold used for the check of
surface pressure obs against lateral boundary (LBC) fields
(if equal to zero then this LBC-QC is not performed at all).
|
1.4 |
| irun_osse |
NEW |
switch for the model run to derive obs values
from the feedback file 'fof':
- = 0: the original observations are used as obs values (default)
- > 0: index of model run (in dimension 'd_veri' of 'fof' file)
from which the simulated obs are used as obs values;
|
0 |
| losse_fg |
NEW |
decides whether first guess check flags
from the feedback file 'fof' is used:
- = true. : first guess check flags from 'fof' are converted into
'dataset' pre-processing flag; observation status
remains 'rejected'
- = false.: first guess check flags are discarded, and the
related observation may be used actively; (default)
|
.FALSE. |
| fperturb |
NEW |
factor to the obs error variances to define the size of random
perturbations added to the observation values (only for
(simulated or original) obs values from feedback file 'fof').
|
0.0 |
| iseed |
NEW |
external seed for random number generator. |
0 |
Back to Contents
Nudging:
With default settings, the main changes, limited to some occasions, are due to
the new quality control of surface pressure observations against the fields
that provide the lateral boundary conditions (LBC-QC).
Without LBC-QC, changes are usually very small, due various minor bug
corrections in the quality control and due to different rounding errors.
However, there is no backward bit reproducibility.
Latent Heat Nudging:
The results of the latent heat nudging have been changed slightly due to changes
of precision for some variables (see comments above).
Dynamics:
Due to the bug fixes, the results are changed in the case itype_fast_waves=2
(and also for ldyn_bbc=.TRUE. in connection with itype_fast_waves=2).
(See above)
Results of applications not using data assimilation or the new fast waves solver
have not been changed!
Back to Contents