21.02.2020
- Bug Fix in Shallow-Convection Interface
- Modifications for GPU Implementation
- Running Surface Modules in Double Precision
- Modifications for EMVORADO
- Modifications for MESSy
- Technical Changes and Bug Fixes
- Changes to the Namelists
- Changes of Results
(by Yannick Boetzel, MCH)
When computing the final tendencies after the shallow convection (itype_conv=3),
erroneously the loop index k was used instead of kdim to
indicate the lowest model level (for variable pdtdiab_con in file
conv_shallow.f90).
This fix modifies the results!
Back to Contents
Performance Optimizations for PGI
Various bugfixes and performance optimizations for PGI have been
implemented. A large part of this is the use of asynchronous kernel
launching in OpenACC: All physical parametrizations and some utility
subroutines now use the 'async' clause in all OpenACC directives.
Additionally some kernels have been improved by use of collapse
and default(present) clause, and reordering OpenACC directives.
- All OpenACC kernels in the physical parametrizations now run asynchronously.
Improved kernels by restructuring and use of collapse and
default(present).
- Added async clause in some subroutines and improved kernels.
- Added fill_array subroutines in utilities.f90.
- Added async clause to enter/exit data directives,
added default(present) to copy kernels in subroutines
src_block_fields.f90 and src_block_fields_org.f90.
- Added async clause to cal_conv_ind and ascent subroutine.
Restructured ascent subroutine by removing ldone and lzcompute arrays
in pp_utilities.f90..
GPU port of Tiedtke-Bechtold scheme and update to latest ICON version
(by Riccardo Scatamacchia, ITAF MET, and colleagues from MCH)
The Tiedtke-Bechtold scheme has been ported to GPUs. At the same time,
the code has been updated technically to the latest ICON version
(Commit c322005cdfc69fa245598c40606e150fb5dfb0e8).
Replaced STELLA by GRIDTOOLS
This is un-noticed by Fortran users (although some changes in lmorg.f90
and organize_dynamics.f90
Moved GCL Bindings to exchg_boundaries
All communications within GPUs are now done in subroutine exchg_boundaries
in module environment.f90.
GCL bindings have been removed therefore in: lmorg.f90, organize_dynamics.f90,
organize_physics.f90, src_nudging.f90, src_output.f90, src_setup.f90,
src_sfcana.f90, src_tracer.f90.
Port computation of LPI to GPU
Computation of LPI is now done on GPU.
The subroutine lightning_potential_index has been ported using OpenACC.
The interface to the subroutine lightning_potential_index has been modified,
to add the start- and end-index for the computations in j-direction
(jstartpar, jendpar). This affects also calc_tracks.f90.
Back to Contents
(by MCH)
It turned out that for small timesteps the changes to the prognostic variables
in the surface schemes are very small and cannot be represented correctly
in single precision. Therefore the possibility has been implemented to run
the surface schemes (TERRA, FLake, Seaice) in double precision, also when
running the rest of the model in single precision.
A new KIND parameter vpp (variable precision for physics) has been implemented
in kind_parameters.f90, which can be set to double precision by setting the
pragma -DVAR_PHYSICS_PREC. But this pragma only is active, if also the pragma
-DSINGLEPRECISION is set.
Chosen Method
The prognostic surface variables are still stored in single precision in long
term memory, and additional single precision fields are stored, which contain
the differences between double and single precision values, instead of storing
the full double precision field.
In this way, the "copy-to/from-block" mechanism still works, which is written
that it can work either for single or for double precision fields. But the
additional "differences fields" are not written to restart files, so at the
moment no restart is possible then.
(Note: this has been added in Version 5.07a)
Other modules of the COSMO-Model, which use variables from sfc_terra_data are
also affected, because these variables have to be transferred from vpp to the working
precision wp.
Back to Contents
(by Ulrich Blahak, DWD)
General Update of EMVORADO. The interfaces in lmorg.f90 have been
updated accordingly.
Back to Contents
(by Astrid Kerkweg)
For the re-unification between the COSMO NWP- and CLM-Versions, also
the MESSy part is updated.
- Introduce additional lalloc_xxx variables to avoid tests
with subsequent IF (ALLOCATED) / IF (ASSOCIATED).
In contrast to the original COSMO code, for MESSy the data fields
need to be pointers and not allocatable targets. To avoid the
distinction between IF ASSOCIATED and IF ALLOCATED in the code we
introduced logicals with the name lalloc_VARIABLE, which are set
to .TRUE. in src_allocation.f90 and used throughout the code
where required.
- Modifications for convection schemes:
Some adaptions are required due to the introduction of the blocking
of the physical schemes:
- The pick off of the convective fluxes in the tiedtke scheme was
adapted to the new structure. Additionally, tracers are not treated
in the convection scheme if running with MESSy.
- For (de-)blocking of additionally required fields, calls to the MESSy
interface have been added for turbulence, convection and microphysics
in src_block_fields_org.f90.
- A bugfix in conv_data.f90: move the compiler directive before the
CONTAINS, otherwise a dummy routine needs to be defined for the case
that ALLOC_WKARR is not defined.
- Run the COSMO-Model with a higher atmosphere:
If COSMO should run with a higher atmosphere, we encounter the problem,
that the parametrisations for saturation adjustment are not valid anymore.
To circumvent this problem, saturation adjustment is switched off above a certain
level. This treatment was adopted from the ICON model.
A new global variable kstart_moist has been introduced in data_modelconfig.f90.
kstart_moist is the level at which saturation adjustment calculation starts.
Per default kstart_moist is set to 1, so that nothing is changed, unless
another value is chosen for kstart_moist
Up to now, kstart_moist is only changed, if MESSy is defined for compilation.
There is a new namelist variable in PHYCTL, htop_moist_proc, with which
the user can specify from which height on the saturation adjustment is calculated.
The DEFAULT of kstart_moist is 1
- For the coupling between chemistry and radiation, coupling to interactive
CO2 and Ozone was introduced.
Additionally, for specific experiments, the solar constant was made variable.
(only with ifdef MESSy).
- Make block memory copy for tracers optional:
So far in COSMO all tracers are copied to the block memory required by the
physical processes. However, for COSMO/MESSy we can avoid to copy all tracers
(might be several hundreds) to the block memory, which reduces the memory
consumption to a bearable amount for atmospheric chemistry simulations.
Therefore the standard COSMO tracer handling has been expanded to treat the
block memory (especially its size) independent of the overall tracer number.
This has been implemented in analogy to the tracer boundary data handling.
(The code modifications have been discussed and approved in July/August
2017 with Xavier Lapillonne).
- Update implementation of STATIC_FIELDS:
In contrast to the standard COSMO in MESSy the tracer memory is static, i.e. the
shift of time levels (nnow, nnew) by index flipping is not possible.
Therefore, when firstly introducing the MESSy interface into COSMO, Oli Fuhrer
defined the preprocessor directive STATIC_FIELDS, which takes into account that
the access to the memory needs to be treated differently in specific cases.
However, using this implementation it turned out, that the distinction between
STATIC and dynamic memory access was required in additional places in the code.
These are added now.
Back to Contents
- Unification with INT2LM: Some utility modules have been harmonized with INT2LM:
- io_utilities.f90:
- mpe_io2.f90:
Proper initialization of filename and current_context_msg in SR mpe_io_open.
- parallel_utilities.f90:
Use proper KIND parameter i4 for communication buffer (AK).
- utilities.f90:
- vgrid_refatm_utils.f90:
Replaced an integer value with _8 by a correct KIND parameter _i8 (AK).
- The KIND parameter iintegers has been removed from several modules.
- calc_tracks.f90:
For calculating the echotops, -repsilon has to be used in a MIN calculation
Modified interface to SR lightning_potential_index for GPU porting.
- conv_tiedtke.f90:
Extend PGI_FIX_CONV_ACCINOUT to all calls of cu_cond_scalar
(because of a bug in PGI compiler).
- data_io.:
Fixed the definition of maximum number of I/O variables for COSMO-ART.
- io_metadata.f90:
- In single precision and resolutions < 2 km (0.02 degree) the rounding errors
when computing the start and end grid points of the COSMO domain are bigger
than the allowed deviation. Therefore these variables are now always computed
in double precision (dp instead of wp).
(routines: set_vcoord_refatm_out, make_grib_init, make_grib_grid)
- Set typeOfSecondFixedSurface for CEILING to 101 (US)
- lmorg.f90:
Add missing field for serialization.
- organize_assimilation.f90:
Replaced variable nal (which is not initialized) by nav (already impl. in 5.06b_3).
- organize_data.f90:
Fix reading/writing of SKT for restart files (itype_canopy=2).
- organize_physics.f90:
Added a model abort, if loldtur=.FALSE. or itype_vdif ≥ 0 on GPUs.
The original code is ported, but there are no regular tests.
Performance and validation needs to be re-evaluated if these features
are required on GPU.
- pp_utilities.f90:
Have to define integer values for dimensions before referencing the
dimensions in a field definition (SR lpi_spatial_filter) (AK).
- radiation_interface.f90: (by Yannick Boetzel)
Repaired a wrong acc statement.
Replaced dosleep by lm_sleep because of unification with INT2LM (US).
- radiation_utilities.f90:
Limited variable zcosthi in SR compute_sunshine_condition to upper value of 1.0
because of problems in single precision version. (US).
- src_block_fields.f90:
Set blockFieldTableSize depending on COSMOART (US).
- src_gridpoints.f90:
Removed variables from old 2-layer soil model (cdzwxx) and hence usage
of sfc_terra_data.
- src_input.f90:
Adapted to modified utilities (now use lm_sleep instead of dosleep).
Removed variables from old 2-layer soil model (cdzwxx) (SR).
- src_lheat_nudge.f90:
Replace WHERE statements within kernels with DO loops for GPU porting.
Fixed a crash for running on GPUs.
- src_meanvalues.f90:
Replaced an integer value with _8 by a correct KIND parameter _i8 (AK).
- src_obs_cdfin_print.f90:
Include a comment concerning a Cray Compiler bug for GPU.
- src_obs_gnss.f90:
Use correct KIND parameters i8 and dp.
Do not do debug output when using the NAG compiler.
- src_output.f90:
Renamed output variable RAPA to RAPA_SPPT.
Repaired a wrong acc statement.
Adapted to modified version of utilities (US).
- src_setup_vartab.f90:
Renamed output variable RAPA to RAPA_SPPT.
Set leveltype for CEILING to 2 (US).
- src_traj.f90:
Replaced an integer value with _8 by a correct KIND parameter _i8 (AK).
- time_utilities.f90:
Modified pragma CPP_DYCORE to CPP_GT_DYCORE.
- utilities.f90:
Unification with INT2LM version (US).
- vgrid_refatm_utils.f90:
Unification with INT2LM version (US)
Replaced an integer value with _8 by a correct KIND parameter _i8 (AK).
Back to Contents
| Group |
Name |
Meaning |
Default |
| /PHYCTL/ |
htop_moist_proc |
NEW |
To switch off saturation adjustment in the higher stratosphere.
(important for high atmosphere version only; only used for MESSy). |
999999.0 |
| ldetrain_conv_prec |
NEW |
Detrain convective rain and snow for Tiedtke-Bechtold scheme. |
.TRUE. |
| /TUNING/ |
tune_capdcfac_tr |
NEW |
Fraction of CAPE diurnal cycle correction applied in the tropics. |
0.5 |
| tune_rprcon |
NEW |
Coefficient for conversion of cloud water into precipitation. |
1.4E-3 |
| tune_rdepths |
NEW |
Maximum shallow convection depth (Pa). |
2.0E4 |
Back to Contents
The bug fix in the shallow convection scheme will modify results for itype_conv=3
Change in the GRIB 2 metadata: CEILING is now written with
- typeOfFirstFixedSurface = 2 [Cloud base level];
scaleFactorOfFirstFixedSurface = MISSING;
scaledValueOfFirstFixedSurface = MISSING
- typeOfSecondFixedSurface = 101 [Mean sea level];
scaleFactorOfSecondFixedSurface = 0;
scaledValueOfSecondFixedSurface = 0
Back to Contents