Performance of deterministic numerical weather forecasts during the FROST-2014 project field...
-
Upload
dylan-boone -
Category
Documents
-
view
217 -
download
1
description
Transcript of Performance of deterministic numerical weather forecasts during the FROST-2014 project field...
Performance of deterministic numerical weather
forecasts during the FROST-2014 project field campaign
Anastasia Bundel (1), Anatoly Muraviev (1), Pertti Nurmi (2), Dmitry Kiktev (1), Alexander Smirnov (1), and
Andrea Montani (3)(1) Hydrometeorological Research Centre of Russia, Moscow, Russian
Federation, (2) Finnish Meteorological Institute,
(3) HydroMeteoClimate Service of Emilia-Romagna Region
What’s new since the 4th FROST meeting
• FROST observations were additionally controlled, scores recalculated
• Verification tools on the FROST site are developed: - Bootstrap CIs are added in Monitoring:
- Point forecast and diagnostic data viewer – selection of stations, CSV export, more user-friendly interface
http://frost2014.meteoinfo.ru/forecast/point-forecast-and-diagnostic-data-viewer
CIs are not available yet for station aggregations
Verification setup
Mountain cluster: 25 stations Coastal cluster: only 6 stations
Period: 29 Jan – 15 March 2014 –> homogeneity among all the models (COSMO-Ru1 started 29 Jan 2014)Only nearest point approach (with its shortcomings)!
48 hr / 1 h 2 km
INCA 24 (precip 10 min)
48 h / 1h (precip 10 min)
1 km
ARPA-DET 2 72 / 3h 7 km
Joint 24 78 / 1 h Pointwise
+
Only uncorrected forecasts were verified!
All the models, T2m MAE, Mountain, 00h run
0
1
2
3
4
5
6 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET
Big errors of KMA-NIMR at the initial steps
Nowcasting effect
highest-resolution GEM-250 best overallHARMONIE best during the daytime (up to ~1500 m)
lowest-resolution 7-km models worst
All the models, T2m MAE, Coastal, 00h run
0
1
2
3
4
5
6 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET
Big daytime errors of GEM
HARMONIE’s daytime minimum-nighttime maximum persists
T2m MAE, 00h run
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru2GEM-2.5KMA-NIMR
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1GEM-1HARMONIENMMBINCA
Mountain: COSMO, INCA, NMMB: min errors during the nighttime GEM, HARMONIE, KMA-NIMR: min errors during the daytime
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru2GEM-2.5KMA-NIMR
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1GEM-1HARMONIENMMBINCA
Coastal: GEM diurnal cycle is opposite to the Mountain one
T2m MAE, 7km models, 00h runEffect of driving model (GME for COSMO-Ru7 vs. ECMWF-IFS for ARPA-DET)
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru7ARPA-DET
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru7ARPA-DET
Mountain Coastal
Effect of resolution, T2m MAE, 00h run
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1COSMO_Ru2COSMO_Ru7
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
GEM-1GEM-2.5GEM-250
• Clear effect of resolution – the higher, the better (closer nearest point due to better orography!)• COSMO-Ru1 and GEM-1 have comparable MAEs• COSMO-Ru2 is closer to COSMO-Ru1 than GEM2.5 to GEM-1• Potential for COSMO-Ru1 (increase in domain, etc.)
Mountain
Effect of resolution, T2m MAE, 00h run
• COSMO family: similar scores• GEM-250: flattened diurnal cycle• GEM-1 and GEM-2 – diurnal maximum is opposite to that of the Mountain cluster
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1COSMO_Ru2COSMO_Ru7
0
1
2
3
4
5
6
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
GEM-1GEM-2.5GEM-250
Coastal
All the models, RelHum, MAE, Mountain, 00h run
0
5
10
15
20
25 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET
Nowcasting effect
GEM-250 best again (as for T2m)Otherwise, no clear dependence on resolution
COSMO-Ru7 worst during the daytimeMAE max at 10-14 UTC for most models
COSMO-Ru1
Effect of resolution, RelHum MAE, Mountain, 00h run
• COSMO family: effect of resolution after initial steps• GEM family: effect of resolution is clear from 10 to 20 UTC
0
5
10
15
20
25
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1COSMO_Ru2COSMO_Ru7
0
5
10
15
20
25
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
GEM-1GEM-2.5GEM-250
All the models, Wind speed MAE, Mountain, 00h run
0
0.5
1
1.5
2
2.5 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET
No nowcasting effect!
No clear dependence on resolution
Big INCA and NMMB MAEs
COSMO-Ru7 vs. ARPA-DET: Almost no influence of driving model
Ski Jump-800, 15 Jan-15 Mar 2014
Wind direction ME is negative overall, Wind speed and gusts are mostly overestimated. Joint, HARMONIE and INCA best for gusts
ME
JointCOSMO-RU2GEM-250mHARMONIENMMBINCAMAE
Effect of resolution, Wind speed MAE, Mountain, 00h run
• COSMO family: COSMO-Ru1 advantage for wind speed (confirmed by forecasters!)• GEM family: GEM-250 worst! (unlike for T2m and RelHum) Can it be related to explicit resolving of large eddies?
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1COSMO_Ru2COSMO_Ru7
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
GEM-1GEM-2.5GEM-250
CONCLUSIONS
• For T2m, the effect of resolution is clear in the mountains (better nearest point due to higher resolution orography), for relative humidity, it is important within model families, for wind speed, there is no clear dependence of errors on resolution
• T2m: - In the mountain cluster, COSMO, INCA, and NMMB have min MAE during the nighttime, while GEM, HARMONIE, and, KMA-NIMR, during the daytime; - In the coastal cluster, for GEM-1 and GEM-2.5, a big daytime error maximum appears- Bigger daytime errors for ARPA-DET vrt. COSMO-Ru7
CONCLUSIONS, cont.
• Relative humidity (in the mountains) :- GEM-0.25 has lowest MAEs, then goes COSMO-Ru1 - ARPA-DET has lower relative humidity MAE vrt. COSMO-Ru7
• Wind speed (in the mountains) : - INCA and NMMB errors are high due to overestimation (INCA is good for gusts) - COSMO-Ru1 has added value for wind (confirmed by forecasters)- GEM-0.25 is worse than GEM-1 and GEM-2.5- Almost no difference between COSMO-Ru7 and ARPA-DET -> no
influence on driving model
PLANS
• To proceed with area aggregation (height levels, etc.)• To add the CIs for the aggregated scores• To add some categorical scores (visibility!)• To gather and analyze the results in a paper
Thank you for your attention!
20COSMO GM Sibiu, 2 - 5 Sept. 2013
T2m Mean errors (BIAS)
• Вставить общие графики ME для Mountain и Coastal!
• Underestimation of temperature during the sunny weather, likely due to insufficient soil warming
T2m MAE for different heights, 15 Jan-15 Mar 2014, 00 runs
HARMONIE’s errors become bigger after ~1500 mJoint and INCA are the best at RKHU-3 during the first hours (nowcasting effect!)
RKHU-3, altitude 2043 m
JointCOSMO-RU2GEM-250mHARMONIENMMBINCA
Models’ parameterizationsModel Source of BC
and ICLarge-scale
dynamicsBoundary
layer param.
ConvectionParam.
Large-scalePrecip.scheme
Cloud scheme Land-surface scheme
Radiation
COSMO GME primitive hydro thermodynamical equations describing compressible non-hydrostatic flow in a moist atmosphere
Surface layer scheme (based on turbulent kinetic energy) including a laminar-turbulent roughness layer.
Tiedtke (1989) mass-flux convection scheme with equilibrium closure based on moisture convergence. Option for a modified closure based on CAPE.
Reduced Tiedtke scheme for shallow convection only.
Precipitation formation by a bulk microphysics parameterization including water vapour, cloud water, cloud ice, rain and snow with 3D transport for the precipitating phases.
Cloud water condensation and evaporation by saturation adjustmen
Multi-layer version of the former two-layer soil model after Jacobsen and Heise (1982) based on the direct numerical solution of the heat conduction equation. Snow and interception storage are included.
δ-two stream radiation scheme after Ritter and Geylen (1992) for short- and longwave fluxes (employing eight spectral intervals); full cloud-radiation feedback.
WRF-NMMB Downscaled from GFS
nonhydrostatic Eulerian gridpoint
Mellor-Yamada-Janjic level 2.5
Betts-Miller-Janjic
Ferrier Ferrier NOAH Rapid Radiative Transfer Model (RRTM)
HARMONIE Downscaled from ECMWF-IFS
GEM GEM global run at 25 km resolution
Semi-Lagrangian Moist TKE PBL scheme
Kain and Fritsch deep convection scheme (10-km only), Kuo-Transient shallow convection scheme (all resolutions)
Precipitation is diagnosed from convective and moist TKE PBL schemes. Grid-scale precipitation and clouds are from Milbrandt-Yau 2-moment bulk microphysics scheme.
Same as large-scale
ISBA land-surface scheme
Li and Barker radiative transfer scheme
WRF-KMA ECMWF T127
Runge-Kutta 3rd order
YSU PBL no CP in 2km not used in 6km and 2km resolution, Kain-Fritsch in 18km only
WDM6 Modified NoahLSM using Penman-Monteith eq. with modified saturation in snow
Band model using RRTM
24
Is it due to model changes, or 2013 warm anomaly, or observation data distribution, amount, and quality?Difficulty for model calibration!
COSMO T2m ME changes its sign from 2011-2012 winter to 2013 winter
1st test period, Dec2011-Feb2012 2nd test period, Jan-Feb 2013COLD winter WARM winter
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru7ARPA-DET
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru2GEM-2.5KMA-NIMR
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48
COSMO_Ru1GEM-1HARMONIENMMBINCA
Wind speed MAE, Mountain, 00h run
~1-km models
7-km models
Almost no influence of driving model
~2-km models
Visibility scores, meters, Mountain, 00h run
0
10000
20000
30000
40000
50000
60000
70000
80000
0 2 4 6 8 10 12 14 16 18 20 22 24 26
GEM-1GEM-2.5GEM-250NMMB
0
10000
20000
30000
40000
50000
60000
70000
80000
0 2 4 6 8 10 12 14 16 18 20 22 24 26
GEM-1GEM-2.5GEM-250NMMB
MAE ME
Difference between GEM and NMMB is due to different maximum values (roughly, 10000 m and 70000 m are the same) Categorical scores are needed!