inSwitzerland - ECMWF · 2015. 11. 18. · 2012 2013 40 35 30 25 20 15 10 -5 skill Annual evolution...
1
Forecaster’s added value with respect to NWP in Switzerland Daniel Cattani, Pierre Eckert, Michel Matter [email protected] What we do We perform a verification of the deterministic precipitation fore- casts for selected regions of Switzerland issued by the bench fore- casters and compare them to ECMWF’s IFS outputs (HRES and EPS). Verified Precipitation Forecasts Precipitation amounts are predicted by forecasters for regions and represent spatial averages. The number of regions depends on the forecast time-range: it is 27 (D1-D2), 11 (D3-D5) or 6 (D6-D7). Plateau West Alps West Ticino South Verified forecasts: VAL: daily regional mean amounts predicted by forecasters; IFS HRES: spatial average over same regions than VAL; IFS EPS: spatial average of the ensemble median. In this exploratory study, the focus is on the shaded regions shown on the map. A sample “climatology” of these regions during the studied period: (2010-2013) ratio of rainy days (>0.2mm) average daily amount [mm] Plateau W 0.45 5.31 Alps W 0.55 6.42 Ticino S 0.40 10.93 Observations Used for the Verification Forecasts are compared with regional average observations provided by a tool recently developed at MeteoSwiss [1] which combines high resolution radar images with raingauge measurements. Verification The following measure of accuracy is used in the verification (x = f cst, y = obs): S (x, y )= 100 if |x - y |≤ μ(x), 100 · 1 - y p -(x+μ(x)) p d if 0 <y p - (x + μ(x)) p <d, 100 · 1 - (x-μ(x)) p -y p d if 0 < (x - μ(x)) p - y p <d, 0 otherwise, with p =2/5, d =3/2 and μ(x)=0.3x +0.1 a tolerance threshold. This scoring rule is a part of the global quality score for communication and administration newly deployed at MeteoSwiss [2]. 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100 observation [mm] score S P (3,⋅) S P (8,⋅) S P (20,⋅) The skill of FCST against REF is measured according to skill = S FCST - S REF 100 - S REF · 100. Forecaster’s added value (2010-2013) D1 D2 D3 D4 D5 D6 D7 60 65 70 75 80 85 90 forecast time-range accuracy Accuracy vs time-range; region Plateau W VAL HRES MEDIAN D1 D2 D3 D4 D5 D6 D7 55 60 65 70 75 80 85 forecast time-range accuracy Accuracy vs time-range; region Alps W VAL HRES MEDIAN D1 D2 D3 D4 D5 D6 D7 55 60 65 70 75 80 85 forecast time-range accuracy Accuracy vs time-range; region Ticino S VAL HRES MEDIAN 2010 2011 2012 2013 50 55 60 65 70 75 80 85 90 D1 D4 D7 accuracy Annual evolution of accuracy; region Alps W VAL HRES MEDIAN 2010 2011 2012 2013 -25 -20 -15 -10 -5 0 5 10 15 20 skill Annual evolution of skill; region Alps W; D1 VAL2HRES VAL2MEDIAN MEDIAN2HRES 2010 2011 2012 2013 -25 -20 -15 -10 -5 0 5 10 15 20 skill Annual evolution of skill; region Alps W; D4 VAL2HRES VAL2MEDIAN MEDIAN2HRES 2010 2011 2012 2013 -25 -20 -15 -10 -5 0 5 10 15 20 skill Annual evolution of skill; region Alps W; D7 VAL2HRES VAL2MEDIAN MEDIAN2HRES 2010 2011 2012 2013 50 55 60 65 70 75 80 85 90 D1 D4 D7 accuracy Annual evolution of accuracy; region Ticino S VAL HRES MEDIAN 2010 2011 2012 2013 -20 -10 0 10 20 30 40 skill Annual evolution of skill; region Ticino S; D1 VAL2HRES VAL2MEDIAN MEDIAN2HRES 2010 2011 2012 2013 -5 0 5 10 15 20 25 30 35 40 skill Annual evolution of skill; region Ticino S; D4 VAL2HRES VAL2MEDIAN MEDIAN2HRES 2010 2011 2012 2013 -5 0 5 10 15 20 25 30 35 40 skill Annual evolution of skill; region Ticino S; D7 VAL2HRES VAL2MEDIAN MEDIAN2HRES More Detailed Analysis 2010 F M A M J J A S O N D 2011 F M A M J J A S O N D 2012 F M A M J J A S O N D 2013 F M A M J J A S O N D -2 0 2 4 6 8 10 bias [mm] and S/10 Monthly evolution of bias (bars) and accuracy (lines): region Alps W; D1 VAL HRES MEDIAN 2010 F M A M J J A S O N D 2011 F M A M J J A S O N D 2012 F M A M J J A S O N D 2013 F M A M J J A S O N D -6 -4 -2 0 2 4 6 8 10 bias [mm] and S/10 Monthly evolution of bias (bars) and accuracy (lines): region Alps W; D7 VAL HRES MEDIAN Region Alps West: Bias D1 VAL HRES MEDIAN Winter 0.82 0.95 0.33 Spring 0.61 1.18 0.35 Summer 0.72 1.50 0.13 Autumn 1.11 1.50 0.62 global 0.82 1.29 0.36 Bias D7 VAL HRES MEDIAN Winter -0.02 0.31 -1.17 Spring 0.22 1.27 -0.74 Summer -0.72 0.37 -2.20 Autumn -1.01 0.65 -1.55 global -0.39 0.66 -1.42 2010 F M A M J J A S O N D 2011 F M A M J J A S O N D 2012 F M A M J J A S O N D 2013 F M A M J J A S O N D -5 0 5 10 bias [mm] and S/10 Monthly evolution of bias (bars) and accuracy (lines): region Ticino S; D1 VAL HRES MEDIAN 2010 F M A M J J A S O N D 2011 F M A M J J A S O N D 2012 F M A M J J A S O N D 2013 F M A M J J A S O N D -10 -5 0 5 10 bias [mm] and S/10 Monthly evolution of bias (bars) and accuracy (lines): region Ticino S; D7 VAL HRES MEDIAN Region Ticino South: Bias D1 VAL HRES MEDIAN Winter -0.03 0.31 0.09 Spring 0.18 0.76 -0.58 Summer -0.58 1.75 0.24 Autumn -0.20 1.07 -0.15 global -0.16 0.98 -0.10 Bias D7 VAL HRES MEDIAN Winter -1.31 -0.26 -1.36 Spring -2.50 0.51 -3.01 Summer -2.15 0.33 -2.73 Autumn -2.51 0.87 -3.04 global -2.13 0.38 -2.57 Conclusion Advices to forecasters: Alps W: consider the EPS median rather than the HRES for all terms; pay attention to precipitation underestimation of EPS median for long-term forecasts, especially in Summer and Autumn. Ticino S: pay attention to underestimation of long-term forecasts. Remarks about model outputs: Alps W: HRES exhibits significant overestimation at short-term. EPS median underestimates precipitation at long-term, especially during Summer. Ticino S: HRES overestimates at short-term especially during Summer. EPS median strongly underestimates at long-term. Further work: Consider other quantiles from EPS, by region/season/time-range, to determine the best first guess. References [1] I. V. Sideris, M. Gabella, R. Erdin, and U. Germann. Real-time radar-raingauge merging using spatiotemporal co-kriging with external drift in the alpine terrain of switzerland. Quaterly Journal of the Royal Meteorological Society, 00:1–22, 2011. [2] D. Cattani, A. Faes, M. Giroud Gaillard, and M. Matter. COMFORT: continu- ous MeteoSwiss forecast quality score. Submitted.
Transcript of inSwitzerland - ECMWF · 2015. 11. 18. · 2012 2013 40 35 30 25 20 15 10 -5 skill Annual evolution...
Forecaster’saddedvaluewithrespect toNWPinSwitzerland
Daniel Cattani, Pierre Eckert, Michel [email protected]
What we doWe perform a verification of the deterministic precipitation fore-casts for selected regions of Switzerland issued by the bench fore-casters and compare them to ECMWF’s IFS outputs (HRES andEPS).
Verified Precipitation ForecastsPrecipitation amounts are predicted by forecasters for regions andrepresent spatial averages. The number of regions depends on theforecast time-range: it is 27 (D1-D2), 11 (D3-D5) or 6 (D6-D7).
Plateau West
Alps West
Ticino South
Verified forecasts:VAL: daily regional mean amounts predicted by forecasters;IFS HRES: spatial average over same regions than VAL;IFS EPS: spatial average of the ensemble median.
In this exploratory study, the focus is on the shaded regions shownon the map. A sample “climatology” of these regions during thestudied period:
(2010-2013) ratio of rainy days (>0.2mm) average daily amount [mm]
Plateau W 0.45 5.31Alps W 0.55 6.42Ticino S 0.40 10.93
Observations Used for the VerificationForecasts are compared with regional average observations providedby a tool recently developed at MeteoSwiss [1] which combines highresolution radar images with raingauge measurements.
VerificationThe following measure of accuracy is used in the verification (x =fcst, y = obs):
S(x, y) =
100 if |x− y| ≤ µ(x),
100 ·(1− yp−(x+µ(x))p
d
)if 0 < yp − (x+ µ(x))p < d,
100 ·(1− (x−µ(x))p−yp
d
)if 0 < (x− µ(x))p − yp < d,
0 otherwise,
with p = 2/5, d = 3/2 and µ(x) = 0.3x+0.1 a tolerance threshold. Thisscoring rule is a part of the global quality score for communication andadministration newly deployed at MeteoSwiss [2].
0 10 20 30 40 50 60 700
10
20
30
40
50
60
70
80
90
100
observation [mm]
scor
e
S
P(3,⋅)
SP(8,⋅)
SP(20,⋅)
The skill of FCST against REF is measured according to
skill =SFCST − SREF100− SREF
· 100.
Forecaster’s added value (2010-2013)
D1 D2 D3 D4 D5 D6 D760
65
70
75
80
85
90
forecast time−range
accu
racy
Accuracy vs time−range; region Plateau W
VAL HRES MEDIAN
D1 D2 D3 D4 D5 D6 D755
60
65
70
75
80
85
forecast time−range
accu
racy
Accuracy vs time−range; region Alps W
VAL HRES MEDIAN
D1 D2 D3 D4 D5 D6 D755
60
65
70
75
80
85
forecast time−range
accu
racy
Accuracy vs time−range; region Ticino S
VAL HRES MEDIAN
2010 2011 2012 201350
55
60
65
70
75
80
85
90
D1
D4
D7
accu
racy
Annual evolution of accuracy; region Alps W
VAL HRES MEDIAN
2010 2011 2012 2013−25
−20
−15
−10
−5
0
5
10
15
20
skill
Annual evolution of skill; region Alps W; D1
VAL2HRESVAL2MEDIANMEDIAN2HRES
2010 2011 2012 2013−25
−20
−15
−10
−5
0
5
10
15
20
skill
Annual evolution of skill; region Alps W; D4
VAL2HRESVAL2MEDIANMEDIAN2HRES
2010 2011 2012 2013−25
−20
−15
−10
−5
0
5
10
15
20
skill
Annual evolution of skill; region Alps W; D7
VAL2HRESVAL2MEDIANMEDIAN2HRES
2010 2011 2012 201350
55
60
65
70
75
80
85
90
D1
D4
D7
accu
racy
Annual evolution of accuracy; region Ticino S
VAL HRES MEDIAN
2010 2011 2012 2013−20
−10
0
10
20
30
40sk
ill
Annual evolution of skill; region Ticino S; D1
VAL2HRESVAL2MEDIANMEDIAN2HRES
2010 2011 2012 2013−5
0
5
10
15
20
25
30
35
40
skill
Annual evolution of skill; region Ticino S; D4
VAL2HRESVAL2MEDIANMEDIAN2HRES
2010 2011 2012 2013−5
0
5
10
15
20
25
30
35
40
skill
Annual evolution of skill; region Ticino S; D7
VAL2HRESVAL2MEDIANMEDIAN2HRES
More Detailed Analysis
2010F M A M J J A S O N D2011F M A M J J A S O N D2012F M A M J J A S O N D2013F M A M J J A S O N D−2
0
2
4
6
8
10
bias
[mm
] and
S/1
0
Monthly evolution of bias (bars) and accuracy (lines): region Alps W; D1
VAL
HRESMEDIAN
2010F M A M J J A S O N D2011F M A M J J A S O N D2012F M A M J J A S O N D2013F M A M J J A S O N D−6
−4
−2
0
2
4
6
8
10
bias
[mm
] and
S/1
0
Monthly evolution of bias (bars) and accuracy (lines): region Alps W; D7
VAL HRES MEDIAN
Region Alps West:
Bias D1 VAL HRES MEDIAN
Winter 0.82 0.95 0.33Spring 0.61 1.18 0.35
Summer 0.72 1.50 0.13Autumn 1.11 1.50 0.62
global 0.82 1.29 0.36
Bias D7 VAL HRES MEDIAN
Winter -0.02 0.31 -1.17Spring 0.22 1.27 -0.74
Summer -0.72 0.37 -2.20Autumn -1.01 0.65 -1.55
global -0.39 0.66 -1.42
2010F M A M J J A S O N D2011F M A M J J A S O N D2012F M A M J J A S O N D2013F M A M J J A S O N D−5
0
5
10
bias
[mm
] and
S/1
0
Monthly evolution of bias (bars) and accuracy (lines): region Ticino S; D1
VAL HRES MEDIAN
2010F M A M J J A S O N D2011F M A M J J A S O N D2012F M A M J J A S O N D2013F M A M J J A S O N D−10
−5
0
5
10
bias
[mm
] and
S/1
0
Monthly evolution of bias (bars) and accuracy (lines): region Ticino S; D7
VAL HRES MEDIAN
Region Ticino South:
Bias D1 VAL HRES MEDIAN
Winter -0.03 0.31 0.09Spring 0.18 0.76 -0.58
Summer -0.58 1.75 0.24Autumn -0.20 1.07 -0.15
global -0.16 0.98 -0.10
Bias D7 VAL HRES MEDIAN
Winter -1.31 -0.26 -1.36Spring -2.50 0.51 -3.01
Summer -2.15 0.33 -2.73Autumn -2.51 0.87 -3.04
global -2.13 0.38 -2.57
ConclusionAdvices to forecasters:Alps W: consider the EPS median rather than the HRES for all terms; pay attention to precipitation underestimation of EPS median forlong-term forecasts, especially in Summer and Autumn.Ticino S: pay attention to underestimation of long-term forecasts.Remarks about model outputs:Alps W: HRES exhibits significant overestimation at short-term. EPS median underestimates precipitation at long-term, especially duringSummer.Ticino S: HRES overestimates at short-term especially during Summer. EPS median strongly underestimates at long-term.Further work: Consider other quantiles from EPS, by region/season/time-range, to determine the best first guess.
References[1] I. V. Sideris, M. Gabella, R. Erdin, and U. Germann. Real-time radar-raingauge
merging using spatiotemporal co-kriging with external drift in the alpine terrainof switzerland. Quaterly Journal of the Royal Meteorological Society, 00:1–22,2011.
[2] D. Cattani, A. Faes, M. Giroud Gaillard, and M. Matter. COMFORT: continu-ous MeteoSwiss forecast quality score. Submitted.
