SREPS Priority Project:final report

C. Marsigli, A. Montani, T. Paccagnella

ARPA-SIMC - HydroMeteorological Service of Emilia-Romagna, Bologna, Italy

F. Gofa, P. Louka

HNMS – Hellenic National Meteorological Service, Athens, Greece

Last FTEs of Chiara were used for this

TASK

Last FTEs of Chiara were used for this

TASK

ROMEO

Outline

COSMO-SREPS methodology

system set-up

analysis of the results on MAP D-PHASE DOP (JJA 2007+SON 2007)

role of different kind of perturbations

error vs spread

boundaries from mm vs different physics

ranking of different driving models and of different physics

conclusions

analysis of the results on MAP D-PHASE DOP (JJA 2007+SON 2007)

role of different kind of perturbations

error vs spread

boundaries from mm vs different physics

ranking of different driving models and of different physics

conclusions

System set-up

16 COSMO runs 10 km

hor. res.40 vertical

levels

COSMO at 25 km on IFS

IFS – ECMWF globalb

y I

NM

S

pain

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global

P1: control (ope)

P2: conv. scheme (KF)

P3: tur_len=1000

P4: pat_len=10000

00 UTCJJA=53 SON=54

DOP

SON=54 runsJJA=53 runs

COSMO observations

intra-group distance

Z500 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

Same driving model

Different model

parameters

Same model parameters

Different

driving model

role of different kind of perturbations

intra-group distanceJJA 2007 - 50 days

t850

COSMO analysis

COSMO-SREPS

intra-group distanceSON 2007 - 49 days

t850

COSMO I7 analysis

COSMO-SREPS

intra-group distance

2mT Northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

SYNOP over D-PHASE area - Nearest grid point

JJA07 SON07

2m T - relationship between error and spread

COSMO I7 analysis

error vs spread Underdispersive

Synop stations on the Alpine area

218 stations

spread/skill relationship

2mT Alpine area (synop stations)

+12h +24h +36h

+48h +60h +72h

spread/skill relationship

2mT Alpine area (synop stations)

+12h +24h +36h

+48h +60h +72h

Underdispersive

t850 relationship between EM error and

spread

COSMO-I7 interpolated on SYNOP stations over the Alpine area

JJA07

t850 relationship between error and

spread

COSMO-I7 interpolated on SYNOP stations over the Alpine area

SON07

spread/skill relationship

t850 Alpine area (COSMO analyses)

+12h +24h +36h

+48h +60h +72h

spread/skill relationship

t850 Alpine area (COSMO analyses)

+12h +24h +36h

+48h +60h +72h

Z500 relationship between error and

spread

COSMO-I7 interpolated on Northern Italy stations and SYNOP stations over the Alpine area

JJA07

Z500 relationship between error and

spread

COSMO-I7 interpolated on SYNOP stations over the Alpine area

SON07

TP 24h – ave 0.5x0.5 JJA07

+30h

noss 1400 700 400 150 50

IT + CH

+54h

Same driving models

Same driving models

Same perturbation

Big impact of multi model BCs!!!!!!!

Looking at the right column it is evident that even with few members the skill does not decrease too much when the driving models are different.

Same perturbation

TP 24h – ave 0.5x0.5

noss 800 500 300 100 50

SON07

+30h

IT + CH

+54h

Same driving model

Same driving model

Same perturbation

Same perturbation

JJA07 +30h

IT

IT + CH

Same driving model

Same perturbation

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

2.40

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

bia

s (K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

2m temperature - BIAS

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

JJA07

ecmwfgmencepukmo

p4

2m temperature - MAE

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

JJA07

2.60

3.00

3.40

3.80

4.20

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

mae

(K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ecmwfgmencepukmo

p4

2m temperature - BIAS

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

SON07

-3.20

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

bia

s (K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16ecmwfgmencepukmo

p4

2m temperature - MAE

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

SON07

2.60

3.00

3.40

3.80

4.20

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

mae

(K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ecmwfgmencepukmo

p4

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT

pert

1mm/24h

5mm/24h

10mm/24h

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

pert

Test of more parameter perturbations (same father)

16 LM runs at 10 km

P1: control (ope)

P2: conv. scheme (KF)

P3: parameter 1

P4: parameter 2

P5: …

IFS – ECMWF global

SON 07

T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

2.00

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

Preliminary ConclusionsPerturbations -Multi Model ICs/BCs & Perturbations on

Ph. Params:

the use of different driving models seems to dominate with respect to physics parameter perturbations as regards the contribution to the spread; these contributions are different in the two seasons (2mT)

the selected parameters produce a detectable spread among members with the same father (driving model)

spread-skill relationship:

a correlation between error and spread exists, but the

system is under-dispersive -> a better representation of model error is needed

the different driving models contribute differently to the ensemble

skill, but there is a strong dependence on forecast range, season, verification area

the different perturbations can contribute differently to the ensemble skill as well

On-going activities and future plans

continue the analysis over the DOP MAP D-PHASE :

statistical analysis of the system

comparison with the other available mesoscale ensemble systems

verification carried out by HNMS

introduce the new parameter perturbations

analyse the impact of adding soil perturbations

COSMO-SREPS methodology

i.c. and b.c. perturbations -> INM multi-model multi-boundary ensemble (SREPS)

LAM perturbations -> physics parameter perturbations

LAM perturbations - smaller scale errors

driving model perturbations (ics and bcs) - larger scale errors

System set-up

16 COSMO runs 10 km

hor. res.40 vertical

levels

COSMO at 25 km on IFS

IFS – ECMWF globalb

y I

NM

S

pain

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global

P1: control (ope)

P2: conv. scheme (KF)

P3: tur_len=1000

P4: pat_len=10000

MAP D-PHASE DOP testing period

COSMO-SREPS was running during the DOP, at 00 UTC

107 runs out of 183 days, 53 in JJA and 54 in SON

ensemble verification

JJA07

intra-group distanceJJA 2007 - 50 days

2mT

Northern Italy

COSMO-SREPS

Same driving model

Different model parameters

Same model parameters

Different driving model

intra-group distanceJJA 2007 – 50 days

tp6h

Northern Italy

Same driving model

Different model parameters

Same model parameters

Different driving model

Mid Term comments

• Mid-upper troposphere: MULTI MODEL IC/BCs give the bigger contribution to the spread

• Surface/lower troposphere: model physics perturbations “gain ground”.

score evaluation

• +30h:

• ROC

• UKMO and GME the best, then ECMWF and NCEP

• P2 (KF) the best, then P4, P3, P1 (similar)

• BSS

• UKMO the best, ECMWF and GME the worst; NCEP improves with threshold

• P are similar, P2 (KF) slightly better

JJA07tp24I

T

score evaluation

• +54 :

• ROC

• similar for low thresholds, NCEP the best for high thresholds, then ECMWF

• P similar, P3 slightly better

• BSS

• ECMWF the best, GME the worst; NCEP improves with threshold

• P2 (KF) the worst, P3 the best but similar to P1 and P4

JJA07tp24I

T

1 5 10 20 30

EC +30 ROC

BSS

+54 ROC

BSS

MO +30 ROC

BSS

+54 ROC

BSS

GME +30 ROC

BSS

+54 ROC

BSS

AVN +30 ROC

BSS

+54 ROC

BSS

1 5 10 20 30

EC +30 ROC **** ** ** ** *

BSS

+54 ROC

BSS

MO +30 ROC ** **** **** **** ****

BSS

+54 ROC

BSS

GME +30 ROC * *** **** *** ****

BSS

+54 ROC

BSS

AVN +30 ROC ** * * * ***

BSS

+54 ROC

BSS

TP 24h – ave 0.5x0.5 JJA07

+30h

noss 700 350 200 60 20

IT

+54h

father

father

pert

pert

TP 24h – ave 0.5x0.5 JJA07

+30h

IT

+54h

father

father

pert

pert

noss 700 350 200 60 20

Grosso impatto del MULTI MODEL ai boundaries!!!!!!! Looking at the right column it is evident that even with few members the skill does not decrease too much when the driving models are different.

score evaluation

• +30h:

• ROC

• UKMO > GME > ECMWF > NCEP

• P2 (KF) the best, the others are similar

• BSS

• NCEP better for high threshold; UKMO > ECMWF > GME

• P2 (KF) the worst, the others are similar

JJA07tp24IT+CH

score evaluation

• +54 :

• ROC

• similar for low thresholds, NCEP and GME best for high thresholds

• P similar; P2 (KF) slightly better for high thresholds

• BSS

• ECMWF the best, NCEP the worst

• P similar; P2 (KF) slightly worse

JJA07tp24IT+CH

TP 24h – ave 0.5x0.5 JJA07

+30h

noss 1400 700 400 150 50

IT + CH

+54h

father

father

pert

pert

TP 24h – ave 0.5x0.5 JJA07

+30h

noss 1400 700 400 150 50

IT + CH

+54h

father

father

pert

pert

The performances reverse eith the leading time.

Bad skill adding switzerlad.

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

pert

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

pert

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

fathernoss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

fathernoss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

pert

1mm/24h

5mm/24h

10mm/24h

noss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

pertnoss 250 100 50

6h accumulated precipitationrelationship between error and

spread

obs Northern Italy

L1

Relationship between error and spread

Northern Italy observations

Nearest grid point

applicata correzione per la quota LAPSE=0.7

eliminati dati minori di –10 e maggiori di +42

t2m

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

all (16 members)

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

GME only (4 members)

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

ECMWF only (4 members)

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

NCEP only (4 members)

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

UKMO only (4 members)

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

2m temperature relationship between error and

spread

COSMO-I7 interpolated on Northern Italy stations and SYNOP stations over the Alpine area

SON07

6h precipitation - BSS

Northern Italy observations

Average over 0.5 x 0.5 deg boxes

6h precipitation – ROC area

Northern Italy observations

Average over 0.5 x 0.5 deg boxes

Daily precipitation - BSS

Northern Italy + Switzerland observations

Average over 0.5 x 0.5 deg boxes

Daily precipitationreliability and resolution

Northern Italy + Switzerland observations

Average over 0.5 x 0.5 deg boxes

Daily precipitation - ROC area

Northern Italy + Switzerland observations

Average over 0.5 x 0.5 deg boxes

Relationship between error and spread

Northern Italy observations

Nearest grid point eliminati dati minori di –10 e maggiori di +42

tp 6h

Relationship between error and spread

Northern Italy observations

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

eliminati dati minori di –10 e maggiori di +42

Relationship between error and spread

SYNOP over MAP area

Nearest grid point

t2m

applicata correzione per la quota LAPSE=0.7

2m temperature relationship between error and

spread

COSMO-I7 interpolated on SYNOP stations over the Alpine area

intra-group distanceSON 2007 - 49 days

2mT

Northern Italy

COSMO-SREPS

Same driving model

Different model parameters

Same model parameters

Different driving model

intra-group distanceSON 2007 - 49 days

tp6h

Northern Italy

COSMO-SREPS

score evaluation

• +30h:

• ROC

• crossing; UKMO and ECMWF slightly better, GME worse

• P2 (KF) the best, the others are similar

• BSS

• ECMWF the best, GME the worst

• P are similar, P2 (KF) slightly better

SON07

tp24I

T

score evaluation

• +54 :

• ROC

• NCEP the best, GME the worst

• P2 (KF) the best, the others are similar

• BSS

• similar

• similar

SON07

tp24I

T

TP 24h – ave 0.5x0.5 SON07

+30h

IT

+54h

father

father

pert

pert

noss 550 300 200 80 50

TP 24h – ave 0.5x0.5 SON07

+30h

IT

+54h

father

father

pert

pert

noss 550 300 200 80 50

score evaluation

• +30h:

• ROC

• similar; for high thresholds ECMWF is the best and GME the worst

• similar; P2 (KF) slightly better

• BSS

• ECMWF the best and GME the worst, , especially with increasing threshold

• P are similar, P4 (tur_len=1000) better for the last thresholds

SON07

tp24IT+CH

score evaluation

• +54 :

• ROC

• NCEP the best, GME the worst

• similar, P2 (KF) slightly better

• BSS

• ECMWF the best, UKMO the worst

• similar

SON07

tp24IT+CH

TP 24h – ave 0.5x0.5 SON07

+30h

IT + CH

+54h

father

father

pert

pert

noss 800 500 300 100 50

TP 24h – ave 0.5x0.5 SON07+30h

IT + CH

+54h

father

father

pert

pert

noss 800 500 300 100 50

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT

pert

1mm/24h

5mm/24h

10mm/24h

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

pert

Deterministic scores – ave 0.5 x 0.5 IT+CH

father

1mm/24h

5mm/24h

10mm/24h

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

father

Deterministic scores – ave 0.5 x 0.5 IT+CH

pert

1mm/24h

5mm/24h

10mm/24h

Deterministic scores – ave 0.5 x 0.5 IT+CH

1mm/24h

5mm/24h

10mm/24h

pert

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

fathernoss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

fathernoss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

pert

1mm/24h

5mm/24h

10mm/24h

noss 250 100 50

Deterministic scores – ave 0.5 x 0.5 IT

1mm/24h

5mm/24h

10mm/24h

pertnoss 250 100 50

Test of more parameter perturbations (same father)

16 LM runs at 10 km

P1: control (ope)

P2: conv. scheme (KF)

P3: parameter 1

P4: parameter 2

P5: …

IFS – ECMWF global

SON 07

run nr.parameter name parameter description range default used1 ctrl ope2 lconv convection scheme T or KF T KF3 tur_len maximal turbulent length scale [100,1000] m 500 1504 tur_len maximal turbulent length scale [100,1000] m 500 10005 pat_len length scale of thermal surface patterns [0,10000] m 500 100006 rat_sea ratio of laminar scaling factors for heat over sea [1,100] 20 17 rat_sea ratio of laminar scaling factors for heat over sea [1,100] 20 608 qc0 cloud water threshold for autoconversion [0.,0.001] 0 0.0019101112

13 c_lnd

surface area density of the roughness elements over land [1,10] 2 1

14 c_lnd

surface area density of the roughness elements over land [1,10] 2 10

15 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 0.116 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 10

crsmincrsminc_soilc_soil

Minimal stomata resistance 15015011 2

0

50200Minimal stomata resistance

Surface area index of the evaporating soilSurface area index of the evaporating soil

[50,200] s/m[50,200] s/m

]0,c_lnd[

]0,c_lnd[

T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

2.00

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

T2m deterministic scores – npo ITpia mon

1.40

1.80

2.20

2.60

3.00

3.40

3.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

1.40

1.80

2.20

2.60

3.00

3.40

3.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

2.40

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (

K)

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

2.40

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (

K)

Td2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

2.20

2.40

2.60

2.80

3.00

3.20

3.40

3.60

3.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

Td2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

Td2m deterministic scores – npo IT

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

pia mon

1.80

2.00

2.20

2.40

2.60

2.80

3.00

3.20

3.40

3.60

3.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

1.80

2.00

2.20

2.40

2.60

2.80

3.00

3.20

3.40

3.60

3.80

0 3 6 9 12 15 18 21 24

forecast range (h)

ma

e (

K)

0.9

1

1.1

1.2

1.3

1.4

6 12 18 24

forecast range (h)

bia

s sc

ore

TP deterministic scores – ave 0.5 x 0.5 IT

1mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

noss 452 479 470 447

0.44

0.46

0.48

0.5

0.52

0.54

0.56

0.58

6 12 18 24

forecast range (h)

thre

at s

core

TP deterministic scores – ave 0.5 x 0.5 IT

1mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10 noss 452 479 470 447

0.28

0.3

0.32

0.34

0.36

0.38

0.4

0.42

6 12 18 24

forecast range (h)

fals

e al

arm

rat

eTP deterministic scores – ave 0.5 x 0.5

IT1mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

noss 452 479 470 447

0.9

1

1.1

1.2

1.3

1.4

6 12 18 24

forecast range (h)

bia

s sc

ore

TP deterministic scores – ave 0.5 x 0.5 IT

10mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

noss 73 81 69 70

0.2

0.24

0.28

0.32

0.36

0.4

6 12 18 24

forecast range (h)

thre

at s

core

TP deterministic scores – ave 0.5 x 0.5 IT

10mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

noss 73 81 69 70

0.44

0.48

0.52

0.56

0.6

0.64

0.68

0.72

6 12 18 24

forecast range (h)

fals

e al

arm

rat

eTP deterministic scores – ave 0.5 x 0.5

IT10mm/6h

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10

noss 73 81 69 70

t BIA t MAE td BIA td MAE tp1 BS

tp1 TS

tp1 FA tp10 BS

tp10 TS

tp10 FA

KF = = > = = < > <> > > < >

tur_len=150 = < = = > = = <> <> < = < =

tur_len=1000 = > = < = = = <> <> <> => <24

=

pat_len=10000

> <> = < = > = > > = < > > <24

< >24

rat_sea=1 = > = < > = > > <> <> > <> = >24

rat_sea=60 = < = > < = > < <> = < < = < = >

qc=0.001 = = = = = = < > = > <24

< >24

crsmin=50 = < = = > = < = = < = > = = =

crsmin=200 = = = = = = = < = = =

c_soil=0 > < < > < > < = < = <>

c_soil=2 < > > > > = < > = = <>

c_lnd=1 = < > > > = = = = = =

c_lnd=10 > < < > = < = = < = < = =

rlam_heat=0.1

<> = > > = > > <> > = > = =

rlam_heat=10

<> = < < <> < <> < = < = =

- -- +++

lconv=KF tur_len=150 tur_len=1000

pat_len=10000 rat_sea=1 rat_sea=60 qc0=0.001

crsmin=50 crsmin=200 c_soil=0 c_soil=2

c_lnd=1 c_lnd=10 rlam_heat=0.1 rlam_heat=10

ctrl

CSPERT fc-ctrl differences +12h04/05/07 06-12 UTC

mm/6h

CSPERT fc-obs differences +12hlconv=KF tur_len=150 tur_len=1000

pat_len=10000 rat_sea=1 rat_sea=60 qc0=0.001

crsmin=50 crsmin=200 c_soil=0 c_soil=2

c_lnd=1 c_lnd=10 rlam_heat=0.1 rlam_heat=10

ctrl

-5 55 20

20 50

-5 -20

-20 -50

mm/6h

CSPERT forecasts +12h

06-12 UTC 04/05/07

Comparisons

SYNOP over MAP area - Nearest grid point

JJA07 SON07

2m T - relationship between error and spread

COSMO-I7 analyses

intra-group distance

2mT Northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

intra-group distance

tp6h northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days

intra-group distance

t850 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

intra-group distance

Z500 COSMO analysis

JJA 2007 - 50 days

SON 2007 - 49 days

JJA07 SON07

pert

father

+30h

IT + CH

JJA07 SON07

pert

father

+30h

IT

JJA07

pert

father

+30h

IT

IT + CH

SON07

pert

father

+30h

IT

IT + CH

score evaluation

• 2m t:

• BIA > 0

GME the largest (+), then UKMO, ECMWF, NCEP mixed

P4 (tur_len=1000) the largest (+), for any father

• MAE

GME the smallest, then UKMO, ECMWF, NCEP mixed

P4 (tur_len=1000) the largest, especially for GME at night

JJA07

score evaluation

• 2m t:

• BIA < 0

ECMWF the largest (-), then GME, NCEP and UKMO

P4 (tur_len=1000) the more positive, for any father; P1, P2, P3 similar

• MAE

ECMWF the largest, then GME, NCEP and UKMO

P4 (tur_len=1000) the smallest; P1, P2, P3 similar

SON07