CONSENS Priority Project Status report COSMO year 2009/2010

CONSENS Priority Project

Status report COSMO year 2009/2010

Involved scientists:

Chiara Marsigli, Andrea Montani, Tiziana Paccagnella, Tommaso Diomede (ARPA-SIMC)

Flora Gofa, Petroula Louka (HNMS)

Andrea Corigliano (Uni BO), Michele Salmi (Uni FE)

Overview

Task 1: Running of the COSMO-SREPS suite suite maintenance implementation of the back-up suite

Task 2: Model perturbations perturbation of physics parameters perturbation of soil fields

Task 3: Ensemble merging COSMO-LEPS – COSMO-SREPS comparison Multi-clustering

Task 4: Calibration

COSMO-SREPS

IFS (15km) – ECMWF global

GME (30km) – DWD global

UM – UKMO global

GFS (50 km) – NCEP global

•INT2LM (v 1.14)

•COSMO (v 4.12)

•00 UTC and12 UTC

•7 km

•40 levels

•16 members

•48 h

•16 physics perturbations

T1 - Running of the COSMO-SREPS suite(C. Marsigli)

Maintenance of the COSMO-SREPS suite at ECMWF

Implementation of the back-up suite:The work involves also DWD (even if implicitly!)

A BC suite is being implemented by DWD at ECMWF, to provide BCs to COSMO-DE-EPS

The BC suite will provide the 4 control members to COSMO-SREPS

Direct nesting on the global models

Domain enlargement and resolution increase (7 km)

12 members are currently run every day (IFS, GME, GFS branches)

Suite set-up

member father itype_conv tur_len pat_len rlam_heat rat_sea crsmin1 ifs 0 150 500 1 20 1502 ifs 1 1000 500 1 20 1503 ifs 0 500 500 0.1 20 2004 ifs 1 500 5000 1 20 1505 gme 0 500 5000 1 20 1506 gme 1 500 500 0.1 20 1507 gme 0 500 500 1 1 2008 gme 1 500 500 1 1 1509 gfs 0 1000 500 1 20 150

10 gfs 1 150 500 1 20 15011 gfs 0 500 500 10 20 15012 gfs 1 500 500 10 20 15013 um 0 500 500 1 60 15014 um 1 500 500 1 60 15015 um 0 500 500 1 20 5016 um 1 500 500 1 20 50

convection scheme:

0 Tiedtke

1 Kain-Fritsch

maximal turbulent length scale

length scale of thermal surface patterns

scaling factor of the laminar layer depth

ratio of laminar scaling factors for heat over sea

minimal stomata resistance

The new COSMO-SREPS suite – first results

Direct nesting of COSMO at 10 km (!) on IFS (15km) and GME (30 km)

Analysis for MAM 2010 (76 dates, suite running from mid March)

Scores computed for: total precipitation

2m temperature and dew-point temperature

2m T – deterministic scores

Northern Italy data - Nearest grid point

ifs

gme

MAM10

BIAS MAE

1.0

0.5

0.0

-0.5

-1.0

-1.5

-2.0

K

3.0

2.5

2.0

1.5

K

2m T – deterministic scores


MAM10

BIAS MAE

pat_len >

pat_len >

tur_len <

tur_len >

rlam_heat <

rlam_heat <

rat_sea <

rat_sea <

crsmin >

1.0

0.5

0.0

-0.5

-1.0

-1.5

-2.0

K

3.0

2.5

2.0

1.5

K

2m Td – deterministic scores


ifs

gme

MAM10

BIAS MAE

1.0

0.5

0.0

-0.5

-1.0

K

1.5 3.0

2.5

2.0

1.5

K

1.0

2m Td – deterministic scores


MAM10

BIAS MAE

pat_len >

pat_len >

tur_len <

tur_len >

rlam_heat <

rlam_heat <

rat_sea <

rat_sea <

crsmin >

1.0

0.5

0.0

-0.5

-1.0

K

1.5 3.0

2.5

2.0

1.5

K

1.0

•Northern Italy network

•Average over 0.5 x 0.5 deg boxes

24h precipitation

0-24h

BSS

ROC

MAM10

Remarks for COSMO-SREPS

IFS and GME driven runs are of similar quality in terms of t and td, but have different BIAS (especially for td)

For precipitation forecasts, a “well-mixed” 4 members ensemble is as skilful as the full 8 member ensemble, even in the members are of different quality

The runs with physics perturbations have similar scores, the main differences are in td

T2.1 - Model perturbations: parameters(F. Gofa, P. Louka, C. Marsigli)

New parameter perturbations are tested in a dedicated test suite (CSPERT), where IC and BCs are not perturbed (IFS operational run)

BUs are provided from Italian Special Projects

New runs of the CSPERT suite were performed, from Spring 2009 to Spring 2010

Analysis of the results for MAM and SON 2009

http://www.hnms.gr/hnms/greek/index_html

tp > 1mm /6h

tp > 10mm /6h

6h precipitation – Northern Italy MAM09

Td2m: RMSE

0

1

2

3

4

5

3 6 9 12 15 18 21 24

Td2m: BIAS

-2.5

-1.5

-0.5

0.5

1.5

2.5

3 6 9 12 15 18 21 24

-2.5

-1.5

-0.5

0.5

1.5

2.5

3 6 9 12 15 18 21 24

T,rlamheat=0.1,crsmin=200 KF,rlamheat=0.1,crsmin=200 T,crsmin=200, ratsea=1 KF,crsmin=200, ratsea=1

T,cloud=5E7 KF,cloud=5E7 T,murain=0 KF,murain=0

T, gscp=3 KF, gscp=3 T, patlen=10000, cloud=5E7 KF, patlen=10000, cloud=5E7

T,cloud=5E7, murain=0 KF,cloud=5E7, murain=0 ctrl T ctrl KF

MAM09 T2m: BIAS

-2.5

-1.5

-0.5

0.5

1.5

2.5

3 6 9 12 15 18 21 24

T2m: RMSE

0

1

2

3

4

3 6 9 12 15 18 21 24

T and Td – Greece

tp > 1mm /6h

tp > 10mm /6h

6h precipitation – Northern Italy SON09

Remarks from the CSPERT suite

Mu_rain=0:Less precipitation for low thresholdImprove the high thresholds, especially Tiedtke

memberCloud_num=5e+07:

No strong impactPat_len=10000:

Increase the precipitation, especially Tiedtke memberLittle POD improvement with small effect on FA

the set crsmin=200 (largest) and rat_sea=1 (smallest) seems to “improve” bias for T and Td, (over Greece)


2.2 Model perturbations: Developing perturbations for the lower boundary

(F.Gofa, P.Louka)

AimImplement a technique for perturbing soil moisture

conditions and explore its impacts

ReasoningThe lack of spread is typically worse near the surface

rather than higher in the troposphere. Also, soil moisture is of primary importance in determining the partition of energy between surface heat fluxes, thus

affecting surface temperature forecasts


T3.1 - Ensemble merging: comparison of the methodologies

(C. Marsigli)

COSMO-LEPS (EPS downscaling + physics perturbations) and COSMO-SREPS (multi-model IC and BCs + physics perturbations) are compared: 12UTC runs, over SON 2009 (34 runs, 12 members each)

During the last year of the project, a more clean comparison has been scheduled: 16 runs of both systems available every day

same model version

same namelists

same perturbations of the physics parameters

+24h med05

+48h med05

T3.2 - Ensemble merging: development of the COSMO-LEPS clustering

(A. Montani, A. Corigliano)

Aim: perform a dynamical downscaling where driving members for COSMO are taken from more than one global ensemble

ECMWF EPS and UKMO MOGREPS have been considered

The cluster analysis is applied on different sets of members coming from the global ensembles

initial conditions by EPS

initial conditions by MOGREPS

Issues

How does the spread/skill relationship of the single-model and mixed global ensembles look like?

Where do the best (and the worst) elements of the reduced ensembles come from? How to they score depending on their “origin”?

What is the impact of the ensemble reduction?Is it worth weighting according to the cluster population? The following ensembles are considered:

EPS (50+1): 51 members MOGREPS (23+1): 24 members MINI-MIX (EPS24 + MOGREPS24): 48 members MEGA-MIX (EPS51 +MOGREPS24): 75 members

Performance of models: spread-skill relation

MOGREPS 24

MEGAMIX 75

Where do the best (and the worst) elements come from?

MEMBER

best worst

RM

best worst

MINIMIXP

erc

en

tag

e a

nd

RM

SE

Impact of RM weighting

MEGAMIX 75: RMSE_EM = 30.7 m

REDU-MEGAMIX: RMSE_EM = 32.4 m

REDU-MEGAMIX weighted: RMSE_EM_W = 31.8 m

Future plans

Continue the work outside the CONSENS project (since

no programming of the work is possible at this stage)

Implement dynamical downscaling: nest COSMO model

in the selected RMs and generate “hybrid” COSMO-LEPS

using boundaries from members of different global

ensembles.

For a number of case, compare operational COSMO-

LEPS and “hybrid” COSMO-LEPS.

T4 - Calibration (T. Diomede)

Data collection:• Data over Switzerland, provided by MeteoSwiss (interpolated with the

SYMAP method on the 417 COSMO-LEPS grid points covering Switzerland; more than 450 stations, originally)

• Data over Germany, provided by DWD (1038 stations, interpolated with an inverse-squared-distance weighting method over the 3566 Germany grid points)

calibration over Switzerland and Germany, also on sub-areas test on the use of the specific humidity at 700 hPa for performing

the analog search test on the application of calibration functions which are specific

for underestimation and overestimation model conditions over ER; comparison among results obtained for different lengths of the

reforecast dataset over Switzerland and Emilia-Romagna; verification of the calibration process by the coupling of QPFs with

an hydrologic model (implemented for the Reno river basin, Emilia-Romagna).

Calibration over Germany

autumn 80th percentile

Autumn 2003-2007 threshold: 80-th percentile fc: +18-42 h

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1forecast probability

ob

serv

ed r

elat

ive

freq

uen

cy

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

serv

ed r

elat

ive

freq

uen

cy

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (

freq

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (

freq

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

+18-42h +66-90h


+18-42h +66-90hSummer 2003-2007 threshold: 80-th percentile fc: +18-42 h

0

0.2

0.4

0.6

0.8

1


ob

serv

ed r

elat

ive

freq

uen

cy

rawCDFANLLRANL Z

no resolution

no skill

Summer 2003-2007 threshold: 80-th percentile fc: +18-42 h

0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

serv

ed r

elat

ive

freq

uen

cy

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (

freq

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (

freq

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

summer 80th percentile


summer


0

0.2

0.4

0.6

0.8

1


ob

se

rved

rel

ati

ve

fre

qu

en

cy

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rved

rel

ati

ve

fre

qu

en

cy

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

autumn


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

95th percentile+66-90h

Calibration over Switzerland

lead time: +18-42 h lead time: +66-90 hSummer 2003-2007 threshold: 80-th percentile fc: +18-42 h

0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

80th percentilesummer

Calibration over Switzerland

lead time: +18-42 h lead time: +66-90 h

80th percentileautumn


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z


0

0.2

0.4

0.6

0.8

1


ob

se

rve

d r

ela

tiv

e f

req

ue

nc

y

rawCDFANLLRANL Z

no resolution

no skill


0.001

0.01

0.1

1


Lo

g 1

0 (f

req

uen

cy o

f u

sag

e)

rawCDFANLLRanl Z

Differences among COSMO regions95th percentile80th percentile

Autumn 2003-2007 threshold: 80-th percentile

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

day 1 day 2 day 3 day 4forecast range (h)

BS

S

raw Germany

cal CDF Germanyraw Switzerland

cal CDF Switzerlandraw Emilia-Romagna

cal CDF Emilia-Romagna

Autumn 2003-2007 threshold: 95-th percentile

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5


BS

S

raw Germany




Summer 2003-2007 threshold: 80-th percentile

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5


BS

S

raw Germany




Summer 2003-2007 threshold: 95-th percentile

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5


BS

S

raw Germany




autumn

summer

Autumn 2003-2007 Brier Skill Score forecast range: + 18-42 h Switzerland

0

0.1

0.2

0.3

0.4

Q75 Q80 Q90 Q95 Q97.5 Q99threshold (mm/24h)

BS

S

CDF 30 yr refor

CDF 12 yr refor

Autumn 2003-2007 Brier Skill Score forecast range: + 66-90 h Switzerland

0

0.1

0.2

0.3

0.4


BS

S

CDF 30 yr refor

CDF 12 yr refor

Summer 2003-2007 Brier Skill Score forecast range: + 18-42 h Switzerland

0

0.1

0.2

0.3

0.4


BS

S

CDF 30 yr refor

CDF 12 yr refor

Summer 2003-2007 Brier Skill Score forecast range: + 66-90 h Switzerland

0

0.1

0.2

0.3

0.4


BS

S

CDF 30 yr refor

CDF 12 yr refor

Impact of using a reduced reforecast data-set+68-92h

autumn

summer

+20-44h

Winter 2005-2007 Brier Skill Score threshold: 95-th percentile Emilia-Romagna

-0.2

0

0.2

0.4

0.6

0.8

20-44 44-68 68-92 92-116forecast range (h)

BS

S

rawLR2LR PP2LR on 1 anl P2LR on 1 anl Z2LR on 1 anl ZQ2LR on 1 anl Q2LR on meanZ2LR on meanQ

Autumn 2005-2007 Brier Skill Score threshold: 95-th percentile Emilia-Romagna

-0.2

0

0.2

0.4

0.6

0.8

20-44 44-68 68-92 92-116forecast range (h)

BS

S


Summer 2005-2007 Brier Skill Score threshold: 95-th percentile Emilia-Romagna

-0.2

0

0.2

0.4

0.6

0.8

20-44 44-68 68-92 92-116forecast range (h)

BS

S


calibration specific for over- and under-estimation

autumn

summer

winter

95th percentile

• using a predictor to identify if the current forecast will fall in the underestimation or in the overestimation category

• the forecast of a certain field compared against to the best analog of the same field, which identify the category

Impact on hydrological predictions

autumn

95th percentile

Autumn 2003-2008 95-th percentile warning - level 2

0

1

2

3

4

5

6

7

8

9

10

20-44 44-68 68-92 92-116forecast range

nu

mb

er

of

mis

se

s

raw

cdf

observed events


0

1

2

3

4

5

6

7

8

9

10

20-44 44-68 68-92 92-116forecast range

nu

mb

er

of

mis

se

s

raw

cdf

observed events


0

1

2

3

4

5

6

7

8

9

10

11

12

20-44 44-68 68-92 92-116forecast range

nu

mb

er

of

fals

e a

larm

s

raw

cdf

Spring 2003-2008 95-th percentile warning - level 2

0

1

2

3

4

5

6

7

8

9

10

11

12

20-44 44-68 68-92 92-116forecast range

nu

mb

er

of

fals

e a

larm

s

raw

cdf

90th percentile

missed

false alarms

Casalecchio Chiusa

Emilia-RomagnaRegion

Remarks and plans

The performance of the calibration methodologies are very much dependent on the geographic area

A multi-variable approach based on the evaluation of upper air fields at different pressure levels and times of the day will be tested

Calibration could be done over all COSMO countries included in the domain (Greece, Italy, Poland, Romania), if dense and long precipitation data series are available

Final Remarks

Next milestones

the back-up suite has been implemented, with 12 members. During next season, it will move to 16 members, probably using only the 3 global models fully available (IFS, GME, GFS)

the new microphysics perturbations will be added to the suites during within autumn 2010

test the soil moisture perturbation technique in the COSMO-SREPS suite over a period (two seasons)

Next milestones

Carry on the intercomparison between COSMO-LEPS and COSMO-SREPS for a period (from now to February 2011): 16 runs of both systems available every day

same model version

same namelists

same perturbations of the physics parameters

EPS now having EnDA+SVs

Decide about the implementation of the calibration of COSMO-LEPS outputs

Hints for discussion

COSMO-SREPS: Problems with the UM boundary conditions

Use of 3 sets of global models only (but still 16 members)

Which are the needs for BCs to run convective-permitting ensembles in the COSMO countries?

Calibration: The performance of the calibration methodology is

dependent on the precipitation threshold and on the considered area => different calibration methods for different areas?

Difficulty in “catching the bias” of precipitation over Emilia-Romagna, dependent on weather type

CONSENS Priority Project Status report COSMO year 2009/2010

Documents

Transcript of CONSENS Priority Project Status report COSMO year 2009/2010