F. Prates/Grazzini, Data Assimilation Training Course March 2006 1 Error Tracking F. Prates/ F....

F. Prates/Grazzini, Data Assimilation Training Course March 2006 1

Error Tracking

F. Prates/ F. Grazzini


Monitoring of the forecasting system is carried out on daily basis by a meteorologist at ECMWF.

The main reason of this activity is to investigate bad or very inconsistent forecast trying to detect deficiencies in the analysis and in the forecasting system.

Investigations are covering all aspects of the system, often dealing with initial conditions (data availability) and data assimilation problems.

INTRODUCTION

ERROR TRACKING BY MEANS OF SYNOPTIC-DIAGNOSIS


Every day we summarize our findings in the MetOps Daily Report. The daily report is posted on our internal web site where can be accessed by people in RD and OD.

Every four months there is a special meeting (OD/RD meeting) in which OD present a summary of the daily reports of the previous months.

Daily Report


Investigations can be divided in the following main steps:

Spot the problem

Find out when the error enter in the system

Where did it happen

What caused the error

TROUBLESHOOTING PROCEDURES


WHEN?

Verification statistics would tell which forecast had a bad performance


ANALYSIS ECMWF D+6 FC

WHEN ?


WHERE?

Different techniques are used to identify the origin of forecast error

1) Error maps:

A sequence of maps shows how initial errors will move downstream.

Focus on the evolution of the most amplified error wave train.

because …

Error patterns become more complex as the forecast range increases.

The energy associated to the wave train is transmitted by their group velocity which is different of phase speed of the individual perturbations.


+144


+120


+96


+84


+72


+60


+48


+36


+24


+12


Winter track

Summer track

The most likely areas for errors(energy) to rapidly amplify(release) are baroclinic regions and developing cyclones

They provide the most efficient mechanism for the “spread of influence” in mid-latitude upper-tropospheric westerlies.

Theoretical and observational studies indicate that the energy associated to the wave packets travel at 30˚/day in midlatitudes.

ERROR PROPAGATION / DOWNSTREAM DEVELOPMENT

(Anders Persson)


WHERE?

2) EPS perturbations:

The perturbation fields computed by EPS can help to identify where the atmosphere is sensitive to possible errors growth.

These perturbations are generated using singular vectors of a linear version of ECMWF, which maximize the total energy norm (phase space) over a 48-hour time interval with a energy peaking at around 700 hPa in regions of strong barotropic and baroclinic energy conversion, at initial time. Thus we expected that small errors in initial conditions will amplify most rapidly affecting the forecast.


WHERE?

520

520

520

520

520

560560

560

560

560

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E 60°E

60°E 80°E

80°E 100°E

100°E 120°E

120°EZ500 hPa ANALYSIS VT: 20020329 12 UTCSTDEV of EPS INITIAL PERTURBATION for T 700 hPa VT: 20020329 12 UTC

0

0.25

0.5

0.75

1

1.5

2

2.5

1104

1104

1152

1152

1152

1152

12001200

12001200

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E 60°E

60°E 80°E

80°E 100°E

100°E 120°E

120°EZ200 hPa ANALYSIS VT: 20020329 12 UTCSTDEV of EPS INITIAL PERTURBATION for V-comp 200 hPa VT: 20020329 12 UTC

0

1

2

3

4

5

6

7


Zoom on wind sensitivity: is suggesting a reshaping of the Jet streak west of Alaska

SENSITIVITY MAPS: KEY ANALYSIS ERROR


COMPARISON WITH OTHERS MODELS

ANALYSIS D+6 ECMWF

D+6 UKMO D+6 T255/UKMO ANA


ANALYSIS DIFFERENCE AT 500 hPa Z


WHERE? –Part 2

H

L

L

L

516 528

52854

0

540

552

552

552

552

552

564

564

576

40°N

60°N

60°W

60°W

40°W

40°W

20°W

20°W

0°

0°

20°E

20°E 40°E

40°E

60°E

60°E

500z20040216 12UTC ECMWF FC t+120 VT: 20040221 12UTC

H H

L

L

L

LL

516

516

528

528

540

540

552

552

552

564

564

564

576

40°N

60°N

60°W

60°W

40°W

40°W

20°W

20°W

0°

0°

20°E

20°E 40°E

40°E

60°E

60°E

500z20040215 12UTC ECMWF FC t+144 VT: 20040221 12UTC


WHERE?

3) Extracting envelope of packets of synoptic waves (Rossby):

Locate wave packets in atmospheric data (wave velocity group).

This method is applied to the field difference between to successive forecast runs .

We do expect …

that local differences in the beginning would propagate in a form of Rossby waves packets in upper troposphere


M

MM

1

1

1 1

9

9

-64

00

27.

15.13.

6. 6.

-20. -12.-10.

-6.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E

Monday 16 Feb 2004 0utc T511/ solid others runs/ dash difference: red/blue (dgpm) 20040216 00UTC ECMWF FC t+132 VT: 20040221 12UTC 500 ** // 20040215 12UTC ECMWF FC t+144 VT: 20040221 12UTC 500 **

1

5

9

13

17

19.48

DIFFERENCE TRACKING: streamfunction+envelope of the difference


M

M

1

1

1

11

5

5

-64-64

0

0

-64

-32

14.

13.11. 6.

0.

-18.-9.

-8.

-7.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E


1

3

5

7

9

10.98



MM 1

1 1

3.56

0

-64

14.

11.10. 5.

3.

2.

0.0.

-9. -8.

-6.

-6.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E


1

2.25

3.5

4.75

6

6.482



M

M

1

3.5

10.

5.4. 1.

1.0.

-9.

-9.-4.

-4.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E


1

2.25

3.5

4.75

6

6.704



M M

M

1

1

11

-32

5.

4.

3.

2.

2.

1.1.

-6.-5.

-1.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E


1

2

3

4

5

5.459



MM4.

2.

1.

1.

1.

0.

-6.-3.

-2.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E


1

1.5

2

2.5

3

3.404



M2.

1.

1.

1.

1.

1.

0.

-4.-2.

-1.

-1.

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E

Monday 16 Feb 2004 0utc T511/ solid others runs/ dash difference: red/blue (dgpm) 20040215 12UTC ECMWF FC t+12 VT: 20040216 00UTC 500 **

ECMWF AN VT: 20040216 00UTC 500 ** //

1

1.25

1.5

1.75

2

2.121



520

520

520

520

520

520

520

520

560

560

560

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E 60°E

60°E 80°E

80°E 100°E

100°E 120°E

120°E

Z500 hPa ANALYSIS VT: 20040216 00 UTCSTDEV of EPS INITIAL PERTURBATION for T 700 hPa VT: 200400216 00 UTC

0

0.25

0.5

0.75

1

1.5

2

2.5

264288

288

288

288

288

312

312

312

312

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E 60°E

60°E 80°E

80°E 100°E

100°E 120°E

120°E

Z700 hPa ANALYSIS VT: 20040216 00 UTCSTDEV of EPS INITIAL PERTURBATION for WINDSPEED 850 hPa VT: 200400216 00 UTC

0

1

2

3

4

5

6

7


5.0m/s

0° 0°

10°N10°N

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

70°N70°N

140°E

140°E 160°E

160°E 180°

180° 160°W

160°W 140°W

140°W 120°W

120°W 100°W

100°W 80°W

80°W 60°W

60°W 40°W

40°W 20°W

20°W 0°

0° 20°E

20°E 40°E

40°E

20040216 0utc exp=01 850hPa INCR (z:10m w:5m/s t:1K) AN (solid black) FG (dash black) OBS (+/-50hPa) used not flagged:navy flagged but used:ochre rejected:red

ANALYSIS INCREMENTS : 20020416 00utc 850hPa


WHAT DATA?

After “when” and “where” has been answered…

ECMWF data base provides records and statistics of available observations in the area

The cause/effect relation between increments and obs is not always trivial

But we can…

Assess the impact of different obs data in the analysis comparing the obs departures from the first-guess and analysis.

With 4DVAR the increments no longer have a local interpretation- the errors may come from other regions as a consequence of the assimilation window.

Other causes…

If one or several observations are wrong → quality control is applied

If the obs errors turn out to be systematic → blacklisting is produced


DROPSONDES and TEMPs at 850 hPa – 23-03utc 20040216

269

272

271

274

277

285BL

270

BL

BL

BL

-42101

-32116

-32132

-42147

7

2155

-22204

7

2203

-22219

-2

2236

3

2248

-4

2250

-42308

3

2303

1

2319

-32322 5

2334

0

2346

-22357

62351

8 4

-4 0

14

0

7 20

-3 23

-5

0

-4 0

6 34

4 0

4 0

3 0

-1 0

-7 0

-8 0

-6 0

-6 0

-6 0

-4 0

-6 0

15 0

9 0

-4

55

4 53

2 114

-2 0

-3 0

2 127

2 144

-2 201

-3 224

-4

226

-4 240

-1

246

-5

253

-3 253

-2

256

20°N 20°N

30°N30°N

40°N 40°N

50°N50°N

60°N 60°N

140°W

140°W 120°W

120°W

20040216 0utc exp=01 850hPa INCR (z:10m w:5m/s t:1K) AN (solid black) FG (dash black) OBS (+/-50hPa) used not flagged:navy flagged but used:ochre rejected:red

SYNOP: 0 DRIBU: 0 AIREP: 0 SATOB: 0 TEMP: 0 PILOT: 0 SATEM: 0 UNFLAGGED 0

POSS ERR 0 PROB ERR 0 ERROR: 0

38 Dropsondes

Analysis mass and wind increments at 850 hPa


OBSERVATIONS STATISTICS - 08082002 12UTC

3D view of the increments isosurfaces: 20040216 00utc

Profile of mass increments at cursor location


-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

OBS - FG

1000 925 850

700

500

400

300

250

200

150

100

70

50

30

20

10

HP

A

TEMPERATUREAREA: (30N ,150W) - (60N ,125W)

06 UTC 16 FEB 2004UPPER AIR SOUNDINGS

NO. OF USED OBS: 359 ( 96 %)NO. OF OBS: 373 BIAS: .3 STD: 1.7

2 /0

13 /0

13 /0

13 /0

13 /0

13 /1

13 /0

11 /0

11 /0

-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

OBS - AN

1000 925 850

700

500

400

300

250

200

150

100

70

50

30

20

10

HP

A


06 UTC 16 FEB 2004UPPER AIR SOUNDINGS


2 /0

13 /0

13 /0

13 /0

13 /0

13 /1

13 /0

11 /0

11 /0

DROPSONDES DEPARTURES from First-Guess and Analysis

The analysis(solid) shows a significant reduction of the departure from

Obs -> strong impact of the observation


-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

OBS - FG (C)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.


All cycles 16 FEB 2004AIRCRAFT


-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

OBS - AN (C)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.


All cycles 16 FEB 2004AIRCRAFT


AIRCRAFT DEPARTURES from First-Guess and Analysis

Fairly strong impact also from aircraft


-6-5.5-5-4.5-4-3.5-3-2.5-2-1.5-1-0.500.511.522.533.544.555.56

OBS - FG (C)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.

BRIGHTNESS TEMPERATUREAREA: (30N ,150W) - (60N ,125W)

All cycles 16 FEB 2004ATOVS AMSU-A-c 4

NO. OF USED OBS: 0 ( 0 % )NO. OF OBS: 11387 BIAS: 1.1 STD: 1.8

Channel: 4

-6-5.5-5-4.5-4-3.5-3-2.5-2-1.5-1-0.500.511.522.533.544.555.56

OBS - FG (C)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.



NO. OF USED OBS: 904 ( 8 %)NO. OF OBS: 11387 BIAS: .2 STD: .6

Channel: 5

-6-5.5-5-4.5-4-3.5-3-2.5-2-1.5-1-0.500.511.522.533.544.555.56

OBS - FG (C)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.



NO. OF USED OBS: 1161 ( 10 %)NO. OF OBS: 11387 BIAS: .0 STD: .2

Channel: 9

ATOVS RADIANCES DEPARTURES from the First-Guess

Weak impact from polar satellite radiances due to very small departure from the First-Guess

Not Used


-20-18-16-14-12-10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20

OBS - FG (M/S)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.

WINDSPEEDAREA: (30N ,150W) - (60N ,125W)

All cycles 16 FEB 2004QSCAT


-20-18-16-14-12-10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20

OBS - AN (M/S)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RE

L. F

RE

Q.

WINDSPEEDAREA: (30N ,150W) - (60N ,125W)

All cycles 16 FEB 2004QSCAT

NO. OF USED OBS: 1469 ( 44 %)NO. OF OBS: 3322 BIAS: -.1 STD: 2.1

QSCAT DEPARTURES from First-Guess and Analysis

Moderate positive impact from Qscat


SUMMARY

Synoptic diagnosis of NWP forecast is a necessary complement to the usual statistical verifications.

Diagnostic tools allow to identify complex problems that often do not show up in objective scores.

Through this type of monitoring we have been able to identify several problems successively taken under consideration by the RD department.


To find out more:

http://www.ecmwf.int/products/forecasts/guide/Monitoring_of_the_data_assimilation_system.html

Persson, A, 2000: Synoptic-dynamic diagnosis of medium range weather forecast systems, ECMWF

Seminar on diagnosis of models and data assimilation systems, 6-10 September 1999.pp.123-137 .

Zimin A. V. et al. , May 2003: Extracting Envelopes of Rossby Wave Packets, Monthly Weather Review,

131, pp. 1011-1017

F. Prates/Grazzini, Data Assimilation Training Course March 2006 1 Error Tracking F. Prates/ F....

Documents

Transcript of F. Prates/Grazzini, Data Assimilation Training Course March 2006 1 Error Tracking F. Prates/ F....