Examination of Observation Impacts Derived from OSEs and Adjoint Models

Examination of Observation Impacts Derived from OSEs and Adjoint Models

Ron Gelaro and Yanqiu Zhu

NASA Global Modeling and Assimilation Office

Ricardo Todling, Ron Errico

(x 1,000,000)2.21.1

Data Types

Brightness temperatureUpper-air virtual temperature

Surface (2m) pressure

Upper-air specific humidityUpper-air meridional windUpper-air zonal wind

Rain rate

The observing system today…a 6-hr ‘snapshot’ GEOS-5 GSI 09-Aug-2007 12UTC All Data: 2,794,770 observations

The data volumes entailed by future observing systems will massively increase over the next 10 years … new approaches to ingest, process, monitor, quality control, assimilate and archive the data will have to be developed. ECMWF 10-year plan, 2006-2015

Observing System Experiments (OSEs)

The traditional method of assessing the impact of observations on forecast skill…

Subsets of observations are removed from the assimilation system and forecasts are compared against a ‘control’ system that includes all observations

Performed intermittently at operational centers but, because of their expense, usually involve a relatively small number of independent experiments, each considering relatively large subsets of observations

But what if one wants to investigate, for example, the impact of all individual channels on a given satellite…and over arbitrary periods of time, or even routinely…?

Outline of Talk

Estimation of observation impact – adjoint (ADJ) method

Observation impact results in NASA’s GEOS-5 DAS

Comparison of ADJ and OSE results

Combined use of ADJ and OSEs

Concluding remarks

Atmospheric forecast model:

)( 0xmx =f

Note that may be viewed as a transformation between a perturbation in state space and a perturbation in observation space

ba0 xxx −=δ

Atmospheric analysis (best estimate of ) :0x

)( bxhyy −=δwhere: (increment, correction vector)

(innovation vector ~106)

Data Assimilation-Forecast System

yKx δδ =0

yKx δδ =0

K determines the scalar weight (gain) given to each observation

Forecast error measure (dry energy, sfc–140 hPa):

Estimating Observation Impact

)()( v0T

v0 xxCxx −−= ffe

gxxx

xxx

x T0

203

0

3

020

2

00 )(...)

6

1

2

1( δδδδδ =+

∂∂+

∂∂+

∂∂= eee

e

Taylor expansion of change in due to change in :e 0x

3rd order approximation of in observation space:

)]()([)( vaTavb

Tb

TT xxCMxxCMKy −+−≈ ffe δδ

eδ

model adjointanalysis adjoint

3T ~)( gyδ=

…summed observation

impact

Transformation to observation-space:

yKxxx δδ =−= ba0

3T ~)( gyδδ ≈e

Properties of the Impact Estimate

0<eδ0>eδ

…the observation improves the forecast

…the observation degrades the forecast

…see Langland and Baker (2004), Errico (2007), Gelaro et al. (2007)

The “weight” vector is computed only once, and involves the entire set of observations; removing or changing the properties of one observation changes the scalar measure of all other observations.

3~g

Valid forecast range limited by tangent linear assumption for TM

The impact of arbitrary subsets of observations (e.g. instrument type, channel, location) can be easily quantified by summing only the terms involving the desired elements of .yδ

GEOS-5 Observation Impact Experiments

Analysis System

3DVAR Gridpoint Statistical Interpolation (GSI) 0.5o resolution, 72 levels

Forecast Model

GEOS-5: FV-core + full physics 0.5o resolution, 72 levels

Adjoint: Exact line-by-line (Zhu and Gelaro 2007)

Adjoint: FV-core 1o resolution + simple dry physics

Experimentation

6h data assimilation cycle, July 2005 and January 2006 24h forecasts from 00UTC to assess observation impact

Separate error response functions for the globe, NH, SH and tropics

January 2006 Global Error Norm

-30

-25

-20

-15

-10

-5

0

5

amsuaamsubairs hirs goes msu raobs

satwindsspssmiaircraftsurfaceqkswnd

January 2006 NH Error Norm

-20

-15

-10

-5

0

5



January 2006 SH Error Norm

-20

-15

-10

-5

0

5



January 2006 Tropical Error Norm

-6

-4

-2

0

2



N. Hemisphere (20o-80o)

S. Hemisphere (20o-80o) Tropics (20o-20o)

Global

Total 24hr Forecast Error Reduction due to Observations

GEOS-5 Adjoint Data Assimilation System

(J/kg)

eδ

(J/kg)

eδ

January 2006 00UTC

July 2005 Global Error Norm

-30

-25

-20

-15

-10

-5

0

5



July 2005 NH Error Norm

-20

-15

-10

-5

0

5



July 2005 SH Error Norm

-20

-15

-10

-5

0

5



July 2005 Tropical Error Norm

-6

-4

-2

0

2



N. Hemisphere (20o-80o)

S. Hemisphere (20o-80o) Tropics (20o-20o)

Global


(J/kg)

eδ

(J/kg)

eδ

Total 24hr Forecast Error Reduction due to Observations

July 2005 00UTC

Global impact of most channels is beneficial on average…with some exceptions

Cha

nnel

sC

hann

els

Forecast Error Reduction (J/kg)

Forecast Error Reduction (J/kg)

Impact of satellite observations by channel

AMSU-A

AIRS

July 2005 00UTC

improve degrade

0.0

0.0-7.0

-0.7


Localized examination of AIRS impactsJuly 2005 00UTC

AIRS impact map

H20

degrade

AIRS impact by channel

Removal of AIRS water vapor

channels improved the

forecast scores

(All Channels)

(20-50N, 0-80E)

degrade

improve

Comparison of ADJ results with OSEs

How do observation impact results based on the ADJ method compare with traditional observing system experiments…OSEs?

Can the two approaches be meaningfully compared?

GEOS-5 OSEs were conducted for July 2005 and January 2006 00UTC forecasts at 1o horizontal resolution

OSE Growth of S.H. Energy Norm

0

5

10

15

20

25

30

35

40

1 2 3 4 5

Forecast Day

Energy (J/kg)

control

no raobs

no amsua

no satwnd

no aircraft

OSE Growth of N.H. Energy Norm January 2006

0

5

10

15

20

25

30

35

40

1 2 3 4 5

Forecast Day

Energy (J/kg)

control

no raobs

no amsua

no satwnd

no aircraft

ControlNo raobNo amsuaNo satwindNo aircraft

Forecast Day Forecast Day

Skill (impact) measured using the same energy-based error metric for the globe, NH, SH and tropics as in the ADJ experiments

eJ/kg

July 2005 S. Hemisphere January 2006 N. Hemisphere

Comparison and Interpretation of ADJ and OSE Results

The ADJ measures the impact of observations in each analysis cycle separately and against the control background, while the OSE measures the impact of removing observational information from both the background and analysis in a cumulative manner

The ADJ measures the impacts of observations in the context of all other observations present in the assimilation system, while the OSE changes/degrades the system (i.e., differs for each OSE member)

The ADJ measures the response of a single forecast metric to all perturbations of the observing system, while the OSE measures the effect of a single perturbation on all forecast metrics

The ADJ is restricted by the tangent linear assumption (valid ~1-3 days), while the OSE is not

…a few things to keep in mind…

K

)()( v0T

v0 xxCxx −−= ffe

)]()([)( vaTavb

Tb

TT xxCMxxCMKy −+−= ffe δδ

‘Direct’ quantitative comparison of ADJ and OSEs

Define the fractional impact of observing system for each approach:jF j

eeF jj δδ / ADJ)( =

ctlctlno /)( OSE)( eeeFjj −=

Measures the % decrease in error due to the presence of observing system with respect to the background forecastj

1 ADJ)( =∑ j jF

Measures the % increase in error due to the removal of observing system with respect to the control forecastj

1 OSE)( ≠∑ j jF

% Impact Jan06 SH

0

5

10

15

20

25

30

35

40

amsua1amsua2amsua3

airsraobs

satwindsaircraftqkswnd

ADJ

OSE

% Contributions to 24hr Forecast Error Reduction January 2006

OSE ADJ% Impact Jan06 Global

0

5

10

15

20

25

30

35

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

% Impact Jan06 NH

-5

0

5

10

15

20

25

30

35

40

45

50

amsua1amsua2amsua3

airsraobs


ADJ

OSE

% Impact Jan06 Tropics

05

101520253035404550556065

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

Tropics (20o-20o)S. Hemisphere (20o-80o)

N. Hemisphere (20o-80o)Global

%

%

% Impact July05 SH

0

5

10

15

20

25

30

35

40

45

50

55

60

amsua1amsua2amsua3

airsraobs


ADJ

OSE

% Impact July05 Tropics

05

10152025303540455055606570

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

OSE ADJ% Impact July05 Global

0

5

10

15

20

25

30

35

40

45

50

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

% Impact July05 NH

0

5

10

15

20

25

30

35

40

45

50

amsua1amsua2amsua3

airsraobs


ADJ

OSE

% Contributions to 24hr Forecast Error Reduction July 2005

%

%



Normalized % Impact Jan06 SH

0

0.05

0.1

0.15

0.2

0.25

0.3

amsua1amsua2amsua3

airsraobs


ADJ

OSE

Normalized % Impact Jan06 Global

0

0.05

0.1

0.15

0.2

0.25

0.3

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

Normalized % Contributions to 24hr Fcst Error Reduction January 2006

OSE ADJ

%

%

Normalized % Impact Jan06 Tropics

0

0.05

0.1

0.15

0.2

0.25

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

Normalized % Impact Jan06 NH

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

amsua1amsua2amsua3

airsraobs


ADJ

OSE



Normalized % Impact July05 SH

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

amsua1amsua2amsua3

airsraobs


ADJ

OSE

Normalized % Contributions to 24hr Fcst Error Reduction July 2005

%

%

Normalized % Impact July05 Global

0

0.05

0.1

0.15

0.2

0.25

0.3

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

Normalized % Impact July05 NH

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

amsua1amsua2amsua3

airsraobs


ADJ

OSE

Normalized % Impact July05 Tropics

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

amsua1amsua2amsua3

airsraobs


ADJ

OSE1

OSE ADJ



0

5

10

15

20

25

30

35

amsuaamsubairs hirs goes

eos_amsua

msuraobssatwinds

spssmiaircraftsurfaceqkswnd

Control

No AMSUA

No Raob

No AIRS

% Contribution to 24h forecast error reduction

January 2006 Global Observations

Removal of AMSUA results in large increase in AIRS (and other) impacts

Removal of AIRS results in significant increase in AMSUA impact

Removal of Raobs results in significant increase in impact of several obs types, with AIRS and Satwinds being a notable exceptions

Combined Use of ADJ and OSEs

ADJ tool applied to each OSE member

Control

No AMSUA

No Raob

No Satwind

-1

0

1

2

3

4

5

6

7

amsuaamsubairs hirs goes

eos_amsua

msuraobssatwinds

spssmiaircraftsurfaceqkswnd

% Contribution to 24h forecast error reduction

July 2005 Tropical Observations

Removal of AMSUA results in large increase in AIRS impact in tropics

Removal of wind observations results in significant decrease in AIRS impact in tropics (in fact, AIRS degrades forecast without Satwinds!)

Combined Use of ADJ and OSEs

ADJ tool applied to each OSE member

Adjoint data assimilation system provides an accurate and efficient tool for estimating observation impact on analyses/short-term forecasts

computed with respect to all observations simultaneously permits arbitrary aggregation of results by data type, channel, location, etc.

Complement and extend, but not replace, traditional OSEs as tools for assessing observation impact…metrics, interpretations differ

Applicable to data quality assessment and selection, understanding DAS behavior, identifying redundancies in the observing system

Conclusions - 1

Excellently suited for real-time monitoring of observation impact: …see NRL page: http://www.nrlmry.navy.mil/ob_sens/

Despite fundamental differences in how impact is measured, ADJ and OSE methods provide comparable estimates of the overall ‘importance’ of most observing systems tested

Conclusions - 2

Used together, ADJ and OSEs illuminate the complex, complementary nature of how observations are used by the assimilation system

Comparisons of impacts in different forecast systems should help clarify deficiencies in data quality vs. assimilation methodology, and hopefully provide useful feedback to data producers.

Examination of Observation Impacts Derived from OSEs and Adjoint Models

Documents

Transcript of Examination of Observation Impacts Derived from OSEs and Adjoint Models