Adaptive localization and adaptive inflation methods with the LETKF

Takemasa MiyoshiUniversity of Maryland, College Park

miyoshi@atmos.umd.edu

With many thanks toE. Kalnay, B. Hunt, K. Ide, J.-S. Kang, H. Li,

UMD Chaos-Weather group

5/20/2010, 5th International EnKF Workshop, Bergen, Norway

Adaptive covariance localization

Covariance localization (e.g., Houtekamer and Mitchell 1998)

fPfP

Empirical treatment for…• reducing sampling noise• increasing the rank

Localized

Difficulties of localizationDifficulties include…• depending on (x, y, z, t)• reducing flow-dependence

95.0 51.0 08.0

Adaptive localization

• Hierarchical filter by Anderson (2007)– Cross-validation by groups of ensembles

• ECO-RAP by Bishop and Hodyss (2009)– Smooth the sample correlations raised to a power

• High sample correlation = more reliable = more weight

• Spatial smoothing to reduce noisiness of the sample correlation

1m

TyXCross-

validationLocalization

weights1m

TyX

1m

TyX1m

TyX

LETKF algorithm (Hunt et al. 2007)

Each grid point is treated independently.

Multiple observations are treated simultaneously.

Local Ensemble Transform Kalman Filter

Localization in LETKF

),,(1 iif

iifim

fi

ai dRYTx1xx

Analysis of the i-th variable:

1. Selecting a subset of global obs for the i-th variable

iif

i dRY ,,

2. Obs error std. is weighted by the localization factor

are composed of only selected local obs.

iii RR 1~ so that far-away obs have large error.

)( mN )( mN )( mm

Two steps of localization:

R-localization, Hunt et al. (2007)

Adaptive localization with LETKFThe localization factor for R-localization is given by two adaptive components:

A. Cross-validation (Anderson 2007)as

a

ccw

2

||1 21

B. Use the sample covariance (Bishop and Hodyss 2009)High sample correlation = more reliable = more weight

bsb cw ||

),( yX Cor

),( yX Cor

c1

c2

m/2-member correl.

m-member correl.baadaptive ww ~

Advantages and disadvantages

• Advantages:– Minimal additional computations– Minimal changes to the existing LETKF code– Automatic inter-variable 4-D localization

• Disadvantages– Sampling error issue remains.

Not the best, but we can effectively avoid optimization.

Results with the Lorenz-96 model

system dim. 40

obs. network 13 regularly

ensemble size 10

inflation 10% multiplicative

adaptivefixedcombined ~~~ noisynot-noisy

Preliminary results with an AGCM

Adaptive covariance inflation

Covariance inflation (e.g., Houtekamer and Mitchell 1998)

Empirical treatment for…• covariance underestimation

Error covariance is underestimated due to various sources of imperfections:

• limited ensemble size• nonlinearity• model errors

inflation

No inflation 5% inflation

Difficulties of inflationFixed covariance inflation has difficulties such as…• ensemble spread exaggerates observing density pattern• should depend on (x, y, z, t); tuning is very difficult

Temperature spread of JMA’s global LETKF w/ fixed multiplicative inflation

~50 hPa~500 hPa

adapted from Miyoshi et al. (2010)

Additive inflation• introduces new directions to span the error space• solves the problem of the spread pattern• has difficulties in obtaining reasonable additive noise• random noise does not grow initially

Temperature spread of JMA’s global LETKF w/ additive inflation

adapted from Miyoshi et al. (2010)

~50 hPa~500 hPa

Adaptive inflation

Li et al. (2009) used the statistical relationship derived by Desroziers et al. (2005):

TfTob

ab

Tob

ob

TfTfTob

ab

ob

abaab

HHPdd

ddRHHPHHPdd

HKdHdxHxHxd

1)(

RHHPdd TfTob

ob using

In the EnKF, )( ff PP TfTo

bab HPHdd )( i.e.,

Tf

Tob

ab

HPHtr

ddtr

)( We estimate the inflation parameter:

Anderson (2007; 2009) developed an adaptive inflation algorithm using a hierarchical Bayesian approach.

Adaptive inflation in the LETKF

),,(1 iif

iifim

fi

ai dRYTx1xx

Analysis of the i-th variable:

)( mN )( mN )( mm

When computing Ti, we compute the following statistics simultaneously:

11

11

)~)((~

)~(

ii

Tfi

fii

ii

Tob

ab

itr

ddtr

RYY

R

iii RR 1~ R-localization, Hunt et al. (2007)

Normalization factor

3-dimensional field of inflation

Time smoothingDue to the limited sample size in the local region at a single time step, it is essential to apply time smoothing to include more samples in time.

We use the Kalman filter approach.

a tuning parameter that determines the strength of time smoothing002.0b

po

1

22

21

2

bo

obtot

For example,

p denotes the number of observations (i.e., sample size)

i.e., more samples, more reliable.

For example, when p = 100, 1.0o

More considerations

Localization function:is a measure of reliability

~

1

o

1~0

Considering the sample size weighted by the reliability:

Results with the Lorenz 96 model

5% inflation adaptive inflation

Observed No obs Observed No obs

6-month average after 6-month spin-up

0.31

3.07

0.31

2.71

Results with the SPEEDY model

Estimated adaptive inflation

Time series of adaptive inflation

bOvershooting

002.0b gives stable performance

4-mo. exp.

still spinning up

~20% reduction of RMSE

4% fixed inflation adaptive inflation

m/s

Too large spread Improved spread

Averages of the final 1 mo. of 4-mo. experiments

m/s

Regular obs network

With regular obs network

4% fixed inflation adaptive inflation

~20% reduction of RMSE

Improved spread

m/s m/s

Expt. with model errors (Ji-Sun Kang, 2010)

CO2 data assimilation with the SPEEDY-C model by Ji-Sun Kang

(u, v, T, q, ps, C, CF)Analyzed variables:

All variables are prognosticusing the SPEEDY-VEGAS model with dynamical vegetation

prognostic

Nature run:

like a parameterwith fixed vegetation

Results with model errors

m/s m/s K 0.1g/kg hPa ppmv 10-8kg/m2/s

by Ji-Sun Kang (2010)

RMSE and Spread

Fixed inflation

Globally constantadaptive inflation

This adaptive inflation

This adaptive inflationwith model bias correction

by Ji-Sun Kang (2010)

CF estimates by Ji-Sun Kang (2010)

RMSE and Spread of CF estimates

Fixed inflation

Globally constantadaptive inflation

This adaptive inflation

by Ji-Sun Kang (2010)

Using both adaptive localization and adaptive inflation simultaneously

Preliminary results with Lorenz-96

Adaptive localization and inflation

Adaptive inflation significantly stabilizes the filter,giving nearly optimal performance.

Future challenges

• Application to other systems– Realistic NWP models– Ocean models– Martian atmosphere models

• Assimilation of real observations– Model errors– Temporally varying observing network

• (e.g., aircraft, satellites)

The LETKF code is available at:

http://code.google.com/p/miyoshi/