Dusanka ZupanskiCIRA/Colorado State University

Fort Collins, Colorado

Ensemble Ensemble Kalman Kalman FilterFilter

Advanced Numerics Seminar8 March 2006, Langen, Germany

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Acknowledgements: M. Zupanski, G. Carrio, S. Denning, M. Uliasz, R. Lokupitya, CSUA. Hou and S. Zhang, NASA/GMAO

Why Ensemble Data Assimilation?

Kalman filter and Ensemble Kalman filter

Maximum likelihood ensemble filter (MLEF)

Examples of MLEF applications

Future research directions

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

OUTLINE

Why Ensemble Data Assimilation?

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Three main reasons :

Need for optimal estimate of the atmosphericstate + verifiable uncertainty of this estimate; Need for flow-dependent forecast errorcovariance matrix; and The above requirements should be applicableto most complex atmospheric models (e.g.,non-hydrostatic, cloud-resolving, LES).

Example 1: Fronts

Example 2: Hurricanes

(From Whitaker et al., THORPEX web-page)Benefits of Flow-Dependent Background Errors

Are there alternatives?

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Two good candidates: 4d-var method: It employs flow-dependent forecasterror covariance, but it does not propagate it in time. Kalman Filter (KF): It does propagate flow-dependent forecast error covariance in time, but it is tooexpensive for applications to complex atmosphericmodels.

EnKF is a practical alternative to KF, applicable tomost complex atmospheric models.

⇓

⇓

A bonus benefit: EnKF does not use adjoint models!

Typical EnKF

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Forecast error Covariance Pf

(ensemble subspace)

DATA ASSIMILATION

Observations First guess

Optimal solution for model statex=(T,u,v,f, )

ENSEMBLE FORECASTING

Analysis error Covariance Pa

(ensemble subspace)

INFORMATION CONTENT ANALYSIS

Tb,ub,vb,fb, α,β,γ

Hessianpreconditioning

Non-GaussianPDFs

Maximum Likelihood Ensemble Filter

x

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Data Assimilation Equations

Equations in model space:

Prior (forecast) error covariance of x (assumed known):

- Dynamical model for model state evolution (e.g., NWP model)M

- Model state vector of dim Nstate ; w - Model error vector of dim Nstate

G - Dynamical model for state dependent model error

Model error covariance (assumed known):

E - Mathematical expectation;

GOAL: Combine Model and Data to obtain optimal estimate of dynamical state x

n - Time step index

- Observations vector of dim Nobs ;

Observation error covariance, includes also representatives error (assumed known):

H - Observation operator

Equations in data space:

- Observation error

Data Assimilation Equations

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

- Time step index (denoting observation times)

Data assimilation should combine model and data in an optimal way.Optimal solution z can be defined in terms of optimal initial conditions xa(analysis), model error w, and empirical parameters α,β,γ.

Approach 1:Approach 1: Optimal solution (e.g., analysis xa) = Minimum varianceestimate, or conditional mean of Bayesian posterior probability densityfunction (PDF) (e.g., Kalman Kalman filterfilter; Extended Extended Kalman Kalman filterfilter; EnKFEnKF)

xa = E x y( ) = xp(x y)! dx = xp(y x)p(x)

p(y)! dx p - PDF

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

How can we obtain optimal solution?Two approaches are used most often:

For non-liner M or H the solution can be obtained employingExtended Extended Kalman Kalman filterfilter, or Ensemble Ensemble Kalman Kalman filterfilter.

Assuming liner M and H and independent Gaussin PDFs⇒ Kalman Kalman filterfilter solution (e.g., Jazwinski 1970)

xa is defined as mathematical expectation (i.e., mean) of the conditionalposterior p(x|y), given observations y and prior p(x).

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Approach Approach 22:: Optimal solution (e.g., analysis xa) = Maximum likelihoodestimate, or conditional mode of Bayesian posterior p(x|y)(e.g., variationalvariational methods; MLEFMLEF)

For independent Gaussian PDFs, this is equivalent to minimizing cost function J:

xk+1

= xk! H

!1"J

k

Solution can be obtained (with ideal preconditioning) in one iteration for liner H andM. Iterative solution for non-linear H and M:

H!1 - Preconditioning matrix = inverse Hessian of J

xa= Maximum of posterior p(x|y), given observations and prior p(x).

xa= max p x y( )!" #$ = max

p(y x)p(x)

p(y)= min % log p x y( )!" #${ }

VARIATIONAL

MLEF

Milija Zupanski, CIRA/CSUZupanskiM@CIRA.colostate.edu

Ideal Hessian Preconditioning

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

xmode xmean

x

p(x)

Non-Gaussian

xmode = xmean

x

p(x)

Gaussian

MEAN vs. MODE

For Gaussian PDFs and linear H and M results of all methods [KF, EnKF (with enoughensemble members), and variational] should be identical, assuming the same Pf and yare used in all methods.

Minimum variance estimate= Maximum likelihood estimate!

KF,EnKF,4d-var,

allcreatedequal?

Does this really happen?!?

TEST RESULTS EMPLOYING A LINEAR MODEL AND GAUSSIAN PDFs

(M.Uliasz)

(D. Zupanski)

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

- Optimal estimate of x (analysis)

Kalman Kalman filter solutionfilter solution

xa = xb + Pf H

T(HPf H

T+ R)

!1y ! H (xb )[ ]

Analysis step:

- Background (prior) estimate of x

Pa = [I ! Pf H

T(HPf H

T+ R)

!1H ]Pf = I ! KH( )Pf

Pa - Analysis (posterior) error covariance matrix (Nstate x Nstate)

Forecast step:

;

Pf = MPa M

T+GQG

T - Update of forecast error covariance

K - Kalman gain matrix (Nstate x Nobs)

Often neglected

Ensemble Kalman Filter (EnKF) solutionEnKF as first introduced by Evensen (1994) as a Monte Carlo filter.

Analysis solution defined for each ensemble member i:

xa

i= xb

i+ Pf

eH

T(HPf

eH

T+ R

e)!1( y

i! H (xb

i))

Mean analysis solution:

xa = xb + Pf

eH

T(HPf

eH

T+ R

e)!1( y ! H (xb ))

Analysis error covariance in ensemble subspace:

Analysis step:

b

i

a= x

a

i- x

a

Analysis ensembleperturbations:

=

p1,1

ap1,2

a. p

1,Nens

a

p2,1

ap2,2

a. p

2,Nens

a

p3,1

ap3,2

a. p

3,Nens

a

. . . .

pNstate,1f

pNstate,2f

. pNstate,Nensf

!

"

######

$

%

&&&&&&

= b1

ab2

a. bNens

a!" $%

Pa

e( )1 2

Pa

e=

1

Nens-1

Pa

e( )1 2

Pa

e( )1 2!

"#$

T

Sample analysis covariance

Ensemble Kalman Filter (EnKF)Forecast step:

Forecast error covariance calculated using ensemble perturbations:

Ensemble forecasts employing a non-linear model M

=

p1,1

fp1,2

f. p

1,Nens

f

p2,1

fp2,2

f. p

2,Nens

f

p3,1

fp3,2

f. p

3,Nens

f

. . . .

pNstate,1f

pNstate,2f

. pNstate,Nensf

!

"

######

$

%

&&&&&&

= b1

fb2

f. bNens

f!" $%

bif= M (xa

i ) ! M (xa )

Pf

e( )1 2

Pf

e=

1

Nens -1Pf

e( )1 2

Pf

e( )1 2!

"#$

T

;

Sample forecast covarianceNon-linear forecast perturbations

There are many different versions of EnKF

Monte Carlo EnKF (Evensen 1994; 2003)

EnKF (Houtekamer et al. 1995; 2005; First operational version)

Hybrid EnKF (Hamill and Snyder 2000)

EAKF (Anderson 2001)

ETKF (Bishop et al. 2001)

EnSRF (Whitaker and Hamill 2002)

LEKF (Ott et al. 2004)

MLEF (Zupanski 2005; Zupanski and Zupanski 2006)

Minimum variancesolution

Maximumlikelihood solution

Why maximum likelihood solution? It is more adequate for employing non-Gaussian PDFs.

Current status of EnKF applications

EnKF is operational in Canada, since January 2005(Houtekamer et al.). Results comparable to 4d-var.

EnKF is better than 3d-var (experiments with NCEP T62GFS) - Whitaker et al., THORPEX presentation ).

Very encouraging results of EnKF in application to non-hydrostatic, cloud resolving models (Zhang et al., Xue et al.).

Very encouraging results of EnKF for ocean (Evensen etal.), climate (Anderson et al.), and soil hydrology models(Reichle et al.).

Theoretical advantages of ensemble-based DA methods aregetting confirmed in an increasing number of practicalapplications.

Examples of MLEF applications

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

- Dynamical model for standard model state x

Maximum Likelihood Ensemble Filter

- Dynamical model for model error (bias) b

- Dynamical model for empirical parameters γ

Define augmented state vector z

Find optimal solution (augmented analysis) za by minimizing J(MLEF method):

And augmented dynamical model F

⇓

,

.

(Zupanski 2005; Zupanski and Zupanski 2006)

INNOVATION !2 TEST (biased model)

(neglect_err, 10 ens, 10 obs)

0.00E+00

2.00E+00

4.00E+00

6.00E+00

8.00E+00

1.00E+01

1.20E+01

1 11 21 31 41 51 61 71 81 91

Analysis cycle

INNOVATION !2 TEST (biased model)

(bias_estim, 10 ens, 10 obs, bias dim = 101)

0.00E+00

2.00E+00

4.00E+00

6.00E+00

8.00E+00

1.00E+01

1.20E+01

1 11 21 31 41 51 61 71 81 91

Analysis cycle

INNOVATION !2 TEST (biased model)

(bias_estim, 10 ens, 10 obs, bias dim = 10)

0.00E+00

2.00E+00

4.00E+00

6.00E+00

8.00E+00

1.00E+01

1.20E+01

1 11 21 31 41 51 61 71 81 91

Analysis cycle

INNOVATION !2 TEST (non-biased model)

(correct_model, 10 ens, 10 obs)

0.00E+00

2.00E+00

4.00E+00

6.00E+00

8.00E+00

1.00E+01

1.20E+01

1 11 21 31 41 51 61 71 81 91

Analysis cycle

NEGLECT BIAS BIAS ESTIMATION (vector size=101)

BIAS ESTIMATION (vector size=10) NON-BIASED MODEL

BIAS ESTIMATION, KdVB model(Zupanski and Zupanski 2006)

It is beneficial to reduce degrees of freedom of the model error.

Both the magnitude and the spatial patterns of thetrue bias are successfully captured by the MLEF.

40Ens

100Ens

True βR

Cycle 1 Cycle 3 Cycle 7

Bias estimation: Respiration bias βR, using LPDM carbontransport model (Nstate=1800, Nobs=1200, DA interv=10 days)

Domain with larger bias (typically land)

Domain with smaller bias (typically ocean)

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Information measures in ensemble subspace

Shannon information content,or entropy reduction

Degrees of freedom (DOF) for signal (Rodgers 2000):

- information matrix in ensemble subspace of dim Nens x Nens

- are columns of Z

- control vector in ensemble space of dim Nens

- model state vector of dim Nstate >>Nens

Errors are assumed Gaussian in these measures.

(Bishop et al. 2001; Wei et al. 2005; Zupanski et al. 2005, 2006)

!i

2

- eigenvalues of C

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

LPDM Model CO2-flux BIAS estimation:Eigenvalue spectrum of (I+C)-1/2

(First 40 eigenvalues, Nens = 1800, 100, and 40)

Eigenvalue spectrum isvery similar for all 3ensemble sizes!

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

PCTM Global Model CO2-flux estimation:Eigenvalue spectrum of C

(Nstate=13104, Nobs=13104, fully observed system, Nens= 500)

Ensemble size of 500 is adequate for describing all DOFs of thisfully observed system.In later cycles more eigenvalues are approaching value 1 (noinformation).

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

GEOS-5 Single Column Model: DOF for signal(Nstate=80; Nobs=80, seventy 6-h DA cycles,assimilation of simulated T,q observations)

Small ensemble size (10 ens), even though notperfect, captures main data signals.

RMS Analysis errors for T, q:------------------------------------10ens ~ 0.50K; 0.566g/kg20ens ~ 0.32K; 0.462g/kg40ens ~ 0.27K; 0.417g/kg80ens ~ 0.20K; 0.362g/kg-------------------------------------No_obs ~ 0.82K; 0.656g/kg

DOF for signalvaries from oneanalysis cycle toanother due tochanges inatmosphericconditions.

DOF for signal (ds)

40 ens, 80 obs of T and Q

0

10

20

30

40

1 11 21 31 41 51 61

Analysis cycle

ds

MLEF application to GEOS-5 single column model: DOF for signalMLEF application to GEOS-5 single column model: DOF for signal((dsds))

!+

=+="

i i

i

strd

)1(])([

2

21

#

#CCI

DOF for signal reflects dynamics of the true state! Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Independentobservation

IFN↑ abovethe inversion,as observed

IFN↓ belowinversion ascloud forms

Courtesy of G. Carrió

MLEF assimilation of MLEF assimilation of radar/lidar radar/lidar IWP and LWP using RAMS/LESIWP and LWP using RAMS/LES

CONTROL

EXP

VERIF

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Non-Gaussian (lognormal) MLEF framework: CSU SWM Non-Gaussian (lognormal) MLEF framework: CSU SWM (Randall et al.)(Randall et al.)

( ) ( )i

N

i

S

T

f

f

Tfobs y

my

my

xJ !=

""#$

%&'

(+))

*

+,,-

."#

$

%&'

())*

+,,-

."#

$

%&'

(+""=

1

11

)(ln

)(ln

)(ln

2

1

2

1)(

xxR

xxxPxx

HHH

Beneficial impact of correct PDF assumption – practical advantagesDusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Cost function derived from posterior PDF( x-Gaussian, y-lognormal):

Lognormaladditional nonlinear term

Normal(Gaussian)

Courtesy of M. Zupanski

!!"

#$$%

& '=

ba

refxxx exp)(H

Future Research Directions

Covariance inflation and localization need further investigations: Arethese techniques necessary?

Model error and parameter estimation need further attention: Do wehave sufficient information in the observations to estimate complex modelerrors?

Information content analysis might shed some light on DOF of modelerror and also on the necessary ensemble size.

Non-Gaussian PDFs have to be included into DA (especially for cloudvariables).

Characterize error covariances for cloud variables.

Account for representativeness error.

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

References for further readingReferences for further reading

Anderson, J. L., 2001: An ensemble adjustment filter for data assimilation. Mon. Wea. Rev., 129,2884–2903.

Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model usingMonte Carlo methods to forecast error statistics. J. Geophys. Res., 99, (C5),. 10143-10162.

Evensen, G., 2003: The ensemble Kalman filter: theoretical formulation and practicalimplementation. Ocean Dynamics. 53, 343-367.

Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter/3D-variational analysisscheme. Mon. Wea. Rev., 128, 2905–2919.

Houtekamer, Peter L., Herschel L. Mitchell, 1998: Data Assimilation Using an Ensemble KalmanFilter Technique. Monthly Weather Review: Vol. 126, No. 3, pp. 796-811.

Houtekamer, Peter L., Herschel L. Mitchell, Gerard Pellerin, Mark Buehner, Martin Charron,Lubos Spacek, and Bjarne Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Monthly Weather Review: Vol.133, No. 3, pp. 604-620.

Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation.Tellus., 56A, 415–428.

Tippett, M. K., J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemblesquare root filters. Mon. Wea. Rev., 131, 1485–1490.

Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbedobservations. Mon. Wea. Rev., 130, 1913–1924.

Zupanski D. and M. Zupanski, 2006: Model error estimation employing an ensemble dataassimilation approach. Mon. Wea. Rev. (in press).

Zupanski, M., 2005: Maximum likelihood ensemble filter: Theoretical aspects. Mon. Wea. Rev.,133, 1710–1726

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Thank you.

Exp1:Randomnoise incycle 1

Exp2:Correlatedrandomnoise incycle 1

(From M.Zupanski et al.,Tellus)

Time evolving model dynamics significantly reduces the noise in the initiallyprescribed perturbations!

Example: CSU SWM model of Randall et al.Solution-Truth shown

23 2-h DA cycles: 18UTC 2 May 1998 – 00 UTC 5 May 1998(Mixed phase Arctic boundary layer cloud at Sheba site)Experiments initialized with typical clean aerosol concentrationsMay 4 was abnormal: high IFN and CCN above the inversionΔx= 50m, Δzmax = 30m (2d domain: 50col, 40lev), Δt=2s, Nens=48Sophisticated microphysics in RAMS/LESControl variables: Θ_il, u, v, w, N_x, R_x (8 species), IFN, CCN (dim= 22 variables x 50 columns x 40 levels = 44000)Radar/lidar real observations of IWP, LWP are assimilated IWP and LWP are vertically integrated quantities (no informationabout the profiles of IFN, CCN is observed)

Acronyms:IFN - Ice Forming NucleiCCN - Cloud Concentration NucleiIWP (LWP) - Ice (Liquid) Water Path

MLEF experiments with CSU/RAMS Large Eddy Simulation (LES) modelMLEF experiments with CSU/RAMS Large Eddy Simulation (LES) model

MLEF is similar to 4dvar because it seeks a maximumlikelihood solution (i.e., minimum of J). It is also similar to EnKF methods because it usesensembles to calculate forecast error covariance. MLEF uses the same definition of transformation matrixas in the ETKF (Bishop et al. 2001). It has a capability to estimate and reduce several majorsources of forecast uncertainties simultaneously: Initialconditions, model error, boundary conditions, andempirical parameters. MLEF has also a capability to take into account non-Gaussian (log-normal) PDFs (Flatcher and M. Zupanski2006)

Dusanka Zupanski, CIRA/CSUZupanski@CIRA.colostate.edu

Basic characteristics of MaximumLikelihood Ensemble Filter (MLEF)

(Zupanski 2005; Zupanski and Zupanski 2006)