Nonlinear data assimilation via hybrid particle …...2017/07/08 · Nonlinear data assimilation...

Nonlinear data assimilation via hybridparticle-Kalman filters and optimal coupling

Walter Acevedo, Sebastian Reich and Maria ReinhardtUniversität Potsdam, Germany

RIKEN International Symposium on Data Assimilation 201727 February – 2 March, 2017, AICS, Kobe, Japan

Sequential Data Assimilation Strategies

Traditional approachesÉ Ensemble Kalman Filters

+ Robust and moderately affordable– Biased for non-Gaussian PDFs

É Traditional Particle Filters+ Non-Gaussianity properly handled– Liable to the "Curse of

Dimensionality”

Some recent developmentsÉ Localized particle filters

[Lei and Bickel, 2011][Poterjoy, 2015]É Hybrid Kalman-particle filters

[Frei and Künsch, 2013][Chustagulprom et al., 2016]É Optimal transportation based particle filters

[Reich, 2013]É Intrinsic Dimension [Agapiou et al,

arXiv:1511.06196],

Nonlinear DA via hybrid particle-Kalman filters and optimal coupling 2

Outline

Motivation

Linear Ensemble Transform Filters

Hybrid Ensemble Transform Filter

Single one-dimensional DA step

Non-spatially extended dynamical systems

Spatially extended systems

Conclusions and Prospect

,Nonlinear DA via hybrid particle-Kalman filters and optimal coupling 3

Linear Ensemble Transform Filter (LETF)

É Forecast and Analysis ensembles:

zfi, za

i, i = 1, . . . ,M

É Update step via a General linear transformation[Reich and Cotter, 2015]

zaj

=M∑

i=1

zfidij

É Transform coefficients satisfy

M∑

i=1

dij = 1

M∑

j=1

dij = Mwi


Ensemble Square Root Filter [Tippett et al., 2003]

ESRF as a LETF:

zaj

=M∑

i=1

zfiwi +

M∑

i=1

(zfi− zf)sij =

M∑

i=1

zfidKF

ij

É Transform coefficients:

dKFij

= dKFij

({zfl},yobs) := sij + wi −

1

M,

É Square root matrix: S = {sij}

S =

�

I +1

M− 1(HAf)TR−1HAf

�−1/2

,

É ESRF “Weights”:

wi =1

M−

1

M− 1eT

iS2(HAf)TR−1(Hzf − yobs),

M∑

i=1

wi = 1


Ensemble Transform Particle Filter [Reich, 2013]

Analysis Mean

za =M∑

i=1

wizfi, wi ∼ exp

�

−1

2(Hzf

i− yobs)TR−1(Hzf

i− yobs)

�

ETPF transform update DF : {dPFij }

zaj

=M∑

i=1

zfidPF

ij

Transform constraints:M∑

i=1

dij = 1,M∑

j=1

dij = wiM, dij ≥ 0,

Minimal Transportation Cost:

DPF = argmin J(D), J(D) =M∑

i,j=1

dij‖zfi− zf

j‖2


Optimal Transportation (OT) Problem

5 10 15 200

0.02

0.04

0.06

0.08

0.1

Imp

ort

an

ce

we

igth

Particle Number

⇒ OptimalCoupling

⇒

5 10 15 200

0.02

0.04

0.06

0.08

0.1

Imp

ort

an

ce

we

igth

Particle Number

Optimal transportation cost: min J(D)

É also known as Earth Mover distanceÉ takes into account shape of the PDFs

Optimal coupling: DPF

É maximizes correlation between prior and posterior ensemblesÉ is deterministic, preserving the regularity of the state fieldsÉ consistent with Bayes’ formula [Reich 2013]É convergence (regarding ensemble size) is faster than SIR-PFs at the

cost of solving an optimal transportation problem


OT Solvers

Exact solution

É Computational complexity O(M3 ln(M))

É Efficient Earth Mover Distance algorithms available, e.g. FastEMD[Pele and Werman, 2009].

Entropic Regularized Approximation [Cuturi, 2013]

J(D) =M∑

i,j=1

�

dij‖zfi− zf

j‖2 +

1

λdij lndij

�

É Sinkhorn’s fixed point iteration can be used.É Computational complexity O(M2 ·C(λ))

1D Approximation(Each variable independently updated)É OT problem reduces to reordering.É Computational complexity O(M ln(M))

É No particle distance needed


Example: Single 1D Assimilation Step

Observation within the convex hull

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6ETKF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

0.0

0.5

1.0

1.5

2.0

2.5

3.0ETPF weights

Obser ation

Prior ensemble

ETPF analysis

ETKF analysis



Observation within the convex hull

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Pos

erior In

dex

ER ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Par icle Index

0.0

0.5

1.0

1.5

2.0

2.5

3.0ER ETPF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Pos

erior In

dex

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Par icle Index

0.0

0.5

1.0

1.5

2.0

2.5

3.0ETPF weights

Observa ion

Prior ensemble

ETPF analysis

En ropic Reg. ETPF analysis



Observation outside the convex hull

0 1 2 3 4 5 6

Prior Inde

0

1

2

3

4

5

6

Posterior Inde

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Inde

−3

−2

−1

0

1

2

3

4

5ETKF weights

0 1 2 3 4 5 6

Prior Inde

0

1

2

3

4

5

6

Posterior Inde

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Inde

0

1

2

3

4

5

6ETPF weights

Observation

Prior ensemble

ETPF analysis

ETKF analysis



Likelihood splitting [Frei and Künsch, 2013]

πY(yobs|z) ∝exp�

−1

2(eyz)TR−1

eyz

�

, eyz = Hz − yobs

∝ exp�

α

�

−1

2(eyz)TR−1

eyz

��

× exp�

(1− α)

�

−1

2(eyz)TR−1

eyz

��

∝ exp

�

−1

2(eyz)T

�

R

α

�−1eyz

�

× exp

�

−1

2(eyz)T

�

R

1− α

�−1eyz

�

⇓ ⇓ETPF ESRF

Bridging parameter extrems:É α = 0 ⇒ Pure Kalman FilterÉ α = 1 ⇒ Pure Particle Filter


Hybrid Ensemble Transform Filter

Models explored:

É Single DA step

É Lorenz 63

É Lorenz 96

É Lorenz 96 coupled to a wave equation

É Shallow water equations (2D)

É Modified shallow water equations (1D)



0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETKF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETPF weights

Bridging Parameter = 1



0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETKF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETPF weights

Bridging Parameter = 0.9



0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETKF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETPF weights




0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETKF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETKF weights

0 1 2 3 4 5 6

Prior Index

0

1

2

3

4

5

6

Posterior Index

ETPF transform

−1.2

−0.9

−0.6

−0.3

0.0

0.3

0.6

0.9

1.2

0 1 2 3 4 5 6 7

Particle Index

−3

−2

−1

0

1

2

3

4

5ETPF weights

Bridging Parameter = 0



Bayesian Inference

for bimodal prior

and

Gaussian likelihood

-10 -5 0 5 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45Baysian inference

priorlikelihoodposterior



ETPF-ESRF

performance vs

ensemble size

101

102

103

RM

S e

rro

r

0

0.1

0.2

0.3

0.4ETPF-ESRF

ensemble size10

110

210

3

optim

alα

0

0.2

0.4

0.6

0.8

1


Fighting ensemble under-dispersion

Particle rejuvenation:

zaj→ za

j+

M∑

i=1

(zfi− zf)

βξijp

M− 1

É β: Rejuvenation parameterÉ {ξij}: i.i.d. Gaussian random variables with mean zero and

variance one

Properties:É Increase ensemble spread while preserving ensemble meanÉ Ameliorates particle degeneracyÉ Does not destroy spatial regularity


Example: Lorenz 63 model

Dynamical system:

x1 = 10(x2 − x1)

x2 = x1(28− x3)− x2

x3 = x1x2 − 8/3x3−20

−100 10

20−50

0

50

0

5

10

15

20

25

30

35

40

45

50

Perfect model DA experiments:É x1 observed every 12 time-stepsÉ Observation error variance R = 8



PF-KF RMSE with exact OT solver

PF-KF CRPS with exact OT solver


Update ordering dependency

Possible updating sequences:É ETPF-ESRF

zhj

=M∑

i=1

zfidPF

ij, dPF

ij:= dPF

ij(α,{zf

l},yobs),

zaj

=M∑

i=1

zhidKF

ij, dKF

ij:= dKF

ij(α,{zh

l},yobs).

É ESRF-ETPF

zhj

=M∑

i=1

zfidKF

ij, dKF

ij:= dKF

ij(α,{zf

l},yobs),

zaj

=M∑

i=1

zhidPF

ij, dPF

ij:= dPF

ij(α,{zh

l},yobs).


Updating order dependency


KF-PF RMSE with exact OT solver


OT solver dependency


PF-KF RMSE solving entropic regularized OT problem


OT solver dependency


PF-KF RMSE solving 1D OT problem


Localization

R-Localization

rLOCqq

(xk) =rqq

ρ�

‖xk−xq‖Rloc

� ,

with ρ a compactly supported tempering function.

É Directly applicable to Kalman FiltersÉ Directly applicable to importance weights

Particle distance Localization [Cheng and Reich, 2015]

cij(x) =

∫

R

ρ

�‖xk − xq‖

Cloc

�

‖zfi(xq)− zf

j(xq)‖2dxq



Dynamical system:

xj = (xj+1 − xj−2)xj−1 − xj + F,

xj = xj+N

where F = 8 and N = 40.

Perfect model DA experiments:É Odd variables observed every 22 time-stepsÉ Observation error variance R = 8É Particle rejuvenation β = 0.2É Localization radius is Rloc = 4É OTP solved using FastEMD algorithm



Skill dependence on

bridging parameter

for different

ensemble sizes

using fixed

likelihood splitting

bridging parameter α0 0.1 0.2 0.3 0.4 0.5 0.6

RM

S e

rror

1.55

1.6

1.65

1.7

1.75hybrid ETPF-LETKF

M=20M=25M=30


Adaptive likelihood splitting

Normalized effective sample size: θ(α) =1

M∑M

i=1 wi(α)2

θ vs α for different

obs. error levels

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

α

θ

R=1

R=2

R=4



Normalized effective sample size: θ(α) =1

M∑M

i=1 wi(α)2

θ vs α for different

obs. error levels

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

α

θ

R=1

R=2

R=4

θinf

Adaptivity criterion: Set α to the maximal value for which θ ≥ θinf .



Skill dependence on

minimum effective

sample size

for different

ensemble sizes

using adaptive

likelihood splitting

0.75 0.8 0.85 0.9 0.95 1

RM

S e

rror

1.55

1.6

1.65

1.7

1.75hybrid ETPF-LETKF

M=20M=25M=30

θinf



Skill dependence on

ensemble size for

both update orders

using optimal fixed

bridging parameter

10 20 30 40 50 60 701.3

1.4

1.5

1.6

1.7

1.8

1.9

2

Ensemble Size

Tim

e a

ve

rag

e R

MS

err

or

Hybrid ETKF−ETPF

hybrid ETPF−LETKF


Conclusions

É Proposed Hybrid scheme

allows consistent localization for both filters,

preserves model state regularity,

outperforms both ETKF and ETPF for a suitable α

É Approximated OT solution approaches are promising

cheaper options, in particular 1D approximation

É Update ordering sensitivity is model-dependent

É Adaptive likelihood splitting benefits spatially extended

systems.


Ongoing Research and Prospect

É ETPF and hybrid scheme being implemented for the

German Weather Service (DWD)

É Second order accurate Ensemble Transform Particle Filter

(under review)

É 1D OT approximation and then multivariate dependence

recovery via Ensemble copula coupling [Schefzik et al., 2013]

É Generation of unbalanced fields through localization

currently being addressed via mollification.


References I

Cheng, Y. and Reich, S. (2015).Nonlinear Data Assimilation, chapter Assimilating data into scientificmodels: An optimal coupling perspective, pages 75–118.

Chustagulprom, N., Reich, S., and Reinhardt, M. (2016).A hybrid ensemble transform particle filter for nonlinear and spatiallyextended dynamical systems.SIAM/ASA J. Uncertainty Quantification, 4(1):592–608.

Cuturi, M. (2013).Sinkhorn distances: Lightspeed computation of optimal transport.In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger,K., editors, Advances in Neural Information Processing Systems 26,pages 2292–2300. Curran Associates, Inc.

Frei, M. and Künsch, H. R. (2013).Bridging the ensemble kalman and particle filters.Biometrika.


References II

Lei, J. and Bickel, P. (2011).A moment matching ensemble filter for nonlinear non-gaussian dataassimilation.Mon. Wea. Rev., 139(12):3964–3973.

Pele, O. and Werman, M. (2009).Fast and robust earth mover’s distances.In ICCV.

Poterjoy, J. (2015).A localized particle filter for high-dimensional nonlinear systems.Mon. Wea. Rev., 144(1):59–76.

Reich, S. (2013).A non-parametric ensemble transform method for bayesian inference.SIAM J Sci Comput, 35:A2013–A2014.

Reich, S. and Cotter, C. (2015).Probabilistic Forecasting and Bayesian Data Assimilation.Cambridge University Press.


References III

Schefzik, R., Thorarinsdottir, T. L., and Gneiting, T. (2013).Uncertainty quantification in complex simulation models usingensemble copula coupling.pages 616–640.

Tippett, M. K., Anderson, J. L., Bishop, C. H., Hamill, T. M., andWhitaker, J. S. (2003).Ensemble square root filters.Monthly Weather Review, 131(7):1485–1490.


Thanks!


Nonlinear data assimilation via hybrid particle …...2017/07/08 · Nonlinear data assimilation...

Documents

Transcript of Nonlinear data assimilation via hybrid particle …...2017/07/08 · Nonlinear data assimilation...