Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning...
Transcript of Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning...
![Page 1: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/1.jpg)
Conditioning and Preconditioning of the Optimal State Estimation Problem
Nancy Nichols, Stephen Haben and Amos Lawless University of Reading
Met Office Website
![Page 2: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/2.jpg)
Outline
• Optimal State Estimation
• Conditioning
• Preconditioning
• Ill-posedness and Regularization
• Conclusions
![Page 3: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/3.jpg)
1.
Optimal State Estimation
![Page 4: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/4.jpg)
State Estimation / Data Assimilation
Aim: Find the optimal estimate (analysis) of the expected states of a system, consistent with both observations and the system dynamics given:
• Numerical prediction model (PDE)• Observations of the system (over time)• Background state (prior estimate)• Estimates of the error statistics
![Page 5: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/5.jpg)
State Estimation / Data Assimilation
![Page 6: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/6.jpg)
Optimal Bayesian Estimate
)][()][(
)()(21)(min
0
1
000
oiii
n
ii
Toiii
bT
b
HH
J
yxRyx
xxBxxx 1
−−+
−−=
∑=
−
−
subject to ),,( 00 xx ttS ii =
i
i
i
b
H
RB
yx - Background state (prior estimate)
- Observations- Observation operator- Background error covariance matrix- Observation error covariance matrix
![Page 7: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/7.jpg)
Significant Properties:
• Very large number of unknowns (107 – 108)• Few observations (105 – 106)• System nonlinear unstable/chaotic• Multi-scale dynamics
![Page 8: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/8.jpg)
Solution of the Problem• Solve iteratively by an approximate Gauss- Newton (incremental variational) method
• Solves a sequence of linear least-squares problems
• Use conjugate gradient/quasi-Newton iteration method to solve inner linear least- squares problem.
• Must solve in real time
![Page 9: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/9.jpg)
Accuracy/rate of convergence depend on the condition number = λmax / λmin of the Hessian:
Conditioning of the Problem
where
![Page 10: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/10.jpg)
Three Questions• What is the effect of the covariance structure on the conditioning of the Hessian
• What is the effect of preconditioning on the problem
• How does the observational system affect the conditioning of the problem
Reference: (HLN) Haben, Lawless and Nichols, UoR Maths Rpt 3/2009
U
![Page 11: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/11.jpg)
2.
Conditioning of the Hessian
![Page 12: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/12.jpg)
Minimizes with respect to initial state :
Optimal Bayesian Estimate – 3DVar
HHH
• Hessian is:
• B and R are covariance matrices that depend on the correlation length-scales of the errors; H is the Jacobian of .
![Page 13: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/13.jpg)
Minimizes with respect to initial state :
Optimal Bayesian Estimate – 3DVar
HHH
• Hessian is:
• B and R are covariance matrices that depend on the correlation lengthscales of the errors; H is the Jacobian of .
![Page 14: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/14.jpg)
Background Covariance MatrixDefine = σb
2 C on a 1D periodic domainby a Gaussian correlation structure:
![Page 15: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/15.jpg)
Circulant Matrices
![Page 16: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/16.jpg)
Condition of B - Gaussian
![Page 17: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/17.jpg)
Condition of Hessian
B = σb2 C , where C is a correlation matrix
![Page 18: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/18.jpg)
Condition Number of Hessian
where α =
and p = number of observations
Assume C has a circulant covariance structure.Bounds on the conditioning of the Hessian are:
α
![Page 19: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/19.jpg)
Conditioning of Hessian
Bij =
Condition Number of (B-1 + HR-1HT) vs Length Scale
Periodic Gaussian Exponential
Blue = condition number Red = bounds
![Page 20: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/20.jpg)
Conditioning of Hessian
Bij =
Condition Number of (B-1 + HR-1HT) vs Length Scale
Periodic Gaussian Exponential Laplacian 2nd Derivative
Blue = condition number Red = bounds
![Page 21: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/21.jpg)
3.
Preconditioning of the Hessian
![Page 22: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/22.jpg)
Control Variable Transform
• z = (x0 – x0b)
• Uncorrelated variable
• Equivalent to preconditioning by
• Hessian of transformed problem is
To improve conditioning transform to new variable :
![Page 23: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/23.jpg)
Preconditioned Hessian
where
= || HCHT ||
changes slowly as a function of length scale.
Bounds on the conditioning of the preconditioned Hessian are:
.,
![Page 24: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/24.jpg)
Condition number as a function of length scale
Preconditioned Hessian - Gaussian
Preconditioned (blue)Bounds (red)
![Page 25: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/25.jpg)
Preconditioned Hessian - Gaussian
Assume two observations at kth and mth grid points
Condition number decreases as the separationof the observations increases
![Page 26: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/26.jpg)
Preconditioned Hessian - Gaussian
Condition number as a function of observation spacing
![Page 27: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/27.jpg)
Extension to 4DVar
where
Convergence depends on condition number of
![Page 28: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/28.jpg)
Preconditioned Hessian
where
Bounds on the conditioning of the preconditioned Hessian are:
• B = σb2 C , C is correlation matrix
• •
![Page 29: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/29.jpg)
4D Background Correlation Matrix
![Page 30: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/30.jpg)
Condition of Preconditioned 4DVar – using SOAR Correlation Matrix
![Page 31: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/31.jpg)
Convergence Rates of CG in 4DVar – using SOAR Correlation Matrix
Haben et al, 2011
![Page 32: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/32.jpg)
4.
Regularization
![Page 33: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/33.jpg)
Preconditioned Hessian – 4DVar
Let
Then the preconditioned optimal state estimation problem may be written in the form of a classical Tikhonov regularized problem:
![Page 34: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/34.jpg)
Rewrite the 4D Var solution as:
where and , , are
the singular values and right and left singular vectors of .
=
Johnson et al, 2005a,b
![Page 35: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/35.jpg)
Johnson et al, 2005a,b
![Page 36: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/36.jpg)
Johnson et al, 2005a,b
![Page 37: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/37.jpg)
5.
Conclusions
![Page 38: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/38.jpg)
Conclusions • The condition number of the Hessian is sensitive to lengthscale
• Preconditioning with generally reduces the condition number and improves the convergence rates
• The condition number of the Hessian increases as the observations become more accurate and as their separation decreases.
![Page 39: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/39.jpg)
References S.A. Haben, A.S. Lawless and N.K. Nichols, Conditioning of incremental variational data assimilation, with application to the Met Office system, Tellus, 63A, 2011, 782 – 792.
S.A. Haben, A.S. Lawless and N.K. Nichols, Conditioning and preconditioning of the variational data assimilation problem, Computers & Fluids, 46, 2011, 252 – 256.
S.A. Haben, Conditioning and preconditioning of the minimisation problem in variational data assimilation, University of Reading, Dept of Mathematics, PhD Thesis, 2011.
C. Johnson, B.J. Hoskins and N.K. Nichols, A singular vector perspective of 4-DVar: Filtering and interpolation, Q. J. Roy. Met. Soc., 131, 2005, 1-20.
C. Johnson, N.K. Nichols and B.J. Hoskins, Very large inverse problems in atmosphere and ocean modelling, Internat. J. for Numer. Methods Fluids, 47, 2005, 759-771.
![Page 40: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/40.jpg)
Future
Many more challenges left!
![Page 41: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/41.jpg)
![Page 42: Conditioning and Preconditioning of the Optimal State ...€¦ · Conditioning and Preconditioning of the Optimal State Estimation Problem Nancy Nichols, Stephen Haben and Amos Lawless](https://reader031.fdocuments.in/reader031/viewer/2022020204/5b795da57f8b9a534c8d2dbc/html5/thumbnails/42.jpg)