Vorobyev's Method of Moments in Applied Mathematics - Part ... · Krylov Subspace Methods,...

Vorobyev’s Method of Moments in AppliedMathematics - Part I (Self-adjoint Operators)

MORE seminar, May 4, 2015

MORE seminar Jan Papež Vorobyev: Method of Moments

Yu. V. Vorobyev: Method ofMoments in Applied Math-ematics; Translated fromthe Russian original (1958)by Bernard Seckler; Gordonand Breach Science Publish-ers, New York, 1965.

Outline

Part I introduction, self-adjoint operatorsPart II generalizations, classificationPart III moment problem for completely continuous (compact)

operatorsPart IV partial realization, examples, mathematical physics,

miscellaneous, . . .

Other references

J. Liesen, Z. Strakoš:Krylov Subspace Methods, Principles and Analysis, Chapter 3.Oxford University Press, (2013).

J. Málek and Z. Strakoš:Preconditioning and the Conjugate Gradient Method in theContext of Solving PDEs.SIAM Spotlight Series, SIAM, Philadelphia, (2015).


Outline of Part I

1 Problem of moments in Hilbert space

2 Method of moments for self-adjoint operators


I. Preliminaries

• We consider a (complete) Hilbert space H with the innerproduct (·, ·) and the induced norm ‖ · ‖ ;

• operators act on H , i.e.,

A,An : H −→ H ;

• (strong) convergence of operators in H, An → A means

limn→∞

‖Anx − Ax‖ = 0 for all x ∈ H ;

• for a subspace Hn ⊂ H , En denotes the orthogonalprojection onto Hn .


I. Problem of moments in Hilbert space

Let z0, z1, . . . , zn be n + 1 linearly independent elements of Hilbertspace H . Consider the subspace Hn generated by all possiblelinear combinations of z0, z1, . . . , zn−1 and construct a linearoperator An defined on Hn such that

z1 = Anz0,

z2 = Anz1,...

zn−1 = Anzn−2,

Enzn = Anzn−1,

where Enzn is the projection of zn onto Hn .


I. Eigenvalues of operator An

Since Enzn is an element of Hn , we can find numbersα0, α1, . . . , αn−1 such that

Enzn = −α0z0 − α1z1 − · · · − αn−1zn−1 .

Using the definition of An ,

Pn(An)z0 := (Ann + αn−1A

n−1n + · · ·+ α0I ) z0 = 0 .

It can be shown that the eigenvalues of An are the roots of the(monic) polynomial

Pn(λ) = λn + αn−1λn−1 + · · ·+ α0 .


I. Approximation of bounded linear operators

Let A be a bounded linear operator in Hilbert space H .Choosing an element z0 , we first form a sequence of elementsz1, . . . , zn, . . .

z0, z1 = Az0, z2 = Az1 = A2z0, . . . , zn = Azn−1 = Anz0, . . .

For the present z1, . . . , zn are assumed to be linearly independent.By solving the moment problem we determine a sequence ofoperators An defined on the sequence of nested subspaces Hn

such thatz1 = Az0 = Anz0,

z2 = A2z0 = (An)2z0,...

zn−1 = An−1z0 = (An)n−1z0,Enzn = EnA

nz0 = (An)nz0.


I. Approximation of bounded linear operators

Using the projection En onto Hn we can write for the operatorsconstructed above (here we need the linearity of A )

An = En AEn .

The orthogonality of En gives

‖An‖ ≤ ‖A‖ .

Matching moments propertyThere holds

(A`nz0, z0) = (A`z0, z0) , ` = 0, 1, . . . , n − 1, n .


I. Convergence An → A

Given z0 , consider the subspace Hz as the closure, w.r.t. thenorm ‖ · ‖ , of the linear manifold

{x ∈ H | x = Q(A)z0 , Q(λ) is an arbitrary polynomial}

Then A can be considered to be an operator on Hz .

Theorem II, p. 21If A is a bounded linear operator and An a sequence of solutionsof the problem of moments, then the sequence An convergesstrongly to A in the subspace Hz .


I. Comments

Linear (in)dependence of z0, z1, . . . , zn, . . .

If zn is linearly dependent on z0, z1, . . . , zn−1 , then Hz = Hn

and An coincides with A on Hz .

Krylov subspacesThe choice of the elements z0, z1, . . . , zn, . . . as above givesKrylov subspaces that are closely connected with the application(described, e.g. by partial differential equations).

Method of moments falls within the general framework ofthe Galerkin’s methodThe approximate operator

An = En AEn .


Section 2

Method of moments for self-adjointoperators


II. Self-adjoint operators

Let A be a bounded self-adjoint operator, i.e.

(Ax , y) = (x ,Ay) for all x , y ∈ H .

Consequently (Ax , x) is a real number and the eigenvalues of Aare also real.Bounds of a self-adjoint operator

m (x , x) ≤ (Ax , x) ≤ M (x , x) for all x ∈ H .


II. Matching moments property for self-adjointoperators

The problem of moments for a self-adjoint operator A and anelement z0 yields the sequence of (nested) subspaces Hn and theself-adjoint operators An = EnAEn that matches the first 2nmoments of the operator A , i.e.

(A`nz0, z0) = (A`z0, z0) , ` = 0, 1, . . . , n, . . . , 2n − 1 .


II. Outline

• spectral representation of self-adjoint operators

• Stieltjes’ moment problem

• Gauss-Christoffel quadrature

• determining the spectrum of An

• orthogonal polynomials

• Lanczos algorithm, Conjugate Gradient Method


II. Finite-dimensional self-adjoint operator

Consider a self-adjoint operator An in some n-dimensional spaceHn and assume for the moment that it has distinct eigenvalues

λ1 < λ2 < . . . < λn .

The corresponding eigenelements (eigenvectors) uk are mutuallyorthogonal, i.e.

(ui , uk) = 0 i 6= k .

We consider eigenelements uk normalized with ‖uk‖ = 1 , so that{uk} form an orthonormal basis of Hn and

x = (x , u1) u1 + (x , u2) u2 + . . .+ (x , un) un , x ∈ Hn


II. Spectral function, n-dimensional case

The spectral function E(n)λ of An is the family of projections

E(n)λ x = 0 λ < λ1

E(n)λ x = (x , u1) u1 λ1 ≤ λ < λ2· · ·E(n)λ x = (x , u1) u1 + . . .+ (x , un−1) un−1 λn−1 ≤ λ < λn

E(n)λ x = x λn ≤ λ

Denoting the value of the jump by ∆kE(n)λ x ≡ (x , uk) uk ,

k = 1, . . . , n, we can write

Anx = λ1(x , u1) u1 + λ2(x , u2) u2 + . . .+ λn(x , un) un

=n∑

k=1

λk∆kE(n)λ x


II. Spectral function, general case

The spectral representation for any bounded self-adjoint operator A

Ax =

∫ M

mλdEλx ,

where the spectral function Eλ of A represents a family oforthogonal projections which is

• non-decreasing, i.e., if µ > ν , then the subspace onto whichEµ projects contains the subspace into which Eν projects;

• Em = 0, EM = I ;• Eλ is right continuous, i.e. lim

λ′→λ+Eλ′ = Eλ .

The values of λ where Eλ increases by jumps represent theeigenvalues of A .


II. Convergence of spectral functions

Let An be the solution of problem of moments for A, z0 and E(n)λ

be the spectral function of An .

As a result of (strong) convergence An → A in Hz , we have

Theorem IX, p. 61If A is a bounded self-adjoint operator and An a sequence ofsolutions of the problem of moments, then the sequence of spectralfunctions E(n)λ of the operators An converges strongly to thespectral function of A

E(n)λ → Eλin the subspace Hz for all λ not belonging to the discrete spectrumof A .


II. Distribution function

The spectral function Eλ and z0 determine the real function(Eλz0, z0) that satisfies• (Eλz0, z0) = 0 for λ < m,• (Eλz0, z0) = ‖z0‖2 for M ≤ λ,• (Eλz0, z0) is non-decreasing .

The function (Eλz0, z0) can be considered as a distribution functionfor a Riemann-Stieltjes integral.

Hereafter, we will for simplicity of notation consider normalizeddistribution functions, i.e., we consider

‖z0‖ = 1 .


II. Simplified Stieltjes’ moment problem

Given a distribution function ω(λ) on [m,M] , find a distributionfunction ω(n)(λ) , with n distinct points of increase in [m,M] ,such that the first 2n moments are matched, i.e. such that∫ M

mλ`dω(λ) =

∫ M

mλ`dω(n)(λ) , ` = 0, 1, . . . , 2n − 1 .

Equivalent formulation: find an n-node (mechanical) quadraturewith positive weights for the given Riemann-Stieltjes integral withdistribution function ω(λ) , that has algebraic degree of exactness2n − 1 , i.e. that is exact for polynomials of degree at most 2n − 1 .

[Liesen & Strakoš, §3.1]


II. Solution of Stieltjes’ moment problem

Gauss-Christoffel quadratureThe n-node (interpolatory) quadrature that has algebraic degree ofexactness 2n − 1 .The (distinct) quadrature nodes are given as the roots of the n-thmonic orthogonal polynomial w.r.t. the Riemann-Stieltjes integral∫ M

mf (λ)dω(λ) .

[Liesen & Strakoš, §3.2]Properties of orthogonal polynomials and the relation to continuedfractions: [Liesen & Strakoš, §3.3]


II. Orthogonal basis of Hn

Using Lanczos method of successive orthogonalization we constructan orthogonal basis of the subspace Hn = span{z0, z1, . . . , zn−1} :

p0 = z0 ,

p1 = Ap0 − a0p0 , a0 is such that (p0, p1) = 0p2 = Ap1 − a1p1 − b0p0 ,

. . .

pk+1 = (A− ak I ) pk − bk−1pk−1 .

Operator A is self-adjoint ⇒ three-term recurrence .

We can writepk = Pk(A) z0 ,

where Pk(λ) is a monic polynomial of degree k .


II. Orthogonal basis of Hn

Since An is the solution of problem of moments, we have

Pn(An) z0 = EnPn(A) z0 = Enpn .

From the orthogonality of pn to each of the elements p0, . . . , pn−1 ,

Enpn = 0 .

The spectrum of An is equal to the roots of Pn(λ) .


II. Orthogonal polynomials

Using the spectral function Eλ of the operator A ,

pk = Pk(A) z0 =

∫ M

mPk(λ)dEλz0 .

The orthogonality of the elements pk gives (for i 6= k)

0 = (pi , pk) =

∫ M

mPi (λ)Pk(λ) d(Eλz0, z0) ,

Therefore the (monic) polynomials Pk(λ) are orthogonal overinterval [m,M] w.r.t. the distribution function (Eλz0, z0) .

The roots of the polynomial Pn(λ) are the nodes of theGauss-Christoffel quadrature for solving the Stieltjes momentproblem with distribution function (Eλz0, z0) .


II. Lanczos algorithm

Denote symbolically Qn = (q1, . . . , qn) , where the columnsq1, . . . , qn form an orthonormal basis of Hn determined by theLanczos process

AQn = QnTn + δn+1qn+1eTn ,

with q1 = z0 (we assume ‖z0‖ = 1 ) . We get

A`n = QnT`nQ∗n , ` = 0, 1, . . .

and the moment matching property reads

eT1 T`ne1 = q∗1A

`q1 = (A`z0, z0) , ` = 0, 1, . . . , 2n − 1 .

[Liesen & Strakoš, §3.5, §3.7]


II. Lanczos algorithm, Jacobi matrix

The matrix Tn of the orthogonalization coefficients has the form

Tn =

γ1 δ2

δ2. . . . . .. . . . . . . . .

. . . . . . δnδn γn

,

where δj > 0 . Eigenvalues of Tn are equal to the eigenvalues ofAn .

Properties of Jacobi matrices: [Liesen & Strakoš, §3.4]


II. Conjugate Gradient Method

Let A be self-adjoint positive definite and the right-hand side f anelement of Hz . We are solving

Ax = f , i.e. x = A−1f =

∫ M

mλ−1 dEλf .

Approximation using the spectral function E(n)λ

xn =

∫ M

mλ−1 dE(n)λ f =

∫ M

mλ−1 dE(n)λ fn = A−1

n fn ,

where fn denotes the projection of f on Hn . Since all theeigenvalues of An lie in the interval [m,M] with m > 0 , theapproximation xn is well-defined for any n .


II. Conjugate Gradient Method

The approximation xn = A−1n fn can be computed by the Conjugate

Gradient Method (CG) by [Hestenes & Stiefel] .

The matrix representation of CG using the Jacobi matrix Tn andthe orthonormal basis Qn of Hn is

Tn yn = ‖f ‖ e1 , xn = Qnyn ∈ Hn .

Provided that f ∈ Hz , convergence An → A in Hz givesxn → x .


II. Model reductions matching 2n moments

Given a self-adjoint operator A and z0 ∈ H :

Vorobyev’s problem of momentsA : Hz −→ Hz , An : Hn −→ Hn , dim(Hn) = n .

Gauss-Christoffel quadraturefrom the distribution function ω(λ) to the distribution functionω(n)(λ) with n points of increase.

Lanczos algorithmfrom A, z0 to Tn, e1 , Tn ∈ Rn×n .

Conjugate Gradient Methodfrom Ax = f to Tn yn = ‖f ‖ e1 (for positive definite self-adjointoperator A ).


Questions for the next talks

• Can we match more than n + 1 moments if A is notself-adjoint?

• Can we consider an oblique (i.e. not-orthogonal) projection En ?• Do eigenvalues of An converge to eigenvalues of A ?• Can we generalize Gauss-Christoffel quadrature for non positivedefinite linear functionals?

• . . .


Thank you for your attention!


Vorobyev's Method of Moments in Applied Mathematics - Part ... · Krylov Subspace Methods,...

Documents

Transcript of Vorobyev's Method of Moments in Applied Mathematics - Part ... · Krylov Subspace Methods,...