Introduction to Numerical Analysis I

MATH/CMPSC 455

Conjugate Gradient Methods

A-ORTHOGONAL BASIS

form a basis of ,

where

is the i-th row of the identity matrix. They

are orthogonal in the following sense:

They are linearly independent, and form a basis.

Introduce a set of nonzero vectors ,

They satisfy the following condition:

We say they are A-orthogonal, or conjugate w.r.t A.

CONJUGATE DIRECTION METHOD

Theorem: For any initial guess, the sequence

generated by the above iterative method,

converges to the solution of the linear system

in at most n iterations. Question: How to find the A-orthogonal bases?

CONJUGATE GRADIENT METHOD

Answer:

Each conjugate direction is chosen to be a linear

combination of the residual and the previous

direction

Conjugate Gradient Method: Conjugate direction method on this particular basis.

CG (ORIGINAL VERSION)

While

End While

Theorem: Let A be a symmetric positive-

definite matrix. In the Conjugate Gradient

Method, we have

CG (PRACTICAL VERSION)

While

End While

Example: