Linear System Remark: (1) has a unique solution We want to solve the following linear system A is...

nnn RbRA ,

Linear System

Remark: (1) has a unique solution

We want to solve the following linear system

bAx

A is invertable

Remark: A is invertable det(A)=0

Remark: A is invertable eigenvaluean not is 0

Example: Solve:

15

11

25

6

4

3

2

1

x

x

x

x 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

nnn RbRA ,

Linear System


bAx

Remark: A is invertable Rank(A)=n

Remark: solutionexact theis 1* bAx

Linear System


bAx

These methods generate a sequence of approximate solutions

Gaussian EliminationLU, Choleski

2 classes of methods

Direct Methods

Iterative Methods

,,,, )3()2()1()0( xxxx

bAxx k 1*)( quickly How :method Good

Iterative Method

bAx These methods generate a sequence of approximate solutions

,,,, )3()2()1()0( xxxx

bAxx k 1*)( quickly How :method Good

,,,, )3()2()1()0( xxxxapprox

residual ,,,, )3()2()1()0( rrrr

)()( kk Axbr

Remark: solutionexact theis )(kx0)( kr

error ,,,, )3()2()1()0( eeee )(*)( kk xxe

Jacobi Method

Consider 4x4 case

15

11

25

6

4

3

2

1

x

x

x

x

0

0

0

0

given )0(x

10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

158 3

11 10 2

253 11

6 2 10

432

4321

4321

321

xxx

xxxx

xxxx

xxx

)8/()15 3 (

10/)11 2(

11/)253 (

10/)6 2 (

)0(3

)0(2

)1(4

)0(4

)0(2

)0(1

)1(3

)0(4

)0(3

)0(1

)1(2

)0(3

)0(2

)1(1

xxx

xxxx

xxxx

xxx

10/)6 2 ( 321 xxx

)8/()15 3 (

10/)11 2(

11/)253 (

324

4213

4312

xxx

xxxx

xxxx

Example

8750.1

1000.1

2727.2

6000.0

)1(x

K=6 K=7 K=8 K=9 K=10x1

x2

x3

X4

K=1 K=2 K=3 K=4 K=5x1

x2

x3

X4

Jacobi Method

0.6000 1.0473 0.9326 1.0152 0.9890 2.2727 1.7159 2.0533 1.9537 2.0114 -1.1000 -0.8052 -1.0493 -0.9681 -1.0103 1.8750 0.8852 1.1309 0.9738 1.0214

1.0032 0.9981 1.0006 0.9997 1.0001 1.9922 2.0023 1.9987 2.0004 1.9998-0.9945 -1.0020 -0.9990 -1.0004 -0.9998 0.9944 1.0036 0.9989 1.0006 0.9998

11.3537 4.9910 2.0299 0.8911 0.3686)(kr

0.1605 0.0671 0.0290 0.0122 0.0053)(kr

)8/()15 3 (

10/)11 2(

11/)253 (

10/)6 2 (

)(3

)(2

)1(4

)(4

)(2

)(1

)1(3

)(4

)(3

)(1

)1(2

)(3

)(2

)1(1

kkk

kkkk

kkkk

kkk

xxx

xxxx

xxxx

xxx

1

1

2

1

*x

K=1 K=2 K=3 K=4 K=5x1

x2

x3

X4

Gauss Seidel Method

)(kr

0.6000 1.0302 1.0066 1.0009 1.0001 2.3273 2.0369 2.0036 2.0003 2.0000 -0.9873 -1.0145 -1.0025 -1.0003 -1.0000 0.8789 0.9843 0.9984 0.9998 1.0000

5.6930 0.4300 0.0662 0.0082 0.0009

1

1

2

1

*x

)8/()15 3 (

10/)11 2(

11/)253 (

10/)6 2 (

)1(3

)1(2

)1(4

)(4

)1(2

)1(1

)1(3

)(4

)(3

)1(1

)1(2

)(3

)(2

)1(1

kkk

kkkk

kkkk

kkk

xxx

xxxx

xxxx

xxx

Note that in the Jacobi iteration one does not use the most recently available information.

)8/()15 3 (

10/)11 2(

11/)253 (

10/)6 2 (

)(3

)(2

)1(4

)(4

)(2

)(1

)1(3

)(4

)(3

)(1

)1(2

)(3

)(2

)1(1

kkk

kkkk

kkkk

kkk

xxx

xxxx

xxxx

xxx

Gauss-Seidel iteration for general n:

Gauss Seidel Method

Jacobi iteration for general n:

end

axaxabxii

n

ij

kjij

i

j

kjiji

ki

1

)(1

1

)()1(

n:1ifor

end

axaxabxii

n

ij

kjij

i

j

kjiji

ki

1

)(1

1

)1()1(

n:1ifor

nnnnn

n

b

b

x

x

aa

aa

11

1

111

DEF: nnRALet with eigenvalues n ,1

Splittings and Convergence

,,, max)( 21 nA

DEF: NMA

spectral radius of A is defined to be

Splitting bxNM bAx

bNxMx A large family of iteration

bNxMx kk ˆ)(1)1(


A large family of iteration

bNxMx kk ˆ)(1)1(

U L D A

23

1312

3231

21

33

22

11

333231

232221

131211

a

aa

aa

a

a

a

a

aaa

aaa

aaa

Diagonal Lower Upper

Jacobi:

)( ULN

DM

Gauss-Seidel:

UN

LDM


THM:

11 N)ρ(M - Convg


bNxMx kk )(1)1(


Example:

15

11

25

6

4

3

2

1

x

x

x

x 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

10 11 10 8

-1 2 -1 0 3 -1

-1 2 0 -1 3 -1

Jacobi: )( , ULNDM

0.4264)( 1 JJ NM

U=triu(A,1)L=tril(A,-1)D=diag(diag(A))eig(inv(M)*N)-0.4264 -0.1040 0.1860 0.3445GS: UNLDM ,

0.0898)( 1 gsgs NM 0 0 0.0839 - 0.0322i 0.0839 + 0.0322i


THM:

11 N)ρ(M - Convg


bNxMx kk ˆ)(1)1(

Proof: (Golub p511)

Remarks:

iAρ max)( )1 )(A )2

2AAρ T

sym)A (if )(A )32

Aρ

1)( iff 0 )4 AρAk


THM:

definite positive

symmetric )0(any for

x

ConvergesGS

Proof: (Golub p512) show that all eigenvalues are less than one.

A

Example:


DEF:

n

ijj

ijii aa1

idominant diagonallystrictly A IF

31

131

131

13

THM:

)0(any for

Jacobi and

x

Converges

GSAdominant

diagonallystrictly

Successive over Relaxation

The Gauss-Seidel iteration is very attractive because of its simplicity. Unfortunately, if the spectral radius is close to one, then convergence is vey slow. One solution for this

UDNLDM )1( ,


NM minimizes that Find 1

Jacobi: 0 GS: 1


UDN

LDM

)1(

,

Successive over Relaxation Example:

15

11

25

6

4

3

2

1

x

x

x

x 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

03.1

K=1 K=2 K=3x1

x2

x3

X4


UDN

LDM

)1(

,


Example:

15

11

25

6

4

3

2

1

x

x

x

x 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 803.1

0.6180 1.0553 1.0055 2.3988 2.0273 2.0004-1.0132 -1.0211 -1.0014 0.8743 0.9905 1.0000

1

1

2

1

*x

MATLAB CODE

Jacobi iteration for general n:

end

axaxabxii

n

ij

kjij

i

j

kjiji

ki

1

)(1

1

)()1(

n:1ifor

Ex:Write a Matlab function for Jacobi

function [sol,X]=jacobi(A,b,x0)n=length(b);maxiter=10;x=x0;for k=1:maxiterfor i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end xnew(i)=(b(i)-sum1-sum2)/A(i,i)endX(1:n,k)=xnew;x=xnew;endsol=xnew;

MATLAB CODE

GS iteration for general n:Ex:Write a Matlab function for GS

function [sol,X]=gs(A,b,x0)n=length(b);maxiter=10;x=x0;for k=1:maxiterfor i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end x(i)=(b(i)-sum1-sum2)/A(i,i)endX(1:n,k)=x;endsol=x;

end

axaxabxii

n

ij

kjij

i

j

kjiji

ki

1

)(1

1

)1()1(

n:1ifor

Another Look

Remark:

Given:

We want to improve this approximate:

)0(x

bAx

)( )0(*)0()1( xxxx

)0()0()1( exx

)( )0(1)0()1( xbAxx

)( )0(1)0()1( AxbAxx

)0(1)0()1( rAxx

)0(1)0()1( rGxx

)0(1)0()1( rAxx

11 AGwhere

)(1)()1( kkk rGxx

Jacobi: DG

GS: LDG

Examples of Splittings

1) Non-symmetric Matrix:

22

TT AAAAA

2) Domain Decomposition:

1

2

PCC

B

BA

r

r

1

11

symmetric Skew-symmetric

Linear System Remark: (1) has a unique solution We want to solve the following linear system A is...

Documents

Transcript of Linear System Remark: (1) has a unique solution We want to solve the following linear system A is...