Numerical methodNumerical method 11

Iterative approaches for solving linear Iterative approaches for solving linear systemssystems

Successive over-relaxationSuccessive over-relaxationPost-nonlinear systemPost-nonlinear systemPartial pivotingPartial pivoting

Lecture 11-IILecture 11-II

Numerical methodNumerical method 22

Derivation ofDerivation of Successive Successive over-relaxationover-relaxation

Numerical methodNumerical method 33

Successive over-relaxationSuccessive over-relaxation methodmethod

D=diag(diag(A));U=triu(A)-D;L=A-triu(A);R=D+w*L;T=inv(R)*((1-w)*D-w*U);c=w*inv(R)*b;

wbxwUDwwLDx ))1(()( 1

wbxwUDwxwLD ))1(()(

Numerical methodNumerical method 44

Successive over-relaxationSuccessive over-relaxation methodmethod

A,b it_xor.mx

A=[10 -1 2 0;-1 11 -1 3; 2 -1 10 -1;0 3 -1 8];b=[6 25 -11 15]';inv(A)*b

Numerical methodNumerical method 55

>> it_xor(A,b) It takes 17 iterations to converge

ans =

1.00000000300255 2.00000000357295 -1.00000000163021 0.99999999724547

Numerical methodNumerical method 66

VariantVariant successive over-successive over-relaxationrelaxation

bxUwDxLDwD )()(

Numerical methodNumerical method 77

Variant Successive over-Variant Successive over-relaxationrelaxation

D=diag(diag(A));U=triu(A)-D;L=A-triu(A);R=w*D+D+L;T=inv(R)*(w*D-U);c=inv(R)*b;

bxUwDxLDwD )()(

bLDwDc

UwDLDwD1

1

)(

)()(T

Numerical methodNumerical method 88

Variant Successive over-Variant Successive over-relaxationrelaxation method method

A,b it_vxor.mx

A=[10 -1 2 0;-1 11 -1 3; 2 -1 10 -1;0 3 -1 8];b=[6 25 -11 15]';inv(A)*b

Numerical methodNumerical method 99

>> it_vxor(A,b,w) It takes 17 iterations to converge

ans =

1.00000000300255 2.00000000357295 -1.00000000163021 0.99999999724547

Numerical methodNumerical method 1010

w=0w=0

>> it_vxor(A,b,0) It takes 10 iterations to converge

ans =

0.99999999998681 1.99999999985957 -0.99999999997639 1.00000000005561

Numerical methodNumerical method 1111

Large wLarge w

>> it_vxor(A,b,3) It takes 90 iterations to converge

ans =

0.99999998371358 1.99999997234472 -0.99999998858309 1.00000003116601

Numerical methodNumerical method 1212

Post-Nonlinear SystemPost-Nonlinear System

Ax=b, Ax=b, A=rand(100,100),b=rand(100,1)A=rand(100,100),b=rand(100,1)

f(Ax)=b, where f is an arbitrary one-f(Ax)=b, where f is an arbitrary one-dimensional nonlinear functiondimensional nonlinear function

Given A,b Given A,b

find f and xfind f and x

Numerical methodNumerical method 1313

f(Ax)=bf(Ax)=b

Multiple solutionsMultiple solutionsx

fmean absolute error: 0.017080mean square error: 0.000755

mean(abs(b-f(Ax)))

Numerical methodNumerical method 1414

f(Ax)=bf(Ax)=b

Multiple solutionsMultiple solutionsx f

mean absolute error: 0.024637mean square error: 0.002738

Numerical methodNumerical method 1515

Matrix inversionMatrix inversion

Solve Gx=fSolve Gx=f ExampleExample

Gf.zip

>> load Gf.mat;>> inv(G);Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 4.206041e-017.

Numerical methodNumerical method 1616

rankrank

ss=cputime;for i=1:1000rank(G);endss2=cputime;fprintf('cputime: rank %f \n',ss2-ss);cputime: rank 1.703125

Numerical methodNumerical method 1717

rankrank

ss=cputime;for i=1:50000rank(G);endss2=cputime;fprintf('cputime: rank %f \n',ss2-ss);cputime: rank 82.546875

Numerical methodNumerical method 1818

pinvpinv

ss=cputime;for i=1:1000pinv(G);endss2=cputime;fprintf('cputime: pinv %f \n',ss2-ss);

cputime: pinv 7.062500

Numerical methodNumerical method 1919

Inverse of submatrixInverse of submatrix

ss=cputime;for i=1:50000inv(G(1:61,1:61));endss2=cputime;fprintf('cputime: pinv %f \n',ss2-ss);cputime: inv of submatrix 28.937500

Time reduced from 50*7.06 to 28.93 sec

Numerical methodNumerical method 2020

Partial pivoting Partial pivoting

Maximal column pivotingMaximal column pivoting

78.46130.629.5 :

17.5917.5916.^10:

212

211

xxE

xxE

The pivot or pivot element is the element of a matrix, which is selected first by an algorithm (e.g. Gaussian elimination, Quicksort, Simplex algorithm), to do certain calculations with the matrix

Numerical methodNumerical method 2121

A=[10.^-16 59.17;5.29 -6.13];b=[59.17 46.78]';B=[A b];

fGauss.m

backward.m

B

0.0 1.0000

ans =

10.0019 1.0000

B=fGauss(B)

x=backward(B)

Numerical methodNumerical method 2222

B =

10.^-16 59.1400 59.1700 5.2900 -6.1300 46.7800

FindExchange row 1 and row 2

2900.5},max{ 2111 bb

temp=B(1,:);B(1,:)=B(2,:);B(2,:)=temp;

Numerical methodNumerical method 2323

U=fGauss(B);x=backward(U)

x =

10.0019 1.0000

B =

5.2900 -6.1300 46.7800 10.^-16 59.1400 59.1700

Numerical methodNumerical method 2424

78.4613.6291.5

59170059140030

21

21

xx

xx

A=[30.00 591400;5.29 -6.13];b=[591700 46.78]';B=[A b];

Numerical methodNumerical method 2525

Scaled partial pivotingScaled partial pivoting

kjj

k as max

k

ki

ik s

a

maxargp where

pivotith theas pselect

Exchange row i and row p

Numerical methodNumerical method 2626

A =

2.1100 -4.2100 0.9210 4.0100 10.2000 -1.1200 1.0900 0.9870 0.8320

Determine the first pivot by[v p]=max(A(:,1)./s);

>> s=max(abs(A'))'

s =

4.2100 10.2000 1.0900

p =

3

Numerical methodNumerical method 2727

Scaled partial pivotingScaled partial pivoting

Function [B,pr]=pss(A,b)Function [B,pr]=pss(A,b)n=length(b);B=[A b]; pr=[]; n=length(b);B=[A b]; pr=[]; s=max(abs(A'))' Pr=1:n;Pr=1:n;for i= 1:nfor i= 1:n

A.A. Find the ith pivot row and set it to pFind the ith pivot row and set it to pB.B. Exchange pr(i) and pr(p)Exchange pr(i) and pr(p)C.C. Exchange row i and row p of matrix BExchange row i and row p of matrix BD.D. For each row j > iFor each row j > i

1)1) Set d to the ratio B(j,i) / B(i,i)Set d to the ratio B(j,i) / B(i,i)2)2) Set B(j,:) to B(j,:)-d*B(i,:)Set B(j,:) to B(j,:)-d*B(i,:)

Return B and prReturn B and pr

Numerical methodNumerical method 2828

ExampleExampleA=[2.11 -4.21 .921;4.01 10.2 -1.12;1.09 .987 .832];b=[2.01 -3.09 4.21]';

spp.m[B,pr]=spp(A,b)

B =

1.0900 0.9870 0.8320 4.2100 0 -6.1206 -0.6896 -6.1396 0 0.0000 -4.9209 -25.1675

pr =

3 1 2

Numerical methodNumerical method 2929

ExampleExampleA=[1.19 2.11 -100 1;14.2 -0.122 12.2 -1;0 100 -99.9 1; 15.3 0.110 -13.1 -1];b=[1.12 3.44 2.15 4.16]';

[B,pr]=spp(A,b);x=backward(B);

x =

0.1768 0.0127 -0.0207 -1.1826

Numerical methodNumerical method 3030

ExerciseExercise

Implement the scaled partial pivoting Implement the scaled partial pivoting method to improve the forward method to improve the forward Gauss elimination method.Gauss elimination method.

Verify your matlab codes by two Verify your matlab codes by two examplesexamples

Numerical methodNumerical method 3131

Avoid row swappingAvoid row swapping

Row swapping results in time Row swapping results in time consumingconsuming

No row swappingNo row swapping Use a vector v to emulate row swappingUse a vector v to emulate row swapping Swap elements in v instead of swapping Swap elements in v instead of swapping

rows in Brows in B How to revise spp.m to avoid row How to revise spp.m to avoid row

swapping ? swapping ?