DEF: nnCALet )det()( xAIxp Characteristic Polynomial (of degree n)

QR - Algorithm

Note: 1) QR – Algorithm is different from QR-Decomposition

2) a procedure to calculate the eigenvalues of a matrix

The eigenvalues of A are the n roots of its characteristic polynomial

123 The set of these roots is called the spectrum of A and

denoted by },,{)( 1 nA

for i=1:50 [Q,R]=qr(A); A=R*Q;end

QR Algorithm:(Basic version)

Find QR factorization

Multiply in the reverse order

DEF: nnCALet

)det()( xAIxp Characteristic Polynomial (of degree n)

Eigenvalues & Eigenvectors

The eigenvalues of A are the n roots of its characteristic polynomial

123 The set of these roots is called the spectrum of A and

denoted by },,{)( 1 nA

4 nA 21)det(

5 nAtrace 21)(

6 If A is symmetric then all eigenvalues are reals

DEF: nnCALet the nonzero vectors x that satisfy)( If A

Eigenvalues & Eigenvectors

xAx are called eigenvectors.

DEF:

similar

,BA Xmatrix r nonsingula AXXB 1

ation transformsimilarity called is X

DEF:

A similar to

y orthogonal B Qmatrix orthogonal AQQB T

THM: similar ,BA )()( BA same eigenvalues

Special Matrices

nna

a

a

22

11

Diagonal

nn

n

ac

b

ac

ba

1

22

11

Tridiagonal

nn

n

a

a

aa

22

111

Upper triangular

nnnn

n

aa

aa

aa

1,

2221

111

Upper Hessenberg

n

n

a

b

a

ba

1

2

11

Upper bidiagonal

Schur Factorization

Schur Factorization:

TAQQ *

nnCA

x

xx

)(unitary * IQQQ

Schur Factorization:

TAQQT

x

x

orthogonal Qsymmetric If nnRA

1) A and T are similar2) They have same eigenvalues3) Eigenvalues of T are the diagonal entries

Example:Compute all eigenvalues of A:

Schur Factorization

TAQQ *

x

xx

IDEA: We need to construct the Schur factorization

for i=1:50 [Q,R]=qr(A); A=R*Q;end

QR Algorithm:(Basic version)

>> eig(A)‘-100.0111 -10.0988 -0.8901

110

1101

01100

A

A5 =

-100.0111 0.0000 0.0000 0.0000 -10.0988 0.0000 0 0.0000 -0.8901

A3 =

-100.0111 0.0010 0.0000 0.0010 -10.0988 0.0007 0 0.0007 -0.8901

All A’s are similarCompute all eigenvalues of A:

Schur Factorization

TAQQ *

x

xx

IDEA: We need to construct the Schur factorization

for i=1:50 [Q,R]=qr(A); A=R*Q;end

QR Algorithm:(Basic version)

AA 1

111 RQA

112 QRA

222 RQA

112 QRA

11*12 )( QAQA

iiii QAQA *

1

All A’s have same eigenvalues

ngularupper triakA

Compute all eigenvalues of A:

Schur Factorization

TAQQ *

x

xx

IDEA: We need to construct the Schur factorization

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxx

xxxx

0

0

0

xx

xx

xxx

xxxx

00

00

0

x

xx

xxx

xxxx

000

00

0

AHHH 123AHH 12AH1A

QR

-fac

tori

zati

on

QAAHHH 123

Idea

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxx

xxxx

0

0

0

xx

xx

xxx

xxxx

00

00

0

x

xx

xxx

xxxx

000

00

0

AHHH 123AHH 12AH1A

QR

-fac

tori

zati

on

QAAHHH 123

TAQQT

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxx

xxxx

0

0

0

xx

xx

xxx

xxxx

00

00

0

*2

*112 HAHHH*

11AHHAH1A

Bad

Idea

All zeros are destroyed

xxx

xxx

xxx

xxxx

0

0

0

Idea TAQQT

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxx

xxxx

0

0

0

xx

xx

xxx

xxxx

00

00

0

*2

*112 HAHHH*

11AHHAH1AB

ad Id

ea

xxx

xxx

xxx

xxxx

0

0

0

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxxx

xxxx

0

0

*11AQQAQ1A

Go

od

Idea

xxx

xxx

xxxx

xxxx

0

0

zeros are not destroyed

):2,:2(

):2,1()1,1(

11

11

nnAe

nAA

111

11

ˆ0

01

):2,:2(

):2,1()1,1(

QnnAe

nAA

AQA 11 *11AQQ

111

111

ˆ):2,:2(

ˆ):2,1()1,1(

QnnAe

QnAA

Idea TAQQT

xxxx

xxxx

xxxx

xxxx

xxx

xxx

xxxx

xxxx

0

0

*11AQQAQ1A

Go

od

Idea

xxx

xxx

xxxx

xxxx

0

0

*112 AQQQ

xx

xxx

xxxx

xxxx

00

0

*2

*112 QAQQQ

AQQ*

xx

xxx

xxxx

xxxx

00

0

It is not upper triangular

HessenbergGivens

Idea TAQQT

xxxx

xxxx

xxxx

xxxx

A

Go

od

Idea

xx

xxx

xxxx

xxxx

00

0

H

x

xx

xxx

xxxx

000

00

0

T

PHASE-I PHASE-II

QR-Algorithm

for i=1:50 [Q,R]=qr(A); A=R*Q;end

QR Algorithm:

for i=1:50 [Q,R]=qr(H); H=R*Q;end

QR Algorithm:

General matrix A Hessenberg H

O(n^3)

All H’s are Hessenberg (why?)We can upper triangularize H with a sequence of Givens

HGGQH TTn 11

),1,( ii iiGG

O(n^2)

6n^2

for k=1:n-1 v=house(A(k+1:n,k); A(k+1:n,k:n)=(I-2*v*v’/v’*v)*A(k+1:n,k:n); A(1:n,k+1:n)=A(1:n,k+1:n)(I-2*v*v’/v*v);end

Reduce H

Idea

A H

T

PHASE-I

TAQQT

PHASE-II

for i=1:50 [Q,R]=qr(H); H=R*Q;end

QR Algorithm:

H

If A is symmetric matrix, then H is Hessenberg and symmetric al tridiagonA

A symmetric

The work reduces to 50% or less

Example

Slow Convergence

A H

T

PHASE-I

PHASE-IIHA = 1 2 3 4 4 5 6 7 2 1 5 0 4 2 1 0

H = 1.0000 -5.0000 -1.5395 -1.2767 -6.0000 8.5556 0.1085 3.6986 0 -5.9182 -2.1689 -1.1428 0 0 -0.1428 3.6133

>> eig(A)' 11.1055 -3.8556 3.5736 0.1765

H20 = 11.1055 -4.7403 3.9060 -4.0296 -0.0000 -3.8526 -0.6899 1.2559 0 -0.0322 3.5706 0.1571 0 0 -0.0000 0.1765

H5 = 11.1328 4.5792 3.9941 -4.0372 -0.0905 -3.8908 0.5327 -1.2342 0 -0.1006 3.5815 0.1342 0 0 -0.0000 0.1765

Any solution for low convergence

Remark:

Remark:Deflation

Remark:We assume that H is unreduced. If not then we have

Decouple and Deflation

xx

xxx

xxx

xxxxx

xxxxxx

xxxxxx

H0

The problem decouples into two smaller problem

x

xxx

xxxx

xxxxx

xxxxxx

xxxxxx

H

0

xx

xx

xxxx

xxxxx

xxxxxx

xxxxxx

H

0

A of spectrum },,{)( If 1 nA sI-A of spectrum },,{)(then 1 sssIA n

QR Algorithm with shift

TPHASE-II

QR Algorithm with shift:

H A20 = 11.1055 -4.7403 3.9060 -4.0296 -0.0000 -3.8526 -0.6899 1.2559 0 -0.0322 3.5706 0.1571 0 0 -0.0000 0.1765

>> eig(A)' 11.1055 -3.8556 3.5736 0.1765

1-i

i1, rate with 0

iia

The theorem says that if we shift an exact eigenvalue, then in exact arithmetic deflation occurs in one step. )(

,k choice good a knnh

nnnn hh

h

h

1,

21

11

nnnn hh

h

h

1,

21

11

H H

end

I;Q*RH

I);-qr(HR][Q,

shift determine

50:1kfor

k

k

k

Example:

QR Algorithm with shift

TPHASE-II

QR Algorithm with shift:

end

I;Q*RH

I);-qr(HR][Q,

shift determine

50:1kfor

k

k

k

H

>> eig(A)' 11.1055 -3.8556 3.5736 0.1765

1-i

i1, rate with 0

iia

H = 1.0000 -5.0000 -1.5395 -1.2767 -6.0000 8.5556 0.1085 3.6986 0 -5.9182 -2.1689 -1.1428 0 0 -0.1428 3.6133

H1 = 3.3236 -5.5508 4.4353 2.6929 -8.0984 2.9128 -6.1528 -2.4688 0 -2.2250 1.1900 1.0296 0 0 -0.0017 3.5736

H2 = 2.4924 -9.7341 -2.5648 2.3834 -5.9560 5.0131 3.0854 -2.9507 0 -1.4702 -0.0791 0.0767 0 0 -0.0000 3.5736

nnk h

deflation

)(,k choice good a knnh

Example:

QR Algorithm with shift

H = 1.0000 -5.0000 -1.5395 -1.2767 -6.0000 8.5556 0.1085 3.6986 0 -5.9182 -2.1689 -1.1428 0 0 -0.1428 3.6133

H1 = 3.3236 -5.5508 4.4353 2.6929 -8.0984 2.9128 -6.1528 -2.4688 0 -2.2250 1.1900 1.0296 0 0 -0.0017 3.5736

H2 = 2.4924 -9.7341 -2.5648 2.3834 -5.9560 5.0131 3.0854 -2.9507 0 -1.4702 -0.0791 0.0767 0 0 -0.0000 3.5736

nnk h

Theory: quadratic convSymmetric case : cubic

H5 = 11.0798 5.2490 -3.2292 -0.0099 0.1708 1.2672 0 -0.0000 -3.8640

H4 11.0926 5.2199 -3.2299 -0.0368 0.1589 1.2723 0 -0.0029 -3.8649

H3 = 11.1418 5.2542 2.9806 -0.1409 0.0453 -1.4768 0 -0.1735 -3.8005

deflation

Eigenvalues of Symmetric Matrices

A T

D

PHASE-I

PHASE-II

for i=1:50 [Q,R]=qr(H); H=R*Q;end

QR Algorithm:

T

If A is symmetric matrix, then H is Hessenberg and symmetric al tridiagonA

A symmetricT is tridiagonal

O(n)

Schur Factorization:

DAQQT

n

1

orthogonal Qsymmetric If nnRA

kkk qAq Moreover,

Wilkinson shift:

QR Algorithm with wilkinson shift

TPHASE-II

QR Algorithm with shift:

end

I;Q*RH

I);-qr(HR][Q,

shift determine

50:1kfor

k

k

k

H

nnakfor choice reasonable a

nn

nn

ab

ba

b

ba

T

1

11

1

11

A more effective choice is to shift by the eigenvalue of

nn

nn

ab

baT

1

11

That is closer to na2

12)( nn bddsignda

2/)( 1 nn aad

Wilkinson has shown that: cubically convergent.

Can we use this shift to non-symmetric matrix