QR-method lecture 1 - Department of Mathematics | … › ... › matber15 ›...
Transcript of QR-method lecture 1 - Department of Mathematics | … › ... › matber15 ›...
![Page 1: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/1.jpg)
QR-method lecture 1SF2524 - Matrix Computations for Large-scale Systems
QR-method lecture 1 1 / 47
![Page 2: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/2.jpg)
So far we have in the course learned about...
Methods suitable for large sparse matrices
Power methodI Computes largest eigenvalue
Inverse iterationI Computes eigenvalue closest to a target
Rayleigh Quotient IterationI Computes one eigenvalue with a starting guess
Arnoldi method for eigenvalue problemsI Computes extreme eigenvalue
Now: QR-method
Compute all eigenvalues
Suitable for dense problems
Small matrices in relation to previous algorithms
QR-method lecture 1 2 / 47
![Page 3: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/3.jpg)
So far we have in the course learned about...
Methods suitable for large sparse matrices
Power methodI Computes largest eigenvalue
Inverse iterationI Computes eigenvalue closest to a target
Rayleigh Quotient IterationI Computes one eigenvalue with a starting guess
Arnoldi method for eigenvalue problemsI Computes extreme eigenvalue
Now: QR-method
Compute all eigenvalues
Suitable for dense problems
Small matrices in relation to previous algorithms
QR-method lecture 1 2 / 47
![Page 4: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/4.jpg)
So far we have in the course learned about...
Methods suitable for large sparse matrices
Power methodI Computes largest eigenvalue
Inverse iterationI Computes eigenvalue closest to a target
Rayleigh Quotient IterationI Computes one eigenvalue with a starting guess
Arnoldi method for eigenvalue problemsI Computes extreme eigenvalue
Now: QR-method
Compute all eigenvalues
Suitable for dense problems
Small matrices in relation to previous algorithms
QR-method lecture 1 2 / 47
![Page 5: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/5.jpg)
So far we have in the course learned about...
Methods suitable for large sparse matrices
Power methodI Computes largest eigenvalue
Inverse iterationI Computes eigenvalue closest to a target
Rayleigh Quotient IterationI Computes one eigenvalue with a starting guess
Arnoldi method for eigenvalue problemsI Computes extreme eigenvalue
Now: QR-method
Compute all eigenvalues
Suitable for dense problems
Small matrices in relation to previous algorithms
QR-method lecture 1 2 / 47
![Page 6: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/6.jpg)
So far we have in the course learned about...
Methods suitable for large sparse matrices
Power methodI Computes largest eigenvalue
Inverse iterationI Computes eigenvalue closest to a target
Rayleigh Quotient IterationI Computes one eigenvalue with a starting guess
Arnoldi method for eigenvalue problemsI Computes extreme eigenvalue
Now: QR-method
Compute all eigenvalues
Suitable for dense problems
Small matrices in relation to previous algorithms
QR-method lecture 1 2 / 47
![Page 7: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/7.jpg)
Agenda QR-method
1 DecompositionsI Jordan formI Schur decompositionI QR-factorization
2 Basic QR-method3 Improvement 1: Two-phase approach
I Hessenberg reductionI Hessenberg QR-method
4 Improvement 2: Acceleration with shifts
5 Convergence theory
Reading instructions
Point 1: TB Lecture 24Points 2-4: Lecture notes “QR-method” on course web pagePoint 5: TB Chapter 28(Extra reading: TB Chapter 25-26, 28-29)
QR-method lecture 1 3 / 47
![Page 8: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/8.jpg)
Agenda QR-method
1 DecompositionsI Jordan formI Schur decompositionI QR-factorization
2 Basic QR-method3 Improvement 1: Two-phase approach
I Hessenberg reductionI Hessenberg QR-method
4 Improvement 2: Acceleration with shifts
5 Convergence theory
Reading instructions
Point 1: TB Lecture 24Points 2-4: Lecture notes “QR-method” on course web pagePoint 5: TB Chapter 28(Extra reading: TB Chapter 25-26, 28-29)
QR-method lecture 1 3 / 47
![Page 9: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/9.jpg)
1 DecompositionsI Jordan formI Schur decompositionI QR-factorization
2 Basic QR-method3 Improvement 1: Two-phase approach
I Hessenberg reductionI Hessenberg QR-method
4 Improvement 2: Acceleration with shifts
5 Convergence theory
QR-method lecture 1 4 / 47
![Page 10: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/10.jpg)
Similarity transformation
Suppose A ∈ Cm×m and V ∈ Cm×m is an invertible matrix. Then
A
andB = VAV −1
have the same eigenvalues.
Numerical methods based on similarity transformations
If B is triangular we can read-off the eigenvalues from the diagonal.
Idea of numerical method: Compute V such that B is triangular.
QR-method lecture 1 5 / 47
![Page 11: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/11.jpg)
Similarity transformation
Suppose A ∈ Cm×m and V ∈ Cm×m is an invertible matrix. Then
A
andB = VAV −1
have the same eigenvalues.
Numerical methods based on similarity transformations
If B is triangular we can read-off the eigenvalues from the diagonal.
Idea of numerical method: Compute V such that B is triangular.
QR-method lecture 1 5 / 47
![Page 12: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/12.jpg)
First idea: compute the Jordan canonical form
Jordan canonical form
Suppose A ∈ Cm×m. There exists an invertible matrix V ∈ Cm×m and ablock diagonal matrix such that
A = V ΛV −1
where
Λ =
J1. . .
Jk
,
where
Ji =
λi 1
. . .. . .. . . 1
λi
, i = 1, . . . , k
QR-method lecture 1 6 / 47
![Page 13: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/13.jpg)
First idea: compute the Jordan canonical form
Jordan canonical form
Suppose A ∈ Cm×m. There exists an invertible matrix V ∈ Cm×m and ablock diagonal matrix such that
A = V ΛV −1
where
Λ =
J1. . .
Jk
,
where
Ji =
λi 1
. . .. . .. . . 1
λi
, i = 1, . . . , k
QR-method lecture 1 6 / 47
![Page 14: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/14.jpg)
Common case: distinct eigenvalues
Suppose λi 6= λj , i = 1, . . . ,m. Then,
Λ =
λ1 . . .
λm
.
Common case: symmetric matrix
Suppose A = AT ∈ Rm×m. Then,
Λ =
λ1 . . .
λm
.
QR-method lecture 1 7 / 47
![Page 15: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/15.jpg)
Common case: distinct eigenvalues
Suppose λi 6= λj , i = 1, . . . ,m. Then,
Λ =
λ1 . . .
λm
.
Common case: symmetric matrix
Suppose A = AT ∈ Rm×m. Then,
Λ =
λ1 . . .
λm
.
QR-method lecture 1 7 / 47
![Page 16: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/16.jpg)
Example - numerical stability of Jordan form
Consider
A =
2 12 1
ε 2
If ε = 0. Then, the Jordan canonical form is
Λ =
2 12 1
2
.
If ε > 0. Then, the eigenvalues are distinct and
Λ =
2 + O(ε1/3)
2 + O(ε1/3)
2 + O(ε1/3)
.
⇒ Not continuous with respect to ε⇒ The Jordan form is often not numerically stable
QR-method lecture 1 8 / 47
![Page 17: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/17.jpg)
Example - numerical stability of Jordan form
Consider
A =
2 12 1
ε 2
If ε = 0. Then, the Jordan canonical form is
Λ =
2 12 1
2
.
If ε > 0. Then, the eigenvalues are distinct and
Λ =
2 + O(ε1/3)
2 + O(ε1/3)
2 + O(ε1/3)
.
⇒ Not continuous with respect to ε⇒ The Jordan form is often not numerically stable
QR-method lecture 1 8 / 47
![Page 18: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/18.jpg)
Example - numerical stability of Jordan form
Consider
A =
2 12 1
ε 2
If ε = 0. Then, the Jordan canonical form is
Λ =
2 12 1
2
.
If ε > 0. Then, the eigenvalues are distinct and
Λ =
2 + O(ε1/3)
2 + O(ε1/3)
2 + O(ε1/3)
.
⇒ Not continuous with respect to ε⇒ The Jordan form is often not numerically stable
QR-method lecture 1 8 / 47
![Page 19: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/19.jpg)
Example - numerical stability of Jordan form
Consider
A =
2 12 1
ε 2
If ε = 0. Then, the Jordan canonical form is
Λ =
2 12 1
2
.
If ε > 0. Then, the eigenvalues are distinct and
Λ =
2 + O(ε1/3)
2 + O(ε1/3)
2 + O(ε1/3)
.
⇒ Not continuous with respect to ε⇒ The Jordan form is often not numerically stable
QR-method lecture 1 8 / 47
![Page 20: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/20.jpg)
Schur decomposition (essentially TB Theorem 24.9)
Suppose A ∈ Cm×m. There exists an unitary matrix P
P−1 = P∗
and a triangular matrix T such that
A = PTP∗.
The Schur decomposition is numerically stable.Goal with QR-method: Numercally compute a Schur factorization
QR-method lecture 1 9 / 47
![Page 21: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/21.jpg)
Schur decomposition (essentially TB Theorem 24.9)
Suppose A ∈ Cm×m. There exists an unitary matrix P
P−1 = P∗
and a triangular matrix T such that
A = PTP∗.
The Schur decomposition is numerically stable.Goal with QR-method: Numercally compute a Schur factorization
QR-method lecture 1 9 / 47
![Page 22: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/22.jpg)
Outline:1 Decompositions
I Jordan formI Schur decompositionI QR-factorization
2 Basic QR-method3 Improvement 1: Two-phase approach
I Hessenberg reductionI Hessenberg QR-method
4 Improvement 2: Acceleration with shifts
5 Convergence theory
QR-method lecture 1 10 / 47
![Page 23: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/23.jpg)
QR-factorization
Suppose A ∈ Cm×m. There exists a unitary matrix Q and an uppertriangular matrix R such that
A = QR
Note: Very different from Schur factorization
A = QTQ∗
QR-factorization can be computed with a finite number of iterations
Schur decomposition directly gives us the eigenvalues
QR-method lecture 1 11 / 47
![Page 24: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/24.jpg)
QR-factorization
Suppose A ∈ Cm×m. There exists a unitary matrix Q and an uppertriangular matrix R such that
A = QR
Note: Very different from Schur factorization
A = QTQ∗
QR-factorization can be computed with a finite number of iterations
Schur decomposition directly gives us the eigenvalues
QR-method lecture 1 11 / 47
![Page 25: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/25.jpg)
Basic QR-method
Didactic simplifying assumption: All eigenvalues are real
Basic QR-method = basic QR-algorithm
Simple basic idea: Let A0 = A and iterate:
Compute QR-factorization of Ak = QR
Set Ak+1 = RQ.
Note:
A1 = RQ = Q∗A0Q ⇒ A0,A1, . . . have the same eigenvalues
More remarkable: Ak → triangular matrix (except special cases)
QR-method lecture 1 12 / 47
![Page 26: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/26.jpg)
Basic QR-method
Didactic simplifying assumption: All eigenvalues are real
Basic QR-method = basic QR-algorithm
Simple basic idea: Let A0 = A and iterate:
Compute QR-factorization of Ak = QR
Set Ak+1 = RQ.
Note:
A1 = RQ = Q∗A0Q ⇒ A0,A1, . . . have the same eigenvalues
More remarkable: Ak → triangular matrix (except special cases)
QR-method lecture 1 12 / 47
![Page 27: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/27.jpg)
Basic QR-method
Didactic simplifying assumption: All eigenvalues are real
Basic QR-method = basic QR-algorithm
Simple basic idea: Let A0 = A and iterate:
Compute QR-factorization of Ak = QR
Set Ak+1 = RQ.
Note:
A1 = RQ = Q∗A0Q ⇒ A0,A1, . . . have the same eigenvalues
More remarkable: Ak → triangular matrix (except special cases)
QR-method lecture 1 12 / 47
![Page 28: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/28.jpg)
Basic QR-method
Didactic simplifying assumption: All eigenvalues are real
Basic QR-method = basic QR-algorithm
Simple basic idea: Let A0 = A and iterate:
Compute QR-factorization of Ak = QR
Set Ak+1 = RQ.
Note:
A1 = RQ = Q∗A0Q ⇒ A0,A1, . . . have the same eigenvalues
More remarkable: Ak → triangular matrix (except special cases)
QR-method lecture 1 12 / 47
![Page 29: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/29.jpg)
Ak → triangular matrix:
* Time for matlab demo *
QR-method lecture 1 13 / 47
![Page 30: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/30.jpg)
Ak → triangular matrix:
* Time for matlab demo *
QR-method lecture 1 13 / 47
![Page 31: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/31.jpg)
Elegant and robust but not very efficient:
Disadvantages
Computing a QR-factorization is quite expensive. One iteration of thebasic QR-method
O(m3).
The method often requires many iterations.
Improvement demo:
http://www.youtube.com/watch?v=qmgxzsWWsNc
QR-method lecture 1 14 / 47
![Page 32: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/32.jpg)
Elegant and robust but not very efficient:
Disadvantages
Computing a QR-factorization is quite expensive. One iteration of thebasic QR-method
O(m3).
The method often requires many iterations.
Improvement demo:
http://www.youtube.com/watch?v=qmgxzsWWsNc
QR-method lecture 1 14 / 47
![Page 33: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/33.jpg)
Elegant and robust but not very efficient:
Disadvantages
Computing a QR-factorization is quite expensive. One iteration of thebasic QR-method
O(m3).
The method often requires many iterations.
Improvement demo:
http://www.youtube.com/watch?v=qmgxzsWWsNc
QR-method lecture 1 14 / 47
![Page 34: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/34.jpg)
Outline:1 Decompositions
I Jordan formI Schur decompositionI QR-factorization
2 Basic QR-method3 Improvement 1: Two-phase approach
I Hessenberg reductionI Hessenberg QR-method
4 Improvement 2: Acceleration with shifts
5 Convergence theory
QR-method lecture 1 15 / 47
![Page 35: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/35.jpg)
Improvement 1: Two-phase approach
We will separate the computation into two phases:
× × × × ×× × × × ×× × × × ×× × × × ×× × × × ×
→Phase 1
× × × × ×× × × × ×
× × × ×× × ×
× ×
→Phase 2
× × × × ×
× × × ×× × ×
× ××
Phases:
Phase 1: Reduce the matrix to a Hessenberg with similaritytransformations (Section 2.2.1 in lecture notes)
Phase 2: Specialize the QR-method to Hessenberg matrices (Section2.2.2 in lecture notes)
QR-method lecture 1 16 / 47
![Page 36: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/36.jpg)
Improvement 1: Two-phase approach
We will separate the computation into two phases:
× × × × ×× × × × ×× × × × ×× × × × ×× × × × ×
→Phase 1
× × × × ×× × × × ×
× × × ×× × ×
× ×
→Phase 2
× × × × ×
× × × ×× × ×
× ××
Phases:
Phase 1: Reduce the matrix to a Hessenberg with similaritytransformations (Section 2.2.1 in lecture notes)
Phase 2: Specialize the QR-method to Hessenberg matrices (Section2.2.2 in lecture notes)
QR-method lecture 1 16 / 47
![Page 37: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/37.jpg)
Improvement 1: Two-phase approach
We will separate the computation into two phases:
× × × × ×× × × × ×× × × × ×× × × × ×× × × × ×
→Phase 1
× × × × ×× × × × ×
× × × ×× × ×
× ×
→Phase 2
× × × × ×
× × × ×× × ×
× ××
Phases:
Phase 1: Reduce the matrix to a Hessenberg with similaritytransformations (Section 2.2.1 in lecture notes)
Phase 2: Specialize the QR-method to Hessenberg matrices (Section2.2.2 in lecture notes)
QR-method lecture 1 16 / 47
![Page 38: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/38.jpg)
Phase 1: Hessenberg reduction
Idea for Hessenberg reduction
Compute unitary P and Hessenberg matrix H such that
A = PHP∗
Unlike the Schur factorization, this can be computed with a finite numberof operations.
Key method: Householder reflectors
QR-method lecture 1 17 / 47
![Page 39: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/39.jpg)
Phase 1: Hessenberg reduction
Idea for Hessenberg reduction
Compute unitary P and Hessenberg matrix H such that
A = PHP∗
Unlike the Schur factorization, this can be computed with a finite numberof operations.
Key method: Householder reflectors
QR-method lecture 1 17 / 47
![Page 40: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/40.jpg)
Phase 1: Hessenberg reduction
Idea for Hessenberg reduction
Compute unitary P and Hessenberg matrix H such that
A = PHP∗
Unlike the Schur factorization, this can be computed with a finite numberof operations.
Key method: Householder reflectors
QR-method lecture 1 17 / 47
![Page 41: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/41.jpg)
Phase 1: Hessenberg reduction
Definition
A matrix P ∈ Cm×m of the form
P = I − 2uu∗ where u ∈ Cm and ‖u‖ = 1
is called a Householder reflector.
Px
ux Properties
P∗ = P−1 = P
Pz = z − 2u(u∗z) can becomputed with O(m)operations.
· · ·
QR-method lecture 1 18 / 47
![Page 42: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/42.jpg)
Phase 1: Hessenberg reduction
Definition
A matrix P ∈ Cm×m of the form
P = I − 2uu∗ where u ∈ Cm and ‖u‖ = 1
is called a Householder reflector.
Px
ux
Properties
P∗ = P−1 = P
Pz = z − 2u(u∗z) can becomputed with O(m)operations.
· · ·
QR-method lecture 1 18 / 47
![Page 43: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/43.jpg)
Phase 1: Hessenberg reduction
Definition
A matrix P ∈ Cm×m of the form
P = I − 2uu∗ where u ∈ Cm and ‖u‖ = 1
is called a Householder reflector.
Px
ux Properties
P∗ = P−1 = P
Pz = z − 2u(u∗z) can becomputed with O(m)operations.
· · ·
QR-method lecture 1 18 / 47
![Page 44: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/44.jpg)
Phase 1: Hessenberg reduction
Definition
A matrix P ∈ Cm×m of the form
P = I − 2uu∗ where u ∈ Cm and ‖u‖ = 1
is called a Householder reflector.
Px
ux Properties
P∗ = P−1 = P
Pz = z − 2u(u∗z) can becomputed with O(m)operations.
· · ·
QR-method lecture 1 18 / 47
![Page 45: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/45.jpg)
Phase 1: Hessenberg reduction
Definition
A matrix P ∈ Cm×m of the form
P = I − 2uu∗ where u ∈ Cm and ‖u‖ = 1
is called a Householder reflector.
Px
ux Properties
P∗ = P−1 = P
Pz = z − 2u(u∗z) can becomputed with O(m)operations.
· · ·
QR-method lecture 1 18 / 47
![Page 46: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/46.jpg)
Householder reflectors satisfying Px = αe1
Problem
Given a vector x compute a Householder reflector such that
Px = αe1.
Solution (Lemma 2.2.3)
Let ρ = sign(x1),
z := x − ρ‖x‖e1 =
x1 − ρ‖x‖
x2...
xn
and
u = z/‖z‖.
Then, P = I − 2uu∗ is a Householder reflector that satisfies Px = αe1.
QR-method lecture 1 19 / 47
![Page 47: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/47.jpg)
Householder reflectors satisfying Px = αe1
Problem
Given a vector x compute a Householder reflector such that
Px = αe1.
Solution (Lemma 2.2.3)
Let ρ = sign(x1),
z := x − ρ‖x‖e1 =
x1 − ρ‖x‖
x2...
xn
and
u = z/‖z‖.
Then, P = I − 2uu∗ is a Householder reflector that satisfies Px = αe1.
QR-method lecture 1 19 / 47
![Page 48: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/48.jpg)
Householder reflectors satisfying Px = αe1
Problem
Given a vector x compute a Householder reflector such that
Px = αe1.
Solution (Lemma 2.2.3)
Let ρ = sign(x1),
z := x − ρ‖x‖e1 =
x1 − ρ‖x‖
x2...
xn
and
u = z/‖z‖.
Then, P = I − 2uu∗ is a Householder reflector that satisfies Px = αe1.
QR-method lecture 1 19 / 47
![Page 49: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/49.jpg)
Householder reflectors satisfying Px = αe1
Problem
Given a vector x compute a Householder reflector such that
Px = αe1.
Solution (Lemma 2.2.3)
Let ρ = sign(x1),
z := x − ρ‖x‖e1 =
x1 − ρ‖x‖
x2...
xn
and
u = z/‖z‖.
Then, P = I − 2uu∗ is a Householder reflector that satisfies Px = αe1.
QR-method lecture 1 19 / 47
![Page 50: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/50.jpg)
* Matlab demo showing Householder reflectors *
QR-method lecture 1 20 / 47
![Page 51: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/51.jpg)
We will be able to construct m − 2 householder reflectors that bring thematrix to Hessenberg form.
Elimination for first column
P1 :=
1 0 0 0 00 × × × ×0 × × × ×0 × × × ×0 × × × ×
=
[1 0T
0 I − 2u1uT1
].
Use Lemma 2.2.1 with xT = [a21, . . . , an1] to select u1 such that
P1A =
× × × × ×× × × × ×× × × ×× × × ×× × × ×
In order to have a similarity transformation mult from right:
P1AP−11 = P1AP1 = same structure as P1A.
QR-method lecture 1 21 / 47
![Page 52: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/52.jpg)
We will be able to construct m − 2 householder reflectors that bring thematrix to Hessenberg form.
Elimination for first column
P1 :=
1 0 0 0 00 × × × ×0 × × × ×0 × × × ×0 × × × ×
=
[1 0T
0 I − 2u1uT1
].
Use Lemma 2.2.1 with xT = [a21, . . . , an1] to select u1 such that
P1A =
× × × × ×× × × × ×× × × ×× × × ×× × × ×
In order to have a similarity transformation mult from right:
P1AP−11 = P1AP1 = same structure as P1A.
QR-method lecture 1 21 / 47
![Page 53: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/53.jpg)
We will be able to construct m − 2 householder reflectors that bring thematrix to Hessenberg form.
Elimination for first column
P1 :=
1 0 0 0 00 × × × ×0 × × × ×0 × × × ×0 × × × ×
=
[1 0T
0 I − 2u1uT1
].
Use Lemma 2.2.1 with xT = [a21, . . . , an1] to select u1 such that
P1A =
× × × × ×× × × × ×× × × ×× × × ×× × × ×
In order to have a similarity transformation mult from right:
P1AP−11 = P1AP1 = same structure as P1A.
QR-method lecture 1 21 / 47
![Page 54: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/54.jpg)
We will be able to construct m − 2 householder reflectors that bring thematrix to Hessenberg form.
Elimination for first column
P1 :=
1 0 0 0 00 × × × ×0 × × × ×0 × × × ×0 × × × ×
=
[1 0T
0 I − 2u1uT1
].
Use Lemma 2.2.1 with xT = [a21, . . . , an1] to select u1 such that
P1A =
× × × × ×× × × × ×× × × ×× × × ×× × × ×
In order to have a similarity transformation mult from right:
P1AP−11 = P1AP1 = same structure as P1A.
QR-method lecture 1 21 / 47
![Page 55: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/55.jpg)
We will be able to construct m − 2 householder reflectors that bring thematrix to Hessenberg form.
Elimination for first column
P1 :=
1 0 0 0 00 × × × ×0 × × × ×0 × × × ×0 × × × ×
=
[1 0T
0 I − 2u1uT1
].
Use Lemma 2.2.1 with xT = [a21, . . . , an1] to select u1 such that
P1A =
× × × × ×× × × × ×× × × ×× × × ×× × × ×
In order to have a similarity transformation mult from right:
P1AP−11 = P1AP1 = same structure as P1A.
QR-method lecture 1 21 / 47
![Page 56: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/56.jpg)
Elimination for second column
Repeat the process with:
P2 =
1 0 0T
0 1 0T
0 0 I − 2u2uT2
where u2 is constructed from the n− 2 last elements of the second columnof P1AP∗
1 .
P1AP1 =
[× × × × ×× × × × ×
× × × ×× × × ×× × × ×
]−→
mult. fromleft with P2
[× × × × ×× × × × ×
× × × ×× × ×× × ×
]
−→mult. from
right with P2
[× × × × ×× × × × ×
× × × ×× × ×× × ×
]= P2P1AP1P2
QR-method lecture 1 22 / 47
![Page 57: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/57.jpg)
* Matlab demo of the first two steps of the Hessenberg reduction *
QR-method lecture 1 22 / 47
![Page 58: QR-method lecture 1 - Department of Mathematics | … › ... › matber15 › qrmethod_lecture1.pdf1 Decompositions I Jordan form I Schur decomposition I QR-factorization 2 Basic](https://reader034.fdocuments.in/reader034/viewer/2022042408/5f23570dc7f9495938674219/html5/thumbnails/58.jpg)
The iteration can be implemented without explicit use of the P matrices.
* show it in matlab *
QR-method lecture 1 23 / 47