Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. ·...
Transcript of Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. ·...
![Page 1: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/1.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Important Matrix Factorizations
LU Choleski QR
LU Choleski QR
Important Matrix Factorizations
![Page 2: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/2.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization
Choleski Factorization
The QR Factorization
LU Choleski QR
Important Matrix Factorizations
![Page 3: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/3.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization: Gaussian Elimination Matrices
Gaussian elimination transforms vectors of the form aαb
,where a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1, to those of the form a
α0
.
This is accomplished by left matrix multiplication as follows: Ik×k 0 00 1 00 −α−1b I(n−k−1)×(n−k−1)
aαb
=
aα0
.
The matrix on the left is called a Gaussian elimination matrix.
LU Choleski QR
Important Matrix Factorizations
![Page 4: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/4.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization: Gaussian Elimination Matrices
Gaussian elimination transforms vectors of the form aαb
,where a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1, to those of the form a
α0
.
This is accomplished by left matrix multiplication as follows: Ik×k 0 00 1 00 −α−1b I(n−k−1)×(n−k−1)
aαb
=
aα0
.The matrix on the left is called a Gaussian elimination matrix.
LU Choleski QR
Important Matrix Factorizations
![Page 5: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/5.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Gaussian Elimination Matrices
The matrix Ik×k 0 00 1 00 −α−1b I(n−k−1)×(n−k−1)
has ones on the diagonal and so is invertible. Indeed, Ik×k 0 0
0 1 00 −α−1b I(n−k−1)×(n−k−1)
−1 =
Ik×k 0 00 1 00 α−1b I(n−k−1)×(n−k−1)
.Also note that Ik×k 0 0
0 1 00 −α−1b I(n−k−1)×(n−k−1)
x0y
=
x0y
.LU Choleski QR
Important Matrix Factorizations
![Page 6: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/6.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization
Suppose
A =
[a1 vT
1
u1 A1
]∈ Cn×m,
with 0 6= a1 ∈ C, u1 ∈ Cm−1, v1 ∈ Cn−1, and A1 ∈ C(m−1)×(n−1).
Then [1 0− u1
a1I
] [a1 vT
1
u1 A1
]∈ Cn×m =
[a1 vT
1
0 A1
], (*)
where A1 = A1 − u1vT1 /a1 .
Repeat m times to get L−1m−1 · · · L−12 L−11 A = Um−1 = U is upper
triangular, soA = LU
where L is lower triangular with ones on the diagonal.
LU Choleski QR
Important Matrix Factorizations
![Page 7: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/7.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization
Suppose
A =
[a1 vT
1
u1 A1
]∈ Cn×m,
with 0 6= a1 ∈ C, u1 ∈ Cm−1, v1 ∈ Cn−1, and A1 ∈ C(m−1)×(n−1).Then [
1 0− u1
a1I
] [a1 vT
1
u1 A1
]∈ Cn×m =
[a1 vT
1
0 A1
], (*)
where A1 = A1 − u1vT1 /a1 .
Repeat m times to get L−1m−1 · · · L−12 L−11 A = Um−1 = U is upper
triangular, soA = LU
where L is lower triangular with ones on the diagonal.
LU Choleski QR
Important Matrix Factorizations
![Page 8: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/8.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
LU Factorization
Suppose
A =
[a1 vT
1
u1 A1
]∈ Cn×m,
with 0 6= a1 ∈ C, u1 ∈ Cm−1, v1 ∈ Cn−1, and A1 ∈ C(m−1)×(n−1).Then [
1 0− u1
a1I
] [a1 vT
1
u1 A1
]∈ Cn×m =
[a1 vT
1
0 A1
], (*)
where A1 = A1 − u1vT1 /a1 .
Repeat m times to get L−1m−1 · · · L−12 L−11 A = Um−1 = U is upper
triangular, soA = LU
where L is lower triangular with ones on the diagonal.
LU Choleski QR
Important Matrix Factorizations
![Page 9: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/9.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Cholesky Factorization
Suppose M ∈ Rn×n, symmetric and positive definite has LUfactorization
M = LU .
ThenL−1ML−T = UL−T
is an upper triangular symmetric matrix.
That is, UL−T = D, where D is diagonal.
Since M is psd, D has positive diagonal entries, so
M = LDLT = LLT where L = LD1/2.
This is called the Cholesky Factorization of M.
LU Choleski QR
Important Matrix Factorizations
![Page 10: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/10.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Cholesky Factorization
Suppose M ∈ Rn×n, symmetric and positive definite has LUfactorization
M = LU .
ThenL−1ML−T = UL−T
is an upper triangular symmetric matrix.
That is, UL−T = D, where D is diagonal.
Since M is psd, D has positive diagonal entries, so
M = LDLT = LLT where L = LD1/2.
This is called the Cholesky Factorization of M.
LU Choleski QR
Important Matrix Factorizations
![Page 11: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/11.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Cholesky Factorization
Suppose M ∈ Rn×n, symmetric and positive definite has LUfactorization
M = LU .
ThenL−1ML−T = UL−T
is an upper triangular symmetric matrix.
That is, UL−T = D, where D is diagonal.
Since M is psd, D has positive diagonal entries, so
M = LDLT = LLT where L = LD1/2.
This is called the Cholesky Factorization of M.
LU Choleski QR
Important Matrix Factorizations
![Page 12: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/12.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Cholesky Factorization
Suppose M ∈ Rn×n, symmetric and positive definite has LUfactorization
M = LU .
ThenL−1ML−T = UL−T
is an upper triangular symmetric matrix.
That is, UL−T = D, where D is diagonal.
Since M is psd, D has positive diagonal entries, so
M = LDLT = LLT where L = LD1/2.
This is called the Cholesky Factorization of M.
LU Choleski QR
Important Matrix Factorizations
![Page 13: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/13.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Cholesky Factorization
Suppose M ∈ Rn×n, symmetric and positive definite has LUfactorization
M = LU .
ThenL−1ML−T = UL−T
is an upper triangular symmetric matrix.
That is, UL−T = D, where D is diagonal.
Since M is psd, D has positive diagonal entries, so
M = LDLT = LLT where L = LD1/2.
This is called the Cholesky Factorization of M.
LU Choleski QR
Important Matrix Factorizations
![Page 14: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/14.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization: Householder Reflections
Given w ∈ Rn we can associate the matrix
U = I − 2wwT
wTw
which reflects Rn across the hyperplane Span{w}⊥. The matrix Uis call the Householder reflection across this hyperplane.
Given a pair of vectors x and y with
‖x‖2 = ‖y‖2, and x 6= y ,
there is a Householder reflection such that y = Ux :
U = I − 2(x − y)(x − y)T
(x − y)T (x − y).
Householder reflections are symmetric unitary tranformations:U−1 = UT = U.
LU Choleski QR
Important Matrix Factorizations
![Page 15: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/15.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization: Householder Reflections
Given w ∈ Rn we can associate the matrix
U = I − 2wwT
wTw
which reflects Rn across the hyperplane Span{w}⊥. The matrix Uis call the Householder reflection across this hyperplane.Given a pair of vectors x and y with
‖x‖2 = ‖y‖2, and x 6= y ,
there is a Householder reflection such that y = Ux :
U = I − 2(x − y)(x − y)T
(x − y)T (x − y).
Householder reflections are symmetric unitary tranformations:U−1 = UT = U.
LU Choleski QR
Important Matrix Factorizations
![Page 16: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/16.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization: Householder Reflections
Given w ∈ Rn we can associate the matrix
U = I − 2wwT
wTw
which reflects Rn across the hyperplane Span{w}⊥. The matrix Uis call the Householder reflection across this hyperplane.Given a pair of vectors x and y with
‖x‖2 = ‖y‖2, and x 6= y ,
there is a Householder reflection such that y = Ux :
U = I − 2(x − y)(x − y)T
(x − y)T (x − y).
Householder reflections are symmetric unitary tranformations:U−1 = UT = U.
LU Choleski QR
Important Matrix Factorizations
![Page 17: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/17.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization
Given A ∈ Rm×n write
A0 =
[α0 aT0b0 A0
]and ν0 =
∥∥∥∥(α0
b0
)∥∥∥∥2
.
Set
H0 = I − 2wwT
wTwwhere w =
(α0
b0
)− ν0e1 =
(α0 − ν0
b0
).
Then
H0A =
[ν0 aT10 A1
].
LU Choleski QR
Important Matrix Factorizations
![Page 18: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/18.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization
Given A ∈ Rm×n write
A0 =
[α0 aT0b0 A0
]and ν0 =
∥∥∥∥(α0
b0
)∥∥∥∥2
.
Set
H0 = I − 2wwT
wTwwhere w =
(α0
b0
)− ν0e1 =
(α0 − ν0
b0
).
Then
H0A =
[ν0 aT10 A1
].
LU Choleski QR
Important Matrix Factorizations
![Page 19: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/19.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
The QR Factorization
Given A ∈ Rm×n write
A0 =
[α0 aT0b0 A0
]and ν0 =
∥∥∥∥(α0
b0
)∥∥∥∥2
.
Set
H0 = I − 2wwT
wTwwhere w =
(α0
b0
)− ν0e1 =
(α0 − ν0
b0
).
Then
H0A =
[ν0 aT10 A1
].
LU Choleski QR
Important Matrix Factorizations
![Page 20: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/20.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
QR Factorization
H0A =
[ν0 aT10 A1
].
Repeat to get
QTA = Hn−1Hn−2 . . .H0A = R,
where R is upper triangular and Q is unitary.
The A = QR is called the QR factorization of A.
LU Choleski QR
Important Matrix Factorizations
![Page 21: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/21.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
QR Factorization
H0A =
[ν0 aT10 A1
].
Repeat to get
QTA = Hn−1Hn−2 . . .H0A = R,
where R is upper triangular and Q is unitary.
The A = QR is called the QR factorization of A.
LU Choleski QR
Important Matrix Factorizations
![Page 22: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/22.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
QR Factorization
H0A =
[ν0 aT10 A1
].
Repeat to get
QTA = Hn−1Hn−2 . . .H0A = R,
where R is upper triangular and Q is unitary.
The A = QR is called the QR factorization of A.
LU Choleski QR
Important Matrix Factorizations
![Page 23: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/23.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Suppose A ∈ Rm×n with m > n, then A =
The QR factorization of A looks like
A = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.The columns of Q1 form and orthonormal basis for the range of Awith
Q1QT1 = the orthogonal projector onto Ran(A)
and
I − Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(A)⊥
LU Choleski QR
Important Matrix Factorizations
![Page 24: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/24.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Suppose A ∈ Rm×n with m > n, then A =
The QR factorization of A looks like
A = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.
The columns of Q1 form and orthonormal basis for the range of Awith
Q1QT1 = the orthogonal projector onto Ran(A)
and
I − Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(A)⊥
LU Choleski QR
Important Matrix Factorizations
![Page 25: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/25.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Suppose A ∈ Rm×n with m > n, then A =
The QR factorization of A looks like
A = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.The columns of Q1 form and orthonormal basis for the range of Awith
Q1QT1 = the orthogonal projector onto Ran(A)
and
I − Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(A)⊥
LU Choleski QR
Important Matrix Factorizations
![Page 26: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/26.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Similarly, if A ∈ Rm×n with m < n, then AT =
The QR factorization of AT looks like
AT = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.The columns of Q1 form and orthonormal basis for the range of AT with
Q1QT1 = the orthogonal projector onto Ran(AT )
and
I−Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(AT )⊥ = Nul(A)
LU Choleski QR
Important Matrix Factorizations
![Page 27: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/27.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Similarly, if A ∈ Rm×n with m < n, then AT =
The QR factorization of AT looks like
AT = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.
The columns of Q1 form and orthonormal basis for the range of AT with
Q1QT1 = the orthogonal projector onto Ran(AT )
and
I−Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(AT )⊥ = Nul(A)
LU Choleski QR
Important Matrix Factorizations
![Page 28: Important Matrix Factorizations - University of Washingtonburke/crs/408/... · 2012. 2. 17. · Householder re ections are symmetric unitary tranformations: U 1 = UT = U. LU Choleski](https://reader035.fdocuments.in/reader035/viewer/2022071408/61015887a569ca35d17d16bf/html5/thumbnails/28.jpg)
Outline LU Factorization Choleski Factorization The QR Factorization
Orthogonal Projections
Similarly, if A ∈ Rm×n with m < n, then AT =
The QR factorization of AT looks like
AT = [Q1, Q2]
[R0
]= Q1R
where the columns of Q1 and Q2 for an orthonormal basis for Rm.The columns of Q1 form and orthonormal basis for the range of AT with
Q1QT1 = the orthogonal projector onto Ran(AT )
and
I−Q1QT1 = Q2Q
T2 = the orthogonal projector onto Ran(AT )⊥ = Nul(A)
LU Choleski QR
Important Matrix Factorizations