The Cholesky decomposition - Scientific Computing and Imaging...
Transcript of The Cholesky decomposition - Scientific Computing and Imaging...
![Page 1: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/1.jpg)
L17-S00
The Cholesky decomposition
MATH 6610 Lecture 17
October 16, 2020
Trefethen & Bau: Lecture 23
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 2: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/2.jpg)
L17-S01Hermitian positive-definite matrices
Assume A P nˆn is Hermitian positive definite.
Our investigation of LU decompositions specializes considerably in this case.
First we note some properties of A:A is invertibleThe diagonal entries of A are real and strictly positiveIf B P mˆn with m § n is of full rank, then BAB˚ is positive-definite
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 3: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/3.jpg)
L17-S01Hermitian positive-definite matrices
Assume A P nˆn is Hermitian positive definite.
Our investigation of LU decompositions specializes considerably in this case.
First we note some properties of A:A is invertibleThe diagonal entries of A are real and strictly positiveIf B P mˆn with m § n is of full rank, then BAB˚ is positive-definite
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 4: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/4.jpg)
![Page 5: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/5.jpg)
L17-S02LU on positive-definite matrices
A general positive-definite matrix A has the form
A “
¨
˚̊˝
a ´ v˚ ´
v A2
˛
‹‹‚.
Consider performing elimination on A:
A “ L1B˚ “
¨
˚̊˝
1 ´ 0 ´va I
˛
‹‹‚
¨
˚̊˝
a ´ v˚ ´
0 A2 ´ vv˚a
˛
‹‹‚
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 6: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/6.jpg)
![Page 7: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/7.jpg)
L17-S02LU on positive-definite matrices
A general positive-definite matrix A has the form
A “
¨
˚̊˝
a ´ v˚ ´
v A2
˛
‹‹‚.
Consider performing elimination on A:
A “ L1B˚ “
¨
˚̊˝
1 ´ 0 ´va I
˛
‹‹‚
¨
˚̊˝
a ´ v˚ ´
0 A2 ´ vv˚a
˛
‹‹‚
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 8: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/8.jpg)
L17-S03Symmetric factorizations
A “ L1B˚
We can perform a single step of Gaussian elimination on B:
B “ L1
¨
˚̊˝
a ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚,
i.e.,
A “ L1
¨
˚̊˝
a ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚L˚1 “ rL1
¨
˚̊˝
1 ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚rL˚1 .
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 9: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/9.jpg)
![Page 10: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/10.jpg)
L17-S03Symmetric factorizations
A “ L1B˚
We can perform a single step of Gaussian elimination on B:
B “ L1
¨
˚̊˝
a ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚,
i.e.,
A “ L1
¨
˚̊˝
a ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚L˚1 “ rL1
¨
˚̊˝
1 ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚rL˚1 .
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 11: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/11.jpg)
L17-S04The Cholesky factorization
A “ rL1
¨
˚̊˝
1 ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚rL˚1 .
Note that A2 ´ vv˚a must be positive definite since rL1 is invertible.
Thus, we can repeat this process:
A “´
rL1rL2 ¨ ¨ ¨ rLn´1
¯ ´rL1
rL2 ¨ ¨ ¨ rLn´1
¯˚
“: LL˚.
TheoremEvery Hermitian positive definite matrix A has a unique symmetric LU, or
Cholesky, decomposition: A “ LL˚, where L is lower-triangular and
invertible.
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 12: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/12.jpg)
![Page 13: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/13.jpg)
L17-S04The Cholesky factorization
A “ rL1
¨
˚̊˝
1 ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚rL˚1 .
Note that A2 ´ vv˚a must be positive definite since rL1 is invertible.
Thus, we can repeat this process:
A “´
rL1rL2 ¨ ¨ ¨ rLn´1
¯ ´rL1
rL2 ¨ ¨ ¨ rLn´1
¯˚
“: LL˚.
TheoremEvery Hermitian positive definite matrix A has a unique symmetric LU, or
Cholesky, decomposition: A “ LL˚, where L is lower-triangular and
invertible.
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 14: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/14.jpg)
L17-S04The Cholesky factorization
A “ rL1
¨
˚̊˝
1 ´ 0 ´
0 A2 ´ vv˚a
˛
‹‹‚rL˚1 .
Note that A2 ´ vv˚a must be positive definite since rL1 is invertible.
Thus, we can repeat this process:
A “´
rL1rL2 ¨ ¨ ¨ rLn´1
¯ ´rL1
rL2 ¨ ¨ ¨ rLn´1
¯˚
“: LL˚.
TheoremEvery Hermitian positive definite matrix A has a unique symmetric LU, or
Cholesky, decomposition: A “ LL˚, where L is lower-triangular and
invertible.
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 15: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/15.jpg)
L17-S05Pivoted Cholesky
One can perform symmetric pivoting on a Hermitian positive-definite matrixA: A “ PLL˚P˚.
This could be used to pivot maximum-magnitude diagonal entries to thefront.
However, pivoted Cholesky decompositions have another use:
TheoremEvery Hermitian positive semi-definite matrix A has a pivoted Cholesky
decomposition: A “ PLL˚P˚, where L is lower-triangular but need not
invertible. This decomposition is in general not unique.
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 16: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/16.jpg)
L17-S05Pivoted Cholesky
One can perform symmetric pivoting on a Hermitian positive-definite matrixA: A “ PLL˚P˚.
This could be used to pivot maximum-magnitude diagonal entries to thefront.
However, pivoted Cholesky decompositions have another use:
TheoremEvery Hermitian positive semi-definite matrix A has a pivoted Cholesky
decomposition: A “ PLL˚P˚, where L is lower-triangular but need not
invertible. This decomposition is in general not unique.
MATH 6610-001 – U. Utah The Cholesky decomposition
![Page 17: The Cholesky decomposition - Scientific Computing and Imaging …akil/docs/courses/2020fall/math6610/... · 2020. 10. 16. · Pivoted Cholesky L17-S05 One can perform symmetric pivoting](https://reader036.fdocuments.in/reader036/viewer/2022062610/610deac4bb16ec33ba727d4a/html5/thumbnails/17.jpg)