LU Decomposition - MATH FOR...
Transcript of LU Decomposition - MATH FOR...
10/7/2015 http://numericalmethods.eng.usf.edu 1
LU Decomposition
Major: All Engineering Majors
Authors: Autar Kaw
http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM
Undergraduates
http://numericalmethods.eng.usf.edu
LU DecompositionLU Decomposition is another method to solve a set of
simultaneous linear equations
Which is better, Gauss Elimination or LU Decomposition?
To answer this, a closer look at LU decomposition is needed.
MethodFor most non-singular matrix [A] that one could conduct Naïve Gauss Elimination forward elimination steps, one can always write it as
[A] = [L][U]where
[L] = lower triangular matrix
[U] = upper triangular matrix
http://numericalmethods.eng.usf.edu
LU Decomposition
http://numericalmethods.eng.usf.edu
How does LU Decomposition work?If solving a set of linear equations
If [A] = [L][U] thenMultiply by
Which givesRemember [L]-1[L] = [I] which leads to
Now, if [I][U] = [U] thenNow, let
Which ends withand
[A][X] = [C][L][U][X] = [C][L]-1
[L]-1[L][U][X] = [L]-1[C][I][U][X] = [L]-1[C][U][X] = [L]-1[C][L]-1[C]=[Z][L][Z] = [C] (1)[U][X] = [Z] (2)
http://numericalmethods.eng.usf.edu
LU DecompositionHow can this be used?
Given [A][X] = [C]
1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than Gaussian Elimination?
Solve [A][X] = [B]
−+=
3204
38| 2
3 nnnTCT DE
( )nnTCT FS 44| 2 −=( )nnTCT BS 124| 2 +=
−+=
3328
38| 2
3 nnnTCT FE
Forward Elimination Decomposition to LU
Back Substitution Forward Substitution
( )nnTCT BS 124| 2 +=
Back Substitution
T = clock cycle time and nxn = size of the matrix
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than Gaussian Elimination?
To solve [A][X] = [B]Time taken by methods
T = clock cycle time and nxn = size of the matrix
So both methods are equally efficient.
Gaussian Elimination LU Decomposition
++
3412
38 2
3 nnnT
++
3412
38 2
3 nnnT
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
( )
++=
+=
3412
38
||2
34 nnnT
CTCTn BSFE
−+=
×+×+=
32012
332
|||
23 nnnT
CTnCTnCT BSFSDE
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
−+
32012
332 2
3 nnnT
n 10 100 1000 10000CT|inverse GE / CT|inverse LU 3.288 25.84 250.8 2501
Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination.
++
3412
38 2
34 nnnT
For large n, CT|inverse GE / CT|inverse LU ≈ n/4
http://numericalmethods.eng.usf.edu
Method: [A] Decomposes to [L] and [U]
[ ] [ ][ ]
==
33
2322
131211
3231
21
000
101001
uuuuuu
ULA
[U] is the same as the coefficient matrix at the end of the forward elimination step.
[L] is obtained using the multipliers that were used in the forward elimination process
http://numericalmethods.eng.usf.edu
Finding the [U] matrixUsing the Forward Elimination Procedure of Gauss Elimination
11214418641525
( ) 11214456.18.40
152556.212;56.2
2564
−−=−= RowRow
( ) 76.48.16056.18.40
152576.513;76.5
25144
−−−−=−= RowRow
Step 1:
http://numericalmethods.eng.usf.edu
Finding the [U] Matrix
Step 2:
−−−−
76.48.16056.18.40
1525
( ) 7.00056.18.40
15255.323;5.3
8.48.16
−−=−=
−− RowRow
[ ]
−−=
7.00056.18.40
1525U
Matrix after Step 1:
http://numericalmethods.eng.usf.edu
Finding the [L] matrix
Using the multipliers used during the Forward Elimination Procedure
101001
3231
21
56.22564
11
2121 ===
aa
76.525
144
11
3131 ===
aa
From the first step of forward elimination
11214418641525
http://numericalmethods.eng.usf.edu
Finding the [L] Matrix
[ ]
=
15.376.50156.2001
L
From the second step of forward elimination
−−−−
76.48.16056.18.40
15255.3
8.48.16
22
3232 =
−−
==aa
http://numericalmethods.eng.usf.edu
Does [L][U] = [A]?
[ ][ ] =
−−
=
7.00056.18.40
1525
15.376.50156.2001
UL ?
http://numericalmethods.eng.usf.edu
Using LU Decomposition to solve SLEs
Solve the following set of linear equations using LU Decomposition
=
227921778106
11214418641525
3
2
1
.
.
.
xxx
Using the procedure for finding the [L] and [U] matrices
[ ] [ ][ ]
−−
==
7.00056.18.40
1525
15.376.50156.2001
ULA
http://numericalmethods.eng.usf.edu
Example
Set [L][Z] = [C]
Solve for [Z]
=
2.2792.1778.106
15.376.50156.2001
3
2
1
zzz
2.2795.376.52.17756.2
10
321
21
1
=++=+
=
zzzzz
z
http://numericalmethods.eng.usf.edu
ExampleComplete the forward substitution to solve for [Z]
( )
( ) ( )735.0
21.965.38.10676.52.2795.376.52.279
2.968.10656.22.177
56.22.1778.106
213
12
1
=−−−=
−−=−=
−=−=
=
zzz
zzz
[ ]
−=
=
735.021.968.106
3
2
1
zzz
Z
http://numericalmethods.eng.usf.edu
ExampleSet [U][X] = [Z]
Solve for [X] The 3 equations become
−=
−−
735021968106
7.00056.18.40
1525
3
2
1
...
xxx
735.07.021.9656.18.48.106525
3
32
321
=−=−−
=++
aaaaaa
http://numericalmethods.eng.usf.edu
Example
From the 3rd equation
050170
7350735070
3
3
3
.a.
.a
.a.
=
=
=
Substituting in a3 and using the second equation
219656184 32 .a.a. −=−−
( )
701984
0501561219684
5612196
2
2
32
.a.
...a
.a..a
=−+−
=
−+−
=
http://numericalmethods.eng.usf.edu
ExampleSubstituting in a3 and a2 using the first equation
8106525 321 .aaa =++
Hence the Solution Vector is:
=
050.170.19
2900.0
3
2
1
aaa
( )
2900025
05017019581062558106 32
1
.
...
aa.a
=
−−=
−−=
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
The inverse [B] of a square matrix [A] is defined as
[A][B] = [I] = [B][A]
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrixHow can LU Decomposition be used to find the inverse?Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication
First column of [B] Second column of [B]
[ ]
=
0
01
1
21
11
nb
bb
A [ ]
=
0
10
2
22
12
nb
bb
A
The remaining columns in [B] can be found in the same manner
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixFind the inverse of a square matrix [A]
[ ]
=
11214418641525
A
[ ] [ ][ ]
−−
==
7000561840
1525
15376501562001
.
.. ..
.ULA
Using the decomposition procedure, the [L] and [U] matrices are found to be
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixSolving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]
Step 1: [ ][ ] [ ]
=
→=
001
15.376.50156.2001
3
2
1
zzz
CZL
This generates the equations:
05.376.5056.21
321
21
1
=++=+=
zzzzzz
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixSolving for [Z]
( )
( ) ( )23
5625317650537650
56215620
56201
213
12
1
....
z.z.z.
.z. z
z
=−−−=
−−=−=−=−=
=
[ ]
−=
=
23562
1
3
2
1
..
zzz
Z
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving [U][X] = [Z] for [X]
−=
−−
3.22.561
7.00056.18.40
1525
31
21
11
bbb
2.37.056.256.18.4
1525
31
3121
312111
=−=−−
=++
bbbbbb
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixUsing Backward Substitution
( )
( ) 04762.025
571.49524.05125
51
9524.08.4
571.4560.156.28.4560.156.2
571.47.02.3
312111
3121
31
=−−−
=
−−=
−=−
+−=
−+−
=
==
bbb
bb
b So the first column of the inverse of [A] is:
−=
571.49524.0
04762.0
31
21
11
bbb
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixRepeating for the second and third columns of the inverse
Second Column Third Column
=
010
11214418641525
32
22
12
bbb
−
−=
000.5417.108333.0
32
22
12
bbb
=
100
11214418641525
33
23
13
bbb
−=
429.14643.0
03571.0
33
23
13
bbb
http://numericalmethods.eng.usf.edu
Example: Inverse of a MatrixThe inverse of [A] is
[ ]
−−−
−=−
429.1000.5571.44643.0417.19524.0
03571.008333.004762.01A
To check your work do the following operation
[A][A]-1 = [I] = [A]-1[A]
Additional ResourcesFor all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html