7/29/2019 Sparse Day 1 LU FACTOR
1/17
Sparse Matr ix Methods
Day 1: Overview
Matlab and examples
Data structures
Ax=b Sparse matrices and graphs
Fill-reducing matrix permutations
Matching and block triangular form
Day 2:Direct methods
Day 3: Iterative methods
D
7/29/2019 Sparse Day 1 LU FACTOR
2/17
(Demo in Matlab)
Matlab sprand
spy
sparse, full
matrix arithmetic and indexing
examples of sparse matrices from different
applications (from UF site)
7/29/2019 Sparse Day 1 LU FACTOR
3/17
Matlab sparse matr ices: Design pr inc ip les
All operations should give the same results forsparse and full matrices (almost all)
Sparse matrices are never created automatically,but once created they propagate
Performance is important -- but usability, simplicity,completeness, and robustness are more important
Storage for a sparse matrix should be O(nonzeros)
Time for a sparse operation should be O(flops)(as nearly as possible)
7/29/2019 Sparse Day 1 LU FACTOR
4/17
Data structu res
Full:
2-dimensional array ofreal or complex numbers
(nrows*ncols) memory
31 0 53
0 59 0
41 26 0
31 41 59 26 53
1 3 2 3 1
Sparse:
compressed columnstorage
about (1.5*nzs + .5*ncols)memory
D
7/29/2019 Sparse Day 1 LU FACTOR
5/17
(Demos in Matlab)
A \ b Cholesky factorization and orderings
7/29/2019 Sparse Day 1 LU FACTOR
6/17
Solving l inear equat ion s
x = A \ b;
Is A square?
no => use QR to solve least squares problem
Is A triangular or permuted triangular?yes => sparse triangular solve
Is A symmetric with positive diagonal elements?
yes => attempt Cholesky after symmetric minimum degree
Otherwise
=> use LU on A(:, colamd(A))
7/29/2019 Sparse Day 1 LU FACTOR
7/17
Matr ix facto r izat ion s in Matlab
Cholesky: R = chol(A);
simple left-looking column algorithm
Nonsymmetric LU:
[L,U,P] = lu(A);
left-looking GPMOD, depth-first search, symmetric pruning
Orthogonal:
[Q,R] = qr(A);
George-Heath algorithm: row-wise Givens rotations
7/29/2019 Sparse Day 1 LU FACTOR
8/17
Graphs and Sparse Matr ices: Cholesky factor izat ion
10
13
2
4
5
6
7
8
9
10
13
2
4
5
6
7
8
9
G(A) G+(A)[chordal]
Symmetric Gaussian elimination:
for j = 1 to n
add edges between js
higher-numbered neighbors
Fill:new nonzeros in factor
7/29/2019 Sparse Day 1 LU FACTOR
9/17
Elim inat ion Tree
10
13
2
4
5
6
7
8
9
Cholesky factor G+(A) T(A)
10
1
3
2
45
6
7
8
9
T(A) : parent(j) = min { i > j : (i,j) inG+(A) }
T describes dependencies among columns of factor
Can compute T from G(A) in almost linear time
Can compute G+(A) easily from T
D
7/29/2019 Sparse Day 1 LU FACTOR
10/17
(Demos in Matlab)
matrix and graph elimination tree
orderings in detail
7/29/2019 Sparse Day 1 LU FACTOR
11/17
Fil l -reduc ing m atr ix permutat ions
Minimum degree:
Eliminate row/col with fewest nzs, add fill, repeat Theory: can be suboptimal even on 2D model problem
Practice: often wins for medium-sized problems
Nested dissection:
Find a separator, number it last, proceed recursively Theory: approx optimal separators => approx optimal fill and flop count
Practice: often wins for very large problems
Banded orderings (Reverse Cuthill-McKee, Sloan, . . .):
Try to keep all nonzeros close to the diagonal
Theory, practice: often wins for long, thin problems
Best modern general-purpose orderings are ND/MD hybrids.
7/29/2019 Sparse Day 1 LU FACTOR
12/17
Fil l -reduc ing permutat ions in Matlab
Nonsymmetric approximate minimum degree:
p = colamd(A); column permutation: lu(A(:,p)) often sparser than lu(A)
also for QR factorization
Symmetric approximate minimum degree:
p = symamd(A);
symmetric permutation: chol(A(p,p)) often sparser than chol(A)
Reverse Cuthill-McKee
p = symrcm(A);
A(p,p) often has smaller bandwidth than A
similar to Sparspak RCM
D
7/29/2019 Sparse Day 1 LU FACTOR
13/17
(Demos in Matlab)
nonsymmetric LU dmperm, dmspy, components
7/29/2019 Sparse Day 1 LU FACTOR
14/17
Matching and block tr iangular form
Dulmage-Mendelsohn decomposition:
Bipartite matching followed by strongly connected components
Square, full rank A:
[p, q, r] = dmperm(A);
A(p,q) has nonzero diagonal and is in block upper triangular form also, strongly connected components of a directed graph
also, connected components of an undirected graph
Arbitrary A:
[p, q, r, s] = dmperm(A); maximum-size matching in a bipartite graph
minimum-size vertex cover in a bipartite graph
decomposition into strong Hall blocks
7/29/2019 Sparse Day 1 LU FACTOR
15/17
Complexi ty of di rect methods
2D 3D
Space (fill): O(n log n) O(n 4/3 )
Time (flops): O(n 3/2 ) O(n 2 )
n1/2n1/3
7/29/2019 Sparse Day 1 LU FACTOR
16/17
The Landscape of Sparse Ax=b Solvers
Pivoting
LU
GMRES,
QMR,
Cholesky Conjugate
gradient
DirectA = LU
Iterativey = Ay
Non-
symmetric
Symmetric
positive
definite
More Robust Less Storage
More Robust
More General
D
7/29/2019 Sparse Day 1 LU FACTOR
17/17
(Demos in Matlab)
conjugate gradients
Top Related