Combinatorial Algorithms in Scientific Computing MS110
Transcript of Combinatorial Algorithms in Scientific Computing MS110
![Page 1: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/1.jpg)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Combinatorial Algorithms in Scientific Computing
MS110
Robert Kirby (Texas Tech)
Michael Wolf (UIUC, Sandia)
2008 SIAM Annual Meeting
7/11/2008
![Page 2: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/2.jpg)
2
Combinatorial Scientific Computing (CSC)
• Long played important role in enabling scientific
computing
• Traditionally researchers spread across several
different communities
• Recent effort to form more cohesive CSC
community
• CSC Workshops
– San Francisco, 2/2004 (SIAM PP04)
– Toulouse, 6/2005
– Costa Mesa, 2/2007 (SIAM CSE07)
– Monterey, 10/2009 (SIAM LA09)
![Page 3: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/3.jpg)
3
Combinatorial Scientific Computing (CSC)
• Numerous conference minisymposia
• DOE SciDAC institute
– Combinatorial Scientific Computing and Petascale
Simulations (CSCAPES)
• For more info on CSC activities
– http://www.cscapes.org/
![Page 4: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/4.jpg)
4
Combinatorial Algorithms in Scientific Computing
• Inherently combinatorial tasks in scientific
computing
– e.g., data partitioning
• More subtle underlying discrete structures
in scientific computing
– Complements analytic structure of problem
– e.g., combinatorial structure in discretized PDEs
![Page 5: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/5.jpg)
5
Combinatorial Algorithms in Scientific Computing
• Michael Wolf
– “Hypergraph-based combinatorial optimization of
matrix-vector multiplication”
• Dmitry Karpeev
– “Using Sieve for particle tracking, embedding meshing
and field-particle interaction computations”
• Kevin Long
– “Combinatorial dataflow analysis for differentiation of
high-level PDE representations”
• Andrew Lyons
– “Exploitation of Jacobian scarcity”
![Page 6: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/6.jpg)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Hypergraph-Based Combinatorial Optimization of
Matrix-Vector Multiplication
Michael M. Wolf
University of Illinois,
Sandia National Laboratories
2008 SIAM Annual Meeting
7/11/2008
![Page 7: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/7.jpg)
7
Optimization Problem
Objective: Generate set of operations forcomputing matrix-vector product with minimalnumber of multiply-add pairs (MAPs)
![Page 8: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/8.jpg)
8
Motivation
• Reducing redundant operations in building finiteelement (FE) stiffness matrices– Reuse optimized code when problem is rerun
Based on reference
element, generate code to
optimize construction of
local stiffness matrices
Can use optimized code
for every element in
domain
![Page 9: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/9.jpg)
9
Related Work
• Finite element “Compilers” (FEniCS project)– www.fenics.org
– FIAT (automates generations of FEs)
– FFC (variational forms -> code for evaluation)
• Following work by Kirby, et al., Texas Tech,University of Chicago on FErari– Optimization of FFC generated code
– Equivalent to optimizing matrix-vector product code
– Graph model
![Page 10: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/10.jpg)
10
Matrix-Vector Multiplication
where
Element
dependentElement
independent
For 2D Laplace equation, we obtain following matrix-vector product to determine entries in local
stiffness matrix
![Page 11: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/11.jpg)
11
Possible Optimizations - Collinear Rows
0
8
2 22
3 33 0
5 55
2 22 0
![Page 12: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/12.jpg)
12
Possible Optimizations - Collinear Rows
0
8
2 22
3 33 0
5 55
2 22 0
1 MAP
![Page 13: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/13.jpg)
13
Possible Optimizations - Collinear Rows
0
8
2 22
3 33 0
5 55
2 22 0
0 MAPs
Special case when
rows identical
![Page 14: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/14.jpg)
14
Possible Optimizations - Partial Collinear Rows
0
8
2 22
3 33 0
5 55
2 22 0
![Page 15: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/15.jpg)
15
Possible Optimizations - Partial Collinear Rows
0
8
2 22
3 33 0
5 55
2 22 0
2 MAPs
![Page 16: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/16.jpg)
16
Graph Model - Resulting Vector Entry Vertices
• Entries in resulting vector represented byvertices in graph model
![Page 17: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/17.jpg)
17
Graph Model - Inner-Product Vertex and Edges
• Additional inner-product (IP) vertex
• Edges connect IP vertex to every other vertex,representing inner-product operation
![Page 18: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/18.jpg)
18
Graph Model - Row Relationship Edges
• Operations resulting from relationships betweenrows represented by edges betweencorresponding vertices
![Page 19: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/19.jpg)
19
Graph Model - Edge Weights
• Edge weights are MAP costs for operations
![Page 20: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/20.jpg)
20
Graph Model Solution
• Solution is minimum spanning tree (MST)– Minimum cost subgraph
– Connected and spans vertices
– Acyclic
MST(5)GraphMatrix
![Page 21: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/21.jpg)
21
Graph Model Solution
• Prim’s algorithm to find MST (polynomial time)• MST traversal yields operations to optimally
compute (for these relationships) matrix-vectorproduct
InstructionsMST TraversalMatrix
![Page 22: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/22.jpg)
22
Graph Model Example
• Matrix used for building FE localstiffness matrices– 2D Laplace Equation
– 2nd order Lagrange polynomial basis
• Simplified version of matrix– Identical rows removed– Several additional rows removed
![Page 23: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/23.jpg)
23
Graph Model Example - Vertices
![Page 24: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/24.jpg)
24
Graph Model Example - Inner Product Edges
![Page 25: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/25.jpg)
25
Graph Model Example - Collinear Edges
![Page 26: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/26.jpg)
26
Graph Model Example - Partial Collinear Edges
![Page 27: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/27.jpg)
27
Graph Model Example - Final Graph
![Page 28: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/28.jpg)
28
Graph Model Example - Solution (MST)
![Page 29: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/29.jpg)
29
Graph Model Example - Instructions Generated
Matrix (16 nz) MST traversal Instructions (9 MAPs)
![Page 30: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/30.jpg)
30
Graph Model Results - 2D Laplace Equation
• Graph model shows significant improvement overunoptimized algorithm
60% decrease
![Page 31: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/31.jpg)
31
Graph Model Results - 3D Laplace Equation
• Again graph model requires significantly fewerMAPs than unoptimized algorithm
59% decrease
![Page 32: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/32.jpg)
32
Limitation of Graph Model
1
0
1 324 44 4
2 020 20 2
• Edges connect 2 vertices• Can represent only binary row relationships• Cannot exploit linear dependency of more than
two rows• Thus, hypergraphs needed
![Page 33: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/33.jpg)
33
Hypergraph Model
• Same edges (2-vertex hyperedges) as graph model• Additional higher cardinality hyperedges for more
complicated relationships– Limiting to 3-vertex linear dependency hyperedges for
this talk
![Page 34: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/34.jpg)
34
Hypergraph Model Solution: Modified Prim’s
• Extended Prim’s algorithm to include hyperedges• Polynomial time algorithm• Solution not necessarily a tree
– {IP,1,3,5}
– {IP,2,4,5}
• No guarantee of optimum solution
![Page 35: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/35.jpg)
35
Hypergraph Model Solution: Modified Prim’s
• No guarantee of optimum solution
9 MAPs
8 MAPs
![Page 36: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/36.jpg)
36
Hypergraph Model Results - 2D Laplace Equation
• Hypergraph solution shows modest improvementover graph solution
• Graph algorithm solutions close to optimal forsome orders?– 3 Columns
14% additional
decrease
![Page 37: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/37.jpg)
37
Hypergraph Model Results - 3D Laplace Equation
• Hypergraph solution significantly better thangraph solution for many orders
28% additional
decrease
![Page 38: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/38.jpg)
38
Ongoing Work: New Hypergraph Method(s)
• Greedy modified Prim’s algorithm yieldssuboptimal solutions for hypergraphs
• Want improved method that yields better
(or optimal) solutions–Improved solution
–Optimality of greedy solution
• New approach: formulate as vertex ordering
![Page 39: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/39.jpg)
39
Ongoing Work: Vertex Ordering Method
• Order vertices–Roughly represents order of calculation for entries
• For given vertex ordering, can determineoptimal solution subhypergraph–Greedy algorithm of selecting cheapest availablehyperedge
–Fast!
• Ordering is challenging part
• Traversal of greedy solution good starting pt.
• Implemented very simple local refinement–Local refinement on starting point
![Page 40: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/40.jpg)
40
Vertex Ordering: Preliminary Results — 2D Laplace
• Simple local refinement method– Pairwise swapping to improve initial ordering
• Slight additional improvement
• Graph/hypergraph solutions close to optimal?– 3 Columns
4% additional
decrease
![Page 41: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/41.jpg)
41
Vertex Ordering: Preliminary Results — 3D Laplace
9% additional
decrease
• Simple local refinement method– Pairwise swapping to improve initial ordering
• Slight additional improvement
• Perhaps more improvement with global vertexordering method
![Page 42: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/42.jpg)
42
Future Work
• Higher cardinality hyperedges- Perhaps useful for 3D FE problems- Implemented 4, 5, 6 vertex hyperedges- Hyperedge explosion- Need efficient hyperedge pruning algorithms
• Improve hypergraph solution methods– Develop global vertex ordering method
• Focus on reducing runtime of resulting operations– Best feasible vertex ordering for given solution
• Other matrices
![Page 43: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/43.jpg)
43
Acknowledgements/Thanks
• Michael Heath, advisor• Robert Kirby, Texas Tech University• Erik Boman, Sandia National Laboratories• Bruce Hendrickson, Sandia National Laboratories• Funding
– DOE CSGF, DE-FG02-97ER25308– SIAM -- travel support– CSCAPES (Sandia) -- travel support
![Page 44: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/44.jpg)
44
Extra
![Page 45: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/45.jpg)
45
2D Laplace Equation Matrices
• 3 Columns
![Page 46: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/46.jpg)
46
3D Laplace Equation Matrices
• 6 Columns
![Page 47: Combinatorial Algorithms in Scientific Computing MS110](https://reader031.fdocuments.in/reader031/viewer/2022020706/61fca49e9d50e757a522010c/html5/thumbnails/47.jpg)
47
Accuracy
Relative Error 2D Laplace Relative Error 3D Laplace
• Single precision input matrices
• Single precision code generation