SIAM CSE’03 Combinatorial Scientific Computing: The Role of Discrete Algorithms in Computational...

SIAM CSE’03

Combinatorial Scientific Computing:

The Role of Discrete Algorithms in

Computational Science & Engineering

Bruce Hendrickson

Sandia National Labs

Discrete Algorithms & Math Department

MIT 4/03

»Compilers, system software, computer architecture, etc.

CS and CS&E

Computer Science Computational Science

What’s in the intersection?


MIT 4/03

» Combinatorial Scientific Computing

CS and CS&E

What’s in the intersection?

Computer Science Computational ScienceAlgorithmics


MIT 4/03

Combinatorial Scientific Computing

Development, application and analysis of combinatorial algorithms to enable scientific and engineering computations

TheoryPractice

This talk


MIT 4/03

World’s Apart

Computer Science =» Graph algorithms, set theory, complexity theory, etc.

Computational Science & Engineering =» Numerical analysis, PDEs, linear algebra, etc.

Differ in many ways» Vocabulary, concepts and abstractions» Culture – mathematics versus engineering» Definition of success » Aesthetics

Not an easy divide to span!


MIT 4/03

Sparse Direct Methods

Reorderings for sparse factorizations» Powerfully phrased as graph problems» Fill reducing orderings

– Minimum degree (greedy)– Nested dissection (divide & conquer)

» Bandwidth reducing orderings– graph traversals, graph eigenvectors

» Heavy diagonal to reduce pivoting (matching) Efficient exploitation of sparsity

» Factorization, triangular solves, etc.


MIT 4/03

Graphs and sparse Gaussian elimination (1961-)

10

13

2

4

5

6

7

8

9

10

13

2

4

5

6

7

8

9

G(A) G+(A)[chordal]

Cholesky factorization:for j = 1 to n add edges between j’s higher-numbered neighbors

Fill: new nonzeros in factor


MIT 4/03

Sparse Direct Methods

Reorderings for sparse factorizations» Powerfully phrased as graph problems» Fill reducing orderings

– Minimum degree (greedy)– Nested dissection (divide & conquer)

» Bandwidth reducing orderings– graph traversals, graph eigenvectors

» Heavy diagonal to reduce pivoting (matching) Efficient exploitation of sparsity

» Factorization, triangular solves, etc.


MIT 4/03

Preconditioning

Incomplete Factorizations» Exploiting sparsity patterns, e.g. level-of-fill» Orderings

Partitioning for domain decomposition Graph techniques in algebraic multigrid

» Independent sets, matchings, etc.

Support Theory» Spanning trees & graph embedding techniques


MIT 4/03

Numerical Optimization

Sparse Jacobian Evaluation» Exploit sparsity to minimize function calls» Graph coloring on column intersection graph

Sparse basis construction» Matroids, graph colorings, spanning trees, etc.

Hybrid of combinatorics and numerics


MIT 4/03

ParallelizingScientific

Computations

Graph Algorithms» Partitioning» Coloring» Independent sets, etc.

Geometric algorithms» Space-filling curves & octrees for particles» Geometric partitioning

Reordering for memory locality


MIT 4/03

Parallelization Strategies

Observation: Parallelization is usually orthogonal to numerics

Issues are non-numerical» Load balancing» Communication minimization» Scheduling, etc.

Almost invariably combinatorial in spirit


MIT 4/03

Mesh Generation

Geometric algorithms &

data structures» Delaunay/Voronoi decompositions» Convex hulls» Intersection checking, etc.

Topology of unstructured meshes» graph algorithms


MIT 4/03

More Mesh Generation

Rich amalgam of mathematical ideas

Differential geometry Harmonic mappings & numerical PDEs Optimization to improve mesh quality


MIT 4/03

Computational Biology

Genomics» Fragment assembly» Sequence analysis, etc.» Lots of string algorithms

Proteomics» Structural comparisons» NMR and Mass Spec analysis

Phylogenics Literature mining Microarray clustering & analysis Etc, etc.


MIT 4/03

Statistical Physics

Ising spin models and percolation theory» Pfaffians, permanents & matching» Very rich graph theory» Several Nobel prizes awarded

Other percolation models» External fields, Connectivity, Rigidity, etc. » Network flow and other graph algorithms


MIT 4/03

Graphs in Chemisty

Categorizing molecules by graph properties» Various topological invariants, graph properties» Used to screen molecules for desired properties

Combinatorics of polymers» Geometric and graph properties

Statistically correct ensembles» Graph enumeration and sampling


MIT 4/03

CS&E Techniques in Computer Science

Continuous methods in discrete optimization » Using matrix eigenvectors to understand graphs» Approximation algorithms via linear or quadratic

programming Linear algebra in information analysis

» Latent semantic indexing (SVD for info retrieval)» Google’s page ranking (eigenvector & Perron-Frobenius)» Kleinberg’s hubs and authorities (SVD)

Multilevel combinatorial algorithms » Dominant paradigm for practical graph partitioning » Being applied to range of combinatorial problems» Origins in algebraic multigrid


MIT 4/03

The Future …

“It’s tough to make predictions, especially about the future”

Yogi Berra

Prediction: All of the aforementioned and more.


MIT 4/03

Info Organization, Analysis & Mining

Graph algorithms and linear algebra» Importance ranking of documents/pages» Information retrieval

Publication mining is key tool in biology» Text analysis & inference

Simulation output already overwhelming» Learning theory» Advanced visualization


MIT 4/03

More Biology

Gene promotion and inhibition» Strings and learning theory

Multi-atom interactions» Protein complexes» Regulatory networks» Geometry, topology and graphs

Biological systems» Whole cell modeling» Ecological models» Topology and graph analysis


MIT 4/03

Fast Algorithms for Huge Problems

For computer science theorists, key distinction is between polynomial & exponential time

For scientific computing, key is often linear versus quadratic time


MIT 4/03

Examples

Approximate max-weight matching» [Monien, Preis, Diekmann], [Drake, Hougardy]» Useful for partitioning» Exact algorithm O(mn) time» 2-approximation in O(m) time

Extreme Case:“Can we understand anything interesting about our data

when we do not even have time to read all of it?” - Ronitt Rubinfeld


MIT 4/03

Sublinear Time Algorithms

Fast Monte Carlo algorithms for finding low-rank approximations to a matrix» [Frieze, Kannan, Vempala]

» Find B0 such that: ||A – B0||F minB ||A - B||F + ε ||A||F

» Run-time independent of size of matrix!

Approximating weight of MST in sublinear time » [Chazelle, Rubinfeld, Trevisani] » Key idea: estimate number of connected components in

time independent of size of graph


MIT 4/03

Hard Questions

How will combinatorial methods be used by people who don’t understand them in detail?

What are the implications …» for teaching?» for software development?» for journals?» for professional societies?


MIT 4/03

Morals

Things are clearer if you look at them from multiple perspectives

Combinatorial algorithms are pervasive in scientific computing and will become more so

Lots of exciting opportunities» High impact for discrete algorithms work» Enabling for scientific computing


MIT 4/03

Thanks

Yogi Berra, Erik Boman, Edmond Chow, Karen Devine, Jean-Loup Faulon, John Gilbert, Mike Heath, Pat Knupp, Esmond Ng, Ali Pınar, Steve Plimpton, Cindy Phillips, Alex Pothen, Robert Preis, Padma Raghavan, Jonathan Shewchuk, Shang-Hua Teng, Sivan Toledo, etc.


MIT 4/03

For More Information

www.cs.sandia.gov/~bahendr

lists.odu.edu/listinfo/csc

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed-Martin Company, for the US DOE under contract DE-AC-94AL85000

SIAM CSE’03 Combinatorial Scientific Computing: The Role of Discrete Algorithms in Computational...

Documents

Transcript of SIAM CSE’03 Combinatorial Scientific Computing: The Role of Discrete Algorithms in Computational...