Parallel Computing Sciences Department

MOV’01

Multilevel Combinatorial Methods in

Scientific Computing

Bruce Hendrickson

Sandia National Laboratories

Parallel Computing Sciences Dept.

Parallel Computing Sciences Department

MOV’01

An Overdue Acknowledgement

Parallel Computing Uses Graph Partitioning

We owe a deep debt to circuit researchers» KL/FM» Spectral partitioning» Hypergraph models» Terminal propagation

Parallel Computing Sciences Department

MOV’01

In Return …

We’ve given you» Multilevel partitioning» hMETIS

Our applications are different from yours» Underlying geometry» More regular structure» Bounded degree» Partitioning time is more important

– Different algorithmic tradeoffs

Parallel Computing Sciences Department

MOV’01

Multilevel Discrete Algorithm

Explicitly mimic traditional multigrid

Construct series of smaller approximations» Restriction

Solve on smallest» Coarse grid solve

Propagate solution up the levels» Prolongation

Periodically perform local improvement» Smoothing

Parallel Computing Sciences Department

MOV’01

Lots of Possible Variations

More complex multilevel iterations» E.g. V-cycle, W-cycle, etc.» Not much evidence of value for discrete problems

Key issue: properties of coarse problems» Local refinement = multi-scale improvement

I’ll focus on graph algorithms» Most relevant to VLSI problems

Parallel Computing Sciences Department

MOV’01

Not a New Idea

Idea is very natural» Reinvented repeatedly in different settings

Focus of this workshop is on heuristics for hard problems

Technique also good for poly-time problems» E.g. Geometric point detection (Kirkpatrick’83)

Parallel Computing Sciences Department

MOV’01

Planar Point Detection

O(n log n) time to preprocess

O(log n) time to answer query

Parallel Computing Sciences Department

MOV’01

Multilevel Graph Partitioning

Invented Independently Several Times» Cong/Smith’93» Bui/Jones’93» H/Leland’93» Related Work

– Garbers/Promel/Steger’90, Hagen/Khang’91, Cheng/Wei’91

– Kumar/Karypis’95, etc, etc.

Multigrid Metaphor H/Leland’93 (Chaco)» Popularized by Kumar/Karypis’95 (METIS)

Parallel Computing Sciences Department

MOV’01

Multilevel Partitioning

Construct Sequence of Smaller Graphs Partition Smallest Project Partition Through Intermediate Levels

» Periodically Refine

Why does it work so well?» Refinement on multiple scales (like multigrid)» Key properties preserved on (weighted) coarse graphs

– (Weighted) partition sizes

– (Weighted) edge cuts

» Very fast

Parallel Computing Sciences Department

MOV’01

Coarse Problem Construction

1. Find maximal matching2. Contract matching edges3. Sum vertex and edge weights

Key Properties:Preserves (weighted) partition sizesPreserves (weighted) edge cutsPreserves planarityRelated to min-cut algorithm of Karger/Stein’96

Parallel Computing Sciences Department

MOV’01

Extension I:Terminal Propagation

Dunlop/Kernighan’85» Skew partitioning to address constrained vertices

Also useful for parallel computing» Move few vertices when repartitioning» Assign neighboring vertices to near processors» H/Leland/Van Dreissche’96

Basic idea:» Vertex has gain-like preference to be in particular partition

Parallel Computing Sciences Department

MOV’01

MultilevelTerminal Propagation

How to include in multilevel algorithm?

Simple idea:» When vertices get merged, sum preferences» Simple, fast, effective» Coarse problem precisely mimics original

Parallel Computing Sciences Department

MOV’01

Extension II:Finding Vertex Separators

Useful for several partitioning applications» E.g. sparse matrix reorderings

One idea: edge separator first, then min cover» Problem: multilevel power on wrong objective

Better to reformulate multilevel method» Find vertex separators directly» H/Rothberg’98

Parallel Computing Sciences Department

MOV’01

MultilevelVertex Separators

Use same coarse constructor» Except edge weights don’t matter

Change local refinement & coarse solve» Can mimic KL/FM

Resulted in improved matrix reordering tool» Techniques now standard

Parallel Computing Sciences Department

MOV’01

Extension III:Hypergraph Partitioning

Coarse construction» Contract pairs of vertices?» Contract hyperedges?

Traditional refinement methodology

See talk tomorrow by George Karypis

Parallel Computing Sciences Department

MOV’01

Envelope Reduction

Reorder rows/columns of symmetric matrix to keep nonzeros near the diagonal

xxx

xxx

xxx

xxx

x

xx

x

xx

xx

x

xxx

Parallel Computing Sciences Department

MOV’01

Graph Formulation

Each row/column is a vertex Nonzero in (i,j) generates edge eij

For row i of matrix (vertex i)» Env(i) = max(i-j such that eij in E)

» Envelope = Env(i)

Find vertex numbering to minimize envelope» NP-Hard

Parallel Computing Sciences Department

MOV’01

Status

Highest Quality algorithm is spectral ordering» Sort entries of Fiedler vector (Barnard/Pothen/Simon’95)

» Eigenvector calculation is expensive» Fast Sloan (Kumfert/Pothen’97) good compromise

Now multilevel methods are comparable» (Boman/H’96, Hu/Scott’01)

Related ordering problems with VLSI relevance» Optimal Linear Arrangement

– Minimize |i-j| such that eij in E

Parallel Computing Sciences Department

MOV’01

Challenges for MultilevelEnvelope Minimization

No precise coarse representation» Can’t express exact objective on coarse problem

No incremental update for envelope metric» I.e. no counterpart of Fiduccia/Mattheyses

Our solution: Use approximate metric» 1-sum / minimum linear arrangement» Allows for incremental update

– But still not an exact coarse problem

» VLSI applications?

Parallel Computing Sciences Department

MOV’01

Results

Disappointing for envelope minimization» We never surpassed best competitor

– Fast Sloan algorithm

» But Hu/Scott’01 succeeded with similar ideas

Better for linear arrangement, but …» Not competitive with Hur/Lillis’99

– Multilevel algorithm with expensive refinement

Parallel Computing Sciences Department

MOV’01

Lessons Learned

Good coarse model is the key» Need to encode critical properties of full problem» Progress on coarse instance must help real one» Must allow for efficient refinement methodology» Different objectives require different coarse models

Quality/Runtime tradeoff varies w/ application» Must understand needs of your problem domain» For VLSI, quality is worth waiting for» All aspects of multilevel algorithm are impacted

Parallel Computing Sciences Department

MOV’01

Conclusions

Appropriate coarse representation is key» Lots of existing ideas for construction coarse problem

– Matching contraction, independent sets, fractional assignment, etc.

Multigrid metaphor provides important insight» We’re not yet fully exploiting multigrid possibilities» Do we have something to offer algebraic multigrid?

Need for CS recognition of multilevel paradigm» Rich, general algorithmic framework, but not in any textbook» Not the same as divide-and-conquer

Parallel Computing Sciences Department

MOV’01

Acknowledgements

Shanghua Teng» "Coarsening, Sampling and Smoothing: Elements of the

Multilevel Method“

Rob Leland Erik Boman Ed Rothberg Tammy Kolda Chuck Alpert

DOE MICS Office