Direct methods forsparse linear systems

SeminarSummer semester 2017

Andreas Potschka

Heidelberg University

April 19, 2017

A. Potschka Direct methods for sparse linear systems – 1

Overview

Organizational matters

Introduction

List of topics

Preparation guidelines for presentations

Introductory round

A. Potschka Direct methods for sparse linear systems – 2

Organizational matters

I Wednesdays, 14–16 UhrI Kickoff: April 19I Location: INF 205, SR1

I Target group: MScI MathematicsI Scientific computingI Computer science

I Language: EnglishI One presentation per session (45–75 min plus discussion)

I Credit Points: 6 CPI Prerequisites: Presentation, regular attendance

A. Potschka Direct methods for sparse linear systems – 3

Grading criteria

I Quality of contentsI Mathematical precisionI Focus on the essential aspects, adapted to audienceI Clear structure

I Presentation styleI Comprehensible pronounciationI Adequate tempo of presentationI Responsiveness to questions from the audience

I Presentation techniqueI Choice: Black board, PowerPoint, LATEXbeamer, etc.I Readable, well-structured, meaningful black board and slidesI Focus on one message per slideI Nominal style instead of full sentencesI Avoid clutterI Handout

A. Potschka Direct methods for sparse linear systems – 4

Sparse matrices

I Matrices with many zero entriesI Simple examples: 0 matrix, identity matrix, band matricesI Memory requirement for sparse n×n matrix: O(n) instead of O(n2)

I Requires special data structuresI Sparsity pattern connected to graphsI Arise in many application problems

A. Potschka Direct methods for sparse linear systems – 5

Applications: Networks

Source: http://www.cise.ufl.edu/research/sparse/matrices/

A. Potschka Direct methods for sparse linear systems – 6

Applications: Circuits

Source: http://www.cise.ufl.edu/research/sparse/matrices/

A. Potschka Direct methods for sparse linear systems – 7

Applications: Linear programming

Source: http://www.cise.ufl.edu/research/sparse/matrices/

A. Potschka Direct methods for sparse linear systems – 8

Applications: Nonlinear programming

Source: http://www.cise.ufl.edu/research/sparse/matrices/

A. Potschka Direct methods for sparse linear systems – 9

Applications: Partial differential equations

Source: http://www.cise.ufl.edu/research/sparse/matrices/

A. Potschka Direct methods for sparse linear systems – 10

Linear equations

Ax = bSolution alternatives:

I Direct methods1. Decomposition: A = LU, A = QR, A = LLT

2. Forward/backward substitutionI To minimize fill-in: Analyze and permuteI Alternative: Iterative methods

fixed-point solvers, Krylov subspace methods, multi-grid, . . .(Seminar Iterative methods for sparse linear systems)

A. Potschka Direct methods for sparse linear systems – 11

Topic: Crash course in graph theory

1R. Diestel. Graph theory. 4th ed. Graduate texts in mathematics. Springer, 2012.2T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals of

Algorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 4–6.

A. Potschka Direct methods for sparse linear systems – 12

Topic: Basics of sparse matrices

I Memory formatsI Matrix modifications and arithmeticI Solution of triangular systems

3R. Diestel. Graph theory. 4th ed. Graduate texts in mathematics. Springer, 2012.4T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals of

Algorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 7–35.

A. Potschka Direct methods for sparse linear systems – 13

Topic: Cholesky decomposition

I A symmetric positive definiteI A = LLT

5T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 37–67.

A. Potschka Direct methods for sparse linear systems – 14

Topic: Orthogonal decomposition

I A = QR,QTQ = II Householder reflectorsI Givens rotations

6T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 69–82.

7G.H. Golub and C.F. van Loan. Matrix Computations. 3rd ed. Baltimore: JohnsHopkins University Press, 1996, pp. 206–247.

A. Potschka Direct methods for sparse linear systems – 15

Topic: LU decomposition

I A = LU

I UMFPACK: Matlab \

8T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 83–94.

9T.A. Davis. “Algorithm 832: UMFPACK – an unsymmetric-pattern multifrontal methodwith a column pre-ordering strategy”. In: ACM Trans. Math. Software 30 (2004),pp. 196–199.

A. Potschka Direct methods for sparse linear systems – 16

Topic: Minimum degree ordering

I Preserving sparsity of matrix factorsI Minimum degree ordering

10T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 99–112.

A. Potschka Direct methods for sparse linear systems – 17

Topic: Maximum matching

I Preserving sparsity of matrix factorsI Maximum matchingI Dulmage–Mendelsohn decomposition

11T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 112–126.

A. Potschka Direct methods for sparse linear systems – 18

Topic: Profile reduction, nested dissection, solution

I Preserving sparsity of matrix factorsI Bandwidth and profile reductionI Nested dissectionI Solution of decomposed systems

12T.A. Davis. Direct methods for sparse linear systems. Vol. 2. Fundamentals ofAlgorithms. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM), 2006,pp. 127–139.

A. Potschka Direct methods for sparse linear systems – 19

Topic: Implicit LU decomposition

I Decomposition with possibility of updates

13R. Fletcher. “Approximation Theory and Optimization. Tributes to M.J.D. Powell”. In:ed. by M.D. Buhmann and A. Iserles. Cambridge University Press, 1997. Chap. Densefactors of sparse matrices, pp. 145–166.

A. Potschka Direct methods for sparse linear systems – 20

List of topics

Nr Date Topic Name1 10.05.2017 Crash course in graph theory2 17.05.2017 Basics of sparse matrices3 24.05.2017 Cholesky decomposition4 31.05.2017 Orthogonal decomposition5 07.06.2017 LU decomposition6 21.06.2017 Minimum degree ordering7 28.06.2017 Maximum matching8 05.07.2017 Profile reduction, nested dissection, solution9 12.07.2017 Implicit LU decomposition

A. Potschka Direct methods for sparse linear systems – 21

Preparation guidelines for presentations

I Who is my audience?Imagine one or two concrete persons!

I How much time do I have?I Structure: Overview, main part, summaryI One week before presentation:

Meet me to discuss slides/black boardI Your presentation is more than your slides

Deliver at least one, better two exercise presentations

A. Potschka Direct methods for sparse linear systems – 22

Introductory round

I NameI CountryI SemesterI Study programI Possible topics for seminar presentation

A. Potschka Direct methods for sparse linear systems – 23