Fast Linear Solvers for Laplacian Systems - UCSD Research...
Transcript of Fast Linear Solvers for Laplacian Systems - UCSD Research...
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Fast Linear Solvers for Laplacian SystemsUCSD Research Exam
Olivia Simpson
University of California, San Diego
November 8, 2013
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Definitions
Let G = (V ,E ) be an undirected edge-weighted graph on nvertices and m edges. We can generalize an unweighted graphto the case where all edge weights are equal to 1.The degree of a vertex v ∈ V is the sum of the weights of theedges adjacent to it,
dv =∑u∼v
w(u, v).
The degree matrix is the diagonal matrix whose entries are thedegrees of the vertices,
(D)vv = dv .
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Definitions
The adjacency matrix is the n × n matrix which is the weightw(e) in the entries corresponding to an edge,
(A)uv =
{w(u, v) if {u, v} ∈ E ,
0 otherwise.
The Laplacian is the matrix
L = D − A.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Example
For the followingsimple graph,
L =
2 0 0 0 00 2 0 0 00 0 4 0 00 0 0 1 00 0 0 0 3
−
0 0 1 0 10 0 1 0 11 1 0 1 10 0 1 0 01 1 1 0 0
=
2 0 −1 0 −10 2 −1 0 −1
−1 −1 4 −1 −10 0 −1 1 0
−1 −1 −1 0 3
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Properties of the Laplacian
• Symmetric
• Diagonally dominant
• All zero row-sums
• Non-positive off-diagonal entries
• Clean quadratic form:
xTLx =∑u∼v
(x(u)− x(v))2
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus
A network of n decision-making agents
• edges are communicationchannels
• xi is the decision value ofeach agent
• values can be influenced bycommunicating neighbors
• two agents vi , vj are said toagree when xi = xj
Goal: all agents reach a common decision value, called the con-sensus
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus
A network of n decision-making agents
The Laplacian potential is a mea-sure of disagreement in the system,
ΨG (x) =1
2xTLx
=1
2
∑vi∼vj
(xi − xj)2.
all agents agree ⇔ ΨG (x) = 0.
Goal: find the vector x which minimizes the Laplacian poten-tial [OM03]
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Why Laplacian Systems?
Laplacian Systems arise in a number of natural contexts,
• Characterizing the motion of coupled oscillators [HS08]
• Approximating Fiedler eigenvectors [ST04]
• ...
• Computing effective resistance in an electricalnetwork [Kir1847]
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
• Edges are wires in the network
• Edge weights correspond to the conductance of each wire
• Goal: set voltages x1, x2, x3 on the vertices to create aflow of electrical current
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Ohm’s law: take conductance(e) = 1/resistance(e), then
current(e) = conductance(e)× |voltage(u)− voltage(v)|
Consider the flow of current from V 1. By Ohm’s law:
current(V 1,V 2) = 1(voltage(V 1)− voltage(V 2))
current(V 1,V 3) = 2(voltage(V 1)− voltage(V 3))
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Kirchhoff’s law [Kir1847]: For every point in the network,
netflow(v) := flowin(v)− flowout(v) = 0,
except at the injection point, netflow = 1, and the extractionpoint, netflow = −1.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
So, at vertex V 1 the voltage x1 must satisfy:
netflow(V 1) = current(V 1,V 2) + current(V 1,V 3)
1 = 1(x1 − x2) + 2(x1 − x3)
1 = 3x1 − x2 − 2x3
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Applying the same rules at V 2 and V 3 yields the following sys-tem of equations:
3x1 − x2 − 2x3 = 1
−x1 + 2x2 − x3 = 0
−2x1 − x2 + 3x3 = −1
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3
x1
x2
x3
=
10−1
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3
x1
x2
x3
=
10−1
A system in the Laplacian of the network.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
When current is injected at V 1 and extracted at V 3, the solutionvector x can be used to compute the effective resistance of theedge (V 1,V 3),
R(V 1,V 3) = |x1 − x3|
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus Again
[OM03] A connected networkof decision-making agents
• edges are communicationchannels
• xi is the decision value ofeach agent
• values can be influenced bycommunicating neighbors
• two agents vi , vj are said toagree when xi = xj
Goal: all agents reach a common decision value, called theconsensus
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus Again
[OM03] A connected networkof decision-making agents
Suppose each agent evolves theirdecision value according to thedistributed linear protocol,
xi (t) =∑vj∼vi
xj(t)− xi (t).
Then the solution to the system
x = −Lx , x(0) ∈ Rn
is the vector of decision values as afunction of t.
Goal: all agents reach a common decision value, called theconsensus
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Directly
Given a system of linear equations, Ax = b, we can solve thesystem directly using Gaussian elimination.
Gaussian elimination takes O(n3) time in general.
- [CW90] show the exponent can be as small as 2.376
- [Wil12] improves this to 2.3727
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Iterative Methods
Iterative methods for solving systems Ax = b are based on asequence of increasingly better approximations.
The idea is to construct a sequencex (0), x (1), . . . , x (i), . . . , x (N), . . . that converges to the truesolution, x ,
limk→∞
x (k) = x .
In practice, the iterative process is stopped after N iterationswhen
||x (N) − x || < ε
for any vector norm and a prescribed error parameter ε.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Iterative Methods
Richardson’s Method([Young53],[GV61])
An iterative method that improves approximations using theresidual error, b − Ax (i), at each step:
x (i+1) = x (i) + (b − Ax (i))
= b + (I − A)x (i)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = b + (I − A)x (i)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = B−1b + (I − B−1A)x (i)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = B−1b + (I − B−1A)x (i)
The term B−1b involves solving a system in B.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
- One criterion uses the residual vector, r (i) = b − Ax (i)
- The minimum iterations N should satisfy
||x − x (N)||||x ||
≤ κ(A)||r (N)||||b||
≤ εκ(A),
where κ(A) = ||A−1|| · ||A|| is the condition number of A for anymatrix norm || · || [QRS07].
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
In the case of preconditioned methods, this becomes
||B−1r (N)||||B−1r (0)||
≤ ε.
In particular, this means the rate of convergence will depend onhow quickly systems in B can be solved.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Preconditioned Chebyshev method will find solutions withabsolute error ε in time
O(mS(B) log(κ(A)/ε)√κ(A,B)), [GO88]
where S(B) is the time required so solve a system in B and
κ(A,B) =
(max
x :Ax 6=0
xTAx
xTBx
)(max
x :Ax 6=0
xTBx
xTAx
).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Factors:
• minimum number of iterations N, depends on κ(A)
• time to solve the system in B
In general, worst case time bounds are O(mn).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:
solve(A,b):
if dimension-check(A) and sparse(A):
return PrecChebyshev(A,b)
B = precondition(A)
A’ = reduce(B)
solve(A’,b)
• Compute B, the preconditioner for A
• Reduce the system in B to a system in A1, a refinedversion of A
• Recursively solve the system in A1
• At the base, use the Preconditioned Chebyshev method
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:
A→ B → A1 → B1 → · · · → Ak
• Compute B, the preconditioner for A
• Reduce the system in B to a system in A1, a refinedversion of A
• Recursively solve the system in A1
• At the base, use the Preconditioned Chebyshev method
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
Two methods are used at each level of the recursion.
reduce(B): Bi → Ai+1
- Reduce the preconditioned matrix by greedily removing rowsand columns with at most two non-zero entries- Done with a partial Cholesky decomposition
precondition(A): Ai → Bi
- Sparsify the graph associated to A to obtain a subgraph H- Set B to be a matrix of H
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
A→ B → A1 → B1 → · · · → Ak
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
LA → LB → LA1 → LB1 → · · · → LAk
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
G → H → G1 → H1 → · · · → Gk
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Spanning Trees
[Vaidya91]: subgraphs serve as good preconditioners
• maximum weight spanning tree as a preconditioning base
• solves SDD systems with non-positive off-diagonal entriesof degree d in time
O(dn)1.75 log(κ(A)/ε)
[ST04]: subgraphs serve as good preconditioners
• a better base tree uses an edge measure called stretch
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
e = {V 1,V 4},w(e) = 1
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
A spanning tree, T
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
e = {V 1,V 4}, Tree path: {V 1,V 3}, {V 3,V 5}, {V 5,V 4}
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
Let T be a spanning tree of a weighted graph G , and let theweight of an edge e = {u, v} be denoted by w(e). Definew ′(e) = 1/w(e), which is the resistance of the edge, re .
Let e1, e2, . . . , ek be the unique path in T from u to v . Thenthe stretch of the edge by T is defined
stretchT (e) =
∑ki=1 w ′(e1)
w ′(e)= 1/re
k∑i=1
rei
The total stretch of a graph G by the tree T is the sum of thestretch of all the off-tree edges.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Low Stretch Spanning Trees
Low stretch spanning tree (LSST) for the preconditioning base↓
A “spine” that keeps resistance low
LSSTs for preconditioners had been implemented before([AKPW95, BH01, BH03]), but combining the high qualitypreconditioners with a recursive solver amounted to analgorithm for solving SDD systems in nearly-linear time.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Low Stretch Spanning Trees
Low stretch spanning tree (LSST) for the preconditioning base↓
A “spine” that keeps resistance low
LSSTs for preconditioners had been implemented before([AKPW95, BH01, BH03]), but combining the high qualitypreconditioners with a recursive solver amounted to analgorithm for solving SDD systems in nearly-linear time.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
ST-Solver
[ST04]: Given a system of equations Ax = b in a symmetric,diagonally dominant matrix, the ST-solver computes a vector xsatisfying
||Ax − b|| < ε and ||x − x || ≤ ε
in time O(m logO(1) m), where the exponent is a large constant.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
Sparsifier of the ST-solver
1 Compute an LSST T of the graph G
2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′
3 Replace T in G by T ′
4 H ← T ′
5 Add off-tree edges to H by sampling with probabilitiesrelated to vertex degree
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
Sparsifier of Koutis et al. [KMP10, KMP11]
1 Compute an LSST T of the graph G
2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′
3 Replace T in G by T ′
4 H ← T ′
5 Add off-tree edges to H by sampling with probabilitiesrelated to effective resistance
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
pro: Probabilities related to effective resistance yield sparsifierswith few edges [SS11]con: Computing effective resistances involves solving a systemof linear equations
This is problem is avoided by instead using upper bounds foredge probabilities,
pe ≥ weRe ,
where Re is the effective resistance of the edge.
The sparsifier of [KMP11] improved the expected time boundto O(m log2 n log(1/ε)) for an approximate solution withabsolute error bounded by ε.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
pro: Probabilities related to effective resistance yield sparsifierswith few edges [SS11]con: Computing effective resistances involves solving a systemof linear equations
This is problem is avoided by instead using upper bounds foredge probabilities,
pe ≥ weRe ,
where Re is the effective resistance of the edge.
The sparsifier of [KMP11] improved the expected time boundto O(m log2 n log(1/ε)) for an approximate solution withabsolute error bounded by ε.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Summary of Methods Discussed
Gaussian elimination O(n3)[CW90] O(n2.376)[Wil12] O(n2.3727)
Prec Chebyshev [GO88] O(mS(B) log(κ(A)/ε)√κ(A,B))
[Vaidya91] O(dn)1.75 log(κ(A)/ε)
[ST04] O(m logO(1) m)[KMP11] O(m log2 n log(1/ε))
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let S be a subset of vertices ina graph.
S = {S1,S2,S3}
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let S be a subset of vertices ina graph.
The vertex boundary of S is theset of vertices not in S whichborder S
δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let b be a vector over the ver-tices of a graph.
Then a vector x over the satis-fies the boundary condition of bfor a subset S when
x(v) = b(v) ∀v ∈ δ(S).
δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
• a network with a leader, anda group of agents
• values xi correspond toposition
• leader moves independently ofagents
Goal: design a distributed protocol for agents to follow the leader
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
The leader is the boundary of thesubset of agents.
So our solution should:- Respect the leader’s position (fixthe solution on the boundary)- Use local information among theagents (compute the solution onthe subset)
Goal: design a distributed protocol for agents to follow the leader
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
[NC10]: Consider a multi-agent system of n agents and oneleader.
1 The dynamics of the leader is
x0 = Ax0,
which is independent.
2 The dynamics of each agent is
xi =∑vj∼vi
xj − xi ,
and the vector x of agentpositions is given by
x = −Lx .
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Satisfying the Boundary Condition
Consider a linear system Lx = b. Suppose there exists a subsetS such that
• the induced subgraph on S is connected
• the boundary δ(S) is non-empty
• support(b) ⊆ δ(S)
• x satisfies the boundary condition b:
x(v) = b(v), ∀v ∈ δ(S).
Goal: find the solution x restricted to S .That is, the solution will satisfy
x(v) =
{1dv
∑u∼v x(u) if v ∈ S
b(v) if v ∈ δ(S).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
When f is a probability distribution vector, f TPk is thedistribution after k random walk steps.
Define ∆ to be the Laplace operator, defined ∆ = I − P.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
? Avoid long walk processes by sampling truncated randomwalks.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
? Avoid long walk processes by sampling truncated randomwalks.◦ Control contribution lost in later step by stopping after enoughsteps K (ε)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
Let pk be the probability of taking k random walk steps,
pk = e−ttk
k!.
Let X be the distribution over values of k.• Then obtain a good approximation by taking the average of rrandom walks of length at most K .• The variables r ,K are both in terms of a prescribed errorparameter, ε.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Let the output of the algorithm be ρt,f . Then, to achieve
(1− ε)ρt,f [v ]− ε ≤ ρt,f [v ] ≤ (1 + ε)ρt,f [v ],
we choose
r ← 16
ε3log s,
K ← log(ε−1)
log log(ε−1).
Then, assuming that (1) taking a random walk step and (2)sampling from a distribution take constant time, the runtime ofthe heat kernel pagerank computation is
r · K =16
ε3log s · log(ε−1)
log log(ε−1)= O
( log s log(ε−1)
ε3 log log ε−1
).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Recall what it means for a solution vector x to satisfy theboundary condition in the system Lx = b.
S = {v ∈ V : b[v ] = 0},
x(v) =
{1dv
∑u∼v x(u) if v ∈ S
b(v) if v ∈ δ(S).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Given a graph G and a vector b of boundary conditions,• Let S be the subset of s = |S | vertices
S = {v ∈ V : b[v ] = 0}.
• Let AδS be the s × |δ(S)| matrix A with columns restricted tothe vertices of δ(S) and rows to vertices of S .• Let LS , AS and DS be L, A and D with rows and columnsrestricted to vertices of S .• Let xS be the solution vector over the vertices of S , and letbδS be b over δ(S).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Then we would like to find a vector x ∈ Rs satisfying:
DSxS = ASxS + AδSbδS
or, equivalently:
xS = (DS − AS)−1(AδSbδS)
= L−1S (AδSbδS),
The inverse L−1S exists when the induced subgraph on S is
connected and the boundary of S is non-empty.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Then we would like to find a vector x ∈ Rs satisfying:
DSxS = ASxS + AδSbδS
or, equivalently:
xS = (DS − AS)−1(AδSbδS)
= L−1S (AδSbδS),
The inverse L−1S exists when the induced subgraph on S is
connected and the boundary of S is non-empty.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
The normalized Laplacian is defined
(L)uv =
1 if u = v ,−1√dudv
if u ∼ v ,
0 otherwise.
• Degree-nomalized version of L,
L = D−1/2LD−1/2
• Symmetric version of ∆ = I − P,
L = D1/2∆D−1/2
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
When G has no isolated vertex and the matrix D is invertible,
Lx = b and Lx ′ = b′
the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.
Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy
x ′S = L−1S (AδSb′δS)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
When G has no isolated vertex and the matrix D is invertible,
Lx = b and Lx ′ = b′
the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.
Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy
x ′S = L−1S (AδSb′δS)
x ′S = L−1S b1.
And computing b1 takes time proportional to the sum of vertexdegrees in δ(S) := vol(δ(S)).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,
L−1S =
∫ ∞0HS,t dt.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,
L−1S =
∫ ∞0HS,t dt.
x ′S = L−1S b1 =
∫ ∞0HS,tb1 dt.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
• Approximate the solution
1 express the solution as an improper integral of heat kernelpagerank
2 approximate with a definite integral by limiting the range3 approximate by taking a finite sum of heat kernel pagerank
vectors4 approximate each term by sampling truncated random
walks
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′,
x ′S =
(∫ ∞0
ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S︸ ︷︷ ︸ dt D
−1/2S
ρt,f
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
1. Ignore the tail, take the integral to a finite T ,
x1 ≈∫ T
0ρt,f dt
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
2. Discretize with a finite Riemann sum for small intervals T/N,
x1 ≈N∑j=1
ρjT/N,f · T/N
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
3. Obtain a good approximation by finitely many samples ofρjT/N,f over values of t.
for r times:
draw j from the interval [1,N]
compute ApproxHKPR(G, jT/N, f)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
[CS13] show that to achieve an approximation vector xsatisfying
||x ′ − x || ≤ O(ε(1 + ||b||)),
the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.
Then using that the time to compute a heat kernel pagerankvector is
O( log s log(ε−1)
ε3 log log ε−1
),
and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
[CS13] show that to achieve an approximation vector xsatisfying
||x ′ − x || ≤ O(ε(1 + ||b||)),
the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.Then using that the time to compute a heat kernel pagerankvector is
O( log s log(ε−1)
ε3 log log ε−1
),
and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
Theorem (Boundary Solver [CS13])
For a graph G and a linear system Lx = b, assume:
• x is required to satisfy the boundary condition b
• S = V \ support(b) and s = |S |• the boundary of S is nonempty
• the induced subgraph on S is connected.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
Theorem (Boundary Solver [CS13])
Then the approximate solution x output by the boundary solversatisfies the following with probability ≥ 1− ε.
1 ||x − x || ≤ O(ε(1 + ||b||))
2 The running time is
O((log s)2(log(ε−1))2
ε5 log log(ε−1)
)with additional preprocessing time O(vol(δ(S))).
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Future Directions
Capitalize on speed.
• Use fast linear solvers to improve existing graph algorithms
• Interior point algorithms [SD08]• Learning problems [ZHS05]• Graph partitioning [ST04],[ACL06]• Sparsification [ST04],[SS11]
• Find more applications for these linear solvers
• Find more applications for heat kernel pagerank• Expected distribution of random walks• Vertex similarity → graph distance
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
ReferencesNoga Alon, Richard M. Karp, David Peleg, and Douglas West (1995)
A Graph-Theoretic Game and its Applications to the k-SeverProblem.
SIAM J. Computing, 21(1), 78–100.
Reid Andersen, Fan Chung, and Kevin Lang (2006)
Local Graph Partitioning Using PageRank Vectors.
FOCS’06, 475–486.
Erik Boman and Bruce Hendrickson (2001)
On Spanning Tree Preconditioners.
Sandia National Laboratories Manuscript.
Erik Boman and Bruce Hendrickson (2003)
Support Theory for Preconditioning.
SIAM J. Matrix Analysis and Applications, 25(3), 694–717.
Sergey Brin and Lawrence Page (1998)
The Anatomy of a Large-Scale Hypertextual Web Search Engine.
Computer Networks and ISDN Systems, 30(1), 107–117.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Fan Chung and Olivia Simpson (2013)
Solving Linear Systems with Boundary Conditions using Heat KernelPagerank.
WAW’13.
Don Coppersmith and Shmuel Winograd (1990)
Matrix Multiplication via Arithmetic Progressions.
J. Symbolic Computation, 9(3), 251–280.
Samuel I. Daitch and Daniel A. Spielman (2008)
Faster Approximate Lossy Generalized Flow Via Interior PointAlgorithms.
STOC’08, 451–460.
Gene H. Golub and Michael L. Overton (1988)
The Convergence of Inexact Chebyshev and Richardson IterativeMethods for Solving Linear Systems.
Numerische Mathematik, 53(5), 571–593.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Gene H. Golub and Richard S. Varga (1961)
Chebyshev Semi-Iterative Methods, Successive OverrelaxationIterative Methods, and Second Order Richardson Iterative Methods.
Numerical Math. 3, 147–168.
Keith D. Gremban, Gary L. Miller, and Marco Zagha (1995)
Performance Evaluation of a New Parallel Preconditioner.
International Parallel Processing Symposium ’95, 65–69.
Aric Hagberg and Daniel A. Schult (2008)
Rewiring Networks for Synchronization.
Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(3).
Gustav Kirchhoff (1847)
Uber die Auflosung der Gleichungen, auf welche man bei derUntersuchung der linearen Vertheilung galvanischer Strome gefuhrtwird.
Annalen der Physik, 148(12), 497–508.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Ioannis Koutis, Gary L. Miller, and Richard Peng (2010)
Approaching Optimality for Solving SDD Linear Systems.
FOCS’10, 235–244.
Ioannis Koutis, Gary L. Miller, and Richard Peng (2011)
A Nearly-m log n Time Solver for SDD Linear Systems.
FOCS’11, 590–598.
Ioannis Koutis, Gary L. Miller, and Richard Peng (2012)
A Fast Solver for a Class of Linear Systems.
Communications of the ACM, 55(10), 99–107.
Wei Ni and Daizhan Cheng (2010)
Leader-Following Consensus of Multi-Agent Systems Under Fixed andSwitching Topologies.
Systems & Control Letters, 59(3), 209–217.
Reza Olfati-Saber and Richard Murray (2003)
Consensus Protocols for Networks of Dynamic Agents.
American Control Conference ’03, 951–956.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Daniel A. Spielman and Nikhil Srivastava (2011)
Graph Sparsification by Effective Resistances.
SIAM J. Computing, 40(6), 1913–1926.
Daniel A. Spielman and Shang-Hua Teng (2004)
Nearly-Linear Time Algorithms for Graph Partitioning, GraphSparsification, and Solving Linear Systems.
STOC’04, 81–90.
Pravin M. Vaidya (1991)
Solving Linear Equations with Symmetric Diagonally DominantMatrices by Constructing Good Preconditioners.
UIUC manuscript based on a talk.
Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri (2007)
Numerical Mathematics, vol 37.
Virginia Vassilevska Williams (2012)
Multiplying Matrices Faster than Coppersmith-Winograd.
STOC’12, 887–898.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
David Young (1950)
On Richardson’s Method for Solving Linear Systems with PositiveDefinite Matrices.
J. Math. and Physics 32, 243–255.
Denyong Zhou, Jiayuan Huang, and Bernhard Sch’olkopf (2005)
Learning From Labeled and Unlabeled Data on a Directed Graph
International Conference on Machine Learning 1036–1043.
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Thank You