CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller...
-
Upload
molly-burns -
Category
Documents
-
view
219 -
download
0
Transcript of CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller...
![Page 1: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/1.jpg)
CMU SCS
Large Graph Mining:Power Tools and a Practitioner’s
Guide
Christos FaloutsosGary Miller
Charalampos (Babis) TsourakakisCMU
![Page 2: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/2.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-2
![Page 3: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/3.jpg)
CMU SCS
Matrix Representations of G(V,E)
Associate a matrix to a graph:
• Adjacency matrix
• Laplacian
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-3
Main focus
![Page 4: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/4.jpg)
CMU SCS
Recall: Intuition
• A as vector transformation
2 11 3
A
10
x
21
x’
= x
x’
2
1
1
3
![Page 5: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/5.jpg)
CMU SCS
Intuition
• By defn., eigenvectors remain parallel to themselves (‘fixed points’)
2 11 3
A0.52
0.85
v1v1
=
0.52
0.853.62 *
1
![Page 6: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/6.jpg)
CMU SCS
Intuition
• By defn., eigenvectors remain parallel to themselves (‘fixed points’)
• And orthogonal to each other
![Page 7: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/7.jpg)
CMU SCS
Keep in mind!• For the rest of slides we will be talking for square
nxn matrices
and symmetric ones, i.e,
KDD'09 Faloutsos, Miller, Tsourakakis P7-7
nnn
n
mm
mm
M
1
111
...
TMM
![Page 8: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/8.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-8
![Page 9: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/9.jpg)
CMU SCS
Adjacency matrix
KDD'09 Faloutsos, Miller, Tsourakakis P7-9
A=1
2 3
4
Undirected
![Page 10: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/10.jpg)
CMU SCS
Adjacency matrix
KDD'09 Faloutsos, Miller, Tsourakakis P7-10
A=1
2 3
410
0.3
2
4
Undirected Weighted
![Page 11: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/11.jpg)
CMU SCS
Adjacency matrix
KDD'09 Faloutsos, Miller, Tsourakakis P7-11
1
2 3
4
ObservationIf G is undirected,A = AT
Directed
![Page 12: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/12.jpg)
CMU SCS
Spectral Theorem
Theorem [Spectral Theorem]
• If M=MT, then
KDD'09 Faloutsos, Miller, Tsourakakis P7-12
Tnnn
T
Tn
T
n
n xxxx
x
x
xxM
111
11
1 ......
Reminder 1:xi,xj orthogonal
0
0
x1
x2
![Page 13: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/13.jpg)
CMU SCS
Spectral Theorem
Theorem [Spectral Theorem]
• If M=MT, then
KDD'09 Faloutsos, Miller, Tsourakakis P7-13
Tnnn
T
Tn
T
n
n xxxx
x
x
xxM
111
11
1 ......
Reminder 2:xi i-th principal axisλi length of i-th principal axis
0
0
1
2
![Page 14: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/14.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-14
![Page 15: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/15.jpg)
CMU SCS
Eigenvectors:
• Give groups
• Specifically for bi-partite graphs, we get each of the two sets of nodes
• Details:
![Page 16: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/16.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-16
Any graph with no cycles of odd length is bipartite
Q1: Can we check if a graph is bipartite via its spectrum?Q2: Can we get the partition of the vertices in the two sets of nodes?
1 4
2 5
3 6
K3,3
![Page 17: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/17.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-17
Adjacency matrix
where
Eigenvalues:
1 4
2 5
3 6
K3,3
Λ=[3,-3,0,0,0,0]
![Page 18: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/18.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-18
Adjacency matrix
where1 4
2 5
3 6
K3,3
Why λ1=-λ2=3?Recall: Ax=λx, (λ,x) eigenvalue-eigenvector
![Page 19: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/19.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-19
1 4
2 5
3 6
1
1
1
1
1
1
1 4
5
6
1 1
1
1
233=3x1
Value @ each node: eg., enthusiasm about a product
![Page 20: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/20.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-20
1 4
2 5
3 6
1
1
1
1
1
1
1 4
5
6
1
1
1
3=3x1
1-vector remains unchanged (just grows by ‘3’ = 1 )
![Page 21: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/21.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-21
1 4
2 5
3 6
1
1
1
1
1
1
1 4
5
6
1
1
1
3=3x1
Which other vector remains unchanged?
![Page 22: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/22.jpg)
CMU SCS
Bipartite Graphs
KDD'09 Faloutsos, Miller, Tsourakakis P7-22
1 4
2 5
3 6
1
1
1
-1
-1
-1
1 4
5
6
-1 -1
-1
-1
-2-3-3=(-3)x1
![Page 23: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/23.jpg)
CMU SCS
Bipartite Graphs
• Observationu2 gives the partition of the nodes in the two sets S, V-S!
KDD'09 Faloutsos, Miller, Tsourakakis P7-23
S V-S
Theorem: λ2=-λ1 iff G bipartite. u2 gives the partition.
Question: Were we just “lucky”? Answer: No
65321 4
![Page 24: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/24.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-24
![Page 25: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/25.jpg)
CMU SCS
Walks
• A walk of length r in a directed graph:
where a node can be used more than once.
• Closed walk when:
KDD'09 Faloutsos, Miller, Tsourakakis P7-25
1
2 3
4
1
2 3
4
Walk of length 2 2-1-4
Closed walk of length 32-1-3-2
![Page 26: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/26.jpg)
CMU SCS
Walks
Theorem: G(V,E) directed graph, adjacency matrix A. The number of walks from node u to node v in G with length r is (Ar)uv
Proof: Induction on k. See Doyle-Snell, p.165
KDD'09 Faloutsos, Miller, Tsourakakis P7-26
![Page 27: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/27.jpg)
CMU SCS
Walks
Theorem: G(V,E) directed graph, adjacency matrix A. The number of walks from node u to node v in G with length r is (Ar)uv
KDD'09 Faloutsos, Miller, Tsourakakis P7-27
r
ijr
ijij aAaAaA .., , , 221
(i,j) (i, i1),(i1,j) (i,i1),..,(ir-1,j)
![Page 28: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/28.jpg)
CMU SCS
Walks
KDD'09 Faloutsos, Miller, Tsourakakis P7-28
1
2 3
4
1
2 3
4i=2, j=4
![Page 29: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/29.jpg)
CMU SCS
Walks
KDD'09 Faloutsos, Miller, Tsourakakis P7-29
1
2 3
4
1
2 3
4i=3, j=3
![Page 30: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/30.jpg)
CMU SCS
Walks
KDD'09 Faloutsos, Miller, Tsourakakis P7-30
1
2 3
4
1
3
4
2
Always 0,node 4 is a sink
![Page 31: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/31.jpg)
CMU SCS
Walks
Corollary: If A is the adjacency matrix of undirected G(V,E) (no self loops), e edges and t triangles. Then the following hold:a) trace(A) = 0 b) trace(A2) = 2ec) trace(A3) = 6t
KDD'09 Faloutsos, Miller, Tsourakakis P7-31
11 2
1 2
3
![Page 32: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/32.jpg)
CMU SCS
Walks
Corollary: If A is the adjacency matrix of undirected G(V,E) (no self loops), e edges and t triangles. Then the following hold:a) trace(A) = 0 b) trace(A2) = 2ec) trace(A3) = 6t
KDD'09 Faloutsos, Miller, Tsourakakis P7-32
Computing Ar may beexpensive!
![Page 33: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/33.jpg)
CMU SCS
Remark: virus propagation
The earlier result makes sense now:
• The higher the first eigenvalue, the more paths available ->
• Easier for a virus to survive
![Page 34: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/34.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-34
![Page 35: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/35.jpg)
CMU SCS
Main upcoming result
the second eigenvector of the Laplacian (u2)
gives a good cut:Nodes with positive scores should go to one
group
And the rest to the other
![Page 36: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/36.jpg)
CMU SCS
Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-36
L= D-A=1
2 3
4
Diagonal matrix, dii=di
![Page 37: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/37.jpg)
CMU SCS
Weighted Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-37
1
2 3
410
0.3
2
4
![Page 38: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/38.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-38
![Page 39: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/39.jpg)
CMU SCS
Connected Components
• Lemma: Let G be a graph with n vertices and c connected components. If L is the Laplacian of G, then rank(L)= n-c.
• Proof: see p.279, Godsil-Royle
KDD'09 Faloutsos, Miller, Tsourakakis P7-39
![Page 40: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/40.jpg)
CMU SCS
Connected Components
KDD'09 Faloutsos, Miller, Tsourakakis P7-40
G(V,E)
L=
eig(L)=
#zeros = #components
1 2 3
6
7 5
4
![Page 41: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/41.jpg)
CMU SCS
Connected Components
KDD'09 Faloutsos, Miller, Tsourakakis P7-41
G(V,E)
L=
eig(L)=
#zeros = #components
1 2 3
6
7 5
4
0.01
Indicates a “good cut”
![Page 42: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/42.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Cheeger Inequality and Sparsest Cut: • Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-42
![Page 43: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/43.jpg)
CMU SCS
Adjacency vs. Laplacian Intuition
KDD'09 Faloutsos, Miller, Tsourakakis P7-43
V-SLet x be an indicator vector:
S
Six
Six
i
i
if ,0
if ,1
Consider now y=Lx k-th coordinate
![Page 44: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/44.jpg)
CMU SCS
Adjacency vs. Laplacian Intuition
KDD'09 Faloutsos, Miller, Tsourakakis P7-44
Consider now y=Lx
G30,0.5
S
k
![Page 45: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/45.jpg)
CMU SCS
Adjacency vs. Laplacian Intuition
KDD'09 Faloutsos, Miller, Tsourakakis P7-45
Consider now y=Lx
G30,0.5
S
k
![Page 46: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/46.jpg)
CMU SCS
Adjacency vs. Laplacian Intuition
KDD'09 Faloutsos, Miller, Tsourakakis P7-46
Consider now y=Lx
G30,0.5
S
kk
Laplacian: connectivity, Adjacency: #paths
![Page 47: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/47.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix– Intuition behind eigenvectors: Eg., Bipartite Graphs
– Walks of length k
• Laplacian– Connected Components
– Intuition: Adjacency vs. Laplacian
– Sparsest Cut and Cheeger inequality:• Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-47
![Page 48: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/48.jpg)
CMU SCS
Why Sparse Cuts?
• Clustering, Community Detection
• And more: Telephone Network Design, VLSI layout, Sparse Gaussian Elimination, Parallel Computation
KDD'09 Faloutsos, Miller, Tsourakakis P7-48
1
2 3
4
5
6 7
8
9
cut
![Page 49: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/49.jpg)
CMU SCS
Quality of a Cut
• Isoperimetric number φ of a cut S:
KDD'09 Faloutsos, Miller, Tsourakakis P7-49
1
2 3
4
#edges across#nodes in smallest partition
![Page 50: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/50.jpg)
CMU SCS
Quality of a Cut
• Isoperimetric number φ of a graph = score of best cut:
KDD'09 Faloutsos, Miller, Tsourakakis P7-50
1
2 3
4
and thus
![Page 51: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/51.jpg)
CMU SCS
Quality of a Cut
• Isoperimetric number φ of a graph = score of best cut:
KDD'09 Faloutsos, Miller, Tsourakakis P7-51
1
2 3
4
Best cut: hard to findBUT: Cheeger’s inequality gives bounds2: Plays major role
Let’s see the intuition behind 2
![Page 52: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/52.jpg)
CMU SCS
Laplacian and cuts - overview
• A cut corresponds to an indicator vector (ie., 0/1 scores to each node)
• Relaxing the 0/1 scores to real numbers, gives eventually an alternative definition of the eigenvalues and eigenvectors
![Page 53: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/53.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-53
Six
Six
i
i
if ,0
if ,1Characteristic Vector x
Then:
SV-S
Edges across cut
![Page 54: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/54.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-54
1
2 3
4
5
6 7
8
9
cut
x=[1,1,1,1,0,0,0,0,0]T
S V-S
xTLx=2
![Page 55: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/55.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-55
Ratio cut
Sparsest ratio cut
NP-hard
Relax the constraint:
Normalize:?
![Page 56: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/56.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-56
Sparsest ratio cut
NP-hard
Relax the constraint:
Normalize:λ2
because of the Courant-Fisher theorem (applied to L)
![Page 57: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/57.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-57
Each ball 1 unit of mass xLx x1 xnOSCILLATE
Dfn of eigenvector
Matrix viewpoint:
![Page 58: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/58.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-58
Each ball 1 unit of mass xLx x1 xnOSCILLATE
Force due to neighbors displacement
Hooke’s constantPhysics viewpoint:
![Page 59: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/59.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-59
Each ball 1 unit of mass
Eigenvector value
Node idxLx x1 xnOSCILLATE
For the first eigenvector:All nodes: same displacement (= value)
![Page 60: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/60.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-60
Each ball 1 unit of mass
Eigenvector value
Node idxLx x1 xnOSCILLATE
![Page 61: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/61.jpg)
CMU SCS
Why λ2?
KDD'09 Faloutsos, Miller, Tsourakakis P7-61
Fundamental mode of vibration: “along” the separator
![Page 62: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/62.jpg)
CMU SCS
Cheeger Inequality
KDD'09 Faloutsos, Miller, Tsourakakis P7-62
Score of best cut(hard to compute)
Max degree 2nd smallest eigenvalue(easy to compute)
![Page 63: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/63.jpg)
CMU SCS
Cheeger Inequality and graph partitioning heuristic:
KDD'09 Faloutsos, Miller, Tsourakakis P7-63
• Step 1: Sort vertices in non-decreasing order according to their score of the second eigenvector
• Step 2: Decide where to cut.
• Bisection
• Best ratio cutTwo common heuristics
![Page 64: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/64.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix• Laplacian
– Connected Components
– Intuition: Adjacency vs. Laplacian
– Sparsest Cut and Cheeger inequality:• Derivation, intuition
• Example
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-64
![Page 65: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/65.jpg)
CMU SCS
Example: Spectral Partitioning
KDD'09 Faloutsos, Miller, Tsourakakis P7-65
• K500 • K500
dumbbell graph
A = zeros(1000); A(1:500,1:500)=ones(500)-eye(500); A(501:1000,501:1000)= ones(500)-eye(500); myrandperm = randperm(1000);B = A(myrandperm,myrandperm);
In social network analysis, such clusters are called communities
![Page 66: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/66.jpg)
CMU SCS
Example: Spectral Partitioning
• This is how adjacency matrix of B looks
KDD'09 Faloutsos, Miller, Tsourakakis P7-66
spy(B)
![Page 67: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/67.jpg)
CMU SCS
Example: Spectral Partitioning
• This is how the 2nd eigenvector of B looks like.
KDD'09 Faloutsos, Miller, Tsourakakis P7-67
L = diag(sum(B))-B;[u v] = eigs(L,2,'SM');plot(u(:,1),’x’)
Not so much information yet…
![Page 68: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/68.jpg)
CMU SCS
Example: Spectral Partitioning
• This is how the 2nd eigenvector looks if we sort it.
KDD'09 Faloutsos, Miller, Tsourakakis P7-68
[ign ind] = sort(u(:,1));plot(u(ind),'x')
But now we seethe two communities!
![Page 69: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/69.jpg)
CMU SCS
Example: Spectral Partitioning
• This is how adjacency matrix of B looks now
KDD'09 Faloutsos, Miller, Tsourakakis P7-69
spy(B(ind,ind))
Cut here!Community 1
Community 2Observation: Both heuristics are equivalent for the dumbbell
![Page 70: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/70.jpg)
CMU SCS
Outline• Reminders
• Adjacency matrix• Laplacian
– Connected Components
– Intuition: Adjacency vs. Laplacian
– Sparsest Cut and Cheeger inequality:
• Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-70
![Page 71: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/71.jpg)
CMU SCS
Why Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-71
• K500 • K500
The onlyweightededge!
Cut here
φ=φ=Cut here
>
So, φ is not good here…
![Page 72: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/72.jpg)
CMU SCS
Why Normalized Laplacian
KDD'09 Faloutsos, Miller, Tsourakakis P7-72
• K500 • K500
The onlyweightededge!
Cut here
φ=φ=Cut here
>Optimize Cheegerconstant h(G),balanced cuts
where
![Page 73: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/73.jpg)
CMU SCS
Extensions
• Normalized Laplacian– Ng, Jordan, Weiss Spectral Clustering – Laplacian Eigenmaps for Manifold Learning– Computer Vision and many more
applications…
KDD'09 Faloutsos, Miller, Tsourakakis P7-73
Standard reference: Spectral Graph TheoryMonograph by Fan Chung Graham
![Page 74: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/74.jpg)
CMU SCS
Conclusions
Spectrum tells us a lot about the graph:
• Adjacency: #Paths
• Laplacian: Sparse Cut
• Normalized Laplacian: Normalized cuts, tend to avoid unbalanced cuts
KDD'09 Faloutsos, Miller, Tsourakakis P7-74
![Page 75: CMU SCS Large Graph Mining: Power Tools and a Practitioner’s Guide Christos Faloutsos Gary Miller Charalampos (Babis) Tsourakakis CMU.](https://reader038.fdocuments.in/reader038/viewer/2022110209/56649e045503460f94af0502/html5/thumbnails/75.jpg)
CMU SCS
References• Fan R. K. Chung: Spectral Graph Theory (AMS)
• Chris Godsil and Gordon Royle: Algebraic Graph Theory (Springer)
• Bojan Mohar and Svatopluk Poljak: Eigenvalues in Combinatorial Optimization, IMA Preprint Series #939
• Gilbert Strang: Introduction to Applied Mathematics
(Wellesley-Cambridge Press)
KDD'09 Faloutsos, Miller, Tsourakakis P7-75