Network Density of StatesKun Dong1 Austin R. Benson2 David Bindel2
Applied Math1, CS2
Background
I Weyl’s Law connects the geometric properties of a manifold with theasymptotic spectral distribution of Laplace-Beltrami operator
I Eigenvalues alone do not uniquely identify an object, but what can we learn?. How do we compute spectrum for large real-world graphs?. What geometric properties can we infer from graph spectrum?. Do non-extremal eigenpairs contain valuable information?
(a) Co-spectral Domains (b) Co-spectral Graphs
“Can you hear the shape of a drum?” - Mark Kac
Density of States
I Given a symmetric graph matrix H ∈RN×N, such as normalized adjacencyI It has an eigendecomposition H = QΛQT
I Λ = diag(λ1, · · · ,λN), and Q = [q1, · · · ,qN] orthogonalI Spectral density, also known as density of states (DOS), is
µ(λ ) =1N
N
∑i=1
δ (λ −λi)
I Pointwise density of states (PDOS) is
µk(λ ) = µ(λ ;ek) =N
∑i=1
qi(k )2δ (λ −λi)
I Integrate these distributions over equal sized intervals for spectral histogram. On the left: Blue bins - actual count; red dot - approximated count. On the right: Vertically stacked PDOS spectral histogram. Color for heights
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
50
100
150
200
250
300
350
400
450
500
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
2.2
No
de In
dex
104
0
0.2
0.4
0.6
0.8
1
Approximating Density of States with the Kernel Polynomial Method (KPM)
I Chebyshev series: Fourier cosine series with a change of variable
Tm(cos(θ )) = cos(mθ )
I Chebyshev polynomials can also be defined with the recurrence
T0(x) = 1, T1(x) = x, Tm+1(x) = 2xTm(x)−Tm−1(x)
I Expand DOS in the dual Chebyshev basis
T∗m(x) = w(x)Tm(x), w(x) =2
π(1 + δ0m)√
1−x2
µ(λ ) =∞
∑m=0
dmT∗m(λ ), dm =∫ 1
−1Tm(λ )µ(λ )dλ =
1N
tr(Tm(H))
I Similarly for PDOS: µk(λ ) =∞
∑m=0
dmkT∗m(λ ), dmk = Tm(H)kk
I Alternatives: Gauss Quadrature and Lanczos (GQL).
Stochastic Estimators for Approximating Chebyshev Moments
I For a probe vector z with i.i.d entries of mean 0 and variance 1,
E[zT f(H)z] = tr(f(H)), E[z�Hz] = diag(f(H))
I Choose Nz independent probe vector Zj’s, then
tr(f(H))≈ 1Nz
Nz
∑j=1
ZTj f(H)Zj , diag(f(H))≈ 1
Nz
Nz
∑j=1
Zj� f(H)Zj
I Use the three-term recurrence to efficiently compute Tm(H)z
Efficiently Removing DOS Spikes Coming From Motifs
I Commonly observe spikes in spectrum from eigenvalues of high multiplicityI They can usually be attributed to local symmetry groups
+1 −1
λ = 0
+1 −1
λ =−1/2
+1 ±1 −1 ∓1
λ =±1/2
+1/√
2 −1/√
2
±1 ∓1
λ =±1/√
2I Collapse the graph onto the quotient space of these localized eigenvectorsI Example: High Energy Physics Theory Collaboration Network
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
No Filter
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
Filter at λ = 0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
Filter at λ =−1/3
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
Filter at λ =−1/2
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
Filter at λ =−1/4
200 400 600 800 1000
#moments
0
0.05
0.1
0.15
0.2
0.25
re
l. e
rro
r
No Filter
Zero Filter
All Filter
Relative Error
Nested Dissection (ND)
I Previous methods rely on Monte Carlo sampling, which converges at O( 1√Nz
)
I Alternative: an exact method using nested dissectionI If we can find vertex separators that bisect the graph, the recurrence becomes
Tm+1(H)11 = 2H11Tm(H)11−Tm−1(H)11 + 2H13Tm(H)31 .
H11
H22
H13
H23
H33HT13 HT
23
0
0
Vertex Separation Nested Dissection Ordering
I Hierarchical partition to avoid off-diagonal computation and storageI First traverse in pre-order for the separator blocks, then fill in the leaf blocks
Comparison between Our Methods
Exact ND KPMNetwork Time[s] Time[s] Time[s] Rel-L2 W1
Musm 0.9 0.9 0.2 3.7e-3 2.3e-2Yeast 1.8 1.9 0.3 4.3e-3 1.8e-2Erdos 13.6 4.7 1.9 3.0e-4 4.8e-3PGP 60.9 11.1 7.9 9.4e-4 2.7e-3
MvlComic 1007.3 57.2 43.0 2.9e-4 1.0e-3Caida 5915.5 43.2 50.1 1.5e-4 1.0e-4
I All methods compute 50Chebyshev moments
I KPM uses N/8 probe vectorsI Rel-L2 is on the moments; W1 is
the Wasserstein distance onspectral histogram
Gallery of DOS/PDOS
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
100
200
300
400
500
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
100
200
300
400
500
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
1200
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
100
200
300
400
500
600
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
20
40
60
80
100
Co
un
t
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
1000
2000
3000
4000
5000
No
de
In
de
x
(a) Erdos Collaboration Network
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
1000
2000
3000
4000
5000
6000
No
de
In
de
x
(b) Autonomous SystemNetwork (1999)
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
1000
2000
3000
4000
5000
6000
No
de
In
de
x
(c) Marvel Characters Network
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
1000
2000
3000
4000
No
de
In
de
x
(d) Facebook Ego Networks
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
500
1000
1500
2000
2500
No
de
In
de
x
(e) Minnesota Road Network
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
200
400
600
800
1000
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
100
200
300
400
500
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
5
10
15
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
5
10
15
20
25
Co
un
t
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
2
4
6
8
10
Co
un
t
104
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
2000
4000
6000
8000
No
de
In
de
x
(f) HepTh Collaboration Network
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
1000
2000
3000
4000
5000
6000
No
de
In
de
x
(g) Autonomous SystemNetwork (2000)
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
50
100
150
No
de
In
de
x
(h) Harry Potter CharactersNetwork
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
50
100
150
200
No
de
In
de
x
(i) Twitter EgoNetworks
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
5
10
15
No
de
In
de
x
105
(j) California Road Network
Computed using KPM (500 moments, 20 probe vectors).
DOS Wasserstein Distance Similarity
0.00
0.25
0.10
0.12
0.28
0.47
0.25
0.44
0.43
0.40
0.25
0.00
0.15
0.14
0.29
0.46
0.24
0.43
0.19
0.16
0.10
0.15
0.00
0.03
0.32
0.52
0.29
0.49
0.34
0.31
0.12
0.14
0.03
0.00
0.35
0.55
0.31
0.51
0.33
0.29
0.28
0.29
0.32
0.35
0.00
0.23
0.05
0.19
0.47
0.44
0.47
0.46
0.52
0.55
0.23
0.00
0.25
0.09
0.51
0.49
0.25
0.24
0.29
0.31
0.05
0.25
0.00
0.21
0.42
0.39
0.44
0.43
0.49
0.51
0.19
0.09
0.21
0.00
0.52
0.50
0.43
0.19
0.34
0.33
0.47
0.51
0.42
0.52
0.00
0.04
0.40
0.16
0.31
0.29
0.44
0.49
0.39
0.50
0.04
0.00
Erdos
HepTh
AS1999
AS2000
MvlC
hars
HP
Road-M
N
Road-C
A
Erdos
HepTh
AS1999
AS2000
MvlChars
HP
Road-MN
Road-CA
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Collaboration Network
Autonomous System
Fictional Characters
Ego Network
Road Network
Computation Time
KPM Single Core Average Computation Time Per MomentNetwork # Nodes # Edges Avg. Deg. Time (s)
Enron 36,692 183,831 10.02 0.046Gplus 107,614 13,673,453 254.12 1.133
Amazon 334,863 925,872 5.53 0.628Neuron 1,018,524 24,735,503 48.57 9.138
RoadNetCA 1,965,206 2,766,607 2.82 2.276Orkut 3,072,441 117,185,083 76.28 153.7
LiveJournal 3,997,962 34,681,189 17.35 14.52Friendster 65,608,366 1,806,067,135 55.06 1,017
I Datasets from StanfordNetwork Analysis Project
I Single Intel Xeon E5 v3CPU at 2.30GHz with200GB memory
I Give ample opportunitiesfor parallel computation
Discussion
I We develop practical spectral density computation for real-world networksI DOS and PDOS facilitate comparisons among graphs and modelsI Implementation available at: https://github.com/kd383/NetworkDOS
https://people.cam.cornell.edu/~kd383/ [email protected]
Top Related