Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and...
Transcript of Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and...
![Page 1: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/1.jpg)
CS 322: (Social and Information) Network AnalysisJure LeskovecStanford University
![Page 2: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/2.jpg)
Definemodularity to beDefine modularity to beQ = (number of edges within groups) –(expected number within groups)(expected number within groups)
Actual Number of Edges between i and j is
Expected Number of Edges between i and j isExpected Number of Edges between i and j is
![Page 3: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/3.jpg)
So Q is a sum of:Where s is a the vector whose So Q is a sum of:
over pairs (i j) that are in the same group
(si, sj) elements are si
over pairs (i, j) that are in the same group Or we can write in matrix form as
Where B is a new characteristic matrix theWhere B is a new characteristic matrix, the modularity matrix
![Page 4: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/4.jpg)
s is the linear combination of the normalized s is the linear combination of the normalized eigenvectors ui of B βi is the eigenvalue of
B corresponding to B corresponding to eigenvector ui
We maximize the coefficient on the largest We maximize the coefficient on the largest eigenvalue by choosing:
![Page 5: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/5.jpg)
Algorithm:Algorithm: Calculate the leading eigenvector of the modularity matrixmodularity matrix
Divide the vertices according to the signs of the elementsthe elements
Note that there is no need to forbid the solution withNote that there is no need to forbid the solution with all the vertices in a single group.
![Page 6: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/6.jpg)
![Page 7: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/7.jpg)
CN = BetweennessCNM = Greedy DA = External Optimization
![Page 8: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/8.jpg)
What is the cluster structure of networks?
How does it scale from small How does it scale from smallto very large networks?
Physics collaborations
How to think about clusters in large networks?
Jure Leskovec, Stanford CS322: Network Analysis 8
Part of a large social network11/12/2009
![Page 9: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/9.jpg)
What is a good cluster? SWhat is a good cluster? Many edges internally Few pointing outsideFew pointing outside
Formally, conductance:S’
SmallΦ(S) corresponds to good clusters
9Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 10: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/10.jpg)
[Leskovec et al. WWW ‘08]
Define:Network community profile (NCP) plotPlot the score of best community of size k
k=5 k=7
log Φ(k)Φ(5)=0.25
Φ(7)=0.18
10
Community size, log k
(7)
Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 11: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/11.jpg)
Meshes grids dense random graphs: Meshes, grids, dense random graphs:
d-dimensional meshes California road network
Jure Leskovec, Stanford CS322: Network Analysis 11
d dimensional meshes
11/12/2009
![Page 12: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/12.jpg)
[Leskovec et al. WWW ‘08]
Collaborations between scientists in networks [Newman, 2005]
log Φ(k)
ductan
ce,
Cond
12
Community size, log k
Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 13: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/13.jpg)
[Clauset‐Moore‐Newman 08]
13Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 14: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/14.jpg)
Natural hypothesis about NCP:Natural hypothesis about NCP: NCP of real networks slopes downward
Slope of the NCP corresponds to the dimensionality of the network
What about large What about large networks?
14Jure Leskovec, Stanford CS322: Network Analysis
We examined more than 100 large networks
11/12/2009
![Page 15: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/15.jpg)
[Leskovec et al. WWW ‘08]
Typical example: General Relativity collaborationsTypical example: General Relativity collaborations(n=4,158, m=13,422)
15Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 16: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/16.jpg)
[Leskovec et al. WWW ‘08]
Jure Leskovec, Stanford CS322: Network Analysis 1611/12/2009
![Page 17: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/17.jpg)
[Leskovec et al. WWW ‘08]
B tt d b tt
nce)
Better and better communities
nduc
tan
Communities get worse and worse
k), (co
nΦ(
Best community has ~100 nodes
k, (cluster size)17Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 18: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/18.jpg)
Each successive edge inside the Each successive edge inside the community costsmore cut‐edges
NCP plotΦ /3 0 33
Φ=2/4 = 0 5
Φ=1/3 = 0.33
Φ=2/4 = 0.5
Φ=8/6 = 1.3
Φ=64/14 = 4.5
Jure Leskovec, Stanford CS322: Network Analysis 18
Each node has twice as many children
11/12/2009
![Page 19: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/19.jpg)
Empirically we note that best clusters (call them Empirically we note that best clusters (call them whiskers) are barely connected to the network
NCP plot
Best cluster.H d it l How does it scale with network size?
Jure Leskovec, Stanford CS322: Network Analysis 19
Core‐periphery structure11/12/2009
![Page 20: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/20.jpg)
[Leskovec et al. Arxiv ‘09]
Practically constant!constant!
Each dot is a different network20Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 21: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/21.jpg)
Whiskers:Edge to cut
21
Whiskers in real networks are non‐trivial (richer than trees)
![Page 22: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/22.jpg)
22
WhiskersWhiskers in real networks are larger than
expected based on density and degree sequence
![Page 23: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/23.jpg)
[Leskovec et al. Arxiv ‘09]
Nothing happens!Nestedness of the
23Jure Leskovec, Stanford CS322: Network Analysis
Nestedness of the core‐periphery structure
11/12/2009
![Page 24: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/24.jpg)
Denser and denser Denser and denser network core
Small good communities
24Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
Nested core‐periphery
![Page 25: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/25.jpg)
What if we allow cuts that give What if we allow cuts that give disconnected communities?
•Cut all whiskers •Compose communities out of whiskersout of whiskers•How good “community” do we get?
25
![Page 26: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/26.jpg)
Rewired network
Bag‐of‐whiskers
Local spectral
whiskers
Metis+MQI
LiveJournalLiveJournal
26
![Page 27: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/27.jpg)
cted clusters
ots are co
nnec
Metis+MQI (red) gives sets
Do
Q ( ) gwith better conductance.
Local Spectral (blue) gives tighter and more well ex
t/int
tighter and more well‐rounded sets.
27
![Page 28: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/28.jpg)
Two ca. 500 node communities from Local Spectral:
Two ca. 500 node communities from Metis+MQI:
28
![Page 29: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/29.jpg)
Denser and denser Denser and denser network core
So, what’s a Small good communities
good model?
29Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
Nested core‐periphery
![Page 30: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/30.jpg)
Flat Down and FlatFlat and Down
Pref. attachment Small World Geometric Pref. Attachment
None of the common models works!
30Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 31: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/31.jpg)
Forest Fire: Model ingredients:
connections spread like a fire New node joins the network Selects a seed node
• Preferential attachment: second neighbor is not uniform at random Selects a seed node
Connects to some of its neighbors Continue recursively
• Copying flavor: ‐ since we burn seed’s neighbors
• Hierarchical flavor:‐ burn Hierarchical flavor: burn around the seed node
• “Local” flavor:‐ burn “near” the seed node
u
the seed node
As cluster grows it blends into the core
31
blends into the coreof the network
Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 32: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/32.jpg)
rewired
networknetwork
32Jure Leskovec, Stanford CS322: Network Analysis11/12/2009
![Page 33: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/33.jpg)
Want to generate realistic networks: Want to generate realistic networks:
Compare graphs properties, e g degree distribution
Given a real network
Generate a synthetic network
Why synthetic graphs? Anomaly detection, Simulations, Predictions, Null‐
e.g., degree distributiony
Anomaly detection, Simulations, Predictions, Nullmodel, Sharing privacy sensitive graphs, …
Q:Which network properties do we care about?
Jure Leskovec 33
Q:Which network properties do we care about? Q:What is a good model and how do we fit it?
![Page 34: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/34.jpg)
Intuition: self‐similarity leads to power‐lawsT t i i i h / it Try to mimic recursive graph / community growth
There are many obvious (but wrong) ways:
Initial graph Recursive expansion
Kronecker Product is a way of generating self‐similar matrices
Initial graph Recursive expansion
![Page 35: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/35.jpg)
[PKDD ’05]
I t di t tIntermediate stage
(9x9)(3x3)
Adjacency matrixAdjacency matrix35
![Page 36: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/36.jpg)
Kronecker product of matrices A and B is given by
N x M K x L
We define a Kronecker product of two graphs as
N*K x M*L
a Kronecker product of their adjacency matrices
36Jure Leskovec
![Page 37: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/37.jpg)
[Leskovec et al. PKDD ‘05]
Kronecker graph: a growing sequence of graphs b it ti th K k d tby iterating the Kronecker product
h k l l ll Each Kronecker multiplication exponentially increases the size of the graph
Kk has N1k nodes and E1
k edges, so we get K1k 1 1densification
One can easily use multiple initiator matrices y p(K1
’, K1’’, K1
’’’ ) that can be of different sizes
37Jure Leskovec
![Page 38: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/38.jpg)
[PKDD ’05]
Continuing multypling with K we Continuing multypling with K1 we obtain K4 and so on …
K1
K4 adjacency matrix38
![Page 39: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/39.jpg)
[Leskovec et al. PKDD ‘05]
39Jure Leskovec
![Page 40: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/40.jpg)
[PKDD ’05]
Kronecker graphs have many properties Kronecker graphs have many properties found in real networks: Properties of static networks Properties of static networks Power‐Law like Degree Distribution Power‐Law eigenvalue and eigenvector distributionPower Law eigenvalue and eigenvector distribution Constant Diameter
Properties of dynamic networks:Properties of dynamic networks: Densification Power Law Shrinking/Stabilizing Diameter
40
![Page 41: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/41.jpg)
[PKDD ’05]
Theorem: Constant diameter: If G has Theorem: Constant diameter: If G1 has diameter d then graph Gk also has diameter d
Observation: Edges in Kronecker graphs:
h X i t dwhere X are appropriate nodes Example:
41
![Page 42: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/42.jpg)
[PKDD ’05]
Create N1N1 probability matrix P1 Create N1N1 probability matrix P1 Compute the kth Kronecker power Pk For each entry puv of Pk include an edge (u,v) with y puv k g ( )probability puv
Probability f d
0.5 0.2 Instance0.25 0.10 0.10 0.040.05 0.15 0.02 0.06
Kroneckermultiplication
of edge pij
0.1 0.3P
Instancematrix K20.05 0.02 0.15 0.06
0.01 0.03 0.03 0.09P10.01 0.03 0.03 0.09
P2=P1P1
flip biased coins
42
![Page 43: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/43.jpg)
[Leskovec et al. 09]
Initiator matrix K1 is a similarity matrixN d i d ib d ith k bi tt ib t Node vi is described with k binary attributes:v1, v2 ,…, vk
Probability of a link between nodes vi, vj:y i, jP(vi,vj) = ∏a K1[vi(a), vj(a)]
0 1
1K a bc d
0 10
1 =c dv2 = (0,1)
1
v3 = (1,0)
43
P(v2,v4) = b·cv3 (1,0)
Jure Leskovec
![Page 44: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/44.jpg)
Given a real network G Given a real network GWant to estimate initiator matrix:
Method of moments [Owen ‘09]
1K a bc d
Method of moments [Owen 09]
Compare counts of and solve system of equationsof equations.
Maximum likelihood [Leskovec‐Faloutsos ICML ‘07]
arg max P( | G ) arg max P( | G1) SVD [VanLoan‐Pitsianis ‘93]
Can solve using SVD2min KKG Can solve using SVD
Jure Leskovec 44
11minF
KKG
![Page 45: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/45.jpg)
[Leskovec‐Faloutsos ICML ‘07]
Maximum likelihood estimationMaximum likelihood estimation
KP( | ) Kronecker
1KP( | ) arg max1G
Naïve estimation takes O(N!N2): N! for different node labelings:
1K a bc d
g Our solution: Metropolis sampling: N! (big) const
N2 for traversing graph adjacency matrix Our solution: Kronecker product (E << N2): N2 E
Do gradient descent Estimate the model in O(E)45Jure Leskovec
![Page 46: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/46.jpg)
Maximum likelihood estimationMaximum likelihood estimation Given real graph G Estimate Kronecker initiator graph Θ (e.g., ) which
We need to (efficiently) calculate)|(maxarg
GP
( y)
A d i i Θ ( i di t d t))|( GP
And maximize over Θ (e.g., using gradient descent)
![Page 47: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/47.jpg)
[ICML ’07]
Given a graph G and Kronecker matrix Θ we G
g pcalculate probability that Θ generated G P(G|Θ)
0.25 0.10 0.10 0.040.05 0.15 0.02 0.060 05 0 02 0 15 0 06
0.5 0.2
1 0 1 1
0 1 0 1
1 0 1 10.05 0.02 0.15 0.060.01 0.03 0.03 0.09
0.1 0.3
Θ
1 0 1 1
1 1 1 1
GΘk GP(G|Θ)
])[1(][)|( vuvuGP ]),[1(],[)|(),(),(
vuvuGP kGvukGvu
47
![Page 48: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/48.jpg)
[ICML ’07]
Nodes are unlabeled0 25 0 10 0 10 0 04
ΘΘk
Graphs G’ and G” should have the same probabilityP(G’|Θ) = P(G”|Θ)
0.25 0.10 0.10 0.04
0.05 0.15 0.02 0.06
0.05 0.02 0.15 0.06
0 01 0 03 0 03 0 09
0.5 0.20.1 0.3
Θ
P(G |Θ) P(G |Θ) One needs to consider all
node correspondences σ
0.01 0.03 0.03 0.09
1 0 1 01G’ σ
All correspondences are a
1 0 1 0
0 1 1 1
1 1 1 1
0 0 1 1
1
2
3
4
)(),|()|(
PGPGP
ppriori equally likely
There are O(N!)d
2
1
41 0 1 1
0 1 0 1
G”
correspondences13
1 0 1 1
1 1 1 1
P(G’|Θ) = P(G”|Θ) 48
![Page 49: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/49.jpg)
[ICML ’07]
Assume we solved the correspondence problem Assume we solved the correspondence problem Calculating
]),[1(],[)|( vukvukGP
Takes O(N2) time Infeasible for large graphs (N ~ 105)
σ… node labeling
]),[(],[)|(),(),( vukGvuvukGvu
Infeasible for large graphs (N 10 )
0.25 0.10 0.10 0.040 05 0 15 0 02 0 06
1 0 1 10 1 0 10.05 0.15 0.02 0.06
0.05 0.02 0.15 0.060.01 0.03 0.03 0.09
0 1 0 11 0 1 10 0 1 1
σ
GP(G|Θ, σ)
Θkc
Part 1‐49
![Page 50: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/50.jpg)
[ICML ’07]
Log‐likelihood Log‐likelihood
Gradient of log‐likelihood
Sample the permutations from P(σ|G,Θ) and average the gradientsaverage the gradients
50
![Page 51: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/51.jpg)
[ICML ’07]
Metropolis sampling Start with a random permutation σ Do local moves on the permutation Accept the new permutation σ’ Accept the new permutation σ If new permutation is better (gives higher likelihood) else accept with prob. proportional to the ratio of likelihoods (no need to calculate the normalizing constant!)likelihoods (no need to calculate the normalizing constant!)
1
2
34
2
3Swap node
labels 1 and 4
1 0 1 0
0 1 1 11 1 1 0
1 1 1 0
2 214
12
12
Can compute efficientlyOnly need to account for changes in 2 rows /
1 1 1 1
0 1 1 1
1 1 1 0
1 1 1 1
0 0 1 1
34
234
changes in 2 rows / columns
51
![Page 52: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/52.jpg)
Need to efficiently calculate the likelihood ratios
But the permutations σ(i)
and σ(i+1) only differ at 2and σ( ) only differ at 2 positions
So we only traverse to d 2 ( l ) fupdate 2 rows (columns) of
Θk We can evaluate the
likelihood ratio efficientlyMetropolis permutation sampling algorithm
j
k
![Page 53: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/53.jpg)
[ICML ’07]
Calculating naively P(G|Θ σ) takes O(N2) Calculating naively P(G|Θ,σ) takes O(N ) Idea: First calculate likelihood of empty graph a graph First calculate likelihood of empty graph, a graph with 0 edges Correct the likelihood for edges that we observe in Correct the likelihood for edges that we observe in the graph
By exploiting the structure of Kronecker productBy exploiting the structure of Kronecker product we obtain closed form for likelihood of an empty graphempty graph
53
![Page 54: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/54.jpg)
[ICML ’07]
We approximate the likelihood: We approximate the likelihood:
The sum goes only over the edges
No‐edge likelihood Edge likelihoodEmpty graph
The sum goes only over the edges Evaluating P(G|Θ,σ) takes O(E) time Real graphs are sparse, E << N2Real graphs are sparse, E N
54
![Page 55: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/55.jpg)
Real graphs are sparse so we first Θk
g p pcalculate likelihood of empty graph
Probability of edge (i,j) is in general pij=θ1
aθ2b θ3
c θ4dθ1 θ2 θ3 θ4
By using Taylor approximation to pij and summing the multinomial series we obtain:
pij =θ1aθ2
bθ3cθ1
d
obta
We approximate the likelihood: Taylor approximationlog(1-x) ~ -x – 0.5 x2
No‐edge likelihood Edge likelihoodEmpty graph
![Page 56: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/56.jpg)
[ICML ’07]
E i t l t Experimental setup Given real graph Stochastic gradient descent from random initial point Stochastic gradient descent from random initial point Obtain estimated parameters Generate synthetic graphsGenerate synthetic graphs Compare properties of both graphs
W d t fit th ti th l We do not fit the properties themselves We fit the likelihood and then compare the graph propertiesproperties
56
![Page 57: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/57.jpg)
Can gradient descent recover true parameters? How nice (smooth without local minima) is optimization How nice (smooth, without local minima) is optimization
space? Generate a graph from random parameters Start at random point and use gradient descent Start at random point and use gradient descent We recover true parameters 98% of the times
How does algorithm converge to true parameters with How does algorithm converge to true parameters with gradient descent iterations?
DiameterLog‐likelihood Avg abs error 1st eigenvalue
![Page 58: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/58.jpg)
[Leskovec‐Faloutsos ICML ‘07]
Real and Kronecker are very close: 1K0.99 0.54
0 49 0 13 Real and Kronecker are very close: 1 0.49 0.13
58Jure Leskovec
![Page 59: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/59.jpg)
[Leskovec et al. Arxiv 09]
What do estimated parameters tell us What do estimated parameters tell us about the network structure?
b edges
1K a bc d a edges d edgesc d
c edges
59Jure Leskovec
![Page 60: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/60.jpg)
[Leskovec et al. Arxiv ‘09]
What do estimated parameters tell us What do estimated parameters tell us about the network structure? 1K 0.9 0.5
0.5 0.1
0.5 edges
Core0 9 edges
Periphery0 1 edges0.9 edges 0.1 edges
0.5 edges
60
Nested Core‐peripheryJure Leskovec
![Page 61: Stanford Universitysnap.stanford.edu/class/cs322-2009/17-ncp-annot.pdf · CS 322: (Social and Information) Network Analysis Jure Leskovec Stanford University. Define modularityto](https://reader031.fdocuments.in/reader031/viewer/2022022503/5ab0db307f8b9ad9788b8ca6/html5/thumbnails/61.jpg)
[Leskovec et al. Arxiv ‘09]
Small and large net orks are er different Small and large networks are very different:
0 99 0 540 99 0 17K61
0.99 0.540.49 0.13
0.99 0.170.17 0.82
K1 = K1 =Jure Leskovec