Towards Validating Social Network Simulations
-
Upload
bruce-edmonds -
Category
Education
-
view
452 -
download
1
description
Transcript of Towards Validating Social Network Simulations
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 1
Towards Validating Social Network Simulations
SMA Abbas1, Shah Jamal Alam2 and Bruce Edmonds1
1Centre for Policy Modelling, Manchester Metropolitan University2School of Geosciences, University of Edinburgh
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 2
The Situation
A Simulation Social
“System”
GeneratesMeasured
Are these “essentially” the
same?
A Class of Networks Another Class of Networks?
Synthetic
Networks
TargetNetworks
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 3
Key questions
• What properties of the synthetic networks, one would expect to observe given how the model has been constructed
• Which of these properties are ‘significant’ in terms of the intended processes in the model
• Which class of target networks one might expect to observe if one could “re-run” reality under the same basic conditions as assumed in the model
• Do these classes match in important respects• How do we know they do given we only have
samples of synthetic and target networks
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 4
The Problem
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 5
Summary of Issue
• The space of possible networks is vast • But many networks will look similar to us, because
our brains can not deal with them but automatically simplifies them as part of perception
• We are not dealing with single networks but classes of networks…
• …though these classes are often implicit when a single network stands for that class (somehow)
• However, in principle, if synthetic and target networks do match (in some way) then this is potentially a strong validation
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 6
A Cautionary Tale – comparing two network modelsPapadopoulos et al. (2012) Popularity versus similarity in growing networks. Nature, 489:537-540.
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 7
But when compared in a different way…
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 8
Some Network Comparison Approaches
Different kinds of things to compare:• Network Measures• Network Distributions• Eigenvalue/Eigenvectors• Subgroup Identification• Functional Comparison• Likelihood of being described by an Exponential
Random Graph Model• Motif Prevalence
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 9
Examples that follow are from work of S.M.A. Abbas
(see papers at https://sites.google.com/site/maliabbas)
An Example of Validating Synthetic vs. Target Networks
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 10
An example of comparing measures
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 11
An example of comparing distributions
1 5 10 50 100 500 1000
5e
-04
5e
-03
5e
-02
5e
-01
Log-log Plot of Degree Distribution
Degree
Cu
mu
lative
Fre
qu
en
cy
ReferenceRandomFAOFPartyHybrid
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 12
Silo Index Comparison
Correlation0.83
-1.0 -0.9 -0.8 -0.7 -0.6 -0.5
-1.0
0-0
.85
-0.7
0
Dorm Silo Index
Reference Dorm
Hybrid D
orm Correlation
0.93
-1.00 -0.90 -0.80 -0.70
-1.0
0-0
.90
-0.8
0
Major Silo Index
Reference Major
Hybrid M
ajo
rCorrelation0.84
-1.0 -0.5 0.0 0.5
-1.0
-0.8
-0.6
Year Silo Index
Reference Year
Hybrid Y
ear
Correlation0.29
-1.0 -0.8 -0.6
-1.0
0-0
.90
High School Silo Index
Reference High School
Hybrid H
igh S
chool
Reference vs. Hybrid Mode Silo Indices
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 13
Assortativity Mixing
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 14
Problems in the Literature
Authors are often not clear about:• Precisely what the links in a synthetic network are
supposed to represent (in terms that would allow an in principle measurement of observed actors)
• Which aspects of the target network are subject to measurement error (or otherwise judged not to be significant) and which should be reproduced by a synthetic network
• Which aspects of the synthetic network are significant in terms of the generating process (and which are essentially accidental)
Readers often cannot judge the extent or meaning of the match/mismatch between synthetic and target networks
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 15
Conclusions
• Many social simulation models assume stereotypic networks (e.g. Watts-Strogatz)
• It is increasingly clear that the exact network structure matters (e.g. Holzhauer ESSA 2013)
• No single ‘Golden Bullet’ technique• More thought needed about what is significant about the
synthetic and target class networks• Multiple approaches needed to show that classes of
networks are similar – a few 1D measures is not enough to show this
• Validating networks could be quite a strong validation of our models…
• …but much more work is needed in this area!
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 16
Thanks!
SMA Abbas
https://sites.google.com/site/maliabbas
Shah Jamal Alam
https://sites.google.com/site/jamialam
Bruce Edmonds
http://bruce.edmonds.name
Slides at:
http://www.slideshare.net/BruceEdmonds