Quantifying the Invisible Audience in Social Networks
-
Upload
michael-bernstein -
Category
Technology
-
view
4.738 -
download
0
description
Transcript of Quantifying the Invisible Audience in Social Networks
stanford hci group
Quantifying the Invisible Audience in Social Networks
Eytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science Team
Michael BernsteinStanford Computer Science Department
Sharing on a social network is like giving a talk from behind a curtain.
Sharing on a social network is like giving a talk from behind a curtain.
2
Quantify the difference between users’ estimated and actual audience
Quantify the difference between users’ estimated and actual audience
Measure audience size uncertainty for 220,000 Facebook users
Our perception of audience size affects our behaviorWe guide our audience’s impression of us[Go!man 1959]
We manage the boundaries of when to engage [Altman 1975]
On social media, we speak to the audience that we expect is listening[Marwick and boyd 2011, Viégas 1999]
Our perception of audience size affects our behaviorWe guide our audience’s impression of us[Go!man 1959]
We manage the boundaries of when to engage [Altman 1975]
On social media, we speak to the audience that we expect is listening[Marwick and boyd 2011, Viégas 1999]
What if our audience size estimates are inaccurate?
Perceived audience vs. reality - survey - folk theories of audience - desired audience size
Predictability of audience size - using friend count - using feedback
Perceived audience vs. reality - survey - folk theories of audience - desired audience size
Predictability of audience size - using friend count - using feedback
MethodData
220,000 U.S. Facebook users who share with friends-only privacy
Collected audience information for their status updates and link shares over 30 days
150,000,000 viewer-story pairs
MethodAudience size measurement
Javascript tracking whether a story remained in the browser viewport for at least 900ms
MethodAudience size measurement
Javascript tracking whether a story remained in the browser viewport for at least 900ms
MethodAudience size measurement
Javascript tracking whether a story remained in the browser viewport for at least 900ms
900ms
MethodAudience size measurement
Javascript tracking whether a story remained in the browser viewport for at least 900ms
MethodAudience size measurement
Javascript tracking whether a story remained in the browser viewport for at least 900ms
Not a direct measure of attention: users remember ~70% of posts they see[Counts and Fisher 2011]
MethodSurvey
Recruited users with recent content (2-90 days ago) via a request at the top of news feed
N=589; 61% female; mean age 33
Audience size surveyShow participants their most recent story“How many people do you think saw it?”“Describe how you came up with that number.”“How many people do you wish saw this content?”
MethodAnalysis
Compare participants’ actual audience size totheir estimated audience size
MethodAnalysis
Compare participants’ actual audience size totheir estimated audience size
Consider your own most recent status update: What percentage of your social network do you think saw it?
Users underestimate by 4xResults
Users underestimate by 4xResults
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Actual audience (% of friends)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Actual audience (% of friends)
Perc
eive
d au
dien
ce (%
of f
riend
s)
Users underestimate by 4xResults
Users underestimate by 4xResults
Accurate estimations along the diagonal
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Actual audience (% of friends)
Perc
eive
d au
dien
ce (%
of f
riend
s)
Users underestimate by 4xResults
Accurate estimations along the diagonal
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Actual audience (% of friends)
Perc
eive
d au
dien
ce (%
of f
riend
s)
overestimates
underestimates
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%Actual audience (% of friends)
Perc
eive
d au
dien
ce (%
of f
riend
s)
Users underestimate by 4xResults
Estimated20 friends = 6% of network
Actual78 friends = 24% of network
R2 = 0.04
overestimates
underestimates
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%Feedback — likes and comments 21%
“I figured about half of the people who see it will ‘like’ it, or comment on it”
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%
“Maybe a third of my friends saw it.”
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%
“Not a lot of people stay up late at night”
Folk theories of audienceResults
Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%
Folk theories of audienceResults
No folk theory was more accurate than a random guess
Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%
Users want larger audiencesResults
same more far morefewerfar fewer
“How many people do you wish saw this content?”
50% 25% 22%
Users want larger audiencesResults
Roughly half want a larger audience...but they already have it.
same more far morefewerfar fewer
“How many people do you wish saw this content?”
50% 25% 22%
Users underestimate their audience by 4x
Common folk theories use feedback and friend count
Users want larger audiences, but already have them
Perceived audience vs. reality - survey - folk theories of audience - desired audience size
Predictability of audience size - using friend count - using feedback
Perceived audience vs. reality - survey - folk theories of audience - desired audience size
Predictability of audience size - using friend count - using feedback
Can we predict a post’s audience using public signals?using the full 220,000 user and 150,000,000 view dataset
35% of friends see median postResults
More friends means higher variability in audience
0.00
0.01
0.02
0.03
0 50 100 150 200Number of friends who saw post
Den
sity
50th percentile by friend count (266)
0.00
0.01
0.02
0.03
0 50 100 150 200Number of friends who saw post
Den
sity
35% of friends see median postResults
More friends means higher variability in audience
25th percentile by friend count (138)
35% of friends see median postResults
More friends means higher variability in audience
0.00
0.01
0.02
0.03
0 50 100 150 200Number of friends who saw post
Den
sity
75th percentile by friend count (484)
Audience size is highly variableResults
Highly variable: 50th percentile range is 20% of friends
0%
20%
40%
60%
80%
100%
lllllllllllllllllllllllllllllllll
0 200 400 600 800Number of friends
Perc
ent o
f frie
nds
who
saw
pos
t
Audience size is highly variableResults
Highly variable: 50th percentile range is 20% of friends
50th percentile range90th percentile range
0%
20%
40%
60%
80%
100%
lllllllllllllllllllllllllllllllll
0 200 400 600 800Number of friends
Perc
ent o
f frie
nds
who
saw
pos
t
Audience size predictionOLS regression
Model predictors R2 Mean absolute error
Friend count 0.12 8% of friend count
Feedback 0.13 8%
Friend count and feedback
0.27 7%
0%
20%
40%
60%
80%
100%
ll
l l l l l l l l l l l l l l l l l l
0 5 10 15Unique friends liking the post
Perc
ent o
f frie
nds
who
saw
pos
tFeedback is not predictiveResults
Rapid audience growth until the post receives feedback from five unique friends
Posts with no likes or comments have especially large variance: 90th percentile is 2%–55%
Model predictors R2 Mean absolute error
Friend count 0.12 8% of friend count
Feedback 0.13 8% of friend count
Friend count and feedback
0.27 7% of friend count
Audience size predictionOLS regression
Model predictors R2 Mean absolute error
Friend count 0.12 8% of friend count
Feedback 0.13 8% of friend count
Friend count and feedback
0.27 7% of friend count
Audience size predictionOLS regression
Model predictors R2 Mean absolute error
Friend count 0.12 8% of friend count
Feedback 0.13 8% of friend count
Friend count and feedback
0.27 7% of friend count
Even with access to all user-visible signals, audience size is still unpredictable.
Audience size predictionOLS regression
How predictable is a user’s cumulative audience?Consider the audience for all of a user’s posts over 30 days instead of a single post
50% of the users in our sample produced five or more pieces of content during the month
0%
20%
40%
60%
80%
100%
ll
lllllllllllllllllllllllllllllll
0 200 400 600 800Number of friends
Cum
ulat
ive a
udie
nce
(% o
f frie
nds)
61% of friends see at least one of the user’s posts each month
However, actual audience size is still highly variable
Discussion
Fundamental mismatch between perceived and actual audienceHow might a 4x underestimate be impacting user behavior?
Type of content shared, sharing volume, motivation
Ambiguous whether a more socially transparent design would be desirable
Fundamental mismatch between perceived and actual audienceHow might a 4x underestimate be impacting user behavior?
Type of content shared, sharing volume, motivation
Ambiguous whether a more socially transparent design would be desirable
Why underestimate audience size?
The wishful thinking hypothesis: more comfortable to blame a noisy distribution channel than to blame yourself for writing bad content
Why underestimate audience size?
The wishful thinking hypothesis: more comfortable to blame a noisy distribution channel than to blame yourself for writing bad content
What role might be played by...The availability heuristic?Algorithmic feed filtering?
Your invisible audience is larger than you probably think.
Your invisible audience is larger than you probably think.Users underestimate audience size by 4xMedian reach is 35% per post and 61% per monthMany want larger audiences but already have them
stanford hci group
Quantifying the Invisible Audience in Social Networks
Eytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science Team
Michael BernsteinStanford Computer Science Department