Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation, at CCS...

Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation

Sauvik Das

Carnegie Mellon University

Adam Kramer

Facebook, Inc.

Laura Dabbish


Jason Hong


1

Summary

2

3

We showed 50,000 facebook users an announcement urging them to explore security tools. Announcements varied in the presence of, specificity, and framing of social proof.

Overview

Social proof increased awareness.

4Overview

Social proof increased overall adoption but not motivation.

5Overview

Simple social proof, with high specificity and no subjective framing, performed best.

6Overview

Background & Motivation

7

Wait, why is this important?

Security Tools Underutilized

8

Today's user-facing security technology can

prevent many of the security breaches average

people experience.

But people do not use user-facing security

technology, for three reasons:

Background & Motivation: Why is this important?

Security Sensitivity

9

AwarenessDo users know about security threats and tools?

MotivationDo users want to use security threats and security tools?

KnowledgeDo users know how to use security tools?


Key Motivation

10

The need for higher security sensitivity remains a

large outstanding problem in computer

security.


How can we best increase security sensitivity?

11Background & Motivation: Why is this important?

Social Proof

12

We look to others for cues on how to act when we are uncertain. If everyone else is doing it, it must be right!


Social proof is known to influence human behavior.- Milgram, Bickman and Berkovitz found that they could get many pedestrians to stop and stare up at the sky if they had a seed group look up at the sky in the middle of the sidewalk.

- On Facebook, Kramer showed that users are more likely to share emotional content that matches the valence of the emotions shared by their friends.


Social-proof interventions can nudge human behavior.- Cialdini et al. found that hotels can reduce guest’s use of towels by showing them a message that previous hotel guests were less wasteful.

- On Facebook, Bond et al. found that showing people that their friends voted made them significantly more likely to vote.


How to increase security sensitivity?

15

Social proof is a key catalyst for security related

behavior change—increasing awareness,

motivation and knowledge (Das, Kim, Dabbish,

Hong, 2014).


Key Observation

16

We may be able to use social proof to increase

security sensitivity.


Background & Motivation Recap

17

Key Observation

18

Key MotivationThe need for higher security sensitivity remains one

large outstanding problems in computer

security.

We may be able to use social proof to increase

security sensitivity.

We may be able to use social proof to help solve one of the large outstanding problems in computer security.

19

Our Contributions

20

Can social proof be used to increase security sensitivity?

21Our Contributions

Does the presentation of the social proof (e.g., its specificity and framing) alter its effect on security sensitivity?

22Our Contributions

Methods

23

Methods: Social Prompt Experiment

24

Controlled, randomized experiment with 50,000

active facebook users.

Part of annual security awareness campaign run by

facebook, promoting the following three voluntary-

use security tools:


Promoted Security Tools

25

Login ApprovalsTwo-factor authentication. Enter in additional random code generated on trusted device for every log in.

Login NotificationsReceive e-mail/SMS notifications on every login attempt.

Trusted ContactsSocial identification. Specify 3-5 “trusted contacts” to vouch for you if you forget password and do not have access to registered e-mail.


Security Awareness Campaign

26

Show people an announcement on their newsfeed.

Call-to-action button Announcement text


Adding Social Proof

27

We modified the text to include social proof.

We created seven variations, varying in the

specificity and framing of the social proof.


Raw Template

28

Very specific (exact number/percent), no subjective

framing.


Only Template

29

Very specific, negative framing, at most 10% of

security tool using friends.


Over Framing

30

Less specific (value rounded down), positive

framing, at least 10% of security tool using friends.


Some framing

31

Least specific, no subjective framing.


Sample picked randomly among:

32

U.S. Facebook Users>= 18 years of age

At least 10 friends who used security toolsHad not themselves used security tools

Logged in at least once in the past month


Sample assignment

33

Each person assigned randomly and evenly to be

shown one of the eight announcements.

n=6,250 shown each announcement


Sample experience

34

The campaign ran for 4 days in November ‘13.

Participants shown their assigned announcement

at each login, but at most three times.

Not shown again if they already clicked the call-to-

action button.


Measures

35

Click-through rate (awareness)

7-day adoptions (motivation)

5-month adoptions (motivation)

Our social interventions did not attempt to

increase knowledge of how to use security tools.


Covariates

36

Demographics Social Network

Behavioral

AgeGenderFriend countAccount length

Mean friend ageFriend age entropyPercent male friendsMean friends’ account lengthFriend country entropyMean friend-of-friend countNumber of feature-using friends

Posts CreatedPosts DeletedComments CreatedComments DeletedFriends AddedFriends RemovedPhotos Added


Results

37

Descriptive Stats

38

46,235 (93%) logged in and saw an announcement.5,971 (13%) clicked on the announcement over all. 1,873 (4%) adopted one of the promoted features in 7 days.4,555 (10%) adopted one of the promoted features in 5 months.

Results

Raw Overview

39

Group N Clicks 7-day adoptions

5-month adoptions

Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)

Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)

Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)

Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)

Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)

Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)

Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)

Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)

Results

Raw Overview

40


5-month adoptions

Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)

Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)

Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)

Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)

Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)

Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)

Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)

Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)

Results

Clicks

41

Does social proof draw more people to explore announcements, and thereby increase people’s awareness of available security tools?

Results

Logistic Regression

42

Modeled clicks with a logistic regression.

DV: Clicked (yes/no)IV: Which announcement shownControls: Previously listed demographic, social network, and behavioral covariates.

Results

Clicks Model Regression Table

43

Variable Coefficient

Treatment: Raw # 0.36

Treatment: Some 0.35

Treatment: Over # 0.29

Treatment: Only # 0.26

Treatment: Over % 0.21

Treatment: Only % 0.19

Treatment: Raw % 0.17

# security feature using friends 0.09

p < 0.001 for allResults


44











Clicks Model Odds Ratios

45

1

1.1

1.2

1.3

1.4

1.5

Raw % Only % Over % Only # Over # Some Raw #

1.431.42

1.341.3

1.231.21

1.19

Odds ratio for clicking on announcement (relative to control)


Clicks Model: Specificity

46

# conditions get 7% more clicks than %

conditions (p=0.0004).

But specificity has a nuanced effect. The two best

performers were very specific (Raw #) and

completely ambiguous (Some).

Results

Clicks Model: Framing

47

Framing of social proof did not have an effect.

Insignificant performance differences between Raw

(13.3%), Over (13.0%), and Only (12.9%) framings

(p=0.54).

Results


48











Clicks Finding Summary

49

1. Social proof can help increase awareness of security tools. And, this effect is amplified when people have more security-feature using friends.

2. Framing had no statistically discernible effect, but the performance of the Raw # condition suggests that wordsmithing is unlikely to help.

3. Specificity had a non-linear effect. #s were better than %s, but both very specific and ambiguous social proof attracted clicks.

Results

Adoptions

50

Does social proof motivate more people to adopt available security tools?

Results

Logistic Regression

51

Modeled short term and long term adoptions with a

logistic regression.

DV: 7-day adoptions, 5-month adoptionsIV: Which announcement shownControls: Previously listed demographic, social network, and behavioral covariates. Also, whether or not user clicked on the announcement.

Results


52

Variable 7-day Coefficient 5-mo. Coefficient

Treatment: Raw # -0.01 -0.001

Treatment: Some -0.18 -0.03

Treatment: Over # -0.07 -0.13

Treatment: Only # -0.16 -0.09

Treatment: Over % -0.12 -0.06

Treatment: Only % -0.12 -0.05

Treatment: Raw % -0.15 -0.06

# security feature using friends 0.17 * 0.20 *

* p < 0.05Results


53










* p < 0.05Results

Overall adoptions

54

0

2.2

4.4

6.6

8.8

11

Control Raw % Over % Only % Only # Over # Some Raw #

10.610.39.59.79.79.79.69.7

4.84.24.343.93.93.83.7

7-day overall adoption rate 5-month overall adoption rate

p=0.003Results


55










* p < 0.05Results

Adoptions Finding Summary

56

1. Social proof can increase overall feature adoptions.

2. However, we found no evidence that social proof increases motivation to use security features more than the non-social control.

3. The Raw # condition (High specificity and no subjective framing) again performed best, yielding the highest adoption rate.

Results

Discussion & Implications

57

What does it all mean? What next?

Social proof can increase both awareness and adoption of security tools.

58

Furthermore, this effect increases in strength as more of one’s friends use security tools.


Finding 1

59

Finding 1: Implication

To maximize awareness and adoption, we should iteratively show non-adopters with many security-using friends social proof announcements.


60

Finding 2

The type of social proof we tested did not significantly affect motivation to use security tools.


61

Finding 2: ImplicationThis does not mean that social proof is ineffective or has a negative effect on motivation:

1. Needs to be timely & in context; and,2. Needs to be reinforced at the interface where decisions are being made.


62

Finding 3

The most effective presentation of social proof appears to be the simplest: high specificity and no subjective framing.


63

Finding 3: Implication

No need for wordsmithing. Simply presenting people with social proof that others they know use security tools is the best way to reap the benefits of social-proof based interventions.


Conclusion

64

We provided some of the first empirical evidence that social proof can be used to increase security sensitivity.

65

We believe our work opens up a new line of inquiry for solving the longstanding problem of getting users to care and take agency over their security.

66

3. The most effective presentation of social proof appears to be the simplest: high specificity and no subjective framing.

1. Social proof can increase both awareness and adoption of security tools.2. The type of social proof we tested did not significantly affect motivation to use security tools, but that does not mean that all social proof would be ineffective.

Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation

Sauvik Das [[email protected]]


Take-aways

67

mailto:[email protected]

mailto:[email protected]

How to increase security sensitivity?

68

AwarenessSecurity announcements and news.

MotivationMake security tools faster, flashier, cooler.

KnowledgeMake security tools more usable, security education.


Security sensitivity remains lower than ideal.


Raw Overview

70Results


5-month adoptions

Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)

Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)

Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)

Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)

Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)

Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)

Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)

Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)

Raw Overview

71Results


5-month adoptions

Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)

Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)

Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)

Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)

Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)

Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)

Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)

Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)

Adoption Models Odds Ratios

72Results

0

0.2

0.4

0.6

0.8

1

Some Only # Raw % Over % Only % Over # Raw #

1

0.880.950.940.940.91

0.97 0.990.93

0.890.890.860.850.83

7-day odds ratio for adoptions (relative to control)5-month odds ratio for adoptions (relative to control)

all insignificant

Challenges

73

Wait, why is this hard?

Challenges

74

Historically, security tool usage has been kept

confidential and data of its adoption has been

siloed and stripped of its social context.

Challenges: Why is this hard?

We lack a global view: we do not know who

uses what security tools, nor whether any of their

social connections use those tools.

As a result...

75

(1) It is difficult to create interventions that increase

security sensitivity with social proof; and,

(2) It is difficult for security tools to diffuse through

social channels.

Challenges: Why is this hard?

Social Media To The Rescue

76

Social media platforms have the elusive global view:

they know who does and does not use different

security tools, as well as how many of their social

connections use those security tools.

Working with facebook, we put social proof to the

test in the context of increasing security sensitivity.

Our Contributions

We have overlooked a potentially fruitful opportunity to use social factors to increase security sensitivity.

77Challenges: Why is this hard?

Sample experience

78

Participants who clicked the call-to-action button

of any of the announcements were taken to the

same interstitial that explained the promoted

features and allowed them to activate the features.


Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation, at CCS...

Technology

Transcript of Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation, at CCS...