A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2,...
-
Upload
arnold-arnold -
Category
Documents
-
view
215 -
download
0
Transcript of A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2,...
![Page 1: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/1.jpg)
A Comparative Usability Study of Two-Factor AuthenticationEmiliano de Cristofaro1, Honglu Du2, Julien Freudiger2, Gregory Norcie3
UCL1, PARC2, Indiana University3
![Page 2: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/2.jpg)
2
Two Factor (2F) Authentication
Website/Service
password
Possession Knowledge Inherence
Token
Phone
Smart Card
Fingerprint
PIN
Pattern
Retina
Palm
A. Adams and M. A. Sasse. Users are not the enemy. 1999
![Page 3: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/3.jpg)
3
Two Factor vs One Factor
+More secure
-Less usable
Slower
Unfamiliar
N. Gunson et al. User perceptions of security and usability of 1F and 2F in automated telephone banking, 2011D. D. Strouble et al. Productivity and usability effects of using a two-factor security system, 2009C. S. Weir et al. Usable security: User preferences for authentication methods in ebanking and the effects of experience , 2010
![Page 4: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/4.jpg)
4
This Presentation
ObservationsLarge offering of two factor solutions
Lack of metrics to measure 2F usability
ProblemIs there a difference in usability among 2F?
ContributionsComparative usability study
Pre-study interview
Explorative quantitative study
![Page 5: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/5.jpg)
5
Pre-Study Interviews
GoalUnderstand popular 2F in use, context and motivations
Participant Recruitment Mailing lists and social media (Google+ and Facebook)
Announced paid interviews for user study on authentication
Online screening survey to know more about potential participants
9 out of 29 mostly from Silicon Valley, familiar with 2F
![Page 6: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/6.jpg)
6
FindingsMotivationForced to
Incentivized
Wanted to
Adoption
Security token
SMS or email Smartphone app
“I use 2F to obtain higher limits on online banking transactions”
“I use 2F to avoid getting hacked”
ContextWork
Personal
Financial
![Page 7: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/7.jpg)
7
QUANTITATIVE SURVEY“An artisan must first sharpen his tools if he is to do his work well.” Confucius
![Page 8: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/8.jpg)
8
Quantitative Survey
Two main challengesHow to recruit participants?
What questions to ask?
Existing usability metricsSUS - System Usability Scale (10 questions)
QUIS - Questionnaire for User Interface Satisfaction (27 questions)
PUEU - Perceived Usefulness and Ease of Use (12 questions)
CSUQ - Computer System Usability Questionnaire (19 questions)
…
Software focused, not for 2F technologies
![Page 9: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/9.jpg)
9
Usability Questions
Quick
EnjoyReuse
Helpful
Not EnjoyUser Friendly
Need Instruction
Concentration
Stressful
MatchFrustrating TrustSecure
Easy
Convenient
A. Karole, etc. A comparative usability evaluation of traditional password managers. In ICISC, 2011.
J. Bonneau, etc. The quest to replace passwords: a Framework for comparative evaluation of web authentication schemes. IEEE Symposium on Security and Privacy, 2012.
![Page 10: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/10.jpg)
10
User Distribution
Online survey219 participants from Mechanical Turk
SUS and 15 other questions on usability
Group 2F Technologies Used
# of Participants
1 Token 11
2 Email/SMS 77
3 App 7
4 Token & Email/SMS 29
5 Token & App 3
6 Email/SMS & App 50
7 All three 41
Total 219
![Page 11: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/11.jpg)
11
ResultsAdoption and Context
AdoptionSMS/Email is the most popular 2F (89.95%)
App (45.20%)
Token (24.20%)
Context
Financial
Personal
Work
10.19%
15.77%
45.36%
69.42%
54.48%
39.18%
20.39%
29.75%
15.46%
Token Email/SMS App
Χ2(4, 582)= 65.18, p<.0001)
![Page 12: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/12.jpg)
12
ResultsMotivations
Token
Email/SMS
App
44.90%
43.52%
37.57%
19.73%
11.65%
9.25%
35.37%
44.48%
53.18%
Forced Incentive Voluntary
Χ2(4, 775)= 14.68, p<.0001)
![Page 13: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/13.jpg)
13
ResultsExploratory Factor Analysis
Quick
EnjoyReuse
Helpful
Not EnjoyUser Friendly
Need Instruction
Concentration
Stressful
MatchFrustrating TrustSecure
Easy
Convenient
A. Karole, etc. A comparative usability evaluation of traditional password managers. In ICISC, 2011.
J. Bonneau, etc. The quest to replace passwords: a Framework for comparative evaluation of web authentication schemes. IEEE Symposium on Security and Privacy, 2012.
![Page 14: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/14.jpg)
14
ResultsExploratory Factor Analysis
Quick
Enjoy
Reuse
Helpful
Not Enjoy
User Friendly
Need Instruction
Concentration
Stressful
Match
Frustrating
Trust
Secure
Convenient
Ease of Use Cognitive Efforts Trustworthiness
32% 15% 14%Variance Explained
![Page 15: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/15.jpg)
15
Usability Comparison
SUS Ease of Use Cog. Efforts Trustworthiness0
1
2
3
4
5
6
7
Token Email/SMS App
![Page 16: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/16.jpg)
16
Usability Comparison
MANOVA analysis (groups 4, 6 & 7)DVs: Ease of use, Cognitive Efforts and Trustworthiness
IV: Technology (2F technologies used)
Covariates: Age and gender
ResultsNo main effect of Technology
Some usability differences w.r.t age and gender:
Email/SMS and Token users (group 4) The elderly (Md=3) need more Cognitive Efforts than the young (Md=2, p=0.003)
Email/SMS and App users (group 6)The elderly (Md=5.5) find that 2F are less trustworthy than the young (Md=6,
p=.0007)
Users of all 3 technologies (group 7) Females (Md=2.75) need more Cognitive Efforts than males (Md=2.0, p=.001)
![Page 17: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/17.jpg)
17
Conclusion
Main resultsDifferent 2F technologies are preferred in different
contexts
Did not find usability difference among three 2F technologies
Identified two additional dimensions of 2F usability: Cognitive Efforts and Trustworthiness
Future workLarger variety of 2F technologies and participants
Develop a usability scale for 2F technologies
![Page 18: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/18.jpg)
18
BACKUP
![Page 19: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/19.jpg)
19
Methodology
Interviews1 on 1 meeting, $10 Amazon Gift Card compensation
Questions1. Which 2F have you used? (Adoption)
2. How does 2F work? (Understanding)
3. Why do you use 2F? (Motivation)
4. Recall last time you used 2F? (Familiarity)
5. What issues do you have with 2F? (Comments)
PIN from a paper/card Digital certificateRSA token codeVerisign token codePaypal token codeGoogle AuthenticatorPIN received by SMS/emailUSB tokenSmartcard
![Page 20: A Comparative Usability Study of Two-Factor Authentication Emiliano de Cristofaro 1, Honglu Du 2, Julien Freudiger 2, Gregory Norcie 3 UCL 1, PARC 2, Indiana.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649e895503460f94b8e4ce/html5/thumbnails/20.jpg)
20
Participants’ Profile
Selected 9/29 from surveyMost of them from silicon valley
Only participants familiar with 2F
Age: 21 to 49
Gender: 5 males, 4 females
Education: High school to PhD
Security: 5/9 background in computer security