User Studies Motivation
description
Transcript of User Studies Motivation
![Page 1: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/1.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/1
User Studies MotivationUser Studies Motivation
January 30, 2007
![Page 2: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/2.jpg)
How do we know whether How do we know whether security is usable?security is usable?
![Page 3: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/3.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/3
Need to observe usersNeed to observe usersWe are not our users!
(you may be surprised by what users really do)
![Page 4: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/4.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/4
Wireless privacy studyWireless privacy study Many users unaware that communications over
wireless computer networks are not private How can we raise awareness?
B. Kowitz and L. Cranor. Peripheral Privacy Notifications for Wireless Networks. In Proceedings of the 2005 Workshop on Privacy in the Electronic Society, 7 November 2005, Alexandria, VA.
![Page 5: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/5.jpg)
Wall of sheepWall of sheep
![Page 6: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/6.jpg)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Photo credit: Kyoorius @ techfreakz.org http://www.techfreakz.org/defcon10/?slide=38
Defcon 2001
![Page 7: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/7.jpg)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Photo credit: http://www.timekiller.org/gallery/DefconXII/photo0003
Defcon 2004
![Page 8: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/8.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/8
Peripheral displayPeripheral displayHelp users form more accurate
expectations of privacy
Without making the problem worse
![Page 9: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/9.jpg)
![Page 10: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/10.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/10
Experimental trialExperimental trialEleven subjects in student workspace
Data collected by survey and traffic analysis
Did they refine their expectations of privacy?
![Page 11: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/11.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/11
ResultsResultsNo change in behavior
Peripheral display raised privacy awareness in student workspace
But they didn’t really get it
![Page 12: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/12.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/12
Privacy awareness increasedPrivacy awareness increased “I feel like my information /activity / privacy
are not being protected …. seems like someone can monitor or get my information from my computer, or even publish them.”
![Page 13: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/13.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/13
But only while the display But only while the display was onwas on
“Now that words [projected on the wall] are gone, I'll go back to the same.”
![Page 14: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/14.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/14
Security and privacy Security and privacy indicatorsindicators
![Page 15: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/15.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/15
Evaluating indicatorsEvaluating indicatorsCase study: Privacy Bird
![Page 16: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/16.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/16
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Platform for Privacy Preferences Platform for Privacy Preferences (P3P)(P3P)
2002 W3C Recommendation
XML format for Web privacy policies
Protocol enables clients to locate and fetch policies from servers
![Page 17: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/17.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/17
Privacy BirdPrivacy Bird P3P user agent
Free download http://privacybird.org/
Compares user preferences with P3P policies
![Page 18: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/18.jpg)
![Page 19: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/19.jpg)
![Page 20: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/20.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/20
Critique Privacy BirdCritique Privacy Bird Security people
• Can attackers spoof it?• What if P3P policy
contains lies?• Can P3P policies be
digitally signed?• What about main-in-
the-middle attacks?
Usability people• Green/red color blind
problem• Do people notice it in
corner of browser?• Do people understand
privacy implications?• Why a bird?
![Page 21: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/21.jpg)
Typical Typical securitysecurity
evaluationevaluation
![Page 22: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/22.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/22
Does it behave correctly when Does it behave correctly when notnot under attack? under attack?
No false positives or false negatives
![Page 23: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/23.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/23
Anti-phishing toolsAnti-phishing tools
Y. Zhange, S. Egelman, L. Cranor, and J. Hong. Phinding Phish: Evaluating Anti-Phishing Tools. In Proceedings of NSSS 2006, forthcoming.
![Page 24: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/24.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/24
Does it behave correctly when Does it behave correctly when under attack?under attack?
Can attackers cause wrong indicator to appear?
![Page 25: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/25.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/25
Correct indicator
Wrong indicatorAttacker redirects through CDN
![Page 26: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/26.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/26
Can it be spoofed or Can it be spoofed or obscured?obscured?
Can attacker provide indicator users will rely on instead of real indicator?
![Page 27: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/27.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/27
![Page 28: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/28.jpg)
Usability evaluationUsability evaluation
![Page 29: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/29.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/29
C-HIP ModelC-HIP Model Communication-
Human Information Processing (C-HIP) Model• Wogalter, M. 2006.
Communication-Human Information Processing (C-HIP) Model. In Wogalter, M., ed., Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51-61.
![Page 30: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/30.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/30
Do users notice it?Do users notice it?If users don’t notice indicator all bets are
off
“What lock icon?”• Few users notice lock icon in browser chrome,
https, etc.
C-HIP model: Attention switch, attention maintenance
![Page 31: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/31.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/31
![Page 32: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/32.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/32
![Page 33: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/33.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/33
Do users know what it means?Do users know what it means?Web browser lock icon:
“I think that it means secured, it symbolizes some kind of security, somehow.”
Web browser security pop-up:“Yeah, like the certificate has expired. I don’t actually know what that means.”
C-HIP Model: Comprehension/Memory
J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12-14 July 2006, Pittsburgh, PA.
![Page 34: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/34.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/34
Netscape SSL icons
Cookie flag
IE6 cookie flagFirefox SSL icon
![Page 35: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/35.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/35
Privacy Bird iconsPrivacy Bird icons
Privacy policymatches user’s
privacy preferences
Privacy policydoes not match user’s privacy
preferences
![Page 36: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/36.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/36
![Page 37: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/37.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/37
Do users know what to do when Do users know what to do when they see it?they see it?
C-HIP Model: Comprehension/Memory
![Page 38: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/38.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/38
![Page 39: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/39.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/39
![Page 40: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/40.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/40
![Page 41: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/41.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/41
Do users believe the Do users believe the indicator?indicator?
“Oh yeah, I have [seen warnings], but funny thing is I get them when I visit my [school] websites, so I get told that this may not be secure or something, but it’s my school website so I feel pretty good about it.”
C-HIP Model: Attitudes/Beliefs
![Page 42: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/42.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/42
Are users motivated to take Are users motivated to take action?action?
May view risk as minimal
May find recommended action too inconvenient or difficult
C-HIP Model: Motivation
![Page 43: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/43.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/43
Do they actually do it?Do they actually do it?“I would probably experience some brief, vague sense of unease and close the box and go about my business.”
C-HIP Model: Behavior
![Page 44: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/44.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/44
![Page 45: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/45.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/45
Do they keep doing it?Do they keep doing it?Difficult to measure in laboratory setting
Need to collect data on users in natural environment over extended period of time
C-HIP Model: Behavior
![Page 46: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/46.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/46
How does it interact with How does it interact with other indicators?other indicators?
Indicator overload?
![Page 47: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/47.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/47
![Page 48: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/48.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/48
Summary: Security evaluationSummary: Security evaluationDoes indicator behave correctly when not
under attack?•No false positives or false negatives
Does indicator behave correctly when under attack?•Can attackers cause wrong indicator to
appear?
Can indicator be spoofed or obscured?•Can attacker provide indicator users will rely
on instead of real indicator?
![Page 49: User Studies Motivation](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815e41550346895dccaf18/html5/thumbnails/49.jpg)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/49
Summary: Usability evaluationSummary: Usability evaluation Do users notice it? Do they know what it
means? Do they know what they
are supposed to do when they see it?
Do they believe it? Are they motivated to do
it? Will they actually do it? Will they keep doing it? How does it interact with
other indicators?