Usable Privacy and Security Carnegie Mellon University Spring 2008 Lorrie Cranor 1 Understanding...

Usable Privacy and Security • Carnegie Mellon University • Spring 2008 • Lorrie Cranor • http://cups.cs.cmu.edu/courses/ups.html/ 1

Understanding the Understanding the Human in the LoopHuman in the Loop

January 16, 2008


HumansHumans“Humans are incapable of securely storing high-quality cryptographic keys, and they have unacceptable speed and accuracy when performing cryptographic operations. (They are also large, expensive to maintain, difficult to manage, and they pollute the environment. It is astonishing that these devices continue to be manufactured and deployed. But they are sufficiently pervasive that we must design our protocols around their limitations.)”

-- C. Kaufman, R. Perlman, and M. Speciner. Network Security: PRIVATE Communication in a PUBLIC World.

2nd edition. Prentice Hall, page 237, 2002.


Humans are weakest linkHumans are weakest linkMost security breaches attributed to

“human error”

Social engineering attacks proliferate

Frequent security policy compliance failures

Automated systems are generally more predictable and accurate than humans


Why are humans in the loop at Why are humans in the loop at all?all?Don’t know how or too expensive to

automate

Human judgments or policy decisions needed

Need to authenticate humans


The human threatThe human threatMalicious humans who will attack system

Humans who don’t know when or how to perform security-critical tasks

Humans who are unmotivated to perform security-critical tasks properly or comply with policies

Humans who are incapable of making sound security decisions


Need to better understand Need to better understand humanshumansDo they know they are supposed to be

doing something?

Do they understand what they are supposed to do?

Do they know how to do it?

Are they motivated to do it?

Are they capable of doing it?

Will they actually do it?


Proposed frameworkProposed framework Cranor interactions article: What do they

"indicate?": evaluating security and privacy indicators

The Handbook of Warnings, edited by Michael S. Wogalter• Wogalter’s Communication-Human Information

Processing (C-HIP) Model Applied C-HIP to security indicators evaluation

from interactions article Expanded it to model other types of human

interaction with secure systems Developed “Human in the loop security framework”

and “Human threat identification and mitigation process” - paper under review

Need validation and more work on mitigation and how to operationalize process

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.


C-HIP ModelC-HIP Model Communication-

Human Information Processing (C-HIP) Model• Wogalter, M. 2006.

Communication-Human Information Processing (C-HIP) Model. In Wogalter, M., ed., Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51-61.


Human in the loop security Human in the loop security frameworkframework

Human ReceiverHuman Receiver

Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery

IntentionsIntentions

Attention Switch

AttentionMaintenance

Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation

Attitudes and Beliefs

KnowledgeAcquisition

CommunicationCommunication BehaviorBehavior

Personal VariablesPersonal Variables

Knowledgeand

Experience

Demographicsand Personal Characteristics

Communication Impediments


Interference

EnvironmentalStimuli

CapabilitiesCapabilities

Usable Privacy and Security • Carnegie Mellon University • Spring 2008 • Lorrie Cranor • http://cups.cs.cmu.edu/courses/ups.html/10

Communication processing Communication processing modelmodelFramework is based on communication

processing model• Many models in the literature• Used to model all sorts of different types of

communications: individual, group, media, etc.

Most end-user security actions are triggered by some form of communication• Pop-up alert, email, manual, etc.

Expert self-discovery of a security process can be modeled as communication to oneself


CommunicationCommunication


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference




Types of security Types of security communicationscommunications Warnings• Alert users to take immediate action to avoid hazard

Notices • Inform users about characteristics of entity or object

Status indicators • Inform users about system status information

Training • Teach users about threat and how to respond

Policy • Inform users about policies


Active versus passive Active versus passive communicationscommunications

FirefoxAnti-Phishing

Warning

FirefoxAnti-Phishing

Warning

Active Passive

Bluetoothindicator in

Mac menu bar

Bluetoothindicator in

Mac menu bar

Indicators with audio

alerts

Indicators with audio

alerts

Indicators with

animation

Indicators with

animation


Communication impedimentsCommunication impediments


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference




Environmental stimuliEnvironmental stimuliDivert user’s attention

Greatest impact on passive communication

Examples• Other communications• Ambient light and noise• User’s primary task


InterferenceInterference Anything that may prevent a communication from

being received as the sender intended

Caused by• Malicious attackers• Technology failures• Environmental stimuli that obscure the communication

Focus of traditional secure systems analysis• How can attacker interfere with communications?


Human receiverHuman receiver


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference



“The human in the loop”


Communication deliveryCommunication delivery Attention switch• Noticing communication

Attention maintenance• Paying attention long enough to process

communication

Breakdowns• Environmental stimuli, interference• Characteristics of communication• Habituation

Tendency for the impact of stimuli to decrease over time

Just because the communication appeared on the user’s screen, doesn’t mean the user actually saw it


Communication processingCommunication processingComprehension• Ability to understand communication

Knowledge acquisition• User’s ability to learn what to do in response

Breakdowns• Unfamiliar symbols, vocabulary, complex

sentences, conceptual complexity

Even if a user understands the communication, they still may not know what they are supposed to do


ApplicationApplicationKnowledge retention• Ability to remember communication

Knowledge transfer• Ability to recognize situations where the

communication is applicable and figure out how to apply it

Some security communications are always applied immediately (for example, pop-up warnings) so retention and transfer may not be necessary


Personal variablesPersonal variablesDemographics and personal characteristics• Age, gender, culture, education, occupation,

disabilities

Knowledge and experience• Education, occupation, prior experience


IntentionsIntentions Attitudes and beliefs• Beliefs about communication accuracy• Beliefs about whether they should pay attention• Self-efficacy - whether they believe they can complete

actions effectively• Response-efficacy - whether they believe the actions

they take will be effective• How long it will take• General attitudes - trust, annoyance, etc.

Motivation• Incentives, disincentives


CapabilitiesCapabilitiesUser’s level of ability• Cognitive or physical skills• Availability of necessary software or devices


BehaviorBehavior


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference




BehaviorBehaviorUsers may complete recommended action,

but do so in a way that follows a predictable pattern that can be exploited by attackers• Example: password choice

Users may intend to comply, but may fail to complete necessary action


GulfsGulfsDon Norman. The Design of Every Day

Things.1988.

Gulf of Execution• Gap between a person’s intentions to carry out an

action and the mechanisms provided by a system to facilitate that action “I can’t figure out how to make it do what I want it to do”

Gulf of Evaluation• When a user completes an action but is unable to

interpret the results to determine whether it was successful “I can’t figure out whether it worked”


Generic Error-Modeling SystemGeneric Error-Modeling SystemJames Reason. Human Error. 1990.

Mistakes• When people formulate action plans that will

not achieve the desired goal

Lapses• When people formulate suitable action plans,

but forget to perform a planned action (for example, skipping a step)

Slips• When people perform actions incorrectly (for

example, press the wrong button)


Human threat identification Human threat identification and mitigation processand mitigation process

TaskIdentification

TaskIdentification

TaskAutomation

TaskAutomation

FailureIdentification


FailureMitigation

FailureMitigation

Human-in-the-loopFramework

User Studies

User Studies

Task identification• Identify all points where the system relies on humans to perform security-

critical functions

Task automation• Find ways to partially or fully automate some of these tasks

Failure identification• Identify potential failure modes for remaining tasks

Failure mitigation• Find ways to prevent these failures

Why don’t users follow Why don’t users follow password policies?password policies?


Typical password policyTypical password policyPick a hard to guess password

Don’t use it anywhere else

Change it often

Don’t write it down


Typical password practiceTypical password practice

Bank = b3aYZ Amazon = aa66x!Phonebill = p$2$ta1


Why don’t users follow password Why don’t users follow password policies?policies?

TaskIdentification

TaskIdentification

TaskAutomation

TaskAutomation



FailureMitigation

FailureMitigation


User Studies

User Studies


Why don’t users follow password Why don’t users follow password policies?policies?


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference



Why don’t user’s heed Why don’t user’s heed browser security warnings?browser security warnings?


Do users notice them?Do users notice them?“What lock icon?”• Few users notice lock icon in browser chrome,

https, etc.


Do users know what they Do users know what they mean?mean?Web browser lock icon:• “I think that it means secured, it symbolizes

some kind of security, somehow.”

Web browser security pop-up:• “Yeah, like the certificate has expired. I don’t

actually know what that means.”

J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12-14 July 2006, Pittsburgh, PA.


Do they do what they advise?Do they do what they advise?“I would probably experience some brief, vague sense of unease and close the box and go about my business.”


Why don’t users heed browser security Why don’t users heed browser security warnings?warnings?

TaskIdentification

TaskIdentification

TaskAutomation

TaskAutomation



FailureMitigation

FailureMitigation


User Studies

User Studies


Why don’t users heed browser security Why don’t users heed browser security warnings?warnings?


Co

mm

un

icat

ion

Pro

cess

ing

Co

mm

un

icat

ion

Pro

cess

ing

Ap

plic

atio

nA

pp

licat

ion

Co

mm

un

icat

ion

Del

iver

yC

om

mu

nic

atio

nD

eliv

ery


Attention Switch


Comprehension

KnowledgeRetention

KnowledgeTransfer

Motivation





Knowledgeand

Experience




Interference



Usable Privacy and Security Carnegie Mellon University Spring 2008 Lorrie Cranor 1 Understanding...

Documents

Transcript of Usable Privacy and Security Carnegie Mellon University Spring 2008 Lorrie Cranor 1 Understanding...