CMU Usable Privacy and Security Laboratoryhttp://cups.cs.cmu.edu/

Sensor-Based Interactions

Kami Vaniea

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

OutlineOutline

Coping with Uncertainty

Making Sense of Sensing Systems: Five Questions for Designers and Researchers

Distributed Mediation of Ambiguous Context in Aware Environments

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

UncertaintyUncertainty

State of knowledge in which one or more alternatives result in a set of possible

specific outcomes, but where the probabilities of the outcomes are neither

known nor meaningful.

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Uncertain CommunicationUncertain Communication

Sensors produce concrete data with uncertain or ambiguous interpretation•Location Data (GPS)

•Temperature

Computers can make bad decisions based on uncertain data• Interrupting user

•Making poor decisions on user’s behalf

•Communicating wrong information

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

OutlineOutline

Coping with Uncertainty

Making Sense of Sensing Systems: Five Questions for Designers and Researchers

Distributed Mediation of Ambiguous Context in Aware Environments

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

HHI and HCIHHI and HCI

Human to Human Interaction (HHI) has been well studied by researchers

How can we use some of the lessons about Human to Human communication to facilitate Human to Computer communication?

Recognize that communication is a shared responsibility and requires management and repair

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Norman’s Seven Stages of Action

Norman’s Seven Stages of Action

Forming the goal

Forming the intention

Specifying an action

Executing the action

Perceiving the state of the world

Interpreting the state of the world

Evaluating the outcome

Goal

Execution

Evaluation

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Five Issues in CommunicationFive Issues in Communication

Address: Directing communication to a system

Attention: Establishing that the system is attending

Action: Defining what is to be done with the system

Alignment: Monitoring system response

Accident: Avoiding or recovering from errors or misunderstandings

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Stages of Action vs. Conversation IssuesStages of Action vs. Conversation Issues

SystemHuman

Address

Attention

Action

Alignment

Accident

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Address: How do I address or of many possible devices?

Address: How do I address or of many possible devices?

GUI Answers• Keyboard

• Mouse

Exposed Challenges• Disambiguate signal-to-noise

• Disambiguate intended target system

• How to not address the system

Possible Problems• No response

• Unwanted response

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Attention: How do I know the system is ready and attending to my actions?Attention: How do I know the system is ready and attending to my actions?

GUI Answers• Graphical feedback such as blinking cursors

Exposed Challenges• Giving feedback so the user knows it has the system’s

attention

• Periphery feedback

Possible Problems• Wasted input if system not responding

• Unintended action

• Privacy and Security problems(?)

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Action: How do I effect a meaningful action, control its extent and

possibly specify a target or targets of my action?

Action: How do I effect a meaningful action, control its extent and

possibly specify a target or targets of my action? GUI Answers

• Use standard GUI widgets, such as clicking and selecting text, to interact with the system

Exposed Challenges• Identifying and selecting possible actions• Avoid unwanted selection• Handling complex operations

Possible Problems• Limited operations available• Failure to execute action• Unintended action

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Alignment: How do I know the system is doing the right thing?

Alignment: How do I know the system is doing the right thing?

GUI Answers• Graphical feedback such as text appearing• Auditory feedback• Detectible new state

Exposed Challenges• Making system state perceivable• Providing distinctive feedback on results and state

Possible Problems• Differentiation problems• Inability to detect mistakes• Unrecoverable problems

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Accident: How do I avoid mistakes?Accident: How do I avoid mistakes?

GUI Answers• Control/guide in direct manipulation

• Stop, cancel, undo or delete

Exposed Challenges• Controlling or canceling system actions in progress

• Disambiguating what to undo in time

• Intervening when user makes obvious error

Possible Problems• Unintended action

• Inability to recover state

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

DiscussionDiscussion

UbiComp is still a new growing field

Now is the time to create new methods of interaction between Humans and Machines

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

OutlineOutline

Coping with Uncertainty

Making Sense of Sensing Systems: Five Questions for Designers and Researchers

Distributed Mediation of Ambiguous Context in Aware Environments

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Ambiguous Context DataAmbiguous Context Data

Many systems are based on the assumption that context data is unambiguous

However, accurately determining context is a hard problem since the system isn’t in possession of all the facts

Techniques such as AI algorithms can be applied but are not always accurate and require some training

Sometimes it is necessary to ask the human in order to get unambiguous context data

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Architecture of the Extended Context Toolkit

Architecture of the Extended Context Toolkit

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Disambiguating Sensor DataDisambiguating Sensor Data

In an ubiquitous system where all the devices behave in a distributed manor disambiguating data is a shared problem

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Issues in Building Realistic Context-Aware Applications

Issues in Building Realistic Context-Aware Applications

Distribution: Devices are in a distributed environment

Storage: When should ambiguous data be stored?

Multiple Subscription Types: Allow subscribers to opt-in/out and choose their type of subscription

Pre-emption of Mediation: Prevent multiple simultaneous mediations

Forced Mediation: Request mediation by other devices

Feedback: Provide feedback to users about what the system thinks is happening

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

StorageStorage

Should ambiguous data be stored?

Should mediated data be stored?

Both?

How do we store ambiguous data since it is actually a graph of options

Storing only unambiguous data is easier

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

SubscribersSubscribers

Opt-in and Opt-out subscriptions allow subscribers to get only what they want and no more

Those who get ambiguous data will have to mediate it, wait for another to mediate it or force mediation

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Pre-Emption of MediationPre-Emption of Mediation

Mediation may occur simultaneously in several components

If several components are mediating at once the first one to succeed pre-empts the others

Does this mean the users will get multiple requests for mediation?

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

Forced MediationForced Mediation

If a subscriber is unable or doesn’t want to mediate it can request another component to mediate

It passes all ambiguous data to the other component which mediates if possible

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

FeedbackFeedback

Provide feedback about what the system is doing and what its current state is

Feedback can come in many forms from messages to speech

• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/

ExamplesExamples

In/Out Board•Monitored when residents of a single building

where present or not

CybreMinder•A situation aware reminder system which can

use situational data to trigger reminders

Word Predictor•Makes typing words easier since it guesses

context

CMU Usable Privacy and Security Laboratoryhttp://cups.cs.cmu.edu/

Questions?