Interactivity & Design

David KirshDept of Cognitive Science

UCSD

Question

Designing for experience

is it similar to

Designing for efficiency

Topics

• Physical interactivity - an example

• Efficiency

• Fundamental Design Principle

• Metrics

• Experience

• Conclusion

Physical Interactivity: an example

• Complex folding sequence– Easy to forget

• Continuous interaction

• Difficult for one person

How would you make this process easier?

Think: context aware, digital elements in around

cardboard.

VideoQuickTime™ and a

YUV420 codec decompressorare needed to see this picture.

Design it better!

• Embed instructions (digital or physical)

– Where, when, how

• Improve manipulability

– Where, when, how

Fundamental Design Principle

Design Principle for Efficiency

right

Info

Tools

Cues

Constraints

Affordances

right Form, Place, Time, Pace

For a given task, design an environment so that it provides the:

to support efficient work activity.

Right for what?

• Pragmatic efficiency (getting job done quickly, few errors)

• Cognitive Efficiency

• Optimizes certain performance metrics

• Enhances experience of working/acting?

right

Info

Tool

Cue

Constraint

Affordance

right Form, Place, Time, Pace

Right(Info, Tools, Cues, Constraints, Affordances)

Constraint: Can’t fold in certain ways, forces compliance

Information: words, ‘fold in’+ instructions

Cues: arrows, colors, symbols of folding, fold marks, next fold lights up.

Tools: jig that holds and re-orients, actuators

Affordances: surfaces made for grasping

All revealed and concealed to manage attention

For cardboard task:

Right Info

• What is the right information to spatialize?

– Recipe study• problems with modularizing info

– Origami study• What activities should you support?

Spatially Distributed Recipe

ParticipantCOOKING TOOLS

INGREDIENTS

POTS & PANS

2/2 OF RECIPE

1/2 OF RECIPE Cooking phase

Assembling phase

RECIPE STIMULI: Spatially Distributed - improve localitySpatially distributed

After learning from pilot

Spatially Distributed Activity Map: Preparation

ParticipantCOOKING TOOLS

INGREDIENTS

POTS & PANS

2/2 OF RECIPE

1/2 OF RECIPE

Laying out ingredients, chopping,washing, measuring

Results

• Participants made better use of space– Used more surface, stabilized better, prepared

better

• BUT: – took much longer, – looked at recipe more

• Wanted to look ahead!

• Implication: we don’t always know what information is needed by users

What do subjects do?Origami

• More than fold– flip over, inflate, rotate, register, point, try out in gesture form …

• Pragmatic actions: needed to complete structure

– on sub-goal trajectory

– fold and some non-folding actions - flatten, flip over, inflate

• But we saw other actions that were not pragmatic but which seemed important for the subject

• Only a fraction of the actions performed are represented explicitly in origami instructions

To support activity we need to know what users do - their routines etc. ..the task structure.

Non-pragmatic Hand Actions

Registration

Verification

Gestural Thought

Focusing Attention

Trying out - exploratory

QuickTime™ and aH.263 decompressor

are needed to see this picture.

Upshot

• Choosing the right information to spatialize is hard– Even when workflow is known– What should be shown, when and where?– But often we have

little idea what thereal workflow involves

Right Form

Cognitive efficiency

– Faster processing

Given some information content, cue, constraint or tool how should it be displayed to support:

That is something up with which I will not put.

I won’t put up with that.

Police police police police police.

Police whom policemen police also police other police.

Topology vs. algebra

• Topological constraints are more natural

Right Form: Modality

Right modality

– Visual decisions are visual

– Audio statements free

visual search

Given some information content, cue, constraint or tool how should it be displayed to be the:

Interface 2

Interface 3

Rely on recognition rather than memory. Show, don’t just tell

Right Form

Visibility

- cue stands out

Given some information content, cue, constraint or tool how should it be displayed to have:

Right Form: visibility

R R R RR RR

R

R

RR R R

RRR P

Is there a non-R?

Right Form: visibility

R R R RR RR

R

R

RR R R

RRR P

Upshot

• Each step or phase in a routine or activity requires information or cues to be in the right form

• The right form may vary with step and task

• There are some general principles

Right Place

• The information or cue should be placed where you need it - given your resources and workflow

Starbuck’s cup

Right Place

On radio - logically

Volume, ChannelButtons

On steering wheel - workflow

Right Place

Interface 2

Goes together semantically goes together visually

Options are where they should be

Right place for cue when there are distracters

reduce descriptive complexityreduce visual complexity

Better place

Upshot• Spatializing information correctly depends

not just on workflow and resources but on:

– Showing semantic or work significant relations between information or cues - what goes with what

– A theory of attention• Should we use P or RRR P?

– Visibility or Place

Right Time

• See what I need to at this stage of activity– Eg. Jigsaw puzzle might show perimeter pieces first

• Show instructions, cues, tools just when needed in workflow – Hide tools in Illustrator™ that cannot be used in current

context

• Encourage right sequence - soft constraints– Travel: If I choose departure time first then calendar for

return time can be autoset– Show horizon of relevant options

Right Time: relevant optionsGiven structure of task - order of sub-goals - show only relevant actions

D H M

A B C E F G I J K L

orand

D H or M P

P

A

B

C

Upshot

• Time, place and form interact

• Get them right and users have what they need to make an informed choice right at their fingertips

Right Pace• Game coming at you too fast • Activity has a natural frequency• Slides in a presentation - comprehension rate

StressedBored

Cognitive Load

AttentivenessLoses focus

Disrupted

optimal

Right Pace: blind to change if slow

Change blindness

Before After

Fast enough and you see it.

Upshot

• Pace is the overall speed users find comfortable when performing their tasks

• Pace can change with mood and other user states

Bottom Line

• To give the right information at the right time … is equivalent to creating a– Dynamic keyhole - cognitive sweet spot

QuickTime™ and aAnimation decompressor

are needed to see this picture.

Metrics

Speed Accuracy & Design How fast can you fold the cardboard house?

Learnability & Design How long did it take to master the folding routine?

Complexity & Design How much more complex structures can you fold?

Error Recovery & DesignIf you make an error how long does it take to recover?

Variance & Design How much variance is there in your performance?

Bean Counters & DesignHow often do you damage or ruin the cardboard?

Interim Summary

• Designing for efficiency is hard• Requires developing a science of:

– Workflow analysis– Cue effectiveness, task informativeness– Cognitive complexity of forms

• E.g. visual simplicity, representational efficiency

– Impact of timing on routine efficiency– Impact of pace on attention, routine efficiency and

comprehension

Experience

Feeling versus Efficiency

Fast, few errors, easy, functional

Efficiency

Beautiful - clean, harmonious,

Kinetic - bodily movement

Tactile - texture, smooth

Emotional - erotic, rage, affectionate, love

Stimulating - intellectual,

Feeling

What feeling is there in state space environments?

Timeless •do things in sequence but state

transitions have no function beside moving to next state

•Qualitative feel of transition is irrelevant

States and Actions

At each node there are:• Possible actions

Goal is to reach end stateby selecting actions

Start

End End

Reflection One

• Experience is essentially in time– People are still experiencing even when system

is stopped– Experience is continuous never discrete

• It’s about the ‘time between’ the state transitions– About the process more than the outcomes

• Longer beautiful route rather than shortest uglier

Coverflow

QuickTime™ and aAnimation decompressor

are needed to see this picture.

What matters is the feel of the transition

Smooth, nice reflections - non-pragmatic properties

Design Principle

• For state space environments:– E.g. Forms, wisards, most websites, database

queries, dialogue based interfaces, most tool based interfaces (photoshop) …

– Design principles needs addition

Right(…) right (….) that makes user feel good

right

Info

Tool

Cue

Constraint

Affordance

right Form, Place, Time, Pace

Reflection Two

• What matters is:

– How agent engages an object or environment • He was hungry and attacked his food with gusto

–How agent feels when acting or watching While patting his dog he felt love and devotion

Continuous Control environmentsWorld acts back

continuously

Part of a system

Actions are continuous so not like state spaceoperators - everything is a transition

Continuous control

Theremin the first musical instrument designed to be played without being touched.

Highly interactive gesture based

Engagement and Feeling

• Depends on inner state: hormones, appetite, conditioning

• Agent can learn to engage or to feel

Reflection ThreeTwo forms of experience:

– Consciousness - qualia– Nature of sensation

– Cut a tomato - sharp knife

– Visceral feeling

– Content - interpreting as, seeing as, categorizing

Ecstasy of Fruit Loops

Qualia

• Varies between individuals• Sensitive to inner state: hormones,

emotions,• Some qualia - visceral feeling are the

result of mirror neurons - physical empathy

• If affected by stage setting, anticipation, scripting then not purely qualia

Interpretation needs anchoring

• Cue effectiveness– Image completion

– Word fragment completion

– Intentional frameworkcausation vs. self-propelling, desires

Rational gestures– Behaving ‘like a friend’

• Criteria for making an ‘inference-rich attribution’

wrd frgmnt

Upshot

• Study of projection of meaning - what is recognized and why - is necessary to determine:– Right (cue) right (Form, time, place, pace)

to engender an interpretation

Further reflections• Pleasure and delight is also derived from self-expression• Thought has an experiential component - intellectual

delight is content related• Absorption in environments often requires commitment

(this work is important) or the willing suspension of disbelief - that avatar is Josh

• Flow states are desirable but mostly ill-understood

Conclusion

• Designing for experience in state space environments is about the ‘time between’

• Designing for experience in continuous environments is about delight, absorption, engagement, hormones, emotions

• Both efficiency and experience design requires developing a science of:

right

Info

Tool

Cue

Constraint

Affordance

right Form, Place, Time, Pace

The End