Making More Senses of UX

UX Cambridge 2014

Alastair Somerville

Twitter@acuity_design

Hashtag#SensoryUX

Housekeeping

Safety

Flotation Aid

Use Table as

What's this workshop about?

LearningObjectives

Confusion&

Uncertainty

Understanding more about:•yourself•how technology works with people•How our senses and minds work•What could happen in the future

Be flexible&

Be questioning

Themes

WearablesSmart Cities

Internet of Things

My Cat is sad because Apple Watch is just another wrist mounted notification screen with health sensor

Sensor Algorithm Visualisation

Action

User

Sensors withoutsocial sensesare nonsense

Sensor Algorithm Visualisation

Action

User

New Stakeholders?

What can we do?

Relearningour senses

Going beyond screens

“History of the future in 100

objects”Adrian Hon

Introductions

1. Name2. What you're doing

MultimodalityMultimodality SensoryFuturesSensoryFutures Q&AQ&A

Event Plan

Meaning & Memory

Meaning & Memory

Accessibility Accessibility

MultimodalityMultimodality PlaytimePlaytime Show & TellShow & Tell

Multimodality

Meaning Meaning

GestureGesture PlaytimePlaytime Show & TellShow & Tell

Meaning & Memory

Memory

Neuroplasticity Neuroplasticity Sensory

SubstitutionSensory

Substitution

Sensory Futures

Sensory Augmentation

Sensory Augmentation

Any questions?

Multimodality

Accessibility

Tactile projects

Confusion

Haptics

Touch

Smell

Taste

Sight

Hearing

Touch

Smell

Taste

Sight

Hearing

1

2

3=

4

Touch

Smell

Taste

Sight

Hearing

CognitiveSlopTime

Chion

More Senses

SightVision

HearingAudition

TasteGustation

SmellOlfaction

TouchSomatosensory

Balance & accelerationEquilibrioceptionTemperature

ThermoceptionKinaesthetic senseProprioception

PainNociception

TimeChronoception

Other internal sensesInteroception

Playtime• Communicating without seeing• Teams create messages with

anything here• Use Touch to communicate• Allocate one person as listener• Prototype quickly, learn fast

Context

In a bar

Discovery

Difficult

Confusion

Metaphor

Emotion

BREAK

Any Questions?

Meaning & Memory

New rules

If we want to talk in new ways through new

technology, we need to agree new rules

Meaning

Meaning1 Understanding

2 Creating3 Sensory

1Understanding

meaning

InformationTheory

2 parts toConversation

What we say

Confirming understanding

Loop

What we say

Confirming understanding

This is why active Social Media matters

2Creatingmeaning

Metaphor

Language isbuilt on

Metaphor

This thing is LIKE these things

SIMILE

This thing is LIKEthis thing

ANALOGY

This thing isthis thing

METAPHOR

SimileSimile

Simile

Simile

SimileSimile

Simile

Analogy Analogy

Metaphor

Mutual agreement of meaning

Proposal of meaning

Agreement of meaning

3SensoryMeaning

Body Language

Spiteful Resting Face

Sign Language

Signs as wordsFaces as grammar

Gesture

Gesture

Words Symbols

MessagesIn symbols

In physical gestures

Translate to Words

Discovery

Makaton

Mutuality

CulturallySpecific

We can use total experience to

create messages

But need to balance definitions &

information across senses

Memory

Warning

Neuroscience is not certain!

Neurotypical behaviourLimited sample numbers

FMRI assumptions

Memory1 Senses

2 Movement3 Social

1Senses

We recall the past with the senses of the

present

The greater the sensory engagement, the stronger the recall

2Movement

We move to think

We think to move

Our journeys define our memories

3Social

Stories

We remember what we want to share

BREAK

Any Questions?

Sensory Futures

Neuroplasticity

Sensory Substitution

Learn from impairment to understand

augmentation

Daniel Kish

BrainPort

USArmy

USArmyStudy

V.E.S.T.

Playing with Sensory Substitution

Heareapp

Route Codescdsljru

cwzvhid

Warning!

Let’s go!

Discovery

Creatingnew senses

Decision to change Decision to change

Necessary but not sufficient Mindfulnes

spractice

Mindfulness

practice

Sensory practiceSensory practice

Neuroplastic changeNeuroplastic change

Necessary but not sufficient

Successful Sensory Substitution

Successful Sensory Substitution

Emotional desire to changeEmotional desire to change Key gateway for success

A behavioral model for Neuroplastic Sensory Substitution

TIME

Augmentation

ConnectedIntimacy

TailoredEcosystems

Co-EvolvedPossibilitiesPerson As ComputerPerson To Computer

Person To Person

Social Technology

Futures

Connectedintimacy

Enabling empathy

Supporting decision making

Ego Depletion

Tailoredeco systems

Shared places

Smarter places

Co-Evolved possibilities

Embodiment

Technology as part of us

FinalThemes

UI is aboutpersonal emotion

UX is aboutcommunal empathy

Connecting information to context

Context in Community

PlaceTime

Emotion

Remember we have9+ senses!

SightVision

HearingAudition

TasteGustation

SmellOlfaction

TouchSomatosensory

Balance & accelerationEquilibrioceptionTemperature

ThermoceptionKinaesthetic senseProprioception

PainNociception

TimeChronoception

Other internal sensesInteroception

We can make moreof ourselves

FinishingUp

Sensory UXdepends on innovation

Newtechnology canbe used now

How well it workswith peopledepends...

It's aboutproduct designservice design

sensesemotions

social connections

Technology works in a human context of

social and sensory needs

What to do?

Talk

Tell stories

Test

Throw stuff away

Keep trying

Be flexible&

Be questioning

Q&A