Designing easily learnable eyes-free interaction Kevin Li University of California, San Diego.
-
date post
22-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of Designing easily learnable eyes-free interaction Kevin Li University of California, San Diego.
touchtouch
mapping eyes free interactionmapping eyes free interaction
humanobject
hearinghearing
music speech sounds
Tapping and Rubbing
Tapping and Rubbing
Simulating buttons
Simulating buttons
PeopleTonesPeopleTones
Mapping language to tactile
Mapping language to tactile
stimulus class
sense
example project
PeopleTonesPeopleTones
lessons for application designerslessons for application designers
Vibratese Language - Communicate via vibrations
Map alphabet and digits to 5 vibrators
user with 65 hours training -> 90% accuracy, 38wpm
System capable of delivering at 67wpm
anything can be learned
• Ok, so let’s just translate all text from visual to auditory– Easily learnable– High bandwidth
• Wait, that sounds familiar
auditory feedback
0 1 2 3 4 5 6 7 8
Was not missing information
Knew position in the menu
Knew what day/time I was at
Felt in control of the conversation
Better for setting meeting times
Prefer if driving and talking
Prefer Overall
blindSight Smartphone
Overall preference
results
1. brevity is good, but use in moderationclarification of navigation overrides brevity
2. predictable/modeless user interface is key
3. auditory feedback goes a long way even during phone call(disclaimer: need to study how it interferes with activities… driving)
lessons
• Requires direct interaction with device
• Not usable in all scenarios– Loud environments (concerts)– Social environments– Driving and talking on the phone
auditory feedback doesn’t always work
Can we create tactile stimuli that feels like tapping or rubbing?
Will people associate it with what they already associate with tapping and rubbing in human-human communication?
research questions
cardboard
battery(metal)
eraser
IBM track pointer (sandpaper texture)
foam
IBM track pointer(rubber)
what feels like a finger?
Task 1: Distinguish
Given two taps of different
strength, which one feels stronger?
Given two taps of different frequency, which one feels faster?
error %for distinguish amplitude
1 2 3 4 5 6 7 1 2 2
2 0 10 0
3 2 6 17 4
4 6 17 15 6
5 14 6 29 8
6 4 13 25
7 2 21
Level of first stimulus
Leve
l of s
econ
d sti
mul
us
1 2 3 4 5 6 7 1 0 4
2 4 4 0
3 0 8 3 0
4 4 17 4 4
5 4 4 8 4
6 0 0 17
7 8 0
1 2 3 4 5 6 7 1 0 4
2 8 15 4
3 0 0 13 8
4 0 0 13 13
5 4 13 46 29
6 8 13 63
7 8 17
error %for distinguish frequency
Level of first stimulus
Level of first stimulus
Leve
l of s
econ
d sti
mul
us
Leve
l of s
econ
d sti
mul
us
ConstantDuration
ConstantNumber
Task 2: Identify
Given a tap rate its strength on a scale of 1-7.
Given a tap rate its speedon a scale of 1-7.
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7
Use
r Rep
orte
d Le
vel
Stimulus Level
Tapping - Amplitude Identification
01234567
1 2 3 4 5 6 7
Use
r Rep
orte
d Le
vel
Stimulus Level
Tapping - ConstantNumber Identification
01234567
1 2 3 4 5 6 7
Use
r Rep
orte
d Le
vel
Stimulus Level
Tapping - ConstantDuration Identification
questionnaire
How would you describe the tactile sensations you just experienced to someone who has not experienced them?
Which aspects of the experience felt natural and which aspects did not?
If your phone could generate these types of sensations, what would you like to use them for?
Tapping has human quality– 13 of 16 used the word ‘tap” in their description– 12 of 16 volunteered it had a human quality
Fast and slow are perceptually different– 12 participants mentioned harder taps don’t feel natural– 5 said fast ones don’t feel natural
Good for alerts– 6 volunteered that single taps would be good for silent
environments– 7 volunteered they would be useful when vibrations can’t
be felt (walking around)
results
Taps have a number of characteristics that make them desirable– Quiet– Strong
Rubbing is more subtle– Useful for in-the-hand scenarios
Number of taps and rubs is key element– Sometimes, this has pre-learned meaning– Limits number of viable distinct icons
design implications
only two states, nearby and far away
when a buddy is near, play their song
if phone is in vibrate mode, vibrate the equivalent
PeopleTones
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Vo
lts
(V
)0
1
.5
3.0
4.5
6.
0
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Vo
lts
(V
)0
1
.5
3.0
4.5
6.
0
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Vo
lts
(V
)0
1
.5
3.0
4.5
6.
0
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Vo
lts
(V
)0
1
.5
3.0
4.5
6.
0
Duty Cycle = 100%
Duty Cycle = 60%
Duty Cycle = 18%
Duty Cycle = 37%
remove noise
isolate 6.6kHz to 17.6kHz components using 8th order Butterworth Filter
use amplitude threshold, to keep only components greater than the average
takerunning sum
take running sum of absolute value, generate 1 value every 20ms
this keeps length consistent
exaggeratefeatures
compose output from previous step with power function
Axn , x is sample, A and n are constants,
10<=A<15, 1<=n<=2
lessons
cues in the wild should be music
higher comprehension rate when users select their own cues
obtrusiveness of music cues was not a concern
mapping music to vibration was most successful for people who knew the songs well
semantic association is key
problem
tactile feedback is always the same, but visual and motor has directional aspect
information is lost in the conversion
• Add state information to tactile feedback– Hover state– Moved to the left/right/up/down
• Where do we put it?– Under the button– Make the sides move– Tapping?– solenoid
solution
researchquestions
what are relevant characteristics of speech when mapping to tactile?
how do users naturally perceive these to be mapped?
pilot study
5 common phrases from text messaging literature
20 vibration sequences
Which phrase does this vibration feel like?
lessons
intonation is important
syllables should match number of pulses
duration should match (roughly)
touchtouch
mapping eyes free interactionmapping eyes free interaction
humanobject
hearinghearing
music speech sounds
Tapping and Rubbing
Tapping and Rubbing
Simulating buttons
Simulating buttons
PeopleTonesPeopleTones
Mapping language to tactile
Mapping language to tactile
stimulus class
sense
example project
PeopleTonesPeopleTones
lessons for application designerslessons for application designers
Jan Apr Jun Sept Dec
2008
Ubicomp Doctoral colloquium
Paper Deadlines
Projects
UISTDoctoral symposium
CHI Doctoral consortium
CHI
Doctoral Symposiums
Mapping language to
tactile
Mapping language to
tactile
Simulating buttons
Simulating buttons
2009
Paper Deadlines
Projects
Jan Apr Jun Sept Dec
Mapping language to tactilePart 2: Field Deployment
Mapping language to tactilePart 2: Field Deployment
CHIUIST
Simulating buttons
Simulating buttons Find a jobFind a job
2010
Paper Deadlines
Projects
Jan Apr Jun Sept Dec
Dissertation
Find a jobFind a job
Formalize music mapping