Mobile Note Taking: Mobile Note Taking: Investigating the Efficacy of Investigating the Efficacy of

Mobile Text EntryMobile Text Entry

Dr. Jo LumsdenDr. Jo LumsdenNRC IIT eNRC IIT e--BusinessBusiness46 Dineen Drive46 Dineen Drive

Fredericton, N.B., Canada, E3B 9W4Fredericton, N.B., Canada, E3B 9W4tel: 506tel: 506--444444--03820382

ee--mail: jo.lumsden@nrc.gc.camail: jo.lumsden@nrc.gc.ca

Andrew GammellAndrew GammellFaculty of Computer ScienceFaculty of Computer ScienceUniversity of New BrunswickUniversity of New Brunswick

Fredericton, N.B., Fredericton, N.B., Canada, E3B 5A3Canada, E3B 5A3

ee--mail: andrew.gammell@unb.camail: andrew.gammell@unb.ca

Mobile HCI 2004 2

introduction...• why has promise & hype of mobile devices surpassed substance?

– predominant perception that devices are hard to use– technology seen to dominate rather than support users

• most UI designs for wearables are based on desktop GUIs but users of wearable/mobile devices …– are in motion– cannot devote all attentional resources to device interaction

• aim: – create an audio-enhanced single-stroke text entry system that

demands as little visual resource as possible– assess the effectiveness of such a system

Mobile HCI 2004 3

a single-stroke text entry system...• support ‘eyes-free’ text entry when mobile

• design:– 3x3 conceptual grid overlaid on 1.3cm x 1.3cm writing pad– stylus used to enter text– character path condensed to ‘bin’ sequence– dynamic audio feedback to guide character entry progress

• 2 audio designs: – stereo-panned earcons– earcons creating virtual boundaries

Mobile HCI 2004 4

the alphabet...• restricted alphabet for testing• close resemblance to Roman alphabet representation

for most characters

Mobile HCI 2004 5

1 2 3

4 5 6

7 8 9

C5 (523.25Hz)

C4 (261.63Hz)

C3 (130.81Hz)

highest note

lowest note

audio-enhancement using stereo-panned earcons to represent bins...

Mobile HCI 2004 6

1 2 3

4 5 6

7 8 9

C5 (523.25Hz)

E4 (329.63Hz)

C3 (130.81Hz)

highest note

G3 (196.00Hz) C2(65.406Hz)

lowest note intensity increase

audio-enhancement using earcons to create “virtual” boundaries...

Mobile HCI 2004 7

• equipment:– Xybernaught MA V running Windows XP

• touch screen in non-preferred hand• stylus in preferred hand

– lightweight headphones to deliver audio

• handwriting style assessed– handprint, cursive, mixed

experimental set-up (1)...

instructed word

handwritten word

Mobile HCI 2004 8

• 3 experimental conditions:– static, no audio– mobile, no audio– mobile, with audio

• printed training material for each session– 5 minutes to practice, last 3 of which replicated session

condition setting

• experimental task:– enter ten 4-word English language phrases per condition

experimental set-up (2)...

Mobile HCI 2004 9

• 4-word phrase projected on one screen; output projected in real time on other screen (order was random)– no visual representation of output on touch screen

• 20m laps around obstacles for mobile conditions– mobile in fairly realistic but controlled environment

• full range of measures taken during experiment– time to complete tasks– accuracy rates (Unified Error Metric) & awareness of accuracy– subjective workload (TLX + annoyance)– percentage preferred walking speed (PPWS)

experimental set-up (3)...projection

screen

proj

ectio

n sc

reen

Mobile HCI 2004 10

• fully counter-balanced, between-groups study

• 2 main hypotheses:– mobility would have significant detrimental effect on text

input accuracy using single stroke alphabet– text input accuracy would be higher under audio conditions

than non-audio conditions when mobile

• also…– audio conditions would have greater detrimental effect on

participants’ PWS than non-audio condition when mobile– handprint writers would outperform others

experimental design...

Mobile HCI 2004 11

• workload:– mobility significantly increased overall workload– no significant difference in overall workload between audio

and non-audio conditions when mobile– audio feedback significantly reduced participants’

performance rating when mobile (greater awareness of error)

• preference:– bin-based audio condition significantly preferred to non-

audio condition

experimental results (1)...

Mobile HCI 2004 12

• accuracy:– cursive writers made significantly fewer errors than handprint

writers– bin-based audio returned significantly fewer errors than

boundary based audio– no significant difference in accuracy between audio mobile

and non-audio mobile conditions

• awareness of accuracy:– availability of audio feedback significantly increased

awareness of error• under mobile conditions, users entered significantly fewer

characters before realising and correcting error when provided with audio feedback

experimental results (2)...

Mobile HCI 2004 13

• handwriting-based techniques that combine audio and gesture have significant potential to support mobile note taking

• dynamic audio feedback improves awareness of text input accuracy when mobile– the simpler the audio feedback the better

• handwriting style influences accuracy– suggests need for tailorability to style

conclusions...

Mobile HCI 2004 14

visit the demos to try out the two different audio-enhanced systems for yourself!

Thank you...any questions?

Contact Details:Contact Details:NRC IIT eNRC IIT e--BusinessBusiness46 Dineen Drive46 Dineen Drive

Fredericton, Canada, N.B., E3B 9W4Fredericton, Canada, N.B., E3B 9W4tel: 506tel: 506--444444--03820382

ee--mail: jo.lumsden@nrc.gc.camail: jo.lumsden@nrc.gc.ca