The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory

Living Environments Lab

Human-ComputerInteractionInstitute

Carnegie Mellon

Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos{stace, anind, hudson, paulos} @ cs.cmu.edu

The Effectiveness of Haptic Cues as an Assistive Technology for Human

Memory

Human Memory

Carnegie Mellon

Memory Impairments

Carnegie Mellon

Visual and Audio Cues

DeVaul, R.W., Pentland, A. and Corey, V.R. The memory glasses: subliminal vs. overt memory support with imperfect information. (2005)

Lin, W. and Hauptmann A.G.A Wearable Digital Library of Personal Conversations. (2002)

Disruptive

Prominent

Carnegie Mellon

Vibro-Tactile Cues

Unobtrusive

Discreet

Carnegie Mellon

Vibro-Tactile Interaction

Carnegie Mellon

Hypothesis

Vibro-tactile cues can improve human memory recall and recognition

Carnegie Mellon

Priming

Effective cross-modal priming for visual and haptic stimuli

Ballesteros S., Reales J.M., Mayas J. and Heller M.A. Selective Attentionmodulates visual and haptic repetition priming: effects in aging and Alzheimer'sdisease. (2008)

Carnegie Mellon

Attention and Navigation

Haptic feedback aids attention and navigation

Young, J.J., Tan, H.Z., Gray, R. Validity of haptic cues and its effect on primingvisual spatial attention. (2003)Kildal, J. and Brewster, S.A. VibroTactile External Memory Aids in Non-Visual Browsing of Tabular Data. (2006)

Carnegie Mellon

Multimodal Priming

Retrieval of audio and video information is assisted with vibro-tactile cues

Carnegie Mellon

Wearable Haptic Feedback Device

Carnegie Mellon

Cue 1: two medium pulses Cue 2: strong oscillating pulses

Cue 3: short soft pulse

Example Cues

Training Phase

Carnegie Mellon

Procedure

Training Phase

Test Phase

Carnegie Mellon

Method

Carnegie Mellon

Four User Studies

“Blending”

Audio Recognition

Visual Recognition

Free Recall

Carnegie Mellon

Blending

24 participants, 15 male, 9 female, ages 21-65

Shape Recognition Test

Carnegie Mellon

Auditory Recognition

20 participants: 12 male, 8 female, ages 21-65 Mandarin word translation test

Carnegie Mellon

Visual Recognition

22 participants: 16 male, 6 female, ages 21-65 Face memory test

Carnegie Mellon

Visual Recognition Test

Carnegie Mellon

Free Recall

24 participants: 16 male, 8 female, ages 21-65

Ebbinghaus nonsense syllable test (“hik”, “lup”)Ebbinghaus, H. Memory: A Contribution to Experimental Psychology. Teacher’sCollege (1913). Dover Publications.

Carnegie Mellon

Results

Audio recognition

Shape recognition

Face recognition

Free recall

+11% (5 trials, p = 0.075)

no effect

-20% (7 trials, p = 0.009)

Carnegie Mellon

Haptic Cues and Free Recall

Significantly lower performance with cues

Declarative vs. Perceptual memory

Our cues did not have semantic meaning

Carnegie Mellon

Haptic Cues and Cognitive Load

Cues were ‘distracting’

Perceptual differences between users

Carnegie Mellon

Haptic Cues as a Memory Aid

Memory aid for people with poor or impaired memory

Memory enhancing treatments often target low performers

Aldenkamp, A. P., et al., Effect of Sabeluzole (R 58 735) on Memory Functions in Patients with Epilepsy. (1995)

Mattay, V. S. et. al.. Effects of Dextroamphetamine on Cognitive Performanceand Cortical Activation. (2000)

Carnegie Mellon

Opportunities

Cue vocabulary

Maximum number of cues

Effectiveness of cues for people with functional memory impairments

Carnegie Mellon

Pulse Duration

Pulse 1 Pulse 2

Carnegie Mellon

Pulse Intensity

Pulse 1 Pulse 2

Carnegie Mellon

Delay Between Pulses

Pulse 1 Pulse 2

Carnegie Mellon

Cue Vocabulary Study

14 Participants, age 18-45Stacey Kuznetsov, Anind K. Dey, Scott E. Hudson, Eric Paulos

{stace, anind, hudson, paulos} @ cs.cmu.edu

Carnegie Mellon

Ratings of Pulse Duration Differences

Carnegie Mellon

Ratings of Pulse Intensity Differences

Carnegie Mellon

Ratings of Differences in Delay Between Pulses

Carnegie Mellon

Ongoing Work

Systematically pick cues from the sampled space

Determine the maximum number of cues that can be remembered

Recruit subjects with functional memory impairments

Carnegie Mellon

Future Work

Adjust cue intensity to individual’s perception

Strive for context-awareness

Add semantic meaning to cues

Carnegie Mellon

The Effectiveness of Haptic Memory Cues

Haptic cues as a memory aid for poor performers

Assistive technology for memory-impaired users

Carnegie Mellon

Vibro-Tactile Devices

Carnegie Mellon

Roberta KlatzkyNSF Intel

Thank you

http://isiria.files.wordpress.com/2008/04/foreign_language.jpghttp://internetservices.readingeagle.com/blog/gavel/archives/cellphone.jpg

http://www.blendfilms.com/movies/handshake_small.movhttp://spiddlement.files.wordpress.com/2008/05/todolist.jpg

http://static.howstuffworks.com/gif/brain-1.jpghttp://www.sytechcorp.com/new_site/SytechCorp/SyTechCorpXY/images/Pager_AA021263.jpg

Image Credits

Carnegie Mellon

Questions

Kuznetsov, S., Dey, A. K., Hudson, S. E. The Effectiveness of Haptic Cues as a Ubiquitous Memory Aid. Pervasive Computing, 2009

Carnegie Mellon

Sixth Sense, MIT Media Lab, 2009

Carnegie Mellon

Point localization thresholds for males for different areas of the body(Weinstein, 1968)

Carnegie Mellon

Questions

Kuznetsov, S., Dey, A. K., Hudson, S. E. The Effectiveness of Haptic Cues as a Ubiquitous Memory Aid. Pervasive Computing, 2009

The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory

Documents

Transcript of The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory

258 IEEE TRANSACTIONS ON HAPTICS, VOL. 8, NO. 3 ...ryokitada.dolphin.jp/img/Pawluk2015review_IEEE_ToH1.pdfDesigning Haptic Assistive Technology for Individuals Who Are Blind or Visually

MSc Version 4a - cs.ubc.ca pedrosa eng. “josé ... of figures_____ v list of tables ... effects of haptic and visual cues on ability to follow rhythms ___ 30

IMGD 3xxx - HCI for Real, Virtual, and Teleoperated ... · IMGD 3xxx - HCI for Real, Virtual, and Teleoperated Environments: Haptic Cues by Robert W. Lindeman gogo@wpi.edu

Haptic playback

Haptic Ppt

Determining the Contribution of Visual and Haptic Cues during …etheses.whiterose.ac.uk/12557/1/Fakhoury_E_Mechanical... · 2016-04-13 · Determining the Contribution of Visual

Haptic material: a holistic approach for haptic texture ...fdanieau.free.fr/pubs/Eurohaptics_2018_HapticMaterial.pdf · \Haptic material": a holistic approach for haptic texture mapping

Bookshare: Large Print using DAISY and Microsoft Word …/Documents/Assistive Technology/… · Web viewand has access to Microsoft Word 2007. These instructions give visual cues

Haptic processing 1 Incidental haptic sensations influence ...web-docs.stern.nyu.edu/marketing/JAckermanPaper.pdf · Haptic processing 1 Incidental haptic sensations influence social

Haptic Technology

The Rice Haptic Rocker: skin stretch haptic feedback with ...€¦ · The Rice Haptic Rocker: skin stretch haptic feedback with the Pisa/IIT SoftHand Edoardo Battaglia 1, Janelle

Haptic rendering for under-actuated 6/3-DOF haptic devices

Touching the invisible: localising ultrasonic haptic cues (World Haptics 2015 talk)

IMGD 3100 – Novel Interfaces for Interactive Environments ...web.cs.wpi.edu/~gogo/courses/imgd3100/slides/imgd3100_06_Haptic… · for Interactive Environments: Haptic Cues ...

Haptic Technologies

Curriculum Vitae of Katherine J. Kuchenbecker · and interact with humans. My goal is to elevate and formalize our understanding of haptic cues while simultaneously uncovering new

Assistive Technology Assistive Technology Assistive Technology. Assistive Technology Jim Ellam Staffordshire County Council.

MOBILE ASSISTIVE TECHNOLOGIES FINAL RG - … is being directed at making mobile phones and other handheld devices accessible via our haptic (touch) and audio sensory channels. We review

Contents Haptic interaction - Heriotruth/year4VEs/Slides08/L13.pdf · Haptic interaction Ruth Aylett Contents Haptic definition Haptic model Haptic devices Measuring forces Haptic

Haptic-Emoticon: Haptic Content Creation and Sharing System to

MSc Version 4a - cs.ubc.ca pedrosa eng. “josé ... of figures___ v list of tables ... effects of haptic and visual cues on ability to follow rhythms _ 30