Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual...
-
Upload
verity-park -
Category
Documents
-
view
217 -
download
0
Transcript of Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual...
![Page 1: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/1.jpg)
Towards a real-time, configurable, and affordable
system for inducing sensory conflicts in a virtual
environment for post-stroke mobility rehabilitation:
vision-based categorization of motion impairments
Babak Taati, Jennifer Campos, Jeremy Griffiths, Mona Gridseth, Alex Mihailidis
Sept 11, 2012ICDVRAT
![Page 2: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/2.jpg)
Outline
Motivation
Background
Problem Definition
Tracking Technologies
Method
Experimental Results
Conclusions & Future Work
2
![Page 3: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/3.jpg)
3
Post-Stroke Rehabilitation
15 million people suffer stroke each year
65% of stroke survivors have difficulty using their upper limbs
The economic cost of stroke is ~$6.3 billion a year
Training exercises that provide patients with visual feedback on the movement of the affected limb facilitate rehabilitation
Motivation
![Page 4: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/4.jpg)
Mirror Box Illusion
View of the affected arm is replaced by the mirror reflection of the healthy arm
Simultaneous motion, or attempt of motion of both arms results in artificial visual feedback which reinforces neurorehabilitation
Mirror box therapy is effective in restoring mobility
4
Background
http://en.wikipedia.org/wiki/Mirror_box
(Ramachandran et al, Nature,1995)
![Page 5: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/5.jpg)
Rubber Hand Illusion
5
http://www.thefrogwhocroakedblue.com/illusions.asp
(Botvinick and Cohen, Nature, 1998)
Simultaneous brushing of the rubber hand and the real hand generates a sensory conflict in the brain between the location of the visual vs. proprioceptive signal
Continued brushing typically results in a sensory takeover of proprioception and the person would feel ownership of visually perceived plastic arm
Background
![Page 6: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/6.jpg)
Objective
Reproduce and combine the Mirror Box and the Rubber Handillusions in a virtual environment
• Computer vision: markerless skeleton tracking
• Computer graphics: visualization + pose augmentation
• Machine learning: classification + latent space projection
6
I felt my arm was here …
… but I see it here!
Problem Def.
![Page 7: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/7.jpg)
Milestones
Duplicate the Rubber Hand Effect
• Visual skeleton tracking & visualization
• Synchronized tactile stimulation
Combine the Rubber Hand with the Mirror Box Effect
• Categorize motion impairments
• Apply a movement gain (normalize / exaggerate)
7
Problem Def.
![Page 8: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/8.jpg)
8
Human Pose Tracking
8
[Kanaujia, 2011]
Tracking Tech.
![Page 9: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/9.jpg)
9
Human Pose Tracking (cont’d)
Marker-based• Elaborate setup
Markerless (vision-based)• Accuracy vs. computational efficiency
9
[CMU Motion Capture Database]
[Flickr: beratus]
[Lee & Nevatia, 2009]
Tracking Tech.
![Page 10: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/10.jpg)
10
Consumer Depth Cameras
10
KinectTM (Microsoft) WAVI Xtion (ASUS / PrimeSense)
Real-time color & depth sensing
Real-time skeleton tracking software libraries
• Microsoft SDK or NITE
Tracking Tech.
![Page 11: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/11.jpg)
11
Challenges
Noise / inaccuracies / systematic bias Missing information Ground truth annotation Evaluation of categorical time series prediction
11
Tracking Tech.
True Labels
Predicted Labels
![Page 12: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/12.jpg)
12
Dataset
7 healthy adults
Simulated 2 of the most common upper limb flexion synergies present in the post-stroke population
• Elbow flexion, shoulder flexion, shoulder abduction
Stereotypical post-stroke movement synergy: Rotating the Trunk
• Elbow flexion
Stereotypical post-stroke movement synergy: Elevating the Shoulder
10x normally, 10x simulating an impaired motion
Method
![Page 13: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/13.jpg)
13
Algorithms
Features• Displacement Vectors• Principal Components Analysis (PCA)
Multi-class categorization• Logistic Regression• Support Vector Machines (SVM)• Random Forest (RF)
Cross validation• 7-fold leave-one-subject-out
Method
![Page 14: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/14.jpg)
Dominant Modes of Motion 1
14
Results
Reaching Across(Elbow flexion, shoulder flexion, shoulder abduction)
With shoulder elevation“Normal”
![Page 15: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/15.jpg)
Dominant Modes of Motion 2
15
Results
Reaching Up(Elbow flexion)
With trunk rotation“Normal”
![Page 16: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/16.jpg)
Classification Accuracy
ClassifierAction N Logistic Reg. SVM RF
Reach Across1 54.3 97.1 97.93 61.4 85.7 97.1
Elbow Flexion1 64.3 95.7 90.03 59.3 85.0 92.9
16
Results
# of PCA components
![Page 17: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/17.jpg)
17
Summary
Inherent biases were observed in the tracking of an elevated shoulder
Despite accuracy and tracking bias issues, it was possible to separate impaired motions
A single most dominant mode of motion, as identified by PCA, was sufficiently discriminative
SVM classifier obtained consistent detection accuracies >95%
Conclusions
![Page 18: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/18.jpg)
18
Future Work
Exaggerate / normalize motions
• Latent space projection
Combine the Rubber Hand and Mirror Box illusions
Real data (real post-stroke patients)
• Transfer learning
Conclusions
![Page 19: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/19.jpg)
19
Acknowledgements
![Page 20: Towards a real-time, configurable, and affordable system for inducing sensory conflicts in a virtual environment for post-stroke mobility rehabilitation:](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce55503460f949b2e55/html5/thumbnails/20.jpg)
Toronto Rehab (iDAPT research facilities)
20
Home Environment Laboratory (HEL) Challenging Environment
Assessment Laboratory (CEAL)