Intelligent Prosthesis - tams. · PDF fileI Electrooculography (EOG) I Electrocorticogram...
-
Upload
nguyenkhuong -
Category
Documents
-
view
236 -
download
3
Transcript of Intelligent Prosthesis - tams. · PDF fileI Electrooculography (EOG) I Electrocorticogram...
MIN-Fakultät
Fachbereich Informatik
Intelligent ProsthesisFrom Sensor Data to Motion Generation
Irina Barykina
Universität HamburgFakultät für Mathematik, Informatik und NaturwissenschaftenFachbereich Informatik
Technische Aspekte Multimodaler Systeme
12. December 2016
Irina � Intelligent Prosthesis 1 / 21
OutlineMotivation Sensors EMG control systems Feedback Nice video References
1. Motivation
2. SensorsTypesMyoelectric sensor
3. EMG control systemsSystem overviewPreprocessing of dataNon-pattern recognition systemsClassi�cation with ANNLimitations
4. Feedback
5. Nice video
6. References
Irina � Intelligent Prosthesis 2 / 21
MotivationMotivation Sensors EMG control systems Feedback Nice video References
I Limb prosthesis for amputeesrehabilitation (approx. 10millions of amputees over theworld in 2008 [4]).
I Limb orthosis forrehabilitation of people witha violation of musculoskeletalsystem.
I Exoskeleton systems forassisting people in dailyroutines.
[www.touchbionics.com]
Irina � Intelligent Prosthesis 3 / 21
MotivationMotivation Sensors EMG control systems Feedback Nice video References
I Limb prosthesis for amputeesrehabilitation (approx. 10millions of amputees over theworld in 2008 [4]).
I Limb orthosis forrehabilitation of people witha violation of musculoskeletalsystem.
I Exoskeleton systems forassisting people in dailyroutines.
[www.touchbionics.com]
Irina � Intelligent Prosthesis 3 / 21
MotivationMotivation Sensors EMG control systems Feedback Nice video References
I Limb prosthesis for amputeesrehabilitation (approx. 10millions of amputees over theworld in 2008 [4]).
I Limb orthosis forrehabilitation of people witha violation of musculoskeletalsystem.
I Exoskeleton systems forassisting people in dailyroutines.
[www.touchbionics.com]
Irina � Intelligent Prosthesis 3 / 21
SensorsType of Input SignalsMotivation Sensors EMG control systems Feedback Nice video References
I Electromyography(EMG)I Mechanomyogram (MMG) or sound myogramI Electroencephalography (EEG)I Electrooculography (EOG)I Electrocorticogram (EcoG)
[www.science.education.nih.gov]Irina � Intelligent Prosthesis 4 / 21
SensorsType of Input SignalsMotivation Sensors EMG control systems Feedback Nice video References
I Electromyography(EMG)I Mechanomyogram (MMG) or sound myogramI Electroencephalography (EEG)I Electrooculography (EOG)I Electrocorticogram (EcoG)
[www.science.education.nih.gov]Irina � Intelligent Prosthesis 4 / 21
SensorsType of Input SignalsMotivation Sensors EMG control systems Feedback Nice video References
I Electromyography(EMG)I Mechanomyogram (MMG) or sound myogramI Electroencephalography (EEG)I Electrooculography (EOG)I Electrocorticogram (EcoG)
[www.science.education.nih.gov]Irina � Intelligent Prosthesis 4 / 21
SensorsType of Input SignalsMotivation Sensors EMG control systems Feedback Nice video References
I Electromyography(EMG)I Mechanomyogram (MMG) or sound myogramI Electroencephalography (EEG)I Electrooculography (EOG)I Electrocorticogram (EcoG)
[www.science.education.nih.gov]Irina � Intelligent Prosthesis 4 / 21
SensorsType of Input SignalsMotivation Sensors EMG control systems Feedback Nice video References
I Electromyography(EMG)I Mechanomyogram (MMG) or sound myogramI Electroencephalography (EEG)I Electrooculography (EOG)I Electrocorticogram (EcoG)
[www.science.education.nih.gov]Irina � Intelligent Prosthesis 4 / 21
SensorsMyoelectric SensorMotivation Sensors EMG control systems Feedback Nice video References
[MyoWare Muscle Sensor DevKit,www.sparkfun.com/products/13772]
[www.mananatomy.com/basic-anatomy]
Irina � Intelligent Prosthesis 5 / 21
EMG control systemOverviewMotivation Sensors EMG control systems Feedback Nice video References
Irina � Intelligent Prosthesis 6 / 21
Data SegmentationConstraintsMotivation Sensors EMG control systems Feedback Nice video References
De�nition
Segment is a time slot for acquiring myoelectric data considered forfeature extraction. [2]
Time constraints:
I Real time constraint: segmentlength + processing time ≤300ms
I Lower bound: 32ms
Signal state constraints:
I Transient state
I Steady state (preferred) [2]
Irina � Intelligent Prosthesis 7 / 21
Data Segmentation: WindowingWindowingMotivation Sensors EMG control systems Feedback Nice video References
I Adjacent Windowing
I Overlapped Windowing
I Continuous Segmentation(plus majority votingtechnique)
[6]
Irina � Intelligent Prosthesis 8 / 21
EMG control systemMajority VotingMotivation Sensors EMG control systems Feedback Nice video References
I Post-processing technique.
I Smoothes class decisions.
I Class at point t = most frequent class at points [t-m; t+m].
m ∗ Tprocess ≤ Tdelay ,where
Tprocess = time consumed during feature extraction,projection and classi�cation;
Tdelay = acceptable response time of the control system.
Irina � Intelligent Prosthesis 9 / 21
EMG control systemNon-pattern recognition systemMotivation Sensors EMG control systems Feedback Nice video References
T.Sono et al.[7] upper limb prosthesis with simple threshold control.
I Biceps deliver �exion, triceps -extension.
I Threshold T = 20% of maxcontraction.
I Open if closing EMG < T &&opening EMG > T
I Close if closing EMG > T &&opening EMG < T
I Otherwise => do nothing (noneed to keep musclescontracted).
[7]
Irina � Intelligent Prosthesis 10 / 21
EMG control systemNon-pattern recognition systemMotivation Sensors EMG control systems Feedback Nice video References
T.Sono et al.[7] upper limb prosthesis with simple threshold control.
I Biceps deliver �exion, triceps -extension.
I Threshold T = 20% of maxcontraction.
I Open if closing EMG < T &&opening EMG > T
I Close if closing EMG > T &&opening EMG < T
I Otherwise => do nothing (noneed to keep musclescontracted).
[7]
Irina � Intelligent Prosthesis 10 / 21
EMG control systemNon-pattern recognition system (cont.)Motivation Sensors EMG control systems Feedback Nice video References
Grasping obeject with 2 �ngers (upper line)and 1 �nger (bottom line). [7]
Irina � Intelligent Prosthesis 11 / 21
EMG control systemFeature ExtractionMotivation Sensors EMG control systems Feedback Nice video References
Red lines = mean values with applied low-pass �lter.Red dashed lines = onset of signal.
[3]
Irina � Intelligent Prosthesis 12 / 21
EMG control systemClassi�cation with ANNMotivation Sensors EMG control systems Feedback Nice video References
Vectorization of data in experiment of M.Gandolla et al. [3]
[3]
Irina � Intelligent Prosthesis 13 / 21
EMG control systemClassi�cation with ANNMotivation Sensors EMG control systems Feedback Nice video References
Calibration procedure in experiment of M.Gandolla et al. [3]
[3]
Irina � Intelligent Prosthesis 14 / 21
EMG control systemLimitationsMotivation Sensors EMG control systems Feedback Nice video References
I No tactile feedback. Visual feedback is not enough to providesubconcious control.
I No individual control on some of the muscles (true for innatelimbs as well).
I Necessity to concentrate and physically react during operation.
Irina � Intelligent Prosthesis 15 / 21
EMG control systemLimitationsMotivation Sensors EMG control systems Feedback Nice video References
I No tactile feedback. Visual feedback is not enough to providesubconcious control.
I No individual control on some of the muscles (true for innatelimbs as well).
I Necessity to concentrate and physically react during operation.
Irina � Intelligent Prosthesis 15 / 21
EMG control systemLimitationsMotivation Sensors EMG control systems Feedback Nice video References
I No tactile feedback. Visual feedback is not enough to providesubconcious control.
I No individual control on some of the muscles (true for innatelimbs as well).
I Necessity to concentrate and physically react during operation.
Irina � Intelligent Prosthesis 15 / 21
FeedbackCould device be included into user's body image?Motivation Sensors EMG control systems Feedback Nice video References
Experiment of Hernandez A. Alejandro et al. [1].
I Uses neural plasticity toassociate an unique event withnew stimuli.
I Substitutes tactile feedbackwith an electrical stimulation inhealthy arm.
I Results: brain shows activationof the sensory area related tothe amputated arm.
[1]
Irina � Intelligent Prosthesis 16 / 21
ConclusionMotivation Sensors EMG control systems Feedback Nice video References
I Patients are able to return to simple functional activities ofdaily life.
I Enhancement with biofeedback helps to prevent phantom painand neuro reconstraction.
I Unfortunately, contemporary solutions are still inconvenient
Irina � Intelligent Prosthesis 17 / 21
ConclusionMotivation Sensors EMG control systems Feedback Nice video References
I Patients are able to return to simple functional activities ofdaily life.
I Enhancement with biofeedback helps to prevent phantom painand neuro reconstraction.
I Unfortunately, contemporary solutions are still inconvenient
Irina � Intelligent Prosthesis 17 / 21
ConclusionMotivation Sensors EMG control systems Feedback Nice video References
I Patients are able to return to simple functional activities ofdaily life.
I Enhancement with biofeedback helps to prevent phantom painand neuro reconstraction.
I Unfortunately, contemporary solutions are still inconvenient
Irina � Intelligent Prosthesis 17 / 21
Nice VideoMotivation Sensors EMG control systems Feedback Nice video References
Upper limb prosthesis from John Hopkins University, AppliedPhysics Laboratory:https://www.youtube.com/watch?v=-0srXvOQlu0
[www.bloomberg.com]Irina � Intelligent Prosthesis 18 / 21
ReferencesMotivation Sensors EMG control systems Feedback Nice video References
[1] H. A. Alejandro, R. Kato, H. Yokoi, T. Arai, and T. Ohnishi.An fMRI Study on the E�ects of Electrical Stimulation asBiofeedback. In 2006 IEEE/RSJ International Conference onIntelligent Robots and Systems, pages 4336�4342, Oct. 2006.DOI:10.1109/IROS.2006.282006.
[2] M. Asghari Oskoei and H. Hu. Myoelectric control systems�Asurvey. Biomedical Signal Processing and Control, 2(4):275�294, Oct. 2007. ISSN 1746-8094.DOI:10.1016/j.bspc.2007.07.009. URLhttp://www.sciencedirect.com/science/article/pii/
S1746809407000547.
Irina � Intelligent Prosthesis 19 / 21
References (cont.)Motivation Sensors EMG control systems Feedback Nice video References
[3] M. Gandolla, S. Ferrante, D. Baldassini, M. Cotti Cottini,C. Seneci, and A. Pedrocchi. Arti�cial Neural-Network EMGClassi�er for Hand Movements Prediction, pages 640�643.Springer International Publishing, Cham, 2016. ISBN978-3-319-32703-7. DOI:10.1007/978-3-319-32703-7_123.URLhttp://dx.doi.org/10.1007/978-3-319-32703-7_123.
[4] M. LeBlanc. Give hope - give a hand, 2011. URLhttps://web.stanford.edu/class/engr110/2011/
LeBlanc-03a.pdf.
[5] C. J. D. Luca, A. Adam, R. Wotiz, L. D. Gilmore, and S. H.Nawab. Decomposition of Surface EMG Signals. Journal ofNeurophysiology, 96(3):1646�1657, Sept. 2006. ISSN0022-3077, 1522-1598. DOI:10.1152/jn.00009.2006. URLhttp://jn.physiology.org/content/96/3/1646.
Irina � Intelligent Prosthesis 20 / 21
References (cont.)Motivation Sensors EMG control systems Feedback Nice video References
[6] E. C. Orosco, N. M. Lopez, and F. di Sciascio.Bispectrum-based features classi�cation for myoelectric control.Biomedical Signal Processing and Control, 8(2):153 � 168,2013. ISSN 1746-8094.DOI:http://dx.doi.org/10.1016/j.bspc.2012.08.008.URL http://www.sciencedirect.com/science/article/
pii/S1746809412000900.
[7] T. Sono, L. Menegaldo, and M. Pinotti. Hand prosthesisprototype controlled by EMG and vibrotactile force feedback. In5th IEEE RAS/EMBS International Conference on BiomedicalRobotics and Biomechatronics, pages 91�95, Aug. 2014.DOI:10.1109/BIOROB.2014.6913758.
Irina � Intelligent Prosthesis 21 / 21