Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering...
Transcript of Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering...
![Page 1: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/1.jpg)
Brain – Computer Interfaces Physiologic basis for feature selection, and decoding techniques
![Page 2: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/2.jpg)
Brain – Computer Interfaces
Hochberg 2012
For dexterous motor control
![Page 3: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/3.jpg)
Brain – Computer Interfaces Control of end effectors
![Page 4: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/4.jpg)
Brain – Computer Interfaces Control of end effectors Communication
![Page 5: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/5.jpg)
Brain – Computer Interfaces Control of end effectors Communication Neuromodulation to
replace lost senses
![Page 6: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/6.jpg)
Brain – Computer Interfaces Control of end effectors Communication Neuromodulation to
replace lost senses Other Neuromodulation / Biofeedback
![Page 7: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/7.jpg)
Brain – Computer Interfaces Control of end effectors Communication Neuromodulation to
replace lost senses Other Neuromodulation / Biofeedback Consumer BCI
![Page 8: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/8.jpg)
Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations
![Page 9: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/9.jpg)
Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations
But in general: need to isolate, translate, and utilize a neural signal
![Page 10: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/10.jpg)
Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations
But in general: need to isolate, translate, and utilize a neural signal
![Page 11: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/11.jpg)
Architecture of a BCI
Classification / Regression
![Page 12: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/12.jpg)
Architecture of a BCI
Classification / Regression
![Page 13: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/13.jpg)
BCI Signal Types
(Picture credit: Wadsworth Center)
EEG (scalp)
ECoG (brain surface)
Intracortical Electrodes
Non-invasive
Invasive
![Page 14: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/14.jpg)
BCI Signal Types Signal Cell count Raw
Magnitude Feature Z (depends)
Spatial Specificity
Signal Stability
EEG (non-invasive)
> 1M ~50 uV 3-5 1-5 cm Long-term?
ECoG (semi-invasive?)
500K ~500 uV 10-20 3-10 mm Months
Intracortical (invasive)
1-??? 10s of mV Very high < 300 um Days
Appropriate modality choice depends on application. Consider subject population. Research/Clinical goals. Stimulation requirements.
![Page 15: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/15.jpg)
Architecture of a BCI
Classification / Regression
![Page 16: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/16.jpg)
Feature extraction, intracortical recordings
![Page 17: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/17.jpg)
Feature extraction, intracortical recordings The quest for single units
mean firing rates
![Page 18: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/18.jpg)
Feature extraction, intracortical recordings The quest for single units
Ensemble spiking
mean firing rates
![Page 19: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/19.jpg)
Feature extraction, intracortical recordings The quest for single units
Ensemble spiking
Local Field Potentials (LFPs)
mean firing rates
![Page 20: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/20.jpg)
Feature extraction, ECoG and LFPs
![Page 21: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/21.jpg)
Feature extraction, ECoG and LFPs
Spectral Estimation: STFFT Wavelets Band filtering and envelope detection Auto-regressive model
![Page 22: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/22.jpg)
Feature extraction, EEG Signal spreads as it passes through meat
![Page 23: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/23.jpg)
Feature extraction, EEG Signal spreads as it passes through meat
1) Correct for spatial spreading
Common Spatial Patterns – Linear combination of electrodes maximizing two class discriminability
![Page 24: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/24.jpg)
Feature extraction, EEG Signal spreads as it passes through meat
1) Correct for spatial spreading
Use of spherical head model as solution to forward model
Common Spatial Patterns – Linear combination of electrodes maximizing two class discriminability
![Page 25: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/25.jpg)
Feature extraction, EEG Signal spreads as it passes through meat
1) Correct for spatial spreading
Use of spherical head model as solution to forward model
Common Spatial Patterns – Linear combination of electrodes maximizing two class discriminability
Subject specific MRI as solution to forward model
![Page 26: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/26.jpg)
Feature extraction, EEG Signal spreads as it passes through meat
1) Correct for spatial spreading
Use of spherical head model as solution to forward model
Common Spatial Patterns – Linear combination of electrodes maximizing two class discriminability
Subject specific MRI as solution to forward model
2) Apply same spectral estimation techniques used in ECoG (50 Hz and below) for SMR and SSVEP Or Simple LPF for EPs
![Page 27: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/27.jpg)
Architecture of a BCI
Classification / Regression
![Page 28: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/28.jpg)
Decoding, intracortical recordings Translation of neural signal to one or more continuous variables
![Page 29: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/29.jpg)
Decoding, intracortical recordings
∑
−=
i ii r
rrmax
0ˆ pd
Population Vector
Translation of neural signal to one or more continuous variables
![Page 30: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/30.jpg)
Decoding, intracortical recordings
Kalman Filter
Estimate
Update
Bonus: Incorporates effector kinematics ∑
−=
i ii r
rrmax
0ˆ pd
Population Vector
Translation of neural signal to one or more continuous variables
![Page 31: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/31.jpg)
Decoding, intracortical recordings
Kalman Filter
Estimate
Update
Bonus: Incorporates effector kinematics ∑
−=
i ii r
rrmax
0ˆ pd
Population Vector
Many Others: Neural Networks, ARMA Models, etc
Translation of neural signal to one or more continuous variables
![Page 32: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/32.jpg)
Decoding, ECoG Translation of neural signal to one or more continuous variables, High SNR allows (causes ) us to be lazy.
![Page 33: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/33.jpg)
Decoding, ECoG Translation of neural signal to one or more continuous variables, High SNR allows (causes ) us to be lazy.
mean
std
dy/dt = (x-mu) / std
![Page 34: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/34.jpg)
Decoding, EEG Much harder computational problem, because of low SNR Neural signal typically translated to discrete variable with pre-defined (and pre-trained) number of states
LDA
Non-linear transform + LDA
SVM, Naïve Bayes, Decision Trees, Random Forest, Neural Network, on and on… BCI competition
![Page 35: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/35.jpg)
An Inherent Problem
![Page 36: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/36.jpg)
The Underlying Model
decoder effector movement
visual feedback
effector position
target position effector movement
decoder effector movement
visual feedback
effector position
target position
effector movement
Task goal Inverse model
Separately capable of adaptation
![Page 37: Brain – Computer Interfaces...Brain – Computer Interfaces Many Applications -> Many Engineering Requirements -> Many Architecture Considerations But in general: need to isolate,](https://reader030.fdocuments.in/reader030/viewer/2022040920/5e98ba0c3346fd46f66a9a0e/html5/thumbnails/37.jpg)
Closed-loop decoder adaptation