Neural Interfaces and How They Use Signal Processing May ...
Transcript of Neural Interfaces and How They Use Signal Processing May ...
![Page 1: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/1.jpg)
Neural Interfaces andHow They Use Signal Processing
Sarah FelixMay 12, 2016
IEEE Signal Processing Society, Santa Clara Chapter Event
![Page 2: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/2.jpg)
Outline
● What is a neural interface?○ Case study: Artificial retina
● Perceiving neural recordings○ Case study: Brain-machine interface○ Case study: Combining ECoG with other information
● Stimulating the senses○ Case study: Encoding sound in a cochlear implant○ Case study: Prosthetic limb with sensory feedback
● Closing the loop
![Page 3: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/3.jpg)
Neural Interface: biology + technologyEngineered system that interacts with the brain and/or peripheral nerves
![Page 4: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/4.jpg)
Neuron 101
Hodgkin & Huxley model (1952)gkin
Cell membrane
Action potential
pump
Ion channels
Dendrites (input)
Axon
Synapse (output)
Cell body
wikipedia
![Page 5: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/5.jpg)
Retina
Bipolar Cells
Rod & Cone Cells
Ganglion Cells
Optic Nerve
Visual Cortex
![Page 6: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/6.jpg)
Artificial Retina
![Page 7: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/7.jpg)
Electronics Package
ElectrodeArray
Antenna
Implant Components
![Page 8: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/8.jpg)
Thin film technology enables neuron-sized electrodes
240 Electrodes
![Page 9: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/9.jpg)
It takes a large team to develop a full system
![Page 10: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/10.jpg)
Considerations for choosing an interface
Recording vs. stimulation Selectivity vs. invasiveness
Invasiveness
![Page 11: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/11.jpg)
Making sense of neural recordings
![Page 12: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/12.jpg)
Electrodes detect voltage fluctuations from neurons
FILTERS MUX ADC
![Page 13: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/13.jpg)
Tradeoffs of different recording types
Single neuronSpikes
More InvasiveSpatially localized
Time domain analysisTime series, probabilistic
Many neuronsOscillations
Less invasiveSpatially aggregated
Frequency domain analysisWavelet, time-frequency
Depends on the goalUnderstanding neural circuitry?
Classifying signals corresponding to brain states?Inferring stimuli or inputs?
Controlling a prosthetic (BMI)?
![Page 14: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/14.jpg)
Single unit recording● Detection
○ Step 1: Transformation, e.g:■ Simple band pass filter■ Wavelet transform■ Likelihood test
○ Step 2: Threshold or criteria
● Classification (“spike sorting”)○ Principal component analysis○ Template matching
OUTPUT: Event times
Voltage
Transform
Software from DataWave Technologies
Kim, K. H. and Kim, S. J., “A wavelet-based method ..."Biomed. Eng., IEEE Transactions, 2003.
![Page 15: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/15.jpg)
Single unit recording● Analyze spike trains
○ Extract features: firing rate, burst rate○ Characterize interspike interval (ISI) distribution○ Map spatial and temporal correlations○ Model as a point process
Histogram of inter-spike intervalsRaster plot of spike activity from 35 neuronsTime
Neu
ron
![Page 16: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/16.jpg)
Decoding for Brain Machine Interface
1. Statistical model of neural spiking
2. State evolution model
3. Fit parameters to the model4. Bayesian estimation algorithm (e.g. Koyama, Eden, Brown, Kass, 2010)
Velocity control signal to cursor
Subject sees target and thinks about intended motion
Firing rate, neuron i
Intended Velocity
Spike sorting may not be necessary (Fraser, Chase, Whitford, Schwartz, 2009)
![Page 17: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/17.jpg)
Field potentials and oscillatory signalsCoherence and spatial correlation
“Spatiotemporal dynamics of word processing in the human brain,” Conolty et al., Frontiers in Neuroscience, 2003
From one electrode
e.g. Mean Phase Locking Valuee.g. Gabor transform
Time-frequency analysis
ECoG electrodes Single electrodeColor = power
Freq
uenc
y (H
z)
TimeFrequency (Hz)
Inte
r-el
ectr
ode
dist
ance
(mm
)
![Page 18: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/18.jpg)
Multimodal decoding of...fear?
ECoG
Audio
Heart rate
Power in frequency bands on each electrode
Human vocal signatures? Pitch?
Rate
Clusters identified by correlating Audio and Heart
● Validate using psychometrics
● Map clusters back to brain location
![Page 19: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/19.jpg)
Stimulating the senses
![Page 20: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/20.jpg)
Examples of neural interfaces that use stimulation
● Therapeutic stimulation○ DBS○ Vagus nerve stimulation○ TMS and tDCS
Wikimedia commons
● Sensory prosthetics○ Cochlear implant○ Artificial retina○ Vestibular implant○ Upper limb prosthetic
Wikipedia
![Page 21: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/21.jpg)
Schematic of a neural prosthetic
Stimulus
Generate stimulus signal
Still healthy neural circuitry
Brain’s perception
Need to get this input output model
Neural circuitry
Stimulus Brain’s perception
Replaces and approximates damaged neural circuitry
Transducer Feature extraction
![Page 22: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/22.jpg)
Encoding sound in the cochlear implant
Wikimedia commons
MICROPHONE
PROCESSOR
Filter bank → separate frequency components
Rectifying low pass filter → amplitude envelope
Pulse generator
ELECTRODES
Series of biphasic “charge balanced” pulses
Different locations on the cochlea are tuned to different frequencies
Active area of research to improve perception of:
● speech with background noise
● music
![Page 23: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/23.jpg)
Relaying touch from a prosthetic hand ● Input/output model based on descriptive feedback from amputee volunteers● Stimulation frequency proportional to force sensor output allowed patient to
handle delicate objects
Tan, Daniel W., et al. "A neural interface provides long-term stable natural touch perception." Science translational medicine 6.257 (2014): 257ra138-257ra138.
Electrode location
Pulse width modulation
Pulse frequency
Hand location
Sensation quality
Intensity
http://engineering.case.edu/Tyler-prosthetic-sensation
![Page 24: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/24.jpg)
Closing the loop
![Page 25: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/25.jpg)
Smart deep brain stimulation
● Epilepsy suppression● Mood disorders
Feedback from ECoG-like electrodes
See: “Smart neural stimulators listen to the body,” T. Dennison, M. Morris, F. Sun, IEEE Spectrum, Jan. 2015.
![Page 26: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/26.jpg)
Bioelectronic medicine - a new frontier
Organ innervation
Vagus nerve stimulation
Feedback from biochemical signatures…?
● Cardiac regulation● Gastro-intestinal health● Inflammation and pain
management
![Page 27: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/27.jpg)
Summary
● Neural interfaces leverage the electrical and network characteristics of human + machine
● Neural recordings present a plethora a SP problems for brain machine interfaces and basic neuroscience
● Sensory prosthetics stimulate neurons to restore lost function - identifying the right I/O model is critical
● Closed loop interfaces are a new frontier promising therapies for hard-to-treat conditions
![Page 28: Neural Interfaces and How They Use Signal Processing May ...](https://reader031.fdocuments.in/reader031/viewer/2022020700/61f60a187bfc895c0b5785fa/html5/thumbnails/28.jpg)
Acknowledgements and other referencesThank you to Sat Pannu at Lawrence Livermore National Laboratoryneurotech.llnl.gov
● Articles and texts not listed in slides○ Rubinstein, Jay T. "How cochlear implants encode speech." Current opinion in
otolaryngology & head and neck surgery 12.5 (2004): 444-448.
○ Gibson, Sarah, Jack W. Judy, and Dejan Markovic. "Spike Sorting." IEEE Signal processing magazine 29.1 (2012): 124.
○ Statistical Signal Processing for Neuroscience and Neurotechnology, Karim Oweiss○ Signal Processing for Neuroscientists, Wim van Drongelen○ Analyzing Neural Time Series Data, Mike X Cohen
● Coursera: Computational Neuroscience, Rajesh Rao & Adrienne Fairhall