Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson...
-
Upload
virgil-gray -
Category
Documents
-
view
215 -
download
1
Transcript of Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson...
![Page 1: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/1.jpg)
Electrosensory data acquisition and signal processing strategies
in electric fish
Mark E. Nelson
Beckman InstituteUniv. of Illinois, Urbana-Champaign
![Page 2: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/2.jpg)
How Electric Fish Work
![Page 3: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/3.jpg)
Distribution of Electric Fish
Fish tank upstairs
blackghost
knifefish
elephant-nosefish
![Page 4: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/4.jpg)
Electric Organ Discharge (EOD) - Spatial
![Page 5: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/5.jpg)
EOD - Temporal
![Page 6: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/6.jpg)
Electric Organ Discharge (EOD)
![Page 7: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/7.jpg)
Principle of active electrolocation
![Page 8: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/8.jpg)
mech
an
o
MacIver, fromCarr et al., 1982
Electroreceptors
~15,000 tuberous electroreceptor organs1 nerve fiber per electroreceptor organ
up to 1000 spikes/s per nerve fiber
![Page 9: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/9.jpg)
Individual Sensors (Electroreceptors)
VIN
nerve spikesOUT
![Page 10: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/10.jpg)
Neural coding inelectrosensory afferent fibers
![Page 11: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/11.jpg)
Probability coding(P-type) afferent spike trains
00010101100101010011001010000101001010
Phead = 0.333
Phead = 0.337 Phead =
0.333
![Page 12: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/12.jpg)
Principle of active electrolocation
![Page 13: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/13.jpg)
Electrosensory Image Formation
![Page 14: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/14.jpg)
Electrosensory Image Formation
![Page 15: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/15.jpg)
Electrosensory Image Formation
![Page 16: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/16.jpg)
Prey-capture video analysis
![Page 17: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/17.jpg)
Prey capture behavior
![Page 18: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/18.jpg)
Fish Body Model
![Page 19: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/19.jpg)
Motion capture softwareMotion capturesoftware
![Page 20: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/20.jpg)
MOVIE: prey capture behavior
![Page 21: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/21.jpg)
Electrosensory Image Reconstruction
![Page 22: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/22.jpg)
Voltage perturbation at skin :
Estimating Daphnia signal strength
waterprey
waterpreyfish ar
rE
/21
/133
electrical contrastprey volume
fish E-field at prey
distance from prey to receptor
THIS FORMULA CAN BE USED TO COMPUTE THE SIGNAL AT EVERY POINT ON THE BODY
SURFACE
![Page 23: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/23.jpg)
MOVIE: Electrosensory Images
![Page 24: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/24.jpg)
System Capabilities
Electric fish can analyze electrosensory images to extract information on target
direction (bearing) distance size shape composition (impedance)
![Page 25: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/25.jpg)
Distance Discrimination
![Page 26: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/26.jpg)
Distance Discrimination
![Page 27: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/27.jpg)
Shape Discrimination
![Page 28: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/28.jpg)
Shape Discrimination
![Page 29: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/29.jpg)
Shape Generalization
![Page 30: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/30.jpg)
Shape “completion”
![Page 31: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/31.jpg)
Impedance Discrimination
![Page 32: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/32.jpg)
How Do They Do It? Electric fish analyze dynamic 2D electrosensory images on the body surface to determine target direction, distance, size, shape and
composition (impedance)
Fish might perform an inverse mapping from 2D sensor data to obtain a dense 3D neural representation of world conductivity sensor data 3D conductivity action
Alternatively, fish might use sensor data to directly estimate target parameters sensor data target parameters action
![Page 33: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/33.jpg)
Parameter estimation
(bearing)
![Page 34: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/34.jpg)
Parameter Estimation (cont.)
![Page 35: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/35.jpg)
Dynamic Movement Strategies
Fish are constantly in motion not a single, static ‘snapshot’ dynamic, spatiotemporal data stream
With respect to target objects in the environment, fish body movements simultaneously influence the relative positioning of the sensor array the electric organ effector organs (e.g. mouth)
![Page 36: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/36.jpg)
MOVIE: Electrosensory Images
![Page 37: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/37.jpg)
Active motor strategies: Dorsal roll toward prey
![Page 38: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/38.jpg)
Probing Motor Acts
chin probing back-and-forth (va et vient )
lateral probing
tangentialprobing
stationaryprobing
![Page 39: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/39.jpg)
Fish exploring a 4 cm cube
![Page 40: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/40.jpg)
CNS Signal Processing Strategies
Multi-scale filtering spatial and temporal
Adaptive background subtraction tail-bend suppression
Attentional ‘spotlight’ mechanisms local gain control
![Page 41: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/41.jpg)
Multiple Maps
![Page 42: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/42.jpg)
Multi-scale Filtering
INPUT
(from skin receptors)
Centromedial map High spatial acuity Low temporal acuity
Centrolateral map Inter spatial acuityInter temporal acuity
Lateral map Low spatial acuityHigh temporal acuity
tem
pora
l
inte
grat
ion
bothspatial
integration
HINDBRAIN PROCESSING
PERIPHERALSENSORS
![Page 43: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/43.jpg)
Adaptive Background Subtraction
![Page 44: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/44.jpg)
Adaptive Background Subtraction
![Page 45: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/45.jpg)
Attentional ‘spotlight’ mechanism
![Page 46: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/46.jpg)
Summary Fish can evaluate direction, distance, size, shape and composition of target objects
How? model-based parameter estimation based on 2D image
analysis, not full 3D reconstruction presumably some sort of (adaptive) (extended)
(unscented) Kalman-like algorithm extensive pre-filtering (virtual sensors?)
self-calibrating, adaptive noise suppression, multi-scale spatial and temporal signal averaging
dynamic control of source and array position
![Page 47: Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cf45503460f949c2f1c/html5/thumbnails/47.jpg)
Acknowledgements
Colleagues Curtis Bell (OHSU) Len Maler (Univ. Ottawa) Gerhard von der Emde (Univ. Bonn)
Nelson Lab Members Ling Chen, Rüdiger Krahe, Malcolm MacIver
Funding Agencies NIMH, NSF