Post on 20-Dec-2021
HAL Id: hal-02316888https://hal.inria.fr/hal-02316888
Submitted on 15 Oct 2019
HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.
Motion anticipation in the retinaBruno Cessac, Selma Souihel
To cite this version:Bruno Cessac, Selma Souihel. Motion anticipation in the retina. NeuroSTIC 2019 - 7e édition desjournées NeuroSTIC, Oct 2019, Sophia-Antipolis, France. �hal-02316888�
Motion anticipation in the retina
Bruno Cessac, Selma Souihel
Biovision INRIA team
Motion anticipation in the retina
Bruno Cessac, Selma Souihel
Biovision INRIA team
Motion anticipation in the retina
Bruno Cessac, Selma Souihel
In collaboration with :
Frédéric ChavaneSandrine Chemla
Olivier MarreMatteo Di VoloAlain Destexhe
Biovision INRIA team
4
The visual flow
Source : Wikipedia
5
The visual flow
Source : Wikipedia
6
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
7
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
8
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
9
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
10
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
11
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
12
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
13
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
Too slow !
14
Visual Anticipation
Source : Benvenutti et al. 2015
15
Visual Anticipation
Source : Benvenutti et al. 2015
Anticipation is carried out by the primary visual cortex (V1) through an activation wave
16
Visual Anticipation
Source :Berry et al.1999
Anticipation also takes place in the retina
17
Visual Anticipation
What are the respective mechanisms underlying retinaland cortical anticipation?
TrajectoryTrajectory
18
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
The retina is NOT a camera
• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV
• Able to detect approaching motion
• Able to detect differential motion
• Sensitive to « surprise » in a visualscene
• Able to perform motion anticipation
19
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
20
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion« Analogic computing »
No spikes (except in GCs)Low energy consumption
Dedicated circuitsSmall number of neurons
Specialized synapses
21
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
22
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
23
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
Horizontal cells
24
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
Horizontal cells
Amacrine cells
25
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
26
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion
Gap junctions
27
The visual flow
Source : Wikipedia
Decoding spike trains
Encoding motion« Analogic computing »
No spikes (except in GCs)Low energy consumption
Dedicated circuitsSmall number of neurons
Specialized synapses
28
Which generic computational paradigms are atwork in the retina ?
29
Visual Anticipation
30
Visual Anticipation
Which animal ?
31
Visual Anticipation
Which animal ?
32
Visual Anticipation
Which animal ?
33
Visual Anticipation
Which animal ?
34
Visual Anticipation
Which animal ?
35
Visual Anticipation
Which animal ?
36
Visual Anticipation
Developping a retino-cortical model of anticipation soas to
understand / propose
possible generic mechanisms for anticipation in theretina and in the cortex.
37
Anticipation in the retina
38
The Hubel-Wiesel view of vision
Ganglion cells
Nobel prize 1981
Ganglion cells response is the convolution of the stimulus with a spatio-temporalreceptive field followed by a non linearity
Ganglion cells are independent encoders
39
The Hubel-Wiesel view of vision
Source : Berry et al. 1999
Ganglion cells
Nobel prize 1981
40
Gain control (Berry et al, 1999, Chen et al. 2013)
Building a 2D retina model for motionanticipation
41
Building a 2D retina model for motionanticipation
Gain control (Berry et al, 1999, Chen et al. 2013)
42
Building a 2D retina model for motionanticipation
Gain control (Chen et al. 2013)
43
Building a 2D retina model for motionanticipation
44
1D results : smooth motion anticipationwith gain control
Bipolar layer Ganglionlayer
45
1D results : smooth motion anticipationwith gain control
Anticipation variability with stimulusparameters
46
Building a 2D retina model for motionanticipation
Ganglion cells are independent encoders
47
Building a 2D retina model for motionanticipation
Ganglion cells are not independent encoders
Gap junctions connectivity
48
Building a 2D retina model for motionanticipation
Gap junctions connectivity
49
Building a 2D retina model for motionanticipation
Gap junctions connectivity
50
Building a 2D retina model for motionanticipation
Gap junctions connectivity
51
Building a 2D retina model for motionanticipation
Diffusive wave of activity ahead of the motion
Gap junctions connectivity
52
Smooth motion anticipation with gapjunctions
53
6
Low gap junction conductance
Smooth motion anticipation with gapjunctions
54
6
Low gap junction conductance High gap junction conductance
Smooth motion anticipation with gapjunctions
55
6
Smooth motion anticipation with gapjunctions
56
Anticipation variability with stimulusparameters
Smooth motion anticipation with gapjunctions
57
Building a 2D retina model for motionanticipation
58
Building a 2D retina model for motionanticipation
Ganglion cells are not independent encoders
Amacrine cells connectivity
59
Amacrine cells connectivity
Building a 2D retina model for motionanticipation
60
Amacrine cells connectivity
Building a 2D retina model for motionanticipation
61
Amacrine cells connectivity
● The circuitry involves amacrine cells connectivity upstream of ganglion cells
Building a 2D retina model for motionanticipation
● A class of RGCs are selective to differential motion
62
Amacrine cells connectivity
Building a 2D retina model for motionanticipation
63
Amacrine cells connectivity
Building a 2D retina model for motionanticipation
64
Amacrine cells connectivity
Diffusive wave of activityahead of the bar
Building a 2D retina model for motionanticipation
65
Building a 2D retina model for motionanticipation
66
Building a 2D retina model for motionanticipation
67
Building a 2D retina model for motionanticipation
68
Building a 2D retina model for motionanticipation
69
Building a 2D retina model for motionanticipation
Spatial profiles w=1
x
70
Building a 2D retina model for motionanticipation
Spatial profiles w=3
x
71
Building a 2D retina model for motionanticipation
Spatial profiles w=5
x
72
Building a 2D retina model for motionanticipation
Temporal profile of the middle cell
73
1D results : smooth motion anticipationwith amacrine connectivity
Bipolar layer Ganglion layer
74
1D results : smooth motion anticipationwith amacrine connectivity
Anticipation variability with stimulusparameters
75
Comparing the performance of the three layers
76
Suggesting new experiments : 2D results
1) Angular anticipation
Stimulus
t = 0 ms 100 200 ms 300 ms 400 ms 500 ms 600 ms 700 ms
Bipolar linearresponse
Bipolar gainresponse
Ganglion linearresponse
Ganglion gainresponse
A)
B) C)
77
Suggesting new experiments : 2D results
1) Angular anticipation
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
● A large part of the « computation » is made by synapses.
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
● A large part of the « computation » is made by synapses.
● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
● A large part of the « computation » is made by synapses.
● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.
● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
● A large part of the « computation » is made by synapses.
● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.
● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).
● Here we propose that lateral connectivity also plays a role in motion anticipationwhere a wave of activity propagates ahead of the motion.
Conclusions
● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.
● A large part of the « computation » is made by synapses.
● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.
● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).
● Here we propose that lateral connectivity also plays a role in motion anticipationwhere a wave of activity propagates ahead of the motion.
● Useful paradigms for :1) Computer vision ?2) Retinal prostheses ?
84
Anticipation in V1
85
Anticipation in V1
86
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
87
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
88
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
Affords a retino thalamic input
89
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
90
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
91
A mean field model to reproduce VSDIrecordings Zerlaut et al 2016
Chemla et al 2018
Response of the cortical model to a LNretina drive
Response of the cortical model to a retinadrive with gain control
Anticipation in the cortex : VSDI dataanalysis (Data courtesy of F.
Chavane et S. Chemla)
Comparing simulation results to VSDIrecordings
Cortex experimentalrecordings
Simulation resultsResponse to an LNmodel of the retina
Simulation resultsResponse to a gaincontrol model of theretina
Conclusions
● We developped a 2D retina with three ganglion cell layers,implementing gain control and connectivity.
● We use the output of our model as an input to a mean field model ofV1, and were able to reproduce anticipation as observed in VSDI
Conclusions
● How to improve object identification ● 1) exploring the model's parameters and
● 2) using connectivity ?
● Is our model able to anticipate more complex trajectories, withaccelerations for instance ?
● How to calibrate connectivity using biology ?
● How does anticipation affect higher order correlations ?
● Would it be possible to design psycho-physical tests clearly showingthe role of the retina in visual anticipation ?
Thank you for your attention !