HAL Id: hal-02316888https://hal.inria.fr/hal-02316888

Submitted on 15 Oct 2019

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Motion anticipation in the retinaBruno Cessac, Selma Souihel

To cite this version:Bruno Cessac, Selma Souihel. Motion anticipation in the retina. NeuroSTIC 2019 - 7e édition desjournées NeuroSTIC, Oct 2019, Sophia-Antipolis, France. �hal-02316888�

Motion anticipation in the retina

Bruno Cessac, Selma Souihel

Biovision INRIA team

Motion anticipation in the retina

Bruno Cessac, Selma Souihel

Biovision INRIA team

Motion anticipation in the retina

Bruno Cessac, Selma Souihel

In collaboration with :

Frédéric ChavaneSandrine Chemla

Olivier MarreMatteo Di VoloAlain Destexhe

Biovision INRIA team

4

The visual flow

Source : Wikipedia

5

The visual flow

Source : Wikipedia

6

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

7

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

8

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

9

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

10

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

11

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

12

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

13

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

Too slow !

14

Visual Anticipation

Source : Benvenutti et al. 2015

15

Visual Anticipation

Source : Benvenutti et al. 2015

Anticipation is carried out by the primary visual cortex (V1) through an activation wave

16

Visual Anticipation

Source :Berry et al.1999

Anticipation also takes place in the retina

17

Visual Anticipation

What are the respective mechanisms underlying retinaland cortical anticipation?

TrajectoryTrajectory

18

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

The retina is NOT a camera

• High transduction rate : 1 photon cantrigger a membrane voltage variation of~1 mV

• Able to detect approaching motion

• Able to detect differential motion

• Sensitive to « surprise » in a visualscene

• Able to perform motion anticipation

19

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

20

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion« Analogic computing »

No spikes (except in GCs)Low energy consumption

Dedicated circuitsSmall number of neurons

Specialized synapses

21

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

22

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

23

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

Horizontal cells

24

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

Horizontal cells

Amacrine cells

25

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

26

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion

Gap junctions

27

The visual flow

Source : Wikipedia

Decoding spike trains

Encoding motion« Analogic computing »

No spikes (except in GCs)Low energy consumption

Dedicated circuitsSmall number of neurons

Specialized synapses

28

Which generic computational paradigms are atwork in the retina ?

29

Visual Anticipation

30

Visual Anticipation

Which animal ?

31

Visual Anticipation

Which animal ?

32

Visual Anticipation

Which animal ?

33

Visual Anticipation

Which animal ?

34

Visual Anticipation

Which animal ?

35

Visual Anticipation

Which animal ?

36

Visual Anticipation

Developping a retino-cortical model of anticipation soas to

understand / propose

possible generic mechanisms for anticipation in theretina and in the cortex.

37

Anticipation in the retina

38

The Hubel-Wiesel view of vision

Ganglion cells

Nobel prize 1981

Ganglion cells response is the convolution of the stimulus with a spatio-temporalreceptive field followed by a non linearity

Ganglion cells are independent encoders

39

The Hubel-Wiesel view of vision

Source : Berry et al. 1999

Ganglion cells

Nobel prize 1981

40

Gain control (Berry et al, 1999, Chen et al. 2013)

Building a 2D retina model for motionanticipation

41

Building a 2D retina model for motionanticipation

Gain control (Berry et al, 1999, Chen et al. 2013)

42

Building a 2D retina model for motionanticipation

Gain control (Chen et al. 2013)

43

Building a 2D retina model for motionanticipation

44

1D results : smooth motion anticipationwith gain control

Bipolar layer Ganglionlayer

45

1D results : smooth motion anticipationwith gain control

Anticipation variability with stimulusparameters

46

Building a 2D retina model for motionanticipation

Ganglion cells are independent encoders

47

Building a 2D retina model for motionanticipation

Ganglion cells are not independent encoders

Gap junctions connectivity

48

Building a 2D retina model for motionanticipation

Gap junctions connectivity

49

Building a 2D retina model for motionanticipation

Gap junctions connectivity

50

Building a 2D retina model for motionanticipation

Gap junctions connectivity

51

Building a 2D retina model for motionanticipation

Diffusive wave of activity ahead of the motion

Gap junctions connectivity

52

Smooth motion anticipation with gapjunctions

53

6

Low gap junction conductance

Smooth motion anticipation with gapjunctions

54

6

Low gap junction conductance High gap junction conductance

Smooth motion anticipation with gapjunctions

55

6

Smooth motion anticipation with gapjunctions

56

Anticipation variability with stimulusparameters

Smooth motion anticipation with gapjunctions

57

Building a 2D retina model for motionanticipation

58

Building a 2D retina model for motionanticipation

Ganglion cells are not independent encoders

Amacrine cells connectivity

59

Amacrine cells connectivity

Building a 2D retina model for motionanticipation

60

Amacrine cells connectivity

Building a 2D retina model for motionanticipation

61

Amacrine cells connectivity

● The circuitry involves amacrine cells connectivity upstream of ganglion cells

Building a 2D retina model for motionanticipation

● A class of RGCs are selective to differential motion

62

Amacrine cells connectivity

Building a 2D retina model for motionanticipation

63

Amacrine cells connectivity

Building a 2D retina model for motionanticipation

64

Amacrine cells connectivity

Diffusive wave of activityahead of the bar

Building a 2D retina model for motionanticipation

65

Building a 2D retina model for motionanticipation

66

Building a 2D retina model for motionanticipation

67

Building a 2D retina model for motionanticipation

68

Building a 2D retina model for motionanticipation

69

Building a 2D retina model for motionanticipation

Spatial profiles w=1

x

70

Building a 2D retina model for motionanticipation

Spatial profiles w=3

x

71

Building a 2D retina model for motionanticipation

Spatial profiles w=5

x

72

Building a 2D retina model for motionanticipation

Temporal profile of the middle cell

73

1D results : smooth motion anticipationwith amacrine connectivity

Bipolar layer Ganglion layer

74

1D results : smooth motion anticipationwith amacrine connectivity

Anticipation variability with stimulusparameters

75

Comparing the performance of the three layers

76

Suggesting new experiments : 2D results

1) Angular anticipation

Stimulus

t = 0 ms 100 200 ms 300 ms 400 ms 500 ms 600 ms 700 ms

Bipolar linearresponse

Bipolar gainresponse

Ganglion linearresponse

Ganglion gainresponse

A)

B) C)

77

Suggesting new experiments : 2D results

1) Angular anticipation

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

● A large part of the « computation » is made by synapses.

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

● A large part of the « computation » is made by synapses.

● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

● A large part of the « computation » is made by synapses.

● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.

● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

● A large part of the « computation » is made by synapses.

● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.

● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).

● Here we propose that lateral connectivity also plays a role in motion anticipationwhere a wave of activity propagates ahead of the motion.

Conclusions

● The retina is able to encode complex visual scene features, fast and reliably, with avery low energy consumption, without spikes, except at the ganglion cells level.

● A large part of the « computation » is made by synapses.

● Starting from the retina architecture one can extract circuits solving « tasks » such asmotion anticipation.

● The role of gain control and local synaptic balance between excitation-inhibition hasbeen experimentally shown to play a central rôle in anticipation (Berry et al, 1999 ; Chenat al, 2013 ; Johnston-Lagando, 2015).

● Here we propose that lateral connectivity also plays a role in motion anticipationwhere a wave of activity propagates ahead of the motion.

● Useful paradigms for :1) Computer vision ?2) Retinal prostheses ?

84

Anticipation in V1

85

Anticipation in V1

86

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

87

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

88

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

Affords a retino thalamic input

89

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

90

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

91

A mean field model to reproduce VSDIrecordings Zerlaut et al 2016

Chemla et al 2018

Response of the cortical model to a LNretina drive

Response of the cortical model to a retinadrive with gain control

Anticipation in the cortex : VSDI dataanalysis (Data courtesy of F.

Chavane et S. Chemla)

Comparing simulation results to VSDIrecordings

Cortex experimentalrecordings

Simulation resultsResponse to an LNmodel of the retina

Simulation resultsResponse to a gaincontrol model of theretina

Conclusions

● We developped a 2D retina with three ganglion cell layers,implementing gain control and connectivity.

● We use the output of our model as an input to a mean field model ofV1, and were able to reproduce anticipation as observed in VSDI

Conclusions

● How to improve object identification ● 1) exploring the model's parameters and

● 2) using connectivity ?

● Is our model able to anticipate more complex trajectories, withaccelerations for instance ?

● How to calibrate connectivity using biology ?

● How does anticipation affect higher order correlations ?

● Would it be possible to design psycho-physical tests clearly showingthe role of the retina in visual anticipation ?

Thank you for your attention !