Feel of Seeing

Feel of Hearing

What is the quality of sensory experience?

J Kevin O’Regan

Laboratoire Psychologie de la PerceptionCentre National de la Recherche

Scientifique & Université René Descartes - Paris 5

No Feel

Quality of sensory modalities

•Old view:–Müller’s specific nerve energy

•New view:–Cortical maps, neural pathways

detailed internalrepresentation

Brain creates experience

standard view

Explanatory gap!

Sensorimotor approach to sensory experience

(O’Regan & Noë, Behavioral and Brain Sciences, 2001)

No more explanatory

gap!

Sensation = exercising a skill

Sensation

• Accessing knowledge that you are currently exercising a certain sensorimotor skill.

• Quality of sensation: laws of sensorimotor contingency

The laws governing how what you do affects

sensory input

Sensorimotor Contingencies(D. M. MacKay, 1956)

Seeing

“Red” is the way red things change the light (Broackes, 1992)

Seeing Red

knowing that sensorimotor contingencies typical of red are currently being

obeyed.

“Biological” reflection properties

for a biological organism reflection properties are constraints over sensory inputs set of reflection properties is finite dimensional

finite number of singular reflection properties

RLMSi LMSr

Universal color categories World color survey: Berlin & Kay (1969)

D. Philipona & J K O’Regan, 2006

Unique hues

D. Philipona & J K O’Regan, 2006

Hue Cancellation

3D Wandell

D. Philipona & J K O’Regan, 2006

“Red” is the way red things change the light (Broackes, 1992)

Aline Bompas with split-field glasses

Forced choice

more yellow-ish more blue-ish

Bompas & O’Regan, 2005, 2006

Seeing

detailed internalrepresentation

Seeing is making an internal

representation

Seeing is visually

manipulating

standard view new view

The impression of seeing “everything”

richness not in the head

have algorithms to access information

you see what you visually manipulate

world as outside memory (O’Regan, 1992; cf. also Minsky, 1988; R. Brooks,

1991)

Refrigerator light analogy (N. Thomas)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

CB during Mudsplashes (O’Regan, Rensink & Clark, 1999)

Reimer & Simons, 2001; Auvray & O’Regan, 2004

• Flicker– Rensink, O’Regan & Clark,1997; 1999

• Eye saccades– Currie, McConkie, Carlson-Radvansky & Irwin, 1995; McConkie & Currie, 1996

• Blinks– O’Regan, Deubel, Clark, Rensink, 1999

• Film cuts, real life– Levin & Simons, 1997

• “Mudsplashes”– O’Regan, Rensink & Clark (Nature, 1999)

Change Blindness

QuickTime™ et undécompresseur DV - PAL

sont requis pour visionner cette image.

•Inattentional blindness

–Neisser–Mack & Rock–D. Simons

Sensation = exercising a skill

Sensation

• Accessing knowledge that you are currently exercising a certain sensorimotor skill.

• Quality of sensation: laws of sensorimotor contingency

big change

expanding flow

shifting flow

nothing

big change

no change

increasing amplitude

asynchrony

big change

nothing

blink:

move forward:

turn sideways:

cover ears:

cover eyes:

SEEING HEARING

Examples of sensorimotor contingencies

Tactile VisualSensory Substitution

Bach y Rita (1972; 1984)

Tongue Display Unit

Sampaio, E., S. Maris., and P. Bach-y-Rita. 2001Brain plasticity: 'Visual' acuity of blind persons via the tongue. Brain Research 908(July 13):204.

Sensory Substitution

• “rewired” ferrets (Sharma, Angelucci & Sur, 2000; Melchner, Pallas & Sur, 2000)

• Phantom limbs

• TVSS (Bach y Rita, 1972, 1984)• substitution of vision through sound

• embodiment in virtual reality– Murray & Sixsmith (1999); Heim (1995)

testingP. Meijer’s“The vOICe”

Auvray & O’Regan, in press

Sensation = exercising a skill

Rubber arm experiment of Botvinick & Cohen, 1998

Hand-image matching

Experiment 2 Responses: picture-matching (shoes) and foot-localisation

Stimuli: the little feet

Sensation = exercising a skill

RETINALSTIMULATION

CORTICALREPRESENTATION

SCENE

Each eye is constitued of a retina made of randomly distributed omnidirectional photosensitive cells, and a diaphragm

Each joint can freely rotate (rotule)Each segment can stretch (piston)

Each light can freely move in 3D space

Space from sensorimotor contingencies

Philipona, O'Regan & Nadal, Neural Computation, 2003.Philipona, O'Regan, Nadal & Coenen, NIPS, 2003.

QuickTime™ et un décompresseur sont requis pour visualiser

cette image.

Philipona, O ’Regan & Nadal, Neural Computation 2003

Aerial Snapshot Agent

Uncalibrated camera, effectors

Aerial Snapshot Agent

– Make linear predictors

– Study commutativity

– Determine basis of translations and rotations

Aerial Snapshot Agent

Sensorimotor embedding

• Self insertion

• Similarity (proximity) judgments

Aerial Snapshot Agent

Isomap with K=4,5,6,7Sensorimotor embeddingPhilipona, Glanois & O’Regan, under review

Summary

• New view on what sensory experience is• Predictions about

– Color– Sensory substitution– Body sense

• Robotic applications– Color– Space– Dimension reduction

• Other work– Sensory feels vs mental feels– Pain– Consciousness

http://nivea.psycho.univ-paris5.fr

• Philosophy – Erik Myin, Antwerp

• Psychology– Malika Auvray, Aline Bompas, Ed Cooke

• Robotics – David Philipona, Fred Glanois

• EU funding:• CoSy Integrated project• ENACTIVE Network of excellence