The architecture of the visual system: What is the grand

design?

April 12, 2010

Goals of the visual system• Create a meaningful representation of the 3D world• Recognize objects• Detect motion, color, edges• Reconstruct 3D depth• Create representation of world useful for action• Store visual memories• Distinguish shadows from real edges• Suppress vision during saccades• …

How are all these different functions organized?

“In biology, if seeking to understand function, it isusually a good idea to study structure.”

-Francis Crick and Christof Koch, Function of the Claustrum

1883

Mapping different visual areas

What happens if you cut the corpus callosum?

Mapping different visual areas

Wandell Neuron 2007

The What and Where PathwaysThe What and

Where Pathways

The Ungerleider & Mishkin (1982) ExperimentThe Ungerleider & Mishkin (1982) ExperimentThe Ungerleider & Mishkin (1982) ExperimentThe Ungerleider & Mishkin (1982) Experiment

Task 1: Task 1:

Object discriminationObject discrimination study an object study an object select the familiar object select the familiar object

(reward) (reward)

parietal lesions parietal lesions impair LANDMARK TASKimpair LANDMARK TASK

temporal lesions temporal lesions impair OBJECT TASKimpair OBJECT TASK

Task 2:Task 2:

Landmark discriminationLandmark discrimination• select foodwell closest to select foodwell closest to the TOWERthe TOWER

Felleman and Van Essen 1991

Callaway 2004:Feedforward, feedback and inhibitory connections in primate visual cortex

Challenges to What/Where

Milner and Goodale: “The A brain-damaged patient (D.F.) …has a profound inability to recognize objects, places and people, in large part because of her inability to make perceptual discriminations of size, shape or orientation, despite having good visual acuity.Yet she is able to perform skilled actions that depend on that very same size, shape and orientation information that is missing from her perceptual awareness.”

Konen and Kastner, “Two hierarchically organized neural systems for object information in human visual cortex” 2008

• evidence from functional brain imaging in humans demonstrates that object representations are not confined to the ventral pathway, but can also be found in several areas along the dorsal pathway.

• In both streams, areas at intermediate processing stages in extrastriate cortex (V4, V3A, MT and V7) showed object-selective but viewpoint- and size-specific responses.

• In contrast, higher-order areas in lateral occipital and posterior parietal cortex (LOC, IPS1 and IPS2) responded selectively to objects independent of image transformations

Sereno and Maunsell 1998 Shape selectivity in primate lateral intraparietal cortex

How are the “what” and “where” information streams reunited to allow for simultaneous understanding of an object’s location and identity? …The binding problem is one of many fundamental, unsolved visual problems that will make visual science a frontier of new knowledge for many years to come.

We know very little about interactions between areas (lawrence…)

Felleman and Van Essen 1991

Lewis and Van Essen 2000

Interactions between areas

• Functional connectivity (determine networks with significantly increased correlated activity; using seed-based and ICA techniques)

• Granger causality (determine whether a lagged time course from area X can predict another time course from area Y)

Identify neurons in HvC projecting to RA

Hahnloser et al., Nature, 2002

An attempt to link neuroanatomy to music, hallucinations, the Iliad, consciousness…