Natural Vision Systems

ECE 847:Digital Image Processing

Stan BirchfieldClemson University

Animal visionWhich has “the most complicated eye in the animal kingdom?”

Tropical mantis shrimp

Sees at least 11 color dimensions!• 8 pigments sensitive to visual light• 3 to ultraviolet

Also• has 4 filters to tune the pigments• sees 2 or 3 planes of polarized light

Lives at depths of 40m, where light is only filtered, dim blue. Why such complex vision?• shrimp communicate with each other by fluorescing their spots

http://www.innovations-report.de/html/berichte/biowissenschaften_chemie/bericht-23390.html

Color vision in bees

flower as seen by a bee

http://www.wonderquest.com/nectar-guides.htm

Bees have ultraviolet filters to help them see their target:

Color vision in mammalsHow does the bull see the cape?

Motion, not color!

Generally speaking, only

primates and humans (among mammals)

can see color

Visual acuity

Raptors (including eagles, hawks, and falcons) can see up to 8 times more clearly than humans

Golden eagle can see a hare a mile away!

Which animal has the sharpest vision?

http://ebiomedia.com/gall/eyes/sharp.html

Speed

fly can process images at 200 Hz

http://www.eyedesignbook.com/ch3/eyech3-c.html

Nautilus eye

What is unusual about the nautilus eye?

It has no lens!

http://www.eyedesignbook.com/ch3/eyech3-b.html

Lobster eye

What is unusual about the lobster eye (and some other crustaceans)?

It focuses light by reflection, not refraction

(design has inspired Lobster Eye telescope)

http://www.answersingenesis.org/creation/v23/i3/lobstereyes.asp

Brittlestar

Where is the brittlestar eye?Its whole body!

Brittlestar secretes calcite crystals that form microlenses

The “oldest” eye

http://www.trilobites.info/eyes.htm

Descartes’ lens design for minimal aberration

Huygens’ lens design for minimal aberration

trilobite

trilobite

trilobite eyes

An exerciseinterpolate this function:

?

?

time

maximum complexity

of animal eye

500M4B

Irreducible complexity

from M. Behe, Darwin’s Black Box

A single photovoltaic cell:

Human visual system

[from P. Gurney]

Visual pathway

Retinal cells

retinal pigment epithelium (RPE)

light

optic nerve fibers

ganglion cells

amacrine cellsbipolar cellshorizontal cells

photoreceptors

[from S. Palmer, Vision Science]

Retinal cells

http://webvision.med.utah.edu/sretina.html

Foveation

central retina: color and form perceptionperipheral retina: light detection, motion detection, night vision

only cones in foveola

Cones

http://mbb.harvard.edu/evolution_of_mind_and_brain/chapter_2.pdf

some birds, insects,and fish see ultraviolet

some snakes,insects,and batssee infrared

Macula Lutea

macula lutea –yellow pigmentation

(Xanthophyll)in foveal area

Xanthophyll in Muller cells absorbs damaging short-wavelength light (420-450 nm)

Inverted retina allows• Muller cells in fovea to protect photoreceptors• RPE, which is opaque, to nourish and protect photoreceptors

• RPE and choroid absorb 25-33 % of light entering eye• Choroid acts as heat sink (has the highest blood flow per gram of tissue of all tissues in the body)

Besides, in foveola• cones are packed tightly together and elongated• layers of retina are spread aside, permitting light to fall directly on cones• Muller cells act like optic fiber plates to transmit light to receptors

Why an inverted retina?

verted retina

(common in invertebrates)

inverted retina

(common in vertebrates)

lightlight

[P. Gurney, Is our inverted retina really bad design?, 1999]

Visual processing

Overlapping fields of view

rabbit person

Characteristics of human vision

• Vision is an interpretive process– The raw data is insufficient– Processes in the brain impose a priori

assumptions to make sense of the data– Examples: brightness, color, and size constancy

• Vision solves specific tasks in specific contexts– Visual skills directly tied to needs and context– Examples: Thatcher illusion, Chernoff faces

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Brightness constancyPerceived brightness is complex function of pixel values

(Image courtesy of Ted Adelson)

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Color Constancy

Pixel color strongly affected by illumination.

Perception of color constancy maintained by the brain

Sunlight Fluorescent light

(Images courtesy of David Heeger)

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Size Constancy

Object size vs. object depth

(Images copyright John H. Kranz, 1999)

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Thatcher Illusion

(Due to P. Thompson)

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Thatcher Illusion

Face processing is sensitive to orientation.

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Chernoff faces

Useful for displaying multivariate data

Integration of Information

• Spatial information in the image must be integrated to perceive a coherent object

Fixation experimentsYarbus 1961

Fixations for “Give theages of the people”

Fixations for “Whatwere they doingbefore the guestarrived?”

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html

Recognition of Objects

(Due to Ron Rensick)(Due to R. James)

Grouping is affected by top-down processes

Context influences recognition of parts

http://www-static.cc.gatech.edu/classes/AY2007/cs4495_fall/html/materials.html