Cosc 6326/Psych6750X

Vision and Visual Displays

Why do we have two eyes?

• Binocular vision has several advantages including– increased field of view– redundancy– increased detection probability– depth perception from

• vergence

• stereopsis

Geometry of Binocular Correspondence

• Images in the two eyes differ (or are disparate) due to separation of vantage points

• Points of light project to the same retinal locations in the two eyes only on the horopter

• Points off the horopter have horizontal and/or vertical disparity

Vieth-Müller circle

Binocular subtense

Nodal points

F

P

O

F'F' P'P'

N1 N2

The Geometrical Point Horopter

LE RE

Fixation PointUncrossed disparity

Crossed disparity

Even flat frontal surfaces have disparity

• Differential perspective• Both horizontal and

vertical disparities• Gradient of vertical size

disparity– Varies with distance

– Affects depth, shape judgments …

Left Eye Image Right Eye Image

Vergence

Vergence eye movements

• opposite (disjunctive) movements of the eyes

• a principle function is to align both high acuity foveae on target of interest

• vergence helps bring images of objects of interest into correspondence (so that the images fall on similar parts of both retinae)

• since eyes are laterally separated in the head the optical axes of the two eyes are parallel only for fixation at infinity

• nearer targets require crossing one’s eyes (convergence)

• Several catergories– tonic vergence– voluntary vergence or proximal vergence

• induced voluntarily or by an object that appears to be at a different distance

– accommodative vergence• when accommodating there is a link with vergence

that drives a compatible vergence change; also vergence accommodation

• disparity vergence– vergence in response to

disparity in the images in the two eyes

– allows for aligning the images of the target in the two eyes

Two main questions for us

1. Does vergence give a sense of depth?• could theoretically triangulate depth of target

from angle of convergence• static vergence is probably a fairly weak

indicator for distance judgements but• indirectly important for size and shape

perception, calibration of stereopsis• e.g. micropsia in the wallpaper effect• size and shape constancy

• Dynamic vergence may play a role in signalling change in depth

• also controversial

• Regan (1986) found no changing depth from changing vergence in large stimuli but weak effect in small stimuli; substantial changes in apparent size

2. Dissociation of accommodation and vergence

• Normally vergence, accommodation and pupil size are controlled in a tightly coupled manner (the ‘near triad’)

• In most binocular displays, the display is focused at a fixed distance and thus accommodation should be fixed

• This fixed accommodation leads to vergence/accommodation mismatch– depth cue conflicts between vergence and

accommodation; stereopsis and blur

– difficulty in obtaining clear vision for stimuli requiring vergence nearer or further than the optical distance of the display OR

– difficulty ‘fusing’ stimuli

– contributes to eyestrain, simulator sickness

– adaptation, re-adaptation, visual stress (seminar next week Wednesday)

Stereoscopic vision

Binocular Disparity

– Horizontal disparity arises from lateral separation of the eyes (parallax)

– Relative depth is coded by disparity scaled inversely by the square of viewing distance

(ad)

D2

– Stereopsis (solid sight) refers to depth perception from disparity

Depth from Disparity

ScreenScreen

Fixation Point, FFixation Point, F

LeftLeftEyeEye

RightRightEyeEye

PP

FFPPRR PPLL

– Horizontal disparity arises from lateral separation of the eyes

2D

adLR

ScreenScreen

Fixation Point, FFixation Point, F

LeftLeftEyeEye

RightRightEyeEye

PP

FF PPRRPPLL

DD

dd

aa

Stereoscopic Voxels

from Davis and Hodges 1995

http://www.pandigitalmedia.com/bgussin

Sensitivity to disparity

• People are extremely sensitive to relative disparity. Stereopsis is a hyperacuity– discrimination thresholds of a few seconds of arc

– Upper limit: patent versus qualitative stereopsis

• much poorer sensitivity to absolute disparity• thus very sensitive to depth differences between

stimuli; poor ability to estimate absolute distance

Sensitivity to disparity

• Disparity has an inverse square dependence on distance

• Thus, stereopsis is most effective for estimating depth and surface shape at near distances

Sensitivity to disparity

• many people are stereo-anomalous– cannot see depth from disparity (about 5% of

population) or– cannot see motion-in-depth from changing disparity

or– have deficits in particular part of the visual field, or

for crossed disparity, or for uncrossed disparity

• cannot rely on all subjects seeing stereoscopic depth

• Two of the main issues in stereopsis are– matching of corresponding parts of the two

images – estimation of depth from disparities

1 2 3

A B

Keplerian projection

Correspondence problem

• If features are similar a matching ambiguity arises

• Human stereopsis usually matches images very effectively

• Often the most difficult problem in stereoscopic computer vision

Random-dot stereograms

• not all features in one eye’s image have a corresponding feature in the other image

• these monocular zones play an important role in perception of objects and surface properties in binocular vision– depth from monocular occlusion– luster – binocular rivalry

.

Opaque object

Left-eye monocular zone

Right-eye monocular zoneBinocular occlusion zone

(a) When a near occluding object is shorter than the interocular distance, both eyes see the region between the occlusion zones.

(b) When the occluding object is longer than the interocular dis-tance, neither eye sees the region between the monocular zones.

. .

Opaque object

Distant surface

Left-eye monocular zone

Right-eye monocular zone

Binocular zone

(c) The near object may not be visible to the left eye if near and far surfaces are similar. The object is camouflaged for this eye. The near object may be visible to the other eye against a different background. It appears on the nasal side of the far object.

Near object

Far object

.

A

d

C

g

fA

d

(d) Tracings of drawings made by Leonardo da Vinci to illustrate monocular occlusion zones created by looking through an aperture. (Adapted from Strong 1979)

Fusion

• Typically a wide range of disparities present in a scene

• If disparities are modest people ‘fuse’ the images of the object in the left and right and perceive a unitary object

• Outside this range objects are seen doubly (diplopic) or one eye’s image is suppressed (binocular rivalry)

Fusion

• Only a range of disparities near the horopter are seen singly (Panum’s fusional area)

• Also a limited range of disparities for which depth can be obtained from disparity. But depth can be seen in diplopic images

• Fusion depends on size, contrast, disparity gradient, temporal factors …

Estimation of Depth from Disparity

• Relation between horizontal disparity and depth varies with distance and location of surface wrt head (recall the curvature of the horopter)

• Need to account for viewing system parameters of distance and eccentricity to judge– metric depth, relief, surface curvature, surface slant …

– absolute retinal disparity does not provide viewing system parameters need: ‘sensed’ eye position, vertical disparities, other depth cues, …

Stereoscopic issues in displays

• Types of display based on ‘ocularity’– Monocular : Image presented to one eye only

• no stereopsis, rivalry with dark images

– Biocular : same image presented to both eyes• Stereopsis specifies a flat surface

• stereoscopic artefacts can arise with misalignment

– Binocular : eye specific (dichoptic) images presented to each eye

• Stereoscopic imagery possible

• Autostereoscopic - binocular display that requires single display and no special eyewear

• Methods of presenting stereoscopic displays– hmds– shutters– polaroid- linear, circular– anaglyphs– barrier, lenticular– holograms– ….

From www.stereographics.com

From www.stereographics.com

Example - 3ality autostereoscopic display

• Many new autostereoscopic displays based on lenticular sheets or parallax barrier methods – trade spatial resolution for stereopsis

• Shutter and some other systems trade temporal resolution for stereopsis (flicker)

• 3ality display codes stereoscopic information in luminance, preserving spatial and temporal resolution (at expense of dynamic range).

• Viewer wears glasses with a pair of orthogonal polarizing filters

• At each pixel, left eye should see a certain amount of light of one polarization, and the right eye a certain amount of the other

• Left and right eyes images mixed by summing luminance of the two images (vector sum) - ‘magnitude’ of the vector

Images from Kleinberger et al, Stereoscopic Displays and Applications X, SPIE vol #5006

• An ‘extra’ liquid crystal panel twists the light for each pixel to the appropriate intermediate polarization - ‘angle’ of the vector

• Eyewear decomposes the combined light into appropriate amount of left and right eye polarization

Autostereoscopic version

• When are stereo displays useful?– 3D visualisation– ambiguous/poor monocular information– static displays– direct 3D manipulation– cluttered, complicated scenes

• some issues– field of view

– depth resolution

– ghosting and crosstalk

– refresh rate for time multiplexed techniques

– distortion, alignment and calibration

– focus

– head tracking, view dependence

– ‘orthostereoscopy’ vs visual comfort

Generating Stereoscopic Displays

• A range of algorithms have been proposed to deal with generating stereoscopic imagery to deal with– fusion, focus problems– head movement– optimising range of depth portrayed

• we’ll look at these in next week’s seminars

• Trade-offs in field of view in binocular displays– 100% overlap

• stereoscopic vision over the FOV

• less distortion effects

• total FOV is same as monocular FOV

– Partial Overlap:• stereopsis restricted to binocular overlap region

• parts of normal binocular field sees monocular view: rivalry possible, no stereopsis

• total FOV exceeds monocular FOV

• distortion introduces disparity even if optics for the two eyes are matched

Calibration and Alignment

• Calibration is critical for stereoscopic displays. Some parameters:– interpupillary distance (IPD)

• varies with subject 51mm ~ 76mm

• vergence changes effective IPD slightly (may be a small artifact in precise AR displays)

– convergence of the optics– relative rotational alignment of the displays– interocular differences in linearity, distortion, image

size

• In stereoscopic displays relative misalignment may distort space

• Alignment is critical for tele-operation systems which fuse images from multiple sensors or in augmented reality

• Distortion– Nonlinear mapping from object space to image

space arising from lens or display systems– radial distortions are common e.g. pincushion

or barrel distortions– for stereoscopic displays, distortion is

particularly troublesome when not matched in the two eyes