Human Factors and Displays for VR Graphics

David Johnson

Goals

• Learn about– Human vision– Methods presenting images to users

Visual Bandwidth

• Bandwidth of vision is probably greater than other sensory modalities– Retina bandwidth ~10 Mbits/sec

• article

– Most important sense for VR?• IMAX movie of rollercoaster can induce sensation

of motion• Images can induce sensations of taste/smell

Why Study Basis of Vision?

• If a display can match human capabilities, we are done

• If the display can’t, nice to know where to concentrate resources.

• Maybe can use tricks to suggest a higher-quality display– Optical illusions

Anatomy of Vision

Eye Anatomy

Blind Spot test

• test

• Test– Brain fills in

Eye Optics

Accomodation

• Change in curvature of lens– Rest focus from 6m to infinity

Light Sensors

• Cones– Mostly in the

fovea– Blue, green,

red/yellow cones, also called short, medium, long

• Rods– Sense low levels

of light

Visual Acuity• Cycles per degree

– Like angular resolution– Humans can resolve 0.93mm spacing at 1m

– Fovea is about 1 Mpixel• 1 arc minute at fovea ( 1/60 degree)

– 20/20 vision letters are 5 arc minute letters (strokes 1 arc minute)

• Around 24K x 24K over field of view– Retina is sensitive to light levels over 10^13 range

• Dynamic

Visual Field

• One Eye– 120 degrees vertically and 150 degrees

horizontally• 60 to nose, 90 to side• 50 up, 70 down

• Binocular 200 degrees horizontal

• Eye can rotate about 50 degrees

Binocular Vision

• Humans have 120 degree binocular overlap– 2 40 degree

monocular regions

• Depth disparity perception– 0.05mm at 500mm– 4mm at 5m

Eye Motion

• Vergence - the motion of the eye to maintain binocular vision– ‘Cross-eyed’ when focus in close

• Version – eye movement in the same direction• Duction – motion of one eye• Eye makes compensatory motions when head

moving– Shake hand vs. shake head

• Saccades – eye movement to use fovea more• Microsaccades – imperceptible motions to

maintain excitation of rods and cones

Frame rate

• 60Hz frame rate is generally considered important– flicker fusion at ~60Hz

• Old movies at low rates but people weren’t sensitized to it

Visual Displays

• How do we get computers to interact with the eye?

Overview

• display technologies

• head-mounted displays (HMD)

• projection-based displays

Overview

•Characteristics of displays

• field of view

• stereo display

• resolution issues

• brightness

HMD

• head-mounted displays (hmd)

• technology

• displays: LCDs, CRTs, OLEDs, other

• totally immersive display experience

• tend to have small FOV (as compared to a more natural FOV)

• often bulky

HMD

• two displays provide image for left and right eye

• see-through displays used for augmented reality

• semi-transparent overlays or video see-through

• What did Brooks say was an advantage of video merge vs optical merge?

HMD characteristics

•what are important characteristics?

• image resolution, brightness, contrast

• tied to visual acuity

• field of view

• the mechanics

• ergonomics, mass, moments of inertia

field of view - FOV

• field of view - can mean different things

• optical field of view - field of view as specified by the HMD optics

• rendered field of view - field of view as specified in software

• Mapping between the two of them

field of view - FOV

• field of regard - the total area over which a user can view

• afford visual integration of larger space

• for HMDs this is generally 360 degrees

• provided the head is tracked

• for projection systems, generally closer to 180 degrees

• CAVE varies

hmd display optics• optics between the image plane

and the user’s eye produce a virtual image farther away from the eye

• reduces accommodative effort

• ideally out a few meters to help cancel out convergence/accommodative rivalry

• optics magnify pixel granularity!

• other distortions?

• Leep optics – radial falloff – needs defocus

eye

optics

screen

virtual imagefocus distance

HMD displays

•CRT-based HMDs

• electron beam aimed at phosphorescent screen, resulting in emitted light

•generally good picture quality

•but often heavier and much more expensive

•Now rare

example of CRT-based HMD

•Datavisor HiRes, Datavisor 80

Datavisor HiRes: 1280x1024, 1.9’/pixel, 42° fov (100% overlap),

about 4lbs, uses monochrome CRT w/ color filter shutters

Datavisor 80: 1280x1024, 3’/pixel, accommodation at

infinity, 80° diagonal fov, 120° with 20% overlap, about 5lbs

nvis nVisor SX• liquid crystal on silicon (LCoS)

• LCD-type device by CRLOpto

• 0.9” microdisplay with 1280x1024 resolution

• horizontal fov: 47°, weight: 2lbs

• good image quality, brightness, and contrast

LCD-based hmds

• in general, LCD HMDs have been less expensive and lighter

•however,

• usually, much lower resolution

• poorer image quality

• lower brightness, contrast

• improvements in LCD technology but VR just now catching up

LCD-based HMDs

•Virtual Research V8

• 640x480 resolution

• approx 45° fov (100% overlap)

• lightweight: ~2lbs

• inexpensive: ~$12k

other HMDs

• Kaiser• resolution 1024x768

• about $15k

• uses active matrix LCD screens

• 2.3’/pixel

• 40° fov

• just over 2lbs

• OLED-based HMDs - better brightness than LCDs, 800x600

• good image quality, lighter

• $5k, but with low fov (28°)

• may have short life span

Kaiser ProView XL50

5DT HMD 800

Sensics

• First new HMD in some time

• Array of microdisplays• OLEDs• Panoramic field of

view• 2.9’/pixel

setting up stereo• monocular: image viewed with one eye only

• bi-ocular: both eyes see the same image

• binocular: each eye sees its own image

• HMDs approximate stereo vision by showing a user left and right eye images• what if the person can’t fuse stereo?

• What about vergence?

• what about IPD?• IPD = inter-pupillary distance

• is this important?

• most HMDs do not provide enough control over the exact settings

perception and fov/stereo

• common explanations for problems in VR:

• too small fov

• didn’t use stereo

• stereo was configured improperly

• truth of the matter is that no one really knows in general for all situations

• requires specific experimentation with

• different setups, tasks, and applications

• FOV is likely important for speed of localizing

• stereo is likely very important for near field interaction

other pros/cons of HMDs

•definite advantages:

• fully immersive

•disadvantages:

• bulky, heavy, obtrusive

• poor resolution, mismatch between accommodation/convergence

•other comments?

projection displays

• setup a projector, aim it at a screen

• you’re doing VR research!

•actually a little more complex than that

•goal is to surround user with the virtual environment

•good concept

• potentially reduces locomotion abilities

• capability to increase resolution dramatically

• tiling the display projectors

cave

• cave

• CAVE Automatic Virtual Environment

• developed at U Illinois-Chicago - 1992

• did you know CAVE is trademarked?

stereo with projection vr

•with HMDs, stereo vision is potentially easier (minus bad artifacts)

• two images, one for each eye

• for projection vr, you have two choices

• active stereo

• passive stereo

active stereo

• shutter glasses are required

• left and right eye shutters on the glasses synchronize with images coming from projector

• crystal eyes

• transmitter synced up with graphics/projector system

• turns right and left shutters on/off

ia state cave

passive stereo

•use the polarization of light to passively send stereo

• two projectors per screen, each with different polarizing filters

• left and right eyes of glasses allow the correct polarized light through

• potentially less expensive

• people just wearing “sun glasses”

important stereo issues

• generally, very expensive!• requires extremely bright projection systems

• projectors must be capable of high refresh rates (120Hz) for active stereo

• bright projectors are required due to issue with light efficiency• first off, about half of light is lost due to left/right

switching glasses

• passive stereo - between 12% and 59% depending on setup

• Screen transmission

other painful issues with projection vr

• alignment matters

• for stereo setups, especially passive stereo, alignment is crucial

• for tile-able displays with high resolution

• images must overlap and blend

other painful issues with projection vr

• actually, should be able to project on any surface

• large set of research devoted to

• automatic alignment

• image blending, color matching

• projection onto arbitrary surfaces

• not just flat walls

Desktop VR

• Also known as “fishtank” VR

• Use computer monitor– Track head– Stereo from shutter

glasses

vision dome and other spherics

Workbench

Autostereoscopic displays

Volumetric displays

• Actuality– Rotating screen