Seeing 3D from 2D Images
description
Transcript of Seeing 3D from 2D Images
Seeing 3D from 2D Seeing 3D from 2D ImagesImages
William and Craig 115 - 164William and Craig 115 - 164
How to make a 2D image How to make a 2D image appear as 3D!appear as 3D!
►Output is typically 2D ImagesOutput is typically 2D Images►Yet we want to show a 3D world!Yet we want to show a 3D world!►How can we do this?How can we do this?
We can include ‘cues’ in the image that We can include ‘cues’ in the image that give our brain 3D information about the give our brain 3D information about the scenescene
These cues are These cues are visual depth cuesvisual depth cues
Visual Depth CuesVisual Depth Cues►Monoscopic Depth Cues (single 2D Monoscopic Depth Cues (single 2D
image)image)►Stereoscopic Depth Cues (two 2D Stereoscopic Depth Cues (two 2D
images)images)►Motion Depth Cues (series of 2D Motion Depth Cues (series of 2D
images)images)►Physiological Depth Cues (body cues)Physiological Depth Cues (body cues)
Monoscopic Depth CuesMonoscopic Depth Cues► InterpositionInterposition
An object that occludes another is An object that occludes another is closercloser
► ShadingShading Shape info. Shadows are included hereShape info. Shadows are included here
► SizeSize Usually, the larger object is closerUsually, the larger object is closer
► Linear PerspectiveLinear Perspective parallel lines converge at a single pointparallel lines converge at a single point
► Surface Texture GradientSurface Texture Gradient more detail for closer objectsmore detail for closer objects
► Height in the visual fieldHeight in the visual field Higher the object is (vertically), the Higher the object is (vertically), the
further it isfurther it is► Atmospheric effects Atmospheric effects
further away objects are blurrierfurther away objects are blurrier► BrightnessBrightness
further away objects are dimmerfurther away objects are dimmer
Stereoscopic Display IssuesStereoscopic Display Issues►StereopsisStereopsis►Stereoscopic Display TechnologyStereoscopic Display Technology►Computing Stereoscopic ImagesComputing Stereoscopic Images►Stereoscopic Display and HTDs.Stereoscopic Display and HTDs.►Works for objects < 5m. Works for objects < 5m. Why?Why?
StereopsisStereopsisThe result of the two slightly different views of the external world that our laterally-displaced eyes receive.
Retinal DisparityRetinal Disparity
f1f2
Left Eye Right Eye
Retinal disparity =
If both eyes are fixated on If both eyes are fixated on a point, f1, in space, then a point, f1, in space, then an image of f1 if focused an image of f1 if focused at corresponding points in at corresponding points in the center of the fovea of the center of the fovea of each eye. Another point, each eye. Another point, f2, at a different spatial f2, at a different spatial location would be imaged location would be imaged at points in each eye that at points in each eye that may not be the same may not be the same distance from the fovea. distance from the fovea. This difference in distance This difference in distance is the retinal disparity.is the retinal disparity.
DisparityDisparity► If an object is closer than the fixation point, If an object is closer than the fixation point,
the retinal disparity will be a negative value. the retinal disparity will be a negative value. This is known as This is known as crossed disparitycrossed disparity because the because the two eyes must cross to fixate the closer object.two eyes must cross to fixate the closer object.
► If an object is farther than the fixation point, If an object is farther than the fixation point, the retinal disparity will be a positive value. the retinal disparity will be a positive value. This is known as This is known as uncrossed disparityuncrossed disparity because because the two eyes must uncross to fixate the farther the two eyes must uncross to fixate the farther object.object.
► An object located at the fixation point or An object located at the fixation point or whose image falls on corresponding points in whose image falls on corresponding points in the two retinae has a the two retinae has a zero disparityzero disparity..
Convergence AnglesConvergence Angles
i
f2
f1
D1
D2a b
c d
1
+a+c+b+d = 180+a+c+b+d = 180+c+d = 180+c+d = 180-- = a+(-b) = = a+(-b) =
1+1+2 = Retinal 2 = Retinal DisparityDisparity
2
Miscellaneous Eye FactsMiscellaneous Eye Facts►StereoacuityStereoacuity - the smallest depth - the smallest depth
that can be detected based on retinal that can be detected based on retinal disparity.disparity.
►Visual DirectionVisual Direction - Perceived spatial - Perceived spatial location of an object relative to an location of an object relative to an observer.observer.
HoroptersHoropters► Corresponding points on Corresponding points on
the two retinae are the two retinae are defined as being the defined as being the same vertical and same vertical and horizontal distance from horizontal distance from the center of the fovea in the center of the fovea in each eye.each eye.
► Horopter - the locus of Horopter - the locus of points in space that fall points in space that fall on corresponding points on corresponding points in the two retinae when in the two retinae when the two eyes binocularly the two eyes binocularly fixate on a given point in fixate on a given point in space (zero disparity).space (zero disparity).
► Points on the horopter Points on the horopter appear at the same depth appear at the same depth as the fixation point.as the fixation point.
f1
f2
Vieth-Mueller Circle
Stereoscopic DisplayStereoscopic Display►Stereoscopic images are easy to do Stereoscopic images are easy to do badly, hard to do well, and impossible badly, hard to do well, and impossible
to do correctly.to do correctly.
Stereoscopic DisplaysStereoscopic Displays►Stereoscopic display systems create a Stereoscopic display systems create a
three-dimensional image (versus a three-dimensional image (versus a perspective image) by presenting each perspective image) by presenting each eye with a slightly different view of a eye with a slightly different view of a scene.scene. Time-parallelTime-parallel Time-multiplexedTime-multiplexed
Time Parallel Stereoscopic Time Parallel Stereoscopic DisplayDisplay
Two ScreensTwo Screens► Each eye sees a Each eye sees a
different screendifferent screen►Optical system Optical system
directs each eye to directs each eye to the correct view.the correct view.
►HMD stereo is done HMD stereo is done this way.this way.
Single ScreenSingle Screen► Two different images Two different images
projected on the projected on the same screensame screen
► Images are polarized Images are polarized at right angles to at right angles to each other.each other.
► User wears polarized User wears polarized glasses (passive glasses (passive glasses).glasses).
Passive Polarized Projection Passive Polarized Projection IssuesIssues
►Linear PolarizationLinear Polarization Ghosting increases when you tilt headGhosting increases when you tilt head Reduces brightness of image by about ½Reduces brightness of image by about ½ Potential Problems with Multiple Screens Potential Problems with Multiple Screens
(next slide)(next slide)►Circular PolarizationCircular Polarization
Reduces ghosting but also reduces Reduces ghosting but also reduces brightness and crispness of image even brightness and crispness of image even moremore
Problem with Linear Problem with Linear PolarizationPolarization
► With linear polarization, With linear polarization, the separation of the the separation of the left and right eye left and right eye images is dependent on images is dependent on the orientation of the the orientation of the glasses with respect to glasses with respect to the projected image. the projected image.
► The floor image cannot The floor image cannot be aligned with both the be aligned with both the side screens and the side screens and the front screens at the front screens at the same time. same time.
Time Multiplexed DisplayTime Multiplexed Display►Left and right-eye views of an image Left and right-eye views of an image
are computed and alternately are computed and alternately displayed on the screen.displayed on the screen.
►A shuttering system occludes the right A shuttering system occludes the right eye when the left-eye image is being eye when the left-eye image is being displayed and occludes the left-eye displayed and occludes the left-eye when the right-eye image is being when the right-eye image is being displayed.displayed.
Stereographics Shutter Stereographics Shutter GlassesGlasses
Screen ParallaxScreen Parallax
P
Left eye position
Right eye position
Pleft
Pright
Pright
Pleft
P
Display Screen
Object with positive parallax
Object with negative parallax
The screen parallax is the distance between the projected locationof P on the screen, Pleft, seen by the left eye and the projected
location, Pright, seen by the right eye (different from retinal disparity).
Screen Parallax Screen Parallax (cont.)(cont.)
f1
p
id
Left eyepoint
Right eyepoint
Projection Plane
D
p = i(D-d)/Dp = i(D-d)/Dwhere p is the amount of screen where p is the amount of screen
parallax for a point, f1, when parallax for a point, f1, when projected onto a plane a projected onto a plane a distance d from the plane distance d from the plane containing two eyepoints. containing two eyepoints.
i is the interocular distance i is the interocular distance between eyepoints and between eyepoints and
D is the distance from f1 to the D is the distance from f1 to the nearest point on the plane nearest point on the plane containing the two eyepointscontaining the two eyepoints
d is the distance from the d is the distance from the eyepoint to the nearest point eyepoint to the nearest point on the screenon the screen
Screen ParallaxScreen Parallax
-65.00
-55.00
-45.00
-35.00
-25.00
-15.00
-5.00
5.00
0 50 100 150 200 250 300 350
Distance from Eye
Scre
en P
aral
lax
Zero parallax at screen, max positive parallax is i, negative parallax is equal to I halfway between eye and screen
Stereoscopic VoxelsStereoscopic Voxels
Left Eye Point
Right Eye Point
1
1
1
1
1
1
1
11
2
2
2
2
2
2
2
3
3
3
3
3
3
3
4
4
4
4
4
4
5
5
5
5
5
2
AB
Screen Parallax and Screen Parallax and Convergence AnglesConvergence Angles
f1f2 f3
Projection Plane
► Screen parallax Screen parallax depends on closest depends on closest distance to screen.distance to screen.
►Different convergence Different convergence angles can all have the angles can all have the same screen parallax. same screen parallax.
► Also depends on Also depends on assumed eye assumed eye separation.separation.
How to create correct left- How to create correct left- and right-eye viewsand right-eye views
►To specific a single view in almost all To specific a single view in almost all graphics software or hardware you graphics software or hardware you must specify:must specify: EyepointEyepoint Look-at PointLook-at Point Field-of-View or location of Projection Field-of-View or location of Projection
PlanePlane View Up DirectionView Up Direction
Basic Perspective Projection Basic Perspective Projection Set Up from Viewing Set Up from Viewing
ParamentersParamentersY
Z
X
Projection Plane is orthogonal to one of the major axes (usually Z). That axis is along the vector defined by the eyepoint and the look-at point.
What doesn’t workWhat doesn’t work
•Each view has a different projection plane
•Each view will be presented (usually) on the same plane
What Does WorkWhat Does Work
i i
Setting Up Projection Setting Up Projection GeometryGeometry
Look at pointEyeLocations
Look at points
EyeLocations
No
Yes
Screen SizeScreen SizeThe size of the window does
not affect the retinal disparityfor a real window.
Once computed, the screen parallaxis affected by the size of the displayscreen
Visual Angle SubtendedVisual Angle Subtended
Screen parallax is measured in terms of visual angle. This is a screenindependent measure. Studies have shown that the maximum anglethat a non-trained person can usually fuse into a 3D image is about
1.6 degrees. This is about 1/2 the maximum amount of retinal disparityyou would get for a real scene.
Accommodation/ Accommodation/ ConvergenceConvergence
Display Screen
Position Dependence Position Dependence (without head-tracking)(without head-tracking)
Interocular DependanceInterocular Dependance
F
Modeled Point
Perceived Point
Projection Plane
True EyesModeled Eyes
Obvious Things to DoObvious Things to Do►Head trackingHead tracking►Measure User’s Interocular DistanceMeasure User’s Interocular Distance
Another ProblemAnother Problem►Many people can not fuse stereoscopic Many people can not fuse stereoscopic
images if you compute the images images if you compute the images with proper eye separation!with proper eye separation!
►Rule of ThumbRule of Thumb: Compute with about ½ : Compute with about ½ the real eye separation.the real eye separation.
►Works fine with HMDs but causes Works fine with HMDs but causes image stability problems with HTDs image stability problems with HTDs (why?)(why?)
Two View Points with Head-TrackingTwo View Points with Head-Tracking
Projection Plane
Modeled Point
Perceived Points
Modeled Eyes
True Eyes
Maximum Depth PlaneMaximum Depth Plane
Maximum Depth PlaneModeled Eyes
True Eyes
EF
Modeled Point
Perceived Point
Projection Plane
Can we fix this?Can we fix this?► Zachary Wartell, "Stereoscopic Head-Tracked Displays: Analysis Zachary Wartell, "Stereoscopic Head-Tracked Displays: Analysis
and Development of Display Algorithms," Ph.D. Dissertation, and Development of Display Algorithms," Ph.D. Dissertation, Georgia Institute of Technology, August 2001. Georgia Institute of Technology, August 2001.
► Zachary Wartell, Larry F. Hodges, William Ribarsky. "An Zachary Wartell, Larry F. Hodges, William Ribarsky. "An Analytic Comparison of Alpha-False Eye Separation, Image Analytic Comparison of Alpha-False Eye Separation, Image Scaling and Image Shifting in Stereoscopic Displays," Scaling and Image Shifting in Stereoscopic Displays," IEEE IEEE Transactions on Visualization and Computer Graphics, Transactions on Visualization and Computer Graphics, April-June April-June 2002, Volume 8, Number 2, pp. 129-143. (related tech report is 2002, Volume 8, Number 2, pp. 129-143. (related tech report is GVU Tech Report 00-09 (GVU Tech Report 00-09 ( Abstract Abstract , , PDFPDF , , PostscriptPostscript .) .)
► Zachary Wartell, Larry F. Hodges, William Ribarsky. "Balancing Zachary Wartell, Larry F. Hodges, William Ribarsky. "Balancing Fusion, Image Depth, and Distortion in Stereoscopic Head-Fusion, Image Depth, and Distortion in Stereoscopic Head-Tracked Displays." Tracked Displays." SIGGRAPH 99 Conference ProceedingsSIGGRAPH 99 Conference Proceedings, , Annual Conference Series. ACM SIGGRAPH, Addison Wesley, Annual Conference Series. ACM SIGGRAPH, Addison Wesley, August 1999, p351-357. (Paper: August 1999, p351-357. (Paper: AbstractAbstract , , PDFPDF , , PostscriptPostscript ; ; SIGGRAPH CD-ROM Supplement, SIGGRAPH CD-ROM Supplement, supplement.zip,supplement.zip, supplement.tar.Zsupplement.tar.Z ). ).
Point of fixationPoint of fixation
Distance in centimeters from eye plane
Chan
ge in
eye
poin
tse
para
tion
in c
entim
eter
s
0.000.050.100.150.200.250.300.350.40
10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
Symmetric convergence
Convergence 20 centimeters to the left of the left eye
Change in eyepoint separation with change in point of fixation.Centers of rotation of the eyes are assumed to be 6.4 centimeters apart.
GhostingGhosting►Affected by the amount of light Affected by the amount of light
transmitted by the LC shutter in its off transmitted by the LC shutter in its off state.state.
►Phosphor persistencePhosphor persistence►Vertical screen position of the image.Vertical screen position of the image.
Ghosting Ghosting (cont.)(cont.)
Extinction Ratio =
Luminance of the correct eye image------------------------------------------------------------Luminance of the opposite eye ghost image
Image Position Red WhiteTop 61.3/1 17/1Middle 50.8/1 14.4/1Bottom 41.1/1 11/1
Ghosting Ghosting (cont.)(cont.)
►Factors affecting perception of Factors affecting perception of ghostingghosting Image brightnessImage brightness ContrastContrast Horizontal parallaxHorizontal parallax Textural complexityTextural complexity
Time-parallel stereoscopic Time-parallel stereoscopic imagesimages
► Image quality may also be affected byImage quality may also be affected by Right and left-eye images do not match in Right and left-eye images do not match in
color, size, vertical alignment.color, size, vertical alignment. Distortion caused by the optical systemDistortion caused by the optical system ResolutionResolution HMDs interocular settingsHMDs interocular settings Computational model does not match Computational model does not match
viewing geometry.viewing geometry.
Motion Depth CuesMotion Depth Cues► Parallax Parallax
created by created by relative head relative head position and position and object being object being viewed.viewed.
►Objects nearer Objects nearer to the eye to the eye move a greater move a greater distancedistance
Pulfrich EffectPulfrich Effect►Neat trickNeat trick►Different levels of illumination require Different levels of illumination require
additional time (your frame rates differ additional time (your frame rates differ base of amount of light)base of amount of light)
►What if we darken one image, and What if we darken one image, and brighten another?brighten another?
►http://dogfeathers.com/java/pulfrich.hthttp://dogfeathers.com/java/pulfrich.htmlml
►www.cise.ufl.edu/~lok/multimedia/www.cise.ufl.edu/~lok/multimedia/videos/pulfrich.avivideos/pulfrich.avi
Physiological Depth CuesPhysiological Depth Cues►Accommodation Accommodation – focusing – focusing
adjustment made by the eye to adjustment made by the eye to change the change the shapeshape of the lens. (up to 3 of the lens. (up to 3 m)m)
►ConvergenceConvergence – movement of the – movement of the eyes to bring in the an object into the eyes to bring in the an object into the same location on the retina of each same location on the retina of each eye.eye.
SummarySummary►Monoscopic – Interposition is strongest.Monoscopic – Interposition is strongest.►Stereopsis is very strong.Stereopsis is very strong.►Relative Motion is also very strong (or Relative Motion is also very strong (or
stronger).stronger).►Physiological is weakest (we don’t even Physiological is weakest (we don’t even
use them in VR!)use them in VR!)►Add as neededAdd as needed
ex. shadows and cartoonsex. shadows and cartoons