SYLLABUS 3D Computer Vision - Scientific Computing and Imaging
Lecture 5. 3-D imaging and vision (2 hours)
19
L. Yaroslavsky. Course 5107212. Selected Topics in Image Processing, Graphics and Computer Vision Lecture 5. 3-D imaging and vision (2 hours) 5.1. Mechanisms of 3-D human vision. Evolution of Two-eyed Vision. Vision in fishes, herbivorous animal and carnivores. Stereoscopic vision. http://www.strabismus.org/all_about_strabismus.html ; http://www.strabismus.org/critical_period_Hubel.html http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html ; http://www.cee.hw.ac.uk/~alison/ai3notes/section2_8_3.html Stereopsis is from the Greek word for "solid sight", and thus refers to perception of three-dimensional shape from any source of depth information. Within psychological writings it refers to the perception of three-dimensional shapes and distances that results from the differential views afforded by the two eyes. 5.2. 3-D displays; In 1838 Charles Wheatstone published the first paper on stereopsis entitled "On some remarkable and hitherto unobserved, phenomena of binocular vision". In this decoratively titled, classic paper Wheatstone pointed out that the positional differences in the two eyes' images due to their horizontal separation yielded depth information. Prior to this time the principal problem in the study of binocular vision was: how is the world seen as single when we have two different views of it? People thought that only objects with the same visual directions, falling on corresponding points would be seen as single, all other points would be double. By means of the stereoscope that he invented Wheatstone demonstrated that not only does the stimulation of non-corresponding points yield singleness of vision, it also results in the perception of depth. Wheatstone's great rival was the professor of physics at St. Andrew's University, David Brewster. Brewster contested the primacy of Wheatstone's discovery of stereopsis and an acrimonious correspondence in the Times ensued. Wheatstone prevailed but Brewster did invent the most popular (http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html#Arditi ) Victorian stereoscope: the lenticular stereoscope. Liquid Crystal Shutter Stereoscopes Ross (1974) established that stereopsis can be obtained if the left and right images are separated by being presented alternately in rapid succession; typically around 20 times per second for each eye. This technique requires electronic shutters in front of the eyes that can be opened and closed in synchrony with presentation of successive images corresponding to the two eyes' views on a screen. This is achieved with liquid crystal shutters driven by a computer in synchrony with the image alteration. Passing an electric current through the liquid crystal changes its state from transparent to opaque so the eyes can alternately see and not see through the shutters. The two eyes shutters are lagged such that when one is transparent the other is opaque, and the stereo half images alternate at the same rate (http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html#Arditi ). Anaglyph method 5.3. Redundancy of stereoscopic images Tolerance of stereoscopic vision to blur of one the images, its theoretical foundation and experimental confirmation Tolerance of color stereoscopic vision to loss of saturation of one of the images. Experimental confirmation. Relationship between stereoscopic and color vision 5.4. New anaglyph method for stereoscopic display: - Image alignment - Blurring one color component - Compression of the depth map dynamic range
Transcript of Lecture 5. 3-D imaging and vision (2 hours)
L. Yaroslavsky. Course 5107212. Selected Topics in Image Processing, Graphics and Computer Vision Lecture 5. 3-D imaging and vision (2 hours)
5.1. Mechanisms of 3-D human vision. Evolution of Two-eyed Vision. Vision in fishes, herbivorous animal and carnivores. Stereoscopic vision. http://www.strabismus.org/all_about_strabismus.html ; http://www.strabismus.org/critical_period_Hubel.html http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html; http://www.cee.hw.ac.uk/~alison/ai3notes/section2_8_3.html
Stereopsis is from the Greek word for "solid sight", and thus refers to perception of three-dimensional shape from any source of depth information. Within psychological writings it refers to the perception of three-dimensional shapes and distances that results from the differential views afforded by the two eyes. 5.2. 3-D displays; In 1838 Charles Wheatstone published the first paper on stereopsis entitled "On some remarkable and hitherto unobserved, phenomena of binocular vision". In this decoratively titled, classic paper Wheatstone pointed out that the positional differences in the two eyes' images due to their horizontal separation yielded depth information. Prior to this time the principal problem in the study of binocular vision was: how is the world seen as single when we have two different views of it? People thought that only objects with the same visual directions, falling on corresponding points would be seen as single, all other points would be double. By means of the stereoscope that he invented Wheatstone demonstrated that not only does the stimulation of non-corresponding points yield singleness of vision, it also results in the perception of depth. Wheatstone's great rival was the professor of physics at St. Andrew's University, David Brewster. Brewster contested the primacy of Wheatstone's discovery of stereopsis and an acrimonious correspondence in the Times ensued. Wheatstone prevailed but Brewster did invent the most popular (http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html#Arditi) Victorian stereoscope: the lenticular stereoscope. Liquid Crystal Shutter Stereoscopes Ross (1974) established that stereopsis can be obtained if the left and right images are separated by being presented alternately in rapid succession; typically around 20 times per second for each eye. This technique requires electronic shutters in front of the eyes that can be opened and closed in synchrony with presentation of successive images corresponding to the two eyes' views on a screen. This is achieved with liquid crystal shutters driven by a computer in synchrony with the image alteration. Passing an electric current through the liquid crystal changes its state from transparent to opaque so the eyes can alternately see and not see through the shutters. The two eyes shutters are lagged such that when one is transparent the other is opaque, and the stereo half images alternate at the same rate (http://pages.slc.edu/~ebj/sight_mind/stereo/stereo.html#Arditi). Anaglyph method 5.3. Redundancy of stereoscopic images Tolerance of stereoscopic vision to blur of one the images, its theoretical foundation and experimental confirmation Tolerance of color stereoscopic vision to loss of saturation of one of the images. Experimental confirmation. Relationship between stereoscopic and color vision 5.4. New anaglyph method for stereoscopic display:
- Image alignment - Blurring one color component - Compression of the depth map dynamic range
How does human 3D vision work? Mono view vision
How does human 3D vision work? Stereoscopic vision
Stereoscopes
Charles Wheatstone’s stereoscope (1838)
By placing a plus lens one focal length in front of the half images parallel convergence of the eyes will be enforced and the images will fuse. With this lens arrangement the size of the half images is limited to the size of the interocular separation. To overcome this limitation Brewster added prisms to his lenticular stereoscope. However this results in some prismatic distortion of the images. Liquid Crystal Shutter Stereoscopes Ross (1974) established that stereopsis can be obtained if the left and right images are separated by being presented alternately in rapid succession; typically around 20 times per second for each eye. This technique requires electronic shutters in front of the eyes that can be opened and closed in synchrony with presentation of successive images corresponding to the two eyes' views on a screen. This is achieved with liquid crystal shutters driven by a computer in synchrony with the image alteration. Passing an electric current through the liquid crystal changes its state from transparent to opaque so the eyes can alternately see and not see through the shutters. The two eyes shutters are lagged such that when one is transparent the other is opaque, and the stereo half images alternate at the same rate.
Methods For 3D Image Projection
Screen
H-Polaraizer
V-Polaraizer Screen
H-Polaraizer
V-Polaraizer
W. Rollman in 1853 was first to illustrate the principle of the anaglyph using blue and red lines on a black field with red and blue glasses to perceive the effect, but this was for line drawings only. In 1858 Joseph D'Almeida began projecting three-dimensional magic lantern slide shows using red and green filters with the audience wearing red and green goggles. It is to Louis Ducas du Hauron that we owe the first printed anaglyphs, produced in 1891. This process consisted of printing the two negatives which form a stereoscopic photograph on to the same paper, one in blue (or green), one in red.
Odd frame Even frame
Right
Left
Redundancy of stereoscopic vision and images
0 1 2 3 4 5 60
5
10
15
20
25
30
35
40
45
Decimation factor
RM
S e
rror
of
para
llax
mea
sure
men
ts,
mic
rons
Standard deviation of parallax measuring versus image blur and (adopted from L.P. Yaroslavsky, On Redundancy of Stereoscopic Pictures, Acta Polytechnica Scandinavica, n. 149. Image Science'85. Proceedings. Helsinki, Finland., 11-14 June 1985, V. 1, p. 82-85.)
Test images
Test depth maps
Computer generated random dot stereo pair
Computer generated random dot anaglyph
Test depth maps and computer generated stereo images
Bela Julesz (1928-2003)
1 2 3 4 0
2
4
6
8
Scale Factor
1 2 3 4 0
5
10
15
20
Scale Factor c) - test image “random dots”, d) - test image “random dots”, Blue curves represent mean values of stereopsis threshold
Stereopsis threshold as a function of image blur
1 2 3 4 5 6 7 8 9 10 0.5
1
1.5
2
2.5
3
1/BlurFactor
a)
1 2 3 4 5 6 7 8 9 10 0.4 0.6 0.8 1
1.2 1.4 1.6
1/BlurFactor
“ideal” low pass filter
b)
a) test image “random dots”, 10 experiments (dots) with one viewer and data average (red) and median (blue); b) - test image “Color random patches”, 3 viewers (dots) and data average (red) and median (blue)
“Daubechie -1 wavelet
shrinkage Haar wavelet shrinkage
a) Random Dots (depth = 0.5)
b) Random dots (depth=0.75)
e) Textile (depth=1.75)
f) Textile (30 sights, depth=1)
c) Piece Wise Noisy BW (depth 1.5)
d) Piece Wise BW (depth 1.5)
g) Texture 2 (30 sights, depth=0.5)
h) Texture 2 (30 sights, depth=1..5)
1/BlurFactor
1 2 3 4 5 6
7
8
1 2 3 4 56 7 8
9
10
1/BlurFact
1 2 3 4 5 5.
5.8
6.
6.
7
1/BlurFactor
1 2 3 4 57
7.4
7.8
8.2
1/BlurFactor
1 2 3 4 5 5.5
6
6.5
7
7.5
5 1 2 3 46
6.
7 7.5
8
8.5
1/BlurFactor 1/BlurFactor
1 2 3 4 56
6.57
7.5 8
1/BlurFa 1 2 3 4 55
5.4
5.8
6.2
1/BlurFactor
3D target localization accuracy as a function of image blur using approximated low pass filter.
(((( ))))RGBRgRGBR −−−−++++====mod
(((( ))))RGBGgRGBG −−−−++++====mod
(((( ))))RGBBgRGBB −−−−++++====mod , (((( )))) 3BGRRGB ++++++++====
Tolerance of 3D color vision to loss of color saturation
Right
Left
Color anaglyphs
Image alignment
Blurring anaglyph red component
Blurring anaglyph red component
Depth map
Depth Map Manipulation
d
h
αααα ββββ
P(i+pdepth(i,j),j)P(i,j)
Pdepth(i,j)
Right image Left image
Depth map
Depth(i,j)
P(i+pdepth(i,j),j)P(i,j)
Pdepth(i,j)
Right image Left image
Depth map
Depth(i,j)
P(i,j)
Pdepth(i,j)
Right image Left image
Depth map
Depth(i,j)
Depth map
synthesis
Depth map manipulation
(i.e. compression of the dynamic range)
Synthesis of modified
stereo images
Synthesis of anaglyph
Raw data anaglyph Synthetic anaglyph
Re-synthesis of stereoscopic images by means of depth map manipulation
