STEREO VISION Computer Vision

Farah Al-Tufaili

04/15/2023

2

Stereo Vision

Two Seeing Eyes = Two Views!

Two Views Used and Fused in the Brain = Stereovision!

Human Beings with Two Eyes that Work Together Have Stere

04/15/2023

3

Computer stereo vision

Computer stereo vision is the extraction of 3D information from digital images. By comparing information about a scene from two vantage points, 3D information can be extracted by examination of the relative positions of objects in the two panels.

04/15/2023

4

Inferring 3D From 2D

Model based pose estimation :If we have single camera that’s calibrated and we have a model we know the geometry of the model so we can determine the pose of the camera with respect to the model called Model based pose estimation

Known model

single (calibrated)

camera

-> Can determine the

pose of the model

04/15/2023

5

Inferring 3D From 2D

Stereo vision: if we have two cameras and we know the relative pose between them we can find 3D information from an arbitrary seen we don’t have to know model in the seen and we can determine the position in that scene from those two cameras

Arbitrary scene

two (calibrated)

camera

-> Can determine the

positions of points in the

scene

Relative pose between cameras is also known

04/15/2023

6

Outline

In traditional stereo vision, two cameras, displaced horizontally from one another are used to obtain two differing views on a scene, in a manner similar to human binocular vision. By comparing these two images, the relative depth information can be obtained, in the form of disparities, which are inversely proportional to the differences in distance to the objects.

To compare the images, the two views must be superimposed in a stereoscopic device, the image from the right camera being shown to the observer's right eye and from the left one to the left eye.

04/15/2023

7

04/15/2023

8

A way of getting depth (3-D) information about a scene from two (or more) 2-D images

- Used by humans and animals, now computers

Left image Right image

Reconstructed surface with image texture

04/15/2023

9

Iright = im2double(imread('pentagonRight.png'));

Ileft = im2double(imread('pentagonLeft.png'));

% Disparity is d = xleft-xright

% So Ileft(x,y) = Iright(x+d,y)

for d=-20:20

d

Idiff = abs(Ileft(:, 21:end-20) - Iright(:, d+21:d+end-20));

imshow(Idiff, []);

pause

end

04/15/2023

10

Stereo Principle

If you know

• intrinsic parameters of each camera

• the relative pose between the cameras

If you measure

• An image point in the left camera

• The corresponding point in the right camera

Each image point corresponds to a ray emanating from that camera

You can intersect the rays (triangulate) to find the absolute point position

04/15/2023

11

Stereo Geometry – Simple Case

Assume image planes are coplanar.

There is only a translation in the X direction between the two coordinate frames.

b is the baseline distance between the cameras.

XL

ZR ZL

XR

xR

P(XL,YL,ZL)

Right camera

disparity: the difference in image location of the same 3D point when projected under perspective to two different cameras

d = xleft-xright

xL

Left camera

f

b

04/15/2023

12

04/15/2023

13

Stereo Geometry – Simple Case

f is the focal lenth, b is the baseline distance between the cameras.

d= , ,

and = = =• We can see as the

disparity increases the Z value is smaller.

• And as disparity decreases the point goes further way

XL

ZR ZL

XR

xR

P(XL,YL,ZL)

Right camera xL

Left camera

f

b

04/15/2023

14

Geometry for parallel cameras

Let us consider the optical setting in the figure, that is also called standard model.

1. L and R are two cameras with parallel optical axes. Let f be the focal length of both cameras.

2. The baseline (that is the line connecting the two lens centers) is perpendicular to the optical axes. Let b be the distance between the two lens centers.

3. XZ is the plane where the optical axes lie, XY plane is parallel to the image plane of both cameras, X axis equals the baseline and the origin O of (X,Y,Z) world reference system is the lens center of the left camera.

04/15/2023

15

Simple Model: Optic axes of 2 cameras are parallel

, , =

(from similar triangles)Y-axis isperpendicularto the page.

04/15/2023

16

3D from Stereo Images: Triangulati

For stereo cameras with parallel optical axes, focal length f,

baseline b, corresponding image points (xl,yl) and (xr,yr), the location of the 3D point can be derived from previous slide’s equations:

Depth z = =

or or

This method ofdetermining depthfrom disparity d is

called triangulation.

04/15/2023

17

Disparity is higher for points closer to the camera

04/15/2023

18

04/15/2023

19

Goal: a complete disparity

Disparity is the difference in position of corresponding points between the left and right images .

04/15/2023

20

Reconstruction Error

Given the uncertainty in pixel projection of the point, what is the error in depth?

Obviously the error in depth (∆Z) will depend on:

Z, b, f

∆xL, ∆ xR

Let’s find the expected value of the error, and the variance of the error

04/15/2023

21

Reconstruction Error

First, find the error in disparity Dd, from the error of locating the feature in each image, ∆ XL and ∆ XR

d= Taking the total derivative of each side

d(d)=d()

∆ d=

Assuming ∆xL, ∆xR are independent and zero mean

04/15/2023

22

Reconstruction Error

And

=

=

So:

=+

=0Because ∆xL, ∆xR are independent and zero mean

04/15/2023

23

Reconstruction Error

Next, we take the total derivative of Z= If the only uncertainty is in the disparity d

∆Z= The mean error is = E[∆ Z]

=0

The variance of the error is = E [(∆ Z- )2]

E [(∆ Z- )2] ==

04/15/2023

24

Example

A stereo vision system estimates the disparity of a point as d=10 pixels

What is the depth (Z) of the point, if f = 500 pixels and b = 10 cm?Z= =(500 pix)

What is the uncertainty (standard deviation) of the depth, if the standard deviation of locating a feature in each image = 1 pixel? =+=2

(500 cm)

04/15/2023

25

Example – continues

Find 3D point corresponding to 2 point P1 and P2 in from right and left camera respectively ,where P1(88,90) ,P2 (100,90). f=500 cm , b=10 bix.

z = =

=

So P which is 3D point is :