Robot\Machine Vision

Computer vision“Computer vision (or machine vision) is the science and technology of machines that see. Here see means the machine is able to extract information from an image, to solve some task, or perhaps “understand” the scene in either a broad or limited sense”

Some applications:sky

water

Ferris wheel

amusement park

Cedar Point

12 E

tree

tree

tree

carouseldeck

people waiting in line

ride

rideride

umbrellas

pedestrians

maxair

bench

tree

Lake Erie

people sitting on ride

ObjectsActivitiesScenesLocationsText / writingFacesGesturesMotionsEmotions…

The Wicked Twister

3D Reconstruction:

Given many images of a certain scene we can use computer vision algorithms to reconstruct the 3D model.

Connection to other disciplines :

Mathematics

Algorithms

Image processing

Artificial intelligence

GraphicsMachine learning

Computer vision

Robotics

I(176,201) = 164 I(194,203) has value 37

width 520j=1

500 height

i=1

Intensity : [0,255]

Image representation on Computer:

R G B

Color images, RGB color space :

RGB to Grayscale

Image formation – Pinhole Camera:

• Pinhole camera is a simple model to approximate imaging process, perspective projection.

If we treat pinhole as a point, only one ray from any given point can enter the camera.

Virtual image

pinhole

Image plane

Perspective Projection

• A 3D orthogonal coordinate system with its origin at O. This is also where the camera aperture is located..

• An image plane where the 3D world is projected through the aperture of the camera. The image plane is parallel to axes X1 and X2. -f where f > 0. f is also referred to as the focal length of the pinhole camera.

• A point R at the intersection of the optical axis and the image plane. This point is referred to as the principal point or image center.

• A point P somewhere in the world at coordinate relative to the axes X1,X2,X3.• The projection line of point P into the camera. This is the green line which passes through

point P and the point O.• There is also a 2D coordinate system in the image plane, with origin at R and with axes Y1 and

Y2 which are parallel to X1 and X2, respectively. The coordinates of point Q relative to this coordinate system is .

• We have 2 similar triangles• So:

Perspective Projection

1 1 11

3 3

2 2 22

3 3

1 1

2 23

y x fxyf x xy x fxyf x x

y xfy xx

• The resulting image is rotated 180 degrees.• In order to produce an unrotated image there are two

possibilities: 1. Rotate the coordinate system in the image plane 180° (in either

direction). This is the way any practical implementation of a pinhole camera would solve the problem; for a photographic camera we rotate the image before looking at it, and for a digital camera we read out the pixels in such an order that it becomes rotated.

2. Place the image plane so that it intersects the X3 axis at f instead of at -f and rework the previous calculations. This would generate a virtual (or front) image plane which cannot be implemented in practice, but provides a theoretical camera which may be simpler to analyze than the real one.

Perspective Projection

EdgeDetection

14/33 עיבוד תמונות ואותות במחשב

Edge Detection - גילוי שפות

מפת שפות של התמונה

15/33 עיבוד תמונות ואותות במחשב

Edge Detection - גילוי שפות

• We look at the image as a continuous function f(x,y) .• The gradient of this function:

• The gradient direction measures change in intensity, and the size of the gradient is the value of the highest slope.

22

yf

xff

yf

xff ,

xfyf

f arctan

Gradient - Example

17/33 עיבוד תמונות ואותות במחשב

Gradient - Example

fxf

yf

18/33 עיבוד תמונות ואותות במחשב

Gradient Example

>> i = double(imread('cameraman.tif'));>> gradFilt = [-1 0 1 ; -2 0 2 ; -1 0 1]/2;>> grad_x = imfilter(i , gradFilt , 'same' , 'replicate');>> grad_y = imfilter(i , gradFilt' , 'same' , 'replicate');>> [x,y] = meshgrid([1:size(i,2)] , [1:size(i,1)]);>> figure; imshow(i , []); hold on; >> quiver(x , y , grad_x , grad_y , 3 , 'm' , 'LineWidth' , 1);

19/33 עיבוד תמונות ואותות במחשב

Yet another examplefrice.png

20/33 עיבוד תמונות ואותות במחשב

Approximation of the gradient

10-1

10-1

10-1

-1-1-1

000

111

10-1

20-2

10-1

-1-2-1

000

121

prewitt sobel

מסנן לחשובxנגזרת בכיוון

מסנן לחשובxנגזרת בכיוון

מסנן לחשובyנגזרת בכיוון

מסנן לחשובyנגזרת בכיוון

21/33 עיבוד תמונות ואותות במחשב

Edge Detection - גילוי שפות

f

22/33 עיבוד תמונות ואותות במחשב

Filtering by gradient sizeT = 100 T = 70 T =40

T = 20 T = 10 T = 2