1

Machine Visionlecture 1

Thomas Moeslund

Computer Vision and Media Technology lab.

Aalborg University

tbm@cvmt.dk

2

• Pose estimation– Pick and place applications

– Bin-picking

Machine vision applications

• Classification

• Quality control

THOR HOF

THOR

Conveyor belt

3

Machine Vision class

• Purpose: – Provide an overview of the different elements

in a machine vision system

4

Machine Vision class• Topics

– Acquisition of images: • Light, camera, lens

– Representation of digital images: • pixels, colors

– Processing of images • filtering, segmentation

– Analysis of images: • feature extraction, pattern recognition

– Your interests and needs?

5

Plan for today

• Where does an image come from?• Image definitions• Camera types• Image formation

– Lens • Creating light for machine vision• ( other applications )

6

Where does an image come from?

7

Where does an image come from?

8

Where does an image come from?

Charged coupled deviceCCD-chip

9

Where does an image come from?

• Integration over time– Exposure time/shutter

time• DK: lukketid

– Maximum charge• Saturation• Blooming

Under exposed

Over exposed

Correct exposed

10

Where does an image come from?

• Image elements, picture elements, pels, pixels

11

Image definitions

12

Digital Image Representation

• Image is seen as a discrete function f(x,y) as opposed to a continuous function

• x and y cannot take on any value!

y

x

f(x,y)

Origin

13

Discrete image coordinate system

y

x

f(x,y)

Origin

y

xf(0,0)

f(2,6)

14

Digital Image Representation

• Pixel representation (bits)– One bit: {0,1}

– One byte = eight bits

– One pixel: one byte = eight bits = one number: [0,255]

– Grey-scale, intensity, black/white: 8 bits = [0,255]

– Binary image: 1 bit {0,1}. Black and white: visualized as: 8 bit {0,255} ( colors: Another lecture )

• Video: 25 – 60 images per second. – Framerate: 25Hz – 60Hz– For example: 500 x 500 pixel at 50Hz => 1.25x107 bytes per second!!!!

15

Digital Image Representation

• An image f(x,y) is represented

as an Array• Width =

number of pixels in x-direction• Height = number of pixels in y-direction• Size (width x height, width > height)• ROI = region of interest

– To reduce the amount of data

Width

Hei

ght ROI

16

Spatial Image Resolution:

• Resolution– The size of an area in a scene that is represented

by one pixel in the image• Different Resolutions are possible (256x256….16x16)

• Lower resolution leads to data reduction!

17

Gray-level Resolution: Quantization

• Different gray-level resolutions: 256, 128, …, 2• Less gray-levels leads to data reduction.• For 256, 128, 64 gray-levels: Difference hardly visible

18

Camera types

19

Interlaced vs. Progressive

20

Sensor Chip Formats

Number of Pixelsfrom 500x500to 5000x5000

Pixel sizefrom 4m x 4 mto 16 m x 16 m

21

Line scan camera

22

Image formation- the lens

23

The lens

• A lens focuses a bundle of rays to one point• Parallel rays pass through a focal point at a distance F beyond the plane of the lens. F is the focal length• O is the optical center• F and O span the optical axis

24

Thin Lens & Gaussian Formula

fbg

111

25

Focus and depth-of-field • Depth-of-field (DK: dybteskarphed)• Distance range in which the blur does not exceed a certain value

fbg

111

26

Aperture (DK: blænde)

• More aperture => better depth-of-field• Downside: less light enters => decrease exposure time =>

risk of blur due to motion

Depth-of-field

27

Field-of-view

• Field-of-view depends on size of chip and focal length• ”Fisheye” lens => small focal length and large field-of-view

28

Lighting in machine vision

29

Levels of Natural Light

[Burke]

30

31

32

33

34

Lighting• ”In machine vision lighting is more than 50%”

• Use controlled lighting!

• Avoid direct/indirect sunlight– Build a housing covering the

field-of-view of the camera

• Avoid highlights

35

Illumination Setups

Directedillumination

Rear illumination

Light field illumination

Diffuse illumination

36

Backlighting

37

38

39

40

41

42

• Viewpoint invariant

• High reflectance in illumination direction

Spherical Marker

43

Infrared Illumination

Visually Opaque IR pass filter

• Fast segmentation by adaptive threshold

• Robust

44

What to remember

45

Exercises (1/3)

• Given a 512 x 512 x 8bit image. How many different images can be made?

• Given a 512 x 512 x 8bit image. How is the memory size reduced when you: – Decrease the grayscale resolution repeatedly by 2– Decrease the x-size and y-size of the image

repeatedly by 2• What do you want to learn in this class?

46

Exercises (2/3)

• Describe the following concepts and provide examples of their usages:– Classification, Quality control, Pose estimation, Bin-

picking

• What is a CCD-chip and how does it operate?• What is Depth-of-field (DK:dybteskarphed)?• Pros and cons of back-lighting?

47

Exercises (3/3)

• Show that the following is true for a thin lens:• Mick is 2m tall and standing 5m from a camera. The

camera’s focal length is 5mm.– At which distance from the lens is Mick in focus?

• How tall (in mm) will Mick be on the CCD-chip?• How tall (in pixels) will Mick be on the CCD-chip?

– The camera has a 1/2” CCD chip

– The camera image has a size of: 480x640 pixels

• What is field-of-view of the camera?

fbg

111

48

Working with images….

• Image manipulation– Simple operations, e.g., scale image

• Image processing– Improve the image, e.g., remove noise

• Image analysis– Analyze the image, e.g., find the person in the image

• Machine vision– Industry, e.g., Quality control, Robot control

• Computer vision– Everything: multiple cameras, video-processing, etc.

49

Image file types

• image.jpg, image.tif, image.gif, image.png, image.ppm, …. • Raw:

– No data is lost – Header + data (234 235 32 21…)– For example: image.pgm – The file can be viewed

• Lossless compression:– No data is lost, but the file cannot be viewed– For example: image.gif

• Lossy compression:– Better compression– Some data is lost (optimized from the HVS’ point of view)– The file cannot be viewed– For example: image.jpg

50

Image file types

• Normally you don’t care about the file type– The application will take care of it for you:– For example: rotate

• Application– image.x => raw– Rotate the raw image– Rotated raw => rotated_image.x

• But to write your own programs from scratch the images need to be in the raw format (without a header).

51

Video Standards

• RS-170– NTSC

• CCIR– PAL, SECAM

• USB, Firewire• Composite• S-video• RGB

52

Applications

53

Digital Image

• Why digital ?• Before 1920: Image transmission from USA to

Europe: more than a week: by ship!• Early 1920s: Bartline cable picture transmission

system: Transmission in three hours!

• Transmission via telegraph/wire, radio signals for newspapers

54

Small Progress

• Small progress in digital imaging until 1964

• Jet Propulsion Lab (JPL) in Pasadena, CA– Transmission and correction of lunar images

from Ranger 7.

• Not so good quality so the images had to be processed before they could be viewed

• Since then many applications…

55

Examples: Image Correction

• Needed when image data is erroneous: – Bad transmission

– Bits are missing: Salt & Pepper Noise

56

Image Deblurring: Motion Blur

• Can be used when a camera or object is moved during exposure

57

Deblurring

• Can be used when the camera was not focused properly!!

58

Image manipulation

• Image improvement, e.g. too dark image

• Rotate + scale

59

Medical Image Processing

• Image Processing is widely used

• E.g. Analysis of microscopic images

60

Medical Image Processing

• MR/CT Imaging of a human body

• Use for Brain Surgery

61

Biometrics

• Recognizing/verifying the identity of a person by analyzing one or more characteristics of the human body

• Characteristics:– Fingerprint, eye (retina, iris), ear, face, heat profile,

shape (3D face, hand), motion (gait, writing), …

• Applications: – Verifying: Access control (bio-passports)

– Recognizing: Surveillance: 9/11

62

Chroma keying

63

Analysis of Sport Motions

• Here: Analysis of motion of Sarah Hughes• 3D Tracking of body parts• Motion interpretation• Action recognition

64

Motion Capture

Andy Serkis

• Special effects– Advertising

– Movies

65

Motion Capture

66

Fundamental Steps in Computer Vision

Knowledge baseProblemdomain Image

acquisition

Preprocessing

SegmentationRepresentationand description

Recognitionand Interpretation

Result

Point 1: 22,33Point 2: 24, 39…..

Actor sitting

67

Shrinking the pinhole

[from Hecht 1987]

68

Shrinking the pinhole even more

[from Hecht 1987]Diffraction causes blur

69

Basic Optics

ReflectionRefraction & DispersionDiffraction

70

Minimum Object Distance

fb

bfMOD

fMODb

max

max

max

111