Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495...

52
Computer Vision Computer Vision Chapter 6 Chapter 6 Color Color

Transcript of Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495...

Page 1: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Computer VisionComputer Vision

Chapter 6Chapter 6

ColorColor

Page 2: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Visible lightVisible light

ColorColor WavelengthWavelength

violet violet 380–450 nm380–450 nm

blue blue 450–495 nm450–495 nm

green green 495–570 nm495–570 nm

yellow yellow 570–590 nm570–590 nm

orange orange 590–620 nm590–620 nm

redred 620–750 nm620–750 nm(high frequency/shorter wavelength)(high frequency/shorter wavelength)

Page 3: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

White lightWhite light

So what is “white” light?So what is “white” light?

White light is composed of approx. White light is composed of approx. equal energy in all wavelengths of equal energy in all wavelengths of the visible spectrum.the visible spectrum.

Page 4: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Perception of colorPerception of color

A blue surface appears blue when A blue surface appears blue when illuminated by white light.illuminated by white light.

The same surface appears violet The same surface appears violet when illuminated by red light.when illuminated by red light.

Page 5: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Perception of colorPerception of color

Dependent on:Dependent on:

– Spectrum of light sourceSpectrum of light source

– Characteristics of illuminated objectCharacteristics of illuminated object

– Spectral sensitivity of the sensorSpectral sensitivity of the sensor

Page 6: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Perception of colorPerception of color

Additional factors:Additional factors:

– Surface is specular or matte (lambertian)Surface is specular or matte (lambertian)

– Distance of surface from lightDistance of surface from light

– Orientation of surface and lightOrientation of surface and light

Page 7: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.
Page 8: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Surface normalsSurface normals

Say we have 3 points, p1, p2, and p3 on our Say we have 3 points, p1, p2, and p3 on our surface element.surface element.

Form two vectors, U=p2-p1 V=p3-p1.Form two vectors, U=p2-p1 V=p3-p1. Then N=UxV is the surface normal (cross Then N=UxV is the surface normal (cross

product).product).– Nx = UyVz – UzVyNx = UyVz – UzVy– Ny = UzVx – UxVzNy = UzVx – UxVz– Nz = UxVy – UyVxNz = UxVy – UyVx

Analog in 2D is UxVy – UyVxAnalog in 2D is UxVy – UyVx Note regarding length of N; note regarding Note regarding length of N; note regarding

direction of N.direction of N.

Page 9: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Sensitivity of receptorsSensitivity of receptors

Page 10: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Types of imagesTypes of images

Binary image = range of I is in {0,1}Binary image = range of I is in {0,1} Gray-scale image = range of I is in {0,1,Gray-scale image = range of I is in {0,1,

…,max}…,max} Multispectral image = vector-valuedMultispectral image = vector-valued

– Ex. Color image = range is in {<r,g,b> | Ex. Color image = range is in {<r,g,b> | (r,g,b) in {0,1,…,max} }, for 8 bits, max=255 (r,g,b) in {0,1,…,max} }, for 8 bits, max=255 (2(288-1)-1)

– <0,0,255> = ?<0,0,255> = ?– <255,0,0> = ?<255,0,0> = ?– <100,100,100> = ?<100,100,100> = ?

Page 11: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Representing colorRepresenting color

RGB (red green blue)RGB (red green blue)– Additive (to black: <0,0,0>)Additive (to black: <0,0,0>)

CMY (cyan magenta yellow)CMY (cyan magenta yellow)– Subtractive (from white: <0,0,0>); based Subtractive (from white: <0,0,0>); based

on printing inks on white paperon printing inks on white paper

– Often CMYOften CMYKK (K=key=black) because black (K=key=black) because black from CMY is poor.from CMY is poor.

HSI (hue saturation intensity)HSI (hue saturation intensity)– aka HSV (hue saturation value)aka HSV (hue saturation value)

Page 12: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Additive (left) vs. Additive (left) vs. subtractive (right)subtractive (right)

Page 13: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.
Page 14: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color cubeColor cube(note RGB values in (note RGB values in

[0.0..1.0])[0.0..1.0])

Page 15: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

color color cube vs. cube vs.

color color wheelwheel

Page 16: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Hue angle in [0..2Pi].Saturation in [0..1].Intensity in [0..255].

Interactive:http://www.had2know.com/technology/hsi-rgb-color-converter-equations.html

Page 17: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.
Page 18: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Other systemsOther systems

Luminance + 2 chromaticity valuesLuminance + 2 chromaticity values YIQYIQ

– luminanceluminance Y = 0.3 R + 0.59 G + 0.11 BY = 0.3 R + 0.59 G + 0.11 B

– red-cyanred-cyan I = 0.6 R – 0.28 G – 0.32 BI = 0.6 R – 0.28 G – 0.32 B

– magenta-greenmagenta-green Q = 0.21 R – 0.52 G + 0.31 BQ = 0.21 R – 0.52 G + 0.31 B

Used by color and black & white broadcast TV (before digital).Used by color and black & white broadcast TV (before digital).

Page 19: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Other systemsOther systems

Luminance + 2 chromaticity valuesLuminance + 2 chromaticity values YUVYUV

Y = 0.3 R + 0.59 G + 0.11 BY = 0.3 R + 0.59 G + 0.11 B

U = 0.493 (B – Y)U = 0.493 (B – Y)

V = 0.877 (R – Y)V = 0.877 (R – Y)

– Used by JPEG and MPEG (compresses Used by JPEG and MPEG (compresses better than RGB)better than RGB)

Page 20: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

COLOR HISTOGRAMSCOLOR HISTOGRAMS

Page 21: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

How can one construct a color How can one construct a color histogram?histogram?

Page 22: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

How can one construct a How can one construct a color histogram?color histogram?

– as 3D color cloudsas 3D color clouds

Page 23: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

How can one construct a How can one construct a color histogram?color histogram?

– as histograms of each as histograms of each individual componentindividual component

Page 24: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

Another method for constructing a color Another method for constructing a color histogram:histogram:– Given an RGB value, construct an integer i = Given an RGB value, construct an integer i =

RR22GG22BB22

where Rwhere R22 = the 2 high order bits of the R = the 2 high order bits of the R component (and similarly for G and B)component (and similarly for G and B)

222222222222 = 2 = 266 = 64 bins in histogram = 64 bins in histogram

Maps 3D RGB values into 1D.Maps 3D RGB values into 1D.

Page 25: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

64 bin color 64 bin color histogram histogram exampleexample

Page 26: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

64 bin color 64 bin color histogram histogram exampleexample

Page 27: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

Another method of constructing a color Another method of constructing a color histogram:histogram:1.1. Quantize RGB into 16 levels (4 bits for each Quantize RGB into 16 levels (4 bits for each

component, 2component, 244 = 16) = 16)

2.2. Create a histogram of red valuesCreate a histogram of red values

3.3. Create a histogram of green valuesCreate a histogram of green values

4.4. Create a histogram of blue valuesCreate a histogram of blue values

5.5. Concatenate these 3 histograms into one Concatenate these 3 histograms into one single 16x3 = 48 bin histogramsingle 16x3 = 48 bin histogram

– Also maps 3D RGB values into 1D.Also maps 3D RGB values into 1D.

Page 28: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

COMPARING HISTOGRAMSCOMPARING HISTOGRAMS

Page 29: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Comparing color histogramsComparing color histograms

Say we have a database of images and Say we have a database of images and their color histograms.their color histograms.– Given an image that is not in our database, Given an image that is not in our database,

we can calculate its histogram.we can calculate its histogram.– And see what histogram it best matches.And see what histogram it best matches.

But But howhow can we compare histograms? can we compare histograms?

Page 30: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Method 1: Comparing two (1D) Method 1: Comparing two (1D) histograms hhistograms hII and h and hMM

image model of histogram

histogram image

histogram thein bins ofnumber theis

,min,onintersecti1

M

I

K

jMIMI

h

h

K

where

jhjhhh

r.denominato always isnumerator :Note

,onintersecti,match

1

K

jM

MIMI

jh

hhhh

Note: match is not symmetric. I.e., match(a,b) <> match(b,a) for all a and b.

Page 31: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Method 2: Comparing two (1D) Method 2: Comparing two (1D) histograms hhistograms hII and h and hMM

1.1. Normalize (divide) each hNormalize (divide) each hII and h and hMM by by total number of pixels (to total number of pixels (to approximate probabilities).approximate probabilities).

2.2. Calculate distance d( hCalculate distance d( hII, h, hMM ) as 2 ) as 2 64- (or 48-) dimensional points.64- (or 48-) dimensional points.

How do we calculate the distance between How do we calculate the distance between 2 64- or 48-) dimensional points?2 64- or 48-) dimensional points?

Page 32: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Method 2: Comparing two (1D) Method 2: Comparing two (1D) histograms hhistograms hII and h and hMM cont’d. cont’d.

1.1. Normalize (divide) each hNormalize (divide) each hII and h and hMM by by total number of pixels (to total number of pixels (to approximate probabilities).approximate probabilities).

2.2. Calculate distance d( hCalculate distance d( hII, h, hMM ) as 2 ) as 2 64- (or 48-) dimensional points.64- (or 48-) dimensional points.

How do we calculate the distance between How do we calculate the distance between 2 64- or 48-) dimensional points?2 64- or 48-) dimensional points?

K

jMIMI jhjhhhd

1

2,

Page 33: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Method 3: Relative entropy (or Method 3: Relative entropy (or the Kullback Leibler distance)the Kullback Leibler distance)

from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006), p. 18.Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006), p. 18.

(all logs are base 2)

Page 34: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006)Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006)

Method 4: Mutual informationMethod 4: Mutual information

(all logs are base 2)

Page 35: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Method 4: Mutual information Method 4: Mutual information cont’d.cont’d.

How to calculate the joint distribution How to calculate the joint distribution or joint p.m.f., p(x,y):or joint p.m.f., p(x,y):

Page 36: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

COLOR SEGMENTATION COLOR SEGMENTATION EXAMPLEEXAMPLE

Page 37: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color segmentation Color segmentation exampleexample

Problem: Find (segment) face in workstation Problem: Find (segment) face in workstation camera image.camera image.

Approach:Approach:1.1. Training phase:Training phase:

Determine clusters from pixels from known faces Determine clusters from pixels from known faces using red-green values.using red-green values.

2.2. Classification phase:Classification phase: Given a pixel that we need to classify (as either face Given a pixel that we need to classify (as either face

or not), produce a labeled image which is then or not), produce a labeled image which is then filtered according to region size.filtered according to region size.

Page 38: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

cluster of face pixel colors

non face pixel colors

Page 39: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.
Page 40: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

END COLOR END COLOR SEGMENTATION EXAMPLESEGMENTATION EXAMPLE

Page 41: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color histogramsColor histograms

Yet another method is to map 3D Yet another method is to map 3D RGB values into a 3D histogram.RGB values into a 3D histogram.

Note that K-means can also be easily Note that K-means can also be easily applied to any of these 3 color applied to any of these 3 color histogram methods.histogram methods.

Page 42: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

HomeworkHomework

Exercises 6.6, 6.8, and 6.9.Exercises 6.6, 6.8, and 6.9.

Page 43: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

HomeworkHomework

Exercise 6.6. Using Algorithm 6.1 Exercise 6.6. Using Algorithm 6.1 (procedure RGB_to_HSI),(procedure RGB_to_HSI),a)a) convert the RGB value (100,150,200) convert the RGB value (100,150,200)

into an HSI value and,into an HSI value and,

b)b) convert the RGB value (0.0, 1.0, 0.0) to convert the RGB value (0.0, 1.0, 0.0) to HSI.HSI.

Page 44: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

HomeworkHomework

Exercise 6.8. Suppose a color Exercise 6.8. Suppose a color camera encodes a given pixel in RGB camera encodes a given pixel in RGB as (200, 50, 100), where 255 is the as (200, 50, 100), where 255 is the highest (most energy) value.highest (most energy) value.a)a) What should be the equivalent triple in What should be the equivalent triple in

the HSI system?the HSI system?

b)b) What should be the equivalent triple in What should be the equivalent triple in the YIQ system?the YIQ system?

Page 45: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

HomeworkHomework

Exercise 6.9. Is the transformation Exercise 6.9. Is the transformation from RGB to YIQ invertible? Is so, from RGB to YIQ invertible? Is so, compute the inverse.compute the inverse.

Page 46: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

Color is a key feature in identifying the real tiger.

Page 47: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

HSI is a natural representation for increased or decreasing the amount of color in an image.

Hue angle in [0..2Pi].Saturation in [0..1].Intensity in [0..255].

original increased saturation decreased saturation

Page 48: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

image after segmentation and labeling (coloring)

Page 49: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

STORING IMAGES IN FILESTORING IMAGES IN FILE

Page 50: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

P2 = ASCII P2 = ASCII graygray PGM PGM

width(cols)

height(rows)

max data value ASCIIdata

comments

{header

Page 51: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

P3 = ASCII P3 = ASCII colorcolor PPM PPM

Each pixel’s color is represented as Each pixel’s color is represented as <red,green,blue>.<red,green,blue>.

P3P3w hw hmaxvalmaxval

vv11rr v v11gg

v v11bb v v22rr

v v22gg v v22bb

. . .. . .

. . . . . . v. . . . . . vw*hw*hrr v vw*hw*hgg

v vw*hw*hbb

Page 52: Computer Vision Chapter 6 Color. Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm.

2D Images Stored as 1D 2D Images Stored as 1D ArraysArrays

Say our 2D color (RGB) image is Say our 2D color (RGB) image is stored as a 1D array, I.stored as a 1D array, I.

How do we calculate the offset to a How do we calculate the offset to a particular element given a particular particular element given a particular row and column?row and column?

– First, do you recall how it’s done for a 2D First, do you recall how it’s done for a 2D gray image?gray image?