Instructor: Mircea Nicolescu Lecture 4 CS 485 / 685 Computer Vision Alternative Color Spaces Various other color representations can be computed from RGB This can be done for: Decorrelating the color channels: principal components Bringing color information to the fore: Hue, saturation and brightness Perceptual uniformity: CIELuv, CIELab, 2 RGB (CIE), RnGnBn (TV - N ational T elevision S tandard C ommittee ) XYZ (CIE) UVW (UCS de la CIE), U*V*W* (UCS modified by the CIE) YUV, YIQ, YCbCr YDbDr DSH, HSV, HLS, IHS Munsel color space (cylindrical representation) CIELuv CIELab SMPTE-C RGB YES (Xerox) Kodak Photo CD, YCC, YPbPr,... Alternative Color Spaces 3 Red Green Blue T -1 Processing Processing Strategy T Red Green Blue 4 Color Transformation - Examples 5 Skin Color RGBrg r g 6 Skin Detection M. Jones and J. Rehg, Statistical Color Models with Application to Skin Detection, International Journal of Computer Vision, 2002.Statistical Color Models with Application to Skin Detection 7 8 Image File Formats Many image formats adhere to the simple model shown below (line by line, no breaks between lines) The header contains at least the width and height of the image Most headers begin with a signature or magic number short sequence of bytes identifying the file format 9 Common Image File Formats GIF (Graphic Interchange Format) PNG (Portable Network Graphics) JPEG (Joint Photographic Experts Group) TIFF (Tagged Image File Format) PGM (Portable Gray Map) FITS (Flexible Image Transport System) 10 PGM Format A popular format for grayscale images (8 bits/pixel) Closely-related formats are: PBM (Portable Bitmap) for binary images (1 bit/pixel) PPM (Portable Pixelmap) for color images (24 bits/pixel) ASCII or binary (raw) storage: ASCII: Binary: Image Filtering 11 f(x,y)g(x,y) filtering 12 Early Vision One Image Classification of image operations Spatial domain methods Point Processing Transformations Area/Mask Processing Transformations Frame Processing Transformations Geometric Transformations Frequency domain methods Image Filtering Methods Spatial Domain Frequency Domain (i.e., uses Fourier Transform) 13 Spatial Domain Methods f(x,y) g(x,y) f(x,y) 14 15 Point Processing Methods The most primitive, yet essential, image processing operations Intensity transformations that convert an old pixel into a new pixel based on some predefined function Operate on a pixel based solely on that pixels value Used primarily for image enhancement Transformation function 16 Point Processing Methods Identity transformationNegative transformation 17 Point Processing Methods Contrast stretching / compression Stretch gray-level ranges where we desire more information 18 Point Processing Methods Thresholding 19 Point Processing Methods Intensity-level slicing Highlight a specific range of gray-levels only Similar to double thresholding 20 Point Processing Methods Non-linear transformations We may use any function, provided that is gives a one-to-one or many-to-one (i.e., single-valued) mapping. Logarithmic Useful for enhancing details in the darker regions of the image at the expense of detail in the brighter regions. 21 Point Processing Methods Exponential The effect is the reverse of that obtained with logarithmic mapping. 22 Point Processing Methods Histogram equalization Low contrast images are usually mostly dark, mostly bright, or mostly gray. Good contrast images exhibit a wide range of pixel values (i.e., no single gray level dominates the image). 23 Point Processing Methods Histogram equalization The histogram of an image (i.e., a plot of the gray-level frequencies) provides important information regarding the contrast of an image Histogram clustered at the low end: dark image Histogram clustered at the high end: bright image Histogram with a small spread: low contrast image Histogram with wide spread: high contrast image 24 Point Processing Methods Histogram equalization is a transformation that stretches the contrast by redistributing the gray-level values uniformly is fully automatic compared to other contrast stretching techniques 25 Point Processing Methods Histogram equalization Area Processing Methods Need to define: (1)Area shape and size (2) Operation output image 26 Area Shape and Size Area shape is typically defined using a rectangular mask Area size is determined by mask size e.g., 3x3, 5x5, 7x7, Mask size is an important parameter 27 Operation Typically, linear combinations of pixel values e.g., weigh pixel values and add them together Different results can be obtained using different weights e.g., smoothing, sharpening, edge detection mask 28 Example MaskModified image data Local image neighborhood 29 Common Linear Operations Correlation Convolution 30 Correlation A filtered image is generated as the center of the mask visits every pixel in the input image g(i,j) h(i,j) f(i,j) filtered image nxn mask 31 Handling Pixels Close to Boundaries pad with zeroes or .0 wrap around 32 Correlation Example 33 Geometric Interpretation of Correlation Suppose x and y are two n-dimensional vectors: The dot product of x with y is defined as: cos() measures the similarity between x and y using vector notation: x y 34 Correlation generalizes the notion of dot product: Normalized correlation (divide by lengths) Geometric Interpretation of Correlation 35 Normalized Correlation = mask Measure the similarity between images or parts of images 36 Application: TV Remote Control 37 Application: TV Remote Control 38 Application: TV Remote Control 39 Application: TV Remote Control 40 Application: TV Remote Control 41 Application: TV Remote Control 42 Normalized Correlation Traditional correlation cannot handle changes due to: size orientation shape (e.g., deformable objects) ? 43 Convolution Same as correlation, except that the mask is flipped, both horizontally and vertically h * f = f * h Notation: For symmetric masks (h(i,j)=h(-i,-j)), convolution is equivalent to correlation 44 Correlation/Convolution Examples Correlation: Convolution: 45 How to Choose the Mask Weights? Depends on the application Usually by sampling certain functions and their derivatives Gaussian 1 st derivative of Gaussian 2 nd derivative of Gaussian Good for image smoothing Good for image sharpening 46 Normalization of Mask Weights Sum of weights affects overall intensity of output image Positive weights Normalize them so that they sum to one Both positive and negative weights Should sum to zero (but not always) 1/91/16 47