CH01 Introduction (also see Machine Vision PPT slides)CH02 IP Fundamentals

Check this link: http://www.gnu.org/software/octave/download.html1. Sensing and bio-inspiration2. What is an image3. Colors and color depth: binary, intensity, indexed, RGB, etc. 4. Image coordinates5. reading images

a. im0c = imread('house01.jpg');b. explore im0c buffersc. [r c] = size( im0c );d. ndims( im0c );e. imshow( im0c );f. im0g = rgb2gray( im0g ); g. imshow( im0g ) % 256 intensity levels by defaulth. imshow( im0g, 32 ) % 32 intensity levelsi. imshow( im0g, [32 128] )% min/max intensity range j. imshow( im0g, [] ) % min/max intensity range = min/max(im0g)k. pixval % command after imshowl. figure and subplot % for multiple image displaym. imviewn. imfinfoo. imwrite( im0g, 'filename.format', 'quality', 0~100);

6. Image Basic Types and conversionsa. Colored: im0cb. Intensity: im0gc. Pseudo color: im0p = im0g – 60; imshow(im0i, jet)d. Indexed: im0i = rgb2ind( im0c, jet(256) );e. Binary: im0b = im2bw( im0g, 0.5);

7. Image plotting (2D and 3D)a. Plot(im0g(:,1)) % plotting a columnb. plot(im0g(1,:)) % plotting a rowc. surf( double( im0g ) );d. try a face imagee. im1g = rand(240, 320);f. im1c(:,:,1) = rand(240, 320); 2, and 3

8. Setting and getting sub-imagesa. x = im0g(20:100, 40:200);b. x(1:20, 1:20) = 0;c. try it with colored images: im0c

9. Images in a programa. A simple program that reads rgb image converts it to intensity and shows

itb. May be another example …

10. HW Assignment (2) : Write a MatLab program thata. Accepts an rgb image file name, colormap idx, threshold, save filename

b. Design your own color map of 256 colors and make it available as one of the options for users to select.

c. Verifies the type of the image is rgb (3 buffs)d. Converts img into intensity, indexed (according to passed idx), binary

(according to passed threshold)e. Writes your initials (two letters) in red pixels on the rgb version at

the upper left corner with a 10% scale of the image.f. In the intensity img Find all pixels within the range [0-50] or [200-

255] and turn them off. g. Displays all 4 images versions on one figureh. Turn in a hard copy of your code and screen shots of your output.i. Grad students only : one page paper on imaging technologies. To be

presented next class.

CH03 Intensity Transformation and Spatial Filtering

1. Intensity Transformation Functionsa. Imadjust(im0g, [li hi], [lo ho], gamma); by default gamma=1;

iout=iineγiin

x = 0:.01:1;y = x .* exp( a * x ); plot(x, y); experiment with a=0, <0, >0.

Imadjust(im0g, [0 1], [1 0]); inversion or complement equivalent to imcomplement(im0g);

It can be applied when a range of intensities (could represent an object) is desired to be highlighted:Imadjust(im0g, [0.5 0.75], [0 1]).

Imadjust(im0g, [], [], gamma); adjusts the brightness lower or higher. Experiment with gamma = .01, .1, 1, 10; plot linear data function

Demo some image examples

b. Logarithmic and Contrast-Stretching Transformation 0.1 * log( double(im0g) + 1); the double is to insure a real img

datatype

iout=c log(1+iin) The shape is similar to gamma < 0; plot linear data function Demo some image examplesc. Contrast Stretching function

x = 0:.01:1;m = .5; E=5; y = 1./(1+(m./(double(x)+eps)).^E); plot(x,y)

iout=1

1+(m / iin )E, m: see figure, E: slope coefficient

Plot linear data function Demo some image examples

2. Histogram Processing What histogram and image histogram? Demo some examples linear and image.

x = rand(100,1); bar( hist(x, 10)); % nBins = 10.read an image to intensity imhist( im0g );Question: how do we relate histogram info to image scene.Can save hist and plot it any way we like:h = imhist( im0g ); area(h);

Histogram EqualizationThe idea of histogram equalization is as follows:Compute h = imhist(im0g), normalize h as hn = h ./ numel(im0g), compute the cumulative sum of hn as cumsum(hn) and use the resulting function as the intensity transformation function for im0g.Demo and plot the trans function.The procedure is implemented in a native function called histeq an example: im1g=.25*im0g;im1h=histeq(im1g,256);subplot(2,2,1);imshow(im1g);subplot(2,2,2);imshow(im1h);subplot(2,2,3);imhist(im1g);subplot(2,2,4);imhist(im1h);

Histogram Matching

The function histeq supports another method for equalization. We can specify a histogram (PDF not CDF) that the histeq will follow to enforce its distribution on the passed image.Use the function: histeqMatch: discuss the functionExample: h = histeqMatch( [.25 77 11; .25 128 23;.50 200 5], 256);area(h)

3. Spatial Filtering Noise:

uniform PDF: imnoise(im0g,'speckle',.1 ) % .1=varianceBinary: imnoise(im0g,'salt & pepper', .1 ) % .1 = noise densityGaussian: imnoise(im0g,'gaussian',.1, .1 ) % mean, variance

Convolution: linear and non-linear

1/9 1/9 1/91/9 1/9 1/91/9 1/9 1/9

0.0 0.5 0.4 0.6 0.7 0.4 0.1 0.9 0.8 0.2 0.3 0.70.3 0.4 0.7 0.8 0.2 0.1 0.5 0.1 0.6 0.2 0.1 0.80.2 0.6 0.6 0.5 0.4 0.9 0.5 0.9 0.8 0.9 0.8 0.80.6 0.8 0.7 0.8 0.1 0.8 0.1 0.8 0.1 0.7 0.4 0.7 0.7

0.1 0.3 0.0 0.1 0.7 0.8 0.5 0.9 0.1 0.7 0.3 0.30.3 1.0 0.4 0.2 0.9 0.2 0.8 0.5 0.9 0.6 0.3 0.10.0 0.4 0.3 0.9 0.8 0.1 0.8 0.5 0.3 0.2 0.3 0.60.0 0.9 0.3 0.6 0.2 0.2 0.7 0.8 0.9 0.2 0.4 0.50.6 0.7 0.1 0.4 1.0 0.3 0.3 0.5 0.6 0.1 0.8 0.60.3 0.8 0.5 0.2 0.3 0.2 0.9 0.2 0.1 0.6 0.7 0.50.4 0.6 0.5 0.8 0.5 0.5 0.4 0.9 0.4 0.5 0.7 0.00.6 0.5 0.2 0.7 0.9 1.0 0.8 0.9 0.6 0.5 0.9 0.30.4 0.9 0.5 0.0 0.3 0.8 0.2 0.3 0.7 1.0 0.6 0.41.0 0.2 0.4 0.1 0.2 0.2 0.1 0.8 0.1 0.3 0.6 0.7

Convert image data to normalized double:im0g = double(im0g) ./ double(max(im0g(:)));

Generate a kernel: k = ones(3) .* 1/9; Convolute an image with k: conv2( im0g , k ); The size of the kernel is arbitrary, typically 3x3 fspecial: filter special

k = fspecial('filterType'); imfilter(im0g, k) 'average' averaging filter, [r c] or size'disk' circular averaging filter, radius

'gaussian' Gaussian lowpass filter, [r c] or size 'log' logarithmic filter, [r c] or size 'unsharp' unsharp contrast enhancement filter, alpha (0~1)

medfilt2(im0g, [r c]) surf: some examples on the kernel shapes

4. HW Assignment (3) Write a matlab program that:1) Accepts an rgb image2) Prompts the user to add noise using type, density, mean and variance.3) Add noise to an intensity (gray-level) copy of the image4) Filters the image using 4 different types of image filters5) Allows the users to enter their customized kernel6) Displays rgb img, intensity img, all filtered imgs in one figure7) Label the different images appropriately on the figure 8) Graduate students only: write a matlab function that builds customized

Gaussian mixture PDF as explained in image equalization via histogram matching section.

CH04 Intro to MatLab GUIDE & Image Arithmetic Logic Operations (ALO)

1. MatLab GUIDE Blank gui Panel: property inspector, set units to pixels Axes: units = pixels Create 4 axes 240x320, in1, in2, out1, out2. Menu editor: tools->menu editor, File->Load Image: callback, hObject,

handles? Global variables as a single structure Load image callback:

function Load_Image_Callback(hObject, eventdata, handles)global gv;[fname pname] = uigetfile('*.*');gv.im1 = imread( strcat(pname,fname) );axes(handles.axes1);im = image(gv.im1);set(im, 'HitTest', 'off');set(handles.axes1, 'XTick', [], 'YTick', []);

Main Menu item: color transformation->rgb2grayglobal gv;gv.im2 = rgb2gray(gv.im1);axes(handles.axes2);colormap( gray(256) );im = image(gv.im2);set(im, 'HitTest', 'off');set(handles.axes2, 'XTick', [], 'YTick', []);

inputdlg:hdrs = {'x-val', 'y-val', 'z-val'};ttl = 'example of message box';numl = 1;def = {'11', '22', '33'};op.Resize = 'ON';

usrin = inputdlg(hdrs, ttl, numl, def, op);if ~isempty(usrin)

x = str2num(usrin{1}); y = str2num(usrin{2}); z = str2num(usrin{3}); disp((x+y+z)/3.0);else disp('canceled ...');end

x= 11, y=99; msg = sprintf('x=%.3f, y=%d', x,y)standard icon: msgbox(msg,'this is a test msgbox','warn');'icon' is 'none', 'error', 'help', 'warn', or 'custom'. The default is 'none'.Custom icon: x = imread(‘imgfilename.jpg’);msgbox(msg,'this is a custom msgbox','custom', x, jet(256));

Main Menu item: color maps->sub items: all mapsPopup-menu implementation for color maps and edit1 (or inputdlg) text for map size

global gv;

contents = get(hObject,'String');op = contents{get(hObject,'Value')};p0 = str2num( get(handles.colorMap_edit1, 'String') );

switch op case 'gray' gv.colorMap = gray(p0); case 'jet' gv.colorMap = jet(p0);

need a function to update the four images. Call it imgupdate. This function should redraw all images with new color map. Colormap command should be in the imageupdate after the image() or imshow() command.

2. Threshold: slider3. Noise: popup-menu. See the noise option above.4. Binary Arithmetic & Logical Operations (ALO)5. Gray scale ALO: im2double(gray) to be able to max-normalize ALO results6. RGB ALO: im2double(rgb) to be able to max-normalize ALO results7. Masking: bin-bin, bin-rgb (loading a 3D bin as a mask), bin-rgb (run time

conversion of bin mask to 3D)8. Main menu item: image ALO, implement all for gray and binary.9. Button group with radio buttons: see example in test.gif and .m

5. HW Assignment(4)& Mid-Term 1)Implement all image processing tools demonstrated above in this class in MatLab GUIDE.Your interface should include at least the following features:1- load/save images2- Image type transformation: gray, indexed, colormaps3- Thresholding4- Noise manipulation: uniform, Gaussian, salt&pepper5- Filtering: mean, median, Gaussian

6- Image ALO

2) Graduate students only: None.

% a demo menu program function [] = test00()

op = -1;

while op ~= 0 disp(' [0] Quit'); disp(' [1] load image'); op = input(' enter your option '); switch op case 0 return; case 1 imshow(rand(100), jet(256)); end end

ALO Handlercontents = get(hObject,'String');op = contents{get(hObject,'Value')};switch op case 'a+b' gv.im{3} = imadd( gv.im{2}, gv.im{4}); case 'a-b' gv.im{3} = imsubtract( gv.im{2}, gv.im{4}); % works as long as the result is positive unless intended otherwise. use imabsdiff below case '|a-b|' gv.im{3} = imabsdiff( gv.im{2}, gv.im{4}); case 'axb' gv.im{3} = immultiply(gv.im{2}, gv.im{4}); case 'a/b' gv.im{3} = imdivide( gv.im{2}, gv.im{4} + eps); % epsilon to avoid devide by zero case 'a&b' gv.im{3} = double( gv.im{2} & gv.im{4} ); % make all logical results doubles case 'a|b' gv.im{3} = double(gv.im{2} | gv.im{4}); case '^' gv.im{3} = double( xor(gv.im{2}, gv.im{4}) ); case '!a' gv.im{3} = imcomplement( gv.im{2} ); case '!b' gv.im{3} = imcomplement( gv.im{4} );

case 'fxa' f = str2num( get(handles.arithmetic_edit1, 'String') ); gv.im{3} = f * gv.im{2}; case 'fxb' f = str2num( get(handles.arithmetic_edit1, 'String') ); gv.im{3} = f * gv.im{4}; case 'a/max' % nothing to be done, see the call for normImg belowend

gv.im{3} = normImg(gv.im{3});

updateImgs();

CH05 IMAGE SEGMENTATION

1. What is image segmentation?Partitioning/Clustering the image space into a finite number of homogeneous regions that share identical or similar features based on criteria including: intensity, color and texture. Segmentation aims at reducing dimensionality of the original image and mapping its space to a significantly smaller space to facilitate further advanced image analysis and understanding.

2. Segmentation Methods Thresholding: given a known intensity region, we can devise a band-pass

filter that can turn off the entire image but the target region.>> example: see image file ‘obj05.jpg’ and try to segment the blue regions.>> extracting original object by maskingThresholding method is limited due to the loss of information in color depth. Furthermore, the output is binary regions.

Histogram: binary low-pass, binary high-pass, binary band-pass

Color space quantization: (gray and rgb)Reducing the intensity/color depth. This method will collapse intervals of intensity/color to the quantized limits and similar colors are merged.

% --- Executes on selection change in colorSpace_listbox11.function colorSpace_listbox11_Callback(hObject, eventdata, handles)% hObject handle to colorSpace_listbox11 (see GCBO)% eventdata reserved - to be defined in a future version of MATLAB% handles structure with handles and user data (see GUIDATA)

% Hints: contents = get(hObject,'String') returns colorSpace_listbox11 contents as cell array% contents{get(hObject,'Value')} returns selected item from colorSpace_listbox11global gv;contents = get(hObject,'String');op = contents{get(hObject,'Value')};

p0 = str2num( get( handles.colorSpace_edit1, 'String' ) );

switch op case 'RGB_Q' gv.im{2} = im2double(grayslice(gv.im{1}, p0)); case 'Gray_Q' gv.im{3} = im2double(grayslice(gv.im{2}, p0)); gv.im{3} = normImg(gv.im{3}); case 'Gray' gv.im{2} = rgb2gray( gv.im{1} ); case 'HSV' gv.im{2} = rgb2hsv( gv.im{1} ); case 'Ind' gv.im{2} = rgb2ind( gv.im{1} ); case 'ntsc' gv.im{2} = rgb2ntsc( gv.im{1} ); case 'YcBCr' gv.im{2} = rgb2ycbcr( gv.im{1} );end

gv.im{2} = normImg(gv.im{2});

updateImgs();

Color Filters: RGB vs. HSI(V,L)

>> Demo color selector with RGB vs. HIS. Set ‘I’ to white and then black, see how RGB is color blind when in this case when you move the cursor over the color space vs. HSI model behavior.

>> Demo HSV-based filter component

Skin Segmentation and masking nametorgb( colorname ); rgb2name( [ r g b ] ); text2speech tts( ‘hello’);

contents = get(hObject,'String');op = contents{get(hObject,'Value')};

switch op case 'name2rgb' % name2rgb name = get(handles.colorname_edit1, 'String'); idx = find( strncmpi( name, gv.nameList, length(name))); rgb = gv.rgbList(idx, :); set(handles.colorname_edit1, 'String', num2str([rgb(1) rgb(2) rgb(3)])); set( handles.colorNamePnl, 'BackgroundColor', rgb(1,:));

case 'rgb2name' % rgb2name rgb = str2num( get(handles.colorname_edit1, 'String') ); d = dist(rgb, gv.rgbList'); idx = find(d==min(d));

name = gv.nameList{ idx(1) }; set(handles.colorname_edit1, 'String', name); set( handles.colorNamePnl, 'BackgroundColor', rgb(1,:));

end

Clustering: kmeans (3D vs. 5D)>> function and demo

% by: Aaron Rababaah% kmeans an image based on 5D (R,G,B,x,y)% x = (double) rgb image% k = number of clusters% org = map to original colors [boolean]% is5d = (boolean) include spatial coords to rgb = 5 dim

function [xo] = im5Dkmeans(x,k,is5d,org)[h w] = size(x(:,:,1));if is5d x = im25d(x); x = reshape(x,h*w,5);else x = reshape(x,h*w,3);end

[idx cen] = kmeans(x,k,'distance','sqEuclidean','Replicates',3);

if org == 0 xo = reshape(idx,h,w);else L = length(idx); r = zeros(L,1); g = zeros(L,1); b = zeros(L,1);

for i=1:k ids = find(idx==i); r(ids) = cen(i,1); g(ids) = cen(i,2); b(ids) = cen(i,3); end

r = reshape(r,h,w); g = reshape(g,h,w); b = reshape(b,h,w);

xo(:,:,1) = r; xo(:,:,2) = g; xo(:,:,3) = b;end

HW Assignment(5)1. Implement Band-pass threshold, Color quantization and HSV-based

segmentation as demonstrated in class in your GUI2. (Grads only): find a research paper that addresses any segmentation related

subjects, read it and report a half-page summary on it. Be ready to share your thoughts with the class.

CH06 REGION PROCESSING

Image Morphology Connected Components Analysis (CCA)

After the foreground image is computed, the connected component analysis (CCA) is applied to

I tF (x , y )

for foreground segmentation into separate regions of interest each of which represents

a TOI Ti. The goal of CCA is to label each physically connected pixels as a single region with

one unique label. So other higher processing algorithms can readily identify the separated

regions by their unique labels. The algorithm of CCA is illustrated in an example in figure 2.18,

and the pseudo-code is given below

Note that: the Neighbor pixels are shown in the figure below:

i-1,j-1 i-1,j i-1,j+1

I,j-1 I,j

Red: the current pixel, yellow: the neighboring pixels considered for labels. These neighbours

are already processed if scanning is done leftright and topbottom

ALGORITHM CCA( I tF (x , y )

)

{

// 1st pass

For each pixel in I tF (x , y )

{

If I tF (x , y )

0{

Nk = getNeighbors(x,y);

If ALL Nk = 0

I tF (x , y )

= newlabel;

Else

{

I tF (x , y )

= minlabel( Nk );

Store equivalentLabels in Nk;

}

}

}

// 2nd pass

For each pixel in I tF (x , y )

{

If I tF (x , y )

0{

I tF (x , y )

= minLabel(equivalentLabels);

}

}

}

Figure 2.18 Left: the foreground image I tF (x , y )after background subtraction. Right: after CCA is

applied, I tF (x , y ) is segmented into N connected components.

1) Build the props string:gv.propsOps = { 'Area' 'Centroid' 'Extrema' ... 'BoundingBox' 'EquivDiameter' 'SubarrayIdx' ... 'Image' 'Solidity' 'MajorAxisLength' ... 'PixelList' 'Extent' 'MinorAxisLength' ... 'PixelIdxList' 'Orientation' 'FilledArea' ... 'Eccentricity'};

2) Call bwlabeln:case 'Do CCA' [gv.im{3} gv.ncca] = bwlabeln( gv.im{2} ); gv.im{3} = normImg( gv.im{3}); % if you decide to norm img

3) Compute and save props

case 'Props' gv.propsMat = []; gv.props = regionprops( gv.ncca * gv.im{3}, gv.propsOps ); % multiplying by ncca since the regprop function requires an int img gv.propsMat(1,:) = [gv.props.Area]; gv.propsMat(2,:) = [gv.props.EquivDiameter]; gv.propsMat(3,:) = [gv.props.Solidity]; gv.propsMat(4,:) = [gv.props.MajorAxisLength]; gv.propsMat(5,:) = [gv.props.Extent]; gv.propsMat(6,:) = [gv.props.MinorAxisLength]; gv.propsMat(7,:) = [gv.props.Orientation]; gv.propsMat(8,:) = [gv.props.FilledArea]; gv.propsMat(9,:) = [gv.props.Eccentricity];

for i=1:gv.ncca % needed for filtering gv.props(i).id = i; gv.props(i).alive = 1; % all objs are initially are alive end

4) Compute boundaries and shape signatures

case 'Boundary pol_Sig' % polar coords boundary signature (1D array) gv.propsSig = {gv.props.PixelList}; % shape signature for i=1:gv.ncca gv.propsSig{i} = signature(gv.propsSig{i}); end

case 'Plot Sig' figure; [h w] = screen2wins( gv.ncca ); for i=1:gv.ncca subplot(h,w,i); plot(gv.propsSig{i}); % p1 = Obj ID (1-ncca) title(['Shape Signature of Object: ' num2str(i) ] ); end

%----------------------------------------------------% this function is needed to compute the proper subplots

% splits the screen into subwins according to the the ratio of the monitor% H = 4/3 W% By: Aaron Rababaah

function [h w] = screen2wins( n ) h = ceil(sqrt(n * 3/4)); w = ceil( n / h );

5) Draw IDs case 'Draw ID' axes(gv.ax{3}); for i=1:gv.ncca p = gv.props(i).BoundingBox; text(p(1)+p(3)/2, p(2)+p(4)/2, num2str(i), 'Color', 'white'); end

6) Prints case 'Print_Obj' % all props per obj for i=1:gv.ncca disp( ['-------------( Object ' num2str(i) ' )-------------']); disp( gv.props(i) ); end case 'Print_Prop' % all objs per prop

disp( ['-------------( Area )-------------']); gv.propsMat(1,:)' disp( ['-------------( EquivDiameter )-------------']); gv.propsMat(2,:)' disp( ['-------------( Solidity )-------------']); gv.propsMat(3,:)' disp( ['-------------( MajorAxisLength )-------------']); gv.propsMat(4,:)'

disp( ['-------------( Extent )-------------']); gv.propsMat(5,:)' disp( ['-------------( MinorAxisLength )-------------']); gv.propsMat(6,:)' disp( ['-------------( Orientation )-------------']); gv.propsMat(7,:)' disp( ['-------------( FilledArea )-------------']); gv.propsMat(8,:)' disp( ['-------------( Eccentricity )-------------']); gv.propsMat(9,:)'

7) Prop filtersfunction [] = cca_filter(op, handles, p1, p2)global gv;

z = [];switch op case 'Area_f' z = and ([gv.props.Area] >= p1, [gv.props.Area] <= p2); case 'Centroid_f' z = and ([gv.props.Centroid] >= p1, [gv.props.Centroid] <= p2); case 'MajorAx_f' z = and ([gv.props.MajorAxisLength] >= p1, [gv.props.MajorAxisLength] <= p2); case 'MinorAx_f' z = and ([gv.props.MinorAxisLength] >= p1, [gv.props.MinorAxisLength] <= p2); case 'Eccent_f' z = and ([gv.props.Eccentricity] >= p1, [gv.props.Eccentricity] <= p2); case 'Orient_f' z = and ([gv.props.Orientation] >= p1, [gv.props.Orientation] <= p2); case 'EqvDiam_f' z = and ([gv.props.EquivDiameter] >= p1, [gv.props.EquivDiameter] <= p2); case 'Solidity_f' z = and ([gv.props.Solidity] >= p1, [gv.props.Solidity] <= p2); case 'Extent_f' z = and ([gv.props.Extent] >= p1, [gv.props.Extent] <= p2);

end

for i=1:gv.ncca if z(i) gv.props(i).alive = 1; x = gv.im{3}; x(gv.props(i).PixelIdxList) = gv.props(i).id / gv.ncca; gv.im{3} = x;

else gv.props(i).alive = 0; x = gv.im{3}; x(gv.props(i).PixelIdxList) = 0;

gv.im{3} = x;

endend

Region Properties and Filtering Perimeter and Shape Signature Analysis

>> Cartesian Boundary>> Polar Boundary>> Chain coding

Signal Filtering Signal Resampling and Amplitude Normalization

HW Assignment(6)1. Implement morphological operators {dilate, erode, open, close, fill,

perimeter}, CCA, region properties. Integrate all to your GUI.2. Extra credit: implement shape signature.3. (Grads only):

CH07 OBJECT REPRESENTATION, CHARACTERIZATION, FEATURE EXTRACTION

Region Properties (check cca) Boundary Signature Chain Coding

Chain code: 0010300111212123233323

function [cc] = chaincode(b,unwrap)% Freeman Chain Code%% Description: Give Freeman chain code 8-connected representation of a

% boundary% Author.....: Alessandro Mannini <mannini@esod.it>% Date.......: 2010, september%% usage:% --------------------------------------------------------% [cc] = chaincode(b,u)%% INPUT:% --------------------------------------------------------% b - boundary as np-by-2 array; % np is the number of pixels and each element is a pair (y,x) of% pixel coordinates% unwrap - (optional, default=false) unwrap code;% if enable phase inversions are eliminated% %% OUTPUT:% --------------------------------------------------------% cc is structure with the following fields:%% cc.code - 8-connected Freeman chain code as 1-by-np array (or% 1-by-(np-1) if the boundary isn't close)% cc.x0,cc.y0 - respectively the abscissa and ordinate of start point% cc.ucode - unwrapped 8-connected Freeman chain code (if required)%

%%% used direction-to-code convention is: 3 2 1% \ | /% 4 -- P -- 0% / | \% 5 6 7% % and in terms of deltax,deltay if next pixel compared to the current:% --------------------------% | deltax | deltay | code |% |------------------------|% | 0 | +1 | 2 |% | 0 | -1 | 6 |% | -1 | +1 | 3 |% | -1 | -1 | 5 |% | +1 | +1 | 1 |% | +1 | -1 | 7 |% | -1 | 0 | 4 |% | +1 | 0 | 0 |% --------------------------%

% check input argumentsif nargin>2 error('Too many arguments');elseif nargin==0 error('Too few arguments');elseif nargin==1

unwrap=false;end % compute dx,dy by a circular shift on coords arrays by 1 elementsb=circshift(b,[-1 0]);delta=sb-b;% check if boundary is close, if not cut last elementif abs(delta(end,1))>1 || abs(delta(end,2))>1 delta=delta(1:(end-1),:);end% check if boundary is 8-connectedn8c=find(abs(delta(:,1))>1 | abs(delta(:,2))>1);if size(n8c,1)>0 s=''; for i=1:size(n8c,1) s=[s sprintf(' idx -> %d \n',n8c(i))]; end error('Curve isn''t 8-connected in elements: \n%s',s);end

% convert dy,dx pairs to scalar indexes thinking to them (+1) as base-3 numbers% according to: idx=3*(dy+1)+(dx+1)=3dy+dx+4 (adding 1 to have idx starting% from 1)% Then use a mapping array cm% --------------------------------------% | deltax | deltay | code | (base-3)+1 |% |-------------------------------------|% | 0 | +1 | 2 | 8 | % | 0 | -1 | 6 | 2 | % | -1 | +1 | 3 | 7 | % | -1 | -1 | 5 | 1 | % | +1 | +1 | 1 | 9 | % | +1 | -1 | 7 | 3 | % | -1 | 0 | 4 | 4 | % | +1 | 0 | 0 | 6 | % ---------------------------------------

idx=3*delta(:,1)+delta(:,2)+5;cm([1 2 3 4 6 7 8 9])=[5 6 7 4 0 3 2 1];

% finally the chain code array and the starting pointcc.x0=b(1,2);cc.y0=b(1,1);cc.code=(cm(idx))';

% If unwrapping is required, use the following algorithm%% if a(k), k=1..n is the original code and u(k) the unwrapped:%% - u(1)=a(1)% - u(k)=g(k), % g(k) in Z | (g(k)-a(k)) mod 8=0 and |g(k)-u(k-1)| is minimized %if (unwrap) a=cc.code;

u(1)=a(1); la=size(a,1); for i=2:la n=round((u(i-1)-a(i))/8); u(i)=a(i)+n*8; end cc.ucode=u';end

end

Central Moments

Invariant Moments

function phi = invmoments(F)%INVMOMENTS Compute invariant moments of image.% PHI = INVMOMENTS(F) computes the moment invariants of the image% F. PHI is a seven-element row vector containing the moment% invariants as defined in equations (11.3-17) through (11.3-23) of% Gonzalez and Woods, Digital Image Processing, 2nd Ed.%% F must be a 2-D, real, nonsparse, numeric or logical matrix.

% Copyright 2002-2004 R. C. Gonzalez, R. E. Woods, & S. L. Eddins% Digital Image Processing Using MATLAB, Prentice-Hall, 2004% $Revision: 1.5 $ $Date: 2003/11/21 14:39:19 $

if (ndims(F) ~= 2) | issparse(F) | ~isreal(F) | ~(isnumeric(F) | ... islogical(F)) error(['F must be a 2-D, real, nonsparse, numeric or logical ' ... 'matrix.']); end

F = double(F);

phi = compute_phi(compute_eta(compute_m(F))); %-------------------------------------------------------------------%function m = compute_m(F)

[M, N] = size(F);[x, y] = meshgrid(1:N, 1:M); % Turn x, y, and F into column vectors to make the summations a bit% easier to compute in the following.x = x(:);y = y(:);F = F(:); % DIP equation (11.3-12)m.m00 = sum(F);

% Protect against divide-by-zero warnings.if (m.m00 == 0) m.m00 = eps;end% The other central moments: m.m10 = sum(x .* F);m.m01 = sum(y .* F);m.m11 = sum(x .* y .* F);m.m20 = sum(x.^2 .* F);m.m02 = sum(y.^2 .* F);m.m30 = sum(x.^3 .* F);m.m03 = sum(y.^3 .* F);m.m12 = sum(x .* y.^2 .* F);m.m21 = sum(x.^2 .* y .* F);

%-------------------------------------------------------------------%function e = compute_eta(m)

% DIP equations (11.3-14) through (11.3-16).

xbar = m.m10 / m.m00;ybar = m.m01 / m.m00;

e.eta11 = (m.m11 - ybar*m.m10) / m.m00^2;e.eta20 = (m.m20 - xbar*m.m10) / m.m00^2;e.eta02 = (m.m02 - ybar*m.m01) / m.m00^2;e.eta30 = (m.m30 - 3 * xbar * m.m20 + 2 * xbar^2 * m.m10) / m.m00^2.5;e.eta03 = (m.m03 - 3 * ybar * m.m02 + 2 * ybar^2 * m.m01) / m.m00^2.5;e.eta21 = (m.m21 - 2 * xbar * m.m11 - ybar * m.m20 + ... 2 * xbar^2 * m.m01) / m.m00^2.5;e.eta12 = (m.m12 - 2 * ybar * m.m11 - xbar * m.m02 + ... 2 * ybar^2 * m.m10) / m.m00^2.5;

%-------------------------------------------------------------------% function phi = compute_phi(e)

% DIP equations (11.3-17) through (11.3-23).

phi(1) = e.eta20 + e.eta02;phi(2) = (e.eta20 - e.eta02)^2 + 4*e.eta11^2;phi(3) = (e.eta30 - 3*e.eta12)^2 + (3*e.eta21 - e.eta03)^2;phi(4) = (e.eta30 + e.eta12)^2 + (e.eta21 + e.eta03)^2;phi(5) = (e.eta30 - 3*e.eta12) * (e.eta30 + e.eta12) * ... ( (e.eta30 + e.eta12)^2 - 3*(e.eta21 + e.eta03)^2 ) + ... (3*e.eta21 - e.eta03) * (e.eta21 + e.eta03) * ... ( 3*(e.eta30 + e.eta12)^2 - (e.eta21 + e.eta03)^2 );phi(6) = (e.eta20 - e.eta02) * ( (e.eta30 + e.eta12)^2 - ... (e.eta21 + e.eta03)^2 ) + ... 4 * e.eta11 * (e.eta30 + e.eta12) * (e.eta21 + e.eta03);phi(7) = (3*e.eta21 - e.eta03) * (e.eta30 + e.eta12) * ... ( (e.eta30 + e.eta12)^2 - 3*(e.eta21 + e.eta03)^2 ) + ... (3*e.eta12 - e.eta30) * (e.eta21 + e.eta03) * ... ( 3*(e.eta30 + e.eta12)^2 - (e.eta21 + e.eta03)^2 );

Intensity Histogram features 3D Color Histogram features

case 'RGB Hists' x = gv.im{1}; r = x(:,:,1); g = x(:,:,2); b = x(:,:,3); for i=1:gv.ncca [h(:,1) x] = imhist(r(gv.props(i).PixelIdxList)); [h(:,2) x] = imhist(g(gv.props(i).PixelIdxList)); [h(:,3) x] = imhist(b(gv.props(i).PixelIdxList)); z{i} = h; end gv.rgbHist = z;

% assumes RGB hists are computed 1st case 'Gray Hist Sig' gv.sigLbl = 'Gray Histogram'; for i=1:gv.ncca h = gv.rgbHist{i}; % convert RGB hits to one gray hist by left multiply by % transformation vector of rgb2gray coefficients gv.grayHist{i} = [0.299 0.587 0.114]* (h'); % observe the transpose gv.propsSig{i} = gv.grayHist{i}; end

Edge Features and Hough Transformo Edge detectorso Hougho Hough histogram signature

Example:

.:

Texture Analysis: o Spatial texture featureso Spectral texture features (FFT-based)

HW Assignment(7)1. Implement Invariant moments, RGB histograms, and Intensity histogram

features in your program.2. (Grads only): Find one other method for object representation, understand

the algorithm, implement and demonstrate it with images of your choice. (be ready to present it in class).

CH08 OBJECT/PATTERN RECOGNITION AND CLASSIFICATION

Template Matching Via Minimum Euclidean Distance (dist function)

Euclidian Distance

Given a vector space of nFeatures x nVectors 1 column = 1 vectorx = rand(3, 5) 5 vectors 3 features each

0.5807 0.9907 0.3991 0.4318 0.5010 0.8303 0.1817 0.1431 0.2480 0.6888 0.1417 0.3634 0.7346 0.2992 0.7990

>> v = rand(3,1)

v =

0.0462 0.8506 0.9934

>> d = dist(x', v) ; observe x transpose (‘)

d =

1.0057 1.3177 0.8319 0.9969 0.5204>> m = min(d)

m =

0.5204

>> id = find( d == m )

id =

5

Correlation-based Classification

Uncorrelated datav0 = rand(100,1);v1 = rand(100,1);scatter(v0, v1)corr(v0,v1) = 0.0607

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Completely correlated datav0 = rand(100,1);v1 = 3.5 * v0;scatter(v0, v1)corr(v0,v1)= 1.0000

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

Partially correlated v1 = (2 .* v0) .^ 2 + 2.5 * v0 - 17 + 7*rand(100,1);corr(v0,v1) = 0.5899scatter(v0, v1);

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-18

-16

-14

-12

-10

-8

-6

-4

Using the same example before (in Euclidean)

>> c = corr(x, v) ; observe no x transpose (‘)

c =

-0.2934 -0.9373 0.2161 -0.9157 0.9728

>> find( c == max(c) ) ; a short cut to compute the index also, ; observe max not min as opposed to Euclidian ; distance

ans =

5

K-Means clustering and classification

>> x = rand(3, 100);>> [id c] = kmeans(x', 3); % observe the x transpose>> c % centers vectors

c =

0.2995 0.4659 0.1925 0.4400 0.3241 0.6824 0.7598 0.7430 0.5419

id: the ids of the 100 vectors of the training set.

Test k-means classification of an unseen vector:

>> v = rand(3,1)

v =

0.6385 0.6984 0.0981

>> d = dist(c', v)

d =

0.7872 0.7653 0.6294

>> find( d == min(d) )

ans =

3

Neural Networks

Figure C.1 The Perceptron, the functional unit in Artificial Neural Networks

Input signal (feature vector), weights, bias & bias weight (threshold), aggregation, activation

function and output signal

Figure C.2 Input feature spaces that demonstrate, Left: linearly separable, right: linearly inseparable vector space

Figure C.3 Multi-layer Perceptron Typical Architecture

Using the same example (training set and vector) of the kmeans:

(1) Training the network

>> [net g] = nntrain(x, id', 5, 10000, 1);TRAINGD, Epoch 0/10000, MSE 1.89081/1e-006, Gradient 3.87868/1e-030TRAINGD, Epoch 1000/10000, MSE 0.0917575/1e-006, Gradient 0.0282827/1e-030

TRAINGD, Epoch 2000/10000, MSE 0.067669/1e-006, Gradient 0.0173372/1e-030TRAINGD, Epoch 3000/10000, MSE 0.0575123/1e-006, Gradient 0.0112892/1e-030TRAINGD, Epoch 4000/10000, MSE 0.0532136/1e-006, Gradient 0.00748766/1e-030TRAINGD, Epoch 5000/10000, MSE 0.0511766/1e-006, Gradient 0.00544244/1e-030TRAINGD, Epoch 6000/10000, MSE 0.0499952/1e-006, Gradient 0.00437156/1e-030TRAINGD, Epoch 7000/10000, MSE 0.0491734/1e-006, Gradient 0.00378341/1e-030TRAINGD, Epoch 8000/10000, MSE 0.0485274/1e-006, Gradient 0.00342915/1e-030TRAINGD, Epoch 9000/10000, MSE 0.0479811/1e-006, Gradient 0.00319522/1e-030TRAINGD, Epoch 10000/10000, MSE 0.0474979/1e-006, Gradient 0.00302987/1e-030TRAINGD, Maximum epoch reached, performance goal was not met.

(2) Testing the network with an unseen vector

>> nnTest(net, v)

ans =

3

Fuzzy Logic

Figure E.1 Concept of Input space to output space mapping process

Figure E.2 Classical Crisp Set Vs. Fuzzy Set

(1) Building the FIS (fuzzy inference System) Add, Edit variables (input and output) Define in/out var names (Temp, Humidity, Pressure, FanSpeed, etc.),

MFS (Tri, Trapezoid, Gauss, Sigmoid, etc.)

(2) Write fuzzy rules

(3) View and test rules

(4) View Surface three (2 in with 1 out) variables at a time 3D limitation

(5) Save model to *.fis tempcontrol.fis(6) Load model to workspace

>> tcontrol = readfis('tempcontrol.fis')

tcontrol =

name: 'temcontrol' type: 'mamdani' andMethod: 'min' orMethod: 'max' defuzzMethod: 'centroid' impMethod: 'min' aggMethod: 'max' input: [1x2 struct] output: [1x1 struct] rule: [1x9 struct]

(7) Use/test the model (the fis object)

>> evalfis( [.5 .5], tcontrol)

ans =

0.5000

>> evalfis( [.75 .1], tcontrol)

ans =

0.4188

Probabilistic Classification:

a = rand(8, 5); % generate class a training samples row-vectorsb = rand(8, 5); % generate class b training samplesc = rand(8, 5); % generate class c training samplesma = [mean(a);mean(b);mean(c)]'; % building the class means matrix row-vectorca(:,:,1) = cov(a); % building the class covars matrix [5x5x3]ca(:,:,2) = cov(b);ca(:,:,3) = cov(c);

x = rand(4, 5); % generate testing data set row-vectors (4 vectors)

p = [0.3 0.5 0.2];% probability of each class sum = 1%if p is not passed, the classes are assumed equally likely

id = bayesgauss(x, ca, ma, p);% id = the classifications of the 4 testing vectors

HW Assignment(8)1. Implement at least two of the Pattern Recognition methods. 2. Graduate Students only: Implement Template Matching, Neural Networks and

Probabilistic-based Classification.

CH09 VIDEO PROCESSING

Image acquisition Image grabber Real time (on-line) processing Off-line Processing: processing a directory of images Applications: motion tracking, surveillance, dynamic sign language, etc.

HW Assignment(9)Given the image sequence (compressed file on blackboard) write a motion tracking algorithm to track the moving objects in the scene. Using the CCA algorithm, compute the following:1. Bounding box2. Center of the bottom edge of the bounding box3. The size4. Velocity5. Classify its posture, standing, squatting and crawling (Graduate Students

only)