PROJECT REPORT ON

FINAL REPORT ON MINI-PROJECT

BHARATI VIDYAPEETHS COLLEGE OF ENGINEERINGA-4,Paschim Vihar, Rohtak Road, New Delhi

AFFILIATED TOGURU GOBIND SINGH INDRAPRASTHA UNIVERSITY (G.G.S.I.P.U.)

Submitted to:IT DEPARTMENT

Submitted by:Bhavna Nakra (25211503110)Diksha Mahajan (25011503110)Surbhi Gupta (10511503110)

Acknowledgement

We are overwhelmed in all humbleness and gratefulness to acknowledge our depth to all those who have helped us to put these ideas, well above the level of simplicity and into something concrete.

We are very thankful to our guide Ms. Parul Yadav for approving of our project with encouragement and her valuable help. She was always there to show us the right track when we needed help. With the help of her valuable suggestions and guidance, we were able to performthis project work.

We would also like to thank our teachers and friends, who often helped and gave us support at critical junctures during the making of this project.

Bhavna NakraDiksha MahajanSurbhi Gupta

Table of Contents

Chapter-1: IntroductionChapter-2: Features and ControlsChapter-3: Concept OverviewChapter-4: Source CodeChapter-5: Output ScreensChapter-6: Future ScopeChapter-7: References

Page | 1

Chapter-1Introduction

In the second module of our minor project we have moved onto Image drawing and Gesture recognition. This has been implemented using OpenCV Version 2.2.0 and CodeBlocks 10.05.

OpenCV 2.2.0

The openCV version 2.2.0 was developed as a development in the 2.x series. The key new features include the new modular structure of the library, the new feature detector/descriptor framework, Latent SVM (Support Vector Machine) detector and Gradient Boosting Trees, opencv-gpu - CUDA module created with support by NVidia, Android port created with support by Google, new Qt backend in highgui with lots of nice features, improved documentation with links to Wiki and hundreds of bug fixes.

Code::Blocks

It is afreeandopen source,cross-platformIDEwhich supports multiplecompilersincludingGCCandVisual C++. It is developed inC++usingwxWidgetsas the GUI toolkit. Using a plugin architecture, its capabilities and features are defined by the provided plug-ins. Currently, Code::Blocks is oriented towardsCandC++. In some cases installing third-partySDKsorframeworksis necessary.Code::Blocks is being developed forWindows,Linux, andMac OS Xand has been ported toFreeBSD The latest stable version as of December 2012is Code::Blocks 12.11, was released on December 6, 2012.Using the above library and platform, we have implemented a gesture drawing mechanism that applies the colour detection and object tracking that we have studied in the first report. This remote sketching takes yellow colour as a pen to draw lines of desired colour (Red, Green or Blue) and thickness. The program also allows one to erase the drawing, clear the screen or exit the program without touching.

Page | 2

Chapter-2Features and ControlsThe Remote Sketching program will offer the following:

Noise Handing:- using area to ignore the noise- using magnitude limit to skip long distance points- using 5x5 median filter to decrease the background noise

Control features:- choosing line colours- changing line thickness based on Y coordinates- clearing the screen + saving the image- exiting the program- saving the image (new)The above features and controls are incorporated in the code using the concepts and logic described next.

Page | 3

Chapter- 3Concept OverviewThe following functions and concepts are used in the code:

1. cvCreateImage: IplImage*cvCreateImage(CvSizesize, intdepth, intchannels)This function creates an image header and allocates the image data.size Image width and heightdepth Bit depth of image elements.channels Number of channels per pixel. This function only creates images with interleaved channels.

2. cvCvtColor: void cvCvtColor(constCvArr*src,CvArr*dst, intcode)Converts an image from one color space to another.src The source 8-bit (8u), 16-bit (16u) or single-precision floating-point (32f) imagedst The destination image of the same data type as the source. The number of channels may be differentcode Color conversion operation that can be specified.

3. cvInRangeS:voidcvInRangeS(constCvArr*src,CvScalarlower,CvScalarupper,CvArr*dst)Checks that array elements lie between two scalars.src The first source arraylower The inclusive lower boundaryupper The exclusive upper boundarydst The destination array, must have 8u or 8s typeThe function does the range check for every element of the input array

4. cvQueryframe:IplImage* cvQueryFrame(CvCapture* capture)Grabs and returns a frame from a camera or file. Capture- Video Capturing structure. The function cvQueryFrame grabs a frame from a camera or video file, decompresses it and returns it. This function is just a combination of GrabFrame and RetrieveFrame , but in one call. The returned image should not be released or modified by the user. In the event of an error, the return value may be NULL.

5. cvFlip: cvFlip(IntPtr src,IntPtr dst,int flipMode) Causes flipping to avoid the mirroring problem.src (IntPtr)-Source array.dst (IntPtr)-Destination array.flipMode(Int32)-Specifies how to flip the array. flip_mode = 0 means flipping around x-axis, flip_mode > 0 (e.g. 1) means flipping around y-axis and flip_mode < 0 (e.g. -1) means flipping around both axes.

Page | 4

6. cvSmooth: cvSmooth Method (src, dst, type, param1, param2, param3, param4)It will smoothen the image using one of several methods. Every of the methods has some features and restrictions listed below:

Blur with no scaling works with single-channel images only and supports accumulation of 8-bit to 16-bit format (similar to cvSobel and cvLaplace) and 32-bit floating point to 32-bit floating-point format. Simple blur and Gaussian blur support 1- or 3-channel, 8-bit and 32-bit floating point images. These two methods can process images in-place. Median and bilateral filters work with 1- or 3-channel 8-bit images and cannot process images in-place.Here we have used themedian filter, whichis a nonlineardigital filteringtechnique, often used to removenoise. Such noise reduction is a typical pre-processing step to improve the results of later processing (for example,edge detectionon an image). Median filtering is very widely used in digitalimage processingbecause, under certain conditions, it preserves edges while removing noise. The main idea of the median filter is to run through the signal entry by entry, replacing each entry with themedianof neighbouring entries. The pattern of neighbours is called the "window", which slides, entry by entry, over the entire signal.

7. Moments: Calculates all moments up to third order of a polygon or rasterized shapevoid cvMoments( const CvArr* arr, CvMoments* moments, int isBinary=0 );arr-Image (1-channel or 3-channel with COI set) or polygon (CvSeq of points of a vector of points).Moments- Pointer to returned moment state structureisBinary- (For images only) If the flag is non-zero, all the zero pixel values are treated as zeroes, all the others are treated as ones.

The functioncvMomentscalculates spatial and central moments up to the third order and writes them tomoments. The moments may be used then to calculate gravity center of the shape, its area, main axises and various shape characeteristics including 7 Hu invariants.GetSpatialMoment: Retrieves spatial moment from moment state structure

double cvGetSpatialMoment( CvMoments* moments, int j, int i );moments- The moment state, calculated bycvMoments.j- x-order of the retrieved moment, j >= 0.i- y-order of the retrieved moment, i >= 0 and i + j = 0.i-y-order of the retrieved moment, i >= 0 and i + j width - 10 ;x++) { if(x+location.x>=src->width) continue; for(y=0;y < overlay->height - 10 ;y++) { Page | 8 if(y+location.y>=src->height) continue; CvScalar source = cvGet2D(src, y+location.y, x+location.x); CvScalar over = cvGet2D(overlay, y, x); CvScalar merged; for(i=0;idepth, //Bit depth per channel 3 //number of channels ); cvSet(imgDrawing, white);Page | 11

CvFont font, fontbig; cvInitFont( &font, CV_FONT_HERSHEY_COMPLEX, 1, .6, 0, 2, CV_AA); cvInitFont( &fontbig, CV_FONT_HERSHEY_COMPLEX, 3, .6, 0, 3, CV_AA);

int confirm_close = 10, confirm_clear = 20; // counters for clear and exit confirmation char buffer [50]; // buffer for cvPutText int image_num = 0; // to keep track of image numbers for saving int posX = 0; int posY = 0;

while(true) { IplImage* frame = 0; frame = cvQueryFrame(capture); if(!frame) break; cvFlip(frame,NULL,1); // flip the frame to overcome mirroring problem

// If this is the first frame, we need to initialize it if(imgScribble == NULL) imgScribble = cvCreateImage(cvGetSize(frame), 8, 3); // cvSet( imgScribble, cvScalar(0,0,0));Page | 12

// Median filter to decrease the background noise cvSmooth( frame, frame, CV_MEDIAN, 5, 5 //parameters for filter, in this case it is filter size );

// Holds the thresholded image (tracked color -> white, the rest -> black) IplImage* imgThresh = GetThresholdedImage(frame,lowerBound,upperBound);

// Calculate the moments to estimate the position of the object CvMoments *moments = (CvMoments*)malloc(sizeof(CvMoments)); cvMoments(imgThresh, moments, 1);

// The actual moment values double moment10 = cvGetSpatialMoment(moments, 1, 0); double moment01 = cvGetSpatialMoment(moments, 0, 1); double area = cvGetCentralMoment(moments, 0, 0);

// Holding the last and current positions int lastX = posX; int lastY = posY;Page | 13

posX = 0; posY = 0;

if(moment10/area>=0 && moment10/area < 1280 && moment01/area >=0 && moment01/area < 1280 && area>area_limit /* to control the limit */ ) { posX = moment10/area; posY = moment01/area; }

CvPoint cvpoint = cvPoint(150,30); // location of the text if(posX < 90 && posY > 400) // clear { lineColor = white; // white color works as eraser cvPutText( frame, "Eraser selected.", cvpoint, &font, white ); sprintf (buffer, "Clearing the screen in %d",confirm_clear); // count-down for clearing the screen cvPutText( frame, buffer, cvPoint(150,70), &font, red ); confirm_clear--; if(confirm_clear < 0) // confirm in 10 frames before clearing {Page | 14 confirm_clear = 20; sprintf (buffer, "d0%d.jpg",image_num++); cvSaveImage(buffer ,imgDrawing); // save the frame into an image ClearScreen(imgScribble,imgDrawing); cvPutText( frame, "Cleared the screen.", cvPoint(150,110), &font, white ); } } else if(posX > 540 && posY > 360) // blue { lineColor = blue; cvPutText( frame, "Blue color selected.", cvpoint, &font, blue ); }

else if(posX > 540 && posY > 200 && posY < 280) // green { lineColor = green; cvPutText( frame, "Green color selected.", cvpoint, &font, green ); }

else if(posX > 540 && posY < 120) // red { lineColor = red; cvPutText( frame, "Red color selected.", cvpoint, &font, red ); }Page | 15

else if(posX > 0 && posX < 90 && posY > 0 && posY < 120) // exit { sprintf (buffer, "EXITING in %d",confirm_close); cvPutText( frame, buffer, cvpoint, &font, red ); confirm_close--; if(confirm_close < 0) // confirm in 10 frames before exit break; } else if(posX < 90 && posY > 130 && posY < 390) // line thickness { lineThickness = 6 - ( posY/60-1 ); // change the thickness of line from 1 - 5 based on posY }

sprintf (buffer, "%d",lineThickness); cvPutText( frame, buffer, cvPoint(40,255), &fontbig, lineColor );

double magnitude = sqrt( pow(lastX-posX,2) + pow(lastY-posY,2) ); // We want to draw a line only if its a valid position //if(lastX>0 && lastY>0 && posX>0 && posY>0) if(magnitude > 0 && magnitude < 100 && posX > 120 && posX