OpenCV Face TrackingANDREW STODGHILL

Goal

The goal of this project was to gain a thorough understanding of the face tracking methods and algorithms used by OpenCV Library.

Literary Focuses Fundamental face tracking and detection theory

Paper by Viola and Jones

Paper done by Lienhart et al.

Basic implementation of the OpenCV face tracking algorithms through Visual Studio

Intro

Open Source Computer Vision Library (OpenCV) is an open source computer vision and machine learning software library.

Used by over 47 thousand dedicated users, with an estimated over 7 million downloads

Used by Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, and Toyota

Literary Focus

Fundamental face tracking and detection theory Chapter 14: Recognition - Computer Vision: Algorithms

and Applications

Paper by Viola and Jones Innovative paper that introduced a cascaded boosting

method

Paper by Lienhart et al. Extended the work done by Viola et al. and was directly

integrated into the OpenCV library

Fundamental Face Tracking Theory

Fundamental Face Tracking Theory

Face Detection Processes that scan over images to find areas

where a face may be

Broken up into three main techniques Feature-based

Template-based

Appearance-based

Fundamental Face Tracking Theory

Appearance-based Approach includes: Clustering and PCA

Neural Networks

Support Vector Machines

Boosting Introduced by Viola and Jones and used by the

OpenCV algorithm

Fundamental Face Tracking Theory

Boosting Involves training a series of increasingly

discriminating simple classifiers and then blending their outputs.

Where are extremely simple functions

Fundamental Face Tracking Theory

Schematic illustration of boosting

Paper by Viola and Jones“RAPID OBJECT DETECTION USING A BOOSTED CASCADE OF SIMPLE FEATURES”

Viola and Jones

Introduced three key contributions to the appearance-based approach to face tracking Integral Image

Learning algorithm based on AdaBoost

Using a cascaded classifier system

Viola and Jones

Integral Image Instead of working directly with the image

intensities, the Integral image contains sums of pixels that allows fast computation of rectangular sums

, ,,

Viola and Jones

Integral image is helpful when using Haar-like features. These features are built with rectangles, so using the Integral image, any Haar-like feature can be computed at any scale or location in constant time.

Viola and Jones

Boosting method based on AdaBoost Each sub-window in the image has far more Haar-

like features than pixels, meaning it is over complete

This is solved by a modification of the AdaBoost procedure The weak learner is constrained so that each weak

classifier returned can depend on only a single feature

1 0

Viola and Jones

Final contribution was a method for combining successively more complex classifiers in a cascade structure Dramatically increases the speed of the detector

by focusing attention on promising regions of the image

Viola and Jones

Summary Three key contributions

Integral Image

Boosting method based on AdaBoost

Method for combining successively more complex classifiers in a cascade structure

Lienhart et al.“EMPIRICAL ANALYSIS OF DETECTION CASCADES OF BOOSTED CLASSIFIERS FOR RAPID OBJECT DETECTION”

Lienhart et al.

Took the work of Viola and Jones and extended it in two ways Introduced a novel set of rotated Haar-like features

to the set used by Viola and Jones

Performed a thorough analysis of the different AdaBoost methods and weak classifiers used

Lienhart et al.

Added 45° rotated features, which added additional domain knowledge to the learning framework

Lienhart et al.

14 feature prototypes shown on previous slide. These prototypes are scaled independently in

vertical and horizontal direction in order to generate a rich, over-complete set of features

1 5,3,6,2,0° 3 7,3,2,2,0°

Lienhart et al.

To compensate for the rotated features, and new Integral Image, renamed SAT and RSAT, had to be created

The SAT and RSAT could be computed in one sweep of the image, and any only required four table look ups

Lienhart et al.

Lienhart et al. took the ideas to use AdaBoost as a learning and training concept for the classifiers and brought it one step further

They tested AdaBoost for three different variations Discrete AdaBoost

Real AdaBoost

Gentle AdaBoost

Also tested different types of cascaded classifiers, and the basic set of Haar-like features and the extended version

Lienhart et al.

Experimental results were performed on the complete CMU Frontal Face Test

A positive result was found if two conditions were met The Euclidian distance between the center of a

detected and actual face was within 30% of the width of the actual face

The width of the detected face was within 50% of the actual face width

Every detected face, which was not a hit, was counted as a false alarm

Lienhart et al.

Results Lienhart et al. found that Gentle AdaBoost

outperformed the other two boosting algorithms

A CART tree with a set number of split nodes was the optimal weak classifier

The extended set of Haar-like features, on average, had a false alarm rate 10% lower than the basic set

Methods

Methods

Found in /opencv/data/haarcascades

Methods

Methods

Methods

Results

References[1] itseez, "opencv.org," OpenCV, 2013. [Online]. Available: opencv.org. [Accessed 29 11 2013].

[2] M. V. Matorell and F. d. l. T. Frade, "Real-time Face Tracking Methods," Universitat Autonoma de Barcelona, Barcelona, Spain, 2009.

[3] P. Viola and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features," in Accepted Conference on Computer Vision and Pattern Recognition, Cambridge, 2001.

[4] R. Lienhart, A. Kuranov and V. Pisarevsky, "Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection," Microprocessor Research Lab, Santa Clara, CA, 2002.

[5] M. H. Yang, D. J. Kriegman and N. Ahuja, "Detecting faces in images: A survey," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 1, no. 24, pp. 34-58, 2002.

[6] R. Szeliski, "Recognition," in Computer Vision: Algorithms and Applications, Springer, 2010, pp. 655 - 745.

[7] Y. Freund and R. E. Schapire, "Experiments with a new boosting algorithm. In Machine Learning," in Thirteenth International Conference in Machine Learning, San Francisco, 1996.

[8] H. Rowley, S. Baluja and T. Kanade, "Neural network-based face detection," IEEE Patt. Anal. Mach. Intell., vol. 20, pp. 22-38, 1998.

[9] "Open Computer Vision Library," [Online]. Available: http:/sourceforge.net/projects/opencvlibrary/.

[10] S. Ahuja, "Getting Started with OpenCV 2.3 in Microsoft Visual Studio 2010 in Windows 7 64-bit," WordPress.com, 18 July 2011. [Online]. Available: http://siddhantahuja.wordpress.com/2011/07/18/getting-started-with-opencv-2-3-in-microsoft-visual-studio-2010-in-windows-7-64-bit/. [Accessed 30 November 2013].

[11] techbitar, "Face detection and tracking with Arduino and OpenCV," instructables, 20 2 2013. [Online]. Available: http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/. [Accessed 30 11 2013].

Questions?