Pedestrian Detection: introduction

• Approaches– Holistic detection: use local search window that meets

criterias– Part-based detection: pedestrian as a collection of parts

(to be found!)– Patch-based detection: local features matched against a

(learned) codebook, then voting for final detection

For a survey on the approaches see: N.Dalal “Finding people in images and videos”, PhD thesis, July 2006.

Holistic approaches

Some remarkable pedestrian detector– Haar wavelets + SVM

P. Papageorgiou and T. Poggio, “A trainable system for object detection,” Intl. J. of Computer Vision, vol. 38, no. 1, pp. 15–33, 2000.

– the popular face detector from Viola Jones (haar+adaboost face-detector) + motion cues

P. Viola, M. Jones, and D. Snow, “Detecting pedestrians using patterns of motion and appearance,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, New York, NY, volume 1, 2003, pp. 734–741.

– Histogram of oriented gradients (HOG)N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition,

San Diego, CA, volume 1, 2005, pp. 886–893.Q. Zhu, S. Avidan, M. C. Yeh, and K. T. Cheng, “Fast human detection using a cascade of histograms of oriented gradients,” in Proc. IEEE Conf. on

Computer Vision and Pattern Recognition, New York, NY, volume 2, 2006, pp. 1491 – 1498.

– Covariance Descriptors + BoostingO. Tuzel, F. Porikli, and P. Meer, “Pedestrian detection via classification on Riemannian manifolds,” To appear in IEEE Trans. Pattern Anal.

Machine Intell., 2008.

Pedestrian detection using covariance descriptors and boosting

rejection cascades approach

Detection on negatives increases

Casc 1 Casc 2 Casc N

Detection on positives decreases

Pedestrian detection using covariance descriptors and boosting

rejection cascades approach

Detection on negatives increases Detection on positives decreases

PerformanceCov-Desc + BoostingHOG + Ker SVMHOG + Lin SVMHOG + Boosting

each black dot is an additional cascade

Inside a cascade: the weak learners1. Add a weak learner to the cascade, that is2. Assign to it a (randomly extracted) sub-region3. Compute covariance-descriptor on this region for

each– Positive sample– Negative sample

4. Are positives and negatives “easily” separable?– No? Go to 1

– Yes? Done with this cascade

Covariance descriptors

Define an image

Region R

Extract Pixel-wise Feature 1

(e.g.: color components, luminance, gradients, …)

Extract Pixel-wise Feature 2

Extract Pixel-wise Feature N

Mean, var

Mean, var

Mean, var

Covariance CR

(NxN matrix, sym

pos def)

Covariance descriptors

• PROs– Versatile and flexible (you can use the pixel-wise

features most suitable for your goal)– Computed very quickly using integral images– Compact (N*(N+1)/2 independent values)

• CONs– Euclidean distance is NOT appropriate over

symmetric positive matrices: they lie over a Riemannian manifolds

Riemannian ManifoldsIn order to use traditional machine learning

techiniques:

• Move back and forth from Riemannian manifold of sym pos def matrices to euclidean space of symmetric matrices using respectively • Exponential of Matrix • Logarithm of Matrix

Expansive computations!

J. Jost, “Riemannian Geometry and Geometric Analysis”. Springer, fourth edition, 2005.

X. Pennec, P. Fillard, and N. Ayache, “A Riemannian framework for tensor computing,” Intl. J. of Computer Vision, vol. 66, no. 1, pp. 41–66, 2006.

Using this detector in real-time surveillance applications

1. Reduce the pedestrian search region where there is (or was) motion• Build a motion history and focus the detection search over the most recent motion but keeping

an eye on oldest motion regions

2. Exploit the background when camera is static with implicit relevance feedback: • In many surveillance scenarios it is possible to assume that the background image does not

contain humans• therefore, enrich the generic pedestrian classifier• training some additional ad-hoc and view-dependent cascades • that tackle the false-positives detected in the background • The enriched classifier is to be used in the ped-detection over the given view

2. Use the pedestrian detection to infer the scene perspective• False detections are very limited, and most of them are out of perspective• Therefore, having defined a perspective model, it is possible to estimate it, rejecting the outliers• Once estimated, it can reject out-of-perspective detections

@ICDSC09: Covariance Descriptors on Moving Regions for Human Detection in Very Complex Outdoor Scenes

Giovanni Gualdi, Andrea Prati and Rita Cucchiara.Univ. of Modena and Reggio Emilia

Using this detector in real-time surveillance applications

Examples

Detections using the generic classifier

Detections using the generic classifier+Perspective model

Detections using the generic classifier+Perspective model+Additional relevance feedback cascades

Videos