A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost

Sheng Yang, Xianmei Liao, UK Borasy

College of Information Science and Engineering, Hunan University Changsha, Hunan, China 410082

yangsh0506@hotmail.com, liaoxianmei0595@163.com

Abstract In this paper, we present a pedestrian detection method based on the combination of Histograms of

Oriented Gradient (HOG) feature and uniform local binary pattern (LBP) feature, which can detect pedestrian accurately. To the problem of low recognition rate for a single feature, we combine contour information and texture information, and propose the cascade of the two types of features, HOG features and LBP features as the feature set. In order to compare the experimental results, Gentle AdaBoost is used to train the pedestrian classifier on the INRIA dataset. The experimental results show that these two features of pedestrian detection algorithm improve the accuracy and reduce the error rate. Our method achieve a detection rate of 94.05% at FPPW = 10-4, which is better than Dalal’s (detection rate of 84% to 89% at 10-4 FPPW).

Keywords: Image recognition, machine learning, pedestrian detection, HOG, LBP

1. Introduction

Pedestrian detection is widely used in the system of computer vision surveillance and pattern recognition. However, due to the influence of posture, illumination and complex background, detecting human from the image background or video quickly and accurately is still a challenging task.

Pedestrian detection research has some success in the computer field. In 2005, Dalal and Trigges [1] presented a pedestrian detection algorithm using Histograms of Oriented Gradients (HOG) and linear SVM as the classifier. Their detection results were excellent at that time. They achieved a detection rate of 88% at FPPW = 10-4. After that, many researchers improved this method. Zhu et al [2] presented a near real-time detection system that they used blocks and cells at multiple scales and integrated a cascade of rejecters approach using AdaBoost algorithm for feature selection. For a 320 * 280 image, the detector can process about 5 to 30 frames per second. The detection rate was greatly enhanced. It greatly improved the detection rate and reduced the detection time. Recently, many researchers explored different ways to combine features, such as the combination of HOG feature, edgelet, and Haar wavelets. Li Zhang and Bo Wu et al [3] explored the combination of HOG feature and edgelet as the feature set and two classification models (AdaBoost and SVM cascade) in IR images to improve the detection performance. Takala et al. [4] introduced a novel real-time tracker using color, texture and motion information, and the experiments showed that their system was better than the versions based on single features. Wang et al. [5] used the combination of HOG and LBP feature as the feature set, which are learned from the training data using linear SVM. LBP operator is a good characterization to describe the texture of the object.

Although many experiments showed that the HOG feature could represent the appearance of the object and shape effectively, a single feature is not enough to express when the background is clustered with noisy edges. LBP is complementary in this aspect. LBP is an effective texture descriptor and has been used in various applications. Therefore, in order to demonstrate pedestrian information, we combine HOG and LBP feature as the feature set, and use Gentle AdaBoost as their strong classifier.

The paper is structured as follows: Section 2 describes the HOG and LBP features and Gentle AdaBoost classifier and the weak classifier. The implementation and comparative results are presented in Section 3. Finally, the main conclusions are summarized.

International Journal of Advancements in Computing Technology(IJACT) Volume4, Number19,October. 2012 doi:10.4156/ijact.vol4.issue19.66

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

553

2. The pedestrian detection framework

The combination of two types of features (HOG and LBP) as the feature set to detect the pedestrian. The framework of HOG-LBP detector is shown in Figure1. Firstly, the input images are normalized size of 64*128. Secondly, for each image we calculated HOG feature and uniformed LBP feature respectively. Thirdly, all HOG-LBP feature vectors are trained by Gentle AdaBoost. Then a HOG-LBP detector can be achieved for testing. In the following section, we give an overview of the two feature extraction and some details in our paper.

Figure 1. The framework of HOG-LBP detector

2.1. HOG feature extraction

HOG feature is an excellent descriptor, which calculates the gradient magnitude and the gradient direction of the local image. It has shown great success in object detection and recognition [1~3]. The computation of HOG features involves the following steps:

a. Calculation of the gradient We first calculate the gradient amplitude of the input image both in the horizontal and vertical

direction with a 1-D mask template, i.e., [-1 0 1].

( , ) ( 1, ) ( 1, )xG x y I x y I x y (1)

( , ) ( , 1) ( , 1)yG x y I x y I x y (2)

Where, ( , )I x y is pixel value of the point ( , )x y , ( , )xG x y and ( , )yG x y denote horizontal

gradient amplitude and vertical gradient amplitude respectively. b. Calculate the gradient of color images Respectively calculate the gradient of the three color channels ( , , )R G B images, and select one of

the largest magnitude values as the gradient of this pixel. Gradient amplitude of the pixel (x, y):

2 2( , ) ( , ) ( , )x yG x y G x y G x y (3)

Gradient direction of the pixel (x, y):

1 ( , )( , ) tan ( )

( , )y

x

G x yx y

G x y (4)

c. Each 16*16 block is divided into 2*2 cells. 16*16 block adds Gaussian weights to adjust the

gradient amplitude, each bin range is ( 1)

, , 0,1, ..., 89 9

k kk

. The gradient magnitude and orient

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

554

of each pixel in the cell is voted into 9 bins with the tri-linear interpolation. Tri-linear interpolation is important, which makes the image block of the border region better allocated to enhance the stability of the characteristics.

d. The histograms of the four cells in each block concatenated on a 36-D feature vector that is normalized by L2-Hys to reduce the influence of the local variation in illumination and foreground-background contrast.

Each 64*128 detection window is represented by 105 overlapped blocks and then are concatenated into a 3780-D feature vector. The process of extracting HOG feature is showed in Figure2.

0 10 20 30 400

0.1

0.2

0.3

0.4

1234567890

500

Figure 2. The process of HOG feature extraction

2. 2. LBP feature extraction

LBP is an excellent texture descriptor for its invariance of gray-scale and rotation. It has been successfully applied to the object detection [5~8].

Generally, we use the notation ,

u

P RLBP , which denotes that P sample points ( 0,1,... )pg p P with

radius r for each pixel, and the number of 0 and 1 transition is no more than u. Uniformed Local Binary Pattern is defined as [9]:

1

, 10

( ) s( ) ( )p

uP R i c i c

i

U LBP g g s g g

(5)

1

,0,

( ) ( ) 2LBP =

1

p

p c P RupP R

s g g if U LBP

P others

(6)

Where:

1 0

s( )0 0

if xt

if x

(7)

The LBP patterns we used are 28,1LBP . The number of uniform model is 58, and other model as a

class, we can get 59-D vector. The LBP features of the four cells in each block are concatenated into a 59-D feature vector of the block.

Each 64*128 detection window is represented by 32 blocks and thus concatenated into 1888-D uniform LBP feature vector. The process of LBP feature extraction is showed in Figure3.

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

555

Figure 3. An illustration of the 28,1LBP pattern (a) The original gray image. (b) 8 sampling points with

radius 1. (c) The binary value of the sampling point. (d) The 28,1LBP pattern of the pixel.

2.3. Gentle AdaBoost algorithm based on the HOG-LBP

The CART algorithm is a simple decision tree [10]. Each branch is just like stump weak classifiers. Variable Y represents the results, and X denotes a D-dimensional feature, which effectively divide the training samples into two partitions using a threshold, and the feature as the root node. Next, we select the left or right with the smaller classification error rate. This sub-sample continues to split into two parts using other threshold. Finally, generate a binary tree, which is shown in Figure 4 below.

Figure 4. The CART weak learners dividing the sample X

The idea of AdaBoost algorithm is to use a simple weaker learner, to change the weights through the T-round training, and eventually superimpose to constitute a strong classifier. AdaBoost algorithm contains Discrete AdaBoost, Real AdaBoost and Gentle AdaBoost. The results [11] showed that Gentle Adaboost could obtain better results. The Gentle AdaBoost algorithm used the Classification and Regression Tress (CART) instead of the stump as the weak learner.

Gentle AdaBoost training algorithm with CART is as follows:

1) For a given the training images were calculated HOG-LBP feature , ,( )i i hog i lbpx x x

2) For a given training sample sets 1 1(( , ),..., ( , ))n nx y x y , which ix X , X represents sample

space. 1,1iy for negative and positive examples respectively; N is the total number of samples.

Initialize the weights of N samples: 1, 1/iD N

3) for 1,...,t T

① A CART algorithm to construct a binary tree as a weak learner

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

556

② According to the weight of each sample fit the classifier to obtain a class probability

estimate ( 1| ( ))DP y f x ( 1| ( ))DP y f x .

③ Set ( ) ( 1| ( )) ( 1| ( ))t D Dh x P y f x P y f x

④ Update ( ) ( ) ( )tH x H x h x

⑤ Update the weights:

( ),

1,

i t i

i

i

y h xt x

t xt

D eD

Z

，where tZ is a normalization factor

4) The final strong classifier:1

( ) ( ( ))T

t it

H x sign h x

2.4. Detection window integration

We used the method proposed in [12] to fuse detection window, and classified all the windows into disjoint subsets, the overlapped area α between two windows must exceed 50%:

1 2

1 2

( )

( )

area w w

area w w

(8)

3. Experimental results and anlysis 3.1. DataSet

We evaluate our algorithm on the INRIA pedestrian dataset, which contains 2416 positive samples of 160×96 pixels and 1218 no-person background images for training. The test set contains 1126 pedestrian images of 134×70 pixels and the 453 no-person background images. Some training samples showed in Figure 5.

Figure 5. Some training samples

3.2. Training and evaluation

We first chose 2146 positive samples that were cropped to standard size of 128×64 pixels and sampled five negative samples randomly from the 1218 no-person background images. So the initial

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

557

training set contained 2416 positive samples and 6090 negative samples. Each window is 5668-D feature vector. All samples were extracted from HOG, LBP, HOG-LBP features, and then used Gentle AdaBoost for 200 times training as a coarse descriptor. Using this descriptor to detect the background image so that we can obtain hard samples, then we re-train the initial training set by adding these hard samples. The entire training process is showed in Figure 6.

Figure 6. The entire training process

To quantify the detection performance, the DET (Detection Error Tradeoff) curve is used in this

paper [1], plots of miss rate versus false positives per window (FPPW). Lower miss rate means better detection performance on the same FPPW. Figure 7 shows each block of the LBP were normalized by L2-Hys, L1-Norm, L2-Norm and L1-sqrt. The DET curve shows that L2-Hys has the best performance. Therefore, in the following experiments, L2-Hys normalized all the blocks.

10-6

10-5

10-4

10-3

10-2

10-1

100

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

FPPW(False positive per window)

mis

srat

e(F

alse

neg

ativ

e ra

te)

DET for classification

HOG-LBP+GenAdaBoost(hys)HOG-LBP+GenAdaBoost(l1sqrt)HOG-LBP+GenAdaBoost(l2)HOG-LBP+GenAdaBoost(l1)Dalal HOG+SVM

Figure 7. Blocks for different normalization

Figure 8 compares the LBP detector, HOG detector and HOG-LBP detector. It is obvious that the

combination of these two features detector performance is better than other single feature detector. At 10-4 FPPW, we achieved 84% detection rate for the HOG detector and 75% detection rate for the LBP detector. For the HOG-LBP detector, which performs best, the detection rate is about 94.05%. Therefore, the combination of HOG and LBP feature can improve the detection rate significantly.

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

558

10-6

10-5

10-4

10-3

10-2

10-1

100

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

FPPW(False positive per window)

mis

srat

e(Fal

se n

egat

ive

rate

)

DET for classification

HOG-LBP+GenAdaBoostHOG+GenAdaBoostLBP+GenAdaBoostDalal HOG+SVM

Figure 8. Classification performance based on three different features

Our HOG-LBP detector is tested on the INRIA pedestrian dataset. Some detect results shows in

figure 9. As can be see, the detector can efficiently discriminate pedestrians from other objects.

Figure 9. Some qualitative results of our algorithm

4. Conclusions

We proposed a pedestrian detection approach based on the combination of HOG feature and LBP feature, the experiments showed that our performance is very well. When the image texture processing

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

559

is abundant, and the contrasting is light, or color is clear, it will be able to detect pedestrians accurately. But when the picture processing is darker or the vertical direction characteristics are similar to pedestrian feature, false detection could occur. The HOG-LBP feature used the Gentle AdaBoost classifier as the strong classifier. A detection rate of 94.05% is obtained at FPPW=10-4, which is better than the Dalal’s (84% to 89%). 5. References [1] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection”, In Proceedings of

IEEE Conference on Computer Vision and Pattern Recognition, pp.886-893, 2005. [2] Q. Zhu, S. Avidan, M.-C. Yeh and K.-T. Cheng, “Fast Human Detection Using a Cascade of

Histograms of Oriented Gradients”, In CVPR, pp.1491-1498, 2006. [3] Li Zhang，Bo Wu and Ram Nevatia, “Pedestrian Detection in Infrared Images Based on Local

Shape Features”, In CVPR, June 2007. [4] Takala. Valtteri, Pietikainen, Matti, “Multi-Object Tracking Using Color, Texture and Motion”, In

CVPR, 2007. [5] X. Wang, T. X. Han and S. Yan, “An HOG-LBP Human Detector with Partial Occlusion

Handling”, In ICCV, pp. 32-39, Kyoto, 2009. [6] Chengbin, Zeng and Huadong, Ma, “Robust Head-Shoulder Detection by PCA-Based Multilevel

HOG-LBP Detector for People Counting”, In ICPR, pp. 2069-2072, 2010. [7] Guolong Gan and Jian Cheng, “Pedestrian Detection Based on HOG-LBP Feature”,

Computational Intelligence and Security (CIS), pp.1184-1187, 2011. [8] Chunmei Qing, Patrick Dickinson, Shaun Lawson and Robin Freeman, “Automatic Nesting

Seabird Detection Based on Boosted HOG-LBP Descriptors”, IEEE International Conference on Image Processing (ICIP), pp. 3577-3580, 2011.

[9] T. Ojala and M. Pietikainen, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 24(7):971-988, 2002.

[10] H.R. Bittencourt and R.T. Clarke, “Use of Classification and Regression Trees (CART) to Classify Remotely-Sensed Digital Images”, IEEE Conference on Geoscience and Remote Sensing Symposium, pp. 3751- 3753, 2003.

[11] J. Friedman, T. Hastie and R. Tibshirani, “Additive Logistic Regression: a Statistical View of Boosting”, The Annals of Statistics, pp. 337-374, 2000.

[12] P. Doll´ar, C. Wojek, B. Schiele, and P. Perona, “Pedestrian Detection: A benchmark”, In CVPR, pp. 304–311, 2009.

A Pedestrian Detection Method Based on the HOG-LBP Feature and Gentle AdaBoost Sheng Yang, Xianmei Liao, UK Borasy

560