Comparison of Pedestrian Detection Systems - Seminar ... · 1. Motivation2. Problem Formulation3....
Transcript of Comparison of Pedestrian Detection Systems - Seminar ... · 1. Motivation2. Problem Formulation3....
Comparison of Pedestrian Detection SystemsSeminar: Mobile Human Detection Systems
Felix Stern, 3142747
Universitat HeidelbergInstitut fur Technische [email protected]
03. Februar 2017
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Overview
1 Motivation
2 Problem Formulation
3 Solution Approach
4 Methods
5 Results
6 Conclusion
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 2 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Motivation
[2]
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 3 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Problem Formulation
[2]
Given: (Stereo-)camera on robot, car, ”Google Glasses”
Required: Bounding boxes of detected pedestrians
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 4 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Solution Approach
Single frame of pedestrian areasLower image quality (motion blur, artifacts, light conditions)Possible reflectionsPartially-occluded pedestrians
Two approaches considered:
1 Using mono-camera:
Create Histograms of Oriented Gradients (HOG)
Train Support Vector Machine (SVM)
Use binary classifier for detecting humans in image blocks
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 5 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Solution Approach
Single frame of pedestrian areasLower image quality (motion blur, artifacts, light conditions)Possible reflectionsPartially-occluded pedestrians
Two approaches considered:
1 Using mono-camera:
Create Histograms of Oriented Gradients (HOG)
Train Support Vector Machine (SVM)
Use binary classifier for detecting humans in image blocks
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 5 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Solution Approach / 2
2 Using additional depth information (stereo camera):
bounding boxes of detected objects (pedestrians) are requiredidentify ground plane in the depth mapuse ground plane to evaluate pedestrian hypotheses
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 6 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients
directional change in intensity / color
dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127
dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255
gradient orientation: θ = tan−1( dydx ) ∗ 180
π θ ≈ 63.5
gradient magnitude:√dx2 + dy2
√2552 + 1272 ≈ 285
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients
directional change in intensity / color
dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127
dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255
gradient orientation: θ = tan−1( dydx ) ∗ 180
π θ ≈ 63.5
gradient magnitude:√dx2 + dy2
√2552 + 1272 ≈ 285
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients
directional change in intensity / color
dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127
dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255
gradient orientation: θ = tan−1( dydx ) ∗ 180
π θ ≈ 63.5
gradient magnitude:√dx2 + dy2
√2552 + 1272 ≈ 285
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients / 2
64x128 pixel detection window
window is divided into cells and blocks:
cell: 8x8 pixelblock: 2x2 cells
blocks have 50% overlap
block cells normalized by L2-Hys norm
gradient orientation quantized into 9 bins(each 20◦), magnitude added to bin
1D feature vector by concatenatinghistogram values
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 8 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients / 2
64x128 pixel detection window
window is divided into cells and blocks:
cell: 8x8 pixelblock: 2x2 cells
blocks have 50% overlap
block cells normalized by L2-Hys norm
gradient orientation quantized into 9 bins(each 20◦), magnitude added to bin
1D feature vector by concatenatinghistogram values
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 8 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Histograms of Oriented Gradients / 3
[1]
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 9 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Support Vector Machine
classifier to predict a class
labeled training data are needed
construct maximum-margin-hyperplane
Source: https://upload.wikimedia.org/wikipedia/commons/f/f2/Svm_intro.svg
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 10 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Stereo Vision
orthogonal depth can be calculated using the intercepttheorem
Figure: f : focal length, z : orthogonal depth, X : reconstructed point,x/x ′: vertical image position, O/O ′: camera originSource: http://docs.opencv.org/3.1.0/dd/d53/tutorial_py_depthmap.html
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 11 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Stereo Vision Depth Map
Source: http://www.360doc.com/content/14/0512/16/17164701_376968762.shtml
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 12 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Multiple Pedestrian Detection
ground plane helps in constraining object detection tomeaningful locations
[2]
hypotheses: bounding boxes around pedestrians (used: ISMdetector)
depth cues: evaluate depth inside bounding box of hypothesis
depth map evidence: probability that the ground planegenerated the depth map (Belief Propagation)
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 13 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Multiple Pedestrian Detection / 2
Figure: ground plane in depth maps could be missing (right)
[1]
using ground plane, bounding box and depth cue to verify ifthe hypothesis is valid
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 14 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Results (HOG)
[1]
best cell size for detecting humans: 8x8, block size 2x2
extremities of pedestrians are about 8px wide
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 15 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Performance Comparison
test datasets and metricsdiffer (FP/W vs. FP/I)
used image size in [2]:640x480
in [1], this would result in73x45 detection windows
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 16 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Conclusion
two pedestrian detection systems compared
only single images/frames needed (no series)
simple approach: Histograms of Oriented Gradients
very accurate on tested datasetsdependent on block and cell size
advanced approach: using additional depth information
first extract ground plane, using Belief Propagationuse those information to validate detections
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 17 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Critique
Approach 1 (HOG):
only one pedestrian per window
undefined maximum occlusion rate for detection
pedestrians have all the same dimensions
Approach 2 (additional depth information):
dependent on pedestrian detector
explanation of the ground plane’s effect missing
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 18 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Critique / 2
[2]
Figure: white boxes: true positives, red boxes: false positives
problems with reflections still existing
faraway pedestrians not detected (> 25m)
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 19 / 21
1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion
Sources
Paper 1: Histograms of Oriented Gradients for HumanDetection, Navneet Dalal, Bill Triggs
Paper 2: Depth and Appearance for Mobile Scene Analysis,Andreas Ess, Bastian Leibe, Luc Van Gool
ISM detector: Pedestrian detection in crowded scenes, Leibe,Seemann, Schiele
Image p.3, p.4, p.13, p.16, p.17: Paper 2
Image p.7, p.8, p.14, p.15: Paper 1
Image p.9: https://upload.wikimedia.org/wikipedia/commons/f/f2/Svm_intro.svg
Image p.10: http://docs.opencv.org/3.1.0/dd/d53/tutorial_py_depthmap.html
Image p.11: http://www.360doc.com/content/14/0512/16/17164701_376968762.shtml
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 20 / 21
Stereo Calculation
ϕ = tan(w/2f ), B = B1 + B2 = Z ∗ tan(ϕL) + Z ∗ tan(ϕR)
x−w/2w/2 = tan(ϕL)
tan(ϕ) ,x ′−w/2w/2 = tan(ϕR)
tan(ϕ)
→ Z = B∗w2∗tan(ϕ)∗(x−x ′)
Source: http://dsc.ijs.si/files/papers/s101%20mrovlje.pdf
03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 21 / 21