Student: Yao-Sheng WangAdvisor: Prof. Sheng- Jyh Wang

ARTICULATED HUMAN DETECTION

1

Department of Electronics Engineering

National Chiao Tung UniversityHsinchu, Taiwan

1Vision Lab 2012

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

2

Introduction Motivation Challenge Representative Works Potential Problems Target

Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

3

4

Why we care about human detection? We are human beings!

Wide range of applications: Automotive safety Surveillance system

Indoor care Crime alert

Human-Computer Interface … etc.

MOTIVATION

Introduction Motivation Challenge Representative Works Potential Problems Target

Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

5

6

What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation

CHALLENGE

7

What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation

CHALLENGE

8

What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation

CHALLENGE

9

What makes human detection so difficult? Illumination condition Cluttered background Change of viewpoints Occlusion Wearing difference Diversity of human Pose variation

CHALLENGE

10

Progress on “Machine Learning” technologyHandle more general and complicate cases.

Definition: “Articulated Human Detection”.

CHALLENGE

Introduction Motivation Challenge Representative Works Potential Problems Target

Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

11

12

Deformable Part Model Root filter (mask). Part filter (mask). Penalty function.

REPRESENTATIVE WORKS (I)

[P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable part model. In CVPR, 2008.]

13

Pose-let:

REPRESENTATIVE WORKS (II)

[Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. In ICCV, 2009.].

Introduction Motivation Challenge Representative Works Potential Problems Target

Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

14

15

Problems: System complexity increased with the complexity of human poses. More detectors needed.

Exhaustive search. Sliding window method + Image pyramid.

Both problems leads to unacceptable speed for applications in real life.

POTENTIAL PROBLEMS

Introduction Motivation Challenge Representative Works Potential Problems Target

Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

16

17

Target in the thesis: Propose a detection scheme with acceptable detection speed in dealing with highly intra- class variation from the change of pose and viewpoint.

TARGET

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

18

19

Better features: Cheap to compute and capture crucial information at the same time. Ex: HOG.

Better classifiers: Linear classifiers.

Ex: Adaboost, Linear-SVM and Random-forests.

Better prior knowledge: Ex: Information about ground plane.

RELATED WORKS

20

Cascades: Cascade the part filters to reduce the searching regions.

RELATED WORKS

[P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with Deformable Part Models. In CVPR, 2010.]

21

Discard non-promising hypotheses. Class-dependent:

Branch and bound. (CVPR, 2008) Class-independent:

What is an object? (CVPR, 2010) Closure boundary, different appearance or salience.

Segmentation as selective search. (ICCV, 2011)

RELATED WORKS

Start

22

Feature response approximation: Feature approximation in testing step. Feature approximation in training step.

RELATED WORKS

[R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at 100 frames per second. In CVPR, 2012.]

[P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

23

24

Recall the memory of the first problem: System complexity increased with the complexity of human poses (include variation of viewpoints).

How can we break the relation between the complexity of system and the one of human poses? Choose stable features or body parts for detection.

IDEA

25

Better prior knowledge:

IDEA

26

Recall the memory of the second problem: Exhaustive search.

“Sliding Window” + “Image Pyramid”. How can we reduce the searching region?

Detect the common feature among these parts.

Use the cumulative characteristic of the feature to handle the variation of scale.

IDEA

27

Common feature Body parts consist of combination of two edge segments.

Cumulative characteristic Edge detector with fixed size + Combination.

IDEA

28

The previous works focus on reducing the searching regions. Specifically against “Exhaustive Search”.

Our method starts from breaking the relation between complexity of system and that of poses. Then, use the common feature and cumulative characteristic to cut down the searching space.

COMPARISON

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

29

30

Fast Part Detection

Part Combinati

on

Combination

Refinement

SYSTEM BLOCK

Bottom-up system:

31

Fast Part Detection

Part Combinati

on

Combination

Refinement

SYSTEM BLOCK

Bottom-up system:

32

Steps: Detection of edge candidates. Production of part candidates. Refinement of part candidates.

FAST PART DETECTION

33

Detection and combination of segments (9 orientations).

DETECTION OF PART CANDIDATES

34

Constraints on combination of edges. Orientation, length ratio and color symmetry.

PRODUCTION OF PART CANDIDATES

Neighbor orientation consideration

35

HOG feature + Random forest training

REFINEMENT OF PART CANDIDATES

Feature = [Length Orientation HOG_features]

feature134

feature33

feature2

? ?

feature400

36

Fast Part Detection

Part Combinati

on

Combination

Refinement

SYSTEM BLOCK

Bottom-up system:

37

Problem: No information about the classes of the limbs

due to the low resolution of images or variation from hand gestures or appearance of shoes...etc.

Need another step to refine the combinations. What information left?

Head-shoulder or head-torso.

PART COMBINATION

38

Any possibility for us to estimate the position and orientation of head-torso based 　　　 on the architecture of current combinations?

PART COMBINATION

39

Problem: How to select body parts belong to specific human from lots of part candidates?

Too much possibilities for exhaustive search.

PART COMBINATION

40

Clues for reducing the number of possible combinations. Center distance, length ration or width ratio between two parts.

Combination with the number of parts more than four.

PART COMBINATION

41

Conclusion for the clues mentioned in the previous slide. Too complicate to combine the parts for the whole body.

Start from low-level combination of parts to reveal the benefits of physical constraints.

Break the problems into two levels. Low-level combination. High-level combination.

PART COMBINATION

42

How far can we reach for low-level combination? 4-parts combination = lower body.

LOW-LEVEL COMBINATION

43

False alarm exists.

Joints relative position + Random Forest

LOW-LEVEL COMBINATION

feature134

feature33

feature2

? ?

feature400

44

Combination between the arms, legs, lower bodies and uncombined single parts from the low-level combination step. Upper bound of the number of combination:

HIGH-LEVEL COMBINATION

45

Fast Part Detection

Part Combinati

on

Combination

Refinement

SYSTEM BLOCK

Bottom-up system:

46

Pose prediction. Detection with DPM detector.

COMBINATION REFINEMENT

47

Feature: Relative size ratio and positions between low-level combinations and architecture of each low-level combination.

Random Forest.

POSE PREDICTION

48

Use DPM detector to cover the intra-class variation.

Model:

DETECTION WITH DPM DETECTOR

49

Much stronger than information of limbs. Head-shoulder to head-torso. Start from head-torso to combine limbs back.

USAGE OF HEAD-SHOULDER INFORMATION

50

SYSTEM ILLUSTRATION

Edge Candidates

Part Candidates

Low Level Part Combine

Pose PredictionHead-Torso Detection

High Level Part Combine

Part Detector

Parts

Low Level Combination

Result ofDetection

High Level Combination

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

51

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

52

Introduction Related Works Idea Proposed Method Experimental Results Conclusion Reference

OUTLINE

53