1
Object Recognition as Ranking Holistic Figure-Ground
Hypotheses
Fuxin Li and Joao Carreira and Cristian Sminchisescu
2
Outline Introduction Method Overview Segment Categorization Segment Post-Processing Experiment Conclusion
3
Introduction Object detector : Top-down approaches
4
Introduction Semantic segmentation results
produced by our algorithm
5
Introduction Ideally segment1. Can model entire object
2. At least sufficiently distinct parts of them
6
7
Introduction Constrained Parametric Min Cuts
algorithm (CPMC) [6]
8
Method Overview
This paper focus
CPMC
Number of segment
9
Method Overview Recognition framework1. Segment categorization
2. Segment post-processing
10
Segment categorization1. Scoring function2. Sort3. Combine high-rank segment
11
12
Segment post-processing
COW
13
Segment Categorization Multiple Segment and Features Learning Scoring Functions with
Regression Learning the Kernel Hyperparameters Compare with Structural SVM
14
Multiple Segment and Features
Model object appearance : 1. Extracted four bag of words of SIFT 2. Two on foreground 3. Two on Background, aim to improve
recognition
15
Multiple Segment and Features
Encode shape information :1. A bag of word of local sharp
contexts [2] : measure similarity between shapes
2. Three pyramid HOGs [5] : classifying images by the object
categories they contain
16
Multiple Segment and Features
Chi-square kernel :
Computed from each histogram feature and use a weighted sum of such kernel for regression
17
Learning Scoring Functions with Regression
: Image I with ground truth segments
: Segmentation algorithm provides a set
of segment : Denote the K object
categories : Indicator function
18
Learning Scoring Functions with Regression
Quality function :
Measure overlap with all denote
the value for and is the maximal overlap with ground truth segments belonging to , and do not appear
19
Learning Scoring Functions with Regression
Learn the function for each :
use nonlinear SVR(Support Vector Regression) to regress against ,the features extracted from
20
Learning Scoring Functions with Regression
Use kernel trick : : support vector from training set : obtained by the SVR optimizer : maximal score of the
segment : final class of the
segment
21
Learning the Kernel Hyperparameters
Fundamental equation (3) is infeasible to estimate all kernel hyperparameter via grid search
Use subset of data comprised segments that best overlap each ground truth segment
22
Compare with Structural SVM The structural SVM(in [3])
formulation for sliding window prediction is :
23
Connections with Structural SVM
Our algorithm VS Structural SVM Structural SVM score the bounding box
and Our algorithm score the segment
Important advantage1. Guarantee the highest rank for the
ground truth2. Correct ranking for all segment
24
Segment Post-Processing Simple decision rule : avoid the post-
processing and direct choose the segment , cannot detect multiple objects
Our methodology : weighted consolidation of segment and sequential interpretation strategy
25
Segment Post-Processing
26
Segment Post-Processing To decide which segments to
combine
Consider segment with intersection
> 0.75 for combination
27
Segment Post-Processing1. Highest-scoring segment as seed2. Group segments that intersect it3. Generated a final mask 4. Proceed with the next higher rank
segment5. Choose segment that are not
overlapping with 3
28
Segment Post-Processing Generate the score for the pixels in
the mask by (9), only pixels with score > 0.65 are displayed in the mask.
29
Experiments Classification : Caltech-101 Detection : ETHZ Shape classes Segmentation : VOC 2009
30
Classification
31
Classification
32
Detection
33
Detection
34
Detection
35
OWT-UCM Masks
36
Segmentation
Bounding box
37
Conclusion CPMC Categorization Post-
processing Achieve good performance Future work : improve the scalability
to be able to process hundreds of thousands of image
Top Related