Object Recognizing We will discuss: Features Classifiers Example ‘winning’ system.
05 - Recognizing a Large Number of Object Classes
Transcript of 05 - Recognizing a Large Number of Object Classes
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
1/27
LargeScale
Recognition and
Retrieval
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
2/27
What does the world look like?
High level image statistics -Object Recognition for large scale searchFocus on scaling rather than
understanding image
Scaling to billions of images
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
3/27
Content-Based ImageRetrieval
Variety of simple/hand-designed cues:
Color and/or Texture histograms, Shape,PCA, etc.
Various distance metrics
Earth Movers Distance (Rubner et al.98)
QBIC from IBM (1999)
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
4/27
Some vision techniques forlarge scale recognition
Efficient matching methods
Pyramid Match Kernel
Learning to compare images
Metrics for retrieval
Learning compact descriptors
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
5/27
Some vision techniques forlarge scale recognition
Efficient matching methods
Pyramid Match Kernel
Learning to compare images
Metrics for retrieval
Learning compact descriptors
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
6/27
Matching features in-category level recognition
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
7/27
Comparing sets of localfeatures
Previous strategies: Match features
individually, vote onsmall sets to verify
[Schmid, Lowe, Tuytelaars et
al.]
Explicit search for one-to-one correspondences
[Rubner et al., Belongie etal., Gold & Rangarajan,
Wallraven & Caputo, Berg etal., Zhang et al.,]
Bag-of-words: Comparefrequencies ofprototype features
[Csurka et al., Sivic &Zisserman, Lazebnik & Ponce]:S lid e cre d it K riste n G ra u m a n
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
8/27
Pyramid match kernel
o p tim a lp a rtia lm a tch in g
: (ptimal match O m3): ( )yramid match O mL= #m features= #L levels in pyramid
[ & ,Grauman Darrell ICCV ]2005
:Slide credit Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
9/27
Pyramid match: main idea
descriptr space
Feature space partitions serve to match the localdescriptors within
successively wider.regions
:Slide credit Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
10/27
Pyramid match: main idea
H isto g ra m in te rse ctio nco u n ts n u m b e r o fp o ssib le m a tch e s a t a
.g ive n p a rtitio n in g:S lid e cre d it K riste n G ra u m a n
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
11/27
Computing the partial matching
Earth Movers Distance [Rubner, Tomasi, Guibas 1998]
Hungarian method [Kuhn, 1955]
Greedy matching
Pyramid match
for sets with features of dimension
[Grauman and Darrell, ICCV 2005]
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
12/27
Recognition on the ETH-80
Recogn
itiona
ccuracy
(%)
Tes
tingtime( s
)
Mean number of features per set (m) Mean number of features per set (m)
ComplexityKernel
Pyramid match
Match [Wallraven et al.]
Slide credit: Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
13/27
Spatial Pyramid Match Kernel, , , .Lazebnik Schmid Ponce 2006
-Dual spacePyramid Matching., .Hu et al 2007
Representing Shape with aPyramid Kernel
& , .Bosch Zisserman 2007
=L 0 =L 1 =L 2
Pyramid match kernel: examples ofextensions and applications by other
groups
cenerecognition haperepresentation edical imageclassification
:Slide credit Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
14/27
wave it downSingle View Human Action
Recognition using Key Pose, & , .Matching Lv Nevatia 2007
-Spatio temporalPyramid Matching for Sports, ., .Videos Choi et al 2008
From OmnidirectionalImages to Hierarchica
,Localization Murillo. .et al 2007
:Pyramid match kernel examples of extensions and applications by other
groups
ction recognition ideo indexing obotlocalization:Slide Kristen Grauma
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
15/27
Some vision techniques forlarge scale recognition
Efficient matching methods
Pyramid Match Kernel
Learning to compare images
Metrics for retrieval
Learning compact descriptors
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
16/27
Learning how to compare images
dissimilar
similar
Exploit( )dis similarity
constraints toconstruct more
useful distancefunction
Number of existingtechniques for
metric learning[Weinberger et al. 2004,Hertz et al. 2004, Frome etal. 2007, Varma & Ray2007, Kumar et al. 2007]
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
17/27
Example sources of similarity constraints
Partially labeled imagedatabases
Fully labeled imagedatabases
Problem-specificknowledge
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
18/27
Locality Sensitive Hashing(LSH)
Gionis, A. & Indyk, P. & Motwani, R. (199Take randomprojections of dataQuantize each projection with few bits
0
1
0
1 0
1
101
Descriptor in
high D space
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
19/27
Fast Image Search for Learned Metrics
Jain, Kulis, & Grauman, CVPR 2008
Less likely to split pairs like those
with similarity constraint
More likely to split pairs like those
with dissimilarity constraint
h( ) = h( ) h( ) h( )
Slide : Kristen Grauman
Learn a Malhanobis metric for LSH
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
20/27
Results: Flickr dataset
slower search faster search
30% of data 2% of data
Error
rate
18 classes, 5400 imagesCategorize scene based on
nearest exemplars
Base metric: Ling &Soattos Proximity
Distribution Kernel (PDK)
Query time:
Slide : Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
21/27
Results: Flickr dataset
slower search faster search
30% of data 2% of data
Error
rate
18 classes, 5400 imagesCategorize scene based on
nearest exemplars
Base metric: Ling &Soattos Proximity
Distribution Kernel (PDK)
Query time:
Slide : Kristen Grauman
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
22/27
Some vision techniques forlarge scale recognition
Efficient matching methods Pyramid Match Kernel
Learning to compare images Metrics for retrieval
Learning compact descriptors
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
23/27
Semantic Hashing
Address Space
Semanticallysimilar
images
Query address
QueryImage
Binarycode
Images in database
[ & , ]Salakhutdinov Hinton 2007 for text documents
Quite different( )to a conventional
randomizing hash
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
24/27
of
semantic hash function
QueryImage
.3 RBM
ComputeGist
Binary code
Gist descriptor
Image 1
Semantic Hash
Retrieved images
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
25/27
Learn mapping
Neighborhood Components Analysis [Goldberger etal., 2004]
Adjust model parameters to move:
Points ofSAME class closer Po in ts o f D IFF E R E N T cla ss
a w a yPo in ts in co d e sp a ce
LabelMe retrieval
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
26/27
LabelMe retrievalcomparison
Size of retrieval
set
%
of
50
true
ne
ig
hb
ors
in
re
tr
ie
va
l
se
t
, ,0 2 000 10 000,20 0000
32-bit learnedcodes do aswell as 512-dim real-valuedinputdescriptor
Learningmethodsoutperform
LSH
-
8/8/2019 05 - Recognizing a Large Number of Object Classes
27/27
Review: constructinga good metric from data
Learn the metric from training data
:Two approaches that do this
, , & , :Jain Kulis Grauman CVPR 2008 Learn Malhanobis.distance for LSH
, , , :Torralba Fergus Weiss CVPR 2008 Directly learn mapping.from image to binary code
( )Use Hamming distance binary codes for speedLearning metric really helps over plain LSH
Learning only applied to metric not