1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval...
1Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Shape
Optimal invariant metrics for shape retrieval
Michael Bronstein
Department of Computer ScienceTechnion – Israel Institute of Technology
2Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
3Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Tagged shapes
Shapes withoutmetadata
Man, person, humanPersonText search
Content-based search
3D warehouse
4Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Outline
Feature descriptor
Geometric words
Bag of words?
5Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Invariance
Local geodesic distance histogram
Rigid Inelastic Topology
Gaussian curvature
Scale
Wang, B 2010
Heat kernel signature (HKS)
Scale-invariant HKS (SI-HKS)
6Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernels
Heat kernel is a fundamental solution of the heat equation with
point heat source at (heat value at point after time )
Initial conditions: heat distribution at time
Solution : heat distribution at time
Heat equation governs heat propagation
on a surface
7Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernel signature
Sun, Ovsjanikov & Guibas SGP 2009
Multiscale local shape descriptor
can be interpreted as probability of Brownian motion to return to
the same point after time (represents “stability” of the point)
Time (scale)
8Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernel signature
Sun, Ovsjanikov, Guibas SGP 2009Ovsjanikov, BB & Guibas NORDIA 2009
Heat kernel signatures represented in RGB space
9Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Original shape Scaled by
HKS= HKS=
Not scale invariant!
10Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale-invariant heat kernel signature
B, Kokkinos CVPR 2010
Log scale-space
Scaling = shift and multiplicative
constant in HKS
log + d/d
Undo scaling
Fourier transformmagnitude
Undo shift
0 100 200 300-15
-10
-5
0
t0 100 200 300
-0.04
-0.03
-0.02
-0.01
0
t0 2 4 6 8 10 12 14 16 18 20
0
1
2
3
4
=2k/T
11Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Heat Kernel Signature Scale-invariantHeat Kernel Signature
12Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Heat Kernel Signature Scale-invariantHeat Kernel Signature
13Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modeling vs learning
Wang, B 2010
14Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Learning invariance
Positives P Negatives N
T
15Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity learning
with high probability
false positive
false negative
positive
negative
16Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity-preserving hashing
-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1
+1 +1 -1 +1+1 +1 +1 -1+1 +1 +1 +1
with high probability
with low probability
Collision:
= # of distinct bits
Gionis, Indik, Motwani 1999Shakhnarovich 2005
17Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Boosting
Construct 1D embedding
Similarity is approximated by
Downweight pairs with
Upweight pairs with
BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005
-1
+1
18Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Boosting
BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005
Construct 1D embedding
Similarity is approximated by
Downweight pairs with
Upweight pairs with
-1
+1
-1 -1
+1 -1
+1 +1
-1 +1
19Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
SHREC 2010 dataset
20Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
SHREC 2010 datasetBB et al, 3DOR 2010
Total dataset size: 1K shapes (715 queries)
Positives: 10K
Negatives: 100K
21Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
ShapeGoogle with HKS descriptorBB et al, 3DOR 2010
22Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
ShapeGoogle with SI-HKS descriptorBB et al, 3DOR 2010
23Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity sensitive hashing (96 bit)BB et al, 3DOR 2010
24Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
WaldHash
B2, Ovsjanikov, Guibas 2010
Construct embedding by maximizing
Remove pairs with
Downweight pairs with
Upweight pairs with
Early decision
and sample in new pairs into the training set
positive
negative
25Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
30%
B2, Ovsjanikov, Guibas 2010
26Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Incommensurablespaces!
BB, Michel, Paragios CVPR 2010
Cross-modal similarity
Modality 1 Modality 2
How to compare apples to oranges?
Triangular meshes Point clouds
27Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modality 1 Modality 2
with high probability
Cross-modality embedding
BB, Michel, Paragios CVPR 2010
28Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modality 1 Modality 2
Cross-modality hashing
-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1
+1 +1 -1 +1+1 +1 +1 -1+1 -1 -1 +1
with high probability
with low probability
Collision:
BB, Michel, Paragios CVPR 2010
29Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Query
8x8 dimensionalbag of expressions
Database
32-dimensionalbag of words
Cross-representation 3D shape retrieval
1052 shapes
BB, Michel, Paragios CVPR 2010
30Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Me
an
av
erag
e p
rec
isio
n
Number of bits
Retrieval performance
BB, Michel, Paragios CVPR 2010