Non-linear Distribution of Descriptors Descriptors are non-linearly distributed.
Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval
description
Transcript of Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval
![Page 1: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/1.jpg)
Selecting Distinctive 3D Shape Descriptors for Similarity
Retrieval
Philip Shilane and Thomas Funkhouser
![Page 2: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/2.jpg)
Large Databases of 3D Shapes
Mechanical CAD(National Design Repository)
Molecular Biology(Protein Databank)
Computer Graphics(Princeton Shape Benchmark)
![Page 3: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/3.jpg)
Shape Retrieval
3D Model Model
Database
BestMatche
s
![Page 4: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/4.jpg)
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
![Page 5: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/5.jpg)
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
![Page 6: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/6.jpg)
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Using many local descriptors is slow.
![Page 7: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/7.jpg)
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Using many local descriptors is slow.Many descriptors do
not represent distinguishing parts.
![Page 8: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/8.jpg)
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Focusing on the distinctive regions improves retrieval time and accuracy.
![Page 9: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/9.jpg)
Related Work
Selecting Local Descriptors• Random
Mori 2001Frome 2004
![Page 10: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/10.jpg)
Related Work
Selecting Local Descriptors• Random• Salient
Gal 2005Lee 2005Frintrop 2004
![Page 11: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/11.jpg)
Related Work
Selecting Local Descriptors• Random• Salient• Likelihood
Johnson 2000Shan 2004
![Page 12: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/12.jpg)
Distinction = Retrieval Performance
QueryDescriptors
The distinction of each local descriptor is based on how well it retrieves shapes of the correct class.
Retrieval Results
![Page 13: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/13.jpg)
Distinction = Retrieval Performance
QueryDescriptors
The distinct descriptors that distinguish between classes are classification dependent.
Retrieval Results
![Page 14: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/14.jpg)
Approach
Descriptors
Distinction
We want a predicted distinction score for each descriptor on the model.
![Page 15: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/15.jpg)
ApproachWe map descriptors into a 1D space where we learn distinction from a training set.
Dis
tinc
tion
1D Parameterization
Descriptors
Distinction
![Page 16: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/16.jpg)
Approach
Descriptors
Distinction
Likelihood of shape descriptors is a 1D function that groups descriptors with similar distinction.
Likelihood Parameterization
![Page 17: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/17.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 18: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/18.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 19: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/19.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 20: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/20.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 21: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/21.jpg)
)(21exp
2)( 1
2
21
xxxdensityt
d
Multi-dimensional normal density [Johnson 2000]
matrix covariance d x d vectorfeaturemean
vectorfeature ldimensiona d
x
Likelihood of Descriptors
![Page 22: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/22.jpg)
Likelihood of Descriptors
)(21)(
))(densityln()(
1
xxxp
xxpt
The likelihood function is proportional to the descriptor density.
matrix covariance d x d vectorfeaturemean
vectorfeature ldimensiona d
x
![Page 23: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/23.jpg)
Map from Descriptors to LikelihoodFlat regions are the most common while wing tips
and the cockpit area are rarer.
Less Likely
More Likely
![Page 24: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/24.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 25: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/25.jpg)
Measuring Distinction
0.33
QueryDescriptors
Evaluation Metric
Evaluate the retrieval performance of every query descriptor.
Retrieval Results
![Page 26: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/26.jpg)
Measuring Distinction
0.33
1.0
QueryDescriptors
Evaluation Metric
Some descriptors are better for retrieval than others.
Retrieval Results
![Page 27: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/27.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 28: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/28.jpg)
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
![Page 29: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/29.jpg)
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
![Page 30: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/30.jpg)
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
![Page 31: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/31.jpg)
Build Distinction FunctionRetrieval performance is averaged within each likelihood bin.
![Page 32: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/32.jpg)
Descriptor DistinctionA likelihood mapping separates descriptors with different retrieval performance.
Less Likely
More Likely
![Page 33: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/33.jpg)
Less Likely
More Likely
Descriptor DistinctionThe most common features are the worst for retrieval.
![Page 34: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/34.jpg)
Predicting Distinction
Distinction Function
Descriptors
Distinction
The likelihood mapping predicts descriptor distinction.
![Page 35: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/35.jpg)
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
![Page 36: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/36.jpg)
Selecting Distinctive DescriptorsThe k most distinctive descriptors with a minimum distance constraint are selected.
Mesh Descriptors DistinctionScores
3 SelectedDescriptors
![Page 37: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/37.jpg)
Matching with Selected Descriptors
k
i
ki
k YXCYX ),(
3D Model Model
Database
BestMatche
s
![Page 38: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/38.jpg)
Results
• Examples of Distinctive Descriptors• Evaluation for Retrieval
![Page 39: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/39.jpg)
Distinctive Descriptor ExamplesDescriptors on the head and neck represent
consistent regions of the models.
![Page 40: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/40.jpg)
Distinctive Descriptor ExamplesDescriptors on the front of the jet are consistent as
opposed to on the wings.
![Page 41: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/41.jpg)
ChallengeThe wheels are consistent features for cars.
![Page 42: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/42.jpg)
Shape Database
• 100 Models in 10 Classes from the Princeton Shape Benchmark
• Models come from different branchesof the hierarchical classification
![Page 43: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/43.jpg)
Shape Descriptors• Mass per Shell Shape Histogram
(SHELLS)Ankerst 1999
• Spherical Harmonics of the Gaussian Euclidean Distance Transform (SHD)
Kazhdan 2003
0.25 0.5 1.0 2.0
Radius of Descriptors Considered
![Page 44: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/44.jpg)
Local vs. Global DescriptorsUsing local descriptors improves retrieval relative to global descriptors.
Global vs Local
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Recall
Prec
isio
n
GlobalAll Local
![Page 45: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/45.jpg)
Focus on Distinctive DescriptorsUsing a small number of distinct descriptors maintains retrieval performance while improving retrieval time.
Global vs Local
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Recall
Prec
isio
n
GlobalAll Local10 Distinct3 Distinct
![Page 46: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/46.jpg)
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Johnson 2000 (DB)
Random
![Page 47: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/47.jpg)
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Johnson 2000(Model)Johnson 2000 (DB)
Random
![Page 48: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/48.jpg)
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Distinctive
Johnson 2000(Model)Johnson 2000 (DB)
Random
Distinction improves retrieval more than other techniques.
![Page 49: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/49.jpg)
Conclusion
• Method to select distinctive descriptors
• Distinctive descriptors can improve retrieval
• Mapping descriptors through likelihood and learned retrieval performance to distinction is better than other alternatives
• Distinction is independent of type of descriptor
![Page 50: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/50.jpg)
Future Work
• Explore other definitions of likelihood including mixture models
![Page 51: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/51.jpg)
Future Work
• Explore other definitions of likelihood including mixture models
• Consider non-likelihood parameterizations
![Page 52: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/52.jpg)
Future Work
• Explore other definitions of likelihood including mixture models
• Consider non-likelihood parameterizations
• Combine descriptors while accounting for deformation [Funkhouser and Shilane, SGP]
![Page 53: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/53.jpg)
Acknowledgements
Szymon RusinkiewiczJoshua PodolakPrinceton Graphics Group
Funding Sources:National Science Foundation Grant CCR-0093343
and Grant 11S-0121446Air Force Research Laboratory Grant FA8650-04-1-
1718
![Page 54: Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval](https://reader030.fdocuments.in/reader030/viewer/2022020418/56815f82550346895dce8883/html5/thumbnails/54.jpg)
The End