Matching and Recognition Matching and Recognition in 3Din 3D

Moving from 2D to 3DMoving from 2D to 3D

• Some things harderSome things harder– Rigid transform has 6 degrees of Rigid transform has 6 degrees of

freedom vs. 3freedom vs. 3– No natural parameterization (e.g. No natural parameterization (e.g.

running FFT or convolution is trickier)running FFT or convolution is trickier)

• Some things easierSome things easier– No occlusion (but sometimes missing No occlusion (but sometimes missing

data instead)data instead)– Segmenting objects often simplerSegmenting objects often simpler

Matching / Recognition in 3DMatching / Recognition in 3D

• Methods from 2DMethods from 2D– Feature detectorsFeature detectors– HistogramsHistograms– PCA (eigenshapes)PCA (eigenshapes)– Graph matching, interpretation treesGraph matching, interpretation trees

Matching / Recognition in 3DMatching / Recognition in 3D

• Other methods (may also apply to Other methods (may also apply to 2D)2D)– Identifying objects in scene: spin Identifying objects in scene: spin

imagesimages– Finding a single object in a database:Finding a single object in a database:

shape distributionsshape distributions– Aligning pieces of the same object:Aligning pieces of the same object:

iterative closest points (ICP)iterative closest points (ICP)

3D Identification Using Spin 3D Identification Using Spin ImagesImages

• Spin images: Johnson and HebertSpin images: Johnson and Hebert

• ““Signature” that captures local shapeSignature” that captures local shape

• Similar shapes Similar shapes similar spin images similar spin images

3D Identification Using Spin 3D Identification Using Spin ImagesImages

• General approach:General approach:– Create database of many objects, Create database of many objects,

many spin images for each objectmany spin images for each object– For each point in unknown scene, For each point in unknown scene,

compute spin imagecompute spin image– Find matches in databaseFind matches in database– Compare object in database to sceneCompare object in database to scene

Computing Spin ImagesComputing Spin Images

• Start with a point on a 3D modelStart with a point on a 3D model

• Find (averaged) surface normal at that Find (averaged) surface normal at that pointpoint

• Define coordinate system centered at this Define coordinate system centered at this point, oriented according to surface point, oriented according to surface normal and two (arbitrary) tangentsnormal and two (arbitrary) tangents

• Express other points (within some Express other points (within some distance) in terms of the new coordinatesdistance) in terms of the new coordinates

Computing Spin ImagesComputing Spin Images

• Compute histogram of locations of Compute histogram of locations of other points, in new coordinate other points, in new coordinate system, ignoring rotation around system, ignoring rotation around normal:normal: n̂ p n̂ p

n̂p n̂p

Computing Spin ImagesComputing Spin Images

Spin Image ParametersSpin Image Parameters

• Size of neighborhoodSize of neighborhood– Determines whether local or global shapeDetermines whether local or global shape

is capturedis captured– Big neighborhood: more discriminatory powerBig neighborhood: more discriminatory power– Small neighborhood: resistance to clutterSmall neighborhood: resistance to clutter

• Size of bins in histogram:Size of bins in histogram:– Big bins: less sensitive to noiseBig bins: less sensitive to noise– Small bins: captures more detail, less storageSmall bins: captures more detail, less storage

Spin Image ResultsSpin Image Results

Range ImageRange Image Model in DatabaseModel in Database

Spin Image ResultsSpin Image Results

Detected ModelsDetected Models

Shape DistributionsShape Distributions

• Osada, Funkhouser, Chazelle, and Osada, Funkhouser, Chazelle, and DobkinDobkin

• Compact representation for entire 3D Compact representation for entire 3D objectobject

• Invariant under translation, rotation, Invariant under translation, rotation, scalescale

• Application: search engine for 3D Application: search engine for 3D shapesshapes

Computing Shape Computing Shape DistributionsDistributions

• Pick Pick nn random pairs of points on the random pairs of points on the objectobject

• Compute histogram of distancesCompute histogram of distances

• Normalize for scaleNormalize for scale

3D Model3D Model ShapeShapeDistributionDistribution

Random Random samplingsampling

Comparing Shape Comparing Shape DistributionsDistributions

SimilaritySimilarityMeasureMeasure

3D3DModelModel

ShapeShapeDistributionDistribution

Shape Distributions for Simple Shape Distributions for Simple ShapesShapes

Robustness ResultsRobustness Results

7 Mugs

7 Missiles

Classification ResultsClassification Results

Classification ResultsClassification Results

3D Alignment3D Alignment

• Alignment of partially-overlappingAlignment of partially-overlapping(pieces of) 3D objects(pieces of) 3D objects

• Application: building a complete 3D Application: building a complete 3D model given output of stereo, 3D model given output of stereo, 3D scanner, etc.scanner, etc.

• One possibility: spin imagesOne possibility: spin images

• Another possibility: correspondences Another possibility: correspondences from user inputfrom user input

Iterative Closest Points (ICP)Iterative Closest Points (ICP)

• Besl & McKay, 1992Besl & McKay, 1992

• Start with rough guess for alignment Start with rough guess for alignment from:from:– Tracking position of scannerTracking position of scanner– Spin imagesSpin images– User inputUser input

• Iteratively refine transformIteratively refine transform

• Output: high-quality alignmentOutput: high-quality alignment

ICPICP

• Assume Assume closestclosest points correspond to points correspond to each other, compute the best each other, compute the best transform…transform…

ICPICP

• … … and iterate to find alignmentand iterate to find alignment

• Converges to Converges to somesome local minimum local minimum

• Correct if starting position “close Correct if starting position “close enough“enough“