Segmentation and Perceptual Grouping

The image of this cube contradicts the optical image

Stochastic Completion Fields

Stochastic Completion Fields

Stochastic Completion Fields

Hough Transform

Hough Transform

Shortest Path

Curve Evolution

Curve Evolution

Thresholding

Histogram

0 50 100 150 200 250

0

200

400

600

800

1000

1200

Thresholding

Thresholding

125

15699

Local Uncertainty

Global Certainty

Local Uncertainty

Global Certainty

Minimum Cut

Texture Examples

Coarse Measurements for Texture

Hilbert Transform

-100 -50 0 50 100-0.01

-0.005

0

0.005

0.01

-100 -50 0 50 1000

0.004

0.008

0.012

0.016

Gaussian Hilbert Transform

1 ( )[ ( )]

f x dxH f x

x y

Filter Bank

Textons

image textons

textonassignment

Normalized Cuts

The Pixel Graph

ij w

Couplings

reflect intensity similarity

{}ij w

Hierarchical Graph

Normalized-Cut Measure

Si

Siui 0

1

Normalized-Cut Measure2( ) ( )ij i j

i j

E S w u u

Si

Siui 0

1

( ) ij i jN S w u u

( )( )

( )

E SS

N S

Minimize:

Recursive Coarsening

iuju

Recursive Coarsening

iuju

lU

kU

Representative subset

1 2( , ,..., )NU U U U

Recursive Coarsening

iuju

2

11

2

.

..

Nn

u

U

P

u

Uu

U

For a salient segment :

P ( )n N ,sparse interpolation matrix

kU

lU

Weighted Aggregation

ijwi

jjlp

aggregate k aggregate l

ijk ik jli j

l p pwW

klWikp

Segment Detection

A Chicken and Egg Problem

Problem:Coarse measurements mix neighboring statistics

Solution: support of measurements is determined as the segmentation process proceed

Adaptive vs Rigid Support

AveragingSWA Geometric

Adaptive vs Rigid Support

InterpolationSWA Geometric

AggregateShape

Boundary Integrity

Sharpen the Aggregates

Top-down Sharpening:• Expand core• Sharpen boundaries

Experiments

• Our SWA algorithm (CVPR’00 + CVPR’01) run-time: 5-10 seconds.

• Normalized cuts (Shi and Malik, PAMI’00; Malik et al., IJCV’01) run-time: about 10-15 minutes.

Software courtesy of Doron Tal, UC Berkeley.

images on a pentium III 1000MHz PC:200 200

Use of Multiscale Variance Vector

Use of Multiscale Variance Vector

Variance: Avoid Mixing

aggregation Moving window

Texture Aggregation

Fine (homogeneous) Coarse (heterogeneous)

Squirrel

Leopard

Texture Composition

WithVariance only

With all measures

Tiger

Butterfly

Bird

Owl

Lion Cub

Polar Bear

Segmentation with Ncuts(Malik et al).

Complexity

• Every level contains about half the nodes of the previous level:

• Total #nodes double #pixels

• All connections are local, cleaning small weights

• Top-down sharpening: constant number of levels

• Linear complexity

• Implementation: 5 seconds for 400x400