BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics

7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics

http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 1/47

!""#$%&'()*%+$),'

-.,")/)0%1/$2+'34'5$%/)/26$2)#'7.&%#+'

'!"#$%&'()'#)*+,-)./0.)

Erik SudderthBrown University

Work by E. Sudderth, A. Torralba, W. Freeman, & A. Willsky

IJCV 2008:Describing Visual Scenes using Transformed Objects & Parts

CVPR 2006:Depth from Familiar Objects: A Hierarchical Model for 3D Scenes

NIPS 2005:Describing Visual Scenes using Transformed Dirichlet Processes

Building on work by Y. W. Teh, M. Jordan, M. Beal, & D. Blei

JASA 2006:Hierarchical Dirichlet Processes



8%)/,$,9':$16';."$2'7.&%#+

model

neural

stochastic

recognition

nonparametric

gradient

dynamical

Bayesian

!

Framework for unsupervised discovery oflow-dimensional latent structure frombag of word representations

Algorithms

Neuroscience

Statistics

Vision

!

!! pLSA: Probabilistic Latent Semantic Analysis(Hofmann 2001)

!! LDA: Latent Dirichlet Allocation(Blei, Ng, & Jordan 2003)

!! HDP: Hierarchical Dirichlet Processes(Teh, Jordan, Beal, & Blei 2006)



5$%/)/26$2)#'<$/$26#%1'=/.2%++'

E

[πj ] =

β

G0 ∼ DP(γ , H )

Gj ∼ DP(α, G0)

J groups of data:documents, images,!



>6$,%+%'?%+1)@/),1'A/),26$+%'



8.2)#'B$+@)#'A%)1@/%+C'D@"%/"$E%#+

•!Partition image into ~1,000superpixels

•!Goal: Reduce dimensionality, aggregateinformation spatially –hopefully not

across object boundaries!

Inspired by the successes oftopic modelsfor text data,

some have proposed learning fromlocal image features



8.2)#'B$+@)#'A%)1@/%+C'F,1%/%+1'?%9$.,+'

Affinely AdaptedHarris Corners

Maximally StableExtremal Regions

Linked Sequencesof Canny Edges

•!Some invariance to lighting & pose variations•!Dense, multiscaleover-segmentationof image



!'<$+2/%1%'A%)1@/%'B.2)G@#)/*'

•! !"#$%&'()*+,'-."/#%$-+"0+

"#')1.%2"1+)1)#/3+

•! 4"$56.)+789:::+;"#*+

*'<2"1%#3+='%+>?$)%1-+

•! @%5+)%<,+0)%.6#)+."+

1)%#)-.+!"#$%&'()*+'

appearance of

featureiin image j

2D position of

featureiin image j

SIFT Descriptors

Lowe, IJCV 2004



;6%'H./#&')+')'()9'.I'B$+@)#'H./&+'

Fei-Fei & Perona,CVPR 2005 Sivic, Russell, Efros, Zisserman, & Freeman,ICCV 2005

Topics asvisual themescomposing a

known set of scene categories

Topics asvisual object classeswithin a

(carefully chosen) image collection



F0)9%+')+'0./%'16),'()9+'.I'A%)1@/%+'

•! A";+*"+B+C1";+.,'-+'-+"<)%1+D)1)%.,+%+<&)%#+-C3E+

•! A";+$%13+D'<3<&)-+%1*+.#'<3<&)-+%$+B+&""C'1/+%.E+

,-.'%*/'(/'0*."12'0)'#3$//4/'"5%2/#'"10)'0)6"7'5)+/&#8'

9-/*/'%*/'5%1.'5)*/'0))&#'%!%"&%:&/':.'%+%6;12'

1)16%*%5/0*"7'%1+'-"/*%*7-"7%&'<%./#"%1'5)+/&#='



B$+@)#'JGK%21'>)1%9./$L)M.,'

•! GOAL:Visuallyrecognize andlocalize object categories

•!Robustlylearn appearance models from few examples



Part-Based Models for Objects

Pictorial Structures Fischler & Elschlager, 1973

Generalized Cylinders Marr & Nishihara, 1978

Recognition by Components Biederman, 1987

Constellation Model Perona, Weber, Welling,

Fergus, Fei-Fei, 2000 to!

Efficient MatchingFelzenszwalb & Huttenlocher, 2005

Discriminative PartsFelzenszwalb, McAllester,

Ramanan, 2008 to!



>.@,M,9'JGK%21+'N'=)/1+'

How many parts? How many objects?



O%,%/)MP%'7.&%#'I./'JGK%21+'

For each image:Sample a reference position

For each feature:

!!Randomly choose one part!!Sample from that part’s feature distribution



JGK%21+')+'<$+1/$G@M.,+'

•!Parts are defined by parameters, whichencode distributions on visual features:

Featureappearance

Featureposition

Pr(appearance | part) Pr(position | part)

•!Objects are defined bydistributionson the

infinitely many potential part parameters:

Pr(part)



!'-.,")/)0%1/$2'=)/1Q()+%&'7.&%#'

4 Images 16 Images 64 Images

# images

# p

a r t s



O%,%/)#$L$,9'!2/.++'>)1%9./$%+'

Can we transfer knowledge from one object category to another?



8%)/,$,9'D6)/%&'=)/1+'

•! FDG)<.-+%#)+"H)1+&"<%&&3+-'$'&%#+'1+%55)%#%1<)+

•! I'-<"=)#+ 6%*0#+-,%#)*+%<#"--+<%.)/"#')-+ –A";+$%13+.".%&+5%#.-+-,"6&*+;)+-,%#)E+

–A";+$%13+5%#.-+-,"6&*+)%<,+<%.)/"#3+6-)E+



5$%/)/26$2)#'<='JGK%21'7.&%#'

G0

J

G!"

#"

H

R

Nw

$"

%"

&"

v

L

G0

1G G2



5$%/)/26$2)#'<='JGK%21'7.&%#'

G0

J

G!"

#"

H

R

Nw

$"

%"

&"

v

L

G0

1G G2

Discrete Data:

Teh et. al., 2004



>6$,%+%'?%+1)@/),1'A/),26$+%'



D6)/$,9'=)/1+C'RS'>)1%9./$%+'

•!Caltech 101 Dataset (Li & Perona)•!Horses (Borenstein & Ullman)•!Cat & dog faces (Vidal-Naquet & Ullman)

•!Bikes from Graz-02 (Opelt & Pinz)•!Google!



B$+@)#$L)M.,'.I'D6)/%&'=)/1+'

Pr(appearance | part)

Pr(position | part)



B$+@)#$L)M.,'.I'D6)/%&'=)/1+'


Pr(position | part)



B$+@)#$L)M.,'.I'=)/1'<%,+$M%+'

Hierarchical Clustering of Pr(part | object)



<%1%2M.,';)+T'



<%1%2M.,'?%+@#1+'

6 Training Images per Category(ROC Curves)

Shared Parts more accurate than

Unshared Parts

Modeling feature positions

improves shared detection, but

hurts unshareddetection



<%1%2M.,'?%+@#1+'

6 Training Images per Category(ROC Curves)

Detection vs. Training Set Size(Area Under ROC)



D6)/$,9'D$0"#$U%+'7.&%#+'



D2%,%+V'JGK%21+V'),&'=)/1+'

Features

Parts

Objects

Scene



>.,1%E1@)#';/),+I%/'8%)/,$,9'



JGK%21'P+4'B$+@)#'>)1%9./$%+'

•! Assume training data contains object category labels•!Discover underlying visual categories automatically



7@#M"#%'JGK%21'D2%,%+'

•!How many cars are there?•!Where are those cars in the scene?

Standard dependent Dirichlet process models (Gelfand et. al., 2005) inappropriate



D")M)#';/),+I./0)M.,+'

•!Let global DP clusters model objectsin acanonical coordinate frame

•!Generate images via a randomset of transformations:

Parameterized family

of transformations

Shift cluster from canonical

coordinate frame to object

location in a given image

Layered Motion Models (Darrell & Pentland 1991, Wang & Adelson 1994, Jojic & Frey 2001)

Nonparametric Transformation Densities (Learned-Miller & Viola 2000)



!';.*'H./#&C''()/+'N'(#.G+'



;/),+I./0%&'<$/$26#%1'=/.2%++'

G0G0

1 2 3G G G

J

G j!"

#"

H

R

Mixture

Parameters Transformations

Nv

$" %"



F0"./1),2%'.I';/),+I./0)M.,+'

HDP TDP



>.@,M,9'N'8.2)M,9'JGK%21+''

•!How many cars are there?•!Where are those cars in the scene?

Dirichlet Processes

Transformations



B$+@)#'D2%,%';<=''

G0#"

H

R

GlobalDensityObject category Part size & shape

Transformation prior

'"

F(

J

G j!" TransformedDensitiesObject category Part size & shape

Instance locations

w v

2D ImageFeatures AppearanceLocation

N

$"

%"o



D1/%%1'D2%,%'B$+@)#'>)1%9./$%+'



D1/%%1'D2%,%'D%90%,1)M.,+'



D%90%,1)M.,'=%/I./0),2%'



WE1%,+$.,C'X<'D2%,%+

•!Segmentation easier in 3D•!Identifying known objectsregularizes depth estimation

Red

Green

Office

Scene

Far

Near



X<'D1/@21@/%'I/.0'D1%/%.

Reference (left) Image Potential Matches Depth Densities

Overhead View

Depth =Disparity

)"



O/%%&*'<%"16'W+M0)1%+

Reference (left) Image Potential Matches Depth Densities

Red Far

Green Near



X<';/),+I./0%&'<=C'JY2%'D2%,%+

Computer Screen

DeskBookshelvesBackground



D1%/%.';%+1'F0)9%

Simultaneousobject recognition & coarse3D reconstruction

BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics

Documents

Transcript of BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics