Joint Optimization of Segmentation and Appearance...
Transcript of Joint Optimization of Segmentation and Appearance...
![Page 1: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/1.jpg)
Joint Optimization of Segmentation and AppearanceModels
David Mandle, Sameep Tandon
April 29, 2013
David Mandle, Sameep Tandon (Stanford) April 29, 2013 1 / 19
![Page 2: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/2.jpg)
Overview
1 Recap: Image Segmentation2 Optimization Strategy3 Experimental
David Mandle, Sameep Tandon (Stanford) April 29, 2013 2 / 19
![Page 3: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/3.jpg)
Recap: Image Segmentation Problem
Segment an image into foreground and background
Figure: Left: Input image. Middle: Segmentation by EM (GrabCut). Right:Segmentation by the method covered today
David Mandle, Sameep Tandon (Stanford) April 29, 2013 3 / 19
![Page 4: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/4.jpg)
Recap: Image Segmentation as Energy Optimization
Recall Grid Structured Markov Random Field:
Latent variables xi ∈ {0, 1} corresponding to foreground/backgroundObservations zi . Take to be RGB pixel valuesEdge potentials Φ(xi , zi ), Ψ(xi , xj)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 4 / 19
![Page 5: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/5.jpg)
Recap: Image Segmentation as Energy Optimization
The Graphical Model encodes the following (unnormalized) probabilitydistribution:
David Mandle, Sameep Tandon (Stanford) April 29, 2013 5 / 19
![Page 6: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/6.jpg)
Recap: Image Segmentation as Energy Optimization
Goal: find x to maximize P(x , z) (z is observed)
Taking logs:
E (x , z) =∑i
φ(xi , zi ) +∑i ,j
ψ(xi , xj)
Unary potential φ(xi , zi ) encodes how likely it is for a pixel or patch yito belong to segmentation xi .
Pairwise potential ψ(xi , xj) encodes neighborhood info aboutpixel/patch segmentation labels
David Mandle, Sameep Tandon (Stanford) April 29, 2013 6 / 19
![Page 7: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/7.jpg)
Recap: Image Segmentation as Energy Optimization
Goal: find x to maximize P(x , z) (z is observed)
Taking logs:
E (x , z) =∑i
φ(xi , zi ) +∑i ,j
ψ(xi , xj)
Unary potential φ(xi , zi ) encodes how likely it is for a pixel or patch yito belong to segmentation xi .
Pairwise potential ψ(xi , xj) encodes neighborhood info aboutpixel/patch segmentation labels
David Mandle, Sameep Tandon (Stanford) April 29, 2013 6 / 19
![Page 8: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/8.jpg)
Recap: Image Segmentation as Energy Optimization
Goal: find x to maximize P(x , z) (z is observed)
Taking logs:
E (x , z) =∑i
φ(xi , zi ) +∑i ,j
ψ(xi , xj)
Unary potential φ(xi , zi ) encodes how likely it is for a pixel or patch yito belong to segmentation xi .
Pairwise potential ψ(xi , xj) encodes neighborhood info aboutpixel/patch segmentation labels
David Mandle, Sameep Tandon (Stanford) April 29, 2013 6 / 19
![Page 9: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/9.jpg)
Recap: Image Segmentation as Energy Optimization
Goal: find x to maximize P(x , z) (z is observed)
Taking logs:
E (x , z) =∑i
φ(xi , zi ) +∑i ,j
ψ(xi , xj)
Unary potential φ(xi , zi ) encodes how likely it is for a pixel or patch yito belong to segmentation xi .
Pairwise potential ψ(xi , xj) encodes neighborhood info aboutpixel/patch segmentation labels
David Mandle, Sameep Tandon (Stanford) April 29, 2013 6 / 19
![Page 10: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/10.jpg)
Recap: GrabCut Model
Unary Potentials: log of Gaussian Mixture ModelI But to deal with tractability, we assign each xi to component ki
φ(xi , ki , θ|zi ) = − log π(xi , ki ) + logN(zi ;µ(ki ),Σ(ki ))
Pairwise Potentials:
ψ(xi , xj |zi , zj) = [xi 6= xj ] exp(−β−1‖zi − zj‖2)
where β = 2 · avg(‖zi − zj‖2)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 7 / 19
![Page 11: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/11.jpg)
Recap: GrabCut Model
Unary Potentials: log of Gaussian Mixture ModelI But to deal with tractability, we assign each xi to component ki
φ(xi , ki , θ|zi ) = − log π(xi , ki ) + logN(zi ;µ(ki ),Σ(ki ))
Pairwise Potentials:
ψ(xi , xj |zi , zj) = [xi 6= xj ] exp(−β−1‖zi − zj‖2)
where β = 2 · avg(‖zi − zj‖2)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 7 / 19
![Page 12: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/12.jpg)
Recap: GrabCut Optimization Strategy
GrabCut EM Algorithm
1 Initialize Mixture Models
2 Assign GMM components:
ki = arg minkφ(xi , ki , θ|zi )
3 Get GMM parameters:
θ = arg minθ
∑i
φ(xi , ki , θ|zi )
4 Perform segmentation using reduction to min-cut:
x = arg minx
E (x , z ; k , θ)
5 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 8 / 19
![Page 13: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/13.jpg)
Recap: GrabCut Optimization Strategy
GrabCut EM Algorithm
1 Initialize Mixture Models
2 Assign GMM components:
ki = arg minkφ(xi , ki , θ|zi )
3 Get GMM parameters:
θ = arg minθ
∑i
φ(xi , ki , θ|zi )
4 Perform segmentation using reduction to min-cut:
x = arg minx
E (x , z ; k , θ)
5 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 8 / 19
![Page 14: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/14.jpg)
Recap: GrabCut Optimization Strategy
GrabCut EM Algorithm
1 Initialize Mixture Models
2 Assign GMM components:
ki = arg minkφ(xi , ki , θ|zi )
3 Get GMM parameters:
θ = arg minθ
∑i
φ(xi , ki , θ|zi )
4 Perform segmentation using reduction to min-cut:
x = arg minx
E (x , z ; k , θ)
5 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 8 / 19
![Page 15: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/15.jpg)
Recap: GrabCut Optimization Strategy
GrabCut EM Algorithm
1 Initialize Mixture Models
2 Assign GMM components:
ki = arg minkφ(xi , ki , θ|zi )
3 Get GMM parameters:
θ = arg minθ
∑i
φ(xi , ki , θ|zi )
4 Perform segmentation using reduction to min-cut:
x = arg minx
E (x , z ; k , θ)
5 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 8 / 19
![Page 16: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/16.jpg)
Recap: GrabCut Optimization Strategy
GrabCut EM Algorithm
1 Initialize Mixture Models
2 Assign GMM components:
ki = arg minkφ(xi , ki , θ|zi )
3 Get GMM parameters:
θ = arg minθ
∑i
φ(xi , ki , θ|zi )
4 Perform segmentation using reduction to min-cut:
x = arg minx
E (x , z ; k , θ)
5 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 8 / 19
![Page 17: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/17.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: Histograms
I K bins, bi is bin of pixel ziI θ0, θ1 ∈ [0, 1]K represent color models (distributions) over
foreground/backgroundI
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 18: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/18.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: HistogramsI K bins, bi is bin of pixel zi
I θ0, θ1 ∈ [0, 1]K represent color models (distributions) overforeground/background
I
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 19: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/19.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: HistogramsI K bins, bi is bin of pixel ziI θ0, θ1 ∈ [0, 1]K represent color models (distributions) over
foreground/background
I
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 20: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/20.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: HistogramsI K bins, bi is bin of pixel ziI θ0, θ1 ∈ [0, 1]K represent color models (distributions) over
foreground/backgroundI
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 21: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/21.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: HistogramsI K bins, bi is bin of pixel ziI θ0, θ1 ∈ [0, 1]K represent color models (distributions) over
foreground/backgroundI
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 22: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/22.jpg)
New Model
Let’s consider a simpler model. This will be useful soon
Unary terms: HistogramsI K bins, bi is bin of pixel ziI θ0, θ1 ∈ [0, 1]K represent color models (distributions) over
foreground/backgroundI
φ(xi , bi , θ) = − log θxibi
Pairwise Potentialsψ(xi , xj) = wij |xi − xj |
We will define wij later; for now, consider pairwise equal to Grabcut
Total Energy:
E (x , θ0, θ1) =∑p∈V− logP(zp|θxp) +
∑(p,q)∈N
wpq|xp − xq|
P(zp|θxp) = θxpbp
David Mandle, Sameep Tandon (Stanford) April 29, 2013 9 / 19
![Page 23: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/23.jpg)
EM under new model
1 Initialize histograms θ0, θ1.
2 Fix θ. Perform segmentation using reduction to min-cut:
x = arg minx
E (x , θ0, θ1)
3 Fix x. Compute θ0, θ1 (via standard parameter fitting).
4 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 10 / 19
![Page 24: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/24.jpg)
EM under new model
1 Initialize histograms θ0, θ1.
2 Fix θ. Perform segmentation using reduction to min-cut:
x = arg minx
E (x , θ0, θ1)
3 Fix x. Compute θ0, θ1 (via standard parameter fitting).
4 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 10 / 19
![Page 25: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/25.jpg)
EM under new model
1 Initialize histograms θ0, θ1.
2 Fix θ. Perform segmentation using reduction to min-cut:
x = arg minx
E (x , θ0, θ1)
3 Fix x. Compute θ0, θ1 (via standard parameter fitting).
4 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 10 / 19
![Page 26: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/26.jpg)
EM under new model
1 Initialize histograms θ0, θ1.
2 Fix θ. Perform segmentation using reduction to min-cut:
x = arg minx
E (x , θ0, θ1)
3 Fix x. Compute θ0, θ1 (via standard parameter fitting).
4 Iterate from step 2 until converged
David Mandle, Sameep Tandon (Stanford) April 29, 2013 10 / 19
![Page 27: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/27.jpg)
Optimization
Goal: Minimize Energy
minxE (x)
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
This is hard!
But efficient strategies for optimizing E 1(x) and E 2(x) separately
David Mandle, Sameep Tandon (Stanford) April 29, 2013 11 / 19
![Page 28: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/28.jpg)
Optimization
Goal: Minimize Energy
minxE (x)
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
This is hard!
But efficient strategies for optimizing E 1(x) and E 2(x) separately
David Mandle, Sameep Tandon (Stanford) April 29, 2013 11 / 19
![Page 29: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/29.jpg)
Optimization
Goal: Minimize Energy
minxE (x)
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
This is hard!
But efficient strategies for optimizing E 1(x) and E 2(x) separately
David Mandle, Sameep Tandon (Stanford) April 29, 2013 11 / 19
![Page 30: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/30.jpg)
Optimization via Dual Decomposition
Consider an optimization of the form
minx f1(x) + f2(x)
where optimizing f (x) = f1(x) + f2(x) is hard
But minx f1(x) and minx f2(x) are easy problems
Dual Decomposition idea: Optimize f1(x) and f2(x) separately andcombine in a principled way.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 12 / 19
![Page 31: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/31.jpg)
Optimization via Dual Decomposition
Consider an optimization of the form
minx f1(x) + f2(x)
where optimizing f (x) = f1(x) + f2(x) is hard
But minx f1(x) and minx f2(x) are easy problems
Dual Decomposition idea: Optimize f1(x) and f2(x) separately andcombine in a principled way.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 12 / 19
![Page 32: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/32.jpg)
Optimization via Dual Decomposition
Consider an optimization of the form
minx f1(x) + f2(x)
where optimizing f (x) = f1(x) + f2(x) is hard
But minx f1(x) and minx f2(x) are easy problems
Dual Decomposition idea: Optimize f1(x) and f2(x) separately andcombine in a principled way.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 12 / 19
![Page 33: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/33.jpg)
Optimization via Dual Decomposition
Consider an optimization of the form
minx f1(x) + f2(x)
where optimizing f (x) = f1(x) + f2(x) is hard
But minx f1(x) and minx f2(x) are easy problems
Dual Decomposition idea: Optimize f1(x) and f2(x) separately andcombine in a principled way.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 12 / 19
![Page 34: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/34.jpg)
Optimization via Dual Decomposition
Original Problem:minx f1(x) + f2(x)
Introduce local variables:
minx1,x2 f1(x1) + f2(x2)
s.t x1 = x2
Equivalent Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
David Mandle, Sameep Tandon (Stanford) April 29, 2013 13 / 19
![Page 35: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/35.jpg)
Optimization via Dual Decomposition
Original Problem:minx f1(x) + f2(x)
Introduce local variables:
minx1,x2 f1(x1) + f2(x2)
s.t x1 = x2
Equivalent Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
David Mandle, Sameep Tandon (Stanford) April 29, 2013 13 / 19
![Page 36: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/36.jpg)
Optimization via Dual Decomposition
Original Problem:minx f1(x) + f2(x)
Introduce local variables:
minx1,x2 f1(x1) + f2(x2)
s.t x1 = x2
Equivalent Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
David Mandle, Sameep Tandon (Stanford) April 29, 2013 13 / 19
![Page 37: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/37.jpg)
Optimization via Dual Decomposition
Primal Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
Lagrangian Dual:
g(y) = minx1,x2
f1(x1) + f2(x2) + yT (x2 − x1)
Decompose Lagrangian Dual:
g(y) = (minx1
f1(x1)− yT x1) + (minx2
f2(x2) + yT x2)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 14 / 19
![Page 38: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/38.jpg)
Optimization via Dual Decomposition
Primal Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
Lagrangian Dual:
g(y) = minx1,x2
f1(x1) + f2(x2) + yT (x2 − x1)
Decompose Lagrangian Dual:
g(y) = (minx1
f1(x1)− yT x1) + (minx2
f2(x2) + yT x2)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 14 / 19
![Page 39: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/39.jpg)
Optimization via Dual Decomposition
Primal Problem:
minx1,x2 f1(x1) + f2(x2)
s.t x2 − x1 = 0
Lagrangian Dual:
g(y) = minx1,x2
f1(x1) + f2(x2) + yT (x2 − x1)
Decompose Lagrangian Dual:
g(y) = (minx1
f1(x1)− yT x1) + (minx2
f2(x2) + yT x2)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 14 / 19
![Page 40: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/40.jpg)
Optimization via Dual Decomposition
g(y) = (minx1
f1(x1)− yT x1)︸ ︷︷ ︸g1(y)
+ (minx2
f2(x2) + yT x2)︸ ︷︷ ︸g2(y)
For all y , g(y) is a lower bound on optimal of the primal problem.
Maximize g(y) w.r.t. y to get the tightest bound
Further, g(y) is concave in y . Many techniques for concavemaximization (subgradient ascent, etc).
Ideally have fast optimization strategies for g1(y) and g2(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 15 / 19
![Page 41: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/41.jpg)
Optimization via Dual Decomposition
g(y) = (minx1
f1(x1)− yT x1)︸ ︷︷ ︸g1(y)
+ (minx2
f2(x2) + yT x2)︸ ︷︷ ︸g2(y)
For all y , g(y) is a lower bound on optimal of the primal problem.
Maximize g(y) w.r.t. y to get the tightest bound
Further, g(y) is concave in y . Many techniques for concavemaximization (subgradient ascent, etc).
Ideally have fast optimization strategies for g1(y) and g2(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 15 / 19
![Page 42: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/42.jpg)
Optimization via Dual Decomposition
g(y) = (minx1
f1(x1)− yT x1)︸ ︷︷ ︸g1(y)
+ (minx2
f2(x2) + yT x2)︸ ︷︷ ︸g2(y)
For all y , g(y) is a lower bound on optimal of the primal problem.
Maximize g(y) w.r.t. y to get the tightest bound
Further, g(y) is concave in y . Many techniques for concavemaximization (subgradient ascent, etc).
Ideally have fast optimization strategies for g1(y) and g2(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 15 / 19
![Page 43: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/43.jpg)
Optimization via Dual Decomposition
g(y) = (minx1
f1(x1)− yT x1)︸ ︷︷ ︸g1(y)
+ (minx2
f2(x2) + yT x2)︸ ︷︷ ︸g2(y)
For all y , g(y) is a lower bound on optimal of the primal problem.
Maximize g(y) w.r.t. y to get the tightest bound
Further, g(y) is concave in y . Many techniques for concavemaximization (subgradient ascent, etc).
Ideally have fast optimization strategies for g1(y) and g2(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 15 / 19
![Page 44: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/44.jpg)
Optimization via Dual Decomposition
Back to our Segmentation problem
Energy
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|+ h(n1)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
Group terms
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
E (x) = E1(x) + E2(x)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 16 / 19
![Page 45: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/45.jpg)
Optimization via Dual Decomposition
Back to our Segmentation problem
Energy
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|+ h(n1)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
Group terms
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
E (x) = E1(x) + E2(x)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 16 / 19
![Page 46: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/46.jpg)
Optimization via Dual Decomposition
Back to our Segmentation problem
Energy
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|+ h(n1)
where n1k =
∑p∈Vk
xp and n1 =∑
p∈V xp
Group terms
E (x) =∑k
hk(n1k) +
∑(p,q)∈N
wpq|xp − xq|︸ ︷︷ ︸E1(x)
+ h(n1)︸ ︷︷ ︸E2(x)
E (x) = E1(x) + E2(x)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 16 / 19
![Page 47: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/47.jpg)
Optimization via Dual Decomposition
Dual Decomposition on E (x) = E 1(x) + E 2(x)
Φ(y) = minx1
(E 1(x1)− yT x1)︸ ︷︷ ︸Φ1(y)
+ minx2
(E 2(x2) + yT x2)︸ ︷︷ ︸Φ2(y)
Maximize Φ(y) w.r.t. y to get the tightest lower bound.
Φ1(y) can be computed efficiently via reduction to min s-t cut.
Φ2(y) via convex minimization (several strategies)
You could now use your favorite concave maximization strategy forΦ(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 17 / 19
![Page 48: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/48.jpg)
Optimization via Dual Decomposition
Dual Decomposition on E (x) = E 1(x) + E 2(x)
Φ(y) = minx1
(E 1(x1)− yT x1)︸ ︷︷ ︸Φ1(y)
+ minx2
(E 2(x2) + yT x2)︸ ︷︷ ︸Φ2(y)
Maximize Φ(y) w.r.t. y to get the tightest lower bound.
Φ1(y) can be computed efficiently via reduction to min s-t cut.
Φ2(y) via convex minimization (several strategies)
You could now use your favorite concave maximization strategy forΦ(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 17 / 19
![Page 49: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/49.jpg)
Optimization via Dual Decomposition
Dual Decomposition on E (x) = E 1(x) + E 2(x)
Φ(y) = minx1
(E 1(x1)− yT x1)︸ ︷︷ ︸Φ1(y)
+ minx2
(E 2(x2) + yT x2)︸ ︷︷ ︸Φ2(y)
Maximize Φ(y) w.r.t. y to get the tightest lower bound.
Φ1(y) can be computed efficiently via reduction to min s-t cut.
Φ2(y) via convex minimization (several strategies)
You could now use your favorite concave maximization strategy forΦ(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 17 / 19
![Page 50: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/50.jpg)
Optimization via Dual Decomposition
Dual Decomposition on E (x) = E 1(x) + E 2(x)
Φ(y) = minx1
(E 1(x1)− yT x1)︸ ︷︷ ︸Φ1(y)
+ minx2
(E 2(x2) + yT x2)︸ ︷︷ ︸Φ2(y)
Maximize Φ(y) w.r.t. y to get the tightest lower bound.
Φ1(y) can be computed efficiently via reduction to min s-t cut.
Φ2(y) via convex minimization (several strategies)
You could now use your favorite concave maximization strategy forΦ(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 17 / 19
![Page 51: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/51.jpg)
Optimization via Dual Decomposition
Dual Decomposition on E (x) = E 1(x) + E 2(x)
Φ(y) = minx1
(E 1(x1)− yT x1)︸ ︷︷ ︸Φ1(y)
+ minx2
(E 2(x2) + yT x2)︸ ︷︷ ︸Φ2(y)
Maximize Φ(y) w.r.t. y to get the tightest lower bound.
Φ1(y) can be computed efficiently via reduction to min s-t cut.
Φ2(y) via convex minimization (several strategies)
You could now use your favorite concave maximization strategy forΦ(y)
David Mandle, Sameep Tandon (Stanford) April 29, 2013 17 / 19
![Page 52: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/52.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:
I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has formy = s1
I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 53: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/53.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:
I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has formy = s1
I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 54: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/54.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1
I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 55: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/55.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1I Implication: maximize Φ(s1) over all possible s
I Φ1(s1) is piecewise-linear concaveF |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 56: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/56.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 57: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/57.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 58: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/58.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 59: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/59.jpg)
Optimization via Dual Decomposition
Turns out can maximize Φ(y) over y in polynomial time using aparametric max-flow technique.
Sketch:I Theorem: Given Φ1(y) and Φ2(y) as described, optimal y has form
y = s1I Implication: maximize Φ(s1) over all possible sI Φ1(s1) is piecewise-linear concave
F |V | breakpoints computed by parametric max-flowF Parametric max-flow also returns between 2 and |V |+ 1 solutions
segmentations x per breakpoint
I Φ2(s1) is piecewise-linear concaveF |V | breakpoints can be enumerated from h(·)
I Φ(s1) is piecewise-linear concave with 2|V | breakpoints, so findingmax over Φ(1s) is easy – it’s one of the breakpoints.
I Given the breakpoint, return the segmentation x with minimum energyfrom set given by PMF.
David Mandle, Sameep Tandon (Stanford) April 29, 2013 18 / 19
![Page 60: Joint Optimization of Segmentation and Appearance Modelsvision.stanford.edu/.../slides/sameep_presentation.pdf · Joint Optimization of Segmentation and Appearance Models David Mandle,](https://reader034.fdocuments.in/reader034/viewer/2022050407/5f83fbdaada75c7fe934145d/html5/thumbnails/60.jpg)
References
MRF Review Slides:http://vision.stanford.edu/teaching/cs231b spring1213/slides/segmentation.pdfDual Decomposition Slides: Petter Strandmark and Fredrik Kahl. Paralleland Distributed Graph Cuts by Dual Decomposition.http://www.robots.ox.ac.uk/vgg/rg/slides/parallelgraphcuts.pdf
David Mandle, Sameep Tandon (Stanford) April 29, 2013 19 / 19