Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture...
-
Upload
gordon-hunter -
Category
Documents
-
view
221 -
download
3
Transcript of Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture...
![Page 1: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/1.jpg)
Probabilistic Models for Images
Markov Random FieldsApplications in Image Segmentation and Texture Modeling
Ying Nian WuUCLA Department of Statistics
IPAM July 22, 2013
![Page 2: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/2.jpg)
Outline•Basic concepts, properties, examples•Markov chain Monte Carlo sampling•Modeling textures and objects•Application in image segmentation
![Page 3: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/3.jpg)
Markov Chains
Pr(future|present, past) = Pr(future|present)future past | presentMarkov property: conditional independence limited dependenceMakes modeling and learning possible
![Page 4: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/4.jpg)
Markov Chains (higher order)
Temporal: a natural orderingSpatial: 2D image, no natural ordering
![Page 5: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/5.jpg)
Markov Random Fields
all the other pixels
Nearest neighborhood, first order neighborhood
Markov Property
From Slides by S. Seitz - University of Washington
![Page 6: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/6.jpg)
Markov Random Fields
Second order neighborhood
![Page 7: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/7.jpg)
Markov Random Fields
Can be generalized to any undirected graphs (nodes, edges)Neighborhood system: each node is connected to its neighbors neighbors are reciprocalMarkov property: each node only depends on its neighbors
Note: the black lines on the left graph are illustrating the 2D grid for the image pixels they are not edges in the graph as the blue lines on the right
![Page 8: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/8.jpg)
Markov Random Fields
What is
![Page 9: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/9.jpg)
Cliques for this neighborhood
Hammersley-Clifford Theorem
normalizing constant, partition function
potential functions of cliques
From Slides by S. Seitz - University of Washington
![Page 10: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/10.jpg)
Cliques for this neighborhood
Hammersley-Clifford Theorem
a clique: a set of pixels, each member is the neighbor of any other member
From Slides by S. Seitz - University of Washington
Gibbs distribution
![Page 11: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/11.jpg)
Cliques for this neighborhood
Hammersley-Clifford Theorem
a clique: a set of pixels, each member is the neighbor of any other member
……etc, note: the black lines are for illustrating 2D grids, they are not edges in the graph
Gibbs distribution
![Page 12: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/12.jpg)
Cliques for this neighborhood
Ising model
From Slides by S. Seitz - University of Washington
![Page 13: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/13.jpg)
Ising model
Challenge: auto logistic regression
pair potential
![Page 14: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/14.jpg)
Gaussian MRF model
continuous
Challenge: auto regression
pair potential
![Page 15: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/15.jpg)
Sampling from MRF Models
Markov Chain Monte Carlo (MCMC)• Gibbs sampler (Geman & Geman 84)• Metropolis algorithm (Metropolis et al. 53)• Swedeson & Wang (87)• Hybrid (Hamiltonian) Monte Carlo
![Page 16: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/16.jpg)
![Page 17: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/17.jpg)
Gibbs Sampler
Simple one-dimension distribution
Repeat: • Randomly pick a pixel • Sample given the current values of
![Page 18: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/18.jpg)
Gibbs sampler for Ising model
Challenge: sample from Ising model
![Page 19: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/19.jpg)
Metropolis Algorithm
Repeat: • Proposal: Perturb I to J by sample from K(I, J) = K(J, I)• If change I to J otherwise change I to J with prob
energy function
![Page 20: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/20.jpg)
Metropolis for Ising model
Challenge: sample from Ising model
Ising model: proposal --- randomly pick a pixel and flip it
![Page 21: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/21.jpg)
Modeling Images by MRFIsing model
Exponential family model, log-linear model maximum entropy model
unknown parameters
features (may also need to be learned)
reference distribution
Hidden variables, layers, RBM
![Page 22: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/22.jpg)
Modeling Images by MRF
Given
How to estimate
• Maximum likelihood • Pseudo-likelihood (Besag 1973) • Contrastive divergence (Hinton)
![Page 23: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/23.jpg)
Maximum likelihood
Given
Challenge: prove it
![Page 24: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/24.jpg)
Stochastic Gradient
Given
Generate
Analysis by synthesis
![Page 25: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/25.jpg)
Texture Modeling
![Page 26: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/26.jpg)
![Page 27: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/27.jpg)
![Page 28: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/28.jpg)
![Page 29: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/29.jpg)
![Page 30: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/30.jpg)
![Page 31: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/31.jpg)
![Page 32: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/32.jpg)
![Page 33: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/33.jpg)
![Page 34: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/34.jpg)
![Page 35: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/35.jpg)
![Page 36: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/36.jpg)
![Page 37: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/37.jpg)
![Page 38: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/38.jpg)
Modeling image pixel labels as MRF (Ising)
( , )i ix y
( , )i jx x
1
real image
label image
Slides by R. Huang – Rutgers University
MRF for Image Segmentation
Bayesian posterior
![Page 39: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/39.jpg)
Model joint probability
label
image
label-labelcompatibility
Functionenforcing
Smoothness constraint
neighboringlabel nodes
local Observations
image-labelcompatibility
Functionenforcing
DataConstraint
( , )
1( , ) ( , ) ( , )i j i i
i j i
P x x x yZ
x y
* *
( , )( , ) arg max ( , | )P
xx x y
region labels
image pixels
model param
.
Slides by R. Huang – Rutgers University
![Page 40: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/40.jpg)
*
1
( , ) ( , )2
2
2
2 2
arg max ( | )
1arg max ( , ) ( | ) ( , ) / ( ) ( , )
1arg max ( , ) ( , ) ( , ) ( , ) ( , )
( , ) ( ; , )
( , ) exp( ( ) / )
[ , , ]
i i
i i
i i i j i i i ji i j i i j
i i i x x
i j i j
x x
P
P P P P PZ
x y x x P x y x xZ
x y G y
x x x x
x
x
x
x x y
x y x y x y y x y
x y
( , )i ix y
( , )i jx xSlides by R. Huang – Rutgers University
MRF for Image Segmentation
![Page 41: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/41.jpg)
Inference in MRFs
– Classical• Gibbs sampling, simulated annealing • Iterated conditional modes
– State of the Art• Graph cuts• Belief propagation• Linear Programming • Tree-reweighted message passing
Slides by R. Huang – Rutgers University
![Page 42: Probabilistic Models for Images Markov Random Fields Applications in Image Segmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics.](https://reader036.fdocuments.in/reader036/viewer/2022062408/56649f035503460f94c1734c/html5/thumbnails/42.jpg)
Summary•MRF, Gibbs distribution•Gibbs sampler, Metropolis algorithm•Exponential family model