Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for...
Transcript of Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for...
![Page 1: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/1.jpg)
1
Dependent Hierarchical Beta Process
for Image Interpolation and Denoising
1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,2David Dunson and 1Lawrence Carin
1Department of Electrical & Computer Engineering, Duke University2Department of Statistical Science, Duke University
3Department of Electrical & Computer Engineering, University of Minnesota
AI & Statistics, Ft. Lauderdale, April 12 2011
Zhou 2011
![Page 2: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/2.jpg)
2
Outline
• Introduction
• Dependent Hierarchical Beta Process (dHBP)
• Dictionary Learning with dHBP
• Image Interpolation and Denoising
• Conclusions
![Page 3: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/3.jpg)
3
Introduction: Background
• Dictionary learning and sparse coding
• Sparse factor analysis model
(Factor/feature/dish/dictionary atom)
• Indian Buffet process and beta process
11
minN
iFi
λ=
− + ∑D,W
X DW w
![Page 4: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/4.jpg)
4
Introduction: Background
• Beta process and Bernoulli process (Thibaux &
Jordan AISTATS2007)
• Indian Buffet process (Griffiths & Ghahramani 2005)
![Page 5: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/5.jpg)
5
Introduction: Motivation
• Exchangeability assumption is not true
Image interpolation with BPFA (Zhou et al. 2009)
80% pixels are missing at random
![Page 6: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/6.jpg)
6
Introduction: Covariate Dependence
• Dependent Dirichlet process
MacEachern (1999), Duan et al. (2007), Griffin & Steel
(2006)
• Non-exchangeable IBP
Phylogenetic IBP (Miller, Griffiths & Jordan 2008)
Dependent IBP (Williamson, Orbanz & Ghahramani 2010)
• Bayesian density regression (Dunson & Pillai 2007)
![Page 7: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/7.jpg)
7
Review of Beta Process (Thibaus & Jordan 2007)
• A beta process is a positive Levy process, whose Levy
measure lives on and can be expressed as
• If the base measure is continuous, then
is drawn from a degenerate beta
distribution parameterized by
• If , then
[ ]0,1Ω ⊗
0B
1
, and kk w k
k
B p pδ∞
=
=∑
01
kk wk
B q δ∞
=
=∑
1
, and ~ Beta( , (1 ))kk w k k k
k
B p p cq c qδ∞
=
= −∑
c
![Page 8: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/8.jpg)
8
Dependent Hierarchical Beta Process
• Random walk matrix
• dHBP
• Covariate-dependent correlations
![Page 9: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/9.jpg)
9
Dictionary Learning with dHBP
BP
dHBP
![Page 10: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/10.jpg)
10
• Missing data
Full data:
Observed: , Missing:
• Sparse spiky noise
• Recoverd data:
Missing Data and Outliers
![Page 11: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/11.jpg)
11
• Independence chain Metropolis-Hastings
• Slice sampling
• Gibbs sampling
MCMC Inference
![Page 12: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/12.jpg)
12
Experiments
• Image interpolation
Missing pixels
Locations of missing pixels are known
• Image denoising
WGN + sparse spiky noise
Amplitudes unknown
Locations of spiky noise are unknown
• Covariates: patch spatial locations
![Page 13: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/13.jpg)
13
Image Interpolation: BP vs. dHBP
BP: 26.9 dB dHBP: 29.92 dB
80% pixels missing at random
![Page 14: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/14.jpg)
14
BP atom usage dHBP atom usage
BP dictionary dHBP dictionary Dictionary atom activation probability map
dHBP recoveryOriginaldHBP recovery
BP recoveryObserved
Image Interpolation: BP vs. dHBP
![Page 15: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/15.jpg)
15
Observed (20%) BP recovery
dHBP recovery Original
dHBP recovery
Image Interpolation: BP vs. dHBP
![Page 16: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/16.jpg)
16
Image Interpolation: BP vs. dHBP
Observed (20%) BP recovery
dHBP recovery Original
dHBP recovery
![Page 17: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/17.jpg)
17
Spiky Noise Removal: BP vs. dHBP
BP denoised imageNoisy image (WGN + Sparse
Spiky noise)BP dictionary
dHBP denoised imageOriginal image dHBP dictionary
![Page 18: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/18.jpg)
18
Spiky Noise Removal: BP vs. dHBP
BP denoised imageNoisy image (WGN + Sparse
Spiky noise)
dHBP denoised imageOriginal image dHBP dictionary
BP dictionary
![Page 19: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/19.jpg)
19
Future Work
• Landmark-dHBP
J landmarks
J << N
• Locality constraint for manifold learning
Covariates: cosine distance between samples
Dictionary atoms look like the data
• Variational inference, online learning
• Other applications
Super-resolution
Deblurring
Video background & foreground modeling
![Page 20: Dependent Hierarchical Beta Process for Image ...€¦ · 1 Dependent Hierarchical Beta Process for Image Interpolation and Denoising 1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,](https://reader033.fdocuments.in/reader033/viewer/2022051815/603f033fc9584f1a4726c43b/html5/thumbnails/20.jpg)
20
• A dependent hierarchical beta process is
proposed
• Efficient hybrid MCMC inference is presented
• Encouraging performance is demonstrated on
image-processing applications
Conclusions