Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University
description
Transcript of Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University
1
Physical Fluctuomatics5th and 6th Probabilistic information processing
by Gaussian graphical model
Kazuyuki TanakaGraduate School of Information Sciences, Tohoku University
[email protected]://www.smapip.is.tohoku.ac.jp/~kazu/
Physical Fluctuomatics (Tohoku University)
Physical Fluctuomatics (Tohoku University) 2
Textbooks
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 7.K. Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002, Section 4.
Physical Fluctuomatics (Tohoku University) 3
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 4
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 5
Markov Random Fields for Image Processing
S. Geman and D. Geman (1986): IEEE Transactions on PAMIImage Processing for Markov Random Fields (MRF) (Simulated Annealing, Line Fields)
J. Zhang (1992): IEEE Transactions on Signal ProcessingImage Processing in EM algorithm for Markov Random Fields (MRF) (Mean Field Methods)
Markov Random Fields are One of Probabilistic Methods for Image processing.
Physical Fluctuomatics (Tohoku University) 6
Markov Random Fields for Image Processing
In Markov Random Fields, we have to consider not only the states with high probabilities but also ones with low probabilities.In Markov Random Fields, we have to estimate not only the image but also hyperparameters in the probabilistic model.We have to perform the calculations of statistical quantities repeatedly.
Hyperparameter Estimation
Statistical Quantities
Estimation of Image
We can calculate statistical quantities by adopting the Gaussian graphical model as a prior probabilistic model and by using Gaussian integral formulas.
Physical Fluctuomatics (Tohoku University) 7
Purpose of My TalkReview of formulation of probabilistic model for image processing by means of conventional statistical schemes.Review of probabilistic image processing by using Gaussian graphical model (Gaussian Markov Random Fields) as the most basic example.
K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A: Math. Gen., vol.35, pp.R81-R150, 2002.
Section 2 and Section 4 are summarized in the present talk.
Physical Fluctuomatics (Tohoku University) 8
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 9
Bayes Formula and Bayesian Network
Posterior Probability
}Pr{
}Pr{}|Pr{}|Pr{
B
AABBA
Bayes Rule
Prior Probability
Event B is given as the observed data.Event A corresponds to the original information to estimate. Thus the Bayes formula can be applied to the estimation of the original information from the given data.
A
B
Bayesian Network
Data-Generating Process
Physical Fluctuomatics (Tohoku University) 10
Image Restoration by Probabilistic Model
Original Image
Degraded Image
Transmission
Noise
Likelihood Marginal
PriorLikelihood
Posterior
}ageDegradedImPr{
}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability.
Bayes Formula
Physical Fluctuomatics (Tohoku University) 11
Image Restoration by Probabilistic Model
Degraded
Image
i
fi: Light Intensity of Pixel iin Original Image
),( iii yxr
Position Vector
of Pixel i
gi: Light Intensity of Pixel iin Degraded Image
i
Original
Image
The original images and degraded images are represented by f = (f1,f2,…,f|V|) and g = (g1,g2,…,g|V|), respectively.
Physical Fluctuomatics (Tohoku University) 12
Probabilistic Modeling of Image Restoration
PriorLikelihood
Posterior
}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Vi
ii fg ),(}|Pr{
}Image Original|Image DegradedPr{
fFgG
fg
Random Fieldsfi
gi
fi
gi
or
Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.
),...,,( Image Degraded of Field Random
),...,,( Image Original of Field Random
||21
||21
V
V
GGG
FFF
G
F
),...,,( Image Degraded ofVector State
),...,,( Image Original ofVector State
||21
||21
V
V
ggg
fff
g
f
Physical Fluctuomatics (Tohoku University) 13
Probabilistic Modeling of Image Restoration
PriorLikelihood
Posterior
}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Eij
ji ff ),(}Pr{
}Image OriginalPr{
fF
f
Random Fields
Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels.
i j
Product over All the Nearest Neighbour Pairs of Pixels
),...,,( Image Original ofVector State
),...,,( Image Original of Field Random
||21
||21
V
V
fff
FFF
f
F
Physical Fluctuomatics (Tohoku University) 14
Bayesian Image Analysis
Eji
jiVi
ii ffgf},{
),(),(
Pr
PrPrPr
gG
fFfFgGgGfF
fg
fF Pr fFgG Pr gOriginalImage
Degraded Image
Prior Probability
Posterior Probability
Degradation Process
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
E : Set of all the nearest neighbour pairs of pixels
V : Set of All the pixels
Physical Fluctuomatics (Tohoku University) 15
Estimation of Original Image
We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels.
}|Pr{maxargˆ gG iiz
i zFfi
2}|Pr{minargˆ zgGzF dzf iiii
}|Pr{maxargˆ gGzFfz
Thresholded Posterior Mean (TPM) estimation
Maximum posterior marginal (MPM) estimation
Maximum A Posteriori (MAP) estimation
if
ii fF\
}|Pr{}|Pr{f
gGfFgG
(1)
(2)
(3)
Physical Fluctuomatics (Tohoku University) 16
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 17
Bayesian Image Analysis by Gaussian Graphical Model
Ejiji ff
Z },{
2
PR 2
1exp
)(
1|Pr
fF
0005.0 0030.00001.0
Patterns are generated by MCMC.
Markov Chain Monte Carlo Method
PriorProbability
,if
E:Set of all the nearest-neighbour pairs of pixels
V:Set of all the pixels
||21},{
2PR 2
1exp)( V
Ejiji dzdzdzzzZ
Physical Fluctuomatics (Tohoku University) 18
Bayesian Image Analysis by Gaussian Graphical Model
2,0~ Nfg ii
Viii gf 2
22 2
1exp
2
1,Pr
fFgG
Histogram of Gaussian Random Numbers
,, ii gf
Degraded image is obtained by adding a white Gaussian noise to the original image.
Degradation Process is assumed to be the additive white Gaussian noise.
V: Set of all the pixels
Original Image f Gaussian Noise n
Degraded Image g
Physical Fluctuomatics (Tohoku University) 19
Bayesian Image Analysis
Eji
jiVi
ii ffgf},{
),(),(
Pr
PrPrPr
gG
fFfFgGgGfF
fg
fF Pr fFgG Pr gOriginalImage
Degraded Image
Prior Probability
Posterior Probability
Degradation Process
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
E : Set of all the nearest neighbour pairs of pixels
V : Set of All the pixels
Physical Fluctuomatics (Tohoku University) 20
Bayesian Image Analysis ,, ji gf
Ejiji
Viii
Eji
ff
jiVi
gf
ii
ffgf
ffgf
jiii
},{
},{
),(
2
),(
2
2
),(),(
2
1exp
2
1exp
},|Pr{
}|Pr{},|Pr{},,|Pr{
gG
fFfFgGgGfF
A Posteriori Probability
Physical Fluctuomatics (Tohoku University) 21
Statistical Estimation of Hyperparameters
zzFzFgG
zgGzFgG
d
d
}|Pr{},|Pr{
},|,Pr{},|Pr{
},|Pr{max arg)ˆ,ˆ(
,
gG
f g
Marginalized with respect to F
}|Pr{ fF },|Pr{ fFgG gOriginal Image
Marginal Likelihood
Degraded ImageVy
x
},|Pr{ gG
Hyperparameters , a s are determined so as to maximize the marginal likelihood Pr{G=g|a,s} with respect to a, .s
zgGzFzf d }ˆ,ˆ,|Pr{ˆ
Physical Fluctuomatics (Tohoku University) 22
Bayesian Image Analysis ,, ji gf
TT2
POS
},{
22
2POS
2
1))((
2
1exp
,,
1
2
1
2
1exp
,,
1
},|Pr{
}|Pr{},|Pr{},,|Pr{
fCfgfgfg
g
gG
fFfFgGgGfF
Z
ffgfZ Eji
jiVi
ii
A Posteriori Probability
Gaussian Graphical Model
||21
TT2POS
2
1))((
2
1exp,, VdzdzdzZ zCzgzgzg
otherwise0
},{1
4
Eji
ji
ji C
|V|x|V| matrix
),...,,(
),...,,(
),...,,(
||21
||21
||21
V
V
V
zzz
ggg
fff
z
g
f
Physical Fluctuomatics (Tohoku University) 23
Average of Posterior Probability
)0,00())()((2
1exp)(
||
||21T2
V
V ,,dzdzdz
mzCI mz mz
gCI
I
CI
IzCI
CI
Iz
CI
IzCI
CI
Izz
gGzFzF
2
||21
T
22
22
||21
T
22
22
||21
2
1exp
2
1exp
},,|Pr{,
V
V
V
dzdzdzgg
dzdzdzgg
dzdzdz
E
Gaussian Integral formula
Physical Fluctuomatics (Tohoku University) 24
gCI
I
zgGzFzf
2
},,|Pr{ˆ
d
Bayesian Image Analysis by Gaussian Graphical Model ,, ii gf
otherwise0
},{1
4
Eji
ji
ji C
Multi-Dimensional Gaussian Integral Formula
Ejiji
Viii ffgf
},{
22
2 2
1
2
1exp},,|Pr{
gGfF
Posterior Probability
Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula
|V|x|V| matrix
E:Set of all the nearest-neghbour pairs of pixels
V:Set of all the pixels
Physical Fluctuomatics (Tohoku University) 25
Statistical Estimation of Hyperparameters
)()2(
),,(}|Pr{},|Pr{},|Pr{
PR2/||2
POS
Z
Zd
V
gzzFzFgGgG
f g
Marginalized with respect to F
}|Pr{ fF },|Pr{ fFgG gOriginal Image
Marginal Likelihood
Degraded ImageVy
x
},|Pr{ gG
||21
TT2
POS
2
1))((
2
1exp
,,
Vdzdzdz
Z
zCzgzgz
g
||21
TPR
2
1exp VdzdzdzZ zCz
Physical Fluctuomatics (Tohoku University) 26
Calculations of Partition Function
AmzA mz
det
)2()( )(
2
1exp
||
||21T
V
Vdzdzdz
T22
||2
||21
T
22
22
T2
||21T
2
T
22
22
POS
2
1exp
)det(
)2(
2
1exp
2
1exp
2
1
2
1exp
),,(
gCI
Cg
CI
gCI
IzCIg
CI
Iz
gCI
Cg
gCI
Cgg
CI
IzCIg
CI
Iz
g
V
V
V
dzdzdz
dzdzdz
Z
(A is a real symmetric and positive definite matrix.)Gaussian Integral formula
CCzz
det
)2(
2
1exp)(
||
||
||21 V
V
VPR dzdzdzZ
Physical Fluctuomatics (Tohoku University) 27
Exact expression of Marginal Likelihood in Gaussian Graphical Model
T
22 2
1exp
det2
det},|Pr{ g
CI
Cg
CI
CgG
V
gCI
If
2ˆ
Multi-dimensional Gauss integral formula
Vy
x
},|Pr{max argˆ,ˆ,
gG We can construct an exact EM algorithm.
)()2(
),,(}|Pr{},|Pr{},|Pr{
PR2/||2
POS
Z
Zd
V
gzzFzFgGgG
Physical Fluctuomatics (Tohoku University) 28
Bayesian Image Analysis by Gaussian Graphical Model
1T
22
2
11||
1
11
1Tr
||
1
g
CI
Cg
CI
C
ttVtt
t
Vt
T22
242
2
2
11
11
||
1
11
1Tr
||
1g
CI
Cg
CI
I
tt
tt
Vtt
t
Vt
0
0.0002
0.0004
0.0006
0.0008
0.001
0 20 40 60 80 100 t
t
g f̂
gCI
Im
2)()()(
ttt
2)(minarg)(ˆ tmztf ii
zi
i
Iteration Procedure in Gaussian Graphical Model
f̂
g
Physical Fluctuomatics (Tohoku University) 29
Image Restoration by Markov Random Field Model and Conventional Filters
2ˆ||
1MSE
Viii ff
V
MSE
Statistical Method 315
Lowpass Filter
(3x3) 388
(5x5) 413
Median Filter
(3x3) 486
(5x5) 445
(3x3) Lowpass (5x5) MedianMRF
Original Image Degraded Image
RestoredImage
V:Set of all the pixels
Physical Fluctuomatics (Tohoku University) 30
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 31
Performance Analysis
1y
x
2y
3y
4y
5y
1x̂
2x̂
3x̂
4x̂
5x̂
Pos
teri
or P
rob
abil
ity
Estimated ResultsObserved
Data
Sample Average of Mean Square Error
SignalA
dd
itiv
e W
hit
e G
auss
ian
Noi
se
5
1
2||ˆ||5
1][MSE
nnxx
Physical Fluctuomatics (Tohoku University) 32
Statistical Performance Analysis
ydxXyYxyhV
x
},|Pr{),,(1
),|(MSE2
),,( yh
gy
Additive White Gaussian Noise
},|Pr{ xXyY
x
},,|Pr{ yYxX
Posterior Probability
Restored Image
Original Image Degraded Image
},|Pr{ xXyY
Additive White Gaussian Noise
Physical Fluctuomatics (Tohoku University) 33
Statistical Performance Analysis
ydxXyYxyV
ydxXyYxyhV
x
Pr1
Pr,,1
),|MSE(
212
2
CI
y
xdyxPxyh
12
)|(,,
CI
)2
1exp(
2
1exp,Pr
22
22
yx
yxxXyYVi
ii
otherwise,0
},{,1
,4
Eji
Vji
ji C
Physical Fluctuomatics (Tohoku University) 34
Statistical Performance Estimation for Gaussian Markov Random Fields
T22
242
22
2
22
||
22
242T
22
||
22
2T
22
||
22
2T
22
||
22T
22
||
2
2
2
T
2
2
2
22
||2
2
2
2
22
||2
22
22
||2
2
2
)(||
1
)(Tr
1
2
1exp
2
1
)(
1
2
1exp
2
1)(
)(
1
2
1exp
2
1
)()(
1
2
1exp
2
1)(
)()(
1
2
1exp
2
1)()(
1
2
1exp
2
1)(
1
2
1exp
2
1)(
1
2
1exp
2
11
},|Pr{),,(1
),|(MSE
xI
xVV
ydxyxxV
ydxyxyxV
ydxyxxyV
ydxyxyxyV
ydxyxxyxxyV
ydxyxxyV
ydxyxxxyV
ydyxxyV
ydxXyYxyhV
x
V
V
V
V
V
V
V
V
C
C
CI
I
CI
C
CI
C
CI
C
CI
I
CI
C
CI
I
CI
C
CI
I
CI
C
CI
I
CI
I
CI
I
CI
I
= 0
otherwise,0
},{,1
,4
Eji
Vji
ji C
Physical Fluctuomatics (Tohoku University) 35
Statistical Performance Estimation for Gaussian Markov Random Fields
xI
xVV
ydyxxyV
ydxXyYxyhV
x
V
22
242T
22
2
22
||
2
2
2
2
)(||
1
)(Tr
1
2
1exp
2
11
},|Pr{),,(1
),|(MSE
C
C
CI
I
CI
I
otherwise,0
},{,1
,4
Eji
Vji
ji C
0
200
400
600
0 0.001 0.002 0.003a
s=40
0
200
400
600
0 0.001 0.002 0.003
s=40
a
),|(MSE x ),|(MSE x
Physical Fluctuomatics (Tohoku University) 36
Contents
1. Introduction2. Probabilistic Image
Processing3. Gaussian Graphical Model4. Statistical Performance
Analysis5. Concluding Remarks
Physical Fluctuomatics (Tohoku University) 37
Summary
Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized.
Probabilistic image processing by using Gaussian graphical model has been shown as the most basic example.
References
K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) .
K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002).
A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002).
Physical Fluctuomatics (Tohoku University) 38
Physical Fluctuomatics (Tohoku University) 39
Problem 5-1: Derive the expression of the posterior probability Pr{F=f|G=g,a,s} by using Bayes formulas Pr{F=f|G=g,a,s}=Pr{G=g|F=f,s}Pr{F=f,a}/Pr{G=g|a,s}. Here Pr{G=g|F=f,s} and Pr{F=f,a} are assumed to be as follows:
Viii gf 2
22 2
1exp
2
1,Pr
fFgG
Ejiji ff
Z },{
2
PR 2
1exp
)(
1|Pr
fF
||21},{
2PR 2
1exp)( V
Ejiji dzdzdzzzZ
Ejiji
Viii ffgf
Z },{
22
2POS 2
1
2
1exp
,,
1},,|Pr{
ggGfF
||21
},{
22
2PR 2
1
2
1exp V
Ejiji
Viii dzdzdzzzgzZ
[Answer]
Physical Fluctuomatics (Tohoku University) 40
Problem 5-2: Show the following equality.
TT2
},{
22
2
2
1))((
2
1
2
1
2
1
fCfgfgf
Eji
jiVi
ii ffgf
otherwise0
},{1
4
Eji
ji
ji C
Physical Fluctuomatics (Tohoku University) 41
Problem 5-3: Show the following equality.
T
2
T
22
22
TT2
2
1
2
1
2
1))((
2
1
gCI
Cg
gCI
IzCIg
CI
Iz
zCzgzgz
Physical Fluctuomatics (Tohoku University) 42
Problem 5-4: Show the following equalities by using the multi-dimensional Gaussian integral formulas.
T22
||2
||21},{
22
2POS
2
1exp
)det(
)2(
2
1
2
1exp,,
gCI
Cg
CI
g
V
VEji
jiVi
ii dzdzdzffgfZ
Cdet
)2(
2
1exp)(
||
||
||21},{
2
V
V
VEji
jiPR dzdzdzffZ
Physical Fluctuomatics (Tohoku University) 43
Problem 5-5: Derive the extremum conditions for the following marginal likelihood Pr{G=g| ,a s} with respect to the hyperparameters a and s.
T
22 2
1exp
det2
det},|Pr{ g
CI
Cg
CI
CgG
V
T
22
2
||
1Tr
||
11g
CI
Cg
CI
C
VV
T
22
242
2
22
||
1Tr
||
1g
CI
Cg
CI
I
VV
[Answer]
Physical Fluctuomatics (Tohoku University) 44
Problem 5-6: Derive the extremum conditions for the following marginal likelihood Pr{G=g| ,a s} with respect to the hyperparameters a and s.
T
22 2
1exp
det2
det},|Pr{ g
CI
Cg
CI
CgG
V
T
22
2
||
1Tr
||
11g
CI
Cg
CI
C
VV
T
22
242
2
22
||
1Tr
||
1g
CI
Cg
CI
I
VV
[Answer]
Physical Fluctuomatics (Tohoku University) 45
Problem 5-7: Make a program that generate a degraded image by the additive white Gaussian noise. Generate some degraded images from a given standard images by setting s=10,20,30,40 numerically. Calculate the mean square error (MSE) between the original image and the degraded image.
Histogram of Gaussian Random Numbers Fi -Gi~N(0,402)
Original Image Gaussian Noise Degraded Image
Sample Program: http://www.morikita.co.jp/soft/84661/
2||
1MSE
Viii gf
V
K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .
Physical Fluctuomatics (Tohoku University) 46
Problem 5-8: Make a program of the following procedure in probabilistic image processing by using the Gaussian graphical model and the additive white Gaussian noise.
1T
22
2
11||
1
11
1Tr
||
1
g
CI
Cg
CI
C
ttVtt
t
Vt
T22
242
2
2
11
11
||
1
11
1Tr
||
1g
CI
Cg
CI
I
tt
tt
Vtt
t
Vt
gCI
Im
2)()()(
ttt
2)(minarg)(ˆ tmztf ii
zi
i
Algorithm: Repeat the following procedure until convergence
Sample Program: http://www.morikita.co.jp/soft/84661/
K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .