Physical Fluctuomatics 11th Probabilistic image processing by means of physical models

Physical Fluctuomatics (Tohoku University) 1

Physical Fluctuomatics11th Probabilistic image processing by means of

physical models

Kazuyuki TanakaGraduate School of Information Sciences, Tohoku University

[email protected]://www.smapip.is.tohoku.ac.jp/~kazu/


Textbooks

Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 7.Kazuyuki Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002.Muneki Yasuda, Shun Kataoka and Kazuyuki Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese).


Contents

1. Probabilistic Image Processing

2. Loopy Belief Propagation3. Statistical Learning

Algorithm4. Summary


Image Restoration by Probabilistic Model

Original Image

Degraded Image

Transmission

Noise

Likelihood Marginal

PriorLikelihood

Posterior

}ageDegradedImPr{}Image OriginalPr{}Image Original|Image DegradedPr{

}Image Degraded|Image OriginalPr{

Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability.

Bayes Formula


Image Restoration by Probabilistic Model

Degraded

Image

i

fi: Light Intensity of Pixel iin Original Image

),( iii yxr

Position Vector of Pixel i

gi: Light Intensity of Pixel iin Degraded Image

i

Original

Image

The original images and degraded images are represented by f = (f1,f2,…,f|V|) and g = (g1,g2,…,g|V|), respectively.


Probabilistic Modeling of Image Restoration

PriorLikelihood

Posterior

}Image OriginalPr{}Image Original|Image DegradedPr{


Eij

ji ff ),(}Pr{

}Image OriginalPr{

fF

f

Random Fields

Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels.

i j

Product over All the Nearest Neighbour Pairs of Pixels

),...,,( Image Original ofVector State

),...,,( Image Original of Field Random

||21

||21

V

V

fff

FFF

f

F


Probabilistic Modeling of Image Restoration

PriorLikelihood

Posterior

}Image OriginalPr{}Image Original|Image DegradedPr{


Vi

ii fg ),(}|Pr{

}Image Original|Image DegradedPr{

fFgG

fg

Random Fieldsfi

gi

fi

gi

or

Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.

),...,,( Image Degraded of Field Random

),...,,( Image Original of Field Random

||21

||21

V

V

GGG

FFF

G

F

),...,,( Image Degraded ofVector State

),...,,( Image Original ofVector State

||21

||21

V

V

ggg

fff

g

f


Bayesian Image Analysis

Eji

jiVi

ii ffgf},{

),(),(

PrPrPr

PrgG

fFfFgGgGfF

fg

fF Pr fFgG Pr gOriginalImage

Degraded Image

Prior Probability

Posterior Probability

Degradation Process

Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.

E ： Set of all the nearest neighbour pairs of pixels

V ： Set of All the pixels


Estimation of Original Image

We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels.

}|Pr{maxargˆ gG iiz

i zFfi

2}|Pr{minargˆ zgGzF dzf iiii

}|Pr{maxargˆ gGzFfz

Thresholded Posterior Mean (TPM) estimation

Maximum posterior marginal (MPM) estimation

Maximum A Posteriori (MAP) estimation

if

ii fF\

}|Pr{}|Pr{f

gGfFgG

(1)

(2)

(3)


Prior Probability of Image Processing

Ejiji ff

Z },{

2

PR 21exp

)(1Pr

fF

2 41

Paramagnetic Ferromagnetic

Digital Images (fi=0,1,…,q1)

Sampling of Markov Chain Monte Carlo Method

V: Set of all the pixels Fluctuations increase near α=1.76….

E: Set of all the nearest-neighbour pairs of pixels

1

0

1

0

1

0 },{

2PR

1 1 ||21exp)(

q

z

q

z

q

z Ejiji

V

zzZ


Bayesian Image Analysis by Gaussian Graphical Model

Ejiji ff

Z },{

2

PR 21exp

)(1|Pr

fF

0005.0 0030.00001.0

Patterns are generated by MCMC.

Markov Chain Monte Carlo Method

PriorProbability

,if

E:Set of all the nearest-neighbour pairs of pixels

V:Set of all the pixels

||21},{

2PR 2

1exp)( VEji

ji dzdzdzzzZ


Degradation Process for Gaussian Graphical Model

2,0~ Nfg ii

Viii gf 2

22 21exp

2

1,Pr

fFgG

,, ii gf

Degraded image is obtained by adding a white Gaussian noise to the original image.

Degradation Process is assumed to be the additive white Gaussian noise.

V: Set of all the pixels

Histogram of Gaussian Random Numbers

Original Image f Gaussian Noise n

Degraded Image g



Eji

jiVi

ii ffgf},{

),(),(

PrPrPr

PrgG

fFfFgGgGfF

fg

fF Pr fFgG Pr gOriginalImage

Degraded Image

Prior Probability


Degradation Process

Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.

E ： Set of all the nearest neighbour pairs of pixels

V ： Set of All the pixels



gG

fFfFgGgGfF

Pr

PrPrPr

fg

fF Pr fFgG Pr gOriginal Image

Degraded Image

Prior Probability


Degradation Process

if

iii fFf\

PrPrˆf

gGfFgG

Computational Complexity

画素

Marginal Posterior Probability of Each Pixel


Posterior Probability for Image Processing by Bayesian Analysis

Ejiji

Viii ffgf

Z },{

22

Posterior 21

21exp

,,1,,Pr

ggGfF

222

},{ 21

41

41exp, jijjiijiji ffgfgfff

1

0

1

0

1

0 },{

22Posterior

1 2 ||21

21exp,,

q

z

q

z

q

z Ejiji

Viii

V

zzgzZ g

Eji

jiji ffZ },{

},{Posterior

,),,(

1Prg

gGfF

Additive White Gaussian Noise 2

1


Bayesian Network and Posterior Probability of Image Processing

Eji

jiji ffZ },{

},{Posterior

,1Pr gGfF

E: Set of all the nearest neighbour pairs of pixels


Probabilistic Model expressed in terms of Graphical Representations with Cycles

gGfFfFgG

gGfF

PrPrPr

Pr

Posterior Probability Bayesian Network


,

,

PrPr

,,,,,,Pr

},{

},{

111111

ijxxPfz

ff

fFfFfFfFfF

ji

z ijjiji

ijjiji

z

LLiiiiii

ii

gGzFgGfF

gG

Markov Random Fields

Eji

jiji ffZ },{

},{Posterior

,1Pr gGfF

Markov Random Fields

∂i: Set of all the nearest neighbour pixels of the pixel i

),( EV ),( EVii


Contents



Algorithm4. Summary


Loopy Belief Propagation

Eji

jiji ffZ },{

},{Posterior

,1Pr gGfF

1

1

115114113112

115114113112

\11 PrPr

z

f

zMzMzMzMfMfMfMfM

fFf

gGfFgG

21

3

4

5

Probabilistic Model expressed in terms of Graphical Representations with Cycles

E: Set of all the nearest neighbour pairs of pixels




MM

Simultaneous fixed point equations to determine the messages

1 2

1

11511411321}2,1{

11511411321}2,1{

221 ,

,

z z

z

zMzMzMzz

zMzMzMfzfM

21

3

4

5


})1210{,,(

,

,

1

0

1

0},{

1

0},{

,q,,,fViij

zMzz

zMfzfM

i

q

z

q

z ikiikjiji

q

z ikiikjiji

jji

i j

i

Step 1: Solve the simultaneous fixed point equations for messages by using iterative method

Belief Propagation Algorithm for Image Processing

i

ji

Step 2: Substitute the messages to approximate expressions of marginal probabilities for each pixel and compute estimated image so as to maximize the approximate marginal probability at each pixel.

})1210{,( Pr 1

0

,q,,,fVifM

fMfF iq

z ikiik

ikiik

ii

i

gG

iiqzi zPfi 1,,1,0

maxargˆ

Belief Propagation in Probabilistic Image Processing



Contents



Algorithm4. Summary


Statistical Estimation of Hyperparameters

z

z

zFzFgG

gGzFgG

}|Pr{},|Pr{

},|,Pr{},|Pr{

},|Pr{max arg)ˆ,ˆ(,

gG

f g

Marginalized with respect to F

}|Pr{ fF },|Pr{ fFgG gOriginal Image

Marginal Likelihood

Degraded ImageVy

x

},|Pr{ gG

Hyperparameters , are determined so as to maximize the marginal likelihood Pr{G=g|,} with respect to , .


||

2

1

Vf

ff

f

Maximum Likelihood in Signal Processing

,Prmaxargˆ,ˆ,

gG

Degraded Image=Data

0 },|Pr{,0 },|Pr{ˆ,ˆˆ,ˆ

gGgG

Extremum Condition

||

2

1

Vg

gg

g

Hyperparameter

Original Image=Parameter

Incomplete Data

Marginal Likelihood

z

zFzFgGgG }|Pr{},|Pr{ },|Pr{


||

2

1

Vf

ff

f

Maximum Likelihood and EM algorithmDegraded Image=Data

0

,Pr,0

,Pr

ˆ,ˆˆ,ˆ

gGgG

Extemum Condition

||

2

1

Vg

gg

g

Hyperparameter

Original Image=Parameter

z

PP

Q

,,ln,,

,,

gGfFgGfF

)(),(,maxarg

)1()1( Update:Step M

)(),(, Calculate :Step E

),(ttQ

t,tα

ttQ

0

,,

0,,

,

,

Q

Q

Expectation Maximization (EM) algorithm provide us one of Extremum Points of Marginal Likelihood.

Q -function

Marginal Likelihood

Incomplete Data

z

zFzFgGgG }|Pr{},|Pr{ },|Pr{


Maximum Likelihood and EM algorithm


gCI

I

zgGzFzf

2

},,|Pr{ˆ

d

Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model

,, ii gf

otherwise0},{1

4Eji

jiji C

Multi-Dimensional Gaussian Integral Formula

Ejiji

Viii ffgf

},{

222 2

12

1exp},,|Pr{

gGfF


Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula

|V|x|V| matrix

E:Set of all the nearest-neghbour pairs of pixels

V:Set of all the pixels


Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model

1T

22

2

11||1

11

1Tr||

1

g

CI

CgCI

C

ttVtt

tV

t

T22

242

2

2

11

11||

1

11

1Tr||

1 gCI

CgCI

I

tt

ttVtt

tV

t

0

0.0002

0.0004

0.0006

0.0008

0.001

0 20 40 60 80 100 t

t

g f̂

gCI

Im2)()(

)(tt

t

2)(minarg)(ˆ tmztf iiz

ii

Iteration Procedure in Gaussian Graphical Model

f̂

g


Image Restorations by Exact Solution and Loopy Belief Propagation of Gaussian Graphical Model

.,,,maxarg1,1,

gttQtt

y

ghf ,ˆ,ˆˆ

0

0.0002

0.0004

0.0006

0.0008

0.001

0 20 40 60 80 100


Exact

0006000ˆ335.36ˆ

LBP

LBP

.

0007130ˆ624.37ˆ

Exact

Exact

.

Image Restorations by Gaussian Graphical Model

Original Image

MSE:315

MSE: 545 MSE: 447MSE: 411

MSE: 1512

Degraded Image

Lowpass Filter Median Filter

Exact

Wiener Filter

2ˆ||

1MSE

Vi

ii ffV

Belief Propagation

MSE:325


Original Image

MSE236MSE: 260

MSE: 372 MSE: 244MSE: 224

MSE: 1529

Degraded Image Belief Propagation

Lowpass Filter Median Filter

Exact

Wiener Filter

Image Restoration by Gaussian Graphical Model


Original Image

MSE:0.98893

Degraded Image

Belief Propagation

Image Restoration by Discrete Gaussian Graphical Model


Q=4

=1

20519.1ˆ05224.1ˆ

LBP

LBP

(t)

(t)MSE: 0.36011

SNR:1.03496 (dB)

SNR: 5.42228 (dB)

Original Image

MSE: 0.98893

Degraded Image

Belief Propagation



Q=4

=1

20294.1ˆ02044.1ˆ

LBP

LBP

(t)

(t)MSE: 0.28796

SNR: 1.07221 (dB)

SNR: 6.43047 (dB)

Original Image

MSE:0.98893

Degraded Image

Belief Propagation



Q=8

=1

660330ˆ96380.0ˆ

LBP

LBP

.

(t)

(t)MSE: 0.37697

SNR:1.89127 (dB)

SNR: 6.07987 (dB)

Original Image

MSE: 0.32060

MSE:0.98893

Degraded Image

Belief Propagation



Q=8

=1

SNR:0.93517 (dB)

SNR: 5.82715 (dB)

623240ˆ85195.0ˆ

LBP

LBP

.

(t)

(t)


Contents



Algorithm4. Summary


Summary

Image ProcessingBayesian Network for Image ProcessingDesign of Belief Propagation Algorithm for Image Processing


iz

LLiiiiii fFfFfFfFfFgGzF

gGfFgG

PrPr

,,,,,,Pr 111111

Practice 10-1

ii z ijjiji

ijjiji

z

fz

ff

,

,

PrPr

},{

},{

gGzFgGfF

For random fields F=(F1,F2,…,F|V|)T and G=(G1,G2,…,G|V|)T and state vectors f=(f1,f2,…,f|V|)T and g=(g1,g2,…,g|V|)T, we consider the following posterior probability distribution:

where ∂i denotes the set of all the nearest neighbour pixels of the pixel i.

and

Eji

jiji ffZ },{

},{Posterior

,1Pr gGfF

Here ZPosterior is a normalization constant and E is the set of all the nearest-neighbour pairs of nodes. Prove that


Practice 10-2

Viii gf 2

22 21exp

2

1Pr

fFgG

Histogram of Gaussian Random Numbers

Original Image f Gaussian Noise n Degraded Image g

Make a program that generate a degraded image g=(g1,g2,…,g|V|) from a given image f =(f1,f2,…,f|V|) in which each state variable fi takes integers between 0 and q1 by according to the following conditional probability distribution of the additive white Gaussian noise:

Give some numerical experiments for =20 and 40.


Practice 10-3Make a program for estimating q-valued original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution for q valued images:

Give some numerical experiments.

Bij

jiij ffZ

,1PrPosterior

gGfF

222

21

81

81exp, jijjiijiij ffgfgfff

1

0

1

0

1

0

222Posterior

1 2 ||21

81

81exp

q

z

q

z

q

zjijjii

V

zzgzgzZ

}1,,2,1,0{ qfi

K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 .M. Yasuda, S. Kataoka and K. Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese).

Make a program for estimating original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution based on the Gaussian graphical model:

Give some numerical experiments.


Practice 10-4

Eji

jiji ffZ },{

},{Posterior

,1Pr gGfF

222

},{ 21

81

81exp, jijjiijiji ffgfgfff

),( if

||21222

Posterior 21

81

81exp Vjijjii dzdzdzzzgzgzZ

Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 and Appendix G.

Physical Fluctuomatics 11th Probabilistic image processing by means of physical models

Documents

Transcript of Physical Fluctuomatics 11th Probabilistic image processing by means of physical models