Post on 13-Dec-2015
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
1
Image Denoising: a Statistical Approach
• Linear estimation theory summary• Spatial domain denoising techniques
• Conventional Wiener filtering• Spatially adaptive Wiener filtering
• Wavelet domain denoising • Wavelet thresholding: hard vs. soft • Wavelet-domain adaptive Wiener filtering
• Experimental Results• Why transform helps?• Why spatial adaptation helps?
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
2
Denoising Problem
aYX ˆ
WXY Noisy measurements
N(0,σw2)
MMSE estimator
Wiener’s idea To simplify estimation by computing the bestestimator that is a linear scaling of Y
Difficulty: we need to know conditional pdf
]|[ˆ YXEX
N(0,σx2)
Orthogonality Principle
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
3
aYX ˆ
0})ˆ{( YXXE
A linear estimator X of a random variable X^
Minimizes E{(X-X)2} if and only if ^
Geometric Interpretation
X
Y
X-X̂
X̂
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
4
Linear MMSE Estimation
YXwx
x22
2
ˆ
),0(~ 2xNX For Gaussian signal
The optimal LMMSE estimation is given by
22
22
wx
xwMMSE
And it achieves
Note: it can be shown such linear estimator is indeedE[X|Y] for Gaussian signal
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
5
What if Signal Variance is Unknown?
222ˆ wx y
Maximum-likelihood estimation of 2x is given by
Since variance is nonnegative, we modify it
],0max[ˆ 222wx y
When multiple observations yi’s are available, we have
]1
,0max[ˆ 2
1
22w
N
iix y
N
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
6
Image Denoising
Theory of linear estimation Spatial domain denoising techniques
• Conventional Wiener filtering
• Spatially adaptive Wiener filtering Wavelet domain denoising
• Wavelet thresholding: hard vs. soft
• Wavelet-domain adaptive Wiener filtering Experimental Results
Why transform helps?Why spatial adaptation helps?
Conventional Wiener Filtering• Basic assumption: image source is modeled by a stationary
Gaussian process• Signal variance is estimated from the noisy observation data• Can be interpreted as a linear frequency weighting
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
7
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
8
Linear Frequency Weighting
),(),(
),(),(
),,(),(),(ˆ
2121
2121
212121
wwSwwS
wwSwwH
wwYwwHwwX
WX
X
22
2
,ˆwx
xaaYX
FT
Power spectrum |X|2
Image Example
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
9
Noisy, =50 (MSE=2500) denoised (MSE=1130)
Image Example (Con’d)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
10
Noisy, =10 (MSE=100) denoised (MSE=437)
Conclusions from the Experiments
• Why did it Fail? • Nonstationary• NonGaussian• Poor modeling
• How to improve?• Achieve spatial adaptation• Use linear transform• Putting them together
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
11
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
12
Spatially Adaptive Wiener Filtering
• Basic assumption: image source is modeled by a nonstationary Gaussian process
• Signal variance is locally estimated from the windowed noisy observation data
]1
,0max[ˆ 2
1
22w
N
iix y
N
T
T
N=T2
Recall
Image Example
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
13
Noisy, =10 (MSE=100) denoised (T=3,MSE=56)
Image Example (Con’d)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
14
Noisy, =50 (MSE=2500) denoised (MSE=354)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
15
Image Denoising
Theory of linear estimation Spatial domain denoising techniques
• Conventional Wiener filtering
• Spatially adaptive Wiener filtering Wavelet domain denoising
• Wavelet thresholding: hard vs. soft
• Wavelet-domain adaptive Wiener filtering Experimental Results
Why transform helps? Why spatial adaptation helps?
From Scalar to Vector Case
1
022
222}||ˆ{||
N
m wm
wmXXE
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
16
1
022
2
][][ˆN
m wm
m mYnX
Suppose X is a Gaussian process whose covariance matrix is adiagonalized matrix RX=diag{ηm}(m=0,…,N-1), the linear MMSEestimator is given by
(A)
and the minimal MSE is given by
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
17
Decorrelating
XTR*
Q: What if X={X[0],…,X[N-1]} is correlated (i.e., Rx is not diagonalized)?
A: We need to transform X into a set of uncorrelated basis and then apply the above result.
The celebrated Karhunen-Loeve Transform does this job!
Diagonal matrix
XX T*'
Karhunen-Loeve Transform
Transform-Domain Denoising
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
18
ForwardTransform
InverseTransform
Denoisingoperation
e.g.,KLTDCTWT
e.g.,Linear Wiener filteringNonlinear Thresholding
Noisysignal
denoisedsignal
The performance of such transform-domain denoising is determinedby how well the assumed probability model in the transform domainmatches the true statistics of source signal (optimality can only beestablished for the Gaussian source so far).
One-Minute Tour of Wavelets
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
19
G0
G1
)(ˆ nxx(n)
H0
H1
y0(n)
y1(n)
x(n)
H0
H1
22 G0
22 G1
)(ˆ nx
s(n)
d(n)
complete expansion (with decimation)
overcomplete expansion (without decimation)
TceTce
-1
Toe Toe-1
Why Wavelet Denoising?• We need to distinguish spatially-localized events (edges) from
noise components
• More about noise components
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
20
Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain
As long as it does not generate exceptions
Additive White Gaussian Noise is just one of them
Wavelet Thresholding
TnY
TnYTnY
TnYTnY
nX
|][|0
][][
][][
][~
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
21
otherwise
TnYifnYnX
0
|][|][][
~
DWT IWTThresholdingY X
~
Hard thresholding
Soft thresholding
Noisysignal
denoisedsignal
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
22
Choice of Threshold
NT elog2
][][
][][
~22
2
nYn
nnX
Donoho and Johnstone’1994
Gives denoising performance close to the “ideal weighting”
Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp. 435-453)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
23
Soft vs. Hard thresholding
|][||][~
|1][
][],[][
~22
2
nYnXn
nanaYnX
● It can be also viewed as a computationally efficient approximationof ideal weighting
|][||][~
|][,][][~
nYnXTnYTnYnX soft
ideal
● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better.
● Shrinking the amplitude by T guarantees with a high probability that.
|][||][~
| nXnX
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
24
Denoising Example
noisy image(σ2=100)
Wiener-filtering (ISNR=2.48dB)
Wavelet-thresholding (ISNR=2.98dB)
2
2
10||
~||
||||log10
XX
YXISNR
X: original, Y: noisy, X: denoised~
Improved SNR
What is Wrong with Wavelets?
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
25
0 1 N-1… …
x(n)
H1
T
-T
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
26
Translation Invariance (TI) Denoising
Toe Toe-1Thresholding
Tce Tce-1Thresholding
Tce Tce-1Thresholding
z
+
x(n) )(ˆ nx
x(n) )(ˆ1 nx
)(ˆ2 nx
2
)(ˆ)(ˆ)(ˆ 21 nxnxnx
Implementation based on overcomplete expansion
Implementation based on complete expansion
z-1
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
27
2D Extension
Noisy image
Tce Tce-1ThresholdingWD =
shift(mK,nK) WD shift(-mK,-nK)
shift(m1,n1) WD shift(-m1,-n1)
Avg
denoised image
(mk,nk): a pair of integers, k=1-K (K: redundancy ratio)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
28
Example
Wavelet-thresholding (ISNR=2.98dB)
Translation-Invariant thresholding (ISNR=3.51dB)
• Challenges with wavelet thresholding• Determination of a global optimal threshold• Spatially adjusting threshold based on local statistics
• How to go beyond thresholding?• We need an accurate modeling of wavelet coefficients –
nonlinear thresholding is a computationally efficient yet suboptimal solution
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
29
Go Beyond Thresholding
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
30
Spatially Adaptive Wiener Filtering in Wavelet Domain• Wavelet high-band coefficients are modeled by a Gaussian
random variable with zero mean and spatially varying variance• Apply Wiener filtering to wavelet coefficients, i.e.,
][][
][][
~22
2
nYn
nnX
estimated in the same wayas spatial-domain (Slide 15)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
31
Practical Implementation
YXwx
x22
2
ˆ
ˆˆ
]1
,0max[ˆ 2
1
22w
N
iix y
N
T
T
N=T2
Recall
Conceptually very similar to its counterpart in the spatial domain
In demo3.zip, you can find a C-coded example (de_noise.c)
(ML estimation of signal variance)
Example
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
32
Translation-Invariant thresholding (ISNR=3.51dB)
Spatially-adaptive wiener filtering (ISNR=4.53dB)
Further Improvements*• Gaussian scalar mixture (GSM) based denoising (Portilla et al.’
2003)• Instead of estimating the variance, it explicitly addresses the
issue of uncertainty with variance estimation• Hidden Markov Model (HMM) based denoising (Romberg et
al.’ 2001)• Build a HMM for wavelet high-band coefficients (refer to the
posted paper)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
33
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
34
Gaussian Scalar Mixture (GSM)*
Model definition: u~N(0,1)
Noisy observation model
Gaussian pdf
scale (variance) parameter
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
35
Basic Idea*In spatially adaptive Wiener filtering, we estimate the variancefrom the data of a local window. The uncertainty with such varianceestimation is ignored. In GSM model, such uncertainty is addressedthrough the scalar z (it decides the variance of GSM). Instead of using a single z (estimated variance), we build a probabilitymodel over z, i.e., E{x|y}=Ez{E{x|y,z}}
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
36
Posterior Distribution*
where
Due to
is so-called Jeffery’s prior
Question: What is E{xc|y,z}?
Bayesian formula
(proof left as exercise)
GSM Denoising Algorithm*
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
37http://decsai.ugr.es/~javier/denoise/index.htmlMATLAB codes available at:
Image Examples
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
38
Noisy, =50 (MSE=2500) denoised (MSE=201)
Image Examples (Con’d)
EE
56
5 A
dva
nce
d I
ma
ge
P
roce
ssin
g C
op
yrig
ht
Xin
Li
20
09
-20
12
39
Noisy, =10 (MSE=100) denoised (MSE=31.7)