Modul 4 Image Restorationthe noise model will takes place on each of the uncorrupted pixel values....

CHAPTER 4Image Restoration

Musa Mohd Mokji

Introduction

• Image enhancement

– A subjective process mainly based on heuristic techniques and what is pleased by human eyes.

• Image restoration

– An objective process based on estimating and modeling the degradation and then applying the inverse. Quality of restoration is measured.

2

Introduction

• Objective is to understand the characteristic of the signal and system so that degraded image can be recovered.

• Graphically:

3

Digitizer DegradedImage

RestorationIdeal

DisplayImage

f(x,y) g(x,y) 𝑓(𝑥, 𝑦)

Causes of Image Degradation

• Noise– Random fluctuations in the signal intensity

– Interfering signals from electronic system of the captor device

• Blur– Contributions of out-of-focus light to the imaging plane

• Illumination (scatter, glare)

• Optic aberrations (spherical aberration, field curvature, line distortion)

4

Degradation/Restoration Model

• The degradation can be modeled as follows:

5

Degradationfunction, H +

Restorationfilter(s)

),( yxf ),(ˆ yxf),( yxg

),( yxNoise

DEGRADATION RESTORATION

Blurring


• The objective is to make f^(x,y) as close as f(x,y).

• In the event that H is linear and position-invariant, then the degraded image g(x,y) can be written as

ℎ(𝑚, 𝑛) – degradation function (a.k.a point spread function, PSF).

𝜂(𝑥, 𝑦) – additive noise

6

),(),(),(),( yxyxfnmhyxg


• In frequency domain the process can be written as (applying Fourier transform):

• The noise is assume as uncorrelated with respect to the image (no relationship between pixel and noise values).

• Noise are normally characterized by the PDF (note: random noise does not possess a Fourier transform).

7

),(),(),(),( vuNvuFvuHvuG

Restoration – Image with Noise Only

• In this case, assume no blurring. Thus,

𝑔 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 + 𝜂(𝑥, 𝑦)

• Or in the frequency domain

𝐺 𝑢, 𝑣 = 𝐹 𝑢, 𝑣 + 𝑁(𝑢, 𝑣)

• Since noise is random we cannot simply generate an approximation of the noise and then subtract it from 𝑔(𝑥, 𝑦) or 𝐺(𝑢, 𝑣).

8


• Estimation of Noise Parameter

– Restoration to image with noise only can be done by obtaining the noise parameter (PDF) in the image and then choose the best spatial filter in reducing the noise.

– One way to obtain the noise characteristic is to capture a set of images of uniform gray level such as a solid gray board under constant illumination.

– If the imaging system is unknown or unavailable, it is possible to estimate the parameters of the noise PDF by considering small patches of constant gray level.

9


• Example 1

10

0

2000

4000

6000

8000

10000

0 50 100 150 200 250

Gray board image captured with phone camera

Histogram showing the presence of Gaussian noise


Sz

ii

i

zpz )()(2

11

• From basic statistics and sample approximations, we can calculate

and

where zi is the pixel in strip S, and p(zi) are the corresponding normalized histogram values.

• For Gaussian PDF, the mean and variance are sufficient since the PDF is characterized by these 2 values.

Sz

ii

i

zpz )(


• This Gaussian noise PDF and other noise PDF characterization will be discussed further in the next subtopic.

• Later, restoration to the image with noise only with spatial filter will be discussed.

• Then, the discussion will continue on restoring image with periodic noise which cannot be characterized with PDF.

12

Noise Model

• Gaussian noise

– Also known as normal noise.

– PDF of Gaussian noise with random variable z :

– z represents gray level, is the mean value of z, and is the standard deviation. 2 is called as the variance of z.

13

2

2

2

)(

2

1)(

z

ezp

Noise Model

• Approximately 70% of its value will be in the range [(- ), ( + )], and about 95% will be in the range [( - 2), ( + 2)].

• Gaussian noise has been used to model noise due to electronic circuit noise and other sensors error due to poor illumination and/or high temperature.

14

z

+ -

2607.0

p(z)

Noise Model

• Rayleigh noise

– PDF is given by

– The mean and variance are

15

azfor

azforeazbzp

b

az

0

)(2

)(

2)(

4/ba

4

)4(2

b

Noise Model

• Rayleigh density is useful for approximating skewed histogram – use for example in histogram modification.

• Also Helpful in characterizing noise phenomena in range imaging.

16

z

b2607.0

2

ba

p(z)

Noise Model

• Erlang (Gamma) noise

– PDF is given as

– a > 0, b is a positive integer. The mean and variance of this density

17

00

0)!1()(

1

zfor

zforeb

za

zp

azbb

a

b

2

2

a

b

Noise Model

• This density is called the gamma density only when the denominator is the gamma function, (b).

• Helpful in modeling noise in laser imaging.

18

a

b 1

K

z

p(z)

)1(1

)!1(

)1(

b

b

eb

baK

Noise Model

• Exponential noise

– The PDF is given by


19

00

0)(

zfor

zforaezp

az

0a

a

1

2

2 1

a

Noise Model

• Note: when b = 1 then Erlang PDF becomes exponential PDF.

• As with gamma noise, exponential noise is helpful in modeling noise in laser imaging.

20

Noise Model

• Uniform noise

– The PDF is given as


21

otherwise

bzaifabzp

0

1

)(

2

ba

12

)( 22 ab

Noise Model

• This type of noise is commonly used as a basis for numerous random number generators.

22

ab

1

a b z

p(z)

Noise Model

• Impulse (salt-and-pepper) noise– The PDF is given as

– If b > a, gray level b will appear as a light dot in the image. On the contrary, level a will appear as a black spot.

– If either Pa or Pb is zero, this noise is called a unipolar noise.

23

otherwise

bzforP

azforP

zp b

a

0

)(

Noise Model

• Noise impulses can be negative or positive.

• Impulse noise is digitized as extreme values (pure black or white) values. Hence a and b normally equal to the minimum and maximum allowed values in the digitized image. As a result negative impulses appear as black (pepper) dots and positive impulses appear as white (salt) spots.

24

bP

aP

a b

Effect of Noise

• We can see the effect of these noise by applying them to some simple test images.

• The original image and its histogram are as shown on the right side.

• The image has somewhat 3 constant gray levels as depicted by the histogram.

25

Effect of Noise

26

Gaussian noise Rayleigh noise Gamma noise

Effect of Noise

27

Exponential noise Uniform noise Salt-and-Pepper noise

Effect of Noise

• Amount of noise in image is high when the distribution of the PDF is wide (high variance) except for impulse noise where it is determine by the value or density of Pa and Pb

• Based on the previous examples, it shows that the shape of the noise model will takes place on each of the uncorrupted pixel values.

• Thus, the objective of restoration is to remove the unwanted pixel values that shapes the noise.

• Basically it can be done by finding the peaks in the histogram, which normally work fine by computing mean.

28

Arithmetic Mean Filter

• Simplest of the mean filters.

• Computes the average value of the corrupted image g(x,y) in the area or window defined by Sxy with size m x n and centered at point (x,y).

• The restored image is computed as

29

xySts

tsgmn

yxf),(

),(1

),(ˆ

3 x 3 window

Noise Reduction with Spatial Filter

• Example

30

Original image Noisy image – additive

Gaussian with 2=0.002


31

Filtered image – using

arithmetic mean filter

Noisy image – additive

Gaussian with 2=0.002

Geometric Mean Filter


• This filter achieves smoothing comparable to arithmetic mean filter, but tends to lose less image detail in the process.

32

mn

Sts xy

tsgyxf

1

),(

),(),(ˆ

Geometric Mean Filter

• Example

33

Filtered image using geometric

mean filter. Notice spurious black

spots. This is the effect when a

pixel in the window is zero or near

zero. The power factor reduces the

value even further causing the

spots.

However, the image is less blur

compared to output from the

arithmetic mean filter

Harmonic Mean Filter


• This filter works well for salt noise, but fails for pepper noise.

34

xySts tsg

mnyxf

),( ),(

1),(ˆ

Contraharmonic Mean Filter

• Q is the filter order.• Meant for salt-and-pepper noise. Positive Q

eliminates pepper noise while negative Q eliminates salt noise.

• If Q=0 filter is arithmetic mean and if Q=-1 filter is harmonic mean.

35

xy

xy

Sts

Q

Sts

Q

tsg

tsg

yxf

),(

),(

1

),(

),(

),(ˆ

Order-Statistics Filters

• Median filter

• Max filter

• Min filter

• Midpoint filter

36

),(median),(ˆ),(

tsgyxfxySts

),(max),(ˆ),(

tsgyxfxySts

),(min),(ˆ),(

tsgyxfxySts

),(min),(max

2

1),(ˆ

),(),(tsgtsgyxf

xyxy StsSts

Order-Statistics Filters

• Median filter: median filter can be applied iteratively but the image will become blur.

• Max filter: finding brightest spots & reduce pepper noise

• Min filter: finding darkest spots & reduce salt noise

• Midpoint filter: computes the midpoint between the max and min intensities in areas Sxy. This filter is good at reducing randomly distributed noise like Gaussian or uniform noise.

37

Alpha-trimmed Mean Filter

• The d/2 lowest and the d/2 highest gray-level values are deleted and gr(s,t) represents the remaining mn – d pixels.

• Useful in situations involving multiple types of noise, such as combination of salt-and-pepper and Gaussian noise.

• d = 0 arithmetic mean

• d = (mn – 1) median filter

38

xySts

rtsg

dmnyxf

),(

),(1

),(ˆ

Adaptive Filters

• Filter coefficients are not global but rather change according to the statistical characteristic of the image inside the predefined window mas.

• Performance-wise is normally increased but at the expense of complexity of the filter as well as computation time.

39

Adaptive Filters

• Adaptive Local Noise Reduction Filter

– Based on mean and variance.

– Recall,

• Mean – measures average gray level over the region of interest.

• Variance – measures average contrast in that region.

– The response of the adaptive filter is based of 4 quantities:• G(x,y) – value of the pixel at location (x,y).

• 2 – variance of noise (x,y)

• mL – local mean of the pixels in Sxy.

• 2L – local variance of noise in Sxy.

40

Adaptive Filters

• The filter should have the following behavioral outcome:

– If 2 = 0, filter output = g(x,y). zero noise case

– If 2L > 2

, filter output should close to g(x.y). A high local variance indicates presence of edges and thus should be preserved.

– If 2L = 2

, filter outputs an arithmetic mean value of pixels in Sxy. Local area has the same property with global area and thus noise should be filtered out.

41

Adaptive Filters

• One implementation of such filter is

• 2 needs to be estimated (the only unknown quantity).

• In the case that 2 > 2

L , the output gray level might results negative value. To prevent this:

– set the ratio to 1 when 2 > 2

L- nonlinear filter.

– Allow negative value but rescale at the end – loss dynamic range.

42

L

L

myxgyxgyxf ),(),(),(ˆ2

2

Adaptive Filters

• Adaptive Median Filter

– The global median filter works fine when the spatial density of the impulse noise is not large, Pa and Pb less than 0.2.

– Adaptive median should be used when greater than the above number.

– Adaptive median filter also smoothes non-impulsive noise while preserving the detail.

43

Adaptive Filters

– Adaptive median increases the size of its window depending on certain conditions (the center is still at the same position (x,y) . Other adaptive filters do not change the size.

– The following notation will be used:• zmin = minimum gray level value in Sxy.

• zmax = maximum gray level value in Sxy.

• zmed = median of gray levels in Sxy.

• zxy = gray level at coordinate (x,y).

• Smax = maximum allowed size of Sxy.

– Adaptive median works in two levels A and B

44

Adaptive Filters

– Level A• A1 = zmed - zmin

• A2 = zmed - zmax

• If A1> 0 AND A2 < 0, go to level B• Else increase the window size• If window size ≤ Smax repeat level A• Else output zxy .

– Level B• B1 = zxy - zmin

• B2 = zxy - zmax

• If B1 > 0 AND B2 < 0, output zxy.• Else output zmed.

45

Check whether zmed is an impulse or not. If yes, it cannot be used to replace the center value. Thus, increase window size to obtain new zmed.

If zmed is not an impulse, check whether zxy is an impulse or not. If yes, output with zmed. If no, preserve the value.

Adaptive Filters

– zmed and zxy are not an impulse when their values are between zmin and zmax (when line 3 and 9 are true)

– Example:

• Suppose in level A, zmed is an impulse, then the window size is increased and level A is repeated.

• The looping is repeated until the algorithm either finds a median value that is not an impulse (go to level B) or the maximum window is reached (in this case the algorithm returns a value of zxy).

46

Adaptive Filters

– Advantage compare to the global median filter:

• It can avoid replacing the center value with an impulse which will worsening the image.

• It will preserve the center value when it is not an impulse, hence reduce the loss of details in the image.

47


• Example

48

255 150 148 255 147

255 0 255 146 0

145 255 255 148 146

145 147 147 0 145

255 0 255 148 146

When the blue pixel is zxy and the initial window size is 3x3:

- At level A, zmed = zmax = 255, which is an impulse. Thus, increase the window size to, let say 5x5.

- Now, zmed = 147, which is not an impulse. Thus, go to level B

- At level B, zxy = zmin = 0, which is an impulse. Thus, output is equals to zmed = 147

Modul 4 Image Restorationthe noise model will takes place on each of the uncorrupted pixel values....

Documents

Transcript of Modul 4 Image Restorationthe noise model will takes place on each of the uncorrupted pixel values....