A Hybrid Filter for SEM Image Enhancement · filter using a stander image with many noise levels...
Transcript of A Hybrid Filter for SEM Image Enhancement · filter using a stander image with many noise levels...
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
119 USA, Michigan
A Hybrid Filter for SEM Image Enhancement
Ahmed Asal Kzar*1
, Hussein Abdullah Alshamarti*2
*Physics Department, Faculty of Science, University of Kufa, Najaf, Iraq.
Corresponding author: Ahmed Asal Kzar, Ph.D. Tel: 009647732386850
Abstract
Scanning electron microscopy (SEM) images is a special and important type
of images that need special enhancement to get more information for the nanoscale
phenomenon. Many studies have been introduced in this purpose with the traditional
enhancement and suggested methods but here in this study a new method represented
in a hybrid filter formed by mean and Sobel that are common filters in enhancement
and edge detection respectively. The results are evidence for the success of the new
filter using a stander image with many noise levels before applying it on the SEM
image. The new filter gave better results comparing with the mean filter as a
traditional enhancement filter. The new filter is added as new technique filter for
Nano scale images.
Keywords:
SEM; Hybrid Filter; image enhancement; mean filter; Soble filter; Gaussian noise.
Introduction
The main tool of getting information for characterization of materials at the Nano
scale is scanning electron microscopy (SEM) [1]. The advanced technology nodes
have size of several structures that are reaching to sub-10nm, resulting in the contact-
edge roughness (CER) and the line-edge roughness (LER) smaller than few
nanometers. Because of this, the ability of the image to give such small structures
becomes critical in both inspection and metrology. Scanning electron microscopy
(SEM) is the most widely used for inspection of information about nanostructure
scale [2]. There are many reasons that produce distortion (noise) that reduces the
information in the image of (SEM). The main two important things in the image are
the color (homogenous part of the object) and the edge (limit part of the object)[3].
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
120 USA, Michigan
After getting resulted image that have noise by a reason or other, we have to find the
best solution for this problem to remove it or to reduce its impact, in the image
processing there are many techniques to solve the noise problem, these techniques
depend on the type of noise that produced with the image.
In information theory there are three forms of information ascending from
lowest level to the highest level: first level is the text form, second level is the sound
form, and third level is the image. Therefore, we found that always the researchers,
scientists, practitioners, engineers, and geologists depend on the image because of the
huge information that it contents. In Nano-technology field the image and its
processing like SEM image and other techniques have been widely used to help in
giving important information about these microscope phenomena. K. S. Sim and M.
E. Nia proved the efficiency and the accuracy of the image noise cross correlation
estimation model, vs. other existing estimators, they applied to different kinds of
scanning electron microscope images[4]. Naresh Marturi and others got the quality of
images obtained by a tungsten gun SEM that has been evaluated by quantifying the
level of image signal-to-noise ratio (SNR). To determine the SNR, an efficient and
online monitoring method is developed based on the nonlinear filtering using a single
image[5]. Myungjun Lee and others enhance the resolution of the CD-SEM
metrology by applying the super-resolution (SR) technique using the advanced signal
processing algorithms. Where the resolution enhancement verified by exploiting the
multiple low resolution (LR) images that contained especial information of an
imaging target through looking at a slightly different position[2]. Mike Williamson
and Andrew Neureuther introduced A recovering more accurate line edge roughness
(LER) data from SEM images by using an image deblurring technique, this study
prove that LER can vary significantly before and after deblurring[6]. E. Giannatou
and others, They trained SEM image Denoising model (SEMD) on synthesized image
data to find the variations of noise and provides as an output a denoised image of the
pattern[7].
The researchers and scientists are still introduce several image techniques to be
the best according to the giving resolution and keeping the image details that
investigated visually by human eye and quantity by statistical measurements. So,
there is no ideal enhancement technique to cover all distortion issues in SEM
imaging. Here in this work we create a new filter by combining two common
different filters from two purposes in image processing methods: mean filter which
use for smoothing and Soble filter which use for edge detection, where both of them
in one process.
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
121 USA, Michigan
Material and method
Mean filter:
Mean filter may be an easy, simple and self-evident to implement technique of
enhancing images, i.e. reducing the high difference between the intensities of the
pixel and the next one. It's usually accustomed cut back noise in the digital images.
The thought of mean filtering is just to interchange every pixel value in an image
with the mean (average) value of its neighbors, as well as itself. This filter eliminates
pixel values that square measure atypical of their surroundings. mean filter calculates
the mean value of the all pixels values of the filter size then be takes the center of the
filter[8].
it is represented by the following equation[9]:
∑ (1)
Where L is the maximum number of filter size, k is the index of pixels.
Sobel filter:
It is represented by the Gradient operator that commonly used for differentiation
in image processing. The gradient of the function I(x, y) is defined as the vector[10]:
yG
xG
y
I
x
I
)y,x(IG
(2)
Gradient operator is applied to a continuous function which produces a vector at
each point whose direction represents maximum change of the function at that point,
and whose magnitude gives magnitude of this maximum change[11, 12]. Of course
an image is not a continuous function, and it cannot be differentiated in the usual
way. Because a digital image is discrete, we use differences instead of derivatives
that are the derivative at a pixel which is approximated by the difference in gray
levels over some local region. These can then be combined together to find the
absolute magnitude of the gradient at each point and the orientation of that gradient.
The gradient magnitude is given by[13, 14]:
22 GyGx)y,x(G (3)
Typically, an approximate magnitude is computed using[15]:
GyGx)y,x(G (4)
The angle of orientation of the edge (relative to the pixel grid) giving rise to the
spatial gradient is given by:
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
122 USA, Michigan
Gx
Gytan 1 (5)
Where:
)y,nx(I)y,nx(IGx (6)
ny,x(I)ny,x(IGy (7)
Where n is a small integer, usually unity. Noted that numbers in Soble filter will be
changed by N of the new filter, so it led to change entire result which represents the
novelty of the new work. The edge detection masks by Sobel filter look for edges in
both of horizontal and vertical directions and then combine this information into
single metric[16]. The Sobel operator performs a 2-D spatial gradient measurement
on an image and so emphasizes regions of high spatial frequency that corresponds to
edges. The operator consists of a pair of (3x3) convolution kernel.
Sobel operator is larger convolution kernel that makes smoothing the input
image to a greater extent, in addition to makes the operator less sensitive to noise[17].
Therefore, we pick it to serve SEM images.
Sobel operator is one of the edge detection operators that return orientation
information (about the direction of the edge) , while others return only the
information about the existence of an edge in each point as Laplace operator. When
we apply the edge detection operators with an image we use the convolution equation
to get on the gradient, as in the following equation:
)jy,ix(I.)j,i(w)y,x(edg
B
Bi
B
Bj
2
1
2
1
2
1
2
1 (8)
Gaussian Noise
Gaussian noise is the most common noise in SEM images[18], therefore we
focus on this type in this work. Gaussian noise is statistical noise which has a
probability density function of the normal distribution. This means, the noise can
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 -1
GX Gy
W(j,i)= W(i,j)=
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
123 USA, Michigan
takes on values as Gaussian-distributed. This type of noise is properly known as the
noise with a Gaussian amplitude distribution. Noise is modeled as additive white
Gaussian noise (AWGN), where all the image pixels values changed from their
original values because of the Gaussian curve[19]. Thus, for each image pixel with
intensity value Iij (1 ≤ i ≤ m, 1 ≤ j ≤ n for an m x n image), the noisy image that
corresponding pixel of gij are presented by the following equation[20]:
gij = Iij +Nij (9)
where, each noise value Nij is drawn as starting from zero then it takes normal
distribution.
Creating a Hybrid Filter
The creating digital filter in image processing requires understanding the main
purpose of this filter exactly, in the other word understanding the physical
phenomenon that represent the image and what the researcher want to do (process),
then representing the process digitally. In the other hand, the digital process be as
matrix contain numbers. When the noise is our problem in the image, we can use one
of the traditional filters of removing noise or enhance the noisy image, mean filter is
the most common filter that used to this task, but there is some image information
loss while it uses, especially the edges of the objects in the image, the matter that
need to maintain these information. So, because the filer number represent the main
impact with image information, the manipulation these number will help us in
maintaining information. We add a new factor with mean filter which is N factor as
in equation (10). In addition to, we mix the common filter for edge detection in
image processing as multiplying N by Sobel filter. Sobel filter helps us in maintain
the edges of objects in the image. As a result, we get enhancement the image by mean
and maintain the edge by Sobel filter with 3x3 size of window for both of them, as in
the new created equation (10 ) which represents the new hybrid filter in this work.
The hybrid filter formed by according to the following equation:
IN=MeanN
+ N*Sobel (10)
Where IN is the enhanced digital number of the resulted image, Mean is the mean
filter for enhancing the digital image, Sobel is Sobel filter for edge detection of the
digital image, and N is the factor of the new filter.
Work Algorithm
Step 1- Read the original image (standard image).
Step 2- Add Gaussian noise over a range of noise levels [0.01, 0.05] by interval
(0.01) to the original image to produce five noisy images.
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
124 USA, Michigan
Step 3- Apply mean filter using equation (1) with size of 3x3 to the entire noisy
image to produce five enhanced images.
Step 4- Calculate MSE between the original image and the first enhanced image by
mean.
Step 5- Repeat step 4 to calculate MSE for all enhanced images by mean filter.
Step 6- Apply the new filter using equation (10) with 3x3 of size the first noisy image
with varying of N value in the equation (10) starting from (0.04) to (0.40)
with interval (0.04) to produce 10 enhanced images.
Step 7- Repeat (step 6) to all other noisy images.
Step 8- Calculate MSE between the original image and the first enhanced image by
the new filter.
Step 9- Repeat step 8 to calculate MSE for all enhanced images by the new filter.
Step10- Pick the best N value that gave the lowest MSE to be considered in the new
filter for applying it with SEM images.
Results and discussion
V=0 V=0.04 V=0.08
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
125 USA, Michigan
V=0.12 V=0.16 V=0.20
Figure 1: the standard image and its noisy images
The new filter that represented by the work algorithm gave clear results in the
main two types of standard results: first, quantity which represented with table and
figures. Second, visually which represented with enhanced images? So the reader can
understand the results sequentially according to the work algorithm steps. Where the
results begin with the stander image then ended with the SEM image of the main
application that push us to achieve this work.
Results of the Stander Image
The main cause of using a stander image is because it is clear from noise and the
required image to enhance (SEM image) contains amount of noise that does not allow
to us to study and test as validation method for the new filter. Therefore, we applied
many steps on it represented by adding amounts of Gaussian noise started from 0.01
to 0.05 with an interval 0.01 as standard deviation in Gaussian noise equation. The
figures below demonstrate the N factor with mean squire error.
Figure 2: the relationship between n factor and MSE at sd=0.01
Sd=0.01
Mean filter
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
126 USA, Michigan
Figure 3: the relationship between n factor and MSE at sd=0.02
Figure 4: the relationship between n factor and MSE at sd=0.03
Figure 5: the relationship between n factor and MSE at sd=0.04
Sd=0.04
Sd=0.02
Sd=0.03
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
127 USA, Michigan
Figure 6: the relationship between n factor and MSE at sd=0.04
The adopted N in equation (10) will be from above figures. Where each n value
has especial MSE which has a not progressive variance from figure to other, this
phenomenon is because of the producing changing of N value to the both kernels of
the used filters (mean and Soble), especially on Soble filter because it has variance in
kernel values (-,0,+) and it has two directions (vertical and horizontal) as we mention
ion the equations (6) and (7). Therefore, we did not get a gradual changing in the
MSE value. Whatever, the adopted n for the main equation of the new filter is
depending on the n which has the highest iteration in these five figures.
The comparison of the new filter with mean filter through the different amounts of
Gaussian noise is demonstrated by the following table.
Results of Real SEM Image
Figures 5 and 6 related with the real images of SEM that are the main images
required for enhancement depending on tests on the standard image.
0
0,5
1
1,5
2
2,5
0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 3,6 4
Iteration
Iteration
Sd=0.
Sd=0.05
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
128 USA, Michigan
Figure 7: the relationship between N factor and its iteration for choosing N
a b c
d e f
Figure 8: a, b, and c the real SEM images. d, e, and f the enhanced images by
applying the new filter with (N=2).
The results of real images of SEM show the impact of the couple of processes
smoothing and edge maintaining that enhance the generated noise and keep one of the
main image information related with objects edges. Also, the new filter results prove
that are very appropriate and necessary for an image like SEM image, because it
contains a lot of shadow between rods that produce high differences between rods
colors and it, where this considered the main cause of noise. The needed filter like the
new filter in this work is very important to element noise and maintains the lines and
edges in micro image like SEM image.
Conclusion
The new hybrid filter proved its success through union of the tow important
processing for the digital image; smoothing (mean filter) and edge detection (Soble
filter) for enhancing important image type which is SEM image in the
nanotechnology. The new filter share in extracting fine information of
nanotechnology that developed by the biggest form of information represented in
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
129 USA, Michigan
digital image. The new filter reduces time and efforts for distinguishing nanostructure
components and help manufacturing of new composite materials.
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
130 USA, Michigan
Reference
[1] K. de Haan, Z. S. Ballard, Y. Rivenson, Y. Wu, and A. Ozcan, "Resolution enhancement in scanning electron microscopy using deep learning," Scientific reports, vol. 9, pp. 1-7, 2019.
[2] M. Lee, J. Cantone, J. Xu, L. Sun, and R.-h. Kim, "Improving SEM image quality using pixel super resolution technique," in Metrology, Inspection, and Process Control for Microlithography XXVIII, 2014, p. 90500U.
[3] A. A. D. Al-Zuky and Ahmed Asal Kzar, "Hit noise reduction in some x-ray images," Al-Nahrain Journal of Science, vol. 12, pp. 78-86, 2009.
[4] K. Sim, M. Nia, and C. P. Tso, "Noise variance estimation using image noise cross‐correlation model on SEM images," Scanning, vol. 35, pp. 205-212, 2013.
[5] N. Marturi, S. Dembélé, and N. Piat, "Scanning electron microscope image signal‐to‐noise ratio monitoring for micro‐nanomanipulation," Scanning: The Journal of Scanning Microscopies, vol. 36, pp. 419-429, 2014.
[6] M. V. Williamson and A. R. Neureuther, "Enhanced quantitative analysis of resist image contrast upon line-edge roughness (LER)," in Advances in Resist Technology and Processing XX, 2003, pp. 423-432.
[7] E. Giannatou, G. Papavieros, V. Constantoudis, H. Papageorgiou, and E. Gogolides, "Deep learning denoising of SEM images towards noise-reduced LER measurements," Microelectronic Engineering, vol. 216, p. 111051, 2019.
[8] S. Tiwari, A. Bansiya, and R. K. Paul, "Elimination of Impulse Noise using Enhanced Mean Median Filter for Image Enhancement," International Journal of Engineering and Management Research (IJEMR), vol. 5, pp. 811-818, 2015.
[9] Z. Lv, J. Shi, and Y. Wang, "AN ADAPTIVE MORPHOLOGICAL MEAN FILTER FOR VERY HIGH-RESOLUTION REMOTE SENSING IMAGE PROCESSING," International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, vol. 42, 2017.
[10] R. C. Gonzalez, "P. wintz digital image processing," Addision-Wesley Publishing Company, pp. 275-281, 1987.
[11] H. Alshamarti, A. K. Hussein, and B. Almayahi, "Application of median filter with the threshold technique to reduce and remove gaussian noise on the image edges produced by sobel operator," International Journal of Computer Engineering & Technology (IJCET), vol. 4, pp. 121-126, 2012.
[12] W. Niblack, "An introduction to digital image processing. 1986," in G. Leedham, C. Yan, K. Takru, J. Tan and L. Mian. Comparison of Some Thresholding Algorithms for Text/Background Segmentation in Difficult Document Images. In Proceedings of the Seventh International Conference on Document Analysis and Recognition, 2003, pp. 859-864.
American Journal of Research www.journalofresearch.us
5-6, May-June 2020 [email protected]
131 USA, Michigan
[13] O. R. Vincent and O. Folorunso, "A descriptive algorithm for sobel image edge detection," in Proceedings of Informing Science & IT Education Conference (InSITE), 2009, pp. 97-107.
[14] H. Khosravi and E. Kabir, "Farsi font recognition based on Sobel–Roberts features," Pattern Recognition Letters, vol. 31, pp. 75-82, 2010.
[15] R. W. Smith, "Computer processing of line images: A survey," Pattern recognition, vol. 20, pp. 7-15, 1987.
[16] S. Gupta and S. G. Mazumdar, "Sobel edge detection algorithm," International journal of computer science and management Research, vol. 2, pp. 1578-1583, 2013.
[17] R. Fisher, S. Perkins, A. Walker, and E. Wolfart, "Hypermedia image processing reference," Department of Artificial Intelligence, University of Edinburg, available on http://www. cee. hw. uk, 1996.
[18] K. S. Sim, V. Teh, and M. E. Nia, "Adaptive noise wiener filter for scanning electron microscope imaging system," Scanning, vol. 38, pp. 148-163, 2016.
[19] C. Xu, S. Zhao, Y. Ma, B. Huang, and F. Liu, "Robust filter design for asymmetric measurement noise using variational Bayesian inference," IET Control Theory & Applications, vol. 13, pp. 1656-1664, 2019.
[20] H. Alshamarti, "Removal of Gaussian noise on the image edges using the Prewitt operator and threshold function technical," Journal of Computer Engineering, vol. 15, pp. 81-85, 2013.