ACCEPTED...
Transcript of ACCEPTED...
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
Tiny surface defect inspection of electronic passive components using Discrete Cosine Transform decomposition and cumulative sum techniques
HONG-DAR LIN
Department of Industrial Engineering and Management,
Chaoyang University of Technology,
168 Jifong E. Rd., Wufong Township, Taichung County, 41349, Taiwan (R.O.C.)
E-mail: [email protected]
Fax: 886-4-2374-2327
Tel: 886-4-2332-3000 Ext 4258
Abstract
Passive components, owing to their low or no power consumption, are widely used in
modern electronic devices. Nevertheless, tiny defects that often appear in the
surface of passive components impair not only their appearances but also their
functions. This paper proposes a global approach for the automated visual
inspection of tiny surface defects in SBL (Surface Barrier Layer) chips, whose
random surface texture contains no repetitions of basic texture primitives. The
proposed method, taking advantage of the DCT decomposition and the cumulative
sum techniques, does not requires textural features, the lack of which often limits the
application of feature extraction based methods. We apply the cumulative sum
algorithm to the odd-odd matrix that gathers most power spectra in the decomposed
DCT frequency domain, and select the large-magnitude frequency values that
represent the background texture of the surface. Then, by reconstructing the
frequency matrix without the selected frequency values, we eliminate random texture
patterns and retain anomalies in the restored image. Experimental results
demonstrate the effectiveness of the proposed method in inspecting tiny defects in
random textures.
Keywords: Tiny surface defect inspection; Electronic passive components; Discrete
cosine transform decomposition; Cumulative sum technique.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
1
1. Introduction
1.1 Popularity of passive components
There are two kinds of electronic components: (1) passive ones that contribute no
power amplification in the circuit system and require only a signal to start their
functions (e.g. resistors, capacitors and inductors) and (2) active ones that are
capable of controlling currents and creating a switching action in the circuit (e.g.
diodes, transistors and integrated circuits). As passive components require low or
no power consumption, their importance and popularity have been increasing these
years when energy conservation issues intensify. Consumers of electronic products
are asking for better energy efficiency, in addition to smaller sizes, lighter weight,
more stylish outlooks, etc. The number of passive components used in Printed
Circuit Boards (PCBs) has grown to a large extent in much smaller electronical
products. More than 80% of the parts in a motherboard are composed of passive
components [1, 2]. Also, the sizes of passive components have shrunk to 5x2.5x2.5
mm for sophisticated products [3].
1.2 Surface Defects of passive components
With the popularity of passive components, inspection of surface defects has
become a critical task for manufacturers who strive to improve product quality and
production efficiency of passive components. Surface defects affect not only the
appearances of passive components but also their functionality, efficiency and
stability. Large and obvious surface defects such as indents, scraps and scratches
are usually inspected by automated visual inspection systems [3, 4, 5, 6, 7, 8]. But
tiny surface flaws such as dust, cavities and pinholes are very difficult to detect
because of their extremely small sizes. Tiny surface defects, frequently occurring
in the manufacturing process of passive components, cause much greater harms and
impact when they appear in high-tech products than in industrial parts. Therefore,
to survive in today’s competitive market of high-tech products, manufacturers of
passive components simply can not afford to ignore tiny surface defects.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
2
To contribute to the field of surface defect inspection on passive components, this
research uses disk-like Surface Barrier Layer (SBL) chips as testing samples. SBL
chips are passive components (ceramics capacitors) commonly used in many
electronic appliances. With a thickness of 0.05 mm and a diameter of 7 mm, SBL
chips are small in size, light in weight, and suitable for large-lot-sized production.
However, in the surfaces of SBL chips often exist tiny defects that cause problems
such as unstable and non-conforming capacitance although the defects occupy only
an extremely small area. For a SBL chip image of 256 x 256 pixels, the size of its
tiny defect falls within the range of 1~15 pixels and occupies 0.0015%〜0.0229%
of the image area. Surface defects of this magnitude are defined as tiny defects in
this research. Figure 1 shows SBL samples with tiny defects marked in circles.
Figure 1. Samples of SBL chips with tiny surface defects
Currently, the most common detection methods for passive component defects are
human visual inspection and electrical functional tests. Human visual inspection is
tedious, time-consuming and highly dependent on the inspectors’ experiences,
conditions, or moods. Erroneous judgments are easily made because of inspectors’
subjectivity and eye fatigues. Electronic functional tests are inherently limited to
offline operations and are normally conducted after passive components have been
fabricated. Difficulties exist in precisely inspecting tiny flaws by machine vision
systems because when product images are being captured, the area of a tiny flaw
could expand, shrink or even disappear due to uneven illumination of the
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
3
environment, complex texture of the product surface, and so on. Seeing the great
need for a real-time, compute-aided, visual detection scheme for passive component
defects, we propose a Discrete Cosine Transform (DCT) based image
reconstruction approach to overcome the difficulties of traditional machine vision
systems.
1.3 Defect detection methods
This paper proposes a global approach for the automated visual inspection of tiny
surface defects in SBL chips, whose random surface texture contains no repetitions
of basic texture primitives. Generally, automated visual inspection techniques for
defects fall into two categories. Approaches of the first category detect defects by
comparing the testing image with a pre-stored faultless item through a pixel-to-
pixel or template-to-template matching process. Disadvantages of the first category
include: (1) time consuming of the matching process, (2) sensitivity to noises, and
(3) large data storage needed for the pre-stored templates and the matching process
[9, 10]. Approaches of the second category select and compute a set of textural
features in a sub-image, and then compare the feature values with those of the
faultless image to find significant differences. Disadvantages of the second
category include: (1) difficulties in deciding what valuable features to extract for
texture discrimination, (2) failure of the valuable features of a specific texture in
depicting other texture patterns, (3) time consuming of the training process for the
addition of a new texture, and (4) sophisticated classifiers needed to discriminate
texture classes when a vector of multiple features results in high dimensionality [4,
11].
Textures can be divided into two types: structural and statistical [12]. Structural
textures consist of repetitions of basic texture primitives such as lines, arcs or
circles. Commonly found in textile fabrics, structural textures have regular and
homogeneous properties. On the other hand, statistical textures do not possess
repetitive primitives or regular patterns. They arise randomly in the surfaces of
objects such as sandpaper and leather.
Techniques that analyze textures based on local features can also be classified into
two kinds: the spatial domain and the frequency domain. First, a traditional
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
4
technique in the spatial domain is the co-occurrence matrix method. Siew et al.
[13] use two-order gray level statistics to build up probability density functions of
intensity changes and apply the co-occurrence matrices to evaluate levels of carpet
wear. Amet et al. [14] present co-occurrence matrices based on wavelet sub-image
characteristics to detect defects of textile fabrics. Secondly, techniques in the
frequency domain extract textural features by conducting frequency transforms
such as Fourier transforms, Gabor transforms, or wavelet transforms. Among the
frequency transforms, wavelet transforms have been a popular alternative for
extracting textural features these years. For example, Tico et al. [15] apply the
wavelet transform of the frequency domain to the field of finger-print recognition.
Sari-Sarraf and Goddard [16] present a wavelet-based preprocessing module for the
detection and localization of woven fabric defects.
Some global approaches have also been developed to detect defects without relying
on local features of textures. Tsai and Hsieh [4] present a new approach for the
automatic inspection of defects in directionally textured surfaces such as textile
fabrics and machined surfaces. Tsai and Huang [5] propose a global approach for
the automatic inspection of defects in randomly textured surfaces such as
sandpaper, castings, leather, and many other industrial materials. Both of the above
two methods make use of the Fourier transform and are based on global image
restoration and reconstruction schemes. Tsai and Chiang [11] present a wavelet
based multi-resolution approach for the inspection of local defects embedded in
homogeneously textured surfaces, such as machined surfaces, natural wood,
sandpaper and textile fabrics. By properly selecting wavelet parameters, the
wavelet based multiresolution approach can effectively remove the global repetitive
texture pattern and reserve only local anomalies in the restored image.
As to inspecting surface flaws of electronic components, some machine vision
systems have been applied to PCBs [9, 17], Ball Grid Array (BGA) substrate
conducting paths [10], and capacitor chips of passive components [3, 18]. Lu and
Tsai [6], and Lu and Tsai [7] use singular value decomposition to inspect surface
defects of Thin Film Transistor-Liquid Crystal Display (TFT-LCD) panels. Jiang et
al. [8] apply analysis of variance and exponentially weighted moving average
techniques to uniformity defect inspection of LCD displays.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
5
1.4 DCT based approaches
The inspection task of this paper involves detecting novel but obscurely faulty
items, tiny defects on electronic passive components. Many of these unanticipated
defects are extremely small in size and can not be described by explicit measures,
thus making automated defect detection difficult. Consequently, we present a new
global image reconstruction scheme using DCT for tiny surface defect detection.
Ahmed et al. [19] first define DCT as one-dimensional (1-D) and suitable for 1-D
digital signal processing. Most of the researchers that apply DCT for image
processing focused on image compression and image reconstruction [20, 21, 22],
because DCT has the property of packing the most information into the fewest
coefficients [23, 24]. However, due to the lack of an efficient algorithm, DCT had
not been applied as widely as its properties imply. Only until recently, many
algorithms and VLSI architectures for the fast computation of DCT have been
proposed [25, 26, 27].
As most coefficients in the DCT domain are small and can be quantized to zero,
manipulating data in the DCT domain is an efficient way to save computer
resources [23]. To enhance edges of remote sensing image data in the DCT
domain, Chen et al. [28] develop new and fast algorithms that involve three steps:
high-pass filtering, gray levels adding, and contrast stretching. Hasan et al. [29]
present an improved thresholding technique for speech enhancement in the DCT
domain. Kim and Han [30] derive DCT properties related to a standard
discontinuity and propose a model-based discontinuity evaluation technique in the
DCT domain. This technique is efficient in detecting discontinuities and robust
against noise.
Since DCT has the advantages of packing the most information into the fewest
coefficients and minimizing reconstruction errors, only a small amount of
information of a detailed image will be lost during the image processing procedures
[23, 24, 31]. Such advantages make DCT suitable and favorable for our study of
tiny defect detection. The main information, or the approximation, of an image can
be represented by a few DCT frequency values of large magnitude. The remaining
frequency values of small magnitude provide detailed information of the image.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
6
Since the SBL surface has random structures, the larger frequency values retain the
information of the statistical patterns and the smaller frequency values retain that of
anomalies in the SBL surface. In the application of SBL defect inspection, we can
set the larger frequency values to zero and reserve the smaller frequency values to
reconstruct the image. Then, the background texture can be removed and
anomalies can be distinctly enhanced in the restored image accordingly.
Requiring neither textural features for detection of local anomalies nor a reference
image for comparison, the proposed method avoids the limitations of feature
extraction and template matching methods. By regarding an input image as a
matrix, we can perform the DCT transformation to transform a spatial domain
image into the frequency domain. From the energy concentration analysis of the
DCT domain, we can decompose the frequency matrix into four decomposed
matrices. We select the proper number of larger frequency values to represent the
random structure features of the SBL surface. Then, we set the selected frequency
values to zero and reconstruct the image. For a faultless SBL chip, the
reconstruction process will result in a uniform image. For a defective SBL chip, the
anomalies will be reserved and the random patterns will be eliminated in the
restored image. Finally, the entropy method is applied to set the threshold for
distinguishing between defective regions and uniform regions in the restored image.
The rest of this paper is organized as follows. Section 2 introduces the properties of
DCT and presents the image reconstruction scheme for inspecting tiny surface
defects of SBL chips. Also discussed in this section are the selection of the proper
sector radius of the DCT frequency components by the cumulative sum algorithm
and the defect separation of the restored image by the entropy method. Section 3
presents the results of the experiments on real SBL chips and evaluates the
sensitivity of the selected radius and the changes of illumination intensity in
lighting conditions. Last but not least, Section 4 concludes with our contributions
and suggestions.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
7
2 Proposed defect detection scheme
2.1 The DCT energy analysis of passive component chips
Eq.(1) gives the expression for computing the DCT of a digital image dx,y of size M
x N. This expression must be computed for all values of u=0, 1, 2, …, M-1, and
also for all values of v=0, 1, 2, …, N-1. Similarly, given Du,v, we can obtain dx,y via
the inverse DCT transform, as shown in Eq.(2) for x=0, 1, 2, …, M-1 and y=0, 1,
2, …, N-1. Eqs.(1) and (2) comprise the two-dimensional DCT pair. While u and v
are frequency variables, x and y are spatial variables.
1 1
, ,0 0
(2 1) (2 1)( ) ( ) cos cos2 2
M N
u v x yx y
x u y vD u v dM N
π πρ ρ− −
= =
+ + = ∑∑ (1)
1 1
, ,0 0
(2 1) (2 1)( ) ( ) cos cos2 2
M N
x y u vu v
x u y vd u v DM N
π πρ ρ− −
= =
+ + = ∑∑ (2)
where
−=−=−=−=
−=
==
−=
==
1...,,2,1,01...,,2,1,0
1...,,2,1,01...,,2,1,0
,1...,,3,2,1,2
0,1
)(,1...,,3,2,1,2
0,1
)(
NyMxNvMu
NvN
vNv
MuM
uMu ρρ
.
The power spectrum P(u, v) of image dx,y is defined as:
2
,( , ) u vP u v D= (3)
That is, the energy of the image can be obtained by adding up the squares of the
DCT coefficients [23].
As the origin of the DCT coefficients has a very huge frequency value, it is
sometimes called the Direct Current (DC) component of the spectrum, while other
coefficients are called the Alternating Current (AC) components. The DC
coefficients in the upper left corner reflect information of lower frequencies,
whereas the AC coefficients in the lower right corner reflect that of higher
frequencies. DCT has the property of concentrating the dominant energy of a
typical image in the low-frequency components. This means that the coefficients of
the high-frequency components are close to zero, and therefore negligible in most
cases [31].
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
8
Figure 2(a) displays an SBL chip with a faultless surface, and Figure 2(b) one with
a defective surface. Figure 2(c) and (d) are the DCT domain images of (a) and (b),
respectively, while Figure 2(e) and (f) present their energy plots in 3D perspective.
We can see clearly in Figure 2(e) and (f) that the high-energy frequency
components concentrate around the top left origin in the DCT spectrum. Owing to
the small size of the tiny defect, the spread sector of the defective surface is not
visualy different from that of the faultless surface. This indicates that the regular
texture pattern is blurred while the anomaly is reserved by restoring only the
frequency components that are outside the sector of a properly selected radius.
Let r* denote the selected radius of the sector in the DCT domain. Let r*=11 for
the faultless surface (Figure 2(a) and (c)), and r*=15 for the defective surface
(Figure 2(b) and (d)), which are selected to reconstruct the textured images. Figure
3(c) and (d) present the reconstruction results after all frequency components
outside the selected sector are set to zero. The restored images in Figure 3 show
that the blurred approximated images of the SBL surfaces, and the random patterns
and edges are nearly eliminated in the faultless and the defective surfaces.
Figure 4(c) and (d) present the reconstruction results after all frequency
components inside the selected sector are set to zero. The restored images in Figure
4 show that the textures are blurred and the edges of the SBL chips are apparently
enhanced. The faultless surface results in an approximately uniform image, and the
anomaly in the defective surface is notably enhanced in the restored image.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
9
(a) (b)
(c) (d)
(e) (f)
Figure 2. (a) An SBL chip with a faultless surface; (b) an SBL chip with a
defective surface; (c) and (d) the respective DCT domain images; (e) and
(f) the respective power spectra in 3D perspective.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
10
(a) (b)
(c) (d)
Figure 3. (a) and (b) The respective DCT domain images of Figure 2(a) and (b)
with Du,v=0 (shown in blank) for all frequency components outside the
sector of radius r*; (c) and (d) the respective restored images.
(a) (b)
(c) (d)
Figure 4. (a) and (b) The respective DCT domain images of Figure 2(a) and (b)
with Du,v=0 for all frequency components inside the sector of radius r*;
(c) and (d) the respective restored images.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
11
Figure 5(a1)-(a4) are the restored images of the faultless surface in Figure 2(a)
when the sector radius is set at 5, 15, 35 or 65 pixels, and Figure 5(b1)-(b4) are
those of the defective surface in Figure 2(b). For faultless surfaces, the larger the
radius is used, the better the gray level uniformity is obtained in the restored images
because more high-energy frequency components are removed. However, for
defective surfaces, an overly large radius may severely blur the anomaly in the
restored images, as seen in Figure 5 (b4). Therefore, the success of the global
image reconstruction scheme for textured surface inspection lies in the selection of
an appropriate radius that sufficiently removes random patterns and yet adequately
reserves the anomalies in the restored images.
(a1) r = 5 (a2) r = 15 (a3) r = 35 (a4) r = 65
(b1) r = 5 (b2) r = 15 (b3) r = 35 (b4) r = 65
Figure 5. (a1)-(a4) The restored images of the faultless SBL surface in Figure 2(a);
(b1)-(b4) the restored images of the defective SBL surface in Figure 2(b)
when the sector radius is set at 5, 15, 35 or 65 pixels.
Let E(r) denote the average energy of the sector with radius r,
2 2 2
1,2,3,..., 21( ) ( , ),
u v rr
r NE r P u vn + =
== ∑
where rn is the total number of points on the arc of radius r sector, and N is the
image width. While the sector grows outwards from the top-left origin of the DCT
spectrum, the average energy generally decreases rapidly. Figure 6 plots the
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
12
avergae energy E(r) with respective to varying values of the radius r for Figure 2(a)
and (b). Note that the E(r)-r curves become flat soon after the radius r reaches the
selected value in both the faultless and the defective images.
0
5000000
10000000
15000000
20000000
25000000
30000000
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
r
E(r)
0
5000000
10000000
15000000
20000000
25000000
30000000
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
r
E(r)
(a) (b)
Figure 6. (a) and (b) The plots of the average energy E(r) as a function of radius r
for the frequency matrices of SBL images in Figure 2(a) and (b).
For exploring more regularities of the average energy plots, we decompose a
frequency image array into four parts: odd-odd (o-o), odd-even(o-e), even-odd (e-
o), and even-even (e-e) matrices based on the alternating decomposition of odd and
even parts of the DCT frequency domain. Figure 7 shows the average energy plots
E(r)-r of the four decomposed matrices in images with faultless and defective ones.
It can be observed from Figure 7(a1) and (b1) that high-energy frequency
components are concentrated and distributed in the decomposed odd-odd matrix.
We further plot the energy percentage diagrams of the decomposed matrices in the
images with faultless and defective image shown in Figure 8. They show that both
of the decomposed odd-odd matrices (Figure 7(a1) and (b1)), respectively, gather
most (up to 90%) of the DCT energy in the two images. Thus, the odd-odd matrix
can represent most information of the repetitive random texture of the SBL surface.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
13
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
(a1) odd-odd matrix (b1) odd-odd matrix
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
05000000
100000001500000020000000
2500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
(a2) odd-even matrix (b2) odd-even matrix
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
0
500000010000000
15000000
2000000025000000
30000000
0 2 4 6 8 101214161820222426283032
r
E(r)
(a3) even-odd matrix (b3) even-odd matrix
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
05000000
1000000015000000200000002500000030000000
0 2 4 6 8 101214161820222426283032
r
E(r)
(a4) even-even matrix (b4) even-even matrix
Figure 7. (a1)-(a4) and (b1)-(b4) Average energy plots of the decomposed matrices
for the SBL images in Figure 2(a) and (b).
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
14
o-o, 87761238, 96%
o-e, 1985646, 2%e-o, 1661487, 2%
e-e, 422396, 0%
o-o
e-e
e-o
o-e
(a)
o-o, 81544875, 96%
o-e, 655016, 1%
e-o, 2365324, 3%
e-e, 282379, 0%
o-o
e-e
e-o
o-e
(b)
Figure 8. (a) and (b) The energy percentage diagrams of the decomposed matrices for
the SBL images in Figure 2(a) and (b).
In most cases, the low frequency components (with larger magnitude) represent the
global approximation of the original image. All other frequency components
(middle and high frequency components) provide the local, detailed information of
the image. Therefore, we can select a proper radius for the frequency components
of larger magnitude to represent the global, repetitive textural feature of the image
and remove such background texture by reconstructing the image without the use of
larger-magnitude frequency components. In the following subsections, we propose
a DCT decomposition and cumulative sum analysis procedure to automatically
determine an appropriate sector radius for image reconstruction.
2.2 DCT decomposition
As given in Eq.(1), D#1u,v is the discrete cosine transform of a testing image d#1
x,y of
size M x N. Let G* be a waveform sign of 1-D frequency arrays and axis be a 1-D
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
15
coordinate vector. Then, G*(axis) is a waveform plot along the axis coordinate and
can be denoted as:
( )* #1,u vG axis D= (4)
where #1 #1,0~( 1) 0~( 1),u N M vaxis D or D− −= ; u=0, 1, 2, …, M-1; v=0, 1, 2, …, N-1. Figure 9
shows the plots of 1-D waveform arrays G*(D#10~(M-1),v) of a DCT frequency matrix
along the u axis and demonstrates that the waveforms of the G*(D#10~(M-1),v) remain
stable until they approach the DC location. And, the closer the waveforms are to
the DC location, the more widely they fluctuate. The magnitude of frequency
components that are far away from the top left origin in the DCT domain falls
rapidly and approximates zero.
( )1#),1(~0
*vMDG −
Figure 9. 1-D waveform plots G*(D#10~(M-1),v).
We decompose the 2-D DCT domain D#1u,v into two matrices: the odd matrix
DOu,v' and the even matrix DEu,v', whose definitions are given below:
#1, ,2u v u vDO D′ ′= (5)
#1, ,2 1u v u vDE D′ ′+= (6)
where u=0, 1, 2, …, M-1; v' =0, 1, 2, …, (N/2)-1.
The odd-number frequencies of the v axis form the odd frequency matrix DOu,v',
while the even-number frequencies make up the even frequency matrix DEu,v' . The
v' axis in either the odd or the even frequency matrix is half as long as the v axis in
the original frequency matrix D#1u,v.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
16
Therefore, the original frequency matrix D#1u,v can be further decomposed into odd-
odd DOOu',v', odd-even DOEu',v', even-odd DEOu',v', and even-even DEEu',v' matrices
under the alternating decomposition of odd and even parts. These four matrices can
be denoted as follows, respectively:
#1', ' 2 ',2 'u v u vDOO D= (7)
#1', ' 2 ',2 ' 1u v u vDOE D += (8)
#1', ' 2 ' 1,2 'u v u vDEO D += (9)
#1', ' 2 ' 1,2 ' 1u v u vDEE D + += (10)
where u' =0, 1, 2, …, (M/2)-1; v' =0, 1, 2, …, (N/2)-1.
After the original frequency matrix D#1u,v is decomposed into four matrices (Eqs.
(7) to (10)), we can use the G* waveform plot to examine the frequency fluctuations
of the waveform matrices. Since each frequency matrix has two dimensions, u' and
v' axes, the four decomposed frequency matrices will have eight sets of 1-D
waveform arrays. Figure 10 demonstrates the decomposition process of the u and v
axes of a 2-D frequency matrix D#1u,v.
To obtain the right sector radii that help improve the analysis results, the negative
45 degree diagonal coefficients of the frequency matrix can be decomposed into a
1-D array DAt when M and N are equal and the top left DC location is set as the
origin. This diagonal matrix can be denoted as follows:
#1,t t tDA D= (11)
where t=0, 1, 2, …, M-1.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
17
1,1 −− NM
1#,vuD
u
v
1)2(,1 −− NM
vuDO ′,u
v'
1)2(,1 −− NM
vuDE ′,u
v'
1)2(,1)2( −− NM
vuDOO ′′,
u'
v'
1)2(,1)2( −− NM
vuDEO ′′,
u'
v'
1)2(,1)2( −− NM
vuDOE ′′,u'
v'
1)2(,1)2( −− NM
vuDEE ′′,
u'
v'
Eq.(5)Eq.(6)
Eq.(7)
Eq.(8)
Eq.(9)
Eq.(10)
Figure 10. The decomposition process diagram of a 2-D DCT spectrum D#1u,v
along the u and v axes.
The frequencies of the waveform plot G*(DAt) of the matrix DAt fluctuate in the
same way as those of the waveform matrices G*(DO0~(M-1),v') and G*(DE0~(M-1),v').
Thus, we can further decompose the matrix DAt into DAOt' and DAEt' matrices.
These two matrices can be denoted as follows:
' 2 't tDAO DA= (12)
' 2 ' 1t tDAE DA += (13)
where t' =0, 1, 2, …, (M/2)-1. Figure 11 shows the diagonal axis decomposition
process of a 2-D DCT frequency matrix D#1u,v.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
18
1,1 −− NM
1#,vuD
u
v
tDAO ′
1)2( −M
t'
tDAE ′
1)2( −M
t'
1−M
t
tDA
Eq.(12)
Eq.(13)
Eq.(11)
Figure 11. The decomposition process diagram of a 2-D DCT spectrum D#1u,v
along the diagonal axis.
After completing the decomposition procedure, we examine the frequency
fluctuation trends of the waveform plots G* of the four decomposed frequency
matrices (Eqs. (7) ~ (10)). We find that the waveform plots of the 1-D odd arrays
show significant and systematic frequency changes. The frequency fluctuation
trends of the odd-odd matrix are smoother and more predictable when compared
with those of the other decomposed matrices. The DCT frequency matrix D#1u,v
needs to be decomposed because most of the regular waveform plots come from the
decomposed odd-odd matrix DOOu',v'. Therefore, the odd-odd frequency matrix
DOOu',v' (including the diagonal matrix DAOt') will be the main target of the later
analyses.
2.3 Cumulative sum algorithm for selecting the proper radius r*
To select the proper sector radius, we propose a transition point detection method
that use the DC location as the origin of the sector area to locate the low-frequency
regions. The proper radius r* of the sector area is determined at the point where the
frequency fluctuation trend gets stable. A stable frequency trend signifies that the
frequencies are no longer in the low-frequency regions. Similar to the waveform
plots G*(axis) of the decomposed frequency matrices, the waveform plots of the
decomposed odd-odd matrix DOOu',v' perform better in detecting the transition
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
19
point when the frequency trend turns stable. In addition, as the value increases
along the axis, the amplitude of the waveform plots G*(axis) decreases and so does
their influence on the determination of the proper sector radius r*. Therefore, this
research analyzes three sets of frequency arrays DOOu',0, DOO0,v', and DAOt' to
determine the proper radius r* of the sector.
The first value in each of the three frequency arrays DOOu',0, DOO0,v' and DAOt' is
the DC value in the DCT domain. This value represents the average gray level of
an image in the spatial domain and is usually an extreme value (the largest or the
smallest) in the frequency matrix. Hence, we set the DC value at zero when
detecting the transition points in the gentle curves to determine the proper radius of
the sector. Setting the DC value at zero can avoid significant variations among the
frequency components. After the individual deviations of the three frequency
arrays are calculated, three transition points can be identified in the gentle curves by
the detection method. The proper radius of the sector r* can be denoted as:
* 2 ( , , ) 1r AVG P P Pα β ω= + (14)
where ( , , ) ( , , 2 )AVG P P P average P P Pα β ω α β ω= . The proper radius r* is selected based
on the average radius of the transition points of the gentle curves along the three
principal frequency arrays in the decomposed odd-odd matrix. This proposed
heuristic method determines the proper radius of the sector in the DCT domain for
image reconstruction.
Figure 12 shows that points αP , βP , and wP are the three transition points of the
gentle curves along u', v', and diagonal axes, respectively. The point wP must be
multiplied by a weight of 2 because this point comes from the diagonal matrix
DAOt' and must be adjusted to be in the same scale as αP and βP . Figure 13
depicts the three frequency arrays DOOu',0, DOO0,v' and DAOt' of the odd-odd
matrix DOOu',v' and their corresponding positions in the original DCT frequency
matrix D#1u,v. Since the u', v' and w' axes all come from the odd arrays of the
original frequency matrix, the point obtained from the function AVG should be
multiplied by 2 and then added by 1 to identify its original location in the DCT
matrix.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
20
Pα
Pβ
PωPω√2
45°
u'
v'
w'
( 0, 0 )
Figure 12. A sector radius diagram obtained from transformation of the three
principal frequency arrays.
1)2(,1)2( −− NM
vuDOO ′′,u'
v'
1,1 −− NM
1#,vuD
u
v
Figure 13. The main elements of the odd-odd matrix DOOu',v' and their
corresponding positions in the original DCT frequency matrix D#1u,v.
2.3.1 Cumulative sum control scheme (CUSUM)
In Statistical Process Control (SPC), the cumulative sum (or cusum) control scheme
performs well in detecting small process shifts. Cusum can effectively detect shifts
of less than 1.5σ as well as indicate when the shifts probably occurred [32, 33, 34].
Hence, this research applies cusum to detect the transition points of gentle curves in
frequency plots for computing the proper sector radius in the DCT domain. The
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
21
cusum chart directly incorporates all the information in the sequence of sample
values by plotting the cumulative sums of the deviations of the sample values from
a target value. Suppose that Z sample sets with sample size n = 1 are collected, and
Xz is the observation of the z-th sample, where z = 1, 2, 3, …, s, …, Z. Then, if µ0 is
the target value, the cumulative sum control chart is formed by plotting the
quantity:
( )01
s
s zz
C X µ=
= −∑ (15)
where s = 1, 2, 3, …, Z, and Eq. (15) can be further expressed as:
( )0 1s s sC X Cµ −= − + . (16)
The cusum scheme works by accumulating deviations from µ0, which are above the
target with one statistic Cs+ and deviations from µ0, which are below the target with
one statistic Cs-. The statistics Cs
+ and Cs- are called one-sided upper and lower
cusums, respectively. They are computed as follows [33]:
0 1max 0, ( )s s sC X K Cµ+ +− = − + + (17)
0 1max 0, ( )s s sC K X Cµ− −− = − − + (18)
where 0 0 0,2
C C K δ σ+ −= = = .
In Eqs. (17) and (18), K as a reference value is often chosen about halfway between
the target µ0 and the out-of-control value of the mean µ1 that we are interested in
detecting quickly. Thus, if the shift is expressed in standard deviation units as µ1
=µ0 + δσ, then K is half the magnitude of the shift. When either Cs+ or Cs
- exceeds
the decision interval H, the sample set is considered to be out-of-control. The value
for H is five times the standard deviation σ. This scheme distinguishes itself in that
we can find the first period following the shift by counting backward from the out-
of-control signal to the time period when the cusum lifted above zero.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
22
2.3.2 Reverse order difference CUSUM method (RD-CUSUM)
CUSUM methods mainly process data that are smooth in the beginning periods and
that deviate slightly in the later periods [33]. To suit the needs of this research, we
reverse the order of the frequency array data because the curves of DCT frequency
components fluctuate sharply in the beginning periods and then turn smooth in the
other periods. We are able to apply the “reverse order” CUSUM method by
moving backward those unstable data that are originally in the beginning periods
and forward those smooth data that are originally in the later periods.
If the cumulative deviation sums reveal the trend of cyclic accumulations, then it
means the data of the sequence are auto-correlated [33]. Cyclic accumulations may
increase the chances of making wrong detection judgments and picking the wrong
transition points. To overcome the problems of cyclic accumulations, we propose
the reverse order difference CUSUM method (RD-CUSUM), which is also based
on the CUSUM scheme to detect transition points. But, the RD-CUSUM method
reverses the data sequence of the frequency array and calculates the difference
between two successive data before the CUSUM algorithm is implemented. The
difference function L(x*z) calculates the new reverse-order difference sequence
L(x*z-1) by subtracting the reverse-order value x*
z-2 in period (z-2) from the reverse-
order value x*z-1 in period (z-1). Through the calculation of the reverse-order
difference sequence, we can eliminate the cyclic accumulation phenomenon in the
cumulative deviation sums.
We first arrange the data x*z of the frequency array in a reverse order and use I(x*
z)
to denote the data in reverse permutation. Then, we calculate the difference
between two successive data for all data and use L(I(x*z)) to denote the difference
sequence L(x*z) in reverse permutation. Finally, we can extend the CUSUM control
scheme in Eqs. (17) and (18) to the case of the reverse-order difference sequence.
We replace Cs+ with Cz
+ (the one-sided upper cusum of the sequence L(I(x*z)), Cs
-
with Cz- (the one-sided lower cusum of the sequence L(I(x*
z)), and Xs with L(I(x*z)),
where µ0 is the mean and σ the standard deviation of the sequence L(I(x*z)).
The data of the three arrays DOOu',0, DOO0,v', and DAOt' are substituted into the
sequence xz*, respectively. We proceed with the procedure of the RD-CUSUM
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
23
method to identify the transition points P(DOOu',0), P(DOO0,v'), and P(DAOt') of the
three frequency arrays and compute the best sector radius r*. The calculation of the
sector radius r* by the RD-CUSUM method is shown in Figure 14, from which we
can see that the cyclic accumulations of the cumulative deviation sums are removed
through the use of the difference function.
-80
-60
-40
-20
0
20
40
60
80
1 11 21 31 41 51 61 71 81 91 101 111 121
-140
-120
-100
-80
-60
-40
-20
0
20
40
60
1 11 21 31 41 51 61 71 81 91 101 111 121
-40
-30
-20
-10
0
10
20
30
40
1 11 21 31 41 51 61 71 81 91 101 111 121
µ
σ
µ
σ
µ
σ
≈
Figure 14. An example of calculating the proper sector radius r* by the RD-
CUSUM method.
The RD-CUSUM method analyzes the frequency plots of the decomposed
frequency arrays to locate transition points, which are points with abrupt changes of
frequency values. The detection of transition points determines the proper sector
radius for the frequency filtering operation, which greatly influences the later
segmentation accuracy of the tiny defects.
2.4 Entropy method for defect segmentation
After the proper sector radius r* is determined, the frequency filtering operation can
accurately specify the non-defect low frequency regions and set the values of their
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
24
frequencies at zero in the DCT domain. Then, we can transform the filtered
frequency image back to the spatial domain for further defect separation. Thus, the
filtered frequency image D#2u,v can be transformed back to the spatial domain to
produce a corresponding restored image d#2x,y by using Eq. (2). When a restored
image with enhanced tiny defects is generated, an automated thresholding
technique called Kapur’s entropy method is applied to separate the tiny defects
from the normal regions.
The Kapur’s entropy method regards the image foreground and background as two
different signal sources, so when the sum of the entropies of the two signal sources
reaches its maximum, the image is said to be optimally thresholded [35]. Let us
consider an image with gray levels in the range [0, L-1]. We assume lS be the
number of pixels in the image having gray level l and the total number of pixels in
the image should be 1
0
Lll
S−
=∑ . Then, lY is an estimate of the probability of
occurrence of gray level l in the image, denoted as 1
0
Ll l ll
Y S S−
== ∑ . From the
definition of entropy, the entropy of an image is 1
0lnL
l llN Y Y−
== −∑ . We aim to
classify the pixels of the image into two opposite classes, namely defect (black) and
normal region (white) by thresholding. The entropy of the defective portion of an
image is ( )ON T and the entropy of the normal portion of an image is ( )BN T .
Therefore, the Kapur’s entropy method is to determine the value of the threshold T,
such that the total entropy N (Eq. (19))of the partitioned image is maximized.
1
0 1
( ) ( ) ln lnT L
O B r r r rr r T
N N T N T Y Y Y Y−
= = +
= + = − −∑ ∑ (19)
The Kapur’s entropy method outperforms many other methods in detecting small
objects [36]. However, the entropy method can not correctly separate tiny defects
if the SBL image does not go through the defect enhancement step of the proposed
DCT frequency filtering operation. This is because many noises in the image may
incur erroneous judgment in defect separation. If an image is processed by the
proposed DCT-based image reconstruction approach in advance, the tiny surface
defects in the image can be precisely located by the Kapur’s entropy method.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
25
3 Experiments and discussion
In this section, we implement the proposed approach and conduct experiments to
evaluate its performance in detecting tiny surface defects. To strengthen the
visibility of tiny surface defects, we make use of the following equipments: a white
ring LED lighting device, a CCD (model WAT-221S of Watec company), a lens
with changeable focal lengths and amplifications of 1 to 10 times, a frame grabber
(model IMAQ PCI-1411 of National Instruments corporation), and an XYZ
electronic control table with a controller. Each image of the SBL chip has a size of
256 x 256 pixels and a gray level of 8 bits. The detection algorithm is edited in C
language and executed on the sixth version of the C++ Builder complier on a
personal computer (Pentium-4 2.66 GHz and 512 MB D-RAM).
3.1 Experimental results
3.1.1 Experiments with a small sample size
To verify the feasibility of the proposed approach, 20 SBL chip images are first
tested. Two (No. 12 and No. 20) of the 20 SBL chips contain no tiny defects, while
each of the remaining 18 chips has at least one tiny defect. Table 1 summarizes the
detection results of the 20 images. The features of detected defects include the
number of detected defects, the number of their pixels, and the positions of their
mass centers (X and Y coordinates). We evaluate the performances of the
transition point detection method with respect to the following indices: statistics of
detected results, numbers of erroneous judgments of defect detections, and area
deviation rates of detected defects. The detection results in Table 1 indicates that
the RD-CUSUM method identifies the transition points with great accuracy. The
smaller the sector radius, the fewer frequency components are inside the selected
sector and have their values set at zero in the DCT domain. A smaller than proper
radius may result in identifying the normal regions as tiny defects or making the
areas of the detected defects larger than those of the real defects. On the other
hand, a larger than proper radius may result in identifying the tiny defects as normal
regions or making the areas of the detected defects smaller than those of the real
defects.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
26
Table 1. Results of small-sample experiments
Samples Image NO. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Real defect NO. 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 0Pixels of a real defect 5 2 4 6 4 2 5 1 4 7 5 9 4 12 10 14 10 1
RD-CUSUM The selected radius r* 11 15 13 15 13 9 17 15 15 9 15 9 11 11 13 13 9 11 13 9Detected defect NO. 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 0Pixels of a detected defect 5 1 1 4 6 4 2 5 4 4 7 5 1 9 4 12 10 14 1 10 1Mass center of a detected defect X 169.2 119 68 162.25 145 89.5 145 156.2 119.5 93.5 97 173.8 147 174.11 90.5 140.83 92.8 207 90 105.5 180
Y 145.4 199 153 96.25 97.5 145.5 62.5 155.6 206.5 164.5 145 101.4 55 95.33 72.5 124 78.5 112.6 195 74.6 185
As Table 1 indicates, the RD-CUSUM method can accurately identify the locations
and the areas (pixels) of tiny defects. The mass centers of the defects detected by
the RD-CUSUM method are all very close to the real ones. Besides, the areas of
most defects in the 20 images are accurately measured; only in images No. 1, 12,
and 17 are the normal regions mistakenly identified as 1-pixel defects. Since tiny
defects are extremely small, how accurately a method measures the area of a
detected defect becomes extraordinarily important in comparing the detection
performance of various methods. Moreover, the RD-CUSUM method is competent
in detecting tiny defects of varying sizes and computing their areas. For instance,
the actual defect area in image No. 17 is as large as 14 pixels and the actual defect
areas of images No. 8 and No. 19 are as small as 1 pixel each. A simplified
version of Table 1 is given in Table 2 to facilitate further discussion of the detection
results.
Table 2. Simplified results of small-sample experiments
Number of Number of correctly Number of erroneously Number of Type I error Type II error Area deviation rate real defects detected defects detected defects missing defects (α) (β) of detected defects
RD-CUSUM 18 18 3 0 0.15 0.00% 3.80%
Method
In Table 2, the RD-CUSUM method is evaluated by the three indices: Type I error
α (false alarms, which regard normal regions as defects.), Type II error β (missing
alarms, which fail to alarm real defects), and area deviation rates of detected
defects. We divide the number of erroneously detected defects by the number of
total testing images to obtain Type I error α, and the number of missing defects by
the number of total defects to obtain Type II error β. To compute the area deviation
rate of detected defects, we first add up the differences between the correctly
detected defect areas and their corresponding actual defect areas; and then divide
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
27
the summation by the total actual defect areas. For all of the three indices, the
smaller the index value, the better the detection result.
As Table 2 indicates, the RD-CUSUM method achieves great performance in
detecting the number of correctly detected defects. Type I error of the RD-CUSUM
method is 0.15; and its Type II errors is 0%. When the major concern of detection
is on the accurate areas of the detected defects, the RD-CUSUM method should be
applied because of its accuracy in detecting defect areas. The area deviation rate of
a detected defect signifies how inaccurately the method detects the area of a defect.
The area deviation rate of the method is 3.8%. The results of our small-sample
experiments show that the proposed method has great feasibility and good
performances. To verify the results, experiments with large testing samples are
conducted in the following subsection.
3.1.2 Experiments with a large sample size
To conduct a large-sample experiment, another 120 SBL chip images are tested.
Each of the first 100 of the 120 SBL chips contains at least one tiny defect, while
the remaining 20 are regular chips without any tiny defect. The proposed approach
takes an average of 4 seconds to process a forward and an inverse DCT
transformations. The processing time can be significantly shortened if the DCT
functionality is incorporated into a single integrated circuit. Actually, if we do not
count the time of the forward and inverse DCT steps, it takes less than 10 ms for the
RD-CUSUM method to process an SBL chip. This means that the methods used in
this research are very time-saving and efficient.
Tables 3 presents the results of large-sample experiments in which we demonstrate
the performance of the proposed method based on the number of total real defects.
Table 3 shows the experimental results of detecting the 120 testing images which
contain a total of 114 real tiny defects. The statistics are computed based on the
number of total real defects, and the results are consistent with those of the previous
preliminary experiment. The RD-CUSUM method has low Type I error value (0.1)
and Type II error value (5.26%). In addition, this method also has a small area
deviation rate (4.81%), which implies that the areas of the detected defects are the
closest to those of real defects.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
28
Table 3. Results of large-sample experiments.
Number of total Number of correctly Number of erroneously Number of Type I error Type II error Area deviation rate real defects detected defects detected defects missing defects (α) (β) of detected defects
RD-CUSUM 114 108 18 6 0.15 5.26% 4.81%
Method
3.2 Experiments on DCT reconstruction effects
To evaluate the performance of the proposed DCT-based image reconstruction
approach, experiments with and without the defect enhancement process are both
conducted. The testing images with and without the DCT defect enhancement
operation are all segmented by the Kapur’s entropy method to examine how they
differ in terms of tiny defect detection. The testing samples as well as the
experimental configurations and procedures are the same as those of the large-
sample experiments. Based on the number of total real defects, Table 4 presents the
performance evaluation indices of tiny defect detection with and without the DCT
defect enhancement process.
Table 4. The performance evaluation table of defect detection with and without
DCT defect enhancement process.
Number of total Number of correctly Number of erroneously Number of Type I error Type II error Area deviation rate real defects detected defects detected defects missing defects (α) (β) of detected defects
DCT defect enhancement + Entropy
Entropy 114 103 64 11 0.561 9.65% 212.50%
6 0.158 5.26% 4.81%
Methods
114 108 18
Table 4 shows that the method with DCT defect enhancement has significantly
lower Type I error and area deviation rate than the method without DCT defect
enhancement. The proposed method outperforms the method without DCT defect
enhancement in reducing the Type I error by 72% and the area deviation rate by
98%. The comparison results of the experiments clearly and objectively
demonstrate the great performance of the proposed DCT-based image
reconstruction approach in detecting tiny defects.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
29
3.3 Sensitivity analysis of the proposed method
3.3.1 Sensitivity of the selected radius r*
In this study, the proper radius r* used for image reconstruction is selected based
on the abrupt changes of the frequency coefficients in the decomposed odd-odd
matrix. A faultless SBL surface and a defective one, as shown in Figure 2(a) and
2(b), are used to evaluate the sensitivity of minor changes in r* value. Figure
15(a1)-(a5) present the final results from sector radii of 7, 11, 15, 30 and 45 pixels
for the faultless texture in Figure 2(a), in which r* = 11 given significantly abrupt
change of frequency values. Figure 15(b1)-(b5) present the final results from radii
of 10, 13, 15, 17 and 30 pixels for the defective texture in Figure 2(b), in which r*
= 15. From Figure 15, we find that the radii in the neighborhood of the selected r*
generates similar final results. It is not crucial for the selection of a precise r*
value. The image reconstruction process can tolerate minor changes of the selected
radius r*. Experiments results show that a minor change of r* values between 1 to
4 pixels will not affect the reconstruction result.
Figure 15. (a1)-a(5) The resulting binary images from the sectors of radii 7, 11,
15, 30 and 45 pixels for the faultless SBL surface in Figure 2(a); (b1)-
b(5) the resulting binary images from the sectors of radii 10, 13, 15, 17,
and 30 pixels for the defective SBL surface in Figure 2(b).
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
30
3.3.2 Changes of illumination intensity
The proposed method in this research is based on a global image reconstruction
scheme for defect detection. It is insensitive to changes in the illumination
intensity. Figure 16 (b1) shows a defective SBL surface under regular lighting
condition. Figure 16 (a1) and (c1) are underexposed and overexposed versions of
the SBL image in Figure 16 (a1). Figure 17 depicts the E(r)-r curves of the odd-
odd matrices for the SBL images in three different illumination intensities. The
selected radii r* for Figure 16(a1)-(a3) are the same as 15 pixels. Figure 16(a2)-
(c2) and (a3)-(c3), respectively, presents the DCT domain images and the restored
images of the three SBL images. Figure 16(a4)-(c4) shows the defect detection
results of Figure 16(a3)-(c3) as resulting binary images. The results reveal that the
defects in all three SBL images are correctly separated in the binary images,
regardless of lighting changes.
3.3.3 Various sizes and numbers of defects in an image
To explore how the proposed method detects defects of different sizes, Figure
18(a1)-(c1) presents defects in increasing sizes (2, 8 and 14 pixels) on SBL
surfaces. The detection results in Figure 18 (a3)-(c3) indicate that the defects are
well enhanced in the restored images, even though the actual defect area is as small
as 2 pixels or as large as 14 pixels. Nevertheless, defects of large sizes cannot be
completely detected because the central parts of the defects may be removed after
the high-pass filtering procedure is conducted.
To examine how the proposed method detects various numbers of defects in an
image, Figure 19(a1)-(c3) presents increasing numbers (4, 5 and 7) of defects on
SBL surfaces. The resulting binary images in Figure 19(a4)-(c4) demonstrate that
tiny defects are well detected, even though a large number of tiny defects exist in
the same image. Experiments results show that the performance of the proposed
method is not restricted by the number of defects in an image.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
31
Figure 16. The effect of changes in illumination intensity: (a1)-(c1) a defective
SBL surface under regular, underexposed and overexposed lighting
conditions; (a2)-(c2) the respective DCT domain images; (a3)-(c3) the
respective restored images; (a4)-(c4) the resulting binary images that
show the defects in black.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
32
0
2000
4000
6000
8000
10000
12000
14000
0 2 4 6 8 10 12 14 16 18
r
E(r) RegularOverexposedUnderexposed
Figure 17. The plots of E(r)-r curves of the odd-odd matrices for the SBL images
in Figures 16(a1)-(c1).
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
33
Figure 18. The effect of changes in size for defects in SBL: (a1)-(c1) SBL images
containing increasing sizes (2, 8, 14 pixels) of defects; (a2)-(c2) the respective DCT
domain images; (a3)-(c3) the respective restored images; (a4)-(c4) the resulting
binary images that show the defects in black.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
34
(b2) (c2)(a2)
(a4) (c4)
(a1) (b1) (c1)
(a3) r* = 9 (b3) r* = 15 (c3) r* = 9
(b4)
Figure 19. The effect of changes in defect number in an image: (a1)-(c1) SBL
images containing increasing numbers (4, 5, 7) of defects; (a2)-(c2) the respective
DCT domain images; (a3)-(c3) the respective restored images; (a4)-(c4) the
resulting binary images that show the defects in black.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
35
4 Conclusion
This paper presents a global approach for the automated visual inspection of tiny
defects in the surfaces of SBL chips. Unlike conventional feature extraction
methods, the proposed method relies on no textural features. Based on an image
reconstruction scheme, it applies DCT decomposition and cumulative sum
techniques to effectively detect tiny defects. DCT has the advantages of packing
the most information into the fewest coefficients and minimizing reconstruction
errors. By analyzing the energy concentration condition of a SBL frequency image,
we can apply the DCT based approach to decompose the frequency domain image
into four decomposed matrices. From the decomposed odd-odd matrix, the
cumulative sum algorithm can be used to select proper frequency values of large
magnitude that represent the normal background texture of the SBL surface. Then,
we take out the selected frequency values of large magnitude and reconstruct the
image from the DCT frequency matrix to remove the global random patterns of the
statistically textured image and reserve local anomalies in the reconstructed image.
The proposed method is based on a global image reconstruction scheme for defect
detection and is insensitive to changes in illumination intensity. If lighting changes
lead to underexposed or overexposed lighting conditions, only the values of the DC
components will be changed after the DCT transformation. Changes of the DC
values do not affect the detection results of the proposed method. Performance of
the proposed method can be limited by defect size, background property, and
processing time. Defects of large sizes cannot be completely detected because the
central parts of the defects may be removed after the high-pass filtering procedure
is conducted. Defects embedded in structural textures cannot be detected because
the proposed method is suitably applied to identify defects in random textures. The
proposed method takes 4 seconds to process a forward and an inverse DCT
transformations in our experimental configurations. This processing time can be
significantly decreased to meet the requests of on-line automatic inspection by
implementing the proposed approach in a single integrated circuit.
The proposed method is effective and efficient in detecting tiny defects of different
kinds, which include dust, cavities and pinholes on the surfaces of SBL chips.
Experimental results show that the proposed method outperforms the traditional
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
36
approach in reducing the Type I error by 72% and in decreasing the deviation of the
defect areas by 98%. Future research can make use of the research results and
investigate the following areas: a global DCT based enhancement approach for
detecting tiny surface defects in structural textures, and the proposed DCT
decomposition procedure applied in the image compression and feature extraction
fields.
Acknowledgements
The author thanks the National Science Council of Taiwan (R.O.C.), for the
financial supports through the Grants NSC 92-2212-E-324-001 and NSC 94-2212-
E-324-001.
References 1. D.M. Stubbs, S.H. Pulko, A.J. Wilkinson, B. Wilson, F. Christiaens, K. Allaert, Embedded
passive components and PCB size-thermal effects, Microelectronics International, 17-2 (2000) 7-10.
2. R.R. Tummala, G.E. White, V. Sundaram, S.K. Bhattacharya, SOP: the microelectronics for the 21st century with integral passive integration, Advancing Microelectronics, 27-1 (2000) 13-19.
3. Chi-Hao Yeh, Ta-Cheng Shen, Ful-Chiang Wu, A case study: passive component inspection using a 1D wavelet transform, International Journal of Advanced Manufacturing Technology, 22 (2003) 899-910.
4. D.M. Tsai, C.Y. Hsieh, Automated surface inspection for directional textures, Image and Vision Computing, 18 (1999) 49-62.
5. Du-Ming Tsai, Tse-Yun Huang, Automated surface inspection for statistical textures, Image and Vision Computing, 21 (2003) 307-323.
6. C.J. Lu, D.M. Tsai, Defect inspection of pattern thin film transistor-liquid crystal display panels using a fast sub-image-based singular value decomposition, International Journal of Production Research, 42-20 (2004) 4331-4351.
7. Chi-Jie Lu, Du-Ming Tsai, Automatic defect inspection for LCDs using singular value decomposition, International Journal of Advanced Manufacturing Technology, 25 (2005) 53-61.
8. B.C. Jiang, C.C. Wang, H.C. Liu, Liquid crystal display surface uniformity defect inspection using analysis of variance and exponentially weighted moving average techniques, International Journal of Production Research, 43-1 (2005) 67-80.
9. Wen-Yen Wu, Mao-Jiun J. Wang, Chih-Ming Liu, Automated inspection of printed circuit boards through machine vision, Computers in Industry, 28 (1996) 103-111.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
37
10. C.H. Yeh, D.M. Tsai, A rotation-invariant and non-referential approach for ball grid array (BGA) substrate conducting path inspection, International Journal of Advanced Manufacturing Technology, 17 (2001) 412-424.
11. Du-Ming Tsai, Cheng-Huei Chiang, Automatic band selection for wavelet reconstruction in the application of defect detection, Image and Vision Computing, 21 (2003) 413-431.
12. A. Pikaz, A. Averbuch, An efficient topological characterization of gray-levels textures using a multiresolution respresentation, Graphical Models and Image Processing, 59 (1997) 1–17.
13. L.H. Siew, R.M. Hodgson, L.K. Wee, Texture measures for carpet wear assessment, IEEE Transactions on Pattern Analysis and Machine Intelligence, 10 (1988) 92-150.
14. A.L. Amet, , A. Ertuzun, A. Ercil, An efficient method for texture defect detection: sub-band domain co-occurrence matrices, Computers and Geosciences, 28 (2002) 763-774.
15. M. Tico, P. Kuosmanen, J. Saarinen, Wavelet domain features for fingerprint recognition, Electronics Letters, 37-1 (2001) 21-22.
16. Hamed Sari-Sarraf, J.S. Goddard, Robust defect segmentation in woven fabrics, Proceedings of Computer Vision and Pattern Recognition, (1998) 938-944.
17. H. Rau, C.H. Wu, Automatic optical inspection for detecting defects on printed circuit board inner layers. International Journal of Advanced Manufacturing Technology, 25 (2005) 940-946.
18. Chi-Hao Yeh, Fang-Chih Tien, Ful-Chiang Wu, Boundary-based passive component inspection approach using eigenvalues of covariance matrices, International Journal of Production Research, 41-17 (2003) 4025-4040.
19. N. Ahmed, T. Natarajan, K.R. Rao, Discrete cosine transform, IEEE Transactions on Computer, 23-1 (1974) 90-93.
20. Yui-Lam Chan, Wan-Chi Siu, Variable temporal-length 3-D discrete cosine transform coding, IEEE Transactions on Image Processing, 6-5 (1997) 758-763.
21. Mandyam Giridhar, Nasir Ahmed, Neeraj Magotra, Lossless image compression using the discrete cosine transform, Journal of Visual Communication and Image Representation, 8-1 (1997) 21-26.
22. Zhishun Wan, Zhenya He, Cairong Zou, J.D.Z. Chen, A generalized fast algorithm for n-D discrete cosine transform and its application to motion picture coding, IEEE Transactions on Circuits and Systems-II: Analog and Digital Signal Processing, 46-5 (1999) 617-627.
23. K.R. Rao, J.J. Hwang, Techniques and Standards for Image, Video, and Audio Coding, Prentice-Hall, Inc., 1996.
24. Ming-Ting Sun, I-Ming Pao, Statistical computation of discrete cosine transform in video encoders, Journal of Visual Communication and Image Representation, 9-2 (1998) 163-170.
25. Nam Ik Cho, Sang Uk Lee, Fast algorithm and implementation of 2-D discrete cosine transform, IEEE Transactions on Circuits and Systems, 38-3 (1991) 297-305.
26. C.W. Kok, Fast Algorithm for computing discrete cosine transform, IEEE Transactions on Signal Processing, 45-3 (1997) 757-760.
27. Yuh-Ming Huang, Ja-Ling Wu, A refined fast 2-D discrete cosine transform algorithm,” IEEE Transactions on Signal Processing, 47-3 (1999) 904-907.
ACCEPTED MANUSCRIPT
ACC
EPTE
D M
ANU
SCR
IPT
38
28. Biao Chen, Shahram Latifi, Junichi Kanai, Edge enhancement of remote sensing image data in the DCT domain, Image and Vision Computing, 17-12 (1999) 913-921.
29. Md. Kamrul Hasan, Sayeef Salahuddin, M. Rezwan Khan, Reducing signal-bias from MAD estimated noise level for DCT speech enhancement, Signal Processing, 84 (2004) 151-162.
30. Tae Yong Kim, Joon Hee Han, Model-based discontinuity evaluation in the DCT domain, Signal Processing, 81 (2001) 871-882.
31. Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing, 2nd Ed., Prentice Hall, 2002.
32. F.F. Gan, An optimal design of CUSUM quality control charts, Journal of Quality Technology, 23-4 (1991) 279-286.
33. Douglas C. Montgomery, Introduction to Statistical Quality Control, 5th Ed., John Wiley & Sons, 2005.
34. J.J. Pignatiello Jr., G.C. Runger, Comparison of multivariate CUSUM charts, Journal of Quality Technology, 22-3 (1990) 173-186.
35. J. Kapur, P. Sahoo, A. Wang, A new method for gray-level picture thresholding using the entropy of the histogram, Computer Vision, Graphics, and Image Processing, 29 (1985) 273-285.
36. M. Sezgin, B. Sankur, Survey over image thresholding techniques and quantitative performance evaluation, Journal of Electronic Imaging, 13-1 (2004) 146– 165.