Advanced (Mid-Level) DIP -- Morphological Image...
Transcript of Advanced (Mid-Level) DIP -- Morphological Image...
1
1
Advanced (Mid-Level) DIP-- Morphological Image Processing
Xiaojun Qi
2
Morphology
• Morphology: It is a tool for extracting image components that are useful in the representation and description of region shape, such as boundaries, skeletons, and the convex hull.
• Morphological operations preserve the basic properties of object while removing irrelevant features.
• Applications: segmentation, enhancement, restoration, edge detection, texture analysis, shape analysis, thinning, and curve filling.
3
• The binary images are normally produced by simple segmentation techniques such as thresholding. They may contain numerous imperfections caused by noise, texture or the inaccurate specification of a threshold. Morphological image processing techniques can remove the imperfections and provide us with information on the form and structureof the image.
• Morphological techniques are also applicable to grayscale images, where they can be used for non-linear smoothing and feature enhancement. 4
• The term “morphological image processing” describes a range of non-linear image processing techniques that deal with the shape of features in an image.
• Morphological techniques typically probe an image with a small shape or template known as a structuring element (SE). The SE is positioned at all possible locations in the image and it is compared with the corresponding neighborhood of pixels.
• Morphological operations differ in how they carry out the comparison. Some test whether the structuring element ‘fits’ within the neighborhood; others test whether it ‘hits’ or ‘intersects’ the neighborhood.
5
C
A
B
o
Probe an image with a structuring element at A, B, and C positions.
Fit or Hit?
6
11111
11111
11111
11111
11111
00100
01110
11111
01110
00100
00100
00100
11111
00100
00100
The dimensions of the matrix determine the overall size of the structuring element, and its shape is determined by the pattern of ones and zeros.
1) What are the sizes of the above three SEs?2) What are the shapes of the above three SEs?
2
7
Fit or Hit?• A structuring element (SE) is said to fit an
image if for each of its pixels that is set to 1, the corresponding image pixel is also 1. The SE pixels that are 0 define points where the corresponding image value is irrelevant.
• The SE is said to hit an image if for any of its pixels that is set to 1, the corresponding image pixel is also 1. We also ignore image pixels for which the corresponding SE pixel is 0. 8
000000000000011111100000011111111100000111111100000011111100000011111110000011111110000001111100000000011000000000000000
Suppose s1 is a 3*3 square SE and s2 is a 3*3 cross-shaped SE, determine whether they are hit or fit in three positions above.
9
Basic Idea of Morphology• Probe an image with a SE and determine
the manner in which the SE fits, hits, (or does not fit or hit) within the image.
• The SE will be small compared to the image.
• By marking the locations where the SE fits or hits within the image, one obtains information about the structure of the image which depends upon the size and shape of the SE.
10
Set Theory and Notation• Translation:• Reflection:• Complement:• Intersection:• Union:• Difference:• SubSet:• Disjoint (Mutually Exclusive)
{ }AaaccA ∈−== ,|ˆ{ }AaxaccA x ∈+== ,)(
{ }AccA c ∉={ })()( BcAccBA ∈∧∈=∩
{ })()( BcAccBA ∈∨∈=∪
{ })()( BcAccBA ∉∧∈=−},|{ BcthenAcifcBA ∈∈=⊆
φ=∩ BAThe elements of the sets are the coordinates of pixels (i.e., members of the 2D integer space Z2) representing objects or other features of interest in an image.
11 12
Logical Operations
These three operations are functionally complete in the sense that they can be combined to form any other logic operation.
3
13 14
Binary Dilation• The dilation of A by B is defined as:
• Set B is commonly referred to as the structuring element in dilation, as well as in other morphological operations.
• Dilation expands the object if the origin is in B.
{ }.})ˆ(|{}|}{{
,,})ˆ(|{
φ
φ
≠∩+=∈+∪=
∈∈+==≠∩=⊕
ApBpBbbA
BbAabaccABpBA p
15
Binary Dilation Implementation View 1
• Position a SE s such that its origin is at image pixel coordinates (x, y) and apply the rule:
repeating for all x and y.• Dilation creates a new image that
shows all the locations of a SE’s origin at which it hits the input image.
=otherwise. 0
. hits if 1),(
fsyxg
16We used a 3*3 square SE to do the dilation
Binary Dilation Example 1
17
Binary Dilation Implementation View 2
Think of the SE as a template, and move it over the image. When the origin of the SE aligns with a black pixel in the image, all of the image pixels that correspond to black pixels in the SE are marked, and will later be changed to black. After the entireimage has been swept by the SE, the dilation calculation is complete. 18
Binary Dilation Example 2-- Origin is part of the SE
Final Dilation Result
4
19
Binary Dilation Example 3-- Origin isn’t part of the SE
20
Binary Dilation Example 4-- Origin is part of the SE
(a) (b) (c) dilation by one pixel each time
1) What kind of SE can achieve (a) (b) and (b) (c)?
2) What kind of SE can achieve (a) (c)?
21
Binary Dilation Example 5-- Effects of the SE size
Here we used square SE
What is the effect after applying the dilation operation? -- Inner and outer boundaries, holes, and gaps
22
Binary Dilation Example 6-- Effects of the SE shape
Here we used a square SE of size 6*6 and a circle SE of radius 3
What is the effect after applying the dilation operation? -- Inner and outer boundaries, holes, gaps and corner
23
Binary Dilate Example 7-- Effects of the SE shape
5*5 Square SE Circle SE with radius of 2
What is the effect after applying the dilation operation? -- Boundaries, holes, and corner
24
Binary Dilation Example 8
We used 3*3 square SE to do the dilation
5
25
Assume the maximum length of the breaks is known to be two pixels.
Binary Dilation Example 9
26
Binary Dilation Summary
27
Binary Erosion• The erosion of A by B is defined as:
• If the translated set is contained in set A, the p is in the erosion of A with B.
• Erosion results in a shrink of the image.• Dilation and erosion are duals of each other with
respect to set complementation and reflection.
( ){ } { }}|)({
|
BbbA
ABppABpBA p
∈−∩=
⊆+=⊆=Θ
BABA cc ˆ)( ⊕=Θ28
Binary Erosion Implementation View 1
• Position a SE s such that its origin is at image pixel coordinates (x, y) and apply the rule:
repeating for all x and y.• Erosion creates a new image that marks all
the locations of a SE’s origin at which it fits the input image.
=otherwise. 0
. fits if 1),(
fsyxg
29
Binary Erosion Example 1
We used 3*3 square SE to do the erosion
30
Binary Erosion Implementation View 2
6
31
Binary Erosion Example 2
(c) (b) (a) erosion by one pixel each time
1) What kind of SE can achieve (c) (b) and (b) (a)?
2) What kind of SE can achieve (c) (a)?32
Binary Erosion Example 3-- Effects of the SE size
Here we used square SE
What is the effect after applying the erosion operation? -- Inner and outer boundaries, holes, and gaps
33
Binary Erosion Example 4-- Effects of the structuring element size
5*5 Square SE Circle SE with radius of 2
What is the effect after applying the erosion operation? -- Boundaries, holes, and corner
34
Binary Erosion Example 5
We used 3*3 square SE to do the Erosion
35
Finding a Region Boundary
Source Code to Generate the BoundaryA = imread('char_a.jpg') ;se1 = strel('square', 3) ; SquareErodeA1 = imerode(A, se1) ;Boundary1 = imsubtract(A, SquareErodeA1) ;SquareDilateA1 = imdilate(A, se1) ;Boundary2 = imsubtract(SquareDilateA1, A) ;
What is the difference between these two
boundaries?
36
Binary Erosion Summary
7
37
• The result of dilation/erosion depends on SE size and shape, with larger SEs having a more pronounced effect.
• The result of dilation/erosion with a large SE is similar to the result obtained by iterated dilation/erosion using a smaller SE of the same shape.
• Dilation by a disc enlarges the object and smoothes its convex corner; Erosion by a disc shrinks the object and smoothes its concave corner.
Summary-- Binary Dilation and Binary Erosion
38
1. What is the structuring element?2. Does erosion followed by dilation fully restore the eroded
objects in this case?3. Can we expand the conclusion from 2 to a general case?
The “active” elements of structuring elements assume the same binary values as the objects of interest.
39
Binary Opening• The opening of A by B is defined as:
• The opening of A by B is the erosion of A by B, followed by a dilation of the result by B.
• Opening generally smoothes the contour of an object, breaks narrow isthmuses, and eliminates thin protrusions. “open” small gaps or spaces between touching objects in an image.
• Remove much of the black pixel noise in the background of the image. (Pepper Noise)
BBABA ⊕Θ= )(o
40
BAoThe boundary of the is established by the points in B that reach the farthest into the boundary of A as B is rolled around the inside of this boundary
})(|){( ABBBA zz ⊆∪=o
41
Binary Opening Example 1-- Eliminate Thin Protrusions
Original Image A
BSE
Opening A by B
42
Binary Opening Example 2-- Break Narrow Connections
Original Image A
BSE
Opening A by B
8
43
Binary Opening Example 3-- Remove Pepper Noise
• Pepper noise consists of collections of small dark objects.
• Open operator can remove pepper noise.• The trick is to find a structuring element which
passes the image but eliminates the pepper noise.• One often uses a symmetric structuring element
so that results do not depend on the orientation of the picture.
• One often starts with small structuring elements B and increases the size systematically.
timesnBBBnBnBnBSnBS
⊕⊕⊕=⊕Θ=
......)(o
44
Binary Opening Example 3-- Remove Pepper Noise (Cont.)
A
B Opening A by B
Can we use a square SE to remove pepper noise by using the opening operation?
45
Binary Opening Example 4-- Remove Noise
(a) A binary image (b) Opening of (a) with a disc SE46
Binary Closing• The closing of A by B is defined as:
• The closing of A by B is simply the dilation of A by B, followed by the erosion of the result by B.
• Closing also tends to smooth sections of contours, but it generally fuses narrow breaks and long thin gulfs, eliminates small holes, and fills gaps in the contour.
• Remove much of the white pixel noise in the foreground (objects) of the image, giving a fairly clean image. (Salt Noise)
BBABA Θ⊕=• )(
47
Roll B on the outside of the boundary
})(|){( φ≠∩∪=• ABBBA zz
48
Binary Closing Example 1-- Eliminate Small Holes
Original Image A
BSE
Closing A by B
9
49
Binary Closing Example 2-- Fuse Narrow Breaks and Long Thin Gulfs
Original Image A
BSE
Closing A by B
50
Binary Closing Example 3-- Remove Salt Noise
• Salt noise consists of small white objects.• The close operator will remove salt noise
in a manner similar to the way the open operator removes pepper noise.
A
B
Closing A by B
51
Binary Closing Example 4
(a)A binary image with five clusters of points. Points within each cluster satisfy the partition property with the distance ρ0, and the clusters are farther from each other than 2ρ0 pixels.
(b)The image of a closed by a disk with a radius just greater than 2ρ0 52
Binary Opening vs. Binary Closing-- Original Image
111111111111111111111111111111
11111111111111111111111111111
1111111111111111111111111111
53
Binary Opening vs. Binary Closing-- Opening (Break Narrow Connections)
1111111111111111111111111111111111111111111111111111111111111111111
54
Binary Opening vs. Binary Closing-- Closing (Fuse Narrow Breaks and Long Thin Gulfs)
11111111111111111111111111111111
111111111111111111111111111111111111
11111111111111111111111111111111
10
55
Properties of Binary Opening and Closing• The opening operation satisfies the following
properties:
• The closing operation satisfies the following properties:
BABBABDBCDC
ABA
ooo
oo
o
=))(3 ofsubset a is then , ofsubset a is If )2
. of (subimage)subset a is )1
BABBABDBCDC
BAA
•=••••
•
))(3 ofsubset a is then , ofsubset a is If )2
. of (subimage)subset a is )1
56
Application of Opening and Closing
(a) An image having many connected objects, (b) Objects can be isolated by opening using the simple structuring element, (c) An image that has been subjected to noise, (d) The noisy image after opening showing that the black noise pixels have been removed.
57
Application of Opening and Closing (Cont.)
The result of closing Figure (d) using the simple structuring
element
Multiple closings for outline smoothing. (a)
after a depth 2 closing, (b) after a depth 3 closing. 58
59
Morphological filtering: (a) original, noisy image; (b) result of erosion; (c) opening of A; (d) result or performing dilation on the opening; (e) final result showing the closing of the opening. 60
Hit-or-Miss Transform• The hit-or-miss transform is a basic tool for
shape detection or object recognition.
• The hit-and-miss transform is a general binary morphological operation that can be used to look for particular patterns of foreground and background pixels in an image.
• The hit-or-miss transform is defined as:Let B ={B1, B2}, where B1 is the set formed from elements of B associated with an object and B2 is the set of elements of B associated with the corresponding background, where B1 and B2 are disjoint.
)2ˆ()1(
)2()1(
BABA
BABABA c
⊕−Θ=
Θ∩Θ=∗
11
61
• A pixel belonging to an object is preserved by the hit-or-miss transform if and only if B1 translated to that pixel fits inside the object and B2 translated to that pixel fits outside the object.
• B1 and B2 cannot intersect, otherwise it would be impossible for both fits to occur simultaneously.
62
Four SEs used for corner finding in binary images using the hit-and-miss transform.
If the foreground and background pixels in the structuring element exactly matchforeground and background pixels in the image, then the pixel underneath the origin of the structuring element is set to the foreground color. If it doesn't match, then that pixel is set to the background color.
63
Four SEs used for corner finding in binary images using the hit-and-miss transform.
64
The Graphical Description of the Hit-or-Miss
Transform
65
Shape Detection-- Find the character ‘n’
B1 =
1 1 0 1 1 1 1 0 0
1 1 1 1 1 1 1 1 0
1 1 1 0 0 0 1 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
1 1 0 0 0 0 0 1 1
B2 =
1 1 1 1 1 1 1 1 1 1 1
1 0 0 1 0 0 0 0 1 1 1
1 0 0 0 0 0 0 0 0 1 1
1 0 0 0 1 1 1 0 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 0 0 1 1 1 1 1 0 0 1
1 1 1 1 1 1 1 1 1 1 166
} and )(|)()(max{))((
bf DxDxsxbxsfsbf
∈∈−+−=⊕
1D Gray-Scale DilationChoose the maximum value of f+b in the interval defined by the shape of the SE, which intersects with the domain of Df.
The range of b intersects with the range of the displaced f.
12
67
1D Gray-Scale Dilation Ex 1
699858974-30-3
-30-3-30-3
-30-3-30-3-30-3-30-3
-30-3-30-39983897
Dilation Result
Original Intensity
68
1D Gray-Scale Dilation Ex 2
551111155
SE
Original Intensity
12321
•What is the size of the dilated result?•What is the final dilated result?
69
At each position of the SE, the value of dilation at that point is the maximum of the sum of f and b in the interval spanned by b.
70
2D Gray-Scale Dilation
• Gray-scale dilation is defined by:
• Where
},;)(),(|),(),(max{),)((
bf DyxDytxsyxbytxsftsbf
∈∈−−+−−=⊕
ly.respective b, and f of domains theare and bf DD
.morphologybinary in case in the
sets of instead functions are and bf DD
71
2D Gray-Scale Dilation Ex 1
Original Image Dilated ImageThe general effect of performing dilation on a gray-scale image is twofold:1) If all the values of the SE are positive, the output image tends to be brighter than the input.2) Dark details either are reduced or eliminated, depending on how their values and shapes relate to the SE used for dilation. 72
1D Gray-Scale Erosion
} and )(|)()(min{))((
bf DxDxsxbxsfsbf
∈∈+−+=Θ
The range of b is completely contained within the range of the displaced f.
Choose the minimum value of f-b in the interval defined by the shape of the SE within the domain of Df.
13
73
1D Gray-Scale Erosion Ex 1
*96369*-30-3
-30-3-30-3
-30-3-30-3
9983897
•What is the result after using the SE = (0 0 0)?•Applying which filter will achieve the same result? 74
2D Gray-Scale Erosion• Gray-scale erosion is defined by:
• Where},;)(),(|),(),(min{
),)((
bf DyxDytxsyxbytxsftsbf
∈∈++−++=Θ
ly.respective b, and f of domains theare and bf DD
.morphologybinary in case in the
sets of instead functions are and bf DD
75
2D Gray-Scale Erosion Ex 1
Original Image Eroded ImageThe general effect of performing erosion on a gray-scale image is twofold:1) If all the elements of the SE are positive, the output image tends to be darker than the input image.2) The effect of bright details in the input image that are smaller in area than the SE is reduced, with the degree of reduction being determined by the gray-level values surrounding the bright detail and by the shape and amplitude values of the SE itself.
76
77
2D Gray-Scale Opening
• The opening of image f by SE b is defined as:
• Opening operations usually are applied to remove small (with respect to the size of the SE) light details, while leaving the overall gray levels and larger bright features relatively undisturbed.
bbfbf ⊕Θ= )(o
78
2D Gray-Scale Closing
• The closing of image f by SE b is defined as:
• Closing operations are generally used to remove dark details from an image, while leaving bright features relatively undisturbed.
bbfbf Θ⊕=• )(
14
79 80
5888468838882778
111121111
Original Image SE
•What is the dilation result?•What is the erosion result?•What is the opening result?•What is the closing results?
81
2D Gray-Scale Opening Ex 1
Original Image Opened Image82
2D Gray-Scale Closing Ex 1
Original Image Closed Image
83 84
The Applications of Binary Morphology
• When dealing with binary images, the principal application of morphology is extracting image components that are useful in the representation and description of shape.– Extract boundaries– Extract connected components– Extract convex hull– Extract the skeleton of a region– Others (Used in pre- or post- processing steps):
Region filling, thinning, thickening, and pruning.
15
85
Basic Binary Morphological Algorithms-- Boundary Extraction
(1) Internal Boundary:(2) External Boundary:(3) Morphological Gradient:
ABAA −⊕= )()(β)()( BAAA Θ−=β
)()()( BABAA Θ−⊕=β
(1) (2) (3)
where B is a 3x3 square SE and A is the solid rectangle
What is the boundary and the thickness of the boundary for each case? 86
Boundary Extraction Example
What is the thickness of the boundary?
87
Basic Binary Morphological Algorithms-- Region Filling
• Consider filling a region surrounded by a curve. Let A be the curve and p a point in the interior of the region A. Then
• The algorithm terminates at iteration step k if Xk = Xk-1.
• The set union of Xk and A contains the filled set and its boundary.
......,3,2,1)(
;
1
0
=∩⊕=
=
− kABXXandpX
ckk
Conditional Dilation
88
Region Filling Example
The goal is to eliminate the black inner spots in each white circle.
How to find the starting points which are contained within the spheres (i.e., black regions completely contained within white region)?
89
Region Filling Example-- Connected Regions in Maze
90
Basic Binary Morphological Algorithms-- Extraction of Connected Components
• Let A be a connected component and p a point on the connected component A. Then
• The algorithm terminates at iteration step k if Xk= Xk-1. The algorithm converges to A = Xk.
......,3,2,1)(;
1
0
=∩⊕==
− kABXXandpX
kk
What is the difference between this formula and the region filling formula?
16
91
Extraction of Connected Components Ex 1
92
Extraction of Connected Components Ex 2
Shown below is an original image, a color-coded image showing the connected components, and an index color palette.
93
Basic Binary Morphological Algorithms-- Thinning
• Thinning removes pixels from a set until only a narrow set remains. It is used to reveal set structure in recognition applications.
• The thinning of a set A by a SE B is defined in terms of the hit-or-miss transform:
• A more useful expression for thinning A symmetrically is based on a sequence of SEs:
where Bi is a rotated version of Bi-1.• The entire process is repeated until no further changes
occur.
cBAABAABA )()( ∗∩=∗−=⊗
}...,,,,{}{ 321 nBBBBB =).)...))((((...}{ 21 nBBBABA ⊗⊗⊗=⊗
94
Thinning Ex 1
Reduce all lines to single pixel thickness.
95
Thinning Ex 2
Original Image Thinned Image
There are broken lines in the thinned image. How to get a thinned image without any broken line? 96
Original Image Thinned Image without broken lines
Solution Preprocess the original image. That is,
dilate the image to connect the lines before thinning it.
17
97
Thinning Ex 3The basic structure is captured by the thinned objects (red). The small ends could be removed by further processing to refine the result.
98
Basic Binary Morphological Algorithms-- Thickening
• The thickening of a set A by a SE B is defined in terms of the hit-or-miss transform:
• A more useful expression for thickening Asymmetrically is based on a sequence of SEs:
where Bi is a rotated version of Bi-1 and all 1’s and 0’s interchanged with the B’s in thinning operation.
}...,,,,{}{ 321 nBBBBB =
)( BAABA ∗∪=•
).)...))((((...}{ 21 nBBBABA •••=•
99
Basic Binary Morphological Algorithms-- Convex Hull
• A set A is said to be convex if the straight line segment joining any two points in A lies entirely within A. That is, any two points can be joined by a straight line that does not go outside the set.
• The convex hull H of an arbitrary set S is the smallest convex set containing S, i.e.,
• The set difference H-S is called the convex deficiency of S.
• The convex set and convex deficiency are needed in many applications such as object description.
.HS ⊆
100
Basic Binary Morphological Algorithms-- Convex Hull
• Let be a set of SEs, then
with
• The convergence result is
• The convex hull of A is:
}...,,,,{}{ 321 nBBBBB =
ik
ik
ik
i XXXD 1where −==
2,...1,k andn ...,2,1,i)( 1 ==∪∗= − forABXX ik
ik
AX i =0
in
iDAC ∪
=
=1
)(
1. The procedure consists of iteratively applying the hit-or-miss transform to A with a series of SEs.
101
Convex Hull Ex 1
(a) (b) (c)
Original Image A sequence of SEs Convex Hull
Origin
What is the minimum convex hull for the original image? 102
Convex Hull Ex 1 (Cont.)
• One obvious shortcoming of the procedure is that the convex hull can grow beyond the minimum dimensions required to guarantee convexity.
• Solution:
Limit growth so that it does not extend past the vertical and horizontal dimensions of the original set of points. That is, Trim C(A) to the bounding box of the component.
What is the final trimming result of the convex hull for the previous example?
18
103
Basic Binary Morphological Algorithms-- Skeletons
• The skeleton of A can be expressed in terms of erosion and openings. That is:
with
K is the last iterative step before A erodes to an empty set.
UK
kk ASAS
0
)()(=
=
BkBAkBAASk o)()()( Θ−Θ=
104
Skeletons Ex 1
105
Skeletons Ex 2
Original Image Skeletons
106
Basic Binary Morphological Algorithms-- Pruning
• Pruning methods are an essential complement to thinning and skeletonizing algorithms because these procedures tend to leave parasitic components that need to be “cleaned up” by post-processing.
• Pruning is a four step procedure. [Refer to section 9.5.8 for details]
107
Pruning Example
Origin
108
Some Applications of Gray-Scale Morphology
• Morphology Smoothing:
The effect is to remove or attenuate both bright and dark artifacts or noise.
.)( bbfs •= o
19
109
Some Applications of Gray-Scale Morphology
• Morphology Gradient: ).()( bfbfg Θ−⊕=
It is a good strategy to use symmetrical SEs since they tend to depend less on edge directionality.
110
Some Applications of Gray-Scale Morphology
• Top-hat Transformation: ).( bffh o−=
It is useful for enhancing detail in the presence of shading.
111
Some Applications of Gray-Scale Morphology
• Textual Segmentation:
1. Use a SE of the size of the small blob to do a closing operation.
2. Use a SE of the size of the large blob to do a opening operation.
3. Thresholding.112
Some Applications of Gray-Scale Morphology
• Granulometry:
1. where b are SEs with increasing size.2. These differences are normalized and then used to
construct a histogram of particle-size distribution.• It is based on the idea that opening operations of a
particular size have the most effect on regions of the input image that contain particles of similar size.
).( bffh o−=
113
Useful Matlab Commands
• strel, getsequence• imdilate, imerode, imopen, imclose• bwhitmiss• bwmorph• bwlabel• imreconstruct, imfill, imclearborder