Localization of Designed Texture on Glass Bottles

8/9/2019 Localization of Designed Texture on Glass Bottles

1/51

Localization of Designed Texture on Glass Bottles Page 1

Technical report, IDE1032, June 2010

Localization of designed texture

On glass bottles

Master Thesis in Intelligent Systems

Abhishek Sharma Pallav Nandi Chaudhari

850326-1250 850908-T314

School of Information Science, Computer and Electrical Engineering

Halmstad University


2/51

Localization of Designed Texture on Glass Bottles


3/51


Preface

This treatise is a masters thesis for the Master of Science program in intelligent systems at

the Halmstad University, Sweden in the IDE department. This thesis deals with the

localization of designed texture on glass bottles at the hot end. This is a research issue from

the company Gedevelop located at Helsingborg, Sweden. The work is carried out with the

guidance of Mr. Kenneth Nilsson. The data for the experiments are provided by Gedevelop.

This project is implemented in the Matlab programming language on a personal computer

system and experimental results on the proposed method are presented.


4/51



5/51


Abstract

Hot end inspection of glass bottles necessitates the localization of designed textures

on that bottle. In this paper we proposed a method to localize designed textures on a test

bottle by matching its image to a certain number of reference images and then mapping the

texture to the test bottle. The intended purpose of this algorithm is to localize the designed

texture on the bottle so that the designed texture areas and other areas on the image of the

bottle can be distinguished. The localization is done by finding the amount of rotation

caused due to the motion of the conveyor belt. The average orientation angle feature is

used in the matching. Reference databases are created which contain necessary information

regarding the different textured regions. A selected portion of the image is used to match

the bottle at the hot end with the same area of the reference bottles. The design texture is

marked, according to the appropriate match. All the methods involved experiments and

results and conclusions based upon them are presented subsequently.


6/51



7/51


Contents

1 Introduction 1

2. Background.... 4

2.1 Real world scenario and their impact.. 5

2.2 Texture recognition and localization..7

2.2.1 Direct template matching7

2.2.2 Feature based matching7

2.3 Propose methods.. 8

2.3.1 Reference model 8

2.3.2 Computations on test image.9

2.3.3 Limitations. .10

2.3.3 Hot spot marking.11

3. Method12

3.1 Feature Extraction12

3.1.1Dominant local direction...13


8/51


3.2 Skew detection and correction and target area

extraction. 15

3.2.1 Skew detection. 15

3.2.2 Target area extraction..16

3.3 Similarity Measure..22

3.4 Hot spot marking.23

4. Experiments and results..24

4.1 Software environment..24

4.2 Data24

4.3Feature Extraction28

4.4 Registration.31

4.5 Matching32

4.6 Hot Spot Marking.36

5. Conclusions..38

References.40


9/51



10/51


1

1 IntroductionFault detection is an essential part of the glass manufacturing process. Different types of

faults occur in glass containers during the manufacturing process [9]. The glass containers

are always inspected by automatic systems. Sometimes human inspection is also done [14].

Most of the inspection is done at the cold end of the manufacturing process. Cold end is a

relatively time consuming process and fault detection is an independent part of this phase.

The company Gedevelop[11] is looking for some method which can detect faults at the hot

end phase, because hot end inspection may give some advantages over regular cold end

inspection as an early warning of problems. Gedevelop is a system development company

which develops and maintains systems for glass manufacturing plants. They have their

camera system which can take pictures of bottles at the hot end and they are interested in

finding manufacturing defects. For detection of faults by the analysis of images, it is

necessary that the system must be able to differentiate between designed texture and

textures made out of defects. One precise way of doing this is by localizing the designed

texture on the glass containers. Once it islocalized, it will be easier to find the areas on the

bottle where inspection should be made for defects.

Hot end phase puts several constraints on the task of fault detection. The environment at

the hot end is not a very friendly environment in which to perform an experiment.

Traditional fault detection equipments cannot work there; no kind of handling of the bottles

is possible. One possible way of inspection is vision based systems. There are also some

limitations imposed by the situation. Multiple numbers of cameras cannot be used. The glass

containers come very fast on the conveyor belt from the molding machine and there is very


11/51


2

limited gap available between two bottles on the conveyor belt. Space is very limited for

fitting multiple numbers of cameras so, normally, only one camera can look at a bottle at a

certain time.

In vision based systems, one of the possible ways to detect different kinds of faults

e.g. cracks, foreign inclusions, etc, can be inspected by texture analysis. In that case, there

must be a classification methodwhich can identify the designed texture on the bottles, so

that the designed texture and textures made out of faults can be differentiated.

In the identification of designed texture on glass bottles at the hot end, there are

certain factors which, to a certain extent, provide the opportunity for it to be modeled in an

easier way, because of its conceptual relevance to some very efficient texture recognition

algorithms, such asoptical character recognition [2, 12]. On the other hand, the real world

situation makes it difficult by imposing some constraints upon the image capturing task.

Practically, when the bottle comes onto the conveyor belt from the forming machine, the

camera can look only at one side of the bottle, which implies that the whole texture area on

the bottle will not be visible to the camera. Due to the high speed of the belt, the bottle may

be rotated to a certain degree. Because of this rotation, the same texture area will not

always be captured by the camera.Keeping in mind all these points, here, we propose a

method to localize the designed texture by finding the rotation angle of the bottle by

matching it to certain number of references and then mapping thetexture to the test bottle.

The method is passive in nature, because it does not directly extract the designedtexture.

In our proposed method, the image pixels are represented by their average directions

extracted from linear symmetric features [4] in a local neighborhood. Rather than using

Matching procedure to find the texture directly, we used matching to find the amount of

rotation. After finding the amount of rotation with respect to a reference image, the

designed texture is marked according to the matching reference.


12/51


3

This treatise is composed as follows:

In chapter 2, the background, an overview of the real world situation in the present

context and motivation for the proposed procedures, is explained. In chapter 3, the

methods involved are discussed in detail, whereas in chapter 4 and chapter 5, the

experiments and their results, and their future scopes, are described.


13/51


4

2 BackgroundTexture recognition is an image processing technique which has a wide variety of

applications in different scientific research areas e .g. optical character recognition[2], face

recognition[15] fingerprint recognition[1][8], biometrical identification of laboratory

animals[13], medical image analysis[6], robot navigation [16]etc. For human eyes it is

possible to distinguish between different textures easily with natural visual capabilities.

Developing such a machine which will have such a competent visual system is difficult.

Usually, geometrical or mathematical representation of arbitrary uneven shapes is difficult

to implement. So, in algorithm design, uneven shapes are always approximated to their

closer geometrical shapes which are mathematically well defined. In certain situations, this

approach is not good enough, e.g. biometric person authentication. In simple terms, it can

be said that there cannot be any universal generalization to categorize textures. Different

researchers have come up with different commendable ideas which fit in to their relevant

application areas. The development of a texture recognition algorithm is mostly dependent

on the application scenario.

The problem of localization of designed texture on glass bottles at the hot end by image

processing has certain number of important facts associated with it which arise out of the

real world situation where the image has been taken.

The inspection of glass containers at the hot end of the manufacturing process is a common

intention of the glass industries all over the world nowadays. Research is being conducted

on this problem using different technologies. However, still today the research on this field

is not in the public domain, as they are bound inside the different companies [10] which are

working on this. So it is difficult to get ideas about the relevant ongoing work in this


14/51


5

particular field. E.g. one of the renowned companies of Europe1, worked on this problem

and they put some information about this in their website. In short, they just disclosed that

they are using the heat radiation property of hot bottles and capturing images of hot bottles

using infrared camera. Information about further processing of these images is not

mentioned in detail.

2.1 Real world scenario and their impact

The hot bottles from the molding machine are put onto a conveyor belt and

transported, one after another, at a certain speed. The camera is placed at a certain

position, oriented orthogonal to the direction of motion of the conveyor belt, such that it can

take pictures when the bottles are passing by it. According to Gedevelop, the bottle can be

rotated around the vertical line passing through its physical center of gravity up to +/-15

degree due to the speed. This will be clearer from figure 2.1, which represents the top view

of a bottle on the conveyor belt. Here, the conveyor belt is moving along XX and the

camera is looking by YY direction. The maximum range of angle is +/-15 degrees. For the

same speed factor, it can be inclined at a certain angle to right or left. It is also not always

necessary that the optical axis of the camera will pass through the center of the bottle.

These facts have a deep impact on the opted strategy of localization in the present scenario.

1 The name cannot be disclosed.


15/51


6

Figure 2.1 The top view of a bottle on the conveyor belt showing the

rotation in interval +/-, where =15 degrees

Because of the rotation, the pictures taken by the camera will not always contain the

same textured region. This indicates that, although textures can be extracted directly in

different ways, their localization will not be easier. This is because of the change of the

image due to rotation. If, for every bottle passing in front of the camera, the same picture is

taken, then the pixel coordinates of the designed texture can be easily defined. As the bottle

is rotating, the same designed texture has changed pixel locations on different images. The

localization of designed texture is difficult in a direct way, because the designed texture and

texture from faults must be differentiated. So the problem is twofold. In the first place, the

texture has to be extracted. Secondly, the designed texture has to be located according to

the change due to rotation. This can be done by finding the rotation angle of the bottle on

the conveyor belt. The rotation angle can be found by matching the image of the test bottle

on the conveyor belt with some reference images of the bottle rotated at different angles.


16/51


7

2.2 Texture recognition and localizationIn texture recognition and localization, the methods can be divided into two

categories in general. One is direct template matching and another is feature based

matching. The main difference between the two is that, while the former uses pixel values of

the images (e.g. for gray images gray values are used) directly, the latter uses different

useful features (e.g. linear symmetry feature) for texture description and matching.

2.2.1 Direct template matchingIn simple applications, template matching is useful because it is easy to

implement and less complicated from a mathematical point of view. However, in complex

situations it is not so useful. In the present case, the designed texture occupies little area

compared to the whole image. As a result, if the matching is done using the whole images,

there will be aberrant results as non textured regions will dominate the calculations. It is

thus necessary to match only those parts of the image which contain the same designed

texture. Extraction of such an image area can be done in a robust way from feature images

than grey images. Using gray scale values as feature is more sensitive to noise, as gray

scale values change with illumination.

2.2.2 Feature based matchingFeature based matching is nowadays applied in wide variety of online and offline

applications. E.g. many optical character recognition algorithms use linear symmetry

features to localize a certain characters in a text [2]. In fingerprint analysis, the test finger

print is matched with the reference finger prints using different features. The two finger

prints are aligned using core and delta points [1], and then matched using the extracted

feature.


17/51


8

2.3 Proposed methods

The matching in the methods mentioned in section 2.2 is done using Schwartz Inequality

[5]. If we try to find and localize the different designed textures on the test bottle directly

by matching the designed textures on the reference images with the test bottle, the

amount of computation will be high, as each and every design texture of each reference

image has to be matched with a single test bottle. We propose a step-by-step method,

which is relatively fast, because a limited number of reference templates are necessary.

Our proposed method is passive in nature.

As it was indicated by the Gedevelop that the rotation is a constraint which imposes a

number of limitations to the task of direct investigation of designed textures, it is important

to find the nature of the image at different degree rotations. This can be done by taking

reference images of the bottle at different angles. These reference images will give

information about the shifts in texture positions due to rotation. The relative position of

designed textures in the image at a certain rotation angle can also be found. If the test

bottle on the conveyor belt is matched with the reference images, then an approximation

about the rotation of the test bottle with respect to the direction of motion of the conveyor

belt and location of the designed texture corresponding to that rotation angle can be found.

In this way, the designed texture can be localized indirectly.

The steps involve some of the well known methods e.g. linear symmetry calculation,

cross correlation etc. in such a way that the design texture can be localized successfully.

The steps show our intention of how we look at the problem and its solution.


18/51


9

2.3.1 Reference model

As indicated by the Gedevelop, there can be +/-15 degree rotation of the bottles on the

conveyor belt; the first step is to create reference models by taking pictures of the bottles

at different rotation angles. The reference model creation will involve the following sub

steps:

(a) Calculate the linear symmetry feature for each image as described in [4] and [5].The linear symmetry feature is defined by second order complex moments, I20 and I11.

(b) Correct the inclination by detecting the skew angle based on the second ordercomplex moment component I20.

(c)Find the vertical boundaries of the bottle by using the second order complex moment I 11[2]and clip the image according to that contours.

(d) Using a mathematical formula, clip out the target area from the image obtained bystep(c). The region should be such that the pixels associated with textures are denser

compared to other parts. Extract the average orientation for each pixel value of this

clipped region, i.e. the angle of the I20 feature.

(e) Tabulate the result of step (d).

(f) The pixel coordinates for the designed texture are assumed to be known for each

reference image.

Steps a, b, c, d and e have to be performed for each image taken for reference creation.

2.3.2 Computations on test image

Test image is the image of the bottle on the conveyor belt, on which the designed

texture has to be located. The necessary computations on the test image can be listed as:


19/51


10

1) Perform steps a,b,c and d of 2.3.1 on the test image.2) The extracted average directions of each pixel of the region obtained by step 2.3.1d

have to be matched with each reference image. The maximum value of the match will

indicate the approximate rotation angle of the test bottle. This is a passive way of

calculating the amount of rotation.

3) Select the reference bottle with which the maximum value out of matching has beenfound.

2.3.3 Limitations

For the full investigation of the problem, it is necessary to have reference images of a

particular type of bottle rotated at different degrees in the interval [-15degree,15

degree].Due to limited data provided by Gedevelop, we have had certain restrictions in our

experiments which cut our journey short towards the full investigation of the problem. E.g.

we do not know the actual rotation angle of the bottles of the images, so we have

approximated it by visualizing. We categorized the images into three classes. In class 1, the

images which are close enough in rotation are assumed to be in 0-5 degrees rotation range;

in class 2, the images which seemed to be not so close are assumed to be in 6-15 degrees

rotation range and, in class 3, the images which are far in rotation are assumed to have

more than 15 degrees rotational difference. Our work was to investigate the correctness of

the matching procedure, i.e. to get a high matching score for the images which are close in

rotation angle and low matching scores for high difference in rotation angle. No study of the

uncertainty in the localization of the design texture could be performed. For real time

implementations, those are necessary points.


20/51


11

2.3.4 Hot spot marking

Hot spot marking is usually done in geographical maps to show the location of a

certain place on the map by putting a mark on that location. We took the same approach to

locate the design texture on the test image.

Knowing the rotation angle, i.e. the appropriate reference bottle, the pixel coordinates

belonging to the designed texture can be marked on the test bottle.

.


21/51


12

3 Methods

3.1 Feature Extraction

The average directions from each local image have to be extracted. The average direction is

calculated from the linear symmetry features of the image. The linear symmetry feature is

nowadays the back bone of various texture recognition algorithms and biometrical analysis.

It has first been intensively investigated by J. Bigun and G.H.Granlund in 1987[4]. In an

image, usually the gray values do not change at a particular direction; rather, the change is

random. Near the edges and textures, the gray values are constant along a line. This

fundamental fact gives rise to linear symmetry approach where, in a local neighborhood, the

ideal local orientation tensor or linear symmetry tensor is calculated. The linear symmetry

calculation is a feature extraction method which extracts and represents the image using

the linear symmetry features.

The orientation tensor or linear symmetry tensor is the representation of the local

orientation of an image in a neighborhood. The details about this can be found in [4][5].

This orientation tensor is represented in complex form for the ease of calculations. Linear

symmetry needs the doubling of the angle of orientation. The reason lies in the difference

between the direction of a line and orientation. There is a subtle difference between the

two. The direction of a line spans over the 0 to 2 angle range. When we talk about

orientation, the orientation is described in the direction orthogonal to a line along which

gray values do not change. This means there could be situations where orientation vectors

will be bilaterally symmetrical around that line. So the doubling of the orientation angle is

necessary.


22/51


13

3.1.1 Dominant Local Directions

In mathematics, moment is a description of the structure formed by a set of points.

In image processing and computer vision field, the moment almost does the same thing. It

is a weighted average of the image pixel intensities. Image moments are extensively used

to deduce certain properties e.g. centroid, orientation etc. The complex moments are

sometimes preferable compared to real moments [5], because of the ease provided by

complex representations in algebraic calculations. More about the use of complex moments

can be found in [7]. Similar to real moments, complex moments are weighted averages of

pixel values represented in complex form in a neighborhood. The complex moments of a

function, z(x,y), is calculated as follows

= + . (, ) (3.1)

The order of the complex moment is (p+q)[5].

The second order complex moments 20and 11, which can be used to estimate the linearsymmetry features of an image in a local neighborhood are defined in the frequency

domain as

20 = + 2 . | ,|2 (3.2)

11 = + . | ,|2 (3.3)

In the spatial domain they can be computed as

20 = , (,) (3.4)

11 = |, | (,) (3.5)


23/51


24/51


15

that the ratio can never be 1. The value will be near to 1. So, one threshold value for this

ratio have to be selected, which will indicate approximate linear symmetry. Only those pixel

positions will be taken for granted where this ratio will be higher than this threshold value.

As cited in [5], the dominant orientation in a local neighborhood can be extracted from I20

as

= 12argument20 (3.7)

The factor1

2in equation (3.7) is due to the double angle representation.

3.2 Skew Detection and Correction and TargetArea Extraction

As indicated in section 2.3, it is important to correct the skew, find the left and right

most vertical contours of the bottle in the image and find some significant texture area. By

doing these steps, the inclination will be corrected, the contours of the bottles will be

aligned and lastly the computational cost of finding the amount of rotation will be small as

we will use only the selected texture portion of the bottle for matching purposes.

3.2.1 Skew detection and Correction

Skew detection and correction is a mandatory part in optical character recognition.

Although our present task is in a totally different environment than OCR, skew detection is

still necessary. Inthe real life scenario, the test bottle may be skewed a little because, after

forming, the hot bottles are put onto very fast conveyor belt and the conveyor belt itself is

moving very fast. So, it may be skewed a little at the time when camera is taking its


25/51


16

picture. For skew detection, we use the method cited by Lalith Premaratne, Yaregal

Assabie and Josef Bigun as described in[2].

As cited in [2], the directional features of the image can be used to detect the skew in the

image. The second order complex moment component, I20 as described in section 3.1.2,

which represents the information about orientation angle in a small neighborhood, is used to

detect the skew angle of the image. The summation of I20 values over the entire image

gives an almost approximate value of the skew angle. The accuracy is improved by

removing the contribution from the background, considering those values of I20, which have

high confidence according to [5].

3.2.2 Target area extraction

Vertical boundary

Once we have corrected the skew, next the vertical boundary detection is done. Here

vertical boundary refers to the left and right most vertical contours of the bottle in the

image. The second order complex moment component I11, which represents the sum of themagnitude of the orientation vectors in a local neighborhood, is used to do this task. This is

done by adding the I11 values of all the columns of the skew corrected image. The

motivation behind using I11 values is that the I11 values are high at the edges and very low

elsewhere. The longest vertical edges are the vertical boundaries of the bottles. So, by

column wise adding of I11 values, we will get high values at the vertical boundaries of the

bottle.


26/51


17

Figure 3.2 The boundaries of the bottle encircled with green ellipses are to

be found

Suppose, Abe an image of size m x n, then its I11 matrix will also be of size m x n.

Column wise addition of I11 matrix will produce an array of length n where each element will

be the sum of all the row values of that corresponding column, from where we can deduce

the approximate column indices of the major vertical borders. Mathematically, it will look

like

= , = 1,2 . . ,=1 (3.8)

From() of equation (3.8), column indices of left and right boundary can be extracted as-

= max=12 (3.9)

= max =2+1 (3.10)


27/51


18

Figure 3.3 The two high peaks rounded with red ellipses are indicating

the vertical contours of the bottle.

When equation (3.8) is applied to I11 matrix of figure 3.2, the resultant|=1 looks like figure 3.3. The two desired boundaries as indicated in figure 3.3 by red ellipses

can now be extracted by equation (3.9) and equation (3.10). The equations (3.9) and

(3.10) will give the column indices of left and right boundary respectively. When the column

indices of the desired boundaries are found, an image portion can be extracted, where the

left and right boundaries will be the marked boundaries of the figure 3.3. Such an extracted

image is shown in figure 3.4


28/51


29/51


20

textured area which is denser compared to other portions of the image and is changing with

rotation. This kind of texture can be selected by looking at the image of a bottle.

To clip out some significant texture area from the image, a method is developed being

motivated from some obstacle avoidance algorithms, which deal with mobile robot

navigation, namely the certainty grid representation for obstacle representation,the VFF

conceptand vector field histogram methods. A brief introduction to these methods is given

in [3]. In these methods, a robots working area is divided into a certain number of grids

and it is assumed that each of the grids exerts some kind of force on the robot. The total

force is the summation of the individual forces applied by each grid. So, logically, it can be

said that each grid is given some weight, depending upon certain conditions, and the result

is an average of the individual grid components. This resultant element gives the obstacle

density at a certain point where the robot was located at that particular time.

We propose a similar concept with an exception that, rather than weight we

concentrate on each pixel value as mass. Our reason for looking into it as a mass is that a

design texture can be anywhere on the bottle, so it is not possible to weight one pixel value

higher or lower than the other. Rather, we should have a method that can weight each pixel

position equally and, at the same time, can extract the design texture area.

The first step is to convert the cropped I11 image, obtained by equation (3.9) and (3.10)

to a binary image. The I11 image highlights the edges of the image, but it does not give

equal value to each edge pixel. So to obtain equal pixels value at the edges and equal pixel

values elsewhere, the I11 image is converted to binary image. This is done by

experimentally setting a threshold value in the I11 images. All the pixel values higher than

this threshold are given value 1 and other pixel values are set to the value 0 . Fig-3.5 is

obtained by converting an I11 image into a binary image.


30/51


21

The next step is to define an filter of size wh x wv, where each value will be of equal

weight. However, the question is what should be the size of the window? The width of the

window is equal to the number of columns in the vertically limited image matrix, i.e the

width of the bottle. The height of the window should be determined carefully by looking at

the image. This should be large enough, so that it covers the desired texture area but, on

the other hand, it should not be so large that it can fit some unnecessary pixel positions in it

while it is covering the desired texture area. E.g. if we look at figure 3.5, it can be noticed

that the dense texture is made in the middle of the image, where some text is inscribed on

it. In the image, this region will change with rotation. Other portions of the image do not

produce any visual effect due to change of rotation angle. So, this portion can only be used

to identify the rotation angle. The height of the window should be such that it covers the

entire texture.

If we convolve this filter with the binary image, it will give maximum value when it

includes the targeted textured area. Near the textured region, the white pixels are denser

than other locations. Simply at that region the pixel density will be the maximum. If

necessary, further abbreviation can be achieved by considering only the central part of thedensest area. If we vertically divide the densest area into three equal parts, we will select

the second part on that. This should be the same for all the reference images and test

images. In this way, the significant texture area from the images can be extracted, which

will have a difference due to rotation but lies within the same pixel coordinates.


31/51


22

Figure 3.5 I11 image converted to binary image.

3.3 Similarity Measure

Two images, I and J, are matched by calculating the similarity measure between the two

images. This is particularly necessary to find the rotation angle of the bottle on the conveyor

belt.

50 100 150 200 250 300

200

400

600

800

1000

1200


32/51


23

The similarity measure gives high value for similar images and gives relatively low value for

dissimilar images. Selected image pixels (see 3.1.1) are represented by the average

direction of its second order complex moment I20. After extracting the texture regions as

described in the previous section, they are matched as

= ( , ).(, ), (3.11)

In equation 3.11, the values of will be in the interval 0 and 1. The value near to 1indicates high degree of similarity, whereas values near to 0 indicate less similarity.

3.4 Hot spot marking

Hot spot marking is achieved by one to one mapping of the pixel coordinates of an

image J to another image . Here, is the test image and is a texture region in thecorresponding reference image. The images and are of the same size. The appropriateregion on is marked according to the location of on the appropriate reference image withwhich has maximum similarity. Suppose the location of a point on the reference image is(a,b), then the hot spot ofwill be (a,b) coordinate point of the matrix. It is simple one toone mapping, where each pixel on the text image is stained according to the location of

texture on the corresponding reference image.


33/51


24

4 Experiments and results

The experiments performed and their results, are presented in this chapter. The results of

preprocessing of images, design of reference models, matching of the test images with the

reference images, interpretation of matching results and then hotspot marking are

described subsequently.

4.1 Software environment

We have performed all our experiments at simulation level. For simulation level

experimentation, MATLAB has a very compact, easy to use and easy programmable

environment. The images provided by the company Gedevelop are in .bmp format, which is

easy to handle in MATLAB. The image data are in unit-8 format. Some experiments require

double data type supportability, and it is a well known fact that different types of imageprocessing techniques can be easily done in MATLAB. Along with some image processing,

there are some mathematical implementations. This is also easier in MATLAB compared to

other well known programming platforms. So we choose MATLAB for experiments and

implementations.

4.2 Data

Gedevelop provided 34 and 39 images in .bmp format respectively of two different

kinds of bottles rotated at different angles denoted as B1 and B2. Out of 34 images in B1,

we have created two sub databases denoted as B1A and B1B. . B1A contains a total of 24

images while B1B contains 10 images. B1A contains those images which have designed


34/51


25

texture made on their front side of the bottle from the camera viewpoint as in figure 4.1.

B1B contains those images which have design texture on the back side from cameraview

point as in figure 4.2. The motivation behind dividing B1 into two groups is that the

designed textures are made on the opposite sides. Possibility of this kind of situation can be

ignored as it requires more than 90 degree rotation, which is practically inconceivable.

Consequently, we wanted to see the effect of matching of these two types of independent

situations. Figure 4.3 is showing an image of B2. Here, one point to notice is that the

textures on the images of B1 are better defined than thoseof B2.

The images provided by company are used to create a reference database. Those

images are 1380 x1012 pixels in size. Because of automatic clipping of some significant

texture area, done according to the steps described in section 3.2.2, we got images of size

100x200 pixels. Figure 4.4 is one such automatically clipped image from B1A.

Second order complex moment I11 values are used for automatic extraction of significant

texture area. Binary image of the boundary detected image was created by setting a

threshold value. The I11 values greater than that threshold was set to 1 and other pixel

values were set to 0. One such binary image is shown in Figure 3.5. Reference models

include:

1. Original image taken by the camera.2. The linear symmetry matrix of the corresponding image.3. The dominant direction information of each pixel of I20 image.4. The vertical boundary limited image.5. The dimensions of the densest texture region.6. The pixel coordinates of the designed textures.

The results of the individual steps as well as of overall experiments are explained in the

following sections.


35/51


26

Figure 4.1 An image of database B1A having texture on front side of the

bottle


36/51


27

Figure 4.2 An image of database B1B having texture on back side of the

bottle

Figure 4.3 An image of database B2


37/51


28

4.3 Feature Extraction

Initially, the second order complex moments, I20 and I11 are calculated. The first step

towards the calculation of I20 is done by filtering the original image with two Gaussian

derivative filters: one in the X-direction and the other in the Y-direction. Then, by

combining these two derivative values, the complex valued gradient image is produced

according to equation (3.6). Next, the complex valued gradient image pixel values were

averaged in a local neighborhood using a Gaussian average filter. I11 was computed by

calculating the absolute value of each pixel of the complex gradient images and then

averaging the local neighborhood pixel values.

In MATLAB implementation, four Gaussian averaging filters were produced; two of them

are in X direction and two are in the Y direction. One set of filters had standard deviation

value 0.8 and other set has standard deviation value 1.2. The filters with standard deviation

value 1.2 are used to take the average value of complex gradient pixels in the local

neighborhood. Two Gaussian derivative filters are produced using standard deviation value

0.8.

I20 values calculated thus are used to correct skew and to calculate average directions for

matching. I11 values are used to detect vertical boundary as indicated in section 3.2.2 and

section 3.2.3. In figure 4.4 and 4.5, two target area extracted images are shown.


38/51


29

Figure 4.4 Result of target area extraction on an image of B1A

Figure 4.5Result of target area extraction on an image of B1B

20 40 60 80 100 120 140 160 180 200

10

20

30

40

50

60

70

80

90

100

20 40 60 80 100 120 140 160 180 200

10

20

30

40

50

60

70

80

90

100


39/51


30

Figure 4.6 Orientation tensor in color of figure 4.4.

20 40 60 80 100 120 140 160 180 200

10

20

30

40

50

60

70

80

90

100


40/51


31

Figure 4.7Orientation tensor in color of figure4.5.

4.4 Registration

We used pixel by pixel matching of the average direction values of the clipped images.

Usually, in such kinds of matching process, alignment of the texture is a regular concept.

However, in this case, it is not, because we are trying to find the rotation angle. If the

textures are aligned, then the matching will give aberrant result. The alignment of the

contour of the whole bottle image is necessary, so that we can get a common arbitrary

origin from where the bottle can be observed. The bottle may be inclined due to high speed

of the conveyor belt and also it may be translated a little at the right or left. Thus, for

accuracy purpose, the image of the whole bottle should be aligned relative to its contour.


41/51


32

The problem out of inclination is corrected due to skew correction. Translational

misalignment is automatically corrected because of the boundary extraction, as described

in 3.4.2. As a consequence, when we go for clipping out the texture region from the vertical

boundary detected images, we always start the calculation from the left upper corner of the

image which is vertically above the left most vertical boundary of the bottle. The

translational mismatch is thus compensated. It can be said that the bottles become aligned

according to the vertical axis passing through its center of gravity. Thus we can match two

complex images by their overlapping area. This kind of matching is done in [13]

The purposes of the methods described in section 3.4 are twofold. They serve both

automatic clipping and registration.

4.5 Matching

Matching is done by computing the normalized similarity score, according to the equation

(3.11).Each image of B1 is matched with all 33 images of the database except itself. Again

each image of B1A is matched with all 23 images of the database. Similarly, it was done for

B1B and B2. One image is not included in each matching step and that is the bottle itself, as

we will get the value 1 for that matching, which is meaningless in a practical situation. In

total we have 34, 24, 10 and 39 such instances for B1, B1A, B1B and B2 respectively where

the similarity score is 1 according to equation (3.11). We have 1122(=34 x 33), 552(=24 x

23), 90 (=10x9), and 1482(=39x 38) scores for B1, B1A, B1B and B2 respectively where animage was matched with other images in the respective database. The similarity score

distributions for B1, B1A and B1B are shown in figure 4.8, figure 4.9 and figure 4.10. The

figure 4.8 depicts the similarity score distribution for B2 images, where matching is done

based on average direction. By looking at the nature of the figures 4.8, 4.9,4.10 and 4.11,


42/51


33

we found that, for B1 images the matching results are more prominent, as for almost closer

images it has matching scores closer to one(above 0.8) . For B2 images, it is a little lower.

In figure 4.8, 4.9, 4.10 and 4.11 the region marked with green circle shows the similarity

scores when the images are in class 1. In the blue circled region, we have the scores for

those images which are in class 2. The red region indicates least similarity. Here, images

are in the class 3. Class 1, class 2 and class 3 are defined in section 2.3.3.

Figure 4.8 Matching result of B1 images matched using average directions


43/51


34

Figure-4.9 Similarity score distribution for B1A using average directions


44/51


35

Figure-4.10 Similarity score distribution for B1B using average directions

Figure-4.11 Similarity score distribution for B2 using average directions


45/51


36

4.6 Hot Spot Marking

In figure 4.12, an image is shown on which design textures are marked. It is done

according to the method described in section 3.4. The pixel coordinates of the designed

textures are indicated by white colour.

Figure 4.12 Texture, marked on a B1A image

In real time application, appropriate match for the test bottle will not be found: it will be

approximated to the closest reference image. A typical situation which may arise is that it

may not cover the entire texture region because of this approximation. In that case,

uncertainty will be there and for this reason, it is necessary to calculate the uncertainty in

the position of the designed texture. According to that uncertainty the marking should be

done so that it covers up the entire texture region on the test bottle, otherwise it will cause

100 200 300

200

400

600

800

1000

1200


46/51


37

problems to further intended processings. We could not investigate this because of

limitations in the data.


47/51


38

5 ConclusionsIn a nutshell, the ideas proposed here can be used to identify the rotation angle of the

bottle on the conveyor belt in a passive way so that the designed texture on the test bottle

can be localized according to the designed texture coordinates on the particular reference

bottle with which it has maximum similarity. We have proposed the use of the average

direction of linear symmetry feature for texture recognition as it is relatively free from

different kinds of errors caused by gray value changes from image to image due to

environmental changes. Then we proposed an idea to mark the location of the designed

texture on the test bottle in a direct way. The test results for finding rotation angle are

positive. Overall the proposed method works satisfactorily on the test data.

Depending upon the shortcomings of the proposed method, the future works include:

1) Fixing a threshold valued for matching results above which the similarity can be taken for

granted. This could be done by creating reference model with the images at 1 degree or 0.5

degree rotation gap.

2) Uncertainty of the localization results has to be calculated.

3) The automatic clipping out method of some significant texture area has to be made more

robust so that it can be applied in the situations where there is less dense texture in a

particular region.

4) The algorithm can be made faster by translating to any machine friendly programming

language.

5) The uncertainty in localization must be used in the hot spot marking by putting an

uncertainty box around each position.


48/51


39

6) It should be modeled in such a way so that it can be applied in a real time environment.


49/51


40

References

1. Kenneth Nilsson, Josef Bigun,Registration of Fingerprints by Complex Filtering and by1D Projections of Orientation Images, Audio and Video based Person Authentication-

AVBPA 2005

2. Lalith Premaratne, Yaregal Assabie and Josef Bigun ,Recognition of Modification-basedScripts Using Direction Tensors. 4th Indian Conf. on Computer Vision, Graphics and

Image Processing(ICVGIP2004), 587-592.

3. J. Borenstein and Y. KorenThe Vector Field Histogram - Fast Obstacle Avoidance for Mobile Robots

IEEE Transactions on Robotics and Automation, Volume 7, No. 3, pp. 278-288, June,

1991

4. J. Bigun and G. Granlund (1987)Optimal Orientation Detection of Linear Symmetry". First int. Conf. on Computer

Vision, ICCV, (London). Piscataway: IEEE Computer Society Press, Piscataway, (Tech.

Report LiTH-ISY-I-0828, Computer Vision Laboratory, Linkoping University, Sweden

1986. Thesis Report, Linkoping studies in science and technology No. 85, 1986).

pp. 433438.

5. Josef Bigun, Vision with Direction, A systematic introduction to Image Processing andComputer Vision, Springer.

6. R.S. Ledley, Automatic pattern recognition for clinical medicineIEEE,Volume: 57 Issue: 11 , page:2017 - 2035


50/51


51/51


15. Hartwig Fronthaler & Klaus Kollreider, Real-time Face Tracking: Enhancing theauthenticity of Recognition. Masters thesis in computer Systems engineering. IDE

School of information Science, Computer and Electrical Engineering. Halmstad

University.

16. Rolland SiegWart, Illah R. Nourbakhsh, Introduction to autonomous mobile robots,Amazon.

Localization of Designed Texture on Glass Bottles

Documents

Transcript of Localization of Designed Texture on Glass Bottles