LabVIEW Machine Vision for Line Detection

EGR315: Instrumentation Spring 2010

Than Aung and Mike Conlow

Department of Physics and Engineering

Elizabethtown College

Elizabethtown, Pennsylvania

Email: aungt@etown.edu, and conlowm@etown.edu.

Abstract – To improve the previous

development of a visual obstacle

avoidance algorithm without using

proprietary software that is not

financially practical to purchase. The

goal is to use more advanced methods to

produce more concise output in terms of

turning angle and the nearest point of

interest.

I. Introduction

The following data is an analysis of

improvements made to a system previously

developed using NI Vision Development

software to detect white lines on non

uniform grass. The need for this arose due to

over complexity of a vision system that is on

an autonomous robot that is used as a

learning platform. The current system [5]

uses a DVT Legend 554C that collects and

filters the images internally and transmits

the relevant data, via TCP/IP string, to a

LabVIEW program that is a closed loop

motor control. During the fall semester of

2009 using the NI Vision Development

package a prototype virtual instrument was

developed to attempt to improve processing

speed by using LabVIEW for the entire

image processing procedure through a USB

webcam [6].

There were several improvements that

needed to be made to the prototype in order

to justify its implementation over the

previous vision system. The turning

algorithm depended on a set of line

detection sub virtual instruments that

generated large amounts of noise due to

inadequate intensity filtering. To resolve

these and other issues the filtering,

thresholding, and line detection were

programmed using base package of

LabVIEW, using LabVIEW IMAQ and

IMAQ USB to capture the images from a

webcam [6].

The result is a great improvement over the

previous version. Further enhancements still

need to be implemented in order to operate

properly in the field, but the goals that were

set for this semester have been met.

II. Background

The previous project mainly employed NI

Vision Development Module 9.0 (Trial

Version), which provides various image

processing and machine vision tools. By

using the edge-detection sub virtual

instrument we implemented the following

line detection algorithm.

The image resolution is set to 320x240

pixels capturing at 8 frames per second.

Each frame is then converted to an 8 bit

gray-scale image and then the image is

segmented into regions as follows [6]:

Figure 1: Edge Detection Regions

White lines are detected with IMAQ Edge

Detection by finding lines in the eight

boarder regions represented in green in

Figure 1. In our algorithm, we use two

vertical lines (VL1 and VL2) and two

horizontal lines (HL1 and HL2) detectors.

VCL (Vertical Center Line) is then

calculated by averaging VL1 and VL2.

Likewise, HCL (Horizontal Center Line) is

calculated by averaging HL1 and HL2. The

line angle is then calculated by find the

angle between HCL and VCL, by using

Where m2 is the slope of HCL and m1 is

the slope of VCL. By using the intersection

point and the angle between HCL and VCL,

the appropriate heading for the robot is

determined.

Although the algorithm seems simple

enough, there are a lot of drawbacks. First,

when converting from 32 bit color images to

8 bit gray-scale images there is a loss of

edge information in every frame. In the

presence of background noises it is very

difficult to detect stable edges, thereby

making line detection less accurate. Second,

using four edge detectors is unnecessarily

redundant, and over-use of edge detectors

results in slower processing. Third, we did

not have time to implement the filters to

eliminate the noises and to threshold the

unnecessary pixels information. Finally,

since we used the 30-day trial version of NI

Vision Development Module, to continue

using the program the only option was to

purchase the three thousand dollar full

version.

Therefore, the primary motivation of our

project was to solve the problems we faced

by using NI Vision Development, and

improve upon the shortcomings of the first

project. With the main goals in mind, we

developed the second version of our line

detection algorithm.

III. Implementation

Our project goals were to reduce the

noises during the image acquisition, enhance

the edge information, and stabilize the

detected line even with the background

reflections and light sources. Therefore, we

divided the project into different modular

processes to achieve these goals.

A. Single Color Extraction

The images acquired from the camera

(Creative VF-0050) are 320x240 32-bit

color images. Although we can simply

convert the 32-bit color (RGB) images to 8-

bit grayscale images by averaging the 32-bit

color, we have learned how to use a better

method for eliminating the noises and

enhancing the edge information. Since the

background of the images is mostly green,

we decided that if we just simply extract the

blue color pixels from RGB the images, we

can reduce the noises and enhance the white

color lines. The thought process behind this

is that the dirt and grass are mostly

composed of reds and greens, so if we were

to only look at objects composed of some

amount of blue the most intense blues would

be whites.

In binary format, 32 bits color is

represented as follows:

Alpha Red Green Blue

xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx

(x is a binary 1 or 0). In order to extract Blue

color information, we performed an AND

operation with the 32 color bits with the

following binary bit mask [2].

0000 0000 0000 0000 0000 0000 1111 1111

Figure 2: 32-bit Color Image

Figure 3: 8-bit Blue Color Image

Figure 4: Blue Color Extraction

By extracting the blue color plane from an

image only pixels with a high blue intensity

will appear white. This will reduce some

noise from high intensity greens and reds.

To eliminate noise from natural reflections a

spatial convolution using a flattening filter

will be used to further enhance the image

edges.

B. Spatial Convolution Filter

To prevent large quantities of noise it was

necessary to implement a convolution using

a 7x7 flatten kernel [2]. Since the image is

represented as a two dimensional matrix of

blue intensity values, after the color

extractions, a convolution using a kernel of

all ones will be applied to the image

reducing high level intensity values. The

reason this is done after the blue plane is

extracted is to prevent the high intensity

greens and reds from mixing with blues

causing the blue plane to be inaccurate when

extracted.

Figure 5: LabVIEW Convolution

The figure above is the virtual instrument

for a convolution, where X is the image

matrix and Y is the convolution kernel.

Since there are many convolutions being

performed the algorithm is using frequency

domain convolution. The way the operation

is performed first requires that the image

matrix is padded horizontally and vertically

by one minus the width and height of the

kernel [4]. Then by shifting the kernel over

the image the padded matrix is given the

values from the convolution, which is the

Fourier Transform of X and Y multiplied

together as a dot product in terms of two

dimensional matrices and then the result is

transformed back to give the value desired

in the original resolution of 320x240 [4].

The following figure is a representation of

how special convolution is utilized to apply

a kernel to a simple set of data.

Figure 6: Padding and Filtering

From the resulting matrix it is possible to

get the average value of every 7x7 value by

dividing the elements by forty-nine to get

the values in the matrix in terms one byte

per index. Now all the high intensity noise

that would have thrown off the later line

detection should be gone as long as the noise

doesn’t appear in large groups.

From the flattened image it should now be

much easier to find the white lines.

However, since the image was flattened the

line edges will not be as intense as they

were. So the next step is to determine the

highest intensity values in the image to

attempt to only detect the highest intensities

in a set range.

C. Intensity Analysis

After the single-color extraction and

filtering, the image seems to be ready for

edge detection. However, there is still one

problem we have to solve before performing

the edge detection. Under non-uniform

background lighting, the maximum image

intensity and intensity distribution of the

image change accordingly, and it is almost

impossible to perform normal thresholding

to detect the edges. Therefore, we need to

analyze the image intensity distribution. In

order to do so, we first tried to acquire the

intensity histogram of the image, which

includes both intensity range and frequency

of each intensity value. Once we know the

intensity values of the image and their

frequencies, it will become easier for us to

determine the edges that we are interested.

Figure 7: Intensity Histogram

In Figure 7, we can see clearly that the

maximum image intensity is around 200,

and minimum image intensity is around 15.

However, even with the different

background lighting, there is one thing we

know for sure: the white lines always have

the maximum intensity. Therefore, if we can

extract the intensity range from 180 to 200,

we can detect the white lines of the image.

Figure 8: Intensity Analysis

In Figure 8 it can be seen that the highest

value found in the histogram is being passes

on to the next part of the program. Also at

the bottom of the figure there is a user

control called Interval that contains the

range of intensities accepted as white. This

is where the adaptive threshold will receive

its maximum and intensity range from.

D. Adaptive Thresholding

Actually, thresholding is the simplest most

powerful method for image segmentation.

Mathematically, thresholding can be

described as [1]:

, where f(x,y) is the input image, g(x,y) is

the threshold output image and T the

threshold. Generally, thresholding uses the

fixed value of T to segment the images. In

this case we will use a variable threshold

value, which will be adjusted according to

the background lighting as discussed in the

previous section. Since we already know the

maximum intensity of the image from the

intensity analysis, then we will calculate the

variable threshold as follows:

Figure 9: Adaptive Thresholding

Figure 10: Thresholded Image/ Interval = 20

E. Hough Transformation

Once we get the edge pixels after the

adaptive thresholding we need to link them

together to form a meaningful line. To

accomplish this task a Hough

Transformation will be used to bridge any

gaps in the line that may appear. This will

give us the position and direction of the line

in the field of view [1] [2].

In Hough Transformations, each pixel (x,

y) is transformed from Cartesian Space to

Hough Space, H (R,θ), as follows:

, where 0 < R < and

. If two pixels (x1, y1) and (x2, y2) are

co-linear, we will get the same value for R

and θ. In other words, a line in Cartesian

Space is represented as a point in Hough

Space. A simple Hough Transformation can

be achieved by using a two-dimensional

accumulator (array), which corresponds to R

and θ. Each cell of the accumulator is

defined by unique R and θ values, and the

value inside each cell is increased according

to the number of co-linear points in

Cartesian space. However, for practical

purposes, this algorithm is too slow for real-

time image processing. Therefore, we must

use the Matlab ‘hough function’ for our line-

detection [2].

Once the accumulator is filled, we look for

the maximum cell value stored in the

accumulator and its related R and θ. The

resulting R and θ represent the line we are

interested.

Figure 11: Hough Space (R, θ)

Once we get the R and θ, we need to shift

them as follows:

The most efficient way to implement these

equations is to use a formula node.

Once the line is generated in the Hough

Transformation the line values are sent to a

line detection algorithm to determine how to

properly handle the possibility of crossing

the line.

F. Line Detection Algorithm

Once we get the values of x, y, x1 and y1,

we use the following line detection

algorithm to calculate the line angle. This

will be used to determine whether the robot

needs to turn, along with what direction and

intensity the turn should be made.

If (x > 0) AND

(y > 0)

If ( α < 30) AND

(α > 0)

Yes

Go Straight Turn Right

No

Yes

If ( α < 30) AND

(α > 0)

Go Straight Turn Left

No

Yes

No

Right = x1 > 160

Left = Not (Right)

Figure 12: Turning Algorithm

Right will tell us if the detected line is

located on the left side of the camera and

Left will tell us if the detected line is located

on the right side of the camera. These are

decided by what x coordinate pixel the

nearest line is on at the bottom row of the

image.

IV. Results and Performance Analysis

In order to test the reliability and

performance of our algorithm, we carried

out a series of tests with different scenarios.

The results were captured and are shown

below; for each set of conditions there is a

picture of what the camera sees followed by

what the program interprets as the proper

avoidance maneuver along with where the

nearest line is.

Test 1: Simple Right

Test 2: Right w/ Obstacle

Test 3: Parallel

Test 4: Simple Left

Test 5: Left w/ Obstacle

According to the test results, we found that

our new algorithm can give more accurate

and reliable results than our old algorithm.

In addition, since we do not use NI Vision

Development, and wrote the whole project

with the intrinsic LabVIEW functions, we

also solved the problems related to software

expiration. One problem that still needs to

be dealt with is if no line is present the

adaptive threshold will give the line

detection of the largest set of intensities.

This has to be solved before the system can

be declared a fully functional obstacle

avoidance utility.

V. Further Improvements

Although our algorithm is satisfactory to

some extent, there is a lot to be improved

upon that would require more time and a

budget plan for additional equipment. First

of all, we use monocular vision system, to

detect the lines. By adding a second camera

the system could be reprogrammed to have

one camera handle the left and the other

dedicated to the right line. This would allow

greater control due to the visual field being

doubled.

For the project to be a feasible substitute

for the current system the algorithm will

need the ability to distinguish whether or not

a line is even present. Without this ability

the system would need the guarantee that

either the left or right line would be in the

field of view. For our purposes this is not an

acceptable loss, but the improvements made

are still enough to show definite

improvement over the previous prototype.

VI. References 1. Davies, E.R. Machine Vision. 2nd ed. San Diego Academic

Press., 1997. 80-269. 2. González, Rafael C.; Woods Richard Eugene; and Eddins

Steven L. Digital Image Processing with MATLAB. Pearson

Prentice Hall., 2004.380-406. 3. Jahne Bernd. Digital Image Processing. 6th ed. Heidelberg:

Springer-Verlag., 2005.331-340.

4. "NI Vision Acquisition Software." National Instrument. 30 Nov 2009. http://sine.ni.com/psp/app/doc/p/id/psp-394 >

5. Painter, James G. Vision System for Wunderbot IV Autonomous

Robot. Elizabethtown College, 9 May 2008. 6. Aung, Than L. & Conlow Michael. Alternative Vision System

for Wunderbot V. Elizabethtown College, 9 Dec 2009.