This article was downloaded by: [University of Delaware]On: 15 April 2013, At: 03:35Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Archives Of Phytopathology And PlantProtectionPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/gapp20

Performance evaluation of a machinevision system for insect pestsidentification of field crops usingartificial neural networksKeyvan Asefpour Vakilian a & Jafar Massah aa Department of Agrotechnology, University of Tehran, Tehran,IranVersion of record first published: 25 Feb 2013.

To cite this article: Keyvan Asefpour Vakilian & Jafar Massah (2013): Performance evaluation of amachine vision system for insect pests identification of field crops using artificial neural networks,Archives Of Phytopathology And Plant Protection, DOI:10.1080/03235408.2013.763620

To link to this article: http://dx.doi.org/10.1080/03235408.2013.763620

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representationthat the contents will be complete or accurate or up to date. The accuracy of anyinstructions, formulae, and drug doses should be independently verified with primarysources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand, or costs or damages whatsoever or howsoever caused arising directly orindirectly in connection with or arising out of the use of this material.

Performance evaluation of a machine vision system for insect pestsidentification of field crops using artificial neural networks

Keyvan Asefpour Vakilian* and Jafar Massah

Department of Agrotechnology, University of Tehran, Tehran, Iran

(Received 24 December 2012; final version received 29 December 2012)

Feature extraction is a crucial part of advanced image recognition systems. In thisresearch, an autonomous detection device was designed and developed for insectpest detection to improve the ability of intelligent systems in order to annihilateharmful insect pests in agricultural crop fields. Device included a dark chamber, aCCD digital camera, a LDR lightening module and a personal computer. The pro-posed programme for precise insect pest detection was based on an image processingalgorithm and artificial neural networks (ANNs). After image acquisition, the insectpests’ images were extracted from original images with Canny filtration. Afterwards,four morphological and three textural features from the obtained images were mea-sured and normalised. Performance of ANN model was tested successfully for Beetarmyworm (Spodoptera exigua) recognition in images using back-propagation super-vised learning method and inspection data. Results showed that proposed systemwas able to identify S. exigua in the images from other species. Such this machinevision system can be used in autonomous field robots to achieve a modern farmer’sassistant.

Keywords: image processing; insect pests; feature extraction; artificial neuralnetworks; morphological features; textural features

1. Introduction

In recent years, image processing has become one of the important techniques in micro-precision agriculture. For the real-time crop monitoring, high-quality machine visionsystems developed non-contact sensing for agricultural products (Huang et al. 2010).For example, they can identify emerging stresses and guide sampling for identificationof the related stressor (Ushada et al. 2007). Also, they can indicate the type of insectpests in the greenhouses. The real-time accurate identification of insect pests of fieldcrops is an inevitable trend of modern plant protection (Horne and Page 2008).

It is also an important issue that must be carefully studied and resolved for today’sprecision agriculture (Watson et al. 2003). In the recognition of insect pests, there arethree important methods: artificial recognition, sound recognition and image recognition(Zhu et al. 2010). El-Helly et al. (2003) developed an integrated image processing systemto automate the inspection of leaf batches to identify the disease type. They showed thatthe leaf spots can be considered as an indicator of crop diseases. Boissard et al. (2008)described a software prototype system for in situ early whitefly pest detection in theimages of different infected leaves. Their proposed system was composed of extensible

*Corresponding author. Email: keyvan.asefpour@ut.ac.ir

Archives of Phytopathology and Plant Protection, 2013http://dx.doi.org/10.1080/03235408.2013.763620

� 2013 Taylor & Francis

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

knowledge-based systems for image concepts (colour distribution, topology and imagetextural features) and natural object recognition, coupled with image processingprogrammes and machine learning.

In a research, Bechar and Moisan (2010) utilised an autonomous system for onlinecounting of some harmful insects in videos in the framework of in situ videosurveillance that aims at the early detection of pest attacks in greenhouse crops with anacceptable performance. Hongtao et al. (2011) used an image of morphological featuresextraction and a fuzzy classifier system in a real-time detection device for identifyingnine species of insect pests. The correct recognition ratio was over 86%. They alsoshowed that the system was practical and feasible.

Han et al. (2012) presented a feasibility study on a real-time in field pest classifica-tion system design based on a remote online classification platform. The authenticationtest showed that the image data were transmitted correctly to the image processing unit.Based on the performance test results on six classes of pests, the average accuracy was82%. Pokharkar and Thool (2012) used an image segmentation method with Gaussianfilter and Sobel operator to extract infected parts of particular plants’ images. AsefpourVakilian and Massah (2012a) designed an automatic robot with real-time imageprocessing system to detect nitrogen deficiency in greenhouse cucumber crops. Imagetextural features were extracted for calculating three parameters: entropy, energy andhomogeneity.

Modern Artificial Neural Networks (ANNs) are non-linear statistical data modellingtools. They are usually used to model complex relationships between inputs and outputsor to find patterns in data. ANNs have the largest body of applications in agriculturaland biological engineering when compared with other soft computing techniques (Huanget al. 2010). ANN models have the privilege of learning and simulating elaborate appli-cations. Various researches have demonstrated that ANN models give a more accurateestimation in comparison with traditional statistical models (Ushada et al. 2007).

Beet armyworm (Spodoptera exigua) is one of the best-known agricultural pestinsects. It is native to Asia, but has been introduced worldwide and is now foundalmost anywhere its many host crops are grown. It has a wide host range, occurring asa serious pest of vegetable, field and flower crops (Mehrkhou et al. 2012).

The objectives of the current study were (1) to develop a machine vision system forautomated identifying Beet armyworm (S. exigua) insect pest in images using imagemorphological and textural features and (2) to improve the ability of proposed systemwith utilising an ANN model.

2. Materials and methods

In 2010, a population of S. exigua was originally collected from a sugar beet field inShiraz, Iran, (29° 36′N, 52° 32′E). The collected larvae were reared on soybean culti-vars until mature stage. Then, 100 emerged adults were transferred to a dark chamberfor image acquisition.

2.1. Image acquisition system

The elements of image acquisition system included four major units: quite dark cham-ber, a CCD digital camera (Canon, Powershot, G12, Japan), a LDR lightening moduleand a personal computer. A 200-LEDs LDR array with view angle of 70° was usedabove the camera to increase the light uniformity for the region of interest. Distance

2 K. Asefpour Vakilian and J. Massah

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

between camera and the samples was set as a 20 cm (Figure 1). Two sequential imageswere taken with camera from each pest samples and these images were transferred tothe computer and then, image averaging was used for analysis to reduce the effect ofrandom electronic noise and to reduce disturbances by factors that would cause thesamples to move.

The captured image dimension was 1600� 1200 pixels and was analysed as a rawbitmap image. In addition, 100 random images from four well-known species of insectpests (Mythimna separata, Agrotis ipsilon, Helicoverpa armigera and Melolonthamelolontha) were added to these 100 images for evaluating proposed system in pestidentification. The programme for the insect pest monitoring system was written withMATLAB version 7.11 software (MATLAB 2010) using Image Processing Toolbox.

2.2. Preprocessing of the insect pests’ images

From each retrieved original colour image (Figure 2(a)), the region of interest (theinsect pest) was extracted from its background through an image segmentation processcalled Canny edge detection (Figure 2(b)). The Canny edge detector is a susceptiblemethod for edge detection in raw unprocessed image data using Gaussian filtration (bellcurve method). The result is a slightly blurred version of the original which is notaffected by a single noisy pixel to any significant degree (Canny 1986). For eachobtained image, two copies were saved in H.D.D. of a computer. One of them wasconverted to grey-scale image in Red–Green–Blue (RGB) colour system. RGB valuesof each pixel were converted to grey-scale values by forming a weighted sum of theR, G and B components (Figure 2(c)).

Then, the histogram of each grey-scale image was drawn using MATLAB version7.11 software (MATLAB 2010) (Figure 2(d)). As shown in this figure, grey-scaleimages of insect pests did not have sufficient contrast and did not fill the entire intensityrange [0, 255] of the histogram. Therefore, a complementary contrast adjustment blockwas utilised to adjust the contrast of an image by linearly scaling the pixel values

Figure 1. Image acquisition system for insect pests.

Archives of Phytopathology and Plant Protection 3

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

between upper and lower limits. After image contrast adjustment, a global thresholdvalue was computed to convert a grey-scale image to a binary image. For this purpose,the Otsu’s method was utilised to choose the best threshold value for minimising theintraclass variance of the black and white pixels (Otsu 1979) (Figure 2(e)).

2.3. Morphological features extraction from the images

In digital image recognition of the insect pests, a set of morphological features wasextracted from obtained binary image. The system mainly measured four morphologicaland structural characteristics: area, perimeter, sphericity and eccentricity. Each of thesemorphological features was defined as follows:

(1) Area: total number of pixels shared by the object in the image.(2) Perimeter: length of object edge in the image.(3) Sphericity: ratio of radiuses of inscribed circle to circumscribed circle of the

object in the image.(4) Eccentricity: ratio of the lengths of the short axis to the long axis of inertia

ellipse; it described the compactness degree of the object in the image.

Figure 2. The image of Beet armyworm (Spodoptera exigua), (a) the original image, (b) theextracted image of S. exigua, (c) the grey-scale image of S. exigua, (d) the histogram ofgrey-scale image and (e) the binary image of S. exigua.

4 K. Asefpour Vakilian and J. Massah

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

In order to eliminate the influence of dimension and magnitude, and make them tobe comparable, the raw features were normalised before they were taken to the ANNmodel (Zhang and Mao 2008).

2.4. Textural features extraction from the images

One of the images copied in the H.D.D. was used to calculate the colour features of theplant. Grey-level co-occurance matrix was used to capture the spatial dependence ofgrey-level values for different angles of pixel relativity (0°, 45°, 90° and 135°) (Jainet al. 1995). Each matrix was run through probability–density functions to calculatedifferent textural parameters. After analysing the colour features of the focused image,the textural features were extracted.

In one review, 21 textural parameters were identified (Zheng et al. 2006). However,another report indicated that only three textural parameters were useful in identifyingplant health – entropy, energy and local homogeneity (Ushada et al. 2007; Story et al.2010; Asefpour Vakilian and Massah 2012b). In this research, three textural parameterswere extracted in identifying the insect type – entropy, energy and local homogeneity(Haralick et al. 1973; Ushada et al. 2007). Therefore, from each image of insect pests,seven image features (four morphological and three textural features) were extractedand considered as a data set.

2.5. ANNs model

From 200 data sets obtained from images of insect pests, 150 data sets were used fortraining ANN model, while the remains were used as inspection data. The network wastrained with Levenberg–Marquardt back-propagation algorithm. Therefore, the ANNarchitecture was considered as a 7-20-1 network which represents seven nodes in theinput layer, 20 nodes in the hidden layer was used for data analysis. Data sets of 20pest images were used for cross validation and remained 30 data sets were used forevaluating the presented ANN model in pest identification. The ANN model was ableto identify that the test data set belongs to the image of S. exigua or not (see Figure 3).

3. Results and discussion

Table 1 indicates an example of the training data. In order to choose the parameters(weights), the network was trained by training set. The process of cross validation isused to monitor capability of the neural network to build generalised outputs. Lastly,testing data was used to validate the quality of proposed ANN model. Stop criteria andweight reset were used in order to cope with under-fitting/over-fitting problems.

Figure 3. ANN architecture.

Archives of Phytopathology and Plant Protection 5

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

Table 1. Examples of the training data set.

InputsOutput

Area(pixels)

Perimeter(pixels) Sphericity Eccentricity Entropy Energy Homogeneity

Pestidentification

21,300 7430 0.167 0.234 456,740 355,600 675,540 This pest isS. exigua

27,340 7895 0.178 0.190 567,325 356,530 544,570 This pest isS. exigua

25,450 8430 0.434 0.341 800,130 245,550 454,575 This pest isnot S. exigua

40,770 6180 0.129 0.130 567,490 454,095 865,565 This pest isnot S. exigua

Figure 4. Regression results for proposed ANN model (a) Training, (b) Validation and (c)Testing data. 0 and 1 values in horizontal and vertical coordinates represent “S. exigua” and ‘‘notS. exigua’’, respectively.

6 K. Asefpour Vakilian and J. Massah

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

Comparison of training and inspection data was made in Figure 4 to show thecloseness between the predicted and measured value. The three axes represent training,validation and testing data. The dashed line in each axis represents the perfectresult� predicted values = observed values. The solid line represents the best-fit linearregression line between outputs and inputs. The R value is an indication of the relation-ship between the outputs and inputs. In this study, the training data indicated a good fit.

The validation and test results also showed that the R values are about 0.9. Thistraining stopped when the validation error increased for six iterations, which occurred atiteration 10. Figure 5 shows a plot of the training errors, validation errors and testerrors. By using these link weights from trained ANN, pest identification wasestimated.

4. Conclusion

In this study, a machine vision system was developed for an insect pest monitoringdevice. To improve the ability of the proposed system for identifying S. exigua fromother species, a back-propagation ANN model with three layers was utilised. Resultsshowed that the ability of this system for insect pest recognition was acceptable. Incommercial settings, it is desirable to develop an intelligent platform for real-time insectpest identification. This could be achieved by a multi-sensing systems equipped with anartificial light source and a multi-sensor platform that moves in the crop fields anddetects insect pests on plants’ leaves. In order to increase the ability of intelligentfarmer assistant robots in real-time detection of wide range of insect pest species,further researches need to be done by entomologists and agrotechnology engineers.

ReferencesAsefpour Vakilian K, Massah J. 2012a. Design, development and performance evaluation of a

robot to early detection of nitrogen deficiency in greenhouse cucumber (Cucumis sativus)with machine vision. Int J Agric Res Rev. 2(4):448–454.

Figure 5. Mean square error values of training, validation and testing data.

Archives of Phytopathology and Plant Protection 7

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13

Asefpour Vakilian K, Massah J. 2012b. Performance evaluation of CCD and CMOS cameras inimage textural features extraction. Acta Tech Corviniensis. 5(3):61–64.

Bechar I, Moisan S. 2010. On-line counting pests in a greenhouse using computer vision. Paperpresented at: VAIB 2010. Proceedings of Conference on Visual Observation and Analysis ofAnimal and Insect Behaviour; Sep 8–10, Inria, France.

Boissard P, Martin V, Moisan S. 2008. A cognitive visual approach to early pest detection ingreenhouse crops. Comput Electron Agric. 62:81–93.

Canny J. 1986. A computational approach to edge detection. IEEE Trans Pattern Anal. 8(6):679–698.

El-Helly M, Rafea A, El-Gammal S. 2003. An integrated image processing system for leafdisease detection and diagnosis. Paper presented at: ICAIA 2003. Proceedings of 1st IndianInternational Conference on Artificial Intelligence; Dec 18–20, Hyderabad, India.

Han R, He Y, Liu F. 2012. Feasibility study on a portable field pest classification system designbased on DSP and 3G wireless communication technology. Sensors. 12:3118–3130.

Haralick RM, Shanmugam K, Dinstein I. 1973. Textural features for image classification. IEEETrans Syst, Man Cy. 3(6):610–621.

Hongtao Z, Bo G, Yuxia H. 2011. Design and realization of a real-time detection device for insectpests of field crops. Paper presented at: EEIC 2011. Proceedings of International Conferenceon Electric and Electronics; Jun 20–22; Nanchang, China.

Horne P, Page J. 2008. Integrated pest management for crops and pastures. 1st ed. Melbourne:Landlinks Press.

Huang Y, Lan Y, Thomson SJ, Fang A, Hoffmann WC, Lacey RE. 2010. Development of softcomputing and applications in agricultural and biological engineering. Comput ElectronAgric. 71:107–127.

Jain R, Kasturi R, Schunck BG. 1995. Machine Vision. 1st ed. New York (NY): McGraw-Hill.MATLAB. 2010. MATLAB User’s Guide, version 7.11. USA: Mathworks Ltd.Mehrkhou F, Talebi AA, Maharramipour S, Hosseini Naveh V, Farahani S. 2012. Development

and fecundity of Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae) on different soybeancultivars. Arch Phytopathol Plant Prot. 45(1):90–98.

Otsu N. 1979. A threshold selection method from gray-level histograms. IEEE Trans Syst Man,Cy. 9(1):62–66.

Pokharkar SR, Thool VR. 2012. Early pest identification in greenhouse crops using imageprocessing techniques. Int J Comput Sci Net. 1(3).

Story D, Kacira M, Kubota C, Akoglu A, An L. 2010. Lettuce calcium deficiency detection withmachine vision computed plant features in controlled environments. Comput Electron Agric.74:238–243.

Ushada D, Murase H, Fukuda H. 2007. Non-destructive sensing and its inverse model for canopyparameters using texture analysis and artificial neural network. Comput Electron Agric.57:149–165.

Watson A, O’Neill M, Kitching I. 2003. Automated identification of live moths (Macrolepidop-tera) using digital automated identification system (DAISY). Syst. Biodivers. 1:287–300.

Zhang H, Mao H. 2008. Image recognition and classification of the stored-grain pests based onsupport vector machine. Paper presented at: CITESS 2008. Proceedings of Conference onInformation Technology and Environmental System Sciences; Beijing, China.

Zheng C, Sun DW, Zheng L. 2006. Recent applications of image texture for evaluation of foodqualities-a review. Trends Food Sci Tech. 17:113–128.

Zhu L, Wang H, Zhang Z. 2010. Automatic acoustical identification of insects based on MFCCand VQ. Acta Entomol Sin. 53:901–907.

8 K. Asefpour Vakilian and J. Massah

Dow

nloa

ded

by [

Uni

vers

ity o

f D

elaw

are]

at 0

3:35

15

Apr

il 20

13