OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

8

Click here to load reader

Transcript of OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

Page 1: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

O

Os

Ta

b

a

ARRA

KCEFARG

1

frsrawdiiapciestm

1

(

0d

Computers and Electronics in Agriculture 75 (2011) 84–91

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journa l homepage: www.e lsev ier .com/ locate /compag

riginal paper

ptimization of an artificial neural network topology using coupled responseurface methodology and genetic algorithm for fluidized bed drying

ayyeb Nazghelichia, Mortaza Aghbashloa,b,∗, Mohammad Hossein Kianmehra

Department of Agrotechnology, College of Abouraihan, University of Tehran, Pakdasht, IranAgricultural Engineering and Technology, University of Tehran, Karaj, Iran

r t i c l e i n f o

rticle history:eceived 14 June 2010eceived in revised form 2 September 2010ccepted 24 September 2010

eywords:arrot Cubes

a b s t r a c t

In this study, an integrated response surface methodology (RSM) and genetic algorithm (GA) are rec-ommended for developing artificial neural networks (ANNs) with great chances to be an optimal one. Amulti-layer feed forward (MLFF) ANN was applied to correlate the outputs (energy and exergy) to thefour exogenous inputs (drying time, drying air temperature, carrot cubes size, and bed depth). The RSMwas used to build the relationship between the input parameters and output responses, and used as thefitness function to measure the fitness value of the GA approach. In the relationship building, five vari-

nergy and exergyluidized bed dryingrtificial neural networkesponse surface methodologyenetic algorithm

ables were used (number of neurons, momentum coefficient and step size in the hidden layer, number ofepochs and number of training times). A polynomial model was developed from training results to meansquare error (MSE) of 50 developed ANNs to generate 3D response surfaces and contour plots. Finally,GA was applied to find the optimal topology of ANN. The ANN topology had minimum MSE when thenumber of neurons in the hidden layer, momentum coefficient, step size, number of training epochs andtraining times were 28, 0.66, 0.35, 2877 and 3, respectively. The energy and exergy of carrot cubes during

pred

fluidized bed drying were

. Introduction

Fluidized bed dryers are extensively used in the food industryor drying of moist food products. The fluid bed dryers significantlyeduce drying time compared with the tray dryers due to goodolids mixing, high rates of heat and mass transfer, and easy mate-ial transport (Mujumdar, 2006). In practical applications drying isprocess that requires high energy input because of latent heat ofater evaporation and relatively low energy efficiency of industrialryers. Thus, one of the most important challenges of the drying

ndustry is to reduce the cost of energy sources for good qual-ty dried products (Syahrul et al., 2002). Energy analysis is a basicnd traditional approach to estimating various energy conversionrocesses. The analysis is based on the concept of energy and itsonservation. The forms of energy can be expressed as enthalpy,nternal energy, chemical energy, work, heat, electricity, etc. The

nergy analysis is not able to recognize different quality of energyuch as the heat quality which is dependent on the heat sourceemperature. For these reasons exergy analysis which provides a

ore realistic view of the process is used to the process analy-

∗ Corresponding author at: Aboureyhan Campus, University of Tehran, P.O. Box1365-4117, Tehran, Iran. Tel.: +98 292 3020345–9; fax: +98 292 3020340.

E-mail addresses: [email protected], [email protected]. Aghbashlo).

168-1699/$ – see front matter © 2010 Elsevier B.V. All rights reserved.oi:10.1016/j.compag.2010.09.014

icted with R2 values of greater than 0.97 using optimal ANN topology.© 2010 Elsevier B.V. All rights reserved.

sis. The exergy analysis is, based on the modern thermodynamicmethod, used as an advanced tool for engineering process evalua-tion (Szargut et al., 1988).

Recently, several studies have been undertaken on energy andexergy analyses of food and agricultural products drying process(Midilli and Kucuk, 2003; Akpinar et al., 2006; Aghbashlo et al.,2009; Ozgener and Ozgener, 2009; Hepbasli et al., 2010).

One of the most important stages of drying technology is mod-eling and optimization of the process to increase the efficiency ofthe drying facilities. In drying processes, complex and highly non-linear phenomena are involved (Omid et al., 2009). Therefore, it isdifficult to quantify the complex relationships between the inputand the output of a drying system based on analytical methods.

Artificial neural networks (ANNs) have high learning ability andcapability of identifying and modeling the complex non-linear rela-tionships between the input and the output of a system (Hashimoto,1997). Due to the ability of ANNs system in modeling complexnonlinear systems, successful applications of this method in dry-ing technology have been extensively reported (Farkas et al., 2000;Erenturk et al., 2004; Movagharnejad and Nikzad, 2007; Erenturkand Erenturk, 2007; Youssefi et al., 2009).

The trail and error procedure to obtain the optimum topologyof ANN is a very daunting, time-consuming and tedious task. Inorder to find optimal ANN topology, a large number of differentconfigurations of ANNs have to be constructed, trained and tested.Thus, it is vital to researchers to find a technique to solve these

Page 2: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

Electronics in Agriculture 75 (2011) 84–91 85

pnufoagamAl

tbogoaat

i

2

2

kmstb

2

sATdprchATwwDrimi

2

a7swtitc

T. Nazghelichi et al. / Computers and

roblems. Recently, Madadlou et al. (2009) optimized the artificialeural network (ANN) topology of re-assembled casein micellessing response surface methodology (RSM). The RSM is a power-ul optimum design tool in many engineering applications. It notnly can save a lot of time but also can build models quickly andccurately in an optimization design. It is very difficult to find thelobal optimal point using RSM when the response is complex, suchs consisting of many independent process variables or involvingultiple local optimal points. There are many parameters affectingNN performance and finding the optimum topology of ANN using

one RSM is very difficult.Genetic algorithm (GA) is a search algorithm that emulate

he adaptive processes of natural biological systems. The GA haseen applied in many complex optimization and search problems,utperforming traditional optimization and search methods. Inte-ration of RSM and GA seems to be good choice for finding the globalptimum topology of ANNs. It is noteworthy that the coupled RSMnd GA has many benefits for finding optimal ANN topology, suchs improving the model performance, choosing the global optimumopology and accelerating the model development.

This work carried out to ascertain the possibility of supplement-ng RSM with GA for the determination of optimal ANN topology.

. Materials and methods

.1. Sample

Fresh carrots (Daucus carota L.) were obtained from a local mar-et in Iran and stored in a refrigerator at about 5 ◦C. The initialoisture content of the carrot was determined by drying of 50 g of

ample in an oven at 105 ± 2 ◦C. Experiments were replicated threeimes. The initial moisture content of the carrots was observed toe 89.3 ± 0.36% (kg water/kg dry matter).

.2. Drying equipment

The drying of carrot cubes was investigated in a laboratorycale fluidized bed dryer (FBD) developed in the “Department ofgrotechnology Laboratory” of Abouraihan Campus, University ofehran, Iran (Fig. 1). The Plexiglas cylindrical chamber was 15 cm iniameter and 30 cm in height. A centrifugal fan provided an air sup-ly, drawn from ambient air. In order to supply the required airflowate for fluidization of carrot cubes, the fan speed was changed byontrolling the speed of the motor which operates the fan. The dryerad an automatic temperature controller with an accuracy of ±1 ◦C.ir velocity was measured using an anemometer (PROVA AVM-07ES, Co., Taipei, Taiwan) with an accuracy of ±0.05 m/s. Weighingas made at every 5 min using a digital balance (Mahak, Co., Iran)ith an accuracy of ±0.1 g for moisture content determination.uring the experiments, drying air velocity, ambient temperature,

elative humidity, and inlet and outlet temperatures of drying airn dryer chamber were recorded. Detail specifications of the dryer,

easurement instruments, recording and processing are availablen (Nazghelichi et al., 2010).

.3. Experimental procedure

At the start of each experiment carrots were washed, peeled,nd cut into cubes having dimensions of 4 mm × 4 mm × 4 mm,mm × 7 mm × 7 mm and 10 mm × 10 mm × 10 mm. Before the

tart of each experiment, the cylindrical drying chamber was filled

ith prepared samples to a chosen height, and after each run

he samples were taken out from the cylinder to determine thenitial weight. After the dryer was reached steady state condi-ions for operation temperature and fluidization velocity, the carrotubes were put in the drying chamber and were dried there.

Fig. 1. Schematic view of the experimental equipment: fan (1); inverter (2); heaterscontrol unit (3); heaters (4); chamber (5); thermocouple (6–8); fluidization cylin-drical chamber (7); and anemometer (9).

Experiments were conducted at three inlet air temperatures of50, 60 and 70 ◦C, initial bed heights of 3, 6 and 9 cm, and carrotcube dimensions of 4 mm × 4 mm × 4 mm, 7 mm × 7 mm × 7 mmand 10 mm × 10 mm × 10 mm. Each set of conditions was testedthree times.

3. Theoretical principle

3.1. Energy analysis

The energy utilization (EU), calculated by applying the first lawof thermodynamics, can be expressed as (Akpinar, 2004):

EU = ma(hai − hao) (1)

where EU is the energy utilization (kJ/s), ma is the mass flow rateof drying air (kg/s), hai is the specific enthalpy of inlet air (kJ/kg dryair) and hao is the specific enthalpy of outlet air.

The mass flow rate of the air (ma) was calculated using Eq. (2)(Aghbashlo et al., 2009):

ma = �aVaAdc (2)

where �a is the density of air (kg/m3), Va is the air velocity (m/s)and Adc is the surface area of drying chamber (m2).

Meanwhile, it was considered that the mass flow rate of dryingair was equally passed throughout the whole cross section of thedrying chamber.

mai = mao = ma (3)

The enthalpy of drying air can be determined as follows (Akpinaret al., 2006):

hai = Cpai(Tai − Tref ) + hfgwai (4)

hao = Cpao(Tao − Tref ) + hfgwao (5)

where Cpai is the specific heat of inlet air (kJ/kg ◦C), Tai is the tem-perature of inlet air (◦C), Tref is the temperature of ambient air, hfgis the latent heat of vaporization of water (kJ/kg dry air), wai is thehumidity ratio of inlet air (kg water/kg drying air), Cpao is the spe-cific heat of outlet air (kJ/kg ◦C), Tao is the temperature of outlet air

Page 3: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

8 Electronics in Agriculture 75 (2011) 84–91

(a

l

C

C

ptd

w

wda

w

w

m

www

l

E

h

3

tte

E

E

f

E

ua

6 T. Nazghelichi et al. / Computers and

◦C) and wao is the humidity ratio of outlet air (kg water/kg dryingir).

The specific heat of inlet and outlet air is calculated using fol-owing equations (Aghbashlo et al., 2009):

pai = 1.004 + 1.88wai (6)

pao = 1.004 + 1.88wao (7)

During the energy and exergy analyses of carrot cubes dryingrocess, the following equation was generally used to transformhe relative humidity to humidity ratio of the inlet air (kg water/kgry air) (Aghbashlo et al., 2009):

ai = 0.622ϕPvs

P − Pvs(8)

here w is the humidity ratio of inlet and outlet air (kg water/kgrying air), ϕ is the relative humidity of inlet and outlet air, P is thetmospheric pressure (kPa) and Pvs is the saturated pressure (kPa).

Humidity ratio of outlet air was calculated by (Akpinar, 2004):

ao = wai + mv

ma(9)

here mv is the mass transfer rate (kg water/s)The (mv) is obtained by Eq. (9):

˙ v = −Wt+�t − Wt

�t(10)

here Wt+�t is the weight of product at time t + �t (kg), Wt is theeight of product at time t and �t is the time between two sampleeighing (s).

The energy utilization ratio of drying chamber (EUR) was calcu-ated using the following equation (Akpinar, 2004):

UR = ma

ma.(hai − hao)(hai − hae)

(11)

ae is the specific enthalpy of environment.

.2. Exergy analysis

The exergy values were calculated using the characteristics ofhe working medium from the first law of energy balance. Forhis purpose, the general form of applicable exergy equation ismployed for steady flow systems (Midilli and Kucuk, 2003).

xai = maCpai

[(Tai − Tref ) − Tref ln

Tai

Tref

](12)

[ ]

xao = maCpao (Tao − Tref ) − Tref ln

Tao

Tref(13)

The exergy loss (kJ/s) in drying chamber can be determined asollows (Akpinar, 2004):

xloss = Exai − Exao (14)

The exergy efficiency (�ex) can be defined as the ratio of exergyse (investment) in the drying of the product to exergy of the dryingir supplied to the system (Akpinar, 2004):

ex = 1 − Exloss

Exai(15)

Fig. 2. Schematic model of artificial neural network for prediction energy and exergyduring fluidized bed drying of carrot cubes.

In this study, the ambient temperature (Tref) and environment airrelative humidity (ϕ) were taken as 25 ◦C and 20%, respectively.

4. Search method

4.1. Artificial neural network development

In this study, 518 data sets obtained from experiments wererandomly used for training (60%), verification (20%), and testing(20%) the neural networks. Fig. 2 illustrates the structure of amulti-layered feed-forward (MLFF) back-propagation ANN modelfor predicting the energy and exergy of carrot cubes during flu-idized bed drying. The input variables for the model were dryingtime, drying air temperature, air velocity, and cubes size and thenetwork output variables including energy utilization, energy uti-lization ratio, exergy loss and exergy efficiency.

To train the networks, GDM algorithm was selected for avoidinglocal minima, speeding up learning and stabilizing convergence.Since the four dependent variables (the energy utilization, energyutilization ratio, exergy loss and exergy efficiency of fluidized beddrying of carrot cubes) were dependent on four exogenous inputs(the drying time, drying air temperature, cubes size and bed depth),therefore four neurons were chosen for the output and the inputlayers.

The MLFF ANN with different hidden layer(s) (1–5) and differenttransfer functions including hyperbolic tangent sigmoid (tansigm),logarithmic sigmoid (logsigm), linear hyperbolic tangent sigmoid,and linear logarithmic sigmoid transfer functions were trained andtested. The variation of training MSE for different configuration ofhidden layer, transfer functions and neuron number is shown inTable 1. It is clear that the one hidden layer ANN with hyperbolictangent sigmoid transfer function is the best selection due to theresults of training errors for the four outputs.

Therefore, one hidden layer MLFF ANN with hyperbolic tangentsigmoid transfer function was selected for optimization process.The mathematical definition of hyperbolic tangent sigmoid wasexpressed in (Eq. (16)).

tansigm = 21 + exp(−2n)

− 1 (16)

The chosen numerical variables for optimization were the num-ber of neurons in the hidden layer (2–40), momentum coefficient(0.1–0.7) and step size (0.1–0.40) in the hidden layer, epoch num-ber (100–3000), and training times (1–5) and the responses sought

were error on the cross-validation data set. The goodness of fit ofthe optimal ANN to the experimental data was based on coefficientof determination, R2, mean square error (MSE) and mean absoluteerror (MAE) for the tested models. These statistical parameters are
Page 4: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

T. Nazghelichi et al. / Computers and Electronics in Agriculture 75 (2011) 84–91 87

Table 1Variation of training MSE for different configuration of hidden layer, neuron number and transfer functions using static ANN (epoch number = 1550; step size 0.25; momentumcoefficient = 0.4 and training run = 3).

Transferfunction

Neuronnumber

Number ofhiddenlayer(s)

MSE on crossvalidation

Transferfunction

Neuronnumber

Number ofhiddenlayer(s)

MSE on crossvalidation

Transferfunction

Neuronnumber

Number ofhiddenlayer(s)

MSE on crossvalidation

tan sigm 2 1 0.001996 tan sigm 21 1 0.001424 tan sigm 40 1 0.001569log sigm 2 1 0.078418 log sigm 21 1 0.072340 log sigm 40 1 0.031482linear log sigm 2 1 0.012915 linear log sigm 21 1 0.010935 linear log sigm 40 1 0.009240linear tan sigm 2 1 0.096194 linear tan sigm 21 1 0.072771 linear tan sigm 40 1 0.052380tan sigm 2 3 0.051430 tan sigm 21 3 0.050186 tan sigm 40 3 0.036350log sigm 2 3 0.083061 log sigm 21 3 0.080790 log sigm 40 3 0.071703linear log sigm 2 3 0.037102 linear log sigm 21 3 0.034286 linear log sigm 40 3 0.029195linear tan sigm 2 3 0.097688 linear tan sigm 21 3 0.076215 linear tan sigm 40 3 0.063102tan sigm 2 5 0.071236 tan sigm 21 5 0.068310 tan sigm 40 5 0.061022log sigm 2 5 0.091022 log sigm 21 5 0.082652 log sigm 40 5 0.080953

55

a

R

M

M

wte

t

4

ctes

tpwti5tMdt

TT

linear log sigm 2 5 0.043102 linear log sigm 21linear tan sigm 2 5 0.098705 linear tan sigm 21

s follows:

2 = 1 −∑N

i=1

(xpi − xdi

)2

∑Ni=1

(xpi − x

)2(17)

SE = 1N

N∑i=1

(xpi − xdi

)2(18)

AE = 1N

N∑i=1

∣∣xpi − xdi

∣∣ (19)

here xpi is the network (predicted) output from observation i, xdi ishe experimental output from observation i, x is the average value ofxperimental output, and N is the total number of data observation.

NeuroSolutions 5.0 software was used for the designing andesting of ANN models.

.2. Response surface methodology

A face-centered full central composite design of five numeri-al factors (number of neurons, step size, momentum coefficients,raining epochs and number of training times) with three levels andight repetitions at the center point was used in this study. Table 2hows the effective parameters and simulation ranks.

First step in RSM is to find a suitable approximation for therue functional relationship between response and the set of inde-endent variables (Montgomery, 2001). The response variablesere transformed to natural log function for making the distribu-

ion of the response variable closer to the normal distribution andmproving the fit of the model to the data using Box Cox plot. The0 different configurations of proposed ANNs were designed and

rained. The Design Expert version 7.0.0 software (Stat-Ease, Inc.,

inneapolis, MN) was used to model the MSE on cross validationata set of proposed ANNs and to obtain the response surfaces andhe contours.

able 2he range of parameters.

Parameter Down Up

X1: Number of neurons 2 40X2: Step size 0.1 0.4X3: Momentum coefficient 0.1 0.7X4: Training epoch 100 3000X5: Number of training runs 1 5

0.040891 linear log sigm 40 5 0.0542930.083102 linear tan sigm 40 5 0.071022

4.3. Genetic algorithms

In this study, an efficient optimization method by combiningRSM and GA is introduced to find optimal topology of ANN forpredicting the energy and exergy during fluidized bed drying ofcarrot cubes. A predictive model for ANN topology in terms ofMSE on cross validation data set was created using RSM. The RSMmodel was integrated with an effective GA to find the optimumtopology of ANN. The roulette wheel method was used to the chro-mosome selection. The crossover operator was applied to createa pair of offspring chromosomes. For each selected pair, a twocut-point crossover operation was used to generate an offspringwith the crossover probability. The one-gene mutation operationwith a preset mutation probability was applied to generate newchromosomes. The MATLAB 7.0 software was used to develop theintegrated GA and RSM approach and to determine the optimalANN topology.

5. Results and discussion

Table 3 shows the results of generated MSE on cross validationdata set for the 50 designed runs in central composite design forfitting a second order model.

The cubic model was selected to express the relationshipbetween the ANN effective factors and the response of ln(MSE). Thismodel obtained after checking the five processes: orders of mean,the linear polynomial, the linear polynomial with two factor inter-action and quadratic model. The cubic model was selected becauseof non-significant lack of fit and high amount of R2. To obtain thesimplified model, the selected cubic model was reduced based onlow P-value and high mean squares values.

Table 4 illustrates the ANOVA for the response surface ofreduced cubic model. The ANOVA contains sum of squares, degreeof freedom, mean squares, F-value and P-value probability for themodel, its terms and lack of fit.

The training epoch had highest effect on ln(MSE) response andthen followed by step size, momentum coefficient, neuron num-ber and number of training times, respectively. Based on analysisof Design expert software (Table 4), the F-value of model wasvery high (=150.47) indicating the model is significant. Such F-value can occur due to noise with 0.01% chance. It is obviousfrom Table 4, the F-value of lack of fit is not significant and so, it

is acceptable. The prediction R2 of 0.975 is in reasonable agree-ment with the adjusted R2 of 0.987. A ratio greater than 4 foradequacy precision is desirable. The ratio of 39.39 indicated an ade-quate signal. Thus, this model can be used to navigate the designspace. Considering the maximum values of PRESS and C.V.% of 1.85
Page 5: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

88 T. Nazghelichi et al. / Computers and Electronics in Agriculture 75 (2011) 84–91

Table 3Obtained value of MSE for different topology of designed ANN by RSM.

ANN topology Neuron number Step size Momentum coefficient Training epoch Number of training run MSE on cross validation

1 21 0.25 0.4 1550 3 0.0013242 2 0.1 0.1 100 5 0.0167813 21 0.1 0.4 1550 3 0.0041894 40 0.1 0.1 3000 1 0.003055 21 0.25 0.1 1550 3 0.0022986 40 0.4 0.1 3000 1 0.0006547 21 0.25 0.4 1550 3 0.0011528 2 0.25 0.4 1550 3 0.0019969 40 0.1 0.7 100 5 0.011274

10 2 0.1 0.7 100 5 0.01252811 21 0.25 0.4 1550 1 0.00120712 21 0.25 0.4 1550 3 0.00144713 2 0.4 0.7 100 5 0.01016414 21 0.25 0.4 1550 3 0.00172115 2 0.1 0.7 3000 1 0.00197916 40 0.1 0.1 100 5 0.01244217 40 0.1 0.7 3000 1 0.00111618 2 0.4 0.7 3000 5 0.00262519 2 0.1 0.1 3000 1 0.0020320 2 0.1 0.7 3000 5 0.00194421 40 0.4 0.7 100 5 0.00751222 21 0.25 0.7 1550 3 0.00079423 21 0.25 0.4 1550 5 0.00132924 40 0.4 0.1 100 1 0.01201825 40 0.25 0.4 1550 3 0.00156926 40 0.1 0.1 3000 5 0.00400527 40 0.4 0.1 3000 5 0.00067928 2 0.4 0.7 3000 1 0.00262629 40 0.4 0.7 3000 5 0.00054830 40 0.1 0.7 3000 5 0.00087631 2 0.4 0.1 100 1 0.01247332 40 0.4 0.7 3000 1 0.00058133 2 0.4 0.7 100 1 0.01022634 40 0.4 0.1 100 5 0.01113735 2 0.1 0.7 100 1 0.01400536 2 0.1 0.1 3000 5 0.00278837 2 0.4 0.1 3000 5 0.0020638 40 0.1 0.1 100 1 0.01277839 21 0.25 0.4 100 3 0.01048940 21 0.4 0.4 1550 3 0.00082141 2 0.4 0.1 100 5 0.01280142 21 0.25 0.4 1550 3 0.00129243 21 0.25 0.4 1550 3 0.0013844 2 0.1 0.1 100 1 0.01985945 40 0.1 0.7 100 1 0.01069546 21 0.25 0.4 1550 3 0.001427

(v

5 − 0

X3X

5X1X

0.38

wTt(ta

osnl

47 2 0.4 0.148 40 0.4 0.749 21 0.25 0.450 21 0.25 0.4

lower than 10), the final modified cubic model in terms of codedalue is:

ln(MSE) = −6.57 − 0.12X1 − 0.81X2 − 0.53X3 − 1.34X4 − 0.005X

− 0.086 X1 X3 − 0.13X1X4 + 0.092 X2X3 − 0.069X2X4 − 0.025

− 0.027X4X5 + 0.24X21 + 0.29 X2

2 + 0.68X24 − 0.092X2

5 + 0.02

− 0.098X1X3X4 + 0.099X2X3X4 − 0.027X3X4X5 + 0.63X21X2 +

here the parameters of X1, X2, X3, X4 and X5 are defined in Table 2.his model checked hierarchically. In other words, any higher ordererms of the model could appear only in presence of their parentlower order) terms. For example the term X1X2 is a significanterm, then the model should also include the parent terms of X1nd X2 to have the hierarchy model.

The response surfaces and contour plots of the factors werebtained from model graphs tool of the Design Expert version 7.0.0oftware. Fig. 3 shows the 3D surface of ln(MSE) versus the neuronsumber and step size while the other factors remained in their zero

evel, constantly. A descending response surface with two opposite

3000 1 0.00208100 1 0.009695

1550 3 0.0016953000 3 0.000722

.13X1X2

4 − 0.32X3X5

2X3 − 0.17X1X2X4

X21X3 + 0.31X2

1X4 − 0.12X1X22

(20)

curvature at the top and bottom of the face can be seen in Fig. 3. Ascan be seen in the contour plot, with increasing in neurons number

from 20 to 30 and step size from 0.1 to 0.4, the ln(MSE) decreasedto −7.06.

Too few hidden neurons limit the ability of the neural networkto model the process. Too many hidden neurons cause the overfitting and increase the calculation time. It could be related to thefact that too many hidden neurons may allow too much freedom

for the weights to adjust and, thus, to result in learning the noisepresent in the database used in training.

The step size value determines the time needed to find the min-imum in the weight space. Too small step size lead to smaller stepstaken in the weight space, the learning is slow and the network is

Page 6: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

T. Nazghelichi et al. / Computers and Electronics in Agriculture 75 (2011) 84–91 89

Table 4ANOVA for response surface reduced cubic model.

Source Sum of squares DOF Mean square F value P-value Prob > F

Model 58.53537 26 2.25136 150.4665 <0.0001X1 0.028872 1 0.028872 1.929593 0.1781X2 1.32714 1 1.32714 88.69757 <0.0001X3 0.564905 1 0.564905 37.7546 <0.0001X4 3.582279 1 3.582279 239.4166 <0.0001X5 0.000856 1 0.000856 0.057225 0.8131X1X2 0.54015 1 0.54015 36.10016 <0.0001X1X3 0.238352 1 0.238352 15.92992 0.0006X1X4 0.576369 1 0.576369 38.52081 <0.0001X2X3 0.269089 1 0.269089 17.98418 0.0003X2X4 0.150374 1 0.150374 10.05002 0.0043X3X4 0.0207 1 0.0207 1.383427 0.2515X3X5 0.032131 1 0.032131 2.14742 0.1563X4X5 0.02327 1 0.02327 1.555251 0.2249X1

2 0.154342 1 0.154342 10.31522 0.0039X2

2 0.220177 1 0.220177 14.71524 0.0008X4

2 1.225442 1 1.225442 81.90071 <0.0001X5

2 0.022169 1 0.022169 1.481651 0.2359X1X2X3 0.019377 1 0.019377 1.295011 0.2668X1X2X4 0.885756 1 0.885756 59.19828 <0.0001X1X3X4 0.306765 1 0.306765 20.50223 0.0002X2X3X4 0.314871 1 0.314871 21.04393 0.0001X3X4X5 0.023355 1 0.023355 1.560929 0.2241X1

2X2 0.735455 1 0.735455 49.15311 <0.0001X1

2X3 0.277265 1 0.277265 18.53064 0.0003X1

2X4 0.186672 1 0.186672 12.47595 0.0018X1X2

2 0.025775 1 0.025775 1.722608 0.2023Residual 0.344138 23 0.014963Lack of fit 0.217012 16Pure error 0.127126 7Cor total 58.87951 49

Fig. 3. Response surface and contour plot of ln(MSE) for the neurons number andthe step size while other factors remained constant.

0.013563 0.746843 0.70450.018161

less capable to escape from local minima in the error surface. Toohigh step size lead to an increasing in the magnitude of the oscil-lations for the mean squared error and resulting slow convergenceto the lower error state.

Fig. 4 displays the response surface of ln(MSE) versus the neu-rons number and training epochs while other factors held as aconstant at a coded value equal to zero. The response surface showsthe falling ridge form within the simulation rank. It is clear that theincreasing in training epoch from 100 to 3000 and neurons num-ber from 20 to 30, decreased the ln(MSE) to a minimum value of−7.23. Too few epochs limit the ability of the ANN to model theprocess, and too many causes an overtraining of ANN and its fail-ure to learn the data leading to an increase in mean square errorvalues.

The response surface plot of step size and momentum coeffi-cient by maintaining other factors constant at coded value of zerois shown in Fig. 5. The response surface had descending surfaceform within the simulation rank. The contour plot shows that withincreasing in step size from 0.1 to 0.4 and momentum coefficientfrom 0.1 to 0.7, the ln(MSE) decreased to −7.45. The maximumnumber of training epoch is inversely related to the square ofmomentum coefficient. This parameter is a measure of how closethe solution decision boundary is to the input patterns. This meansthat if the input classes are difficult to predict it will take manyepochs for the algorithm to converge.

The fitness function (Eq. (20)) is then used to measure the fit-ness value for each chromosome in the GA procedure. The criticalparameters in GA’s are the size of the population, mutation rate,number of generations (i.e., iterations) and etc. In this study, pop-ulation size of 30, crossover probability of 0.95, mutation rate of

0.15, and the number of iteration of 2000 are employed.

Table 5 shows the five ANN topology parameters and their cor-responding output responses for the fitness function. The MSE ofproposed ANN topology is clearly smaller than the values in Table 3.Therefore, it can be concluded that the proposed approach does

Page 7: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

90 T. Nazghelichi et al. / Computers and Electronics in Agriculture 75 (2011) 84–91

Ft

fir

3p3Atctpeuaat

el

TT

ig. 4. Response surface and contour plot of ln(MSE) for the neurons number andraining epochs while other factors remained constant.

nd the optimal topology of ANN with the satisfactory outputesponses.

The obtained X1 parameter value falls between levels 2 and. The X2 parameter value falls between levels 2 and 3. The X3arameter is closed to level 3. The X4 falls between levels 2 and. The X5 is equal to the level 2. The proposed optimal topology ofNN was designed, trained and tested. The MSE of optimal ANN

opology on cross validation set data was 4.68 × 10−4 and it islearly smaller than the values in Table 3. This is extra evidento the applicability of GA approach to find the global optimumoint. The values of R2, MSE and MAE for energy utilization,nergy utilization ratio, exergy loss and exergy efficiency obtainedsing the optimal ANN topology were (0.9722, 0.9811, 0.9827nd 0.9779), (3.70 × 10−3, 4.14 × 10−4, 1.83 × 10−3 and 6.24 × 10−4)nd (3.57 × 10−2, 2.60 × 10−2, 1.81 × 10−2 and 2.12 × 10−2), respec-

ively.

Fig. 6 shows the plot of the experimental data of energy andxergy versus the predicted values of energy and exergy calcu-ated using optimum ANN topology. It is obvious that the predicted

able 5he best solutions found by the integrated RSM and GA approach.

Parameters

X1 X2 X3 X4

28 0.35 0.66 287

Fig. 5. Response surface and contour plot of ln(MSE) for the step size and momen-tum coefficient while other factors remained constant.

energy and exergy using optimal topology of ANN is very close tothat obtained from experiment, demonstrating the suitability ofthe optimum ANN model for thermodynamically simulation of flu-idized bed drying of carrot cubes. The optimal topology of ANNobtained using coupled RSM and GA provided good quality pre-dictions for the four exogenous outputs. This approach showed aclear superiority over lone RSM and trail-and-error methods to findoptimum topology of ANNs.

It is worth noting that the coupled RSM and GA decreases thenumber of required ANN topology to find the global optimal one,so the computational operations cost and time can be dramaticallyreduced with respect to the cost and time required when the trialand error is used. For example, by considering four different levelsof each ANN variable, 1024 (=45) different configurations of ANN

would be required. But the coupled RSM and GA uses only 50 ANNstopology for finding optimal ANN. The computational cost of trialand error method was approximately twenty times greater thanthe computational cost of integrated RSM and GA, in this study.

Response

X5 MSE on cross validation

7 3 0.00027171

Page 8: OPTIMIZATION OF NEURAL NETWORK WITH RESPONSE SURFACE METHOD.pdf

T. Nazghelichi et al. / Computers and Electronics in Agriculture 75 (2011) 84–91 91

R2

= 0.9722

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

Pre

dic

ted

en

erg

y u

tilizati

on

(kJ/s

) R

2

= 0.9811

0

0.1

0.2

0.3

0.4

0.5

0.6

0 0.1 0.2 0.3 0.4 0.5 0.6

Experimental energy utilization ratio

Pre

dic

ted

en

erg

y u

tilizati

on

rati

o

R2

= 0.9827

0.0

0.3

0.6

0.9

1.2

1.5

1.8

0.0 0.3 0.6 0.9 1.2 1.5 1.8

Pre

dic

ted

exerg

y lo

ss (

kJ/s

)

R2

= 0.9779

0.00

0.20

0.40

0.60

0.80

1.00

0.00 0.20 0.40 0.60 0.80 1.00

Pre

dic

ted

exerg

y e

ffic

ien

cy

Experimental energy utilization (kJ/s)

outpu

fwtoc

6

tdfistsfwswaae

R

A

A

A

Experimental exergy loss (kJ/s)

Fig. 6. Comparison of predicted and desired

Indeed, the coupled RSM and GA is a superior, cheaper andaster optimization technique to find appropriate ANN topologyhen compared to the traditional trial and error method. Therefore,

he integrated RSM and GA approach is an appropriate alternativever lone RSM and trail-and-error methods, which can reduce theomputational cost and accelerate the ANN development.

. Conclusion

A multilayer feed forward artificial neural network (ANN)rained by back propagation algorithms was developed to pre-ict the energy and exergy based on the four input variables. Thenal optimal ANN model obtained using integrated RSM and GA,uccessfully learned the relationship between the input parame-ers and outputs parameters. The integrated RSM and GA approachhowed to be a useful tool to find the global optimum ANN topologyor prediction the energy and exergy during fluidized bed dryinghich is considered as a tedious and time-consuming task. For ANN

imulation, significant reduction of the total computational effortith a relative good precision attained using coupled RSM and GA

pproach. The developed models could be used for determining theppropriate drying conditions of carrot cubes to reach the optimalnergy efficiency in fluidized bed drying.

eferences

ghbashlo, M., Kianmehr, M.H., Arabhosseini, A., 2009. Performance analysis of dry-ing of carrot slices in a semi-industrial continuous band dryer. Journal of FoodEngineering 91, 99–108.

kpinar, E.K., 2004. Energy and exergy analyses of drying of red pepper slices inconvective type dryer. International Journal of Heat and Mass Transfer 31 (8),1165–1176.

kpinar, E.K., Midilli, A., Bicer, Y., 2006. The first and second law analyses of 354thermodynamic of pumpkin drying process. Journal of Food Engineering 72 (4),320–331.

Experimental exergy efficiency

t values using optimal MLFF ANN topology.

Erenturk, S., Erenturk, K., 2007. Comparison of genetic algorithm and neural networkapproaches for the drying process of carrot. Journal of Food Engineering 78,905–912.

Erenturk, K., Erenturk, S., Tabil, L.G., 2004. A comparative study for the estimationof dynamical drying behavior of Echinacea angustifolia: regression analysis andneural network. Computers and Electronics in Agriculture 45 (1–3), 71–90.

Farkas, I., Remenyi, P., Biro, A., 2000. Modelling aspects of grain drying with a neuralnetwork. Computers and Electronics in Agriculture 29 (1–2), 99–113.

Hashimoto, Y., 1997. Applications of artificial neural networks and genetic algo-rithms to agricultural system. Computers and Electronics in Agriculture 18 (71),72.

Hepbasli, A., Erbay, Z., Colak, N., Hancioglu, E., Icier, F., 2010. Exergetic perfor-mance assessment of three different food driers. Proceedings of the Institutionof Mechanical Engineers, Part A. Journal of Power and Energy 224, 1–12.

Madadlou, A., Emam-Djomeh, Z., Ebrahimzadeh Mousavi, M., Ehsani, M.R., Javan-mard, M., Sheehan, D., 2009. Response surface optimization of an artificial neuralnetwork for predicting the size of re-assembled casein micelles. Computers andElectronics in Agriculture 68, 216–221.

Midilli, A., Kucuk, H., 2003. Energy and exergy analyses of solar drying process ofpistachio. Energy 28, 539–556.

Montgomery, D.C., 2001. Design and Analysis of Experiments, 5th ed. John Wiley &sons Inc, New York.

Movagharnejad, K., Nikzad, M., 2007. Modeling of tomato drying using artificialneural network. Computers and Electronics in Agriculture 59 (1–2), 78–85.

Mujumdar, A.S., 2006. Hand Book of Industrial Drying, 3rd ed. Marcel Dekker, NewYork.

Nazghelichi, T., Aghbashlo, M., Kianmehr, M.H., Omid, M., 2010. Prediction of energyand exergy of carrot cubes in a fluidized bed dryer by artificial neural networks.Drying Technology: An International Journal, in press.

Omid, M., Baharlooei, A., Ahmadi, H., 2009. Modeling drying kinetics of pistachio nutswith multilayer feed–forward neural network. Drying Technology: An Interna-tional Journal 10, 1069–1077.

Ozgener, L., Ozgener, O., 2009. Exergy analysis of drying process: an experimen-tal study in solar greenhouse. Drying Technology: An International Journal 27,580–586.

Syahrul, S., Hamdullahpur, F., Dincer, I., 2002. Exergy analysis of fluidized bed dryingof moist particles. Exergy: An International Journal 2, 87–98.

Szargut, J., Morris, D.R., Steward, F.R., 1988. Exergy Analysis of Thermal, Chemicaland Metallurgical Processes. Hemisphere Publishing Corporation, New York.

Youssefi, Sh., Emam-Djomeh, Z., Mousavi, S.M., 2009. Comparison of artificial neuralnetwork (ANN) and response surface methodology (RSM) in the prediction ofquality parameters of spray-dried pomegranate juice. Drying Technology: AnInternational Journal 27, 910–917.