ORIGINAL ARTICLE

A tuned hybrid intelligent fruit fly optimization algorithmfor fuzzy rule generation and classification

Seyed Mohsen Mousavi1 • Madjid Tavana2,3 • Najmeh Alikar1 • Mostafa Zandieh4

Received: 28 December 2016 / Accepted: 15 June 2017 / Published online: 1 July 2017

� The Natural Computing Applications Forum 2017

Abstract Fuzzy rule-based systems (FRBSs) are well-

known soft computing methods commonly used to tackle

classification problems characterized by uncertainties and

imprecisions. We propose a hybrid intelligent fruit fly

optimization algorithm (FOA) to generate and classify

fuzzy rules and select the best rules in a fuzzy if–then rule

system. We combine a FOA and a heuristic algorithm in a

hybrid intelligent algorithm. The FOA is used to create,

evaluate and update triangular fuzzy rule-based and

orthogonal fuzzy rule-based systems. The heuristic algo-

rithm is used to calculate the certainty grade of the rules.

The parameters in the proposed hybrid algorithm are tuned

using the Taguchi method. An experiment with 27

benchmark datasets and a tenfold cross-validation strategy

is designed and carried out to compare the proposed hybrid

algorithm with nine different FRBSs. The results show that

the hybrid algorithm proposed in this study is significantly

more accurate than the nine competing FRBSs.

Keywords Fuzzy rule-based system � Classification

system � Fruit fly optimization � Hybrid algorithm � Taguchi

method

1 Introduction

Fuzzy rule-based systems (FRBSs) are popular computa-

tional intelligence and soft computing methods commonly

used for classification problems. FRBSs utilize linguistic

variables which are accurate and easy to interpret in real-

world problems involving uncertainty and ambiguity.

Accuracy and interpretability are two essential require-

ments of linguistic fuzzy modeling [34]. Accuracy and

interpretability represent the ability to predict and under-

stand the behavior of the system. FRBSs have been suc-

cessful in solving a wide range of classification problems

[5, 6, 7, 8, 11, 40, 43].

Meta-heuristics have played an important role in

employing fuzzy if–then rules for classification problems.

The genetic algorithm (GA) is the most widely used meta-

heuristic in FRBSs. Pourpanah et al. [32] used a hybrid

model for data classification and rule extraction in which

fuzzy ARTMAP with Q-learning was applied for incre-

mental learning of data and a GA was implemented for

feature selection and rule extraction. Tsakiridis et al. [43]

utilized a genetic fuzzy rule-based classification system

with applied differential evolution as its learning algo-

rithm. Derhami and Smith [5] applied a mixed-integer

programming model to extract fuzzy rules from datasets,

& Madjid Tavana

tavana@lasalle.edu;

http://tavana.us

Seyed Mohsen Mousavi

mousavi@siswa.um.edu.my

Najmeh Alikar

najme.alikar@gmail.com

Mostafa Zandieh

m_zandieh@sbu.ac.ir

1 Department of Mechanical Engineering, Faculty of

Engineering, University of Malaya, Kuala Lumpur, Malaysia

2 Business Systems and Analytics Department, Distinguished

Chair of Business Analytics, La Salle University,

Philadelphia, PA 19141, USA

3 Business Information Systems Department, Faculty of

Business Administration and Economics, University of

Paderborn, 33098 Paderborn, Germany

4 Department of Industrial Management, Management and

Accounting Faculty, Shahid Beheshti University, Tehran,

Iran

123

Neural Comput & Applic (2019) 31:873–885

DOI 10.1007/s00521-017-3115-4

where the model found multiple rules by converting the

obtained optimal solutions into a set of taboo constraints

that prevented the model from re-finding the previously

obtained rules. Rudzinski [34] proposed a multi-objective

genetic algorithm to simultaneously obtain systems with

various levels of compromise between accuracy and

interpretability. Cintra et al. [2] used a genetic fuzzy sys-

tem to improve the interpretability of the fuzzy rule bases

and select the final rule base. Fazzolari et al. [10] analyzed

36 training set selection methods using genetic fuzzy rule-

based classification systems where a wide range of datasets

were utilized to evaluate the proposed method. Sanz et al.

[35] used a post-processing genetic tuning step to enhance

the performance of fuzzy rule-based classification systems

by applying the concept of interval-valued fuzzy sets.

Fazzolari et al. [9] applied a fuzzy procedure which was

integrated within a multi-objective evolutionary algorithm

for performing a tuning and a rule selection process.

Stepnicka et al. [41] used a fuzzy rule-based method to

combine several individual forecasts. In this method, the

weights of the combination were obtained by fuzzy rule

bases according to time series features as well as trend,

seasonality or stationary. Alikar et al. [1] improved the

performance of fuzzy rule-based classification systems on a

dataset by combining particle swarm optimization (PSO)

with a heuristic algorithm.

In order to carry out this, a hybrid algorithm consisting a

FOA and a heuristic proposed by Ishibuchi and Nakaskima

[12] is developed to create fuzzy if–then rules and to obtain

the certainty grade of the rules. In addition, some fuzzy

methods are used to show the features in the dataset. The

fruit fly optimization algorithm (FOA) is a population-

based and global optimization meta-heuristic algorithm

proposed by Pan [30], inspired by the food finding behavior

of the insect of fruit fly. While there is no work in the

literature that uses FOA to improve the performance of

fuzzy rule-based classification systems, FOA has been

becoming very popular in recent years in other fields.

Sheng and Bao [38] optimized a kind of fractional-order

fuzzy PID controller, which was applied to an electronic

throttle. Mousavi et al. [20, 21] solved a multi-item multi-

period inventory-location allocation problem, formulated

into a mixed-integer binary nonlinear programming, using

a modified FOA. Mousavi et al. [22] improved an FOA for

solving an integrated multi-item multi-period two-echelon

supply chain and location allocation problems. Wang et al.

[44] used an improved version of an FOA with swarm

collaboration and random reputation to joint replenishment

problems, which outperformed FOA, PSO and DEA algo-

rithms in terms of accuracy and reliability. Zheng and

Wang [45] optimized an unrelated parallel machine

scheduling problem with additional resource constraints,

formulated into a mixed-integer linear programming

model, in which the aim was to minimize the makespan

using a two-stage adaptive FOA. Lei et al. [18] proposed a

fruit fly clustering optimization to identify dynamic protein

complexes by integrating a fruit fly optimization algorithm

with gene expression profiles. Pan et al. [29] optimized

continuous function optimization problems, where a new

control parameter was applied to adaptively calibrate the

search scope around its swarm location.

In this paper, a design of experiment procedure, i.e.,

Taguchi approach, is used to tune the algorithm parameters

which is applied for the first time in the literature, for rule-

based classification systems. Khanlou et al. [13] used the

Taguchi method to investigate the effects of polymer

concentration and electrospinning parameters on the

diameter of electrospun polymethyl methacrylate (PMMA)

fibers. Several studies have successfully used other design

of experiment methods as well as the response surface

methodology. Khanlou et al. [14] used a regression model

to obtain the dominant factors on the responses where the

molecular weight was appeared to successfully find out the

final objective with a lesser number of experimental runs.

Khanlou et al. [16] applied a central composite design of

RSM to conduct a mathematical model as well as to

characterize the optimum condition where a three-layered

feed-forward artificial neural network was established for

the prediction of the response factor. Khanlou et al. [17]

used an ANFIS model to predict the surface roughness of

titanium biomaterials. Ong et al. [27] applied three algo-

rithms of the Cuckoo search algorithm, flower pollination

algorithm and particle swarm optimization to optimize the

extrusion process parameters in which RSM was used to

tune the algorithms’ parameters. Keshtegar and Heddam

[15] proposed a modified response surface method and a

multilayer perceptron neural network to formulate daily

dissolved oxygen concentration.

2 Fuzzy rule-based classification systems

The old fuzzy rule-based systems are often used for pattern

classification [42]. Pattern classification generally involves

a manual classification of training samples by experts,

where a classifier automatically assigns an unseen data

sample to a set of pre-defined classes [37]. Figure 1 shows

a fuzzy inference system, which includes a fuzzifier for

translating crisp inputs into fuzzy values; a heuristic

algorithm for obtaining a fuzzy output by applying a fuzzy

reasoning mechanism; a defuzzifier for translating this

fuzzy output into a crisp value; and a knowledge base for

storing the fuzzy rules. An if–then rule generally takes the

form of:

If antecedent proposition then consequent proposition

874 Neural Comput & Applic (2019) 31:873–885

123

This work uses fuzzy if–then rules of the following type

for the C = 2 class pattern classification problem:

Rule Rj : If x1 is Lj1 and . . . and

xn is Ljn then ClassCj with CFj for j ¼ 1; 2; . . .;Nð Þð1Þ

where Rj is the label of the j th fuzzy if–then rule, Lj1, …,

Ljn are antecedent of fuzzy sets on the unit interval [0, 1],

Cj is the consequent class (in this study there are 2 classes),

and CFj is the grade of certainty of the fuzzy if–then rule

Rj. This study proposes the antecedents of fuzzy sets, using

two orthogonal and triangular fuzzy sets, as depicted in

Figs. 2 and 3, respectively.

In the next subsection, we apply the proposed hybrid

intelligent algorithm to generate N fuzzy if–then rules, where

each one of the rules follows the form as shown in Eq. (1).

3 The proposed hybrid intelligent algorithm

A hybrid intelligent algorithm comprising a fruit fly opti-

mization algorithm and an extended version of the heuristic

algorithm proposed by Ishibuchi and Nakaskima [12] with

fuzzy features is explained in this section. We generated

and coded several other algorithms including GA, PSO and

SA, in addition to the FOA proposed in this study. In all

instances, the proposed FOA outperformed all other com-

peting algorithms. Furthermore, the proposed FOA is an

enhanced version of the population-based algorithms such

as GA and PSO, and in most cases, the FOA has a superior

performance in comparison with the GA and PSO. Finally,

studies that have compared the FOA to other recent works

show results that are in its favor. Therefore, FOA is a

strong meta-heuristic algorithm for solving the problem in

hand [20, 22]. An introduction of the FOA is presented,

followed by a detailed explanation of the hybrid algorithm,

in which the proposed FOA is integrated with the heuristic

algorithm.

Pan [30] introduced the fruit fly optimization algorithm,

inspired by the behavior of the fruit fly in search of food.

The fruit fly is generally superior to other species because

of its strong sensing and perception. The fruit fly can find

all sorts of scents from far away and use its powerful vision

to find and fly toward the food [30]. The particle foraging

process of the fruit flies is illustrated in Fig. 4.

The hybrid intelligent algorithm is developed to gener-

ate fuzzy if–then rules and classify the dataset explained in

the following stages:

1. Initializing the parameters and population In this

study, the parameters of the FOA algorithm are C1 and C2,

which are two fixed parameters; Pop1 is the number of

articles, Pop2 is the number of fruit flies around each

Fig. 1 Proposed fuzzy

inference system (fuzzy rule-

based system) diagram

Fig. 2 Orthogonal membership functions of two linguistic values

Fig. 3 Membership functions of five linguistic values Fig. 4 Particle foraging process of the fruit flies

Neural Comput & Applic (2019) 31:873–885 875

123

particle, and Gen is the number of generations. Also, the

initial population of fuzzy if–then rules is generated ran-

domly. There are two methods available to represent and

initialize the fuzzy if–then rules. The first method is to

initialize the solutions based on the Pittsburgh approach

applied to orthogonal fuzzy sets. A particle structure for the

orthogonal approach is shown as follows:

where a and b are called the point and the length of

membership function edges, and Lj1, …, Ljn, j = 1, 2,…,

N are antecedent of fuzzy sets, respectively. In the second

method, triangular fuzzy sets, depicted in Fig. 3, represent

five linguistic values, in addition to ‘‘don’t care,’’ and are

denoted by the six symbols listed in Table 1. A particle

structure for the triangular fuzzy sets is displayed as follows:

Aj1 Aj2 Aj3 Aj4 Aj5 Aj6 Aj7 Aj8 Aj9

where vector Aj is the antecedent of triangular fuzzy sets

for rule j (j = 1,2,…, N) which can take one of the six

linguistic values proposed in Table 1.

To clarify how to initialize the rules, a representation of

a particle (fruit fly) for triangular fuzzy if–then rules with

three rules is shown as follows:

A11 A12 A13 A14 A15 A16 A17 A18 A19

A21 A22 A23 A24 A25 A26 A27 A28 A29

A31 A32 A33 A34 A35 A36 A37 A38 A39

2. Generating fruit flies for each particle The number of

Pop2 fruit flies is generated randomly around each one of

the Pop1 particles.

3. Evaluating each fruit fly and particle After the gen-

eration of particles and fruit flies around them, each fuzzy

if–then rule is evaluated by the fitness value. The fitness

value of the fuzzy if–then rule (i.e., Rj) used in this study is

evaluated by the following fitness function [12]:

FitnessðRjÞ ¼ WNCP:NCPðRjÞ �WNMP:NMPðRjÞ ð2Þ

where WNCP is the reward for the correct classification,

NCP(Rj) is the number of correctly classified training

patterns by Rj, WNMP is the penalty for the misclassifica-

tion, and NMP(Rj) is the number of misclassified training

patterns by Rj.

To classify each rule to each class, first calculate the

certainty grade and then obtain the fitness value for that

rule. Calculating the grade of certainty is different for each

one of the orthogonal and triangular fuzzy sets. Assume

that m training patterns (data) xp ¼ ðxp1; xp2; . . .; xpnÞ, p ¼1; 2; . . .;m are given for an n-dimensional and C-class

pattern classification problem. For the orthogonal fuzzy

sets, the grade of certainty for rule Rj is calculated by the

following steps:

Step 1 Obtain the membership functions of each variable

of each training pattern xp p ¼ 1; 2; . . .;m with the fuzzy

if–then rule Rj, as shown in Eq. (3) (fuzzifier phase).

lLowðxpÞ ¼1 xp\a

ðaþ bÞ � xp

ba� xp �ðaþ bÞ

0 xp [ ðaþ bÞ

8><

>:; lHighðxpÞ

¼ 1 � lLowðxpÞð3Þ

where lLowðxpÞ and lHighðxpÞ are the membership

functions of labels Low and High to each training pattern

xp, and a and b are the start point and the length of

membership function edges, respectively.

Step 2 Find the truth value of each rule Rj. The truth

value is computed by using the min operator in fuzzy

logic and combining the antecedent clauses in a fuzzy

fashion, resulting in a continuous value representing the

rule’s degree of activation. Here, the truth value of each

rule Rj is demonstrated by ljðxpÞ.Step 3 For class h, compute bclasshðjÞ as follows:

bclasshðjÞ ¼X

xp2hljðxpÞ ð4Þ

Step 4 Find class hj that has the maximum value of

bclasshðjÞ, as follows:

bclass hðjÞ ¼ max1 � k � C

fbclasskðjÞg ð5Þ

It should be noted that we specify Cj as Cj = [ when

the consequent class Cj of the rule Rj cannot be determined

uniquely, because two or more classes have the maximum

Table 1 Decoding linguistic values

Linguistic value Decoding

Small (S) 1

Medium small (MS) 2

Medium (M) 3

Medium large (ML) 4

Large (L) 5

Don’t care (DC) 6

a1 a2 … an b1 b2 … bn L11 L12 … L1n … LN1 … LNn

876 Neural Comput & Applic (2019) 31:873–885

123

value. In this case, we specify Cj as Class h when a single

class h takes the maximum value. The grade of certainty

CFj is determined as:

CFj ¼bclass hðjÞ � �bP

h bclass hðjÞð6Þ

with

�b ¼P

h 6¼h bclass hðjÞC � 1

ð7Þ

Choose the winning rule using the following formula

[12]:

ljðxpÞ:CFj ¼ max1 � k � C

flkðxpÞ:CFkg ð8Þ

In triangular fuzzy sets, calculation of the grade of

certainty of rule Rj is similar to the steps proposed for

orthogonal fuzzy sets with this difference: In the first step,

the training pattern xp should be normalized in the interval

[0, 1]. Then, one obtains bclass hðjÞ ¼P

xp2h ljðxpÞ using

step 1, where ljðxpÞ is computed as:

ljðxpÞ ¼ minflj1ðxp1Þ; . . .; ljnðxpnÞg ð9Þ

4. Smell-based and vision-based search In the smell-

based search process, all articles and fruit flies around each

particle are updated. First, Pop1 fruit flies are gathered

around each particle and use the position and velocity of

the flies to construct a subpopulation. We obtain the

position of the flies by using:

xrskþ1 ¼ xrsk þ vrskþ1 ð10Þ

where xrsk is the position of the rth fly in dimension s and

iteration k, r ¼ 1 to Pop2, s ¼ 1; 2; . . .; S, vrskþ1 is its

velocity, and Pop2 is the number of flies. Moreover, the

velocities of the flies used in Eq. (11) are calculated using

Eq. (11), as follows:

vrskþ1 ¼ w � vrsk þ C1 � rand � ðpbrsk � xrsk Þ þ C2 � rand � ðgbsk� xskÞ

ð11Þ

In Eq. (11), w is the inertia weight for controlling the

magnitude of the old velocity vrsk when calculating the new

velocity vrskþ1, pbrsk is the position of the best local fly, gbsk is

the position of the best global fly, C1 represents the sig-

nificance of pbk, C2 represents the significance of gbsk, and

rand represents a uniformly distributed real random num-

ber between 0 and 1.

Naka et al. [26] and Shi and Eberhart [39] have shown

that introducing a linearly decreasing inertia weight into

the original PSO significantly improves its performance.

Naka et al. [26] and Shi and Eberhart [39] express the

linear distribution of the inertia weight as follows:

w ¼ wmax �wmax � wmin

Gen� d ð12Þ

where d is the current number of iterations and Gen is the

maximum number of iterations. Equation (12) shows how

the inertia weight is updated, where wmax and wmin are the

minimum and the maximum weights, respectively. The

related results of the two parameters wmax ¼ 0:9 and

wmin ¼ 0:4 are reported in [26, 39].

5. Global vision-based search In this search, the algo-

rithm considers the fitness values of the flies and finds the

best one in a subpopulation.

6. Stopping criterion Several stopping criteria have been

proposed in the literature to stop a meta-heuristic (i.e., a

specific CPU time, a specific fitness function value or a

specific number of generations). We use a specific number

of generations as the stopping rule in our Gen meta-

heuristic algorithm. The flowchart of the proposed hybrid

intelligent algorithm is presented in Fig. 5.

4 Design of experiments

In this section, a Taguchi method, whose parameters are

C1, C2, Pop1, Pop2 and Gen, is applied to tune the FOA

parameters. The Taguchi method utilizes an orthogonal

array to run experiments and to analyze results. We used

the Taguchi method in this study for two reasons: (1)

reducing the time and cost in experimental design. For

example, consider an L12 (211) orthogonal array. In this

example, we can reduce the number of experiments to 12

sets instead of 211 = 2048 sets of experiments and

achieve similar results for a full factorial experimental

design; (2) using a full factorial experimental design does

not allow for adjustment on the level of the factor value,

since the value cannot be adjusted to robustness once it

has been set to be the optimal value. In contrast with the

Taguchi method, once the value has been set to be the

optimal value, it can be adjusted if the factor is recog-

nized to be neutral or an adjustment factor. We used the

Taguchi method in this study because it is widely rec-

ognized as a suitable statistical method for achieving the

closest value to the target in an experimental design

[1, 20, 21, 22, 24, 25, 28].

Obviously, increasing the number of parameters has a

strong influence on the performance of the meta-heuristic

algorithms. This section investigates the behavior of all the

parameters of the proposed FOA. All different combina-

tions of the parameters C1, C2, Pop1, Pop2 and Gen yield

many alternative algorithms. The most frequently applied

and exhaustive approach is a full factorial experiment [19].

This method is not always appropriate, because if the

number of factors becomes significantly high, then it

Neural Comput & Applic (2019) 31:873–885 877

123

Fig. 5 Proposed hybrid

intelligent algorithm flowchart

878 Neural Comput & Applic (2019) 31:873–885

123

becomes increasingly difficult to implement [23]. Frac-

tional factorial experiments (FFEs) are improved by

Cochran and Cox [3] to decrease the number of required

tests. FFEs only allow a portion of the total number of

possible combinations to estimate the main effect of factors

and some of their interactions. Ross [33] used the Taguchi

technique and improved a family of FFE matrices by

decreasing the number of experiments, still producing

adequate information. The orthogonal arrays are used in the

Taguchi method to peruse decision variables with limited

experiments. The factors are divided into controllable and

noise factors, where noise factors cannot be controlled

directly. The Taguchi method determines the optimal level

of the controllable factors according to the concept of

robustness [31].

Taguchi categorizes objective functions into three dis-

tinct groups: the smaller-is-better type, the larger-is-better

type and the nominal-is-best type. Here, a three-level

orthogonal array is used and the objective function is of the

maximizing type. So, the objective function in fuzzy

classification problems is segmented into the larger-the-

better type, where its corresponding S/N ratio is given by

Eq. (13), as follows:

S=N ratio ¼ �10 log1

n1

Xn1

t¼1

1

d2t

!

ð13Þ

In Eq. (13) dt is the objective function value of the

experiment row t, log is the logarithm function, and n1 is

the run times of the objective function. Also, a general

symbol for a three-level standard orthogonal array is rep-

resented by L9, where 9 is the number of rows of the

experiment.

Thus, to determine the best value of the algorithm’s

inputs, a parameter adjustment technique is applied to tune

the control parameters of C1, C2, Pop1, Pop2, and Gen in

FOA. The proper design of the parameters of each algo-

rithm depends on the type of problem. However, many

researchers manually fix parameters and operators

according to the reference values in prior studies. In this

article, the readers are referred to [23] for more information

about the application of the Taguchi method.

The hybrid intelligent algorithm has been coded in

MATLAB R2013a, and the code has been executed on a

computer with 3.8 GHz and 4 GB of RAM. Furthermore,

all the designs of experiments and statistical analyses have

been performed on the Wisconsin dataset using the

MINITAB software version 15.

In this study, the orthogonal array L9 (9 rows) is gen-

erated to analyze using the Taguchi method, in which each

factor includes three levels. Since the goal of the model is

to maximize the classification accuracy, ‘‘the larger than

better type’’ is the most suitable one to use to calculate the

S/N ratio using Eq. (13). Tables 2 and 3 display the values

of the FOA parameters for each level for triangular and

orthogonal fuzzy sets, respectively.

Figures 6 and 7 depict the mean S/N ratio plot for dif-

ferent levels of the parameters for both the FOATFRB and

FOAOFRB approaches, respectively. In this study, the

objective function is run 5 times for each experiment.

Table 4 depicts the Taguchi design of experiments for

the FOA parameters for the orthogonal fuzzy sets accord-

ing to the red lines drawn in Fig. 6. Also, Table 5 proposes

the Taguchi design of experiments for the FOA parameters

for the orthogonal fuzzy sets as shown in Fig. 7.

5 Experimental framework

In this section, we consider 27 real-world classification

datasets for evaluating the proposed fuzzy rule-based sys-

tems experimentally, which are publicly available at the

UCI repository of machine learning datasets and the KEEL

dataset repository. Table 6 presents the characteristics (i.e.,

patterns, features and classes) of the current datasets. The

datasets are chosen for this study to derive a comparison

with the results obtained for the nine various approaches

presented by Cintra et al. [2] and Sanz et al. [36]. The

hybrid intelligent algorithm is used for both triangular and

orthogonal fuzzy classification rule-based approaches,

where their accuracy extracted from each dataset is com-

pared to these nine approaches. We use a tenfold cross-

validation strategy to evaluate the accuracy of the

approaches.

Table 7 shows the accuracy results of all the approaches

extracted from each one of the 27 used datasets. The first

eight approaches described in the second to ninth columns

Table 2 FOA parameter levels

for the orthogonal fuzzy setsParameters 1 2 3

A: C1 1.5 2 2.5

B: C2 1.5 2 2.5

C: Pop1 4 5 7

D: Pop2 50 70 100

E: Gen 300 500 1000

Table 3 FOA parameter levels

for the triangular fuzzy setsParameters 1 2 3

A: C1 1.5 2 2.5

B: C2 1.5 2 2.5

C: Pop1 3 4 7

D: Pop2 60 80 100

E: Gen 200 500 1000

Neural Comput & Applic (2019) 31:873–885 879

123

of Table 7 are explained by Cintra et al. [2] in detail, while

FCA based is explained by Sanz et al. [36].

The results of Table 7 show that on the average,

FOATFRB and FOAOFRB outperform the other approa-

ches in terms of accuracy for the 27 datasets. In Table 7,

the best performance among these datasets is identified in

bold. According to the results in this table, FOAOFRB is

the best in 13 out of the 27 datasets, whereas FOATFRB is

the best in 11 out of the 27 datasets. The last row of the

table depicts the average rankings of each approach.

To compare the accuracy of the clustering approaches

used in this study, several experimental tests are applied to

assess the performance of these approaches. The proposed

FOATFRB and FOAOFRB approaches are compared with

the other methods using Wilcoxon’s signed-rank test to

evaluate whether there are statistically significant

321

-0.23

-0.24

-0.25

-0.26

-0.27

321 321

321

-0.23

-0.24

-0.25

-0.26

-0.27

321

A

Mea

nof

SNra

tios

B C

D E

Main Effects Plot for SN ratiosData Means

Signal-to-noise: Larger is better

Fig. 6 Mean S/N ratio plot of

different parameter levels for

FOATFRB

321

-0.24

-0.25

-0.26

-0.27

-0.28

321 321

321

-0.24

-0.25

-0.26

-0.27

-0.28

321

A

Mea

nof

SNra

tios

B C

D E

Main Effects Plot for SN ratiosData Means

Signal-to-noise: Larger is better

Fig. 7 Mean S/N ratio plot of

different parameter levels for

FOATFRB approaches

880 Neural Comput & Applic (2019) 31:873–885

123

differences among the methods. Tables 8 and 9 show the

results of the statistical comparisons versus FOATFRB and

FOAOFRB, respectively. It is found that Wilcoxon’s test

detects significant differences between FOATFRB and all

the remaining models. The Wilcoxon’s signed-rank test

also detects a significant difference between FOAOFRB

and all the other methods mentioned in Table 7. Consid-

ering the higher predictive accuracy obtained by

FOATFRB and FOAOFRB, it can be concluded that both

algorithms outperform the other methods, while there is no

significant deference between the accuracy results obtained

by FOATFRB and the results using CA based on all the 27

datasets.

The second statistical test performed in this article is

comparing the accuracy results of the methods with each

other using the Bonferroni method. Figure 8 shows an

interval plot of the Bonferroni test for the average of the

results of the approaches on the used datasets with

a = 0.05. The results are in favor of the FOATFRB and

FOAOFRB methods.

Figure 9 compares the average ranking of each classifier

approach using Friedman’s method [4]. Friedman’s and

Iman and Davenport’s tests explored the significant dif-

ferences among the approaches considered. Table 10

shows that both statistical tests reject equivalence of results

between the investigated methods. In the next step, we

applied Holm’s method as the post hoc test to distinguish

the significant difference between the FOATFRB as the

control algorithm and the other approaches.

Table 11 depicts the Holm post hoc test for comparing

all the approaches, where the Holm procedure rejects all

first nine hypotheses, since the corresponding p values are

smaller than the adjusted a. This shows that FOATFRB

performs significantly better than FS, FS ? R, IVFSE,

IVFSWI, IVFSWIE, FARC, FURIA, IVTURS and FCA

based at the significance level a = 0.05. FOATFRB is not

significantly better than FOAOFRB; however, FOATFRB

obtains the best ranks in the Friedman test, according to

Fig. 9.

6 Conclusions and future research directions

In this paper, two linguistic fuzzy rule-based classification

methods (i.e., triangular and orthogonal fuzzy approaches)

are introduced to represent various features of datasets. To

extract the fuzzy rule-based classification and find the best

rules, a hybrid intelligent algorithm is proposed to integrate

a fruit fly optimization algorithm with triangular fuzzy

Table 4 Optimal parameter levels for the FOA algorithm and

orthogonal fuzzy sets

Algorithm Factors Optimal levels

FOA C1 2.5

C2 2.5

Pop1 7

Pop2 70

Gen 1000

Table 5 Optimal parameter levels for the FOA algorithm and tri-

angular fuzzy sets

Algorithm Factors Optimal levels

FOA C1 2

C2 1.5

Pop1 7

Pop2 100

Gen 500

Table 6 Characteristics of the datasets used in the experimental

analysis

Datasets Patterns Features Classes

Australian 690 14 2

Balance 625 4 3

Cleveland 297 13 5

Contraceptive 1473 9 3

CRX 653 15 2

Dermotology 358 34 6

Ecoli 336 7 8

German 1000 20 2

Haberman 306 3 2

Hayes-Roth 160 4 3

Heart 270 13 2

Ionosphere 351 33 10

Iris 150 4 3

Magic 1902 10 2

New-Thyroid 215 5 3

Page-blocks 195 22 2

Penbased 1992 16 10

Pima 768 8 2

Saheart 462 9 2

Spectfheart 267 44 2

Tae 151 5 3

Titanic 2201 3 2

Twonorm 740 20 2

Vehicle 846 18 4

Wine 178 13 3

Winequiality-Red 1599 11 11

Wisconsin 683 9 2

Neural Comput & Applic (2019) 31:873–885 881

123

rule-based (FOATFRB) and orthogonal fuzzy rule-based

(FOAOFRB) approaches. Moreover, to calibrate the

parameters of the FOA, a Taguchi method is derived to get

the best results in the shortest amount of time. In total, 27

popular datasets in the literature are used to evaluate the

efficiency of the proposed hybrid algorithm according to

nine different classifier approaches. The results show the

FOATFRB and FOAOFRB have the best performance,

while the FOATFRB has the highest average in comparison

with the other approaches. Furthermore, some well-known

Table 7 Results obtained by the different approaches

Dataset FS FS ? R IVFSE IVFSWI IVFSWIE FARC FURIA IVTURS FCA based FOATFRB FOAOFRB

Australian 86.38 84.93 85.07 83.48 85.51 85.51 86.09 85.8 88.10 88.24 88.24

Balance 80.96 82.40 82.08 82.40 82.88 87.36 83.68 85.76 79.65 87.36 85.76

Cleveland 58.92 54.54 57.58 57.25 59.59 57.92 56.57 59.60 71.40 69.55 69.88

Contraceptive 53.02 53.90 52.68 52.68 52.55 52.68 54.17 53.36 53.10 53.90 55.63

CRX 85.45 86.83 86.53 82.85 87.75 86.53 86.37 87.14 91.70 91.70 85.48

Dermotology 92.75 90.49 94.69 94.69 94.42 89.94 93.86 94.42 95.10 94.89 96.20

Ecoli 78.60 80.07 76.20 77.69 77.39 80.07 80.06 78.58 71.90 81.20 73.42

German 71.10 73.80 71.90 71.00 70.70 71.60 73.30 73.10 72.10 73.80 73.40

Haberman 73.84 72.19 72.22 73.2 73.51 71.22 72.55 72.85 77.90 76.21 79.14

Hayes-Roth 76.41 79.46 80.23 77.18 80.23 80.20 81.00 80.23 85.40 83.42 80.54

Heart 88.15 86.67 86.3 86.30 85.19 84.44 78.15 88.15 90.10 90.50 90.50

Ionosphere 91.18 90.60 91.17 92.33 90.33 90.32 88.91 89.75 95.90 90.71 96.20

Iris 96.00 94.67 96.67 96.67 96.00 94.00 94.00 96.00 96.30 96.67 96.13

Magic 79.13 80.33 78.81 76.18 78.97 80.49 80.65 79.76 80.10 80.33 83.25

New-Thyroid 92.09 94.42 92.09 93.02 93.49 95.35 95.88 95.35 92.30 95.88 95.30

Page-blocks 94.52 94.89 94.71 93.42 94.52 94.34 95.25 95.07 94.40 94.73 95.40

Penbased 91.82 92.00 91.09 91.16 90.18 92.64 92.45 92.18 81.80 90.09 91.46

Pima 75.71 76.17 74.99 72.00 74.35 74.08 76.17 75.90 78.30 79.14 77.20

Saheart 72.28 69.70 69.28 70.99 68.40 70.77 70.33 70.99 79.30 71.45 76.84

Spectfheart 77.51 77.87 80.52 77.13 80.51 78.64 77.88 80.52 89.20 89.20 91.35

Tae 52.37 54.43 53.68 50.39 54.34 48.41 47.08 50.34 63.50 52.37 65.82

Titanic 77.06 78.87 77.65 77.65 77.65 78.87 78.51 78.87 79.20 80.06 79.23

Twonorm 89.19 90.54 93.24 91.49 92.97 89.32 88.11 92.3 79.70 95.85 92.87

Vehicle 64.66 66.9 65.26 62.89 63.48 68.44 70.21 67.38 56.10 73.20 70.21

Wine 93.24 95.49 96.08 97.76 94.95 96.62 93.78 97.19 98.10 97.19 95.43

Winequiality-Red 59.54 59.66 58.91 56.35 58.97 53.96 51.29 58.28 59.50 61.73 64.20

Wisconsin 96.49 96.78 96.34 95.90 96.78 96.63 96.63 96.49 98.20 96.49 98.42

Average 79.57 79.95 79.85 79.04 79.84 79.64 79.37 80.57 81.42 82.80 83.24

Table 8 Wilcoxon’s test results for FOATFRB

FOATFRB versus W z value p value Hypothesis

FS 353 3.9401 8.1448e-05 Rejected

FS ? R 338 3.5799 3.4379e-04 Rejected

IVFSE 346.5 3.7841 1.5428e-04 Rejected

IVFSWI 351 3.8922 9.9343e-05 Rejected

IVFSWIE 349 3.8440 1.2105e-04 Rejected

FARC 330 3.3875 7.0526e-04 Rejected

FURIA 311 3.4414 5.7866e-04 Rejected

IVTURS 302 3.2128 0.0013 Rejected

CA based 274 2.0423 0.0411 Rejected

Table 9 Wilcoxon’s test results for the triangular fuzzy sets

Triangular versus W z value p value Hypothesis

FS 312 4.0226 5.7563e-05 Rejected

FS ? R 275 3.5714 3.5504e-04 Rejected

IVFSE 334 4.0256 5.6835e-05 Rejected

IVFSWI 335 4.0510 5.1004e-05 Rejected

IVFSWIE 365 4.2284 2.3536e-05 Rejected

FARC 336 4.0765 4.5711e-05 Rejected

FURIA 328 3.8732 1.0742e-04 Rejected

IVTURS 311 3.9957 6.4510e-05 Rejected

882 Neural Comput & Applic (2019) 31:873–885

123

statistical tests including the Wilcoxon’s signed-rank, the

Friedman, Iman–Davenport, Bonferroni and Holm post hoc

tests were conducted for comparison purposes. The results

have been in favor of FOATFRB and FOAOFRB, where

FOATFRB showed slightly better performance over the

FOAOFRB.

For future work, we propose integrating hybrid intelli-

gent algorithms with other meta-heuristic algorithms such

FOAO

RB

FOATFRB

FCA-Based

IVTURS

FURIA

FARC

IVFSWIE

IVFSWI

IVFSE

FS+RFS

90

85

80

75

70

Data

95% Bonferroni CI for the MeanInterval Plot of the proposed approachesFig. 8 Bonferroni interval plots

7.41

6.17

7.228.04

7.19 7.11

6.415.61

4.78

3.06 3.02

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

Aver

age

rank

ings

Fig. 9 Accuracy of the Friedman rankings

Table 10 Friedman’s and

Iman–Davenport’s test results

(a = 0.05)

Test Statistics Critical value p value Null hypothesis

Friedman 72.805 18.307 1.27E-11 Rejected

Iman and Davenport 9.60 1.87 1.338E-13 Rejected

Neural Comput & Applic (2019) 31:873–885 883

123

as Harmony search. In addition, other fuzzy linguistic

approaches can be considered to improve the classification

accuracy of the datasets.

Compliance with ethical standards

Conflict of interest The authors declare that they have no conflict of

interest.

References

1. Alikar N, Abdullah S, Mousavi SM, Niaki STA (2013) A hybrid

particle swarm optimization and fuzzy rule-based system for

breast cancer diagnosis. Int J Soft Comput 8(2):126–133

2. Cintra M, Camargo H, Monard M (2016) Genetic generation of

fuzzy systems with rule extraction using formal concept analysis.

Inf Sci 349:199–215

3. Cochran WG, Cox GM (1957) Experimental designs, 2nd edn.

Wiley, New York

4. Demsar J (2006) Statistical comparisons of classifiers over mul-

tiple data sets. J Mach Learn Res 7(Jan):1–30

5. Derhami S, Smith AE (2017) An integer programming approach

for fuzzy rule-based classification systems. Eur J Oper Res

256:924–934

6. Derhami S, Smith AE (2016) A technical note on the paper

‘‘hGA: hybrid genetic algorithm in fuzzy rule-based classification

systems for high-dimensional problems’’. Appl Soft Comput

41:91–93

7. Elkano M, Galar M, Sanz J, Bustince H (2016) Fuzzy rule-based

classification systems for multi-class problems using binary

decomposition strategies: on the influence of n-dimensional

overlap functions in the fuzzy reasoning method. Inf Sci

332:94–114

8. Fallahpour A, Moghassem A (2013) Yarn strength modelling

using adaptive neuro-fuzzy inference system (ANFIS) and gene

expression programming (GEP). J Eng Fabr Fibers (JEFF)

8(4):6–18

9. Fazzolari M, Alcala R, Herrera F (2014) A multi-objective evo-

lutionary method for learning granularities based on fuzzy dis-

cretization to improve the accuracy-complexity trade-off of fuzzy

rule-based classification systems: D-MOFARC algorithm. Appl

Soft Comput 24:470–481

10. Fazzolari M, Giglio B, Alcala R, Marcelloni F, Herrera F (2013)

A study on the application of instance selection techniques in

genetic fuzzy rule-based classification systems: accuracy-com-

plexity trade-off. Knowl Based Syst 54:32–41

11. Gorzałczany MB, Rudzinski F (2016) A multi-objective genetic

optimization for fast, fuzzy rule-based credit classification with

balanced accuracy and interpretability. Appl Soft Comput

40:206–220

12. Ishibuchi H, Nakaskima T (1999) Improving the performance

of fuzzy classifier systems for pattern classification problems

with continuous attributes. IEEE Trans Ind Electron

46(6):1057–1068

13. Khanlou HM, Ang BC, Talebian S, Afifi AM, Andriyana A

(2015) Electrospinning of polymethyl methacrylate nanofibers:

optimization of processing parameters using the Taguchi design

of experiments. Text Res J 85:356–368

14. Khanlou MH, Ang BC, Talebian S, Barzani MM, Silakhori M,

Fauzi H (2015) Multi-response analysis in the processing of poly

(methyl methacrylate) nano-fibres membrane by electrospinning

based on response surface methodology: fibre diameter and bead

formation. Measurement 65:193–206

15. Keshtegar B, Heddam S (2017) Modeling daily dissolved oxygen

concentration using modified response surface method and arti-

ficial neural network: a comparative study. Neural Comput Appl.

doi:10.1007/s00521-017-2917-8

16. Khanlou MH, Ang BC, Kim JH, Talebian S, Ghadimi A (2014)

Prediction and optimization of electrospinning parameters for

polymethyl methacrylate nanofiber fabrication using response

surface methodology and artificial neural networks. Neural

Comput Appl 25:767–777

17. Khanlou MH, Sadollah A, Ang BC, Barzani MM, Silakhori M,

Talebian S (2015) Prediction and characterization of surface

roughness using sandblasting and acid etching process on new

non-toxic titanium biomaterial: adaptive-network-based fuzzy

inference System. Neural Comput Appl 26:1751–1761

18. Lei X, Ding Y, Fujita H, Zhang A (2016) Identification of

dynamic protein complexes based on fruit fly optimization

algorithm. Knowl Based Syst 105:270–277

19. Montgomery DC (2008) Design and analysis of experiments.

Wiley, New York

20. Mousavi SM, Alikar N, Niaki STA (2016) An improved fruit fly

optimization algorithm to solve the homogeneous fuzzy series–

parallel redundancy allocation problem under discount strategies.

Soft Comput 20(6):2281–2307

21. Mousavi SM, Sadeghi J, Niaki STA, Tavana M (2016) A bi-

objective inventory optimization model under inflation and dis-

count using tuned Pareto-based algorithms: NSGA-II, NRGA,

and MOPSO. Appl Soft Comput 43:57–72

22. Mousavi SM, Alikar N, Niaki STA, Bahreininejad A (2015)

Optimizing a location allocation-inventory problem in a two-

echelon supply chain network: a modified fruit fly optimization

algorithm. Comput Ind Eng 87:543–560

23. Mousavi SM, Hajipour V, Niaki STA, Alikar N (2013) Opti-

mizing multi-item multi-period inventory control system with

discounted cash flow and inflation: two calibrated meta-heuristic

algorithms. Appl Math Model 37:2241–2256

24. Mousavi SM, Hajipour V, Niaki STA, Aalikar N (2014). A multi-

product multi-period inventory control problem under inflation

and discount: a parameter-tuned particle swarm optimization

algorithm. Int J Adv Manuf Technol 70(9–12):1739–1756

25. Mousavi SM, Hajipour V, Niaki STA, Alikar N (2013) Opti-

mizing multi-item multi-period inventory control system with

discounted cash flow and inflation: two calibrated meta-heuristic

algorithms. Appl Math Model 37(4):2241–2256

26. Naka S, Genji T, Yura T, Fukuyama Y (2001) Practical distri-

bution state estimation using hybrid particle swarm optimization.

In: Proceeding IEEE Power engineering society winter meeting,

vol 2. USA, p 815–820

Table 11 Holm’s test results

i Approach z p value a/(k - i) Null hypothesis

1 FS 4.862 0.0000 0.0050 Rejected

2 FS ? R 3.488 0.0002 0.0056 Rejected

3 IVFSE 4.657 0.0000 0.0063 Rejected

4 IVFSWI 5.560 0.0000 0.0071 Rejected

5 IVFSWIE 4.616 0.0000 0.0083 Rejected

6 FARC 4.534 0.0000 0.0100 Rejected

7 FURIA 3.754 0.0001 0.0125 Rejected

8 IVTURS 2.872 0.0020 0.0167 Rejected

9 CA based 1.949 0.0257 0.0250 Rejected

10 FOAOFRB 0.041 0.4836 0.0500 Failure to reject

884 Neural Comput & Applic (2019) 31:873–885

123

27. Ong P, Chi DDVS, Ho CS, Ng CH (2016) Modeling and opti-

mization of cold extrusion process by using response surface

methodology and metaheuristic approaches. Neural Comput

Appl. doi:10.1007/s00521-016-2626-8

28. Pasandideh SHR, Niaki STA, Mousavi SM (2013) Two meta-

heuristics to solve a multi-item multiperiod inventory control

problem under storage constraint and discounts. Int J Adv Manuf

Technol 69(5–8):1671–1684

29. Pan Q-K, Sang H-Y, Duan J-H, Gao L (2014) An improved fruit

fly optimization algorithm for continuous function optimization

problems. Knowl-Based Syst 62:69–83

30. Pan W-T (2012) A new fruit fly optimization algorithm: taking

the financial distress model as an example. Knowl Based Syst

26:69–74

31. Phadke MS (1995) Quality engineering using robust design.

Prentice Hall PTR, Englewood Cliffs

32. Pourpanah F, Lim CP, Saleh JM (2016) A hybrid model of fuzzy

ARTMAP and genetic algorithm for data classification and rule

extraction. Expert Syst Appl 49:74–85

33. Ross PJ (1996) Taguchi techniques for quality engineering: loss

function, orthogonal experiments, parameter and tolerance

design. McGraw-Hill Professional, New York

34. Rudzinski F (2016) A multi-objective genetic optimization of

interpretability-oriented fuzzy rule-based classifiers. Appl Soft

Comput 38:118–133

35. Sanz J, Fernandez A, Bustince H, Herrera F (2011) A genetic

tuning to improve the performance of fuzzy rule-based classifi-

cation systems with interval-valued fuzzy sets: degree of igno-

rance and lateral position. Int J Approx Reason 52(6):751–766

36. Sanz JA, Fernandez A, Bustince H, Herrera F (2013) IVTURS: a

linguistic fuzzy rule-based classification system based on a new

interval-valued fuzzy reasoning method with tuning and rule

selection. IEEE Trans Fuzzy Syst 21(3):399–411

37. Schaefer CF, Anthony K, Krupa S, Buchoff J, Day M, Hannay T,

Buetow KH (2009) PID: the pathway interaction database.

Nucleic Acids Res 37(suppl 1):D674–D679

38. Sheng W, Bao Y (2013) Fruit fly optimization algorithm based

fractional order fuzzy-PID controller for electronic throttle.

Nonlinear Dyn 73(1–2):611–619

39. Shi Y, Eberhart RC (1999) Empirical study of particle swarm

optimization. In: Proceedings of the 1999 Congress on Evolu-

tionary Computation, vol 3. p 1945–1950

40. Singh S, Olugu EU, Fallahpour A (2014) Fuzzy-based sustainable

manufacturing assessment model for SMEs. Clean Technol

Environ Policy 16(5):847–860

41. Stepnicka M, Burda M, Stepnickova L (2015) Fuzzy rule base

ensemble generated from data by linguistic associations mining.

Fuzzy Sets Syst 285:140–161

42. Sugeno M (1985) An introductory survey of fuzzy control. Inf Sci

36(1):59–83

43. Tsakiridis NL, Theocharis JB, Zalidis GC (2016) DECO 3 R: a

differential evolution-based algorithm for generating compact

fuzzy rule-based classification systems. Knowl Based Syst

105:160–174

44. Wang L, Shi Y, Liu S (2015) An improved fruit fly optimization

algorithm and its application to joint replenishment problems.

Expert Syst Appl 42(9):4310–4323

45. Zheng X-L, Wang L (2016) A two-stage adaptive fruit fly opti-

mization algorithm for unrelated parallel machine scheduling

problem with additional resource constraints. Expert Syst Appl

65:28–39

Neural Comput & Applic (2019) 31:873–885 885

123