Multi-ObjectiveAnt Colony OptimizationTECHNI SCHEUNI VERSI TTDARMSTADTniversidad deranadaUGManuel Lpez-Ibez InfanteOctober 2003 July 2004Supervisors:Luis PaqueteThomas SttzleLicenseMulti-objectiveAntColonyOptimization.DiplomaThesisbyManuelLopez-Iba nezCopyright c _ 2004 Manuel Lopez-Iba nez. All rights reserved.This work is licensed under the Creative Commons AttributionLicense.To view a copy of this license, visit http://creativecommons.org/licenses/by/2.0/orsendalettertoCreativeCommons, 559NathanAbbottWay,Stanford, California 94305, USA.Youarefree:to copy, distribute, display, and perform the workto make derivative worksto make commercial use of the workUnderthefollowingconditions:Attribution. You must give the original author credit.For any reuse or distribution, you must make clear to others the licenseterms of this work.Any of these conditions can be waived if you get permission from thecopyright holder.Your fair use and other rights are in no way aected by the above.This is a human-readable summary of the Legal Code(http://creativecommons.org/licenses/by/2.0/legalcode).PrefaceRealworldproblemsusuallyinvolveseveralandoftenconictiveobjectivesthat must be simultaneously optimized in order to achieve a satisfactory solu-tion. Multi-objective optimization has its roots at the end of the nineteenthcentury, in the studies of Edgeworth and Pareto, but had not experimentedagreatdevelopmentuntil thefties. Sincelastdecade, thereisanongo-ing research eort to develop approximate algorithms and metaheuristic ap-proaches to solve multi-objective problems. As a result, various metaheuris-tic approaches have been applied to multi-objective optimization,includingsimulatedannealing(SA), taboosearch(TS)andevolutionaryalgorithms.Nevertheless, therearemanyimportantopenquestionsinmulti-objectiveoptimization, e.g., theadequateperformanceassessmentof multi-objectiveoptimization algorithms.Ant Colony Optimization (ACO) is a metaheuristic approach applied suc-cessfully to single objective combinatorial problems, but few work has beendone to apply ACO principles to multi-objective combinatorial optimization(MOCO) and most of the proposed algorithms are only applicable to problemswheretheobjectivescanbeorderedaccordingtotheirimportance. There-fore, itisstill unclearhowACOalgorithmsfortheseproblemsshouldbedesigned.Thisworkexaminestheconceptsinvolvedonthedesignof ACOalgo-rithmstotackleMOCOproblems, e.g., denitionof multi-objectivepher-omone information, multiple cooperative ant colonies, pheromone updatestrategiesinthemulti-objectivecontextandlocalsearchmethodsformul-tipleobjectives. Mostof theseconceptshavebeenpreviouslyusedintheliterature. Nevertheless, weexaminethemindependentlyofanyparticularproblem and we propose alternative concepts still not considered. Moreover,we study these concepts in relation to the available knowledge about the bestperforming ACO algorithms for single objective optimization and the searchstrategies currently used in multi-objective optimization. As well, these con-cepts are studied as components of a general multi-objective ACO (MO-ACO)algorithm.Next, weapplytheseconceptstodesignanACOalgorithmforthebi-objective case of a recently proposed multi-objective variant of the quadraticiv Prefaceassignment problem (QAP). Therefore, we consider various congurations ofthe MO-ACO algorithm and compare their performance when applied to thebi-objective QAP. These congurations use the //A-/1^Ant System asthe underlying algorithm because it is considered the best performing ACOalgorithmforthesingleobjectiveQAP. Inaddition, theparticularcompo-nents of each conguration are chosen in such a way that the congurationreects a particular search strategy. This search strategy is either based ondominance criteria or based on scalarizations of the objective vector. To ad-dress the performance assessment of these experiments, we use a methodologythat combines up-to-date ideas from the research area of multi-objective op-timization with well-known statistical techniques from experimental design.The experimental results show that the use of local search with MO-ACOisessential forthebi-objectiveQAP. Moreover, otherparametersbecomeless signicant when local search is applied. The underlying search strategyfollowedbyeachcongurationplaysanimportantroleintheshapeofthePareto set obtained by the algorithm.The organization of concepts of MO-ACO into modular components whichcanbecombinedintovariouscongurationsallowsapplyingMO-ACOtootherMOCOproblems. TheavailableknowledgeofsingleobjectiveACO,multi-objective optimization and about particular problems may be used todesignnewcomponents, e.g., dierentlocal searchmethods, whichcanbecombined into congurations of MO-ACO in order to tackle dierent MOCOproblems.vThis document is structured as follows:Chapter1introduces the basic concepts of multi-objective optimization interms of Pareto optimality.Chapter2reviewsthesingleobjectiveQAP,describingtheproblemde-nition and briey the current knowledge about types of QAP instancesand related measures. Afterward, this chapter introduces the proposalson multi-objective QAP, and nally it describes the bi-objective QAPinstances used as test instances.Chapter3gives a brief introduction to Ant Colony Optimization, the ap-plicationofACOtothesingleobjectiveQAPandthe //A-/1^Ant System.Chapter4examines the design of multi-objective ACO algorithms. Partic-ularly, we discuss the use of multiple colonies, several pheromone updatestrategies and dierent search strategies. Finally, several congurationsare considered in order to tackle the bQAP.Chapter5describestheassessmentmethodologyfollowedtoevaluatethecongurations considered to tackle the bQAP.Chapter6analyzes the results of the experiments performed with respectto the assessment methodology.Chapter7concludes and indicates future research directions.ContentsPreface iii1 Multi-objectiveOptimization 11.1 Multi-objective Optimization Problems . . . . . . . . . . . . . 11.2 Pareto Optimality . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Relations between Pareto Sets . . . . . . . . . . . . . . . . . . 31.4 Weighted Sum Scalarization. . . . . . . . . . . . . . . . . . . 41.5 Multi-objective Combinatorial Optimization Problems . . . . 42 TheQuadraticAssignmentProblem 72.1 The Single Objective QAP . . . . . . . . . . . . . . . . . . . . 72.1.1 Types of QAP Instances. . . . . . . . . . . . . . . . . 82.1.2 Measures of QAP Instances . . . . . . . . . . . . . . . 92.2 The Multi-objective QAP (mQAP). . . . . . . . . . . . . . . 92.2.1 Bi-objective QAP (bQAP) Instances . . . . . . . . . . 103 AntColonyOptimization 133.1 ACO Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Ant System Applied to the QAP . . . . . . . . . . . . . . . . 153.3 Improvements on AS. . . . . . . . . . . . . . . . . . . . . . . 163.4 //A-/1^Ant System. . . . . . . . . . . . . . . . . . . . 174 Multi-objectiveACO 194.1 ACO Algorithms for MOCO Problems . . . . . . . . . . . . . 194.2 Multi-objective Pheromone Information . . . . . . . . . . . . 214.2.1 Multiple Pheromone Information . . . . . . . . . . . . 224.2.2 Single Pheromone Information . . . . . . . . . . . . . 254.2.3 Computational Eciency . . . . . . . . . . . . . . . . 264.3 Pheromone Update Strategies . . . . . . . . . . . . . . . . . . 274.4 Multiple Colonies. . . . . . . . . . . . . . . . . . . . . . . . . 294.4.1 Weight Vectors in the Multi-colony Approach. . . . . 304.4.2 Candidate Set in the Multi-colony Approach . . . . . 314.4.3 Pheromone Update Strategies with Multiple Colonies 314.5 Local Search Methods for MO-ACO . . . . . . . . . . . . . . 33viii Contents4.5.1 Local Search for Single Objective Problems . . . . . . 344.5.2 Pareto Local Search (PLS) . . . . . . . . . . . . . . . . 344.5.3 Bounded Pareto Local Search (BPLSA) . . . . . . . . 344.6 MO-ACO Applied to the bQAP . . . . . . . . . . . . . . . . . 355 PerformanceAssessment 395.1 Binary-measure . . . . . . . . . . . . . . . . . . . . . . . . . 395.2 Unary-measure . . . . . . . . . . . . . . . . . . . . . . . . . 415.2.1 Lower Bound . . . . . . . . . . . . . . . . . . . . . . . 415.2.2 Analysis of Variance (ANOVA) . . . . . . . . . . . . . 425.3 Median Attainment Surface . . . . . . . . . . . . . . . . . . . 435.4 Reference Solutions . . . . . . . . . . . . . . . . . . . . . . . . 446 Experiments 456.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . 456.2 Analysis of Experimental Results . . . . . . . . . . . . . . . . 466.2.1 Analysis Based on Binary-measure . . . . . . . . . . 466.2.2 Analysis Using Unary-measure and ANOVA . . . . . 526.3 Median Attainment Surfaces . . . . . . . . . . . . . . . . . . 577 Conclusions 637.1 Multi-objective ACO. . . . . . . . . . . . . . . . . . . . . . . 637.2 MO-ACO Applied to the bQAP . . . . . . . . . . . . . . . . . 647.3 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . 65AANOVAAssumptions 67Bibliography 73ListofFigures3.1 Algorithmic schema of ACO algorithms . . . . . . . . . . . . 155.1 Pictorial view of the lower bound for the bQAP. . . . . . . . 426.1 Lower bounds. . . . . . . . . . . . . . . . . . . . . . . . . . . 536.2 Interactions: unstructured ( = 0.75) instance, using LS . . . 556.3 Main factors: unstructured ( = 0.00) instance, using LS . . . 566.4 Interactions: unstructured ( = 0.00) instance, using LS . . . 566.5 Interactions: unstructured ( = 0.75) instance, using LS . . 576.6 Median attainment surfaces: unstructured instances . . . . . 606.7 Median attainment surfaces: structured instances. . . . . . . 61A.1 ANOVA assumptions: unstructured instances, using LS . . . 69A.2 ANOVA assumptions: unstructured instances, not using LS . 70A.3 ANOVA assumptions: structured instances, using LS. . . . . 71ListofTables2.1 bQAP unstructured instances . . . . . . . . . . . . . . . . . . 112.2 bQAP structured instances . . . . . . . . . . . . . . . . . . . 114.1 Search strategies when applying MO-ACO to the bQAP. . . 384.2 Congurations of the MO-ACO algorithm for the bQAP. . . 386.1 Results using LS methods vs. not using LS methods . . . . . 476.2 Results using Sall vs. using Sone. . . . . . . . . . . . . . . . . 486.3 Results using class D vs. using class S . . . . . . . . . . . . . 496.4 Results using iteration-bestvs. using best-so-farstrategies . . 506.5 Results using 1 colony vs. using 5 colonies . . . . . . . . . . . 516.6 ANOVA: unstructured ( = 0.75) instance, using LS . . . . . 556.7 ANOVA: unstructured ( = 0.00) instance, using LS . . . . . 556.8 ANOVA: unstructured ( = 0.75) instance, using LS . . . . 566.9 ANOVA: unstructured ( = 0.75) instance, not using LS. . . 586.10ANOVA: unstructured ( = 0.00) instance, not using LS. . . 586.11ANOVA: unstructured ( = 0.75) instance, not using LS. . 586.12Empirical correlation of bQAP instances . . . . . . . . . . . . 59Chapter1Multi-objectiveOptimizationManyoptimizationproblemsarisinginpracticeinvolvemorethanoneoptimizationcri-terion,requiringthesimultaneousoptimizationofseveral,perhapsconicting,objectivefunctions. Hence, a single optimal solution is not as interesting as information about thetrade-obetweenthevariousobjectives. Inmulti-objectiveoptimization, theabstractconcept of trade-o is usually dened in terms of Pareto optimality. Thus, the goal is toobtainthesetofall Paretooptimal solutionsor,atleast,agoodapproximationofthisset. Analternativemethodwouldbetotransformthemulti-objectiveproblemintoasingleobjectiveproblemusingaweightedsumofthemultipleobjectives. Then,Paretooptimal solutionscanbefoundbysolvingseveral ofsuchsingleobjectiveproblemsus-ingvariousweights. However, ifcertainconditionsarenotmet, notall Paretooptimalsolutionswill befoundbymeansof suchmethod. Inthisworkweareonlyinterestedin a certain class of multi-objective problems, multi-objective combinatorial optimizationproblems.1.1 Multi-objectiveOptimizationProblemsAsitsverynamesuggests, inamulti-objectiveoptimizationproblem(alsocalled multi-criteria optimization and vector optimization problem) every so-lutionismeasuredaccordingtomorethanoneobjectivefunction, eachofwhich must be minimized or maximized. LetQ be the number of objectivesinamulti-objectiveproblem, let obethesetof feasiblesolutionsandletZdenotethesetof all objectivevaluevectorsof feasiblesolutions. Eachelement ofZis a vector referred to as objective vectorformed by the valuesof the objective functions. Formally,s o, z Z, z = (z1, . . . zQ):___z1 =f1(s)...zQ=fQ(s)The feasible set ois a subset of a space called the searchspace. The spacefrom which objective values are taken is called the objective space.2 Multi-objectiveOptimizationLet us assume that allQ objectives should be minimized, without loss ofgenerality. Thus, the goal of multi-objective optimization is to solveminsS(f1(s), . . . , fQ(s)) (1.1)where the meaning of min, i.e., the concept of optimality in multi-objectiveoptimization, will be dened in the following section.1.2 ParetoOptimalityGiven two objective vectorsu, v Z :u = (u1, . . . , uQ),v = (v1, . . . , vQ), weneed to dene whether u is better than v in order to nd an optimal objectivevector which is the solution to the multi-objective problem described above.Whenobjectivescanbeorderedaccordingtotheirimportance, optimalitycan be dened in terms of lexicographic order. Let us suppose objectives arearranged in decreasing order of importance, i.e.,q = 1 is the most importantobjective andq =Q is the least important one. Then, u 0, the probability of choosing a specic solution component is neverzero.Lastly, thepheromonetrailsareinitializedtomax, toachieveahigherexploration of solutions at the start of the algorithm. Actually, all the pher-omone trails would be initialized to some arbitrarily high value,ij(0) = ,andaftertherstiterationwillbesettoij(1)=max(1), beingforcedtotake values within the imposed pheromone limits.Empirical results given in [51] show that //AS is among the best avail-able algorithms for the QAP, where it is compared with HAS-QAP [23] (otherACOalgorithm), Robust TabooSearch(RoTS) [52] andagenetichybrid(GH)methodwhichusesshorttaboosearchrunsforthelocal search[19].//AS outperforms HAS-QAP for all classes of instances and RoTS and GHfor structured instances and achieves a similar performance for unstructuredinstances (see Section 2.1.1 on page 8 for a review of the classes of instancesof the QAP).Anotherstudydescribedin[40]compared //ASwithRobustTabooSearch(RoTS) [52], ReactiveTabooSearch(ReTS) [1], Fast Ant Colony(FANT)incorporatinglocal search[54, 55], SimulatedAnnealing(SA)[5]18 AntColonyOptimizationand a memetic algorithm described by the authors. The results showed thatfor unstructured instances, //AS achieves a similar performance asRoTSandReTSandoutperforms all theothers algorithms. For structuredin-stances, //AS outperformed all algorithms except the ne-tuned memeticalgorithm.Consequently, it makes sense to use the concepts of //AS in order todesign a multi-objective ACO algorithm to tackle the bQAP.Chapter4Multi-objectiveACODespitepreviousresearcheortsontheapplicationofACOprinciplestomulti-objectiveoptimization, very few studies have actually tackled MOCO problems dened in terms ofPareto optimality. To address the design of a multi-objective ACO algorithm (MO-ACO)forthistypeof problem, several conceptsmustbereviewed. Themanagementof thepheromone information in MOCO turns out to be a complex task that involves the de-nition of the pheromone information, how dierent pheromone information is aggregatedusingweights, whichsolutionsareselectedtoupdatethepheromoneinformationandhowthesesolutionsmodifythepheromoneinformation. Inaddition,someauthorspro-posetheuseof multiplecolonies, sothat eachcolonyweighs dierentlytherelativeimportanceofthemultipleobjectives. Whenmultiplecoloniesareconsidered,theman-agement of thepheromoneinformationisevenmorecomplicated. Finally, theuseoflocal search methods must be considered. All these features can be seen as componentsof a certain conguration of a general MO-ACO algorithm. With regard to the bQAP weconsidercongurationsthatmayberelatedtotwoessentiallydierentsearchstrategiesused to tackle MOCO problems with approximate algorithms, namely (i ) based on dom-inancerelationsor(ii )basedonseveral scalarizationsof theobjectivevector. Finally,knowledgeof thebestperformingACOalgorithmsfor singleobjectiveoptimization,e.g.,ACSand //AS,canbetransferredintothemulti-objectivecontextusingequivalentprinciplestodesignMO-ACOalgorithms.4.1 ACOAlgorithmsforMOCOProblemsRelatively few approaches of ACO for MOCO problems have been proposedsofar. Moreover, manyof theproposedalgorithmsareonlyapplicabletoproblems where a lexicographic ordering of the objectives is given, i.e., wherethe objectives can be ordered according to their importance [22, 26, 39].Gambardella, Taillard & Agazzi [22] studied a bi-objective vehicle routingproblem with time window constraints, where the rst objective, the numberof vehicles, is consideredtobemoreimportant thanthesecondone, thetotal travel time. Theyproposedanalgorithmthatusestwoantcolonies,20 Multi-objectiveACOoneforeachobjective. Acommonbest-so-farsolutionisusedtoupdatethe pheromone information in both colonies. The rst colony tries to nd asolution with one vehicle less than the best-so-far solution while the secondcolonytriestoimprovethebest-so-farsolutionwithrespecttothesecondobjective. Whenever the rst colony nds a better solution with respect tothe rst objective, it becomes the new best-so-far solution.Gravel, Price & Gagne [26] tested an ACO algorithm for solving a singlemachinetotal tardinessproblemwithchangeovercostandtwoadditionalobjectives, which arises in a real-world scheduling problem for an aluminumcastingcenter. Intheirapproach, theheuristicinformationisconstructedby taking all the objectives into account, however for the pheromone updateonly the most important objective is considered.Mariano & Morales [39] studied a multi-colony approach for the design ofwater irrigation networks where for each objective there exists one colony ofantsandeveryobjectiveisdeterminedknowingonlytherelevantpartofasolution. An order is imposed on the colonies corresponding to the order ofimportance of the multiple objectives. In every generation, each ant from acertain colony receives a partial solution from the previous colony and triestocompletethepartialsolutionwithrespecttotheobjectiveassociatedtothatcolony. Whenthelastcolonycompletesthesolutionconstruction, allthe nondominated solutions are used to update the pheromone information.None of the above approaches can be applied to problems where the ob-jectives cannot be ordered according to their importance. A notable excep-tion is the work of Iredi, Merkle & Middendorf [29], who tested a multi-colonyACO algorithm for a bi-objective single machine total tardiness problem withchangeover costs. They proposed using onepheromone matrix for each ob-jective, that is, two pheromone matrices. For the solution construction, eachant uses a weight to combine the two pheromone matrices and to combine thedierent heuristic information of each objective. In order to weigh the rela-tive importance of each objective dierently, each ant uses a dierent weight(but the same one at each iteration). Each colony has its own two pheromonematrices and set of weights and,thus,can focus the search on approximat-ing to a certain region of the optimal Pareto set. Nondominated solutionsfoundinthecurrentiterationareusedtoupdatethepheromonematrices.Collaboration between the colonies is achieved by using the solutions of othercolonies to detect dominated solutions and by using nondominated solutionsfrom other colonies to update the pheromone matrices.In the following sections, we examine the concepts involved in the designof anACOalgorithmforMOCOproblemsintermsof Paretooptimality.Manyof theseconceptshavebeenconsideredpreviouslyintheliterature.Nevertheless, we would like to formalize these concepts so that they can beapplied to any MOCO problem. In addition, we discuss alternatives not yetproposedandweseektorelatethesealternativestotheconceptsfoundinthe literature. All of these conceptscan be seen as parameters of a certain4.2.Multi-objectivePheromoneInformation 21congurationofageneral multi-objectiveACO(MO-ACO)algorithm. Fi-nally,westudypossiblecongurationsofthisMO-ACOalgorithminorderto tackle the bi-objective QAP (bQAP).4.2 Multi-objectivePheromoneInformationPerhaps the most important step when designing an ACO algorithm for a spe-cic problem is the denition of the pheromone and heuristic information.1ACOalgorithmsforsingleobjectiveproblemsrepresentthepheromonein-formation by a pheromone matrix (or vector) where each entry in the matrixcorresponds to the desirability for a certain solution component. Thus, howthe pheromone information is actually represented depends on the denitionof solutioncomponents. AgoodexampleforillustratingthisisthesingleobjectiveQAP(cf.Section3.2onpage15),whereasolutioncomponentisdened as the assignment of a facility j to a location i in the current solution, thus each entry ij of the pheromone matrix corresponds to the desirabilityfor settingi toj(i = j).In contrast, each objective in a MOCO problem may dene the solutioncomponents dierently. For instance, let us consider a bi-objective schedulingproblem, where the rst objective depends on which position of the schedulea certain job is placed, whereas the second objective depends on the relativeposition of a certain job with respect to the previous jobs.2In this case, so-lution components for the rst objective can best be dened as assignmentsof jobstopositionsintheschedule, whereasforthesecondobjectivetheyare dened depending on the predecessor/successor relationship among jobs.Therefore, given a feasible solution to this problem, which represents a cer-tain sequence of jobs, the pheromone information for the rst objective is thedesirability that job j is on place i of the schedule (i = j), whereas the phero-mone information for the second objective corresponds to the desirability thatjob j comes immediately after job h in the schedule (i = j i1 = h). Thepheromone information for both objectives cannot be expressed as one pher-omone matrix because an entry ij of the pheromone matrix cannot representthedesirabilityforbothsolutioncomponentsatthesametime. Moreover,if thetotal numberof jobsislargerthanthenumberof jobsthatcanbescheduled at one time, then the dimension of the pheromone matrix for therst objective is dierent from the one for the second objective.On the other hand, for the multi-objective QAP (mQAP), the pheromoneinformationforall theobjectivescorrespondstothedesirabilitythatafa-cilityjisassignedtolocationiinthecurrentsolution(i=j), thatis,1Inthefollowingsectionswemainlystudythedenitionofthepheromoneinformation.Yet,theconclusionscanalsobeappliedtotheheuristicinformation.2This is the case, for example, for the Single Machine Total Tardiness ProblemwithChangeoverCoststhatisdescribedin[29].22 Multi-objectiveACOall objectives dene the solution components in a unique manner. Hence, inother multi-objective optimization problems, solution components and, con-sequently, pheromone information may have a single meaning.In general, in the case of MOCO problems, there may be as many dierentmeanings for the pheromone information as the number of objectives.In order to represent the pheromone information for a MOCO problem, aninitial approach may be to keep the pheromone information for only one of theobjectives. Then, solution components are dened only for this objective andthepheromoneinformationisrepresentedconsequentlyasif thisobjectivewastheonlyoneconsideredintheproblem. ThisistheapproachusedbysomeACOalgorithms[22, 26] forMOCOproblemswhereoneobjectiveisconsidered more important than the others. As would be expected, when onlyone pheromone matrix is considered for a certain objective, solutions obtainedare better for that objective to the detriment of the remaining objectives [29].Thus, thisapproachisnotadequatewithoutapriori knowledgeabouttherelative importance of the dierent objectives.Forthisreason,wewilldiscusstwodierentapproachestodeningthepheromoneinformationwhenobjectivescannotbeorderedbyimportance.Therstone, proposedbyIredietal.[29], usesmultiplepheromoneinfor-mation, such that the pheromone information is dened dierently for eachobjective, whereasinthesecondapproach, weproposethedenitionof asingle pheromone information for all the objectives simultaneously.4.2.1 MultiplePheromoneInformationWewillassumethataMO-ACOalgorithmusesmultiplepheromoneinfor-mationwhendierentpheromoneinformationisdenedforeachobjectiveand weightsare used to aggregate them into a single value, in a similar waythat weightsare used toaggregate dierent objectives in theweighted sumscalarization of a multi-objective problem (cf. Section 1.4 on page 4).Since each objective may dierently dene the solution components, dif-ferent values may be needed by an ant in order to dene the possible solutioncomponentsthatcanbeaddedtoitspartialsolution. Thesevaluescanbeconsideredascomponentsofastatevector whichdenesthecurrentstateofthepartialsolutionbeingconstructedbyanant. InthisstatevectorS,two objectives (q and r) with dierent denitions of the solution componentswill have dierent values on their corresponding elements of the state vector(Sq ,= Sr), whereas if they have the same denition of a solution componentthen they will have the same value (Sq=Sr). As well,an objectiveqmaydenethesolutioncomponentsonlyintermsof thenextelement j tobeadded to the partial solution. Consequently, their corresponding element ofthe state vector will be null (Sq = ).Let us assume an arbitrary MOCO problem withQ objectives. Then, an4.2.Multi-objectivePheromoneInformation 23antk addsjto its partial solution with the following probability:pkSj =_Q

q=1_qSqj_q_

_Q

q=1_qSqj_q_

lNkS__Q

q=1_qSql_q_

_Q

q=1_qSql_q__ ifj ^kS(4.1)where S = S1, . . . , SQ is the state vector of the current partial solution, andeach elementSqof this vector is value needed by each objectiveq = 1, . . . , Qtodeneasolutioncomponent;^kSisthefeasibleneighborhoodof ant kgiven the current state vector S; qsqj is the pheromone information of j givenSqfortheqthobjectiveinthecurrentiterationand, inanequivalentway,qSqjistheheuristicinformationforjgivenSqthatcorrespondstotheqthobjective. Finally, qisavaluethatweighstherelativeimportanceoftheqthobjectiveanditcanbeseenastheqthcomponentofaweightvectortaken from the set , as dened in (1.2) on page 4. Thus, whenq= 0 theqthobjective is not considered and when q = 1 it is the only one considered.For instance, in the bi-objective scheduling problem described above, eachsolutioncomponent for therst objectiveis anassignment of ajobtoaposition, whereasforthesecondobjectiveitisanassignmentof ajobtothepreviousjobscheduled. Thus, inordertodecidewhichthenextjobjscheduledis,anantneedstoknowatwhichpositionithejobshouldbescheduled and which the previous job h is. Therefore, in this case, the currentstate of a partial solution isS = i, h. Following Eq. (4.1) withQ = 2, anantk adds a jobjto its current schedule with probability:pk{i,h}j =__1ij_(1)

_2hj__

__1ij_(1)

_2hj__

lNk{i,h}___1il_(1)

_2hl__

__1il_(1)

_2hl___ ifj ^k{i,h}where ^k{i,h}isthefeasibleneighborhoodofantk, thatis, thosejobsthathave not been already scheduled;1ijand1ijare the pheromone and specicheuristicinformationfortherstobjective; 2hjand2hjarethepheromoneandspecicheuristicinformationforthesecondobjective; andnally, [0, 1] is a value that allows the weighing of the relative importance of the twoobjectives dierently.In the case of the mQAP, the solution components for all objectives aredened as assignments of facilities to locations. Thus, in order to decide thenextfacilityjtoplace, anantonlyneedstoknowatwhichlocationithefacility is going to be placed, therefore S = i. For simplicity, let us restrictourselvestothebi-objectivecase(Q=2). FollowingEq. (4.1), anant k24 Multi-objectiveACOplaces facilityjwith probability:pkij =__1ij_(1)

_2ij__

__1ij_(1)

_2ij__

lNki___1il_(1)

_2il__

__1il_(1)

_2il___ ifj ^ki(4.2)where ^kiis the feasible neighborhood of ant k, that is, those locations whichare still free; 1ij and 2ij are the pheromone information in the current iterationforeachof thetwoobjectivesof thebQAP; 1ijand2ijareQAPspecicheuristicinformationforeachof thetwoobjectives; andnally, asbefore, [0, 1] is a value that allows us to weigh the relative importance of the twoobjectives dierently.Equation(4.2)illustratesapotential issuewhenusingmultiplephero-moneinformationforproblemswhereall theobjectivesdenethesolutioncomponents in the same manner. In this type of problem, when updating themultiplepheromonematrices, if thesolutioncomponentsreceivethesameamountofpheromoneindependentlyoftheparticularobjectivethenitoc-curs that1ij = 2ij = , and thus the aggregation of the pheromone matricesis useless since (1) = . Therefore, when multiple pheromone items aredenedintermsofequivalentsolutioncomponents, thepheromoneupdatestrategy must ensure that the multiple pheromone items are in fact dierent.We will address this issue when discussing the pheromone update strategies(Section 4.3 on page 27).Finally, an important issue is the denition of the weight vector. Multiplepheromoneinformationisaggregatedintoasinglevalueusingweights, ormore formally a weight vector as in Eq. (4.1). This weight vector regulatesthe relative importance of the dierent objectives. We are not interested inobtaining a single solution but a Pareto set. Thus,we cannot use only oneweight vector but a nite set of maximally dispersed weight vectors [47],i.e., the weight vectors in are as much distributed as possible in the innitespace dened by .Iredi et al. [29] proposed assigning each weight vector k to a dierentantk. Thus,thewholesetofweightvectorsisusedateveryiteration. Aswell, intheACOalgorithmsweareconsidering(ASand //AS), atanygiven iteration each ant constructs its solution independently from the others.Therefore, it does not matter the order in which weights are assigned to ants.Analternativemethodwouldbetoassigneachweightvectort toadierent iterationt. In this case, all the ants use the same weight at a giveniterationandadierentweightinthefollowingiteration. Sinceantsfromagiveniterationarenotindependentfromthoseinthepreviousiteration,itmakessenseinthiscasetoassigneachweighttoeachiterationinsuchanorderthattwoconsecutiveweightvectorsdieronlyby zinanytwocomponents [47], wherez is the minimum dierence between any two values4.2.Multi-objectivePheromoneInformation 25of any component for all the weight vectors in .Theweightvectorcanbeseenasasearch direction intheobjectivespace. Then, the rst method searches in all possible directions at each itera-tion, while the second method tries to nd a good solution in one direction atone iteration and using this solution as a starting point in the next iterationbut in a slightly dierent direction. The best performing method will dependon the underlying multi-objective landscape [31].4.2.2 SinglePheromoneInformationWe will assume that a MO-ACO algorithm uses singlepheromoneinforma-tionwhen the pheromone information is dened in a unique manner for allobjectives and thus weights are not needed. The problem once again is howto dene the pheromone information for those objectives where the solutioncomponentshaveadierentmeaning. Theapproachtosolvethisproblemwill besimilartousingmultiplepheromoneinformation, thatis, touseastatevector Sthatdenesthecurrentstateof thepartial solutionbeingconstructed by an ant.Therefore, when using single pheromone information, the pheromone in-formationofasolutioncomponentisexpressedasjSforalltheobjectives,which is the desirability forj given the stateS. Then, an antk addsj to itspartial solution with the following probability:pkS,j =_Sj

_Sj

lNkS__Sl

_Sl_ ifj ^kS(4.3)where[Sj] isamulti-dimensional matrixwithasmanydimensionsasele-ments of the state vectorS, namely the number of ways that solution com-ponents may be dened by the objectives, plus one more dimension forj.For the bi-objective scheduling problem, the state vector contain the posi-tion i where a new job will be scheduled and also the job h that was scheduledin the previous position, S= i, h. Therefore, the single pheromone infor-mation is represented by a pheromone matrix [ihj] with dimensions I J Jwhere Iis the size of the schedule and Jis the number of jobs. Then, apply-ing Eq. (4.3) withQ = 2, an antk adds a jobjto its current schedule withprobability:pkihj =_ihj

_ihj

lNkih__ihl

_ihl_ ifj ^kihIn the case of the mQAP, this approach is even more natural, since a so-lution component has a unique meaning for all objectives and the state vectorhasonlyoneelement, i.e., thelocationiwherethenextfacilityshouldbe26 Multi-objectiveACOplaced,S = i. Therefore, the single pheromone information is representedbyapheromonematrix[ij]asinASand //ASforthesingleobjectiveQAP.4.2.3 ComputationalEciencyAspectsofMulti-objectivePheromoneInformationInterms of computational resources required, at rst it might seemthatusingsinglepheromoneinformationisveryexpensivebecauseoftheuseofamultidimensional matrix, buttherearesomesituationsinwhichtheuseof single pheromone information is more ecient in terms of both space andcomputation time required than the use of multiple pheromone information.From the point of view of required space,single pheromone informationis represented with a multi-dimensional matrix with as many dimensions asthe number of manners that solution components may be dened by theQobjectives, plus one more dimension. This number is equivalent to the size ofthe state vector 1 [S[ Q plus one. Therefore, the single pheromone infor-mation leads to a memory requirement of O_n|S|+1_, wheren is the largestdimension ofthepheromonematrix. In theworst casealltheQobjectivesdene the solution components dierently and then, the single pheromone in-formation is represented with aQ-dimensional matrix, leading to a memoryrequirement of O_nQ+1_. On the other hand,multiple pheromone informa-tion is represented in any case with Q dierent matrices, leading to a memoryrequirementof O_Q(nn)_. Howeverinsomeproblems,aswehaveseenfor the mQAP, the number of meanings of the solution components may besmallcomparedtothenumberofobjectives,andthereforethememoryre-quirements for using multiple pheromone information are much higher thanfor using single pheromone information.Withregardtocomputationtime, whenmultiplepheromoneinforma-tionisused, thecalculationof pkSjimpliestheevaluationof Qq=1_qSqj_q.Consequently, the computation time required is O(Q). Moreover, the powerand the product are very expensive operations in terms of computation time.Hence, therewouldbealargemultiplicativeconstanthiddenintheprevi-ousexpression. Ontheotherhand, whensinglepheromoneinformationisconsidered, thereisonlyonepheromonevalue, thustheevaluationtakesaconstant time without expensive operations. As well, the evaporation proce-dure implies that all the entries of the pheromone matrices must be modied.Therefore, when single pheromone information is used, the computation timeof the evaporation procedure is O_n|S|+1_ where n is again the largest dimen-sion of the pheromone matrix,whereas for multiple pheromone informationthe computation time is O_Q (nn)_.In conclusion, which denition of the multi-objective pheromone informa-tion actually requires fewer computational resources depends on the numberofobjectivesoftheparticularproblembeingconsideredandhowtheseob-4.3.PheromoneUpdateStrategies 27jectives dene the solution components.4.3 PheromoneUpdateStrategiesThe best performing ACO algorithms for single objective problems typicallyupdatethepheromoneinformationbyusingonlythebestorafewof thebest solutions found in the current iteration (iteration-beststrategy) or sincethe start of the algorithm (best-so-farstrategy) [11, 51]. In multi-objectiveproblems, the best solutions can also be taken from a candidate set includingall solutions found in the current iteration (ibor since the start of the algo-rithm (bf. The real diculty lies in the denition of the best solutions of thecandidate set.With a single objective, the comparison criterion between solutions is evi-dent because the best solution is that one with the best value of the objectivefunction. Ontheotherhand, withmultipleobjectives, thestraightforwardcriterion is the Pareto optimality and thus the best solutions of the candidateset(either (ibor (bf)arethosethatarenondominated, orinotherwords,those solutions that belong to the Pareto set. We will refer to this strategyas selection by dominance.When using selection by dominance and multiple pheromone information,if each objective denes the solution components in a dierent manner, theselectedsolutionsupdateeachpheromonematrixdierentlybecauseeachsolutioncomponenthasadierentmeaningforeverypheromonematrix.3However, a problem arises when dierent pheromone matrices dene the so-lution components in the same manner because they will be reinforced equally,becoming the same matrix, causing the aggregation to turn out to be useless.Furthermore, when using selection by dominance and multiple pheromone in-formation, the meaning of each pheromone matrix is no longer the desirabilityof a certain objective, but for a certain set of solution components (which aredened by each pheromone information) and then the weight vector regulatesthe relative importance of dierent solution components.On the other hand, when selection by dominance is used with single pher-omoneinformation, thepheromonetrailsof thesolutioncomponentsthatbelongtonondominatedsolutionsarereinforcedinanatural way. Thereisexactlyoneentryof thesinglepheromoneinformationforeachpossiblesolutioncomponent, thuseachselectedsolutionwill depositpheromoneinthe entries of the single pheromone information associated with its solutioncomponents.Alternatively, we can dene a weight vector and select the best solutionwith respect to a weighted sum scalarization of the multiple objectives. How-ever, this selection by scalarizationis even more problematic because (i) for3This is exactlythe strategyfollowedbythe algorithmproposedbyIredi, Merkle &Middendorf[29].28 Multi-objectiveACOeach weight vector all the solutions are scalarized and compared in order toselect the best solution with respect to the weight vector (ii)it is not clearhow the selected solutions should update the multi-objective pheromone in-formation, particularly when multiple pheromone information is considered.Therefore,wewill restrictourselvestoasimplied formoftheselectionbyscalarization. Inthissimplerstrategy, onlythebestsolutionswithrespectto each objective are selected to update the pheromone information. Then,when multiple pheromone information is considered, each pheromone matrixassociated with each objective is updated by the solution with the best ob-jective value for the respective objective. Consequently, this simplied formof the selection by scalarization is called selection by objective.Selection by objective has two main advantages over selection by scalar-ization. First, only one ant per pheromone matrix will be allowed to depositpheromone as in //AS and ACS, which has led to improved performanceovertheoriginal AS(seeSection3.3onpage16). Asaresult, advancedtechniques used in these algorithms can be easily adapted to multi-objectiveproblems. Second, each pheromone matrix focuses on one objective, thus theaggregation of all of them by means of a weight vector truly regulates the rel-ative importance of each objective. This is not the situation in the selectionbydominancewhenusingmultiplepheromoneinformation. Therefore, theproblemdescribedabovewhichoccurswhenthesolutioncomponentshavethesamemeaningfordierentpheromoneinformationisavoidedwiththisstrategy,because each pheromone matrix is updated by a dierent solutionand thus the pheromone matrices are actually dierent.4In this way, the con-cept of multiple pheromone information can be applied to MOCO problemssimilar to the mQAP.Itmustbenotedthatwhentwosolutionsbothhavethebestvaluefora certain objective, the strategy of selection by objective always chooses theonethatdominatestheother. However, withQ 3itmayhappenthattwonondominatedsolutionshavethebestvalueforanobjective. Inthissituation, either we can choose one of them or we can select both, althoughin the latter case more than one ant is allowed to deposit pheromone and wegive up one of the properties of this strategy.Finally,the amount of deposited pheromone must be dened. The onlyrestriction is that the solution cost cannot be used at all because the valuesofdierentobjectivefunctionsarenotcomparable. Whenmultiplephero-mone information is considered, we must take special care to ensure that theamount of deposited pheromone is independent from the pheromone matrixonwhichit is deposited, otherwisesomeobjectives areimplicitlyconsid-4Of course, itmayhappenthatanantisthebestof thecandidatesetwithrespecttomore than one objective and then it will deposit pheromone in more than one pheromonematrix. Whenthissituationoccursfrequently,weshouldaskourselvesifusingmultiplepheromoneinformationisthecorrectapproachtosolvethatspecicproblem.4.4.MultipleColonies 29ered more important than others. Taking these restrictions into account, theamount of deposited pheromone can be dened in any manner. Finally, withregard to the pheromone evaporation procedure, the only dierence betweenthe multi-objective and the single objective case is the size and the numberof pheromone matrices,thus in principle any method can be used althoughecient techniques would of course be preferred.4.4 MultipleColoniesThe image of the optimal Pareto set in the objective space is a trade-o sur-face between the dierent objectives. Over this surface, two solutions can besaid to belong to dierent regionswith respect to the dierences in their ob-jective vectors, e.g., if the distance between their respective objective vectorsis more than a given value or if we divide the objective space somehow intohyper-boxes such that their objective vectors belong to dierent hyper-boxes.Notice that the solution vector in the search space is not considered, thus thedesirable solution components of two Pareto optimal solutions belonging todierent regions could be very dierent or not. Nevertheless, as the real cor-relation between the objectives decreases,it is to be expected that solutioncomponents that are desirable for minimizing one objective will not be desir-able for minimizing the others. Therefore, it could be advantageous to focusthe search on a certain region, independently from the search in other regions.In a multi-colony approach, the total number of ants is divided into dis-joint sets, each of these sets being called a colony. In the following, we willassumethateverycolonyhasthesamenumberof ants, ormoreformally,thattherearemants, ccoloniesandmcantspercolony. Multi-colonyantalgorithms have been previously used to parallelize ACO algorithms in orderto solve single objective problems. In the multi-objective context, the mainidea behind the multi-colony approach is not parallelization, but the fact thatdierentcoloniescanweightheobjectivesdierently[29]. Inotherwords,eachcolonycanbeseenastryingtoapproximateadierentregionoftheoptimal Pareto set.Each colony is independent from the others in the sense that it has its ownantsandpheromoneinformation, suchthateachantfromacertaincolonyconstructsitssolutionguidedonlybythepheromoneinformationfromitsowncolony. Ifnocooperationtakesplacebetweencolonies,thenwewouldhave a multi-start single colony ant algorithm where at each restart the al-gorithmtriestoapproximateadierentregionof theoptimal Paretoset.Nonetheless, the solutions found by one colony can help to identify weakso-lutions (i.e., dominated solutions) found by other colonies and one colony canndgoodsolutionsbychancewhichbelongtotheregionofothercolonies.Therefore, cooperation between colonies can increase the overall performanceof thealgorithm. Cooperation isachieved byexchangingsolutionsbetween30 Multi-objectiveACOcoloniessotheupdatingof thepheromonesof onecolonyisinuencedbysolutions from other colonies.Three aspects dene the behavior of the multiple colonies: (i)the set ofweight vectors used to aggregate multiple pheromone information can forceeach colony explicitly to approximate a dierent region of the optimal Paretoset, (ii) a candidate set formed by solutions from all colonies enforces cooper-ation between colonies, and (iii) the pheromone update strategy must selectthe colony where each solution updates the pheromone information.4.4.1 WeightVectorsintheMulti-colonyApproachThe use of single pheromone information does not in itself force each colonyto focus on a certain region. When multiple pheromone information is used,on the other hand, the set of weight vectors that each colony uses in order toaggregateitsmultiplepheromoneinformationdenesinsomewayaregionin the objective space on which the colony focuses the search.Theinnitesetdenesall thepossibledirectionsthatcanbetakentoapproximatetheoptimal Paretoset. Anynitesubsetof maximallydispersedweight vectors denes a region for the whole optimal Pareto set.5Thus, a partition of denes regions in the optimal Pareto set that can beeither disjoint or overlapping regions depending on whether disjoint partitionsof are considered or not. Then, the multiple colonies can use the same set, disjoint or overlapping partitions of it.For the bi-objective case, a single value is enough to dene each weightvector 1 , . Then,given the number of coloniesc and the number ofweight vectors per colonyw, Iredi et al. [29] proposed the following possibil-ities to dene each weight valueki withk = 1, . . . , w andi = 1, . . . , cSingle Region: for all colonies the values of are in the interval [0, 1].ki =k 1w 1Disjoint Regions:ki =(i 1)w + (k 1)cw 150% Overlapping regions: the interval of values of for colony i overlapsby 50% with the interval for colonyi 1 and colonyi + 1.ki =(i 1)(w 1) + 2(k 1)(w 1)(c + 1)(4.4)5Nonsupportedsolutionscannotbeobtainedusingaweightedsumscalarization. There-fore,aregionthatonlycontainsnonsupportedsolutionscannotbedenedusingweightvectors.4.4.MultipleColonies 314.4.2 CandidateSetintheMulti-colonyApproachTurningtothecandidatesetfromwhichthebestsolutionsareselectedinorder to update the pheromone information, there are two alternatives:eitherthe set is formed with solutions from all colonies or each colony denes its ownsetwithitsownsolutions. Whenthecandidatesetisformedbysolutionsof onlyonecolonyatatime, thereisnocooperationmechanismbecausethedecisionabout whichsolutions areselectedtoupdatethepheromoneinformation only depends on the solutions found in that colony. Nevertheless,cooperation can still be imposed by other mechanisms, simply by exchangingtheselectedsolutionsbetweencoloniesorevenmorecomplicatedmethodsthat have been studied for parallelized ACO algorithms. In the present work,restrictions on communications between colonies are not considered.6Hence,the cooperation between colonies is enforced by using a candidate set formedwith solutions from all colonies.As for the single colony approach, the candidate set can be the set of allsolutions obtained by all colonies in the current iteration (ib(iteration-beststrategy) or since the start of the algorithm (bf(best-so-far strategy).4.4.3 PheromoneUpdateStrategieswithMultipleColoniesThepheromoneupdatestrategiesdescribedforthesinglecolonyapproachcan also be used with multiple colonies. To enforce the specialization of thecolonies, each ant deposits pheromone on only one colony.The selectionbydominancemethod is straightforwardly adapted to themulti-colony approach, i.e., the ants belonging to the Pareto set of the can-didate set are distributed between colonies and are allowed to deposit pher-omone.The case of the selection by scalarizationis somewhat more complicated.Let us consider only the simplied method described above, namely selectionbyobjective. Inthesinglecolonyapproach, thismethodupdatesthepher-omoneinformationusingthebestsolutionwithrespecttoeachobjective.Solution components of these selected solutions will have a high probabilityofbelongingtosolutionconstructedinthenextiteration. Sinceall oftheselected solutions update the same pheromone information, solution compo-nents which belong to more than one selected solution at one time will have ahigher probability of being part of solutions constructed in the next iteration.However, if the selected solutions are distributed between several colonies, thesolution components common to solutions which are the best with respect tothe dierent objectives will not have a higher probability. Furthermore, whenmultiple pheromone information and multiple colonies are considered, somepheromonematricesarenotupdated. Inthiscase,foreachcolonythereis6Restrictionsoncooperationcancomefromdistributedsystemswherecommunicationsmaybeexpensive.32 Multi-objectiveACOonepheromonematrixforeachobjective. Then, whenusingselectionbyobjective, if solutions selected from the candidate set are distributed amongcolonies, the number of selected solution will be much smaller than the num-berof pheromonematriceswhichmustbeupdatedbecausethemaximumnumber of solutions selected to update is equal to the number of objectives.For these reasons, we propose a denition of selection by objective withmultiplecolonieswhichdistributesthecandidatesolutionsamongcoloniesbeforeselectingthebest solutions.7Under this denition, theminimumnumberof antsallowedtoupdatethepheromoneinformationinacolonyis equal to the number of objectives. As well, since an ant cannot update thepheromoneinformationofmorethanonecolony, theselectionbyobjectivemethod enforces the specialization of the colonies.TheselectionbyscalarizationwithasinglecolonyalsoassuresthataselectedsolutionalwaysbelongstotheParetosetofthecandidateset. Aswell, thecollaborationbetweencoloniesinthemulti-colonyapproachaimstodetectweaksolutions(dominatedbyknownsolutions). Inordertoalsohavethesepropertieswhenselectionbyscalarizationisusedwithmultiplecolonies, the set that is distributed among colonies is actually the Pareto setof the solutions from all the colonies.In summary, in selection by objective, the Pareto set of the candidate setis somehow distributed among the colonies and then for each colony the bestsolutionwithrespecttoeachobjectiveisallowedtodepositpheromoneei-ther on the single pheromone matrix (when single pheromone information isconsidered)oronthepheromonematrixassociatedwiththerespectiveob-jective (when multiple pheromone information is considered). Finally, whenthe number of colonies is one,this denition of selection by scalarization isconsistent with the one previously proposed.Whenever multiple colonies are considered, the pheromone update strat-egy must also decide on which colony each solution updates the pheromoneinformation, i.e., it must dene how solutions are distributed among the mul-tiplecolonies. Iredi et al. [29] proposedtwomethods. Therststrategy,calledupdatebyorigin, allowseachanttodepositpheromoneonlyinthecolony from which that ant comes. Hence, this is not essentially a collabora-tivestrategybecauseitdoesnotenforcetheexchangeofsolutionsbetweencolonies. Moreover,itdoesnotexplicitly forcethecoloniestofocusondif-ferent regions. The second method is called updatebyregionand explicitlyforces each colony to focus on a dierent region. The regions are dened im-plicitly by sorting the Pareto set of the candidate set and then dividing it asequally as possible into parts that are distributed among the colonies. Thismethod can only be used with bi-objective problems because for more than7Thedrawbackofthisapproachisthatifthenumberofsolutionsassignedtoacolonyisless than the number of objectives, then a solution will update the pheromone informationofthecolonymorethanonetime,howeverthiscanalsooccurwithasinglecolony.4.5.Local SearchMethodsforMO-ACO 33two objectives a set of non-dominated objective vectors can only be partiallysorted. More formally,let (be the Pareto set of the candidate set (either(ibor (bf) and letc be the number of colonies. Firstly, the update by regionstrategysorts (inorderofincreasingvaluesoftherstobjective. Next,(isdividedintodisjointpartitions (1, (2, . . . , (cinsuchawaythattheirrespective sizes dier by at most one.8Lastly, all solutions in (iupdate thepheromone information of colonyi.Theorderinwhichthesolutionsaresortedonlymattersif weareus-inganothermechanismtoforcecoloniestofocusondierentregions. Forinstance, whenusingmultiplepheromoneinformation, theweightscanbedened to force colonies to focus on dierent regions. Therefore, if the antsfrom the rst colony construct solutions considering more importance to therstobjectivebecauseof theweightvectorsandtheantsfromthecolonycattachmoreimportancetothesecondobjective,thenthesolutionsin (must be sorted in increasing order of the rst objective.4.5 LocalSearchMethodsforMO-ACOFormanysingleobjectivecombinatorialoptimizationproblems,ACOalgo-rithms obtain the best performance only when the solutions constructed bytheantsareimprovedusinglocal searchmethods[11, 21, 38, 48, 49, 50].Furthermore, the best conguration of settings and parameters for ACO al-gorithms depends greatly on whether local search is used or not. Therefore,when local search can be eciently applied to a certain problem and it mayimprovetheperformanceofanACOalgorithm, thenthealgorithmshouldbe analyzed, taking into account the use of local search since the beginningof the analysis.Local searchmethods for MOCOcanbebasedonmethods for singleobjective problems applied to several scalarizations of the objective vector ofthe MOCO problem. On the other hand, they can use the Pareto optimalitycriteriatokeepanarchiveof Paretolocallyoptimal solutions, asdonebyPareto Local Search (PLS) [42, 43]. In the former case, the number of Paretolocally optimal solutions obtained is bounded by the number of scalarizationsconsidered. In the latter case, the size of the optimal locally Pareto set maybe exponential in the instance size. For this reason, we propose adapting theconceptsforboundedarchivinginmulti-objectiveevolutionaryalgorithms(MOEA) [33, 37] in order to obtain a more ecient variant of PLS.8Noticethat |C| =c p + r, henceeachpartition C1, . . . , Crhas sizep + 1andeachpartitionin Cr+1, . . . , Cchassizep.34 Multi-objectiveACO4.5.1 LocalSearchforSingleObjectiveProblemsLocalsearchmethodsforsingleobjectiveproblemscanbeappliedtotheirmulti-objective variants because any MOCO problem can be transformed intoa single objective problem using a weighted sum scalarization of the objectivevector. Therefore, local search methods for MOCO problems may be basedon several scalarizations of the objective vector by altering the weight vectors.Itiswell-knownthatgloballyoptimal solutionsforscalarizedproblemsarealsoParetogloballyoptimal solutions. Furthermore, local optimaforscalarizedproblemswherealltheweightsarepositivearealsolocalParetooptima,i.e., thereisnoothersolutioninitsneighborhoodthatdominatesit [42]. However, these local search methods will not be able to nd nonsup-ported solutions.4.5.2 ParetoLocalSearch(PLS)Local search methods for MOCO problems can be based on the Pareto op-timalitycriterion. Forexample, ParetoLocalSearch(PLS)[43]isamulti-objectiveextensionoflocal searchalgorithmsforsingleobjectiveproblemswhichusestheParetooptimalityastheacceptancecriterion. PLSstartsfromasolutionandexaminesitsneighborhood. Next, anynondominatedsolutionfoundisaddedtoanarchiveandthedominatedonesareremovedfromit. PLSterminateswhenalltheneighboringsolutionsofallsolutionsin the archive have been explored. The size of the locally optimal Pareto setobtained may be exponential in the instance size. Thus, for some problemsor specically in some instances, because of the large size of the archive, PLSmay be very inecient in terms of space and computation time required, evenmore so if it is to be used by a MO-ACO algorithm.4.5.3 BoundedParetoLocalSearch(BPLSA)In order to address the previously described problem, we propose a boundedvariantofParetoLocal Search(BPLSA), whichisasimpliedapproachofconcepts previously studied in MOEA [33, 37].The only dierence with respect to PLS is the procedure which adds a newneighborhood solution to the archive. This procedure ensures an upper boundto the size of the archive by placing a hyper-grid in the objective space andonly allowing one objective vector to be in each grid cell at one same time [37].When a new solution is added to the archive, there are three possibilities: (i)the solution is dominated by any in the archive; it is discarded and thus thesize of the archive does not change; (ii)the solution dominates at least onesolutioninthearchive; all dominatedsolutionsarediscardedandthenewsolution is added to the archive; thus the size of the archive does not changeormayevendecrease; or(iii)inanyothercase, thesolutioniscompared4.6.MO-ACOAppliedtothebQAP 35with the ones in the archive in order to ensure that there is no other solutionin the same cell.Each cell is represented by a boxvectorand two solutions belong to thesame cell when they have the same box vector. GivenA as the upper boundof the size of the archive, then the number of divisions at each axis is givenbyA+12. Thus, foranysolutionwithobjectivevectora= a1, . . . , aQ, itscorresponding box vector isB(a) = B1, . . . , BQ, withBi =_(aimini)(A + 1)2(maximini)_i = 1, . . . , Qwheremaxiandminiarethe respectivemaximum and minimum valuesofobjectiveiforallsolutionsinthearchiveandA+12(maximini)isthelengthofeach grid cell in the dimensioni.The ranges (maxi, mini) are adapted over time [36]. When a new solutionwithobjectivevectora= a1, . . . , aQisaddedtothearchiveandforanyobjectivei a value is lower than the respective minimum range (ai< mini),thentherespectiveminimumrangeisupdated(mini=ai). Inthiscase,themaximumvaluesarealsocheckedandupdated, ifanyvalueofthere-maining objectives is greater than its respective maximum value (aj> maxj,j = 1, . . . , Qj ,= i). Whenever the ranges for objectivei change, the corre-sponding components of the box vector for every solution in the archive mustbe calculated again and all the box vectors must be compared to ensure thatevery solution belongs to a dierent grid cell.In the case where two solutions belong to the same grid cell, one of themshould be removed. In order to keep the tails of the archive, solutions withan objective value equal to any minimum are protected from removal. In anyother case, we keep the oldest solution.4.6 MO-ACOAppliedtothebQAPAn ACO algorithm applied to a certain MOCO problem may use any com-bination of the concepts described above. A certain combination of conceptsor strategies can be seen as a particular congurationof a general MO-ACOalgorithm. Therefore, we would like to test several congurations in order tond the best one. Ideally,the available knowledge from previous or similarexperiments would help us to decide which congurations may be interestingto consider. Nowadays, there is very limited knowledge about the mQAP orabout MO-ACO algorithms. In contrast, there is a vast literature on multi-objective evolutionary algorithms (MOEA). Two essentially dierent searchstrategiesareusuallyconsideredinMOEA,eitherbasedonthedominancecriteria(classD)orbasedonseveral scalarizationsof theobjectivevector(class S). Consequently, in the study of MO-ACO applied to the bQAP, wewill focus on straightforward congurations which reect these dierent typesof search strategies.36 Multi-objectiveACOThe use of multiple pheromone matrices which are aggregated by meansof weights resembles the scalarization of the multiple objective functions. Ontheotherhand, whensinglepheromoneinformationisconsidered, thereisno need of weights. Moreover, if we use the dominance criteria to select thebestsolutionsthatwillupdatethesinglepheromoneinformation(selectionby dominance), then the search strategy is based only on the dominance cri-teria. For the bQAP in particular, the selection by dominance method doesnot suggest the utilization of multiple pheromone matrices in a natural man-ner simply because a solution component has a unique meaning. Thus, someothermechanismisneededtodistinguishbetweenthemultiplepheromonematrices. In contrast, the strategy of selection by objective allows us to usemultiple pheromone information. For these reasons, we consider a congura-tion of class D that comprises the use of single pheromone information andselectionbydominance, andacongurationof classSthatcomprisestheuse of multiple pheromone information (Eq. 4.2) and selection by objective.In addition, weights can be dened so that the algorithm searches in all di-rections or in one direction at each iteration. Therefore, we consider searchcongurations of class Sall and Sone, respectively.TheavailableknowledgeaboutACOforthesingleobjectiveQAPindi-cates that multi-objective local search methods may improve the performanceoftheMO-ACOalgorithmappliedtothebQAP. Therefore, wewillstudycongurationswhichapplylocalsearchtothesolutionsconstructedbytheants versus congurations which do not use local search. In order to be con-sistent with the search conguration of class D and class S, we use two essen-tially dierent local searches for each class but with the following similarities.Firstly, both local searches use the same underlying 2-exchange neighborhood(2-opt), that is, the set of all solutions where the location of two dierent fa-cilities is exchanged. Secondly, both explore the whole neighborhood beforeapplyingtheacceptancecriterion, thatis, theyarebestimprovementlocalsearches. Lastly, bothmakeuseofthefastdeltaevaluationofthebQAP,which is well-known in the single objective case [52] and was extended to themQAP by Paquete & St utzle [43]. In contrast,the local search method forclass S is based on the weighted sum scalarization of the objective functionvector(W-LS), whereasforclassDisbasedonParetoLocal Search(PLS)oritsboundedvariant(BPLSA). ForW-LS, inparticular, eachsolutionisimproved by W-LS using the same weight that the ant used before while con-structing the solution. The various classes of search strategy are summarizedin Table 4.1.We would also like to compare the multi-colony approach to the use of asingle colony. Hence, we consider a cooperative multi-colony approach wheretheexchangeof solutionsbetweencoloniesisperformedusingthe updateby region strategy and the candidate set is comprised of solutions from allcolonies. In particular, when combined with search congurations of class S,4.6.MO-ACOAppliedtothebQAP 37the set of weight vectors is dened using 50% overlapping regions (Eq. 4.4).A similar conguration has already been tested over a bi-objective schedulingproblem[29], butbyusingasearchstrategybasedonmultiplepheromoneinformation and selection by dominance.As well, we examine congurations where the candidate set is the set ofall the solutions obtained by all colonies (or the single colony) in the currentiteration (ib(iteration-best strategy) or since the start of the algorithm (bf(best-so-farstrategy).Additionally, theconceptsof //ASareappliedtotheMO-ACOal-gorithmforthebQAPwithminordierences. Firstly, eachantdepositsaconstant amount of pheromone when updating the pheromone information.Hence, the pheromone limits are calculated by means of this constant value.Secondly, some congurations allow more than one ant to update the pher-omoneinformationatatime. Forthisreason, theupperpheromonelimit(max) is onlyusedtoinitializethepheromoneinformationbut does notrestrictthemaximumvaluethatthispheromoneinformationcantakeon.Finally, evaporation is done as in Eq. (3.2) on page 16.The congurations considered in the design of a MO-ACO algorithm forthe bQAP are summarized in Table 4.2. The following chapter explains theexperimental setup and the performance assessment methodology used in theexperiments performed with these congurations.38 Multi-objectiveACOClass Selection Ph. Inform. WeightsD by Dominance Single (Eq. 3.1) Sallby Objective Multiple (Eq. 4.2) In all directionsSoneby Objective Multiple (Eq. 4.2) In one directionTable 4.1:Classes of search strategy used by MO-ACO when applied to the bQAP.Selection methods are explained in Sec. 4.3; the possible denitions of multi-objectivepheromone informationare discussed in Sec. 4.2; the dierent alternatives in orderto dene the set of weight vectorsare examined at the end of Sec. 4.2.1.Colonies CandidateSet Class LSmethodSingleD PLS, BPLSA, w/o LS(ibSallW-LS, w/o LSSoneW-LS, w/o LSD PLS, BPLSA, w/o LS(bfSallW-LS, w/o LSSoneW-LS, w/o LSMultipleD PLS, BPLSA, w/o LS(ibSallW-LS, w/o LSSoneW-LS, w/o LSD PLS, BPLSA, w/o LS(bfSallW-LS, w/o LSSoneW-LS, w/o LSTable4.2: Congurations of the MO-ACO algorithm for the bQAP. The columnColoniesrefers to the singlecolony approach compared to use of multiplecolonies.For both approaches, an iteration-best strategy ((ib) or a best-so-far strategy ((bf)may dene the candidate set. For each of these possibilities, various classes of searchstrategies (either D, Sall or Sone) will be tested. Finally, the appropriate local searchmethodfor each class will be applied (PLSor BPLSA for class D and W-LSfor classS) versus the same congurations without local search (w/o LS).Chapter5PerformanceAssessmentTheperformanceassessmentofamulti-objectivealgorithmisnotaneasytask. Inthesingleobjectivecasethequalityoftheoutcomeofanalgorithmisdenedwithrespecttotheobjectivevalue. However,itisnotclearhowtodenethequalityofaParetosetof objective vectors. Consequently,many dierent quality measures have been proposedintheliterature. Thesequalitymeasurescanbeclassiedintounarymeasures, whichassignaqualityvaluetoaParetosetindependentlyfromotherParetosets,andbinarymeasures, whichcompare twoParetosets. The binary -measure indicates whetheraParetoset is better thananother interms of Paretooptimalityandthenumber oftimesthatanoutcomeobtainedbyonealgorithmisbetterthananoutcomeobtainedbytheothercangiveanideaof theirperformance. Unfortunately, inmanycasestheoutcomesmaybeincomparableintermsofParetooptimality. Anunary-measurecanbedenedinthecasethattheoptimal Paretosetisknown. Thisunary-measureisusedtodiscriminateamongincomparableParetosets. BecausetheParetooptimal setis not knownfor thebQAPinstances considered, wecalculatealower boundof theParetooptimal set andusethislower boundinorder todenetheunary-indicator.Additionally, anAnalysisof Variance(ANOVA)of thevaluesof thisunary-measuremay indicate which parameters of an algorithm contribute to its overall performance andwhichinteractionsoccurbetweentheseparameters.5.1 Binary-measureNowadays, an important research eort in multi-objective optimization aimsto clarify the performance assessment of multi-objective algorithms. In singleobjective problems, the quality of the outcome of an optimization algorithmis dened in terms of the scalar value of the single objective function. In con-trast, in multi-objective optimization, the concept of the quality of a Paretoset is rather vague.The relations between Pareto sets provide a solid quality classication inordertocomparetheoutcomesofmulti-objectiveoptimizationalgorithms.Particularly, the relationbetter is the most general form of superior quality40 PerformanceAssessmentbetween two Pareto sets. Then, given that the outcome of the algorithm AistheParetoset Aandtheoutcomeof algorithm istheParetoset B,whenA Bwecansaythatalgorithm Aperformsbetterthanalgorithmfor these two outcomes. However, the outcomes of two algorithms may beincomparable (A | B) in terms of Pareto optimality.Which other quality criteria can be dene over Pareto sets is still an openquestion. Despite this fact, many dierent quality measures have been denedin the literature. These quality measures can be classied in unary measures,which assign a quality value to a Pareto set independent of other Pareto setsunder consideration, and binarymeasures, which evaluate the comparison oftwo Pareto sets. Binary measures require to calculateN(N 1) values inorder to compare NPareto sets because for each pair of Pareto sets a binarymeasure returns two values.1On the other hand,unary measures are mostcommonly used in the literature because they are more ecient to calculate,i.e., onlyNvalues must be calculated to compareNPareto sets.Unfortunately, thereisnounarymeasurewhichcanindicatewhetheraPareto set is better than another [58]. On the other hand, some binary mea-sures are able to indicate whether a Pareto set is better than another. Thus,theinferencepowerofunarymeasuresisinsucienttoachieveconclusionsas solid as those that can be obtained using binary measures.Thebinary-indicatorgivesthefactorbywhichanapproximationsetisworsethananotherwithrespecttoall objectives[58]. Formally, giventwoParetosets, AandB, ofanarbitrarymulti-objectiveproblemwithQobjectives, the binary-indicator can be calculated asI

(A, B) = maxbBminaAmaxqQ_aqbq_(5.1)Using this indicator we candene a binary -measure whichdetectswhether a Pareto set is better than another as followsI

(A, B) 1 I

(B, A) > 1 ABIn order to compare two groups of outcomes, obtained for example by twodierent algorithms, we calculate the percentage of outcomes from one groupthatarebetterthananoutcomefromthesecondgroupusingthisbinary-measure. Wemust alsocalculatethepercentageof outcomes fromthesecond group that are better than an outcome from the rst group. The sumofthesepercentagesisnevergreaterthan100%. Moreover, theremainderpercentage not included in the sum of those percentages corresponds to theoutcomes that are incomparable or equal.1Forsimplication,wedonottakeintoaccountsymmetricbinarymeasuresorcombina-tionsofunarymeasuresbecausetheirpropertiesaresimilartothepropertiesofunarymeasures[58].5.2.Unary-measure 415.2 Unary-measureWe would also like to distinguish between Pareto sets which are incomparableintermsof Paretooptimality. Toaddressthisissue, wedeneanunary-measure using a lower bound of the optimal Pareto set that gives the factorbywhichaParetoset is worsethanthelower boundwithrespect toallobjectives.Whentheoptimal Paretoset Tisknown, anunary-indicatorcanbedened asI

(A) = I

(A, T)This indicator gives the factor by which a Pareto setA is worse that theoptimal Pareto set Twith respect to all objectives [58]. Thus, I

(A) 1 andI

(A)=1impliesA= T. Althoughthereisnounarymeasurewhichcandetect whether a Pareto set is better than other, the above unary -indicatoris able to detect whether a Pareto set is not worsethan other [58]. Thus, anunary-measure can be dened asI

(A) < I

(B) BAAswell, B A(A B) (A |B) (A =B),thusthisunary-measure is consistent with the Pareto optimality criterion: if the value of I

for one Pareto set is lower than for another, then the former is better, thus weget the right answer, or both are incomparable, thus we are choosing amongtwo incomparable sets.Additionally, thisunary-measurecanberelatedtotheworstcaseap-proximation ratio used in approximation algorithms and specially to the no-tion of-approximate Pareto curve[41]. Given a Pareto setA withI

(A) =1 + , then there is not other Pareto setBsuch that for anyb Bthen forall a A, bq(1 + )aqfor someq = 1, . . . , Q. Hence, this is a worst casemeasure and a reasonable answer to present to the decision-maker.Nevertheless, the optimal Pareto set for the bQAP is usually not known.In this case, a lower bound B, such that BT, can be used instead. Hence,I

(A) = I

(A, B). It turns out that I

(A)I

(T, B), i.e., the unary -measureis always greater or equal than the factor by which the optimal Pareto set isworse than the lower bound.5.2.1 LowerBoundIn order to compute a lower bound to the optimal Pareto set for the bQAP,weextendtheGilmore-Lawlerlowerbound[25]. ForthesingleobjectiveQAP,theGilmore-Lawlerboundisgivenbytheoptimalobjectivevalueofan associated Linear Assignment Problem (LAP) which can be solved in cubictime (see [3] for more details).22CodeinFortranisavailableatQAPLIB.42 PerformanceAssessmentFigure 5.1: Pictorialviewof thelowerboundfor the bQAP.Inthiscase, wedeneabi-objectiveLAP(bLAP) where each objective is the LAP associ-ated to each objective of the bQAP. Then,wesolve several scalarizations of the bLAP, obtain-inganumberofsolutionswhicharenotdomi-nated by any Pareto optimal solution of the as-sociatedbQAP. However, thesesolutions arenot guaranteed to dominate all the Pareto opti-mal solutions of the bQAP. Therefore, we addedpointsasfollows. First, wesorttheobjectivevectors lexicographically and for each successivepair of objective vectorsu, vwe added anotherpointw with coordinatesw1 = minu1, v1 andw2=minu2, v2. Theresultingset is guar-anteedtodominateall Paretooptimal solutions of theassociatedbQAP.Figure5.1givesapictorialperspectiveofthisworstcaselowerbound: thewhitepointscorrespondstotheoptimal pointsforseveralscalarizationsofthe bLAP and the dark points refer to the additional points. For the instancesconsidered, we used 5000 weight vectors maximally dispersed in [0, 1].5.2.2 AnalysisofVariance(ANOVA)Theperformanceassessmentofanalgorithmorasetofalgorithmsusuallyinvolves setting the values of certain parameters, e.g., the evaporation factorin ACO algorithms. Each experiment characterizes a certain conguration ofthe values of these parameters. In order to decide which algorithm or whichof itspossiblecongurationsperformsbetterthantheothers, itnormallysucestorepeatseveraltimeseachexperiment, calculatethequalitymea-sure for each repetition and compare the mean values of the measure for eachexperiment. However, this sort of analysis does not generally explain why acertaincongurationisbetterorworsethananother. AnalysisofVariance(ANOVA) is a statistical technique that may be used to nd out which param-eters of an algorithm are relevant to explain its performance. Furthermore,ANOVA may also indicate the relations between these parameters.The parameters which vary depending on the experiment and may aectthe performance are called factors. The levelsof the factor are the dierentvaluesthateachfactortakesonforeveryexperiment. Finally, thequalitymeasure used to assess performance is the response variable. In our analysis,theresponsevariableisthevalueof theunary-measureusingthelowerbound and the factors are the dierent components of a certain congurationof MO-ACO when applied to the bQAP.Essentially, twokeyquestioncanbeasked: doesafactorinuencetheresponse variable?is the eect of a factor in the response variable modiedbytheinuenceofanotherfactor? Therstquestioniswhetherthereisa5.3.MedianAttainmentSurface 43main eect, whereas the second one asks for an interaction eect. The initialhypothesisisthereisnoteect, i.e., foreachfactororinteractionthenullhypothesis isthatitdoesnotinuenceontheresponsevariable. ANOVAteststhisnull hypothesisandgivestheprobabilitythatitisinfacttrue.This probability is called a p-value. When the p-value is lower than a certainvalue, byconvention0.05, itissaidtobestatisticallysignicantandthenthe null hypothesis is rejected, i.e., there is a signicant eect of the factor orinteraction. On the other hand, when thep-value is not statistically signi-cant the null hypothesis cannot be rejected, thus the experimental results donot evidence any signicant dierence on the response variable for any levelof the factor.The results of ANOVA are expressed as a characteristic table. Each rowshows the results for a certain factor or for a certain interaction. Each columnshows the results of calculations involved in the test of the hypothesis. Themost important element of this table is the column of thep-value.Inthecaseofmaineects, ifthenull hypothesisisrejectedthensomelevels of a factor have a dierent eect on the response variable than the otherlevels of the factor. However, ANOVA does not tell which pairs of levels aresignicantlydierent. Forthispurpose, all possiblepairsofthemeansforeachlevel canbecomparedusingcondenceintervals. TheTukeymethodobtains condence intervals at the 95% for all the pairwise comparisons. Eachintervalindicatesthatthereisa95percentofprobabilitythattheintervalcontains the real mean of the response variable for a certain level. Therefore,if the intervals do not themselves overlap, the means are signicantly dierent.For interaction eects, the means of the levels of one factor can be plottedfor eachlevel of other factor. Theinteractioneect is indicatedbynonparallel lines. However, this interaction plot neither proves whether there isasignicantinteractioneectnoritshowsthereal dierencebetweenthecombinationsoflevels. TherstissueisachievedbyANOVA.Thesecondoneisattainedbyplottingerrorbarsthatrepresentstheupperandlowerboundsof a95percentcondenceinterval aroundeachmean. Thus, theTukeymethodisusedtocalculatetheseintervalsforeachcombinationoflevels.The ANOVA is based on some assumptions that must be checked beforeperforming the actual statistical analysis. Appendix A explains briey howthese assumptions are checked.5.3 MedianAttainmentSurfaceThe probability of obtaining an arbitrary goal in the objective space duringa single run of an arbitrary algorithm can be represented by an attainmentfunction [27]. This attainment function can be estimated using data collectedfrom several runs of the particular algorithm. The median attainment surface44 PerformanceAssessmentis the line that connects objective vectors assigned to an empirical frequencyof 50% of being attained [27]. The median attainment surface gives a clearvisual informationof howtheobjectivespaceiscoveredbytheoutcomesof analgorithm. Particularly, wewill usemedianattainment surfaces toassessthesimilaritiesanddierencesbetweenthosecongurationsthatareincomparable with respect to the binary-measure.5.4 ReferenceSolutionsWewouldliketoobtainreferencesolutionsinordertocomparewiththeoutcomesof thecongurationsof MO-ACOstudiedhere. However, thereisnoexactalgorithmforthebQAP. RobustTaboosearch(RoTS)[52] isamong the best performing algorithms for unstructured instances of the singleobjective QAP and it obtains good solutions for structured instances [19, 51].Paquete&St utzle[43] usedRobustTabooSearch(RoTS)tosolveseveralscalarizationsofthebQAPusingmaximallydispersedweightvectors. Thesolutionsgeneratedbytheseveralrunsarelteredbyremovingdominatedsolutions in order to obtain a Pareto set. This algorithm is calledW-RoTS.Chapter6Experiments6.1 ExperimentalSetupThe experimental setup involved three factors: (i)the search conguration,whichcanbeof classD, SallorSone; (ii) iteration-bestversusbest-so-farstrategies for dening the candidate set; and (iii) one or multiple colonies. Inaddition,each combination of these factors was run with and without localsearch. The local search methods tested wereW-LS for classes Sall and Sone;andPLS and BPLSA for class D. For BPLSA we tested three dierent valuesof the upper bound limit of the archive size, A 100, 500, 1000. Therefore,when local search was used, the search congurations were: class D with PLS,BPLS100, BPLS500, andBPLS1000; classSallwithW-LSandclassSonewithW-LS.The total number of ants (m) was always equal to the size of the instance.Five colonies (c = 5) were used in the multi-colony approach, thus there werem/5 ants per colony. We followed the rules of //AS for the managementof pheromoneswith=1and=0(heuristicinformationisnotused);=0.9forthepheromoneevaporation; pbest=0.05toderivethefactorbetween the lower and the upper pheromone trail limits; and maxwas set tothe theoretically largest value [51].The algorithms were tested on the six instances of sizen = 50 presentedinSection2.1.1onpage8. Threeof theseinstancesareunstructuredandtheotherthreearestructuredinstances. Thesethreeinstancesweregener-ated with 0.75, 0.00, 0.75, whereis a parameter that inuences thecorrelation between the ow matrices.Every experiment was run for a maximum time of 300 CPU seconds andwas repeated 20 times. The algorithms were coded in C and the experimentsweredoneonacomputerwithaPentiumIII933MHzCPUand512MBRAM running under Debian Linux.46 Experiments6.2 AnalysisofExperimentalResultsTheanalysisof theresultsis doneinthreephases. Therst phaseusesthebinary-measure todetect which congurations performclearly better.Inthesecondphase, alowerboundoftheoptimalParetosetiscalculatedfor each instance and we perform an Analysis of Variance (ANOVA) on thevalues of the unary-measure with respect to the lower bound. Finally, theresults are visualized using median attainment surfaces.Additionally, we obtained reference solutions using W-RoTS. We ran W-RoTSfor a total time of 300 CPU seconds for each instance, as done for each ex-perimentofMO-ACO. Eachrunoftheunderlyingtaboosearchalgorithmwas stopped after 100n iterations. With this time limit, approximately 136scalarizationscouldberun. Thesescalarizationsusedasetof maximallydispersed weight vectors in the interval [0, 1].6.2.1 AnalysisBasedonBinary-measureThe rst phase of the experimental analysis allows us to obtain solid conclu-sions about the performance of the dierent congurations. The aim of thisphase is to identify which congurations of MO-ACO perform clearly betterthan others. The binary-measure indicates whether one outcome is betterthan another, the latter is better than the former or they are incomparable interms of Pareto optimality. Using this binary -measure, we compare two setof outcomes,comparing each outcome from the rst set with each outcomefrom the second set. First, we calculate the percentage of times one outcomefrom the rst set is better than one outcome from the second set. Next, wemust also calculate the percentage of times one outcome from the second setis better than one outcome from the rst set. The remainder of the sum ofthese two values is the percentage of times one outcome from any of the setsis incomparable or equal to one outcome from the other set.Table6.1comparescongurationsusinglocalsearch methodswith con-gurationsnotusinglocal search. Comparisonaregroupedaccordingthelocal search methods tested: BPLS100, BPLS500, BPLS1000andPLS for classDandW-LSforclassS. Withineachgroup, separatedbyavertical line,congurations using the respective local search are compared with the samecongurations not using local search (w/o). Each value of the two columnswithin each group, that is, each of the pair of values between vertical lines,is the percentage of times an outcome corresponding to one column is betterthan an outcome corresponding to the other column. For each pair of percent-ages, therstone, correspondingtocongurationsusingaparticularlocalsearchmethod, isalwaysgreaterthan25%, whilethesecondvalue, corre-sponding to the same congurations without local search, is always less than0.8%withonlytwoexceptions. Moreover, thedierencebetweentherstvalue minus the second one is always greater than 15% and for any instance6.2.AnalysisofExperimental Results 47Type BPLS100w/o BPLS500w/o BPLS1000w/o PLS w/o W-LS w/oUn. 0.75 35.4 15.0 82.7 0.0 97.4 0.0 99.9 0.0 100.0 0.00.00 99.1 0.0 99.1 0.0 99.1 0.0 98.8 0.0 99.4 0.00.75 69.9 0.0 90.5 0.0 93.0 0.0 86.1 0.0 100.0 0.0Str. 0.75 88.4 0.0 86.8 0.0 79.3 0.0 34.3 5.7 99.5 0.00.00 97.7 0.0 95.4 0.0 93.5 0.0 44.3 0.8 99.9 0.00.75 82.4 0.0 72.7 0.0 61.7 0.0 26.3 0.4 100.0 0.0Table 6.1: Comparison of congurations using local search methods (eitherBPLS100,BPLS500,BPLS1000andPLS for class D andW-LS for class S) with con-gurations not using local search (w/o). Each group of comparisons is separated bya vertical line. Each entry gives the percentage of times an outcome correspondingto one of the two columns of the group was better than an outcome correspondingto the other column.thereisalwaysatleasttwolocal searchmethodsforwhichthisdierenceisgreaterthan80%. Therefore, theresultsshowthatcongurationsusinglocal search methods obtain better results that congurations not using localsearch. Furthermore, for the structured instances (Str.), the three last rows ofTable 6.1 show that the highest dierences of percentages are obtained whenusingW-LS, always greater than 99%, and the bounded variant BPLSA, al-waysgreaterthan80%foratleastonevalueof A= 100, 500, 1000. Incontrast, for PLS this dierence is less than 45% for any structured instance.Thus, BPLSAis a better local search method thanPLS for congurations ofclass D when tackling structured instances.Additionally, we have the intuition that the results obtained with certaincongurations canbeverydierent dependingonwhether local searchisused. In the following, in order to show this dierence, we study separatelythose congurations which use local search of the ones that do not use it.Next,we compare those congurations based on scalarizations in all di-rections at each iteration (Sall) with congurations based on scalarizations inone direction at one iteration and a dierent direction in the next iteration(Sone). ResultsaresummarizedinTable6.2, forcongurationswhichuseW-LS, whichareofclassS, andforcongurationsofclassSwhichdonotuse any local search. Since the dierences between the pairs of percentagesare almost zero independently of the instance and of the use of local search,we conclude that the strategies (Sall) and (Sone) are mainly incomparable interms of Pareto optimality. This incomparability, however, can be caused bytwo dierent situations: either the outcomes of congurations of classes Salland Soneare very similar, that is, the shape of the outcomes is very similar,or the outcomes are very dierent but incomparable in terms of Pareto opti-mality, that is, the outcomes are dierent but for both strategies Sall and Sonethe outcomes obtained using one strategy are not better than the outcomesobtained by the other. This rst phase of the analysis cannot answer to this48 ExperimentsInstance W-LS w/o LSType SallSoneSallSoneUn. 0.75 7.1 10.1 11.9 16.90.00 0.7 0.3 2.9 6.00.75 0.0 0.0 2.1 2.8Str. 0.75 0.0 0.0 4.1 4.00.00 0.0 0.0 4.4 1.80.75 0.0 0.0 2.8 3.3Table6.2: Comparison of congurations using Sallwith congurations using Sone.Each entry gives the percentage of times an outcome obtained by a conguration ofthatcolumnwasbetterthananoutcomeobtainedbyacongurationoftheothercolumn. The same comparison is made separately for congurations using W-LS andfor congurations not using local search (w/o LS).question. Nonetheless, wevisualizedtheseresultsbymeansofthemedianattainment surfaces (as explained in Section 5.3) to obtain the answer. Wend out that for congurations where the only dierence was the use of Sallor Sone, the outcomes were very similar. Therefore, we conclude that for thebQAP there is no dierence between the approaches used by Sall and Sone.The next question is which approach among class D and class S obtainsthebestoutcomes. Inordertohavethesamenumberofoutcomesineachgroup, we consider only one of the search congurations of class S, particularlywe consider only Sall.1Table 6.3(a) compares congurations of class S usingW-LS with cong-urations of class D using either BPLS100, BPLS500, BPLS1000orPLS. In theunstructuredinstancewith=0, thepercentageoftimesanoutcomeob-tainedbyclassDusinganylocal searchwasbetterthatoneobtainedbyclass S usingW-LS is at least 30%, while none of the outcomes obtained byclass S usingW-LS was better than one obtained by class D using any localsearch. In the case of the unstructured instance with = 0.75, the highestdierence(12%)isobtainedwhenclassDusesBPLS1000. Fortheunstruc-tured instance with = 0.75, the highest dierence is obtained when class DusesPLS (32.2 21.6 = 10%),while this dierence is 0.0 97.7 = 97.7%forBPLS100, 1.3 63.3 = 62%forBPLS500, and11.6 30.6 = 19%forBPLS1000, where the negative percentage means that class S usingW-LS wasbetterthan classDusingthatparticularlocalsearch. In summary,fortheunstructured instances the results obtained by class D improve as the upperboundAonthearchivesizeofBPLSAincreases,andclassDusingPLSorBPLS1000 is slightly better than class S usingW-LS.21Asexplainedinthepreviousparagraph,theresultsareverysimilarifSoneisconsideredinsteadofSall.2Actually,PLSisequivalenttoBPLSAwhenA = .6.2.AnalysisofExperimental Results 49Type DBPLS100SW-LSDBPLS500SW-LSDBPLS1000SW-LSDPLSSW-LSUn. 0.75 0.0 97.7 1.3 63.3 11.6 30.6 32.2 21.60.00 40.7 0.0 46.1 0.0 45.7 0.0 30.2 0.00.75 0.1 0.0 8.9 0.0 12.0 0.0 1.3 0.0Str. 0.75 0.0 2.4 0.0 3.7 0.0 16.6 0.0 83.60.00 0.0 0.0 0.0 1.0 0.0 2.6 0.0 63.70.75 0.0 1.1 0.0 3.0 0.0 8.9 0.0 71.2(a)Dvs. SwithLSType Dw/oLSSw/oLSUn. 0.75 59.5 0.10.00 52.6 0.00.75 27.7 1.0Str. 0.75 29.2 0.50.00 25.9 0.10.75 28.8 0.1(b)Dvs. SwithoutLSTable6.3: Comparison of congurations of class D with congurations of class S.Withineachgroupdelimitedbyvertical lines, eachentrygivesthepercentageoftimesanoutcomeobtainedbyacongurationofthatcolumnwasbetterthananoutcomeobtainedbyacongurationof theothercolumn. Thesamecomparisonismadeseparatelyfor(a)congurationsusinglocal search, W-LSforclassSandeitherPLS or BPLSAwithA = 100, 500, 1000 for class D, and (b)congurationsnot using local search (w/o LS).With regard to the structured instances, the results are completely dier-ent: the results obtained by class D improve as the sizeA of the archive inBPLSA decreases. For all the values of , at least 63.7% of times class S usingW-LS was better than class D usingPLS, while if class D uses BPLS100thatvalue is never greater than 2.4%. Thus, in the case of structured instances,classDwithPLSisoutperformedbyclassSwithW-LS, butifBPLS100isused instead of PLS, then the outcomes obtained by class D and class S aremainly incomparable.When local search is not used, Table 6.3(b) shows that in all the six in-stancesthepercentageof timesclassDwasbetterthanclassSisalwaysgreater than 25%, while the percentage of times class S was better is almostzero. Therefore, congurations of class D are slightly better than congura-tions of class S when they do not use any local search method.Since results vary remarkably depending on which local search method is50 ExperimentsInstance with LS w/o LSType ib bf ib bfUn. 0.75 8.5 17.9 3.5