A Comparative Study of Dynamic ResamplingStrategies for Guided Evolutionary

Multi-Objective OptimizationKanGAL Report Number 2013008

Florian Siegmund, Amos H.C. Ng

Virtual Systems Research CenterUniversity of Skovde

Hogskolevagen, 54128 Skovde, Sweden{florian.siegmund, amos.ng}@his.se

Kalyanmoy Deb, IEEE Fellow

Department of Mechanical EngineeringIndian Institute of Technology Kanpur

PIN 208016, Indiadeb@iitk.ac.in

Abstract—In Evolutionary Multi-objective Optimization manysolutions have to be evaluated to provide the decision maker witha diverse choice of solutions along the Pareto-front, in particularfor high-dimensional optimization problems. In Simulation-basedOptimization the modeled systems are complex and require longsimulation times. In addition the evaluated systems are oftenstochastic and reliable quality assessment of system configurationsby resampling requires many simulation runs. As a countermea-sure for the required high number of simulation runs caused bymultiple optimization objectives the optimization can be focusedon interesting parts of the Pareto-front, as it is done by theReference point-guided NSGA-II algorithm (R-NSGA-II) [9]. Thenumber of evaluations needed for the resampling of solutions canbe reduced by intelligent resampling algorithms that allocate justas much sampling budget needed in different situations duringthe optimization run. In this paper we propose and compareresampling algorithms that support the R-NSGA-II algorithm onoptimization problems with stochastic evaluation functions.

Keywords—Evolutionary multi-objective optimization,simulation-based optimization, guided search, reference point,decision support, stochastic systems, dynamic, resampling.

I. INTRODUCTIONIn Multi-objective Optimization many solutions have to be

evaluated to obtain a diverse and converged set of solutions thatcovers the Pareto-front. This set of Pareto-optimal solutions ispresented to the decision maker who chooses only one of themfor implementation. In particular for many-objective optimiza-tion problems a large number of Pareto-optimal solutions isrequired to provide the decision maker with a diverse set tochoose from. In Simulation-based Optimization the numberof available optimization function evaluations is very limited.This is because the modeled systems are usually complex andrequire time-consuming simulation runs. This leads to that itis not possible to find a diverse, converged set of solutions inthe given time. If preference information is available however,the available function evaluations can be used more effectivelyby guiding the optimization towards interesting, preferredregions. One such algorithm for guided search is the Referencepoint-guided NSGA-II (R-NSGA-II) [9]. It is an EvolutionaryAlgorithm (EA) and Multi-Criteria Decision Making (MCDM)technique that uses reference points in the objective spaceprovided by the decision maker and guides the optimizationtowards areas of the Pareto-front close to the reference points.

In Simulation-based Optimization the modeled and sim-ulated systems are often stochastic. To perform an as exactas possible simulation of the systems this stochastic behavioris often built into the simulation models. When running thesimulation this modeled uncertain behavior expresses itself indeviating result values of the simulation. This means that ifthe simulation is run multiple times for a selected parametersetting the result value is slightly different for each simulationrun. In the literature this phenomenon of stochastic evalua-tion functions sometimes is called Noise, respectively NoisyOptimization [1], [3].

If an EA is run without countermeasure on an optimizationproblem with a noisy evaluation function the performancewill degrade in comparison with the case if the true meanobjective values would be known. The algorithm will havewrong knowledge about the solutions’ quality. Two cases ofmisjudgement will occur. The EA will see bad solutions asgood and select them into the next generation. Good solutionsmight be assessed as inferior and can be discarded. Theperformance can therefore be improved by increasing theknowledge of the algorithm about the solution quality.

Resampling is a way to reduce the uncertainty in theknowledge the algorithm has about the solutions. Resamplingtechniques sample the fitness of a solution several timesand use the mean fitness values. In this paper, resamplingmethods are investigated that are designed to help R-NSGA-IIto make good and reliable selection decisions. Another aspectof resampling, which is not considered, is the accuracy of theknowledge about solution quality after the optimization hascompleted. After the end of the optimization, the user has todecide which solution shall be implemented. For this purpose,the user needs accurate knowledge about the solutions andnecessary samples needed to improve accuracy in the end arenot available to the optimization procedure anymore.

There are basic resampling techniques that assign the samenumber of samples to all solutions or for example they assignthe samples dependent on the elapsed optimization time. Moreadvanced techniques base their sampling allocation on theindividual characteristics of the solutions. Those DynamicResampling techniques assign a different number of samplesto different solutions. After each new sample drawn from thefitness of a solution the sample mean usually will change.

More advanced dynamic resampling techniques perform Se-quential Sampling [3], which means that they adjust theirsampling allocation after each new evaluation of the fitness,if new information about the fitness is available. DynamicResampling techniques that perform Sequential Sampling forsingle-objective optimization problems have been proposed byfor instance [6], [11], [4], [20]. Examples of multi-objectivedynamic resampling algorithms are the Multi-Objective Opti-mal Computing Budget Allocation (MOCBA) [13] which hasbeen applied to an EA [5] and Confidence-based DynamicResampling (CDR) [17]. Those algorithms are advanced anddue to their complexity they are difficult to implement byoptimization practitioners and they have limitations. That iswhy in this paper we propose and compare elementary and fun-damental dynamic resampling algorithms with straight-forwardimplementation for multi-objective evaluation functions withthe aim to support guided search. In addition, we presenta resampling algorithm specifically adapted to R-NSGA-II.Some of the evaluated resampling algorithms are extensionsof the single-objective versions. A classification of resamplingalgorithms is given.

For the resampling techniques we evaluate how well theycan support R-NSGA-II as an example for a guided Evolution-ary Multi-objective Optimization (EMO) algorithm. R-NSGA-II is based on the Nondominated Sorting Genetic AlgorithmII [8] which is a widely-used multi-objective evolutionaryalgorithm. NSGA-II sorts the solutions in the combined set ofpopulation and offspring into different non-dominated fronts.Selected are all solutions in the fronts that fit completely intothe next population, starting with the best fronts. From thefront that only fits partially, those solutions are selected intothe next population that have a high crowding distance, whichis a diversity metric. The R-NSGA-II algorithm replaces thecrowding distance operator by the distance to reference points.Solutions that are closer to a reference point get a higherselection priority. The reference points are defined by thedecision maker in areas that are interesting and where solutionsshall be found. As a diversity preservation mechanism R-NSGA-II uses clustering. Solutions that are too close to eachother are considered a low priority. The reference pointscan be created, adapted, or deleted interactively during theoptimization run. If no reference point is defined R-NSGA-II works as NSGA-II. A general flowchart of R-NSGA-II isdisplayed in Figure 1. Due to the complexity of the selectionstep it is displayed in detail in Figure 2.

The rest of the paper is structured as follows. In Section IIa more detailed introduction to dynamic resampling is givenand a classification for resampling algorithms is proposed. InSection III different resampling techniques are presented thatare independent of the optimization algorithm, i.e. they canbe used to support all multi-objective optimization algorithmsand thereby also guided EMO algorithms. The resamplingtechniques are classified according to Section II. In SectionIV a dynamic resampling technique explicitly designed forR-NSGA-II is proposed. In Section V the described resam-pling techniques together with R-NSGA-II are evaluated onbenchmark functions. The test environment is described andthe experiment results are analyzed. In Section VI conslusionsare drawn and possible future work is pointed out.

StartGenerate initial

population P

Select parent solutions for mating(Tournament selection based on

reference point rank)

Crossover andmutation

Selection of new population out of

P ∪ Q

Evaluation of new offspring solutions Q

Generation limitreached?

no

Return final population

yes

Fig. 1. Flowchart R-NSGA-II algorithm.

Non-domination sorting on P ∪ Q

Clustering for each individual front with

cluster radius ε

Current front contains less than the

number of clusters needed?

Select all clusters from this front

Go to next front

yes

Select remaining needed clusters

with best reference point rank

no

Return selected population

Start

Next front available?

yes

no

Start with first front

Choose a representative solution from each of the clusters and select it into

the next population

Remove the selected representative solutions

from the solution set

Select representative

solutions from the clusters

Calculate the reference point rank for all

solutions in the clusters in the front

Set the reference point rank of the clusters to

the best reference point rank of their

solutions

Fig. 2. Flowchart R-NSGA-II algorithm - Selection step.

II. NOISE COMPENSATION VIA RESAMPLINGTo be able to assess the quality of a solution according

to a stochastic evaluation function, statistical measures likesample mean and sample standard deviation can be used. Byexecuting the simulation model multiple times, a more accuratevalue of the solution quality can be obtained. This processis called resampling. We denote the sample mean value ofobjective function Fi and solution s as follows: µn(Fi(s)) =1n

∑nj=1 F

ji (s) and the sample variance of objective function

i: σ2n(Fi(s)) =

1n−1

∑nj=1 (F

ji (s)− µn(Fi(s)))2. The perfor-

mance degradation evolutionary algorithms experience throughthe stochacity of evaluation functions can be compensatedpartly through resampling. However, a price has to be paidfor the knowledge gain through resampling and the resultingperformance gain of the optimization. In a scenario withlimited simulation time the use of resampling will reduce thetotal number of solutions that can be evaluated. Thereby, thesearch space cannot be explored as much as when each solutionis evaluated only once. A trade-off situation arises betweensampling each solution several times and sampling differentsolutions in the search space. In many cases, resampling isimplemented as a Static Resampling scheme. This means thatall solutions are sampled an equal, fixed number of times.The need for resampling however is often not homogeneouslydistributed throughout the search space. Solutions with asmaller variance in their objective values will require lesssamples than solutions with a higher objective variance. Inorder to sample each solution the required number of times,dynamic resampling techniques are used, which are describedin the following section.

A. Dynamic ResamplingDynamic Resampling allocates a different sampling budget

to each solution, based on the evaluation characteristics of thesolution. The general goal of resampling a stochastic objectivefunction is to reduce the standard deviation of the mean of anobjective value σ(µ(Fi(s))), which increases the knowledgeabout the objective value. A required level of knowledge aboutthe solutions can be specified. For each solution a differentnumber of samples is needed to reach the required knowledgelevel. In comparison with Static Resampling, Dynamic Resam-pling can save sampling budget that can be used to evaluatemore solutions and to run more generations of an evolutionaryalgorithm instead.

With only a limited number of samples available, thestandard deviation of the mean can be estimated by thesample standard deviation of the mean, which usually is calledstandard error of the mean. It is calculated as follows [11]:

sen(µn(Fi(s))) =σn(Fi(s))√

n

By increasing the number of samples n of Fi(s) the standarddeviation of the mean is reduced. For the standard error ofthe mean however, this is not guaranteed, since the standarddeviation estimate σn(Fi(s)) can be corrected to a higher valueby drawing new samples of it. Yet, the probability of reducingthe sample mean by sampling s increases asymptotically asthe number of samples is increased.

An intuitive dynamic resampling procedure would be toreduce the standard error until it drops below a certain thresh-old sen(µn(F (s))) < sethr. The required sampling budget

for the reduction can be calculated as n >(σn(F (s))sethr

)2.

However, since the sample mean changes as new samplesare added, this one-shot sampling allocation might not beoptimal. The number of fitness samples drawn might be toosmall for reaching the error threshold, in case the samplemean has shown to be larger than the initial estimate. Onthe other hand, a one-shot strategy might add too manysamples, if the initial estimate of the sample mean was toobig. Therefore Dynamic Resampling is often done sequentially.

For Sequential Dynamic Resampling often the shorter termSequential Sampling is used.

Sequential Sampling adds a fixed number of samples at atime. After an initial estimate of the sample mean and calcu-lation of the required samples it is checked if the knowledgeabout the solution is sufficient. If needed, another fixed numberof samples is drawn and the number of required samples isrecalculated. This is repeated as long as no additional sampleneeds to be added. The basic pattern of a sequential samplingalgorithm is described in Algorithm 1. Through this sequentialapproach the number of required samples can be determinedmore accurately than with a one-shot approach. It guaranteesto sample the solution sufficiently often, and can reduce thenumber of excess samples.

input : Solution s

1. Draw bmin initial samples of the fitness ofs, F (s)

2. Calculate mean of the available fitnesssamples for each of the m objectives:µn(Fi(s)) =

1n

∑nj=1 F

ji (s), i = 1, . . . ,m

3. Calculate objective sample standarddeviation with available fitness samples:

σn(Fi(s)) =√

1n−1

∑nj=1(F

ji (s)− µn(Fi(s))),

i = 1, . . . ,m4. Evaluate termination condition based on

µn(Fi(s)) and σn(Fi(s)), i = 1, . . . ,mStop if termination condition is satisfied,Otherwise sample the fitness of s another ktimes and go to step 2.

Algorithm 1: Basic sequential sampling algorithm pattern

Dynamic resampling techniques can therefore be classifiedas one-shot sampling strategies and sequential sampling strate-gies. In the following section we present several other aspectswhich can be used to classify resampling strategies.

B. Resampling Algorithm ClassificationIn this section we present a classification of resampling

techniques. In the previous section two classification categorieshave been described. First, sampling algorithms that considerobjective value variance and solutions that do not considervariance like Static Resampling. And second, Dynamic Re-sampling algorithms that calculate the sampling budget as one-shot and Dynamic Resampling algorithms that calculate thesampling budget sequentially. More classification categoriescan be defined. A list of categories is presented in Table I.

TABLE I. CLASSIFICATION OF RESAMPLING STRATEGIES

Not considering variance Considering varianceOne-shot sampling Sequential sampling

Single-objective by aggregation Multi-objectiveIndependent resampling Coupled resampling

Individual resampling Comparative resampling

Another way to determine the number of samples is toaggregate the objective values in a single value. For example,the R-NSGA-II algorithm provides such a scalar aggregationvalue on which the sampling budget allocation can be based:The distance to the reference point. A Distance-based Resam-pling strategy could allocate few samples in the beginning

of the optimization runtime when the found solutions are faraway from the reference point. Towards the end of the runtimemore objective value samples can be drawn which helps todistinguish the many solutions that are close to the Pareto-front. On the contrary, a truly multi-objective resamplingstrategy would be to assign the maximum number of samplesto solutions with Pareto-rank 1 and least samples to solutionswith the maximum Pareto-rank.

If we use Distance-based Resampling on R-NSGA-II weclassify the use of this resampling technique as a coupledresampling strategy application. On the other hand if StaticResampling is applied on an algorithm we classify its use asan independent resampling strategy application, since it is nottaking any characteristics of the optimization algorithm intoaccount. This classification category is not always applicablein clear way. If for example the Pareto-rank-based Resamplingalgorithm is applied on the R-NSGA-II algorithm then onlyone part of the algorithm selection procedure is supported.The secondary selection criterion, the distance to a referencepoint, is not supported.

The sampling strategies introduced in the previous sectionconsider only one single solution at a time when makingsampling allocation decisions. The Pareto-rank-based sam-pling strategy however is comparing solutions when allocatingsamples. The former strategy can therefore be classified asIndividual Resampling and the latter strategy as ComparativeResampling.

C. Non-linear sampling allocationMany Dynamic Resampling strategies limit the number

of samples that can be allocated to a solution by an upperbound and an explicit lower bound, as for example Distance-based Resampling. For each solution a value is calculated thatindicates the relative need of resamplings. Based on this valuethe algorithm designer can then decide how many samplesshall be allocated to a solution. The relative sampling needcan be translated directly into samples. For example, if asampling need of 50% is indicated 50% of the span betweenthe maximum and minimum allowed number of samples canbe allocated, on top of the minimum resamplings. This wouldbe a linear sampling allocation.

On the other hand the algorithm designer might transformthis linear allocation. A deferred or decelerated samplingallocation for example might assign 10% of the maximumallowed samples to solutions with 50% sampling need and 80%of the maximum samples to solutions with sample need 90%.The opposite, an accelerated function could reach 50% of theallowed samples already at 10% sampling need. The formulasfor the decelerated and accelerated transformation functions ofthe sampling need x could be: xν , and ν

√x, ν > 1. Mixtures

of both function types are also meaningful. One example isthe cumulative distribution function of the Weibull distribution[12]:

1− e−( xλ )k

with scale parameter λ > 0 and shape parameter k > 1. Thedistribution allows to delay the allocation in the beginning forlower sampling needs and to accelerate for higher samplingneeds. Another example is the logistic growth [16]:

1

(1 + e−γ(x−µ))1ν

where γ is the growth rate, µ the point of highest growth, andν > 0 determines if the maximum growth occurs close to thelower or the upper asymptote. The transformation functionsare displayed in Figure 3.

0 20 40 60 80 1000

10

20

30

40

50

60

70

80

90

100

← x1/4

1− e−(x/0.5)5→

1(1+e−60(x−0.63))0.25

→← x4

Sampling need x (% of max)

Ass

igne

d sa

mpl

es (

% o

f max

)

acceleratedWeibull CDFlogisticdecelerated

Fig. 3. Transformation functions: The number of assigned samples inpercent of the maximum number of samples based on the sampling needin percent. The displayed functions are accelerated growth, the Weibullcumulative distribution function, logistic-, and decelerated growth.

III. ALGORITHM INDEPENDENT STRATEGIESIn this section several resampling algorithms are proposed

and described that can be used to support all types of multi-objective optimization algorithms, inclusive guided EMO algo-rithms like R-NSGA-II. They do not use any specific algorithmcharacteristics and are universally applicable. We classify themaccording to Table I.

We denote: Sampling need for solution s: xs, minimumand maximum number of samples for an individual solution:bmin and bmax. The transformation functions explained inthe previous section are applied on xs to obtain the relativeresampling budget bs. It is then discretised in the followingway which guarantees that bmax is assigned already for bs < 1:

min {bmax, bbs(bmax − bmin + 1)c+ bmin}A. Static Resampling

Static Resampling samples all solution the same amountof times. Regarding the classification, it is not consideringvariance, a one-shot allocation, the number of objectives is notconsidered, it is an independent strategy as all strategies in thissection, and an individual sampling allocation. The samplingbudget is constant, bmin = bmax.

B. Time-based ResamplingTime-based Resampling allocates a small sampling budget

in the beginning of the optimization and a high samplingbudget towards the end of the optimization. The strategy ofthis resampling technique is to support the algorithm whenthe solutions in the population are close to the Pareto-front andto save sampling budget in the beginning of the optimizationwhen the solutions are still far away from the Pareto-front.

Time-based Resampling is a dynamic resampling techniquethat is not considering variance, a one-shot allocation, theobjective functions are not considered, and an individual

sampling allocation. We denote: B = maximum overall numberof simulation runs, Bt = current overall number of simulationruns.

xs =BtB

C. Domination-Strength ResamplingThe term domination-strength is inspired by the strength

value that is used in the SPEA algorithm [19]. It indi-cates the potential of a solution to dominate other solutions.Domination-Strength Resampling assigns more samples tosolutions that dominate many other solutions. Those solutionshave a higher probability to stay in the population duringthe next generations. Therefore, accuracte knowledge abouttheir fitness will be beneficial for algorithm performance. Adisadvantage is that Domination-Strength Resampling is noteffective in many objective optimization where almost allsolutions are non-dominated during most of the optimizationruntime.

Domination-Strength Resampling is not considering vari-ance, performs sequential sampling, is purely multi-objective,and a comparative resampling technique. We denote: S = solu-tion set of current population and offspring, dom(s, S) = num-ber of dominated solutions by s in S, M = maxs∈Sdom(s, S).

xs =

dom(s, S)

M forM > 1

max{0.5, dom(s, S)M } forM = 1

1 forM = 0

For M = 1 we increase the sampling need for solutions that donot dominate other solutions, in order to balance the allocationamong non-dominated solutions. We assume that if M = 1also most dominated solutions with dom(s, S) = 0 will beclose to the Pareto-front and should receive more resamplingbudget.

D. Rank-based ResamplingRank-based Resampling is similar to Domination-Strength

Resampling. It assigns more samples to solutions in the leadingfronts and less samples to solutions in the last fronts, to saveevaluations. In a well-converged population most solutions willhave Pareto-rank 1 and get the maximum number of samples.Like Domination-Strength Resampling it is not applicable inmany-objective optimization.

Rank-based Resampling is not considering variance, per-forms sequential sampling, is purely multi-objective, and acomparative resampling technique. We denote: S = solutionset of current population and offspring, rs = Pareto-rank ofsolution s in S, R = maxs∈Srs.

xs = 1− r − 1

R− 1

E. Standard Error Dynamic ResamplingThe working principle of Standard Error Dynamic Re-

sampling (SEDR) was introduced in Section II-A. It addssamples to a solution sequentially until the standard error ofthe objectives falls below a chosen threshold value. It wasproposed in [11] for single-objective optimization problems.In this study we apply SEDR on multi-objective problemsby aggregating all objective values to a scalar value. Asaggregation the median of the objective standard errors is used.A pseudocode description of SEDR is listed in Algorithm 2.

Regarding the classification, it is considering variance, isa sequential sampling procedure, uses an aggregated valueof the objectives, and determines the resampling budget forall solutions individually. Since the algorithm is using thestandard deviation multiple initial resamplings are needed forall solutions.

input : Solution s

1. Draw bmin ≥ 2 initial samples of the fitnessof s, F (s)

2. Calculate mean of the available fitnesssamples for each of the m objectives: µi (s),i = 1, . . . ,m

3. Calculate objective sample standarddeviation with available fitness samples:

σi(s) =√

1n−1

∑nj=1(F

ji (s)− µi(s))2, i = 1, . . . ,m

4. Calculate objective standard errorssei(s) =

σi(s)√n, i = 1, . . . ,m

5. Calculate an aggregation se(s) of theobjective standard errors sei(s)

6. Stop if se(s) < sethr or if bs = bmax,Otherwise sample the fitness of s another ktimes (not more than bmax) and go to step 2.

Algorithm 2: Standard Error Dynamic Resampling (SEDR)

F. Combinations of multiple criteriaThe resampling algorithms described so far all have short-

comings. Combinations of several algorithms which considermultiple criteria are expected to improve the sampling alloca-tion. The following combinations are conceivable:• Time-based + SEDR• Domination strength/Pareto-rank + Time-based• Domination strength/Pareto-rank + SEDR• Even combinations of all three are imaginable

Combinations with time can be realized by increasing bmaxwith time. Combinations with SEDR can be done in two ways:The progress indicated by the supporting resampling techniquecan control either bmax or sethr.

In the next section we present a resampling algorithm thatis specifically adapted to the R-NSGA-II algorithm.

IV. DISTANCE-BASED DYNAMIC RESAMPLINGIn this section we propose a resampling strategy that uses

the information that is available through the reference points ofR-NSGA-II. It assigns more samples to solutions that are closeto the reference points. We call it Distance-based DynamicResampling. The intention of allocating samples in this way isthat solutions that are close to a reference point are likely tobe close to the preferred area on the Pareto-front. They haveconverged and it is important to be able to tell reliably whichsolutions are dominating other solutions.

According to the classification in Table I Distance-basedDynamic Resampling is not considering variance, performssequential sampling, uses the reference point distance whichis an aggregated form of the objective values, is a coupledresampling method that allocates the sampling budget foreach solution individually. We denote: Rs is the referencepoint that is closest to solution s. In this study, we use anAchievement Scalarization Function [15] as distance metric.δASF (F (s), Rs) is the absolute reference point distance. Forthe sampling allocation we use the normalized reference point

distance ds = min{1, δASF (F (s),Rs)D }, where D is the maxi-

mum reference point distance of the initial population of R-NSGA-II. An intuitive way to assign resamplings based onreference point distance is as follows:

xs = 1− dsReference points are usually not chosen on the Pareto-

front. Therefore adaptive non-linear transformation functionsare needed which are described in the following section.

A. Adaptive Transformation FunctionsIn Section II-C we proposed transformation functions

with the purpose of optimizing the sampling allocation. ForDistance-based Resampling however, more complex transfor-mation functions are needed since in most cases the referencepoints will not be situated on the Pareto-front. They are eitherfeasible or infeasible and both cases need to be consideredseparately. The definition of a feasible reference point is thatit is part of the Pareto-front or a solution can be foundthat dominates it. An infeasible reference point is not on thePareto-front and no solution can be found that dominates it.Both infeasible and feasible cases require an approach thatadapts the transformation function dynamically during theoptimization runtime. It is not known from the beginningwhether the reference point is feasible or infeasible. In thefollowing two subchapters we propose adaptive transformationfunctions for both cases. First it is assumed that the referencepoint is infeasible and as soon as a solution is found thatdominates the reference point the transformation function forthe feasible case is used.

1) Adaptive Strategy for Infeasible Reference Points: Inthis section we propose a method how to adapt the transfor-mation function in the case of an infeasible reference point.We use this transformation function as long as no solutionis found that dominates the reference point. That means thatfeasible reference points on the Pareto-front are handled inthe same way as infeasible reference points. In case of aninfeasible reference point the reference point distance of thesought-after, focused Pareto set cannot be reduced below acertain distance δ by optimization. This scenario is displayed inFigure 4. That means that if this lower bound δ of the referencepoint distance is large enough and a linear or delayed distance-based allocation scheme is used the maximum sampling budgetwill never be reached during the optimization runtime. Toofew samples will be assigned to the last populations thatinclude the best found solutions. Therefore, in order to create a

min

f2

min f1

R

Pareto-front

Result population

Fig. 4. Bi-objective min/min-problem scenario with infeasible reference pointR. The distance of the result population to R is limited by a lower bound δ.

good distance-based sampling strategy for infeasible referencepoints, the budget allocation has to be accelerated. Then, itis not necessary to come close to the reference point to getthe highest number of samples. The full number of sampleswill already be allocated to solutions at a certain distance tothe reference point. Since the lower bound of the referencepoint distance is not known the sampling allocation has to beaccelerated iteratively. The challenge is to adapt the transfor-mation automatically so that in the end of the optimization runthe samples are distributed among all found solutions as if thereference point was feasible and on the Pareto-front.

To achieve this the distance indicator is combined with twoadditional criteria, as it was done for the algorithm independentstrategies in Section III-F:

1. The progress of the population during the optimizationruntime and 2. The elapsed time since the start of the opti-mization. In contrary to the algorithm independent strategiesthis algorithm has access to distance information to referencepoints and can measure progress. The optimization progresswill slow down as soon as the population approaches thePareto-front which is at some distance to the reference point.In this way it can be detected that the algorithm is convergingtowards the preferred area and that more samples per solutionare needed. However, the reference point distance and theoptimization progress alone would be not sufficient since thealgorithm might temporarily get slowed down by local optimain an early stage and show only little progress. This can forexample occur for the ZDT4 function [18] which featuresmany local Pareto-fronts. In that case the goal is to escape thelocal optimum which does not require more exact knowledgeabout the solutions and no increased number of samples. Thatis why in the case of infeasible reference points the referencepoint distance, progress, and elapsed time together are takeninto account for the sampling allocation.

We propose to measure the optimization progress by theaverage reference point distance of the population, on averagefor the last n populations. This metric will be described indetail in the experiment section of this paper, Section V-A. Theaverage improvement in reference point distance of the last npopulations is used as progress indicator. The transformationfunction is designed as an ascending polynomial function.Initially, it will assign less than the maximum allowed samplesto the solution that is closest to the reference point in orderto be prepared for the case of a feasible reference point. Ifthe reference point is feasible the solutions that dominate thereference point should get the highest number of samplesinstead. As long as we can assume infeasibility and if theoptimization progress per generation drops the slope of thetransformation function will be increased in several steps. Ifthe progress for a generation is less than 10% per generationthe transformation function will assign the maximum allowednumber of samples only to the (hypothetical)solutions that areclosest to the reference point. If the average progress is below5% the transformation function will be adapted in a way thatat least 10% of the population are guaranteed full samples.Accordingly, a progress < 2.5% corresponds to 20% of thepopulation with full samples and a progress < 1% to 40%.

As transformation function x2s = (1 − ds)2 is used. Thecalculation is done by taking a percentage of the populationand measuring the maximum relative distance m of this subsetS to the reference point, m = maxs∈Sds. The transformationfunction is adapted through a factor r that makes sure that the

maximum number of samples is reached already at distancem to the reference point.

r(1−m)2 := 1 ⇒ r =1

(1−m)2

However, as mentioned above, if the average populationprogress is above 10% then the sampling allocation will bereduced by using r = 1 − m, where m is the maximumdistance of the best 10% of the population. In case that itwill be detected that the reference point is feasible this allowsa smooth transition to the feasible case which uses a similarallocation function. The transformation functions are definedas

min{1, r(1− ds)2}.This allocation scheme is combined with a sample alloca-

tion based on the elapsed optimization runtime. The elapsedtime is measured by the number of executed solution evalu-ations in relation to the maximum number of evaluations, asit is done by Time-based Resampling in Section III-B. Thetransformation function is adapted by reducing r to delay theincline of the sampling allocation. Before 50% of the time haspassed the distance m will be reduced to m0 = 0, i.e. r = 1.Until 65% m will be reduced to m1 = 1/3m and until 80% itwill be reduced to m2 = 2/3m. The transformation functionsare shown in Figure 5.

0 20 40 60 80 1000

10

20

30

40

50

60

70

80

90

100

2.52x2 →1.76x2 →1.3x2 →

← x2← 0.75x2

← δ

Sampling need x=1−d (% of max)

Ass

igne

d sa

mpl

es (

% o

f max

)

Time ≥ 80%Time < 80%Time < 65%Time < 50%Progress ≥ 10%

Fig. 5. Transformation functions for infeasible reference point. The samplingneed x in percent based on the relative reference point distance d is translatedinto a relative number of samples that are allocated to a solution.

As soon as a solution is found that dominates the referencepoint the sampling allocation will switch to the allocationfunction for the case of feasible reference points, startingwith the next generation of R-NSGA-II. If this happens at anearly stage then only a lesser change in sampling allocationwill occur and a smooth transition will happen since thenthe transformation function for infeasible reference points issimilar to the one for feasible reference points. The feasiblecase is described in the next section.

2) Adaptive Strategy for Feasible Reference Points: In thebeginning of the optimization runtime the decision makermight define a reference point that is feasible. That meansthat it is possible to find a solution that features exactly the

same objective values as the reference point. This may happensince in most real-world cases the Pareto-front is not knownbefore the optimization is started. The reference point mightbe set too conservatively and during the optimization run itmight be detected that solutions better than the reference pointcan be found. A scenario with a feasible reference point isdisplayed in Figure 6. R-NSGA-II can handle this case [9],

min

f2

min f1

R

Pareto-front

Population

Fig. 6. Bi-objective min/min-problem scenario with feasible reference pointR.

i.e. the optimization will still move towards the preferred partof the Pareto-front. This is because the Pareto-dominance is theprimary fitness criterion that is checked before the solutionsare compared by the reference point distance.

For the sampling allocation however, this would mean thatthe allocation would be reversed. Solutions that are closerto the Pareto-front are further away from the reference pointand would be sampled less. We therefore propose a samplingallocation that is based on an imaginary reference point thatis not dominated by any solution in the population.

For R-NSGA-II we recommend that the reference pointis adapted in case of a feasible reference point, given thatthe decision maker is available. This is because an infeasiblereference point or a point on the Pareto-front will help R-NSGA-II to convergerge faster. This is important since weassume a limited optimization time. One might also thinkof automatic methods for reference point adaption based onpreviously found information about the optimization problem.In this paper those approaches are not considered. Instead anadaptive transformation function is proposed that will increasethe sampling allocation for solutions that are expected to beclose to the interesting area on the Pareto-front.

We propose a distance-based sampling allocation functionthat is activated as soon as a solution is found that dominatesthe reference point. It is designed to imitate the samplingallocation as if the reference point would be infeasible orsituated on the Pareto-front. For this purpose the referencepoint is replaced by an imaginary reference point and thesampling allocation is based on the distances to this point.As the imaginary reference point the objective vector of thesolution is used that is non-dominated, dominates the referencepoint, and is closest to it. This point is only used for thesampling allocation, not for guidance of the algorithm.

If the imaginary reference point is far from the interestingarea then this could lead to a suboptimal sampling distribution.This is likely to happen if the Pareto-front is for examplealmost vertical or horizontal in a 2D objective space. Theinteresting area is a set of solutions on the Pareto-front that are

closest to the reference point, for example the closest solutionand all solutions on the Pareto-front that have a 10% higherdistance. A solution that is in the interesting area but does notdominate the reference point will be closer to the referencepoint than the imaginary reference point which is far from theinteresting area. Thereby, it will not get the maximum possiblenumber of samples. This is a disadvantage of our approach,to only define solutions as imaginary reference points thatdominate the reference point. This disadvantage is accepted inorder to keep the approach as simple as possible. An examplescenario is displayed in Figure 7. However, this suboptimalsampling allocation is only temporary. R-NSGA-II itself isnot directly influenced by the imaginary reference point, onlythe sampling allocation is. As the optimization progresses the(sub-)population and thereby the imaginary reference pointwill move towards the interesting area.

min

f2

min f1

RPareto-front

Population

IR

P

Fig. 7. Bi-objective min/min-problem scenario with feasible reference pointR and almost horizontal Pareto-front. The imaginary reference point IR is thesolution that is non-dominated, dominates R, and is closest to R among thesolutions that dominate R. In this scenario IR is distant from the preferredarea due to the characteristics of the Pareto-front. Solution P however ispreferred but will not get the maximum number of samples.

Likewise to the infeasible case we would like to usethe progress and the elapsed time as additional criteria forsampling allocation. The time can be used in the same way asfor the infeasible case but the progress is difficult to measure.The imaginary reference point will move as the optimizationis progressing which means that if the population follows thispoint then the progress is not measurable by the distance to theimaginary reference point. For progress measurements a fixedpoint close to the preferred area on the Pareto-front wouldbe needed. For reasons of simplicity we set aside progressmeasurements for the feasible case in this paper.

The transformation function maps the relative samplingneed xs = 1− ds to the sampling allocation in percent, whereds is normalized reference point distance as defined above,however to the imaginary reference point. The function that isused is defined as

r(1− ds)2, r ≤ 1.

The factor r is controlled by the elapsed time. Until some timehas passed it is expected that the population is still progressing.Many solutions would be assigned a high sampling budgetsince the imaginary reference point is defined as a solutionthat is part of the population which is converging towardseach other in a cluster close to the preferred area. Thereforethe allocation is delayed initially. Unless 50% of the time haspassed the sampling allocation is slowed down with r = 1/4.

Until 65% has passed r is chosen as r = 1/2, until 80% r is3/4. After 80% of the time the allocation is not slowed down.The transformation functions are displayed in Figure 8.

0 20 40 60 80 1000

10

20

30

40

50

60

70

80

90

100

x2 →

0.75x2 →

0.5x2 →

0.25x2 →

Sampling need x=1−d (% of max)

Ass

igne

d sa

mpl

es (

% o

f max

)

Time ≥ 80%Time < 80%Time < 65%Time < 50%

Fig. 8. Transformation functions for feasible reference point. Assumption ofan imaginary reference point.

B. Considering VarianceAnalogously to Section III-F Distance-based Dynamic Re-

sampling can be combined with SEDR. Then all of distance,progress, time, and variance would be considered together.

V. NUMERICAL EXPERIMENTSIn this section the proposed resampling techniques in

combination with R-NSGA-II are evaluated on benchmarkfunctions. Experiments are done for the two-dimensional ZDTbenchmark function ZDT1 [18], with an additive noise ofσ = 0.15.

A. EvaluationTo measure the result quality of the optimization, each

solution that is part of the found non-dominated set is sampledadditional 100 times after the optimization is finished. In thisway accurate objective values can be obtained. The samplingbudget needed for the post-processing is not taken from thetotal sampling budget of the experiments.

As evaluation metric, we use the average distance of theresult population to the reference point. To eliminate theinfluence of outliers, only an α-subset of the population is usedin the calculation, α ∈ [0, 1]. The subset contains the solutionsthat are closest to the reference point. For the experimentsα = 0.5 is used, which means that half of the populationinfluences the performance measurement.

Performance measurements are made after every genera-tion. Due to the resampling techniques, the number of eval-uations per generation varies between different generations,algorithms, and even different optimization runs. In order toobtain measurement values in equidistant time intervals, themeasurement data is interpolated. All experiments are run5 times, and average interpolated performance measurementvalues are used to compare the algorithms.

B. Parameter ConfigurationFor all experiments a simulation budget of 5000 is used.

R-NSGA-II pameter values are: population size=50, number ofoffspring=50, crossover probability=0.8, and mutation proba-bility=0.07.

C. Proof of conceptIn Figure 9 the optimization process of R-NSGA-II with

Time-based Resampling is displayed. The optimization processwith SEDR is shown in Figure 13, and Distance-based Re-sampling in Figure 14. The color values indicate the number

Fig. 9. EMO with R-NSGA-II and Time-based Resampling on ZDT1, noiselevel σ = 0.15, reference point (0.5, 0), 5000 evaluations. The color valuesindicate the number of samples (red=low, green=high).

of samples (red=low, green=high). The sampling allocationof the time-based experiment is shown in Figure 10. If aftersome time, the number of allocated samples has increased, thenew number of resamplings will be applied as soon as a newgeneration has started. This short-time allocation delay is donein order to keep the allocation transparent and understandable.A linear transformation function was used in this experiment.If a delaying transformation function would be used, theallocation looks as in Figure 11. With the delayed allocationmore different solutions can be evaluated, since the samplesare spent more economically on the solutions.

Fig. 10. Sampling distribution of Time-based Resampling as in Figure 9.Linear transformation function bs = xs. 5000 samples, which corresponds toapproximately 2500 different solutions.

Fig. 11. Sampling distribution of Time-based Resampling. Delayed transfor-mation function bs = x2s . 5000 samples, which corresponds to approximately3400 different solutions.

D. Performance measurementsIn this section the resampling strategies are tested in

combination with R-NSGA-II. They are Static, Time-based,Domination strength-based, Pareto rank-based, SEDR, andDistance-based. The upper sampling limit is bmax = 3. Theperformance metric values over time are shown in Figure 12.It can be observed that Static Resampling performs worst, thetime-based strategy and Distance-based Resampling performbest. The Domination Strength- and Rank-based algorithmsshow medium performance. They allocate a higher samplingbudget throughout the runtime which gives them a disadvan-tage. The same effect, but worse, can be seen for SEDR whichallocates many samples to compensate for the high noise level.

0 20 40 60 80 1000

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Time (%)

Ref

eren

ce P

oint

Con

verg

ence

Static 3SEDR 1−3Domination−strength 1−3Rank−based 1−3

Distance−based bs=x

s3

Time−based 1−3

Distance−based bs=x

s5

Fig. 12. R-NSGA-II performance over time supported by different resamplingalgorithms on ZDT1, noise level σ = 0.15. The X-axis shows the time inpercent. The Y-axis shows the average distance to the reference point (0.5, 0)of α = 50% of the population. 5000 evaluations.

VI. CONCLUSIONSIt has been shown that all proposed resampling strate-

gies perform better than Static Resampling. The specificallyadapted Distance-based Resampling algorithm for R-NSGA-II and Time-based Resampling show the best performance.Comparative resampling algorithms like Domination StrengthResampling are promising methods.

A. Future workFor future work the following studies are proposed:• The performance metric that is used measures only

how well the algorithm has converged towards thereference point. A future task would be to createa performance metric for guided search that alsomeasures diversity.

• Another worthwile study would be to create a re-sampling algorithm that explicitly supports selectionsampling, i.e. that maximizes the probability of correctselections.

• The trade-off between achieving a good optimizationresult and an accurate knowledge of the fitness of theresult population will be investigated.

Fig. 13. EMO with R-NSGA-II and SEDR on ZDT1, noise level σ = 0.15,reference point (0.5, 0), 5000 evaluations.

Fig. 14. EMO with R-NSGA-II and Distance-based Resampling on ZDT1,noise level σ = 0.15, reference point (0.5, 0), 5000 evaluations.

ACKNOWLEDGMENTSThis study was partially funded by VINNOVA, Sweden,

through the FFI-HSO project. The authors gratefully acknowl-edge their provision of research funding.

REFERENCES[1] T. Bartz-Beielstein, D. Blum, and J. Branke, “Particle swarm optimiza-

tion and sequential sampling in noisy environments,” Metaheuristics –Progress in Complex Systems Optimization, pp. 261–273, 2007.

[2] J. Branke and J. Gamer, “Efficient sampling in interactive multi-criteriaselection,” in Proceedings of the 2007 INFORMS Simulation SocietyResearch Workshop, pp. 42–46, 2007.

[3] J. Branke and C. Schmidt, “Sequential sampling in noisy environments,”Parallel Problem Solving from Nature - PPSN VIII - LNCS, vol. 3242,pp. 202–211, 2004.

[4] C. H. Chen, C. H., D. He, M. Fu, and L. H. Lee, “Efficient SimulationBudget Allocation for Selecting an Optimal Subset,” Informs Journalon Computing, vol. 20, no. 4, pp. 579–595, 2008.

[5] C. H. Chen and L. H. Lee, Stochastic simulation optimization -An optimal computing budget allocation. World Scientific PublishingCompany, 2010.

[6] C. H. Chen, J. Lin, E. Yucesan, and S. E. Chick, “Simulation budgetallocation for further enhancing the efficiency of ordinal optimization,”Journal of Discrete Event Dynamic Systems: Theory and Applications,vol. 10, pp. 251–270, 2000.

[7] K. Deb, Multi-Objective Opimization using Evolutionary Algorithms.John Wiley & Sons, 2001.

[8] K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, “A fast and elitistmulti-objective genetic algorithm: NSGA-II,” IEEE Transactions onEvolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002.

[9] K. Deb, J. Sundar, U. Bhaskara Rao N., and S. Chaudhuri, “Referencepoint based multi-objective optimization using evolutionary algorithms,”International Journal of Computational Intelligence Research, vol. 2,no. 3, pp. 273–286, 2006.

[10] K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, “Scalable test problemsfor evolutionary multiobjective optimization,” Evolutionary Multiobjec-tive Optimization: Theoretical Advances and Applications, ch. 6, pp.105–145, 2005.

[11] A. Di Pietro, L. While, and L. Barone, “Applying evolutionary algo-rithms to problems with noisy, time-consuming fitness functions,” inProceedings of the IEEE Congress on Evolutionary Computation 2004,vol. 2, pp. 1254–1261, 2004.

[12] A. M. Law, W. D. Kelton, Simulation Modeling and Analysis. Thirded., McGraw Hill, 2000.

[13] L. H. Lee, E. P. Chew, S. Y. Teng, and D. Goldsman, “Finding thenon-dominated Pareto set for multi-objective simulation models,” IIETransactions, vol. 42, no. 9, pp. 656–674, 2010.

[14] F. Siegmund, A. H.C. Ng, and K. Deb, “Finding a preferred diverse setof Pareto-optimal solutions for a limited number of function calls,” inProceedings of the IEEE Congress on Evolutionary Computation 2012,Brisbane, pp. 2417–2424, 2012.

[15] F. Siegmund, J. Bernedixen, L. Pehrsson, A. H.C. Ng, and K. Deb,“Reference point-based Evolutionary Multi-objective Optimization forIndustrial Systems Simulation,” in Proceedings of the Winter SimulationConference 2012, Berlin, 2012.

[16] A. Tsoularis, “Analysis of Logistic Growth Models,” Research Lettersin the Information and Mathematical Sciences, vol. 2, pp. 23–46, 2001.

[17] A. Syberfeldt, A. H.C. Ng, R.I. John, and P. Moore, “Evolutionaryoptimisation of noisy multi-objective problems using confidence-baseddynamic resampling,” European Journal of Operational Research vol.204, no. 3, pp. 533–544, 2010.

[18] E. Zitzler, K. Deb, and L. Thiele, “Comparison of multiobjectiveevolutionary algorithms: Empirical results,” Evolutionary Computation,vol. 8, no. 2, pp. 173–195, 2000.

[19] E. Zitzler and L. Thiele, “Multiobjective Evolutionary Algorithms: AComparative Case Study and the Strength Pareto Approach,” IEEETransactions on Evolutionary Computation, vol. 3, no. 4, pp. 257–271,1999.

[20] S. Zhang, L. H. Lee, E. P. Chew, C. H. Chen, and H. Y. Jen, “Animproved simulation budget allocation procedure to efficiently selectthe optimal subset of many alternatives,” in Proceedings of the 2012IEEE International Conference on Automation Science and Engineering(CASE), pp. 230–236, 2012.