Tabu Search When Noise is Present: An Illustration in the Context of Cause and Effect Analysis

P1: JSN/VSK P2: JSN

Journal of Heuristics KL577-01-Costa April 13, 1998 8:54

Journal of Heuristics, 4: 5–23 (1998)c© 1998 Kluwer Academic Publishers

Tabu Search When Noise is Present: An Illustrationin the Context of Cause and Effect Analysis

DANIEL COSTAGroupe de Statistique, Universite de Neuchatel, 2002 Neuchatel, Switzerland

EDWARD A. SILVERFaculty of Management, University of Calgary, Calgary, Alberta, Canada, T2N 1N4

Abstract

In the field of combinatorial optimization, it may be possible to more accurately represent reality through stochas-tic models rather than deterministic ones. When randomness is present in a problem, algorithm designers facenew difficulties which complicate their task significantly. Finding a proper mathematical formulation and a fastevaluation of the objective function are two major issues. In this paper we propose a new tabu search algorithmbased on sampling and statistical tests. The algorithm is shown to perform well in a stochastic environmentwhere the quality of feasible solutions cannot be computed easily. This new search principle is illustrated in thefield of cause and effect analysis where the true cause of an undesirable effect needs to be eliminated. A set ofnpotential causes is identified and each of them is assumed to be the true cause with a given probability. The time toinvestigate a cause is a random variable with a known probability distribution. Associated with each cause is thereward obtained if the cause is really the true cause. The decision problem is to sequence then potential causesso as to maximize the expected reward realized before a specified time horizon.

Key Words: cause and effect analysis, sampling, statistical tests, stochastic combinatorial optimization, tabusearch

1. Introduction

Combinatorial optimization is a domain of discrete mathematics that has attracted theattention of many researchers in recent years. This ever increasing interest is due to thefact that many real world situations can be modeled in terms of combinatorial optimizationproblems. Given a finite setX of feasible solutions and an objective functionf assigningto each solutions∈ X a real valuef (s), the goal is to find a solutions∗ ∈ X for whichf (s∗) is maximum. Restriction to function maximization is without loss of generality sincemin f (s)=−max(− f (s)).

General search methods, such as tabu search, simulated annealing and evolutionaryalgorithms, have already proven to be very effective for solving difficult problems in diversedomains (Osman and Laporte (1996)). Most of the time, however, these methods have beenused in a very specific framework. Until now they have essentially been applied to situationswhere all components of the underlying problem (e.g., data, variables, objective function,constraints, etc.) are assumed to be known with certainty. Unfortunately, such is not oftenthe case in reality. Uncertainties in the real world are such that most practical applications

P1: JSN/VSK P2: JSN


6 COSTA AND SILVER

incorporate stochastic and dynamic elements. In order to further bridge the gap betweentheory and practice, new implementations of the above search techniques should morecarefully take account of these uncertainties, i.e., not only deal with the expected or themost likely values of the random elements. Those who design algorithms usually face twomajor difficulties in a stochastic environment. First, it is hard to find a proper mathematicalformulation of the problem under study. Sometimes, the introduction of randomness, such asin a stochastic process, even destroys the combinatorial structure of the problem (Bertsimasand Simchi-Levi (1996)). Second, the objective function can seldom be evaluated accuratelyor efficiently.

Besides these two key issues, it is necessary to take account of various situations where thelack of certainty about the set of feasible solutions gives rise to memorization, combinationand trajectory problems. Although these situations should be handled from case to case,on the basis of the method implemented and the optimization problem under study, we willshow that general statistical procedures can be useful for implementing a heuristic method ina stochastic environment. In this paper we place emphasis on neighborhood search methods,and more specifically on the well-known tabu search method developed by Glover (1986),not only because we believe it is one of the most prominent neighborhood search techniquesin the deterministic case, but also because attractive features can be readily incorporatedthat take into account the stochastic aspects of a problem.

The research leading to this paper was motivated by a stochastic problem in the field ofcause and effect analysis (Silver and Rohleder (1997)) where it is hard to compute the objec-tive function f . However, the extend of this study is not limited to this particular problem.There are a large number of applications where noise is present andf cannot be evalu-ated simply, including, for example, problems in inventory management, job scheduling,location and routing. Various search techniques have already been developed for solvingthese problems, but, as already said, most problems were tackled in a deterministic way bymaking certain assumptions to cope with uncertainties.

Combinatorial optimization problems with random elements have received little attentionin the literature. It is only in the late 80s that researchers started to investigate such prob-lems. Several studies have been devoted to traveling salesman or vehicle routing problemsin which uncertainty (in customer’s presence, location or demand, etc.) plays an essentialrole. Different types of models and stochastic programming techniques have been pro-posed in (Jaillet (1988), Bellalouna, Murat, and Paschos (1995), Bertsimas and Simchi-Levi(1996), Gendreau, Laporte, and S´eguin (1995), Benton and Rossetti (1993)). Other stochas-tic optimization algorithms have been examined, for example, in (Yan and Mukai (1992),Andradottir (1995), Barton and Ivey (1996)). The main drawback of these algorithms is thatthey are based on rather severe assumptions and, thus, are not readily usable in practice. Toour knowledge, the only neighborhood search approach capable of handling the stochasticnature of a real world combinatorial optimization problem is due to Gendreau, Laporte,and Seguin (1996). The major contribution of their paper is the development of an easilycomputed approximation for the objective function, to be used in the evaluation of potentialmoves at each step of a standard tabu search algorithm.

The remainder of this paper is organized as follows. In the next section we describe theproblem to be solved in the context of cause and effect analysis. An optimal solution of this

P1: JSN/VSK P2: JSN


TABU SEARCH WHEN NOISE IS PRESENT 7

problem can be found with the branch and bound algorithm presented in Section 3. Dueto its exponential complexity this algorithm is applicable only to problems of small size.Section 4 is devoted to the detailed presentation of a new version of tabu search where theobjective function is evaluated through sampling and the search for good potential moves isgoverned by statistical tests. The proposed algorithm is tested on a set of randomly generatedproblems. The experimental results are reported in Section 5. The paper concludes withsome summary comments and directions for future research.

2. Problem description

Cause and effect analysis (Rohleder and Silver (1997)) is a common approach used inquality or process improvement. The context in which it is used involves an ascertained,undesirable effect and the search for its underlying cause. Early in the analysis a number ofpossible causes are identified. The identification of the most probable causes is usually theresult of a subjective and creative process that involves, for example, group brainstorming.In this paper, we will assume that there is a single true cause of the undesirable effect andthat a set ofn possible causes has been identified with a priori probabilitiespi that causei is the true cause (i = 1, . . . ,n). The problem is how to decide on the order in whichwe should investigate the possible causes. The investigation of causes must be completedbefore a specified amount of timeT . The timeti to investigate causei is a random variablewith a known cumulative distribution functionFi (t). Once the investigation of a cause iscommenced, it must be carried through to completion. Each causei yields a quantifiablerewardr i if it is the true cause. The rewards can vary between causes because some causescould lead to more serious problems if not eliminated. We assume that the investigationtimesti are independent and that the probabilitiespi and the rewardsri are known a prioriand fixed throughout the ordering process.

Knowing the total investigation timeT and the coefficientspi , ti , ri of each possiblecause (i = 1, . . . ,n), the objective is to establish a fixed sequence of then causes so asto maximize the expected reward by timeT . In this study we refer to this problem asthestochastic ordering problem with time constraint(SOPTC). Its mathematical formulationis as follows:

Max f (i1, i2, . . . , i n)= ri1 · pi1 · Fi1(T)+ ri2 · pi2 · Fi1i2(T)+ · · ·+ rin · pin · Fi1···i n(T)

s.t. (i1, i2, . . . , i n) ∈ Perm(n)

(1)

where

• i k ∈ {1, . . . ,n} is the cause investigated in thekth position (k = 1, . . . ,n).• Fi1···i k(T) = Prob(ti1 + ti2 + · · · + tik ≤ T) is the probability that the total time of inves-

tigating causesi1, . . . , i k is less than or equal toT (the convolution of the distributionsof ti1, ti2, . . . , tik ).• Perm(n) is the group of all permutations of{1, . . . ,n}.

P1: JSN/VSK P2: JSN


8 COSTA AND SILVER

Thekth term of (1) represents the expected reward if thekth cause investigated is the truecause and we are able to complete its investigation before timeT .

Specifying the entire sequence of investigation ahead of time is equivalent to making twoimplicit assumptions. First, the investigation of any specific cause does not give any newinformation on the relative values of the probabilities of the remaining causes. Second, weare not permitting the remaining sequence to be dynamically changed as a function of thetime remaining.

In the situation where theti ’s are known exactly, the above problem can be formulatedas a knapsack problem. Indeed, letyi = 1 if causei is investigated before timeT and 0otherwise. Then the problem amounts to selecting the binary variablesyi so as to

Maximizen∑

i=1

ri · pi · yi

s.t.n∑

i=1

ti · yi ≤ T.

(2)

The knapsack problem is known to be a NP-hard problem (Garey and Johnson (1979)).Thus, so is the problem we face in this study.

There are numerous situations that can be represented by a mathematical formulationsimilar to (1). Let us consider an object hidden in one ofn boxes. Letpi be the prob-ability of finding the object in boxi, ti the time required to perform a search in boxiandri = 1∀ i = 1, . . . ,n. We wish to organize the search through then boxes in order tomaximize the chance of finding the object before a time horizonT .

Another situation where expression (1) holds is the manuscript submission problem(Chun, Moskowitz, and Plante (1995)). Suppose we want to submit a manuscript to a journalfor possible publication. In totaln journals have been identified as possible outlets. Sincea manuscript cannot be submitted to more than one journal simultaneously, there is a needto decide where to send it first. The manuscript is accepted in journali with probability pi ,yielding a rewardri to the author. The turn-around time between submission and acceptance(or rejection) is given byti . The manuscript is sequentially submitted to different journalsuntil accepted and thus the goal is to arrange the order of journals so as to maximize theexpected reward within a certain period of timeT (perhaps the time remaining until atenure or promotion decision is taken). The manuscript submission problem differs slightlyfrom SOPTC inasmuch as the true cause (i.e., the journal accepting the manuscript) is notnecessarily unique. However, this does not affect the objective function in (1) because theinvestigation of causes is stopped as soon as one “true cause” has been identified. Moreover,the solution of (1) does not require that the sum of thepi ’s be no larger than unity.

Chun, Moskowitz, and Plante (1995) give a brief review of the literature on deterministicordering problems when no time constraints are present. A number of issues are addressedin their paper and it is shown that, in some cases, it is possible to derive an optimal solutionby sequencing the alternatives (i.e., the causes in our context) on the basis of simple orderingparameters. However, they show that the case where the process must be terminated withina limited period of time is hard to solve (like the knapsack problem) and an algorithm basedon dynamic programming is proposed.

P1: JSN/VSK P2: JSN



Silver and Rohleder (1997) provide some relatively simple mathematical models tofacilitate decision making (by a non-technical audience) in the field of cause and effectanalysis. The situation where there is a time constraintT and uncertainties in the investiga-tion timesti is addressed. The authors propose a greedy heuristic which builds a sequencestep by step by repeatedly selecting the causei k ∈ {1, . . . ,n}\{i1, . . . , i k−1} that maximizesthe expressionr ik · pik · Fik(T − t1− · · · − tk−1), wheret1, . . . , tk−1 are the observed inves-tigation times of the causes examined earlier in the sequence. The procedure stops as soonas the true cause has been identified ort1 + · · · + tk ≥ T , which means that it failed infinding the true cause within the elapsed time.

Unless the individual distributionsFi ’s have a very specific form, it is usually difficult,if not impossible, to evaluate the termsFi1···i k(T) in expression (1). In order to comparethe performance of the stochastic tabu search algorithm developed in Section 4 with itsdeterministic version, as well as with an exact procedure, it is essential to consider a classof distributions for whichFi1...i k(T) can be computed easily. For this reason we have decidedto restrict our attention to normal variables. In the remainder of this paper, the investigationtimesti will be assumed to be normally distributed with meanµi and varianceσ 2

i . In short,we will write ti ∼ N(µi , σ

2i ) and we will use the symbol8 for the cumulative distribution

function of a standard normal variable. Normal variables are additive and, thus,Fi1···i k(T) issimply the distribution function of a normal variable with meanµi1 + · · · +µi k and varianceσ 2

i1+· · ·+σ 2

i k(the latter based on an independence assumption). To limit the occurrence of

negative investigation times which make no sense from a practical standpoint, we impose0<σi <

µi

3 . Under these circumstances the probability ofti taking on a negative value isat most8(−3) = 0.0013. It should be stressed that the modified tabu search, proposed inthis paper, doesnot require normally distributed variables.

3. An exact procedure

This section is devoted to the presentation of an implicit enumeration procedure (IE) forsolving SOPTC optimally. Such a procedure needs to be carefully structured so that only asmall fraction of then! possible sequences is examined. The results achieved with IE willbe useful to evaluate the performance of the heuristic discussed in the next section.

One approach for implicitly enumerating all possible solutions is provided by the branchand bound technique. This technique and variations of it have been applied with some suc-cess to a variety of combinatorial optimization problems (Ibaraki (1987a, 1987b)). The ideais to define a bound on the objective function and to discard entire subsets of the solutionspace if the bound reveals that these subsets cannot contain a solution better than the bestone encountered so far.

Let us denote byPk(δ1, . . . , δk) = {(i1, . . . , i n) ∈ Perm(n) | i1 = δ1, . . . , i k = δk; 1 ≤k ≤ n} the subset of Perm(n) containing all permutations whose firstk components arefixed. By conventionP0 = Perm(n). An upper bound on the expected reward of the bestsequence inPk(δ1, . . . , δk) is given by the expression:

UBk(δ1, . . . , δk) = f (δ1, . . . , δk)+∑

i 6∈{δ1,...,δk}ri · pi · Fδ1...δki (T) (3)

P1: JSN/VSK P2: JSN


10 COSTA AND SILVER

This bound is based on the fact that the inequality

Fδ1...δki (T) ≥ Fδ1...δki j (T) (4)

holds for any nonnegative variablet j ( j 6∈ {δ1, . . . , δk, i }). In our case, however, the inves-tigation times are assumed to be normally distributed and there is need to be careful sincethe inequality (4) may not be satisfied all the time (see Appendix).

Let f ∗ = f (i ∗1, . . . , i∗n) be the expected cost of the best sequences∗ = (i ∗1, . . . , i ∗n) found

so far in the enumeration procedure. Initially we sets∗ = (1, 2, 3, . . . ,n) and f ∗ =f (1, 2, 3, . . . ,n). The enumeration starts with the subproblemPn−2(1, 2, . . . ,n−2). As ageneral rule, oncePk(δ1, . . . , δk) has been considered, three basic steps can be performed:

(1) If k = n− 1, thenδn is the only cause not yet assigned and the resulting sequences=(δ1, . . . , δn) is evaluated. The variabless∗ and f ∗ are updated iff (δ1, . . . , δn)> f ∗.We move backward toP (δ1, . . . , δ

′`) where` is the largest index≤k such that there

exists a causeδ′` > δ` andδ′` 6∈ {δ1, . . . , δ`−1}. If causeδ′` is not unique, preference isgiven to the cause with the smallest number.

(2) If k < n− 1 and UBk(δ1, . . . , δk)≤ f ∗, then there is no chance to find a sequence inPk(δ1, . . . , δk) better thans∗. As above we move backward toP (δ1, . . . , δ

′`).

(3) If k < n−1 and UBk(δ1, . . . , δk)> f ∗, then we move forward toPk+1(δ1, . . . , δk, δk+1)

whereδk+1 = min {i | i 6∈ (δ1, . . . , δk)}.

The procedure stops before backtracking toP0 (in step (1) or (2)) after having exploredimplicitly all possible sequencess.

This branch and bound approach has two inevitable disadvantages that are typical ofimplicit enumeration methods. First, the computational requirements will be severe forlarge problems. Second, even for relatively small problems, there is no guarantee that anoptimal solution can be obtained quickly, since the extent of the partial enumeration dependson the data of the problem, more specifically on the total investigation timeT .

4. Tabu search in the presence of noise

Since most instances of realistic size cannot be tackled by the procedure outlined in the pre-vious section, a heuristic is required to produce in a reasonable amount of time solutions thatare as good as possible. In this paper we propose a tabu search method, called NTS (NoisyTabu Search), for solving the ordering problem with time constraint. Although NTS isdeveloped in a specific context, the method is well suited for optimization problems wherethe evaluation of the objective function is computationally expensive due to the presenceof noise in the problem definition.

Initially proposed by Glover (1986), and independently by Hansen and Jaumard (1990),tabu search (TS) is a neighborhood search method that has been continuously enhanced andapplied to an increasingly wide spectrum of problems (see Glover et al. (1993b) or Glover(1996)). Associated with each solutions ∈ X is a set of neighboring solutionsN(s) ⊂ Xthat can be obtained froms by performing a specific type of move. TS explores part of thesolution spaceX by moving at each iteration to the best solutions′ in the neighborhood

P1: JSN/VSK P2: JSN



N(s) of the current solutions. In order to avoid becoming stuck at a local optimum, themove is performed even if it induces a deterioration of the objective functionf . The riskof cycling is prevented by making unaccessible (or tabu) the solutions visited in the lastseveral iterations. To save time and memory, it is customary to store in a cyclic listI notthe tabu solutions themselves but rather some attributes of the moves that lead to them.More advanced versions of TS include intensification and diversification strategies thathave proved very useful for guiding the search process in the long term. For a more detaileddescription of TS, the reader is referred to Glover and Laguna (1993a) or Hertz, Taillard,and de Werra (1997).

The main features of NTS are outlined in the following subsections. In the context ofSOPTC, the solution spaceX is the group Perm(n) of all permutations of{1, . . . ,n}. Tostart the search an initial sequences is randomly generated inX. The statistically estimatedbest sequence encountered so far is denoted bys◦.

4.1. Objective function

When it is not possible to analytically compute the termsFi1...i k (T) in expression (1),simulation is the most appropriate tool for evaluating the quality of a solutions.

Let W = (wi j ) be an × m matrix where each columnw j ( j = 1, . . . ,m) is a vectorgiving investigation times for then possible causesi . Two different methods of selectingthe sample scenariosw j are possible. One obvious approach is to simply randomly generateeach componentwi j from a normal distributionN(µi , σ

2i ). This can be done, for example,

by using the polar method (Law and Kelton (1991)) or the (Kinderman and Ramage (1976))algorithm. We refer to this approach as random sampling. An alternate approach, usingso-called descriptive sampling, is illustrated in (J¨onsson and Silver (1996)). The idea is tocompute, for each causei , them equispaced quantiles of the distributionN(µi , σ

2i ) and to

assign them randomly to the components in the sample{wi 1, wi 2, . . . , wim}. Saliby (1990)showed that substantial variance reduction is possible by using descriptive sampling in thecontext of Monte Carlo simulation.

Given a sequences = (i1, . . . , i n), each scenariow j yields an expected reward of theform:

r j (s) = ri1 · pi1 + ri2 · pi2 + · · · + ri`( j ) · pi`( j ) (5)

where`( j ) is the largest index such thatwi1 j + · · · + wi`( j ) j ≤ T . Clearly, the expectedrewardsr j (s) can take on at mostn + 1 different values, namely, 0, ri1 · pi1, ri1 · pi1 +r i2 · pi2, . . . , ri1 · pi1 + ri2 · pi2 + · · · + rin · pin . An estimation of the objective functionf (s) = f (i1, . . . , i n) is obtained by simply computing the average value of the expectedrewardsr j (s):

fes(s) = 1

m·

m∑j=1

r j (s) (6)

The computation offes(s) has complexityO(m · n). An illustration of the results of thisapproach withn = 10,m = 40 andT = 0.5 · (µ1 + · · · + µn) is given in figure 1 where

P1: JSN/VSK P2: JSN


12 COSTA AND SILVER

Figure 1. Histogram of the frequencies associated with the rewardsr j (s) (n = 10,m= 40).

the matrixW was obtained by using descriptive sampling. For the chosen sequences,the 40 scenariosw j produce 5 different rewardsr j (s) (out of 11 possible values). Thecomputation offes(s) yields a total expected reward of 50.4402, whereas the true expectedreward f (s) is equal to 50.3958.

Preliminary experiments have shown that descriptive sampling yields slightly better re-sults than random sampling, particularly if the number of generated scenariosw j is small.Another advantage of descriptive sampling over random sampling is that the quality ofthe estimationfes is not related to the quality of the random number generator used. Forthese reasons, we decided to restrict our attention to descriptive sampling in the rest of theresearch.

4.2. Neighborhood structure

We define the neighborhoodN(s) of a sequences as consisting of all permutations whichcan be obtained froms by exchanging the positions of two different causes. In other words,two sequencess = (i1, i2, . . . , i n) ands′ = (i ′1, i

′2, . . . , i

′n) are neighbors if there existj

andk( j 6= k) such thati ′j = i k, i ′k = i j andi ′` = i` ∀ ` ∈ {1, . . . ,n} \ { j, k}.With this neighborhood structure, each sequence hasCn

2 neighbors. In order to speed upthe search the number of potential neighbors should be substantially reduced. A simple wayof achieving this is to restrict the search for neighbors to a random subsetN∗(s) ⊂ N(s)of sizenneigh, wherenneigh<Cn

2 is a parameter.

4.3. Tabu list

After each move involving the switch of position of causesi j andi k the two pairs (i j , j )and (i k, k) are stored in the tabu listI. This means that, for a certain number of iterationsθ ,sequencess with causei j in position j or causei k in positionk cannot be visited. The

P1: JSN/VSK P2: JSN



numberθ has been set experimentally to twice the average number of iterations in eachdescent over the last 10 descents. In this way, the length ofI adapts itself during the searchon the basis of the local structure of the problem. Initially, the average number of iterationsin each descent is set equal to 1.

As a rule, the process moves at each iteration from the current sequences to the bestnon-tabu sequences′ ∈ N∗(s). However, the tabu status of a move is overridden whenf (s′)is larger than the value ofs◦.

4.4. Search for a good neighbor

As seen in Section 4.1, it is not possible to quickly evaluate the change infes for eachpotential move. A possible approach to accelerate the search for a good neighbor is toconsider a smaller number of scenariosw j and to use a statistical test to decide whether agiven neighbor is better than the best neighbor examined so far at the current iteration.

Let sc be the neighbor under consideration andsb the best neighbor encountered so far inN∗(s). Initially sb is the first neighbor considered. In order to quickly evaluate the qualityof sc we randomly generate a subset of scenariosWc ⊂ {w1, . . . , wm} and we compute thecorresponding expected rewardsr j (sc). The decision as to whether or not the valuefes(sc)

is higher thanfes(sb) is made by performing the following hypothesis test:

H0: f (sc) ≤ f (sb)

H1: f (sc) > f (sb)

For this purpose the two samplesRb = {r1(sb), . . . , rm(sb)} andRc = {r j (sc) | w j ∈ Wc}are taken into account. The test is performed at a level of significanceα, i.e., the probabilityof rejecting the null hypothesis even though it is true is equal toα. The size ofWc, whichwe shall denotemc, is a parameter that will be discussed in Section 5.

If the hypothesis H0 is accepted, then we discard the sequencesc (having avoided takingm−mc samples) and we keepsb as the best neighbor encountered so far inN∗(s). If the testaccepts the hypothesis H1, then it is likely thatsc is a better neighbor thansb. To ascertain thiswe consider all of the remainingm−mc scenariosw j 6∈ Wc and we compute the estimatedvalue fes(sc). If fes(sc) > fes(sb), thensc replacessb. Once all neighbors inN∗(s)have beenexamined the algorithm moves to the supposedly best neighbors′ = sb. In Section 5, thisapproach will be shown to reduce the neighbors investigation time significantly. However,by operating in this fashion we can certainly miss the best neighbor inN∗(s). Fortunately,such an outcome is not that important inasmuch as other studies (see, for example, Glover(1995)) have shown that the best move criterion is not essential in tabu search. This isparticularly true when the neighborhood is large or expensive to examine.

The three following tests were considered in this study.

(i) The t-test. The t-test (see, for example, Johnson and Bhattacharyya (1992)) testswhether the difference between the means of the two samplesRb andRc is significant.Let us temporarily assume that the observations inRb andRc are independent randomvariables with meansηb andηc and that the populations sampled have roughly theshape of normal distributions with equal standard deviations.

P1: JSN/VSK P2: JSN


14 COSTA AND SILVER

Let Rb = fes(sb), Rc, vb andvc be the sample means and sample variances ofRb

andRc, respectively. Ifηb = ηc, then the statistic:

t = Rc − Rb√(m−1) · vb+(mc−1) · vc

m+mc−2 · ( 1m + 1

mc

) (7)

follows thet-distribution withm+mc − 2 degrees of freedom. Thus, the hypothesisH0 is rejected at a level of significanceα if t > tm+mc−2,1−α, wheretm+mc−2,1−α is the(1− α) quantile of thet-distribution withm+mc − 2 degrees of freedom.

The use of a “goodness-of-fit” test, like the Kolmogorov-Smirnov test (Gibbons(1985)), allows us to determine whether the observations inRb or Rc can be regardedas normal variables. Experimentations with different setsRhave shown that the rewardsr j ∈ R cannot be assumed to be normally distributed. The reason is that ther j ’s arediscrete variables that can take on only a small number of values (at mostn + 1).Despite this shortcoming, thet-test will still be used and it will be shown to performwell in the computational experiments of Section 5.

Contrary to thet-test, the two tests presented below do not impose the condition thatthe sampled populationsRb and Rc be normally distributed. It is only assumed thatthe distributions are continuous and similar in shape and variance. Both the Wilcoxonand the Median tests check the hypothesis that the mediansvb andvc of Rb and Rc

are equal. It should be noted that this is equivalent to comparing the means of thetwo populations when the corresponding density functions are symmetric with finitemeans.

(ii) The Wilcoxon test. The Wilcoxon test (Gibbons (1985)) compares the mediansvb andvc of the sampled populationsRb andRc by considering the ranks of the observationsin the pooled sampleRb ∪ Rc rather than the observations themselves. The smallestobservation in the pooled sample receives rank 1, the next smallest rank 2, etc. If two ormore observations have the same magnitude, then each of them is assigned a rank whichis the mean of the consecutive ranks to which they would have been assigned if tieswere broken randomly. LetVmc measure the sum of the ranks of themc observationsin Rc. If vb = vc andm andmc are large enough then the distribution of the statisticVmc has a mean ofmc · (m+mc+ 1)/2, a variance ofm ·mc · (m+mc+ 1)/12, and asymmetric shape that approaches normality. This means that the ratio:

V = Vmc − mc · (m+mc+1)2√

m ·mc · (m+mc+1)12

(8)

is distributed approximately like a standard normal variable. Under these circum-stances, we reject H0 at the level of significanceα if V > φ1−α, whereφ1−α is the(1− α) quantile of the standard normal distribution.

(iii) The Median test. Like the Wilcoxon test, the Median test (Gibbons (1985)) is adistribution-free test that will give information as to whether it is likely thatRb andRc

have been drawn from populations having the same median value. Letv be the median

P1: JSN/VSK P2: JSN



of the pooled sampleRb ∪ Rc. If m+ mc is odd, thenv is simply the observationof rank (m+mc+ 1)/2. If not, thenv is the average value of the observations ofranks(m+mc)/2 and 1+ (m+mc)/2. Let Nmc be the number of observations inRc that are larger thanv plus one half of the observations that are exactly equal tov.If vb= vc, then the distribution ofNmc is hypergeometric. Whenm andmc are largeenough the variableNmc tends to be normally distributed with meanmc/2 and variancem · mc/4 · (m+ mc − 1). Hence, we reject H0 at the level of significanceα if theratio:

N = Nmc − mc2√

m ·mc4 · (m+mc+1)

(9)

is larger thanφ1−α.

4.5. Diversification

The SOPTC is unusual in the sense that its local optima are located on very large plateausfrom which it is difficult to escape without introducing appropriate mechanisms in thesearch process. The existence of these plateaus is due to the fact that the permutationof two causes located either near the beginning or near the end of a sequence yields anegligible change in the objective function. To ensure that the search does not always focuson the same region in the solution space, the permutation of two causes whose positions inthe sequence are less thandn/2e units apart is forbidden temporarily. This diversificationstrategy is activated duringnidiv iterations everyγ (nidiv) iterations (γ (nidiv)>nidiv) if thebest solution reached so far,s◦, has not been improved in the lastγ (nidiv)− nidiv iterations.Also, because of the restriction on the positions, the size ofN∗(s) is reduced by half duringeach diversification phase.

4.6. Additional remark

In a deterministic environment, the more iterations that are performed in TS, the better isthe quality of the final sequences◦. In the context of NTS, this is not necessarily true. Itmay happen that the sequences◦ is replaced, at a certain stage, by a sequences better thans◦ in terms of fes but not in terms off , i.e., fes(s) > fes(s◦) and f (s) < f (s◦). To reducethis risk, we keep track of thens best sequences encountered so far in a cyclic listS◦ ={s◦1, . . . , s◦ns}, where fes(s◦1) ≥ · · · ≥ fes(s◦ns). If a sequences such thatfes(s) > fes(s◦ns) isfound, then it is inserted at the right place inS◦ ands◦ns is removed.

The algorithm stops automatically after performing a given numbernimaxof iterations.To better estimate which of thenssequences of the final listS◦ is the best, we generateκ(m)new scenariosw j (κ(m) > m) to estimate the expected reward of the sequences inS◦ witha higher accuracy. The best sequence inS◦, according to this new evaluation criterion, isthe solutions◦ produced by NTS.

P1: JSN/VSK P2: JSN


16 COSTA AND SILVER

5. Computational experiments

The algorithms presented in Sections 3 and 4 were implemented in Pascal and run on a IBM7015-R30 SMB system. Computing times are reported in CPU seconds of this machine. Foreach value ofn, we constructed 20 problem instances by randomly generating the probabil-ities pi , the rewardsri ∈ {1, . . . ,100}, the mean investigation timesµi ∈ {1, . . . ,10} andthe standard deviationsσi ∈ ]0, µi /3[ (i = 1, . . . ,n). All results subsequently shown areaverage results over these 20 problem instances. Recall that the investigations timesti wereassumed to be normally distributed so as to make the evaluation off (s) relatively easy.For each problem instance, the total investigation timeT is set topropT · (µ1+ · · · + µn),wherepropT ∈ [0, 1]. Unless mentioned explicitly,propT will be chosen equal to 0.5. Thefollowing parameter values have been established experimentally:nneigh= n, nidiv= 5,γ (nidiv) = 15,ns= 5, κ(m) = 5 ·m.

The first experiments deal with the efficiency of the three hypothesis tests described in theprevious section. For this purpose NTS is run independently of these tests and all neighborsare evaluated by taking into account thempossible scenarios. At each iteration and for eachneighborsc ∈ N∗(s), we consider the best neighborsb ∈ N∗(s) found so far, we computethe true rewardsf (sb) and f (sc), we generate a subsetWc of scenarios and we perform thethree tests. By comparing the valuesf (sb) and f (sc), we can tell if a test makes the rightchoice or not. Over a large number of iterations, these observations allow us to compute theempirical probabilities of type I and type II errors associated with each test. A type I erroris rejecting the null hypothesis H0 when it is true (f (sb) ≥ f (sc)), whereas a type II erroris failing to reject the null hypothesis when it is false. Each test has a parameterα which isthe probability of type I error given that the underlying assumptions of the tests are valid.

For the three tests considered, the empirical probabilities of type I and type II errors aredenoted bya(t), a(wil), a(med),b(t), b(wil) andb(med), respectively. Figure 2 reports theestimations we have obtained after running NTS over 20 problem instances withn = 20,nimax= 50,m= 40,mc = 0.5 ·m= 20 andα ∈ {0.01, 0.1, 0.2, . . . ,0.9, 0.99}.

Figure 2. Empirically estimated probabilities of type I and type II errors.

P1: JSN/VSK P2: JSN



The experiments have revealed that, when allm scenarios are used, the null hypothesisis verified more often than the alternate hypothesis (on average, H0 is satisfied 8 times outof 10). This difference is due to the fact that the best neighborsb is updated after examiningeachsc ∈ N∗(s) and thus we are more and more in the situation wheref (sb)≥ f (sc). Thisimplies that we are more likely to be in a situation where a type I error could occur ratherthan a type II error. Thus, it seems natural to give more importance to the dashed lines infigure 2. However, it should be noted that incurring a type I error is less serious than havinga type II error. Indeed, in the latter case, we discard a neighbor better thansb, whereas inthe former case, we just keep sampling when it is not really necessary.

The differences between the selectedα and the experimental curvesa(·)are a consequenceof violating some of the assumptions underlying the use of the tests. Indeed, the sampledpopulationsRb andRc are drawn from distributions that are not continuous and that do notnecessarily have the same standard deviation. Also, these distributions are not completelyindependent since ther j ’s are associated with two solutions that are neighbors of the samesolutions. Moreover, for thet-test, the Kolmogorov-Smirnov test revealed that ther j ’scannot be considered as normally distributed. These issues are not fundamental in thecontext of NTS since the tests are primarily implemented for accelerating the search. Thenumberα should be considered more as a control parameter to get decent results than asthe actual probability of type I error associated with a test.

Figure 2 shows that the performances of thet-test and the Wilcoxon test are comparable,with a slight advantage to thet-test. The Median test seems to be less sensitive to the chosenlevel of significance. Forα < 0.7, it is definitely better than the two other tests from thestandpoint of the type II error but it is worse with regard to the type I error. As a generalrule, the quality of the results, as well as the running time of NTS, grow slightly withα.This is because the null hypothesis is rejected more often and, thus, the quality of neighborsis evaluated with a higher accuracy by considering the whole setW of scenarios. All threetests are of the same order of complexity and no significant difference was observed in therunning times. Based on the above observations we have decided to concentrate on thet-testin the remaining experiments. A level of significanceα = 0.75 offers a good compromisewith a(t) < 0.3 andb(t) < 0.05.

The influence of the number of scenarios on the performance of NTS is studied below.To do so we consider two sets of 20 problem instances withn = 100 causes. The first setis generated as mentioned earlier while the second one is obtained from the first one bydividing all standard deviationsσi by two (the intention being to investigate the effect ofvariability on the best choice ofm). Figure 3 plots the true average rewardsf (s◦) achievedby running NTS withmc = 0.5 · m andm ranging from 5 to 400. The search is stoppedafter 3 and 6 s, independently of the number of iterations performed. Clearly, the smallerm is, the higher is the number of iterations.

Figure 3 shows that the selection of an appropriate value form depends on the amountof variation in the data as well as on the running time of the algorithm. The best valueof m increases with the variation in the data and with the available running time of NTS.For a given running time, it is worthwhile considering a smaller number of scenarios andperforming more iterations when little noise is present in the data. Plots similar to figure 3were obtained with values ofn different from 100. These experiments have shown that

P1: JSN/VSK P2: JSN


18 COSTA AND SILVER

Figure 3. Effect of the number of scenariosm on the performance of NTS.

the number of causes does not have a direct influence on the best number of scenarios fora limited running time. As a general rule, when the restriction on the running time is nottight, the higherm is, the better the results of NTS are likely to be. This phenomenon isevident in Table 1, which will now be described.

Table 1 summarizes the results obtained by NTS withn ∈ {20, 50, 100}, nimax= 4 · n,m ∈ {100, 200} andmc ∈ {0.02 ·m, . . . ,0.75 ·m,m}. No hypothesis test is performed inthe situation wheremc = m. The results are compared with those obtained by the samealgorithm as NTS with the estimated objective function valuesfes(s) replaced by their truevaluesf (s). This algorithm, which we simply call TS, uses neither sampling nor hypothesistests. For the use of TS, the average actual rewards obtained are presented. In the case ofNTS, we compute the true reward of each solutions◦ and we report the average percentagedeviation from the corresponding TS value. TS is expected to outperform NTS since nostatistical estimation is used. In Tables 1–3 all CPU times are given in parentheses.

Table 1 makes it clear that the use of an hypothesis test speeds up the search of neighborssignificantly. The quality of the results improves, but only slightly, asmc increases. For

Table 1. Effect of m and mc on the performance of NTS.

n 20 50 100

TS 41.5523 (<0.1) 43.7971 (<0.1) 43.7992 (3.0)

mmc

100 200 100 200 100 200

0.02 · m 0.17 (0.1) 0.03 (0.3) 0.12 (1.0) 0.11 (1.6) 0.11 (5.2) 0.10 (7.7)

0.05 · m 0.14 (0.1) 0.04 (0.3) 0.14 (0.9) 0.10 (1.6) 0.10 (4.9) 0.10 (7.1)

0.10 · m 0.21 (0.1) 0.04 (0.3) 0.10 (0.9) 0.10 (1.7) 0.08 (4.7) 0.09 (7.4)

NTS 0.25 · m 0.08 (0.1) 0.02 (0.3) 0.11 (1.1) 0.08 (2.2) 0.08 (5.5) 0.07 (10.2)

0.50 · m 0.08 (0.2) 0.02 (0.4) 0.10 (1.5) 0.08 (2.9) 0.08 (8.4) 0.04 (16.9)

0.75 · m 0.07 (0.2) 0.02 (0.5) 0.10 (1.8) 0.07 (3.7) 0.08 (8.4) 0.05 (22.5)

no test 0.07 (0.3) 0.02 (0.6) 0.10 (2.2) 0.06 (4.7) 0.07 (13.4) 0.05 (27.3)

P1: JSN/VSK P2: JSN



Table 2. Results achieved with IE, TS and NTS (n = 10, 12, 14).

IE TS NTS

n propT f (s∗) (CPU) apd/mpd (CPU) success apd/mpd (CPU) success

0.25 30.1028(<0.1) <0.01/<0.01 (<0.1) 18 0.03/0.25 (0.1) 4

10 0.50 43.5896 (0.3) <0.01/0.02 (<0.1) 15 0.04/0.33 (0.1) 7

0.75 50.0684 (4.1) <0.01/<0.01 (<0.1) 18 0.02/0.21 (0.1) 3

0.25 30.7379 (0.1) 0.01/0.26 (<0.1) 17 0.04/0.52 (0.1) 3

12 0.50 43.2872 (9.2) 0.04/0.39 (<0.1) 12 0.03/0.26 (0.1) 1

0.75 49.3538 (107.3) <0.01/0.05 (<0.1) 16 0.02/0.15 (0.1) 2

0.25 30.2654 (1.0) 0.03/0.60 (<0.1) 18 0.04/0.55 (0.2) 1

14 0.50 42.2534 (107.4) 0.02/0.13 (<0.1) 10 0.04/0.22 (0.2) 2

0.75 48.0854 (30,041.0) <0.01/0.08 (<0.1) 18 0.03/0.29 (0.2) 3

Table 3. Results achieved with TS and NTS (n = 50, 100, 200, 500).

TS NTS

n propT f (s◦) (CPU) apd/mpd (CPU)

0.25 31.8567(<0.1) 0.11/0.65 (1.9)

50 0.50 43.7971(<0.1) 0.08/0.28 (2.2)

0.75 49.2833(<0.1) 0.02/0.08 (2.0)

0.25 32.1395 (3.0) 0.06/0.21 (7.2)

100 0.50 43.7992 (3.0) 0.07/0.30 (10.2)

0.75 49.2074 (3.0) 0.02/0.06 (9.9)

0.25 31.9823 (24.4) 0.07/0.19 (39.0)

200 0.50 43.6240 (23.4) 0.05/0.15 (60.9)

0.75 48.9354 (23.8) 0.02/0.06 (59.0)

0.25 32.5236 (345.9) 0.05/0.08 (465.8)

500 0.50 44.0748 (336.7) 0.03/0.14 (798.2)

0.75 49.2144 (333.1) 0.02/0.12 (855.0)

example, the difference betweenmc = 0.25 ·m andmc = m (no test), in terms of averagepercentage deviation, never exceeds 0.02. In contrast, the running times can differ by afactor of almost 3. Under these circumstances, it was decided to run NTS withmc = 0.25·min the following experiments.

Now that the most important control parameters have been adjusted, the performance ofNTS is evaluated on the basis of the optimal solutions produced by the implicit enumera-tion procedure (IE) of Section 3. Results are presented in Tables 2 and 3 (nimax= 4 · n,m = 200). Where possible, we report the average (apd) and maximum (mpd) percentagedeviation of the valuesf (s◦) from the corresponding optimal values. For large values

P1: JSN/VSK P2: JSN


20 COSTA AND SILVER

of n, where it was not possible to obtain the optimum solution, comparison is madeto the corresponding TS values, as in Table 1. The total investigation timeT is set topropT · (µ1+ · · · + µn), with propT ∈ {0.25, 0.5, 0.75}. The variablesuccessgives thenumber of times (out of 20) that the corresponding heuristic found an optimal solution.

As expected, the use of IE becomes computationally prohibitive for problems involvinga large number of causes. We observe that the running time of IE is also strongly relatedto the lengthT of the time horizon. This is because the efficiency of the upper bound UBquickly decreases asT increases.

Although some exceptions occur, the comparisons between the various results reported inTables 2 and 3 reveal that TS outperforms NTS both on solution quality and computationaltime. The latter is expected because, as mentioned earlier, in TS we are using the truevalues of f (s) instead of having to statistically estimate them. However, it is important tonote that the observed maximal percentage deviation of NTS from IE or TS is extremelysmall (less than 1%). Table 3 shows that the quality (in terms of both apd and mpd) ofthe solutions produced by NTS improves asn gets larger. For a given number of causes,the differences in the running times of TS observed in Table 3 are not significant and aresimply due to the random nature of the algorithm. As far as NTS is concerned, the runningtime depends significantly onpropT for a givenn(n ≥ 50). We observe that the algorithmrequires a larger amount of time whenpropT = 0.5 (n ≤ 200) orpropT = 0.75 (n = 500).Extensive additional experiments (not reported here) have shown that these differences aredue to the combination of two factors to now be discussed.

First, the evaluation of a potential move requires more time on average aspropT tendsto 0.5. To see this, let us consider a move defined by the switch of two causes lying atpositionspos1 < pos2. In the current sequences, each scenariow j contributes to a rewardr j (s) by allowing the investigation of( j ) causes beforeT = propT · (µ1+ · · · +µn) (seeSection 4.1). Ifpos2 < `( j ) or pos1 > `( j ), then for scenariow j the switch of positionsunder consideration causes no change in the estimated objective function. The only situationthat requires specific computation is whenpos1 ≤ `( j ) ≤ pos2. Since the positionspos1andpos2 are randomly generated in{1, . . . ,n}, the probability ofpos1 ≤ `( j ) ≤ pos2 ismaximal when ( j ) = dn/2e. This occurs precisely whenpropT = 0.5.

Second, we observed that the hypothesis H0 is rejected forpropT = 0.75 more oftenthan forpropT ∈ {0.25, 0.5}. This means that more sampling is performed in the formercase. This is due to the fact that empirically the pooled variance of the sampleRb ∪ Rc,or equivalently the denominator of thet statistic (see Section 4.4), is a concave function ofpropT which takes on its maximum value around 0.25. As expected, the pooled varianceof the sampleRb ∪ Rc tends towards 0 aspropT → 0 or propT → ∞. The smallerthe denominator oft is, the more often thet-test rejects the null hypothesis and, thus, moresampling is performed. In comparison with the first effect, the importance of the secondeffect prevails asn increases. This explains why the running time of NTS is larger forpropT = 0.75 only whenn = 500.

6. Summary

In this paper, we have investigated the problem of finding the true cause of an undesirableeffect when there is uncertainty in the investigation times of the possible causes (SOPTC).

P1: JSN/VSK P2: JSN



Some other potential applications of this problem were identified. An implicit enumerationprocedure (IE) and a tabu search technique heuristic based on sampling and hypothesistests (NTS) have been developed. It should be recalled that SOPTC is a difficult problemto solve by neighborhood search because the solution space contains several large plateaus(i.e., many neighboring solutions have values extremely close to that of the current solution).Also, when it is not possible to analytically compute the objective function, simulation isan appropriate tool for evaluating the quality of the solutions encountered.

Various tests have been carried out to select appropriate values of the parameters govern-ing the search in NTS. The experiments we performed clearly demonstrate the computationalsuperiority of TS over NTS. However, in the actual problem situation, sampling is the onlyway to evaluate the quality of a solution. The results show the effectiveness of NTS inaddressing large combinatorial problems in the presence of data uncertainty.

In this research, we have assumed that the investigation timesti are independent withknown probability distributions and that the probabilitiespi and the rewardsri are knowna priori and fixed throughout the ordering process. This study can be extended in severaldirections by changing the assumptions made regarding these factors. Also, possible ex-tensions would be to the cases where there is more than one true cause of the effect, wheresome causes can be investigated in parallel, where the investigation of causes is subject topreemption or where the sequence of investigation can be dynamically adapted as a func-tion of the remaining time available for investigation. However, each of these new aspectscauses the modeling of the problem to become much more complicated.

NTS can be modified easily to deal with a variety of optimization problems where thereis uncertainty in the data, specifically when the evaluation of the objective function iscomputationally expensive or simply impossible without simulation. The results achievedwith NTS are quite satisfactory and are encouraging for future research in the field of generalheuristics when noise is present in the problem under study.

Appendix

The inequality

Prob(ta ≤ T) ≥ Prob(ta + tb ≤ T), (10)

which is similar to (4) withta = tδ1 + · · · + tδk + ti andtb = t j , holds for any nonnegativevariabletb. For normal variables such that 0< σb <

µb

3 , (10) is violated only when bothsides of the inequality take extremely small values. More specifically:

Property

Let ta ∼ N(µa, σ2a ) andtb ∼ N(µb, σ

2b ) be two normally distributed variables such that

0< σb <µb

3 and σbσa= K .

P1: JSN/VSK P2: JSN


22 COSTA AND SILVER

If inequality (10) does not hold, thenz= T−µa−µb√σ 2

a+σ 2b

>T−µa

σa. In such a case, it can be shown

that the maximum value ofz is given by

z∗(K ) = 3 · K

1−√1+ K 2.

The probability8(z∗(K )) = Prob(ta+ tb ≤ T) ranges from 0 to8(−3) = 0.0013, asKgoes from 0 to infinity. In the enumeration procedure of Section 3 the bound UBk(δ1, . . . , δk)

is used fork ≥ 1 and thusK = σ j√σ 2δ1+···+σ 2

δk+σ 2

i

( j 6∈ {δ1, . . . , δk, i }) is very unlikely to takelarge values.

Consequently, when (4) does not hold, the probability Prob(ta + tb ≤ T) is extremelylow and it is considered as insignificant. Extensive experiments with complete enumerationschemes have shown that the bound UBk never discarded an optimal solution in the implicitenumeration procedure of Section 3.

Acknowledgments

The research leading to this paper was supported by the Natural Sciences and EngineeringResearch Council of Canada under grant No. A1485, by the Fonds National Suisse de laRecherche Scientifique and by the Carma Chair at the University of Calgary. This workwas carried out while Dr. Daniel Costa was a Postdoctoral Assistant at the University ofCalgary. The authors wish to thank Dr. Xavier de Luna for his valuable comments regardinginferential statistics.

References

Andradottir, S. (1995). “A Method for Discrete Stochastic Optimization,”Management Science41, 1946–1961.Barton, R. and J. Ivey. (1996). “Nelder-Mead Simplex Modifications for Simulation Optimization,”Management

Science42, 954–973.Bellalouna, M., C. Murat, and V.Th. Paschos. (1995). “Probabilistic Combinatorial Optimization Problems on

Graphs: A New Domain in Operational Research,”European Journal of Operational Research87, 693–706.Benton, W.C. and M.D. Rossetti. (1993). “The Vehicle Scheduling Problem with Intermittent Customer Demands,”

Computers and Operations Research19(6), 521–531.Bertsimas, D.J. and D. Simchi-Levi. (1996). “A New Generation of Vehicle Routing Research: Robust Algorithms,

Addressing Uncertainty,”Operations Research44(2), 286–304.Chun, Y.H., H. Moskowitz, and R. Plante. (1995). “Sequencing a Set of Alternatives Under Time Constraints,”

Journal of the Operational Research Society46(9), 1133–1144.Garey, M.R. and D.S. Johnson. (1979).Computers and Intractability: A Guide to the Theory of NP-Completeness.

New York: W.H. Freeman.Gendreau, M., G. Laporte, and R. S´eguin. (1995). “An Exact Algorithm for the Vehicle Routing Problem with

Stochastic Demands and Customers,”Transportation Science29, 143–155.Gendreau, M., G. Laporte, and R. S´eguin. (1996). “A Tabu Search Heuristic for the Vehicle Routing Problem with

Stochastic Demands and Customers,”Operations Research44(3), 469–477.Gibbons, J.D. (1985). “Nonparametric Statistical Inference,”Statistics: Textbooks and Monographs. New York:

Marcel Dekker Inc., vol. 85.Glover, F. (1986). “Future Paths for Integer Programming and Links to Artificial Intelligence,”Computers and

Operations Research13(5), 533–549.

P1: JSN/VSK P2: JSN



Glover, F. (1995). “Tabu Search Fundamentals and Uses,” Research Report, School of Business, University ofColorado, Boulder.

Glover, F. (1996). “Tabu Search and Adaptive Memory Programming—Advances, Applications and Challenges.”In Barr, Helgason, and Kennington (eds.),Interfaces in Computer Science and Operations Research. KluwerAcademic Publishers (to appear).

Glover, F. and M. Laguna. (1993a). “Tabu Search.” In C.R. Reeves (ed.),Modern Heuristic Techniques forCombinatorial Problems. Oxford: Blackwell Scientific Publishing, pp. 70–141.

Glover, F., M. Laguna, E. Taillard, and D. de Werra. (eds.) (1993b). “Tabu Search,”Annals of Operations Research,vol. 41.

Hansen, P. and B. Jaumard. (1990). “Algorithms for the Maximum Satisfiability Problem,”Computing44, 279–303.Hertz, A., E. Taillard, and D. de Werra. (1997). “Tabu Search.” In E.H.L. Aarts and J.K. Lenstra (eds.),Local

Search in Combinatorial Optimization. New York: Wiley.Ibaraki, T. (1987a). “Enumerative Approaches to Combinatorial Optimization—Part I,”Annals of Operations

Research, vol. 10.Ibaraki, T. (1987b). “Enumerative Approaches to Combinatorial Optimization—Part II,”Annals of Operations

Research, vol. 11.Jaillet, P. (1988). “A Priori Solution of a Traveling Salesman Problem in Which a Random Subset of the Customers

are Visited,”Operations Research36(6), 929–936.Johnson, R.A. and G.K. Bhattacharyya. (1992).Statistics: Principles and Methods(2nd edition). New York:

Wiley.Jonsson, H. and E.A. Silver. (1996). “Some Insight Regarding Selecting Sets of Scenarios in Combinatorial

Stochastic Problems,”International Journal of Production Economics45, 463–472.Kinderman, A.J. and J.G. Ramage. (1976). “Computer Generation of Normal Random Variables,”Journal of the

American Statistical Association71, 893–896.Law, A.M. and W.D. Kelton. (1991).Simulation Modeling and Analysis(2nd edition). New York: McGraw-Hill.Osman, I.H. and G. Laporte. (1996). “Metaheuristics: A Bibliography,”Annals of Operations Research63,

513–623.Rohleder, T.R. and E.A. Silver. (1997). “A Tutorial on Business Process Improvement,”Journal of Operations

Management15, 139–154.Saliby, E. (1990). “Descriptive Sampling: A Better Approach to Monte Carlo Simulation,”Journal of the Opera-

tional Research Society41, 1133–1142.Silver, E.A. and T.R. Rohleder. (1997). “Some Simple Mathematical Aids for Cause-and-Effect Analyses,”Journal

of Quality Technology(to appear).Yan, Di and H. Mukai. (1992). “Stochastic Discrete Optimization,”SIAM Journal of Control and Optimization

30, 594–612.

Tabu Search When Noise is Present: An Illustration in the Context of Cause and Effect Analysis

Documents

Transcript of Tabu Search When Noise is Present: An Illustration in the Context of Cause and Effect Analysis