Genetic Algorithms for Dynamic Combinatorial Problems
-
Upload
mari-jefferson -
Category
Documents
-
view
45 -
download
0
description
Transcript of Genetic Algorithms for Dynamic Combinatorial Problems
Genetic Algorithms for Dynamic Genetic Algorithms for Dynamic Combinatorial ProblemsCombinatorial Problems
Dyn. Environments
GA & Dyn. Problems
Experimentation
Future Work
OutlineOutline Definitions
CategorizationDifficulties
DiversityImplementation CostRobustness & Flexibility
TSPBM GeneratorResults
Dynamic Problem, a definition
Dynamic EnvironmentsDynamic Environments
Real world problems are dynamic in nature
An optimization problem consists of – Optimization goal(s) – Decision variables – Restrictions
Any change in ingredients Change in optimum
If it changes with time then it is a Dynamic problem
whenever the environment
changes it is very likely that the optimal solution changes as well
If it changes with time then
it is a Dynamic problem
at t1
at t2
at t3F
itne
ssF
itne
ssF
itne
ss
Dynamic EnvironmentsDynamic Environments
Real world problems are dynamic in nature
Vehicle
Routing
Job shop
scheduling
Routes added and deleted
Cost of route changes
Vehicles break down
New Jobs arrive continuously
Raw material changes
Procedures, frequency , tools modified
Adding dynamism brings new challenges
Dynamic EnvironmentsDynamic Environments
Static Dynamic
One start!
all orders known a priori.
A schedule must change in response to new or altered requests
Business Rules less Restrictivesame model for pickup and delivery
More restrictionsCan’t add a delivery vehicle has left
Well-known and studied extensively
Still new
No Benchmark problems
Not all Dynamic problems are interesting
What makes a dynamic problem interesting
Dynamic EnvironmentsDynamic Environments
• Information on the problem is time-dependent.
• Finding solutions while time proceeds concurrently with incoming information.
• Change is not too large and permits partial reuse of old solutions.
The interesting dynamic problem requires an approach which is adaptive to changes
Difficulties
Meta-heuristicsMeta-heuristics and Dynamic problems and Dynamic problems
Originally developed for static problems
When considering dynamic problems, the difficulty:
population tends to converge near the optimum
1 Pop. converged to opt.
2Pop. far from new opt.
77
GA, OverviewGA, Overview“Genetic Algorithms are good at taking large, potentially huge search
spaces and navigating them, looking for optimal combinations of things, solutions you might not otherwise find in a lifetime.”
- Salvatore Mangano
Computer Design, May 1995
An effective and flexible optimization tool
Manipulates a set of candidate solutions
Mimics the evolutionary process in nature
robust , Good for “noisy” environments
Easily exploit previous or alternate solutions
Modular, separate from application
Supports multi-objective optimization
Evolutionary technique … adaptive to changes
GA and Dynamic problemsGA and Dynamic problemsWhy GA for Dynamic Problems
GA and Dynamic problemsGA and Dynamic problems
0
5
10
15
20
25
87 88 89 90 91 92 93 94 95 96 97 98 99
Year
Pu
bli
ca
tio
ns
1010
How do GA’s approach the dynamic problem?
Increasing interest
Evolutionary optimization in dynamic environments
(Branke 2002)
“Ignore” dynamism: no more exploitation
Approaching Dyn. ProblemsApproaching Dyn. Problems
“Ignore” dynamism: no more exploitation
Approaching Dyn. ProblemsApproaching Dyn. Problems
Straight forward… But
x Time consuming
x No adaptation ... old knowledge discarded
x Not suitable if :
“Restart” from the beginning: no exploitation,
• Changes not too large. Permitting partial reuse of old sol.
• Changes can’t be detected directly
• Continuous changes.No benefits from restarting every gen.
• Available time doesn’t permit a restart from scratch.
• Part of the old solution has already been implementedcv
“Ignore” dynamism: no more exploitation
Approaching Dyn. ProblemsApproaching Dyn. Problems
“Restart” from the beginning: no exploitation,
“Adapt old solutions”:
– Partial Restart ( random immigrants)
– Hyper mutation: Scatter the population
– Use Explicit Memory: Save old solutions & seed
Still new, ample opportunity to: - Refine, Combine, Add - Examine on combinatorial problems
Benchmark Problems
Adaptation Cost vs. Solution QualityA multi-objective problem
When adapting old solutions not possible…Choose the most robust
When several adaptable optima… choose the most flexible
Dynamic Comb. IssuesDynamic Comb. Issues
ExperimentationExperimentation
ObjectivesObjectives
Test Dynamic TSP using an adaptive form of GA
Test two mutation models in dynamic landscapes:
-Traditional Mutation - Adaptive (Dynamic) Mutation
Benchmark GeneratorBenchmark Generator
Generates a sequence of static problems. Solves each one separately
problem 1 s1
problem 2
problem 3
s2
s3
generations (time)
S1, S2, S3, … are optimal or “near” optimal solutions
Benchmark GeneratorBenchmark Generator
Later, the sequence of static problems is introduced as sub-problems of one dynamic problem
problem 1
problem 2
problem 3
The goodness of the dynamic solver is measured as how close d1, d2, d3, … are to S1, S2, S3, …
d1
d2
d3
d1, d2, d3, … will
be solutions of a dynamic solver
LandscapeLandscape
All the optima shift randomly over time
Three general modes of shift
– Edge Change: Change the distance b/w cities (traffic jam).
– Add/Delete cities: adding or canceling assignments.
– City Swap: interchange labels of two cities.
The user controls how cost changes– Severity ( # of steps in any change )– Frequency ( # of generations between changes )– Cycling (remove changes in reverse order)
Dynamic Solver, settingDynamic Solver, setting Each experiment used :
– a generational GA hybridized with LS
– path representation
– Tournament selection ( tournament size = 2) with Elitism
– 2 point Order Crossover
– varying mutation rate
– Population size = 50
– 200 different instances in 3000-generation runs.
– Severity: 1, 10, 100 steps per shifts
– Frequency: 10,100, 1000 generations between shifts
– Statistics based on 10 runs per experiment
GA… Mutation ModelsGA… Mutation Models
Test two simple mutation models are tested:
- Traditional Fixed Mutation FM. P= constant - Dynamic Variable Mutation VM P = P0 at change in environment P = 0 at the next change
Several values of P and P0 were tested
ResultsResults Cost changes randomly
Offline performance P=0.1
40
50
60
70
80
90
100
110
0 20 40 60 80 100 120 140 160 180 200 220
Generations
Perf
orm
an
ce
VM
FM
OptimumGoal
ResultsResults
Cost changes randomly, continued.
Online performance P=0.1
40
60
80
100
120
140
0 20 40 60 80 100 120 140 160 180 200 220
Generations
Perf
orm
an
ce
VM
FM
OptimumGoal
ResultsResults
Cost changes randomly, continued.
Inluence of mutation
40
50
60
70
80
90
100
110
0 20 40 60 80 100 120 140 160 180 200 220
Generations
Off
lin
e P
erfo
rman
ce 0.01
0.05
0.1
0.3
0.7
OptimumGoal
ResultsResults
Cost changes randomly, continued.
Inluence of mutation
40
50
60
70
80
90
100
110
120
130
140
0 20 40 60 80 100 120 140 160 180 200 220
Generations
On
lin
e P
erfo
rman
ce 0.01
0.05
0.1
0.3
0.7
OptimumGoal
ResultsResults
Leg cost increased
Offline performance P=0.3
70
80
90
100
0 20 40 60 80 100 120 140 160 180 200 220
Generations
Per
form
ance
VM
FM
OptimumGoal
ResultsResults
Leg cost increased , continued.
Online performance P=0.3
70
80
90
100
110
120
130
140
0 20 40 60 80 100 120 140 160 180 200 220
Generations
Per
form
ance VM
FM
OptimumGoal
ConclusionsConclusions Optimization of Dynamic problems is growing. Needs further research
GA’s almost used exclusively in static applications… although their concept may suggest otherwise
Not all dynamic problems are challenging
DTSP was approached using an adaptive HGA
BM generator was developed for DTSP
VM showed some improvements over FM
High values of initial mutations are recommended
ConclusionsConclusions
Future work
Enhance the VM: mut. rate = f(performance)
Extend the scope from TSP to VRP
Compare HGA with other techniques, CPUT
Classifying and Prediction
Classifying and PredictionClassifying and Prediction
Classifying Input ANN
Predicting ChangesANN
Input Time Series
Optimization GA
OutputTracking Optimum
Thank You
Genetic Algorithms for Dynamic Vehicle Routing ProblemGenetic Algorithms for Dynamic Vehicle Routing Problem . . 3131
----------------------------------------
Recent DevelopmentsRecent Developments Adaptation of Genetic operators for dynamic
problems (Back 1997 Grefenstette 1999)
Hybridization of a GA and local search for VRPTW ( Braysy
2000)
Adaptive Tabu Search for dynamic VRPTW
(Gendreau 1999)
Little on GA in dynamic functionsNothing on VRP
Genetic Algorithms for Dynamic Vehicle Routing ProblemGenetic Algorithms for Dynamic Vehicle Routing Problem . . 3232
ObjectivesObjectives & Previous Work … & Previous Work …
Adapting the operators through externally imposed heuristics (Davis 1989, Back 1992)
Self-adapting mutation rates in static problems (Back and Schwefel
1993)
Self-adaptation of Genetic operators for searching dynamic fitness landscape (Back 1997)
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3333
ResultsResults
How well the moving optimum is tracked?
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3434
• Gradual shiftHyper mut. is better than ordinary
• Abrupt shiftOrdinary mut. unable to explore adequately
Base mutation rate = 0.03
Gradual
Abrupt
FHGHFMGM
Generations
Generations
Cur
rent
Bes
tC
urre
nt B
est
Tracking Performance on dynamic landscape
Performance of ordinary mut. models starts to deteriorate after 50 gen
Lo performance since initial pop is random
Performance deteriorates suddenly every 20 generations
Changing the base mut. Rate
In gradual shifting LS Base mutation rate
FHGHFMGM
Base mutation rate
Onl
ine
Per
form
ance
Cur
rent
Bes
t
Base mutation rate
0.1
Results…Results…
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3535
Hyper mut. Gives better performance than ordinary mut.GH & FH nearly same performance
FM has Lo performance
Improves beyond rate .o3
Too high mutation rate lowers performance
Model approaches rand. search
Results…Results…
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3636
FH is best GH FH FM improves after rate .03 All models deteriorate
beyond rate 0.1
Changing the base mut. Rate In abrupt shifting LS
Similar performance to gradual LS
Current Best
Percentage of pop under Hyp
Results…Results…
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3737
The level of hypermutation:– decreases as population converges near optimum. – Increases when landscape shifts
Relation between change in LS & level of hyper mutation
ConclusionsConclusions
Evolvability in Dyn. Fitness Landscapes: GA Approach. Evolvability in Dyn. Fitness Landscapes: GA Approach. 3838
Alternative models studied
–Models with same-mutation level to all–Models, Genetically controlled mut
Hyper mutation models perform well in all LS.
Hyper mutation can be genetically controlled
When genetically controlled , level of hypermutation:– decreases as population converges near optimum. – Increases when landscape shifts
What’s Adaptation?What’s Adaptation?
A characteristic that is often attributed to Intelligent Systems
Adaptation : to recognize change through inputs and
to adjust accordingly
RMLP capable of adaptation (Cotter and Conwell)
Adaptation from Fixed-Weight Dynamic Networks. Adaptation from Fixed-Weight Dynamic Networks. 3939
Our main question Can adaptive capability be induced directly from training ?
ResultsResults
Training was difficult
BUT
performance was good
Adaptation from Fixed-Weight Dynamic Networks. Adaptation from Fixed-Weight Dynamic Networks. 4040
Network performance.
Interpolative and extrapolative performance.
Network performance for switching time series.
Network performance for noisy time series.
Categoriztion useful to ; know the strategy And to appreciate the difficulty of BM design Dynamic but not noisy not noisey fitness .. noisy still approached as a static problem and the noise is treated in some specific way. not covered here. frequency of change In practice, we actually need the not the period between changes but the time allowed to the GA to find the sol to the new instance. average no of eval. is used iso time Severity of change It should be specified in conjucncion with the definition of neighborhood which in turn depends on the representation scheme of the individuals. In other words how many simple steps alterations or mutaions are to be applied on
the old optimal solution in order to reach the new one. Pattern of change Studying the pattern of changes can give insight to predict the direction , frequency or severity of change. Such information can be used in advance by the algortgm to figure out the best approach to tackle to oncoming
instances. Even if the pattern is completely random, knowing this fact might help in finding the proper strategy. Repetitiveness How often and how close does the old environment states are revisited? The main purpose here is to decide whether to use an explicit memory to remember old solutions or not and what is the length of the list … SEE TS And we add this categorization Detectability Are changes obvious i.e can be detected directly or not ? Adding a new assighmnet , vehicle breakdown…. Are detectable directly. While road jamming, deterioration in machine and manpower performance, and changes in quality of raw material are examples of envriomental changes that are not usually given explicitly. If the changes are not given explicitly the algoritm might not react in time to these changes.. In these canses, some kind of indicators that monitor performance can be used to trigger reactions to changes. Some of the used indicators are : Deterioration of the population performance REF , Time averaged best performance REF . These indicators assume that environmental changes will reduce the fitness of the individuals… however this is not necessarily true…
fitness values of all individuals might increase after a change in environment, in other cases the shift in enviornmnet might make the current population as a whole nearer to the new optimum and hence solution quality enhances.
In another method, used by Brankd 99, several individuals are revaluated every generation and a change in environment is detected if the fitness of at least one individuals has changed. Others REF compare the actual environment with a maintained model and conclude that the environment had changed if the difference between the actual and model environments is significant. =+++++++++++++++++ Optimization in dynamic environments is gaining increasing interest from researches due to the simple fact that almost all real-world problems are dynamic to some degree or another. Metaheuristics that had proved their effectiveness for static problems
are being modified by different adaptation strategies for the use in dynamic environments. In addition, benchmark problems were generated to model the dynamic environments. The current paper tests a Genetic Algorithm under different adaptation strategies to tackle the Dynamic Travelling Salesman Problem. It is expected that the GA as an evolutionary technique will work well in with dynamic problems. Another
contribution of this paper is a benchmark generator to create the dynamic instances necessary for testing and comparing these strategies. With integer spaces it is not easy, as in real space problems, to develop functions with adjustable parameters to simulate a shifting landscape. Here, we need to think of the dynamic environment in terms of possible scenarios in
which changes of a particular problem can happen over time. There can be an infinite number of such scenarios, which, we believe, is a reason behind the deficiency in benchmarks for dynamic combinatorial problems in general.
DETAILSDETAILS
Dynamic LandscapeDynamic Landscape at Generation 0
at Generation 5
at Generation 10
X
X
X
Fitn
ess
Fitn
ess
Fitn
ess
X- sections of fitness landscape
x
Am
plit
ude
(A)
Width (S)
Center (C)
2
2
2
) , (
S
Cd
eAG
X
X
Parameters A,C and S changed to create peaks with different widths, heights & locations
Dynamism introduced by changing fitness landscape with Generations
With real space
it is relatively easy to create dynamic landscapes as
time-varying functions :
by altering a few runtime parameters, one can generate
indefinite # of distinct landscapes with controllable characteristics
In the literature...since the late fifties..– orders to customers dispersed.– elderly or disabled passengers– cargo between seaports– work-in process between workstations
Importance– transportation cost constitutes a large share.– Benefits to business & the country.
VRP, OverviewVRP, Overview
Efficient routing of a fleet of vehicles to reduce transportation cost … that is the essence of VRP
Genetic Algorithms for Dynamic Vehicle Routing ProblemGenetic Algorithms for Dynamic Vehicle Routing Problem . . 4444
Intrigued researchers for years
Easy to describe, hard to solve
Typical of the NP-hard combinatorial problems
Often the case that TSP led to progress on other
combinatorial problems
Simply stated: if a traveling salesman wishes to visit exactly once each of a list of cities and then return to the home city, find the shortest route?
TSP TSP
focus on finding robust solutions.
if adapting old solutions is not possible
Robust SolutionsRobust Solutions
Robust Solution
Unstable Solution
Robust solutions are those which function well over wide ranges of environmental changes.
Environment changes too fast
Changes cannot be detected quickly enough,
Old solutions are already implemented.
Examples Specifications cannot be produced exactly. Tolerance needed
Scheduling: variation in processing times, malfunctions, or
adding new jobs w/o a total reordering of production plan.
Control Problems: it may be difficult to detect gradual changes
machines wear or raw material properties changes
Adaptation Not PossibleAdaptation Not Possible
ClassifyingClassifying Several strategies in the literature to tackle dynamic
problems: ignore, restart, adopt, … and hybridizations.
Important to be able to have some measurements.
How good a strategy. depends on:
speed of change, severity of change, repetitiveness, detect ability
Input Time Series
Classifying Input ANN
Use ANN to measure and classify the data
Use this classification to trigger which strategy the GA should use
PredictionPrediction A dynamic problem requires finding solutions while
time proceeds concurrently with incoming info.
Having insight to future info:
1) gives the GA the necessary time to solve Or
2) at least to switch to a better strategy
Predicting ChangesANN
Input Time Series
Use ANN to study past pattern and try to predict changes
Classifying and PredictionClassifying and Prediction
Classifying Input ANN
Predicting ChangesANN
Input Time Series
Optimization GA
OutputTracking Optimum
Dynamic LandscapeDynamic Landscape at t1
t2
t3F
itne
ssF
itne
ssF
itne
ss
Optimization goal changes
from finding an opt. sol of the static prob,
to continuously tracking the moving optimum in a changing (dynamic) env.
x1
x2
x3