Lecture: 5 Optimization Methods & Heuristic Strategies

Ajmal Muhammad, Robert ForchheimerInformation Coding Group

ISY Department

Outline

Introduction to Optimization & Intractable Problems Linear Programming Heuristic Metaheuristic Linear Programming Formulation

Shortest path Problem

Max Flow Problem

Wavelength Assignment Problem

Optimization Problem

Problem that aims to find the best solution from all feasible solutions

The best solution can be the minimum or maximum solution

Examples:

Find the route from point A to point B that takes the shortest timesDetermine how a production factory can maximize its profit using limited materials

The Travelling Salesman Problem (TSP)

A classic combinatorial optimization problem

Given n cities on a map, find the shortest route that visits all the cities once, and starts and ends at the same city

Starting and ending in Paris, whichroute allow us to visit major citieswith the least amount of travelling ?

The 1-Dimensional Bin Packing Problem

Given n items of different (1D) sizes, and given some fixed- capacity bins, pack the items into a minimum number of bins

Is this the minimal number of Bins needed for these items?

TSP Growth Rates A route around a map can be represented as a permutation of the n

cities:

E.g. for 5 cities, [B,A,C,E,D] means “start at city B, then go to city A, then city C, then E, then D, and return to city B”

Given n cities, there is a total of n! permutations Some permutations represent the same routes - there are actually

½(n-1)! different routes in total To return the provably optimal solution to any instance of the TSP

the algorithm needs to check the majority of – if not all – possible routes

The number of routes grows exponentially quickly making the problem intractable:

Linear Programming (LP)

An optimization problem in which the objective function and all the constraints are expressed as linear functions

even if just one of them is not a linear function, the problem is not an LP problem

Linear programming problem

Nonlinear programming problem

LP Formulation

Matrix expression for LP formulation

Nomenclature in LP Problem

Boundary is a constraint that express the upper or lower bound of an inequality or equality

Feasible region is an area delineated by the boundaries

Corner point is an intersection of the boundaries

Example

Let z= x+y max zRewrite as y= -x + z

Solution by Moving y = -x +z

Simplex Method

Heuristics

• Provide a shortcut to solve difficult problems.• Used under limited time and/or information to make a

decision.• Problem-dependent techniques, i.e., adapted to the

problem at hand to take advantage of the particularities of the problem.

• Lead to a good decision most of the time.

Heuristics are good when Making a spur-of-the-moment decision Having limited information and cannot obtain more The decision is not that important

Heuristics

Heuristics are bad whenHaving plenty of time and information to make an important decisionNeed to be right 100% of the time

Examples of heuristicsDeciding to eat at restaurant B rather than restaurant A only because B has more cars in its parking lotWhen the level in the fuel tank drops to 1/2 tank or less, always filling up the fuel tank at the very next re-filling station

Metaheuristic

Meta: in an upper level

Heuristic: to find

A metaheuristic is formally defined as an iterative generation process which guides a subordinate heuristic by combining intelligently different concepts for exploring and exploiting the search space, learning strategies are used to structure information in order to find efficiently near-optimal solutions.

A metaheuristic is a function that consists of heuristic subroutines. It is a compound heuristic.

Fundamental Properties of Metaheuristics

Metaheuristics are strategies that guide the search process to avoid being trapped in confined areas of the search space

The goal is to efficiently explore the search space in order to find (near)-optimal solutions

Techniques which constitute metaheuristic algorithms range from simple local search procedures to complex learning processes

Metaheuristic algorithms are approximate and usually non-deterministic

Metaheuristics are not problem-specific

Metaheuristic Examples

Genetic algorithms (1960s/1970s), Evolutionary strategy (Rechenberg & Swefel 1960s), Evolutionary programming (Fogel et al. 1960s).

Simulated annealing (Kirkpatrick et al. 1983), Tabu search (Glover 1980s), Ant colony optimization (Dorigo 1992), Genetic programming (Koza 1992), Particle swarm optimization (Kennedy & Eberhart 1995), Differential evolution (Storn & Price 1996/1997),

Harmony search (Geem et al. 2001), Honeybee algorithm (Nakrani & Tovey 2004), ..., Firefly algorithm (Yang 2008), Cuckoo search (Yang & Deb 2009), ...

Genetic Algorithms: TSP Example

Genetic Algorithms: TSP Example

Genetic Algorithms: TSP Example

Genetic Algorithms: TSP Example

Genetic Algorithms: TSP Example

Ant Colony Optimization (in one Slide)

A. Ants in a pheromone trail between nest and food; B. an obstacle interrupts the trail; C. ants find two paths to go around the obstacle; D. a new pheromone trail is formed along the shorter path.

LP Formulation for Shortest Path Problem

Network is represented by directed graph G(V,E), where V is the set of vertices (nodes) & E is the set of links

Link from node i to node j is expressed by (i,j); dij is the link cost of (i,j)

Xpqij : traffic volume from node p to node q routed through (i,j)

Max flow Problem

Network model with link capacities

Max flow routing from node 1 to node 6

ILP formulation for max flow problem ?Assignment

Wavelength Assignment (WA) Problem

Optical path requests

Graph construction for coloring

LP Formulation for Graph Coloring

Let W be a set of λ, where W = {λ1,λ2,··· ,λ|W|}

Let xλv and yλ, be binary variables.

If λ is assigned to a path corresponding v, xλv = 1, otherwise xλ

v = 0.

If λ is used at least one time yλ=1, otherwise yλ=0.

Heuristic: Largest Degree First (LDF)

Wavelength assignment process of LDF

Step 1: select the uncolored vertex with largest degree.

Step 2: choose the min indexed colorfrom the colors that are not used by the adjacent vertices.

Step 3: color the selected vertex using the color chosen in step 2.

Step 4: if all vertices are colored, LDF stops. otherwise, LDF returns to step 1.