Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis.

Nature’s Nature’s AlgorithmsAlgorithms

David C. UhrigDavid C. Uhrig

Tiffany SharrardTiffany Sharrard

CS 477R – Fall 2007CS 477R – Fall 2007

Dr. George BebisDr. George Bebis

What are “Nature’s What are “Nature’s Algorithms”?Algorithms”?

Combinatorial Optimization Combinatorial Optimization ProblemsProblems No easy way to examine all No easy way to examine all

combinationscombinations Use algorithms based on scientific Use algorithms based on scientific

principles, nature, and geneticsprinciples, nature, and genetics This does not give optimal solutions, This does not give optimal solutions,

but is more practical than an but is more practical than an exhaustive searchexhaustive search

Example: Traveling Example: Traveling SalesmanSalesman

The Traveling Salesman problem is The Traveling Salesman problem is one of the most famous one of the most famous combinatorial optimization combinatorial optimization problems: problems: Given a set of cities (nodes), find the Given a set of cities (nodes), find the

shortest path required to visit all cities shortest path required to visit all cities exactly once.exactly once.

Example: Traveling Example: Traveling SalesmanSalesman

There are (There are (nn-1)!/2 possible tours for -1)!/2 possible tours for nn cities. cities. 2520 tours for 8 cities2520 tours for 8 cities 653,837,184,000 tours for 16 cities653,837,184,000 tours for 16 cities

Would require a computer nearly 8 days to Would require a computer nearly 8 days to compute them all! (for 1/1,000,000 sec per compute them all! (for 1/1,000,000 sec per tour generation)tour generation)

Not practical to compute all tours…Not practical to compute all tours…

Example: Location Example: Location AllocationAllocation

Given a set of facilities, each of Given a set of facilities, each of which can serve a certain number of which can serve a certain number of nodes on a graph, place the facilities nodes on a graph, place the facilities in such a way that the distance in such a way that the distance between each node and the facility is between each node and the facility is minimized.minimized. Police stations and population densitiesPolice stations and population densities Schools and student populationsSchools and student populations

Example: Location Example: Location AllocationAllocation

Issues with Combinatorial Issues with Combinatorial Optimization ProblemsOptimization Problems

Large set of discrete solutionsLarge set of discrete solutions Number of solutions grows exponentiallyNumber of solutions grows exponentially

In the Travelling Salesman problem, the In the Travelling Salesman problem, the solution size grows by solution size grows by nn*(*(nn-1)! for each city -1)! for each city that is added.that is added.

In the Location Allocation problem, the In the Location Allocation problem, the solution size is affected by both the number solution size is affected by both the number of nodes but also the connections between of nodes but also the connections between nodes nodes andand the number of (and size of) the number of (and size of) facilities.facilities.

Evolution and the Genetic Evolution and the Genetic AlgorithmAlgorithm

Mimics the process of evolution in Mimics the process of evolution in developing a solution.developing a solution.

Begins with a large set of solutions Begins with a large set of solutions to the problem in a candidate pool.to the problem in a candidate pool.

Each of these algorithms is tested Each of these algorithms is tested for ‘fitness’for ‘fitness’


After the fitnesses are calculated, a After the fitnesses are calculated, a new set of solutions is randomly new set of solutions is randomly picked from the ‘most fit’ algorithms.picked from the ‘most fit’ algorithms. Mimics Natural SelectionMimics Natural Selection

Next, two algorithms are paired Next, two algorithms are paired together and swap pieces of each together and swap pieces of each solutionssolutions

Then each algorithm changes Then each algorithm changes (mutates) slightly(mutates) slightly


This pattern of evaluation, This pattern of evaluation, reproduction, swapping, and reproduction, swapping, and mutating mimics the way DNA is mutating mimics the way DNA is processed during sexual processed during sexual reproduction.reproduction.

This process continues until a This process continues until a satisfactory solution is much better satisfactory solution is much better than the rest of the population of than the rest of the population of solutions.solutions.

Metallurgy and the Simulated Metallurgy and the Simulated Annealing ProblemAnnealing Problem

In metallurgy, when molten metal cools In metallurgy, when molten metal cools too quickly it can lose certain properties, too quickly it can lose certain properties, such as strength or magnetism.such as strength or magnetism.

Annealing is the process of slowly Annealing is the process of slowly cooling, which can help retain or amplify cooling, which can help retain or amplify desired traits.desired traits.

In Computer Science, Simulated In Computer Science, Simulated Annealing Algorithms are usually an Annealing Algorithms are usually an improvement on “Greedy Local Search” improvement on “Greedy Local Search” algorithms.algorithms.


1.1. Start with a proposed solution.Start with a proposed solution.

2.2. Change the solution slightlyChange the solution slightly• i.e. Swap two cities in the traveling i.e. Swap two cities in the traveling

salesman problem.salesman problem.

3.3. Check to see if the new solution is Check to see if the new solution is better than the original. If not, undo better than the original. If not, undo the change.the change.

4.4. Repeat steps 2 and 3 until it is Repeat steps 2 and 3 until it is apparent that any small change makes apparent that any small change makes the solution worse.the solution worse.


A greedy local searchA greedy local searchwill only find Locallywill only find LocallyOptimal Solutions.Optimal Solutions.

This also occurs with Genetic This also occurs with Genetic Algorithms when mutations stop Algorithms when mutations stop producing positive changes.producing positive changes.


Try to improve on Greedy Local Try to improve on Greedy Local Searches by taking occasional risks by Searches by taking occasional risks by accepting a worse solution.accepting a worse solution. Compute ΔCompute ΔCC, the difference between the , the difference between the

cost of the current solution and the proposed cost of the current solution and the proposed solution. Then:solution. Then:

for Random(for Random(rr), 0 < ), 0 < rr < 1, if < 1, if rr < < e e --

ΔΔCC//TT

Where Where TT is the metaphoric “temperature” is the metaphoric “temperature” that is iteratively decreased until an that is iteratively decreased until an acceptable temperature is reached.acceptable temperature is reached.


Theoretically, the temperature should Theoretically, the temperature should be lowered proportionally to 1/log(n), be lowered proportionally to 1/log(n), but this rate is too slow.but this rate is too slow.

Realistically, the temperature is Realistically, the temperature is lowered at a linear or exponential rate.lowered at a linear or exponential rate.

Even with this decreased cooling time, Even with this decreased cooling time, simulated annealing has proven to be simulated annealing has proven to be very effective in finding a balance very effective in finding a balance between exploring every solution and between exploring every solution and avoiding locally optimal results.avoiding locally optimal results.

EconomicsEconomics

The idea of “commodities” and The idea of “commodities” and “consumers” competing for them“consumers” competing for them

Prices are determined by Prices are determined by competitioncompetition

Consumers buy what they can affordConsumers buy what they can afford Prices are influenced by consumersPrices are influenced by consumers

Computer NetworksComputer Networks

Consumer = ProcessConsumer = Process Competing commodities: processor Competing commodities: processor

time, access to network printer, etctime, access to network printer, etc

ExampleExample

Two terminals want to share two Two terminals want to share two database filesdatabase files

The files will run slower if they are The files will run slower if they are accessed from the other location accessed from the other location than from the local locationthan from the local location

Decision should be made based on Decision should be made based on how much of each file should be how much of each file should be stored at each terminalstored at each terminal

SolutionSolution

Give each file a starting priceGive each file a starting price Give each terminal a fixed amount of Give each terminal a fixed amount of

moneymoney With constraints of the amount of money, With constraints of the amount of money,

terminals ask for a certain portion of each terminals ask for a certain portion of each filefile

Price changes based on the proportion of Price changes based on the proportion of how much of the files were requestedhow much of the files were requested

Done until equilibrium is reachedDone until equilibrium is reached

Simulated N-Body Simulated N-Body AlgorithmAlgorithm

Study of moving particles in physicsStudy of moving particles in physics Particles are allowed to move Particles are allowed to move

around the grapharound the graph To minimize potential energy in the To minimize potential energy in the

system, particles tend to spread outsystem, particles tend to spread out A “boundary” is formed to keep the A “boundary” is formed to keep the

particles from being pushed to the particles from being pushed to the outer edgeouter edge

Aggregation and Aggregation and DecompositionDecomposition

Every particle is grouped into a larger Every particle is grouped into a larger particle with combined massparticle with combined mass

Smallest particle is removedSmallest particle is removed Small and the largest particle moved Small and the largest particle moved

to where the potential energy is the to where the potential energy is the lowestlowest

Take next smallest particle and do the Take next smallest particle and do the samesame

Result: reasonably good configurationResult: reasonably good configuration

Particle PhysicsParticle Physics

Simulation used to solve traveling Simulation used to solve traveling salesman problemssalesman problems Use of charged particles connected by Use of charged particles connected by

electric bandselectric bands

How Does It Work? How Does It Work?

Number of Cities = Number of negatively- Number of Cities = Number of negatively- charged particles = Number of positively-charged particles = Number of positively-charged particlescharged particles

Positive particles are placed in a circlePositive particles are placed in a circle Solution:Solution:

Particles are set freeParticles are set free When particle hits a city-stopsWhen particle hits a city-stops When all have stopped- elastic string traces a When all have stopped- elastic string traces a

path through each city path through each city

Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis.

Documents

Transcript of Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis.