Local Search:walksat, ant colonies, and genetic algorithms

Tonight

Course evaluations Local search Final exam

Issue

How to search really large spaces 10^30,000 + states systematic search hopeless!

Local search iterative repair, hill climbing, gradient

descent, … guess a start state repeat until satisfied: make small

change move to a local neighbor

Examples Learning weights in a neural network

Space: set of weights Neighborhood: small delta

Learning CPT’s in a Bayesian network EM algorithm

These are optimization problems… what about local search for decision problems?

N-Queens

N-Queens Demo

GSAT

Guess a random truth assignment

while (#unsat clauses > 0){flip a variable that minimizes number of unsatisfied clauses

}

Room for improvement Less dramatic performance on

structured problems (e.g. planning) Too greedy – can become stuck in local

minima Solution: allow some “uphill” moves –

many different strategies possible… Simulated annealing Tabu lists Alternate GSAT + random flips

Random Walk

Guess a random truth assignment

while (#unsat clauses > 0){pick an unsat clause

flip any variable in the clause

} Suppose clauses are of length 2;

what is probably a flip is “correct”? Solves 2-SAT in O(n^2) time!

Greediness + Randomness

If clause length > 2, then pure random walk is very unlikely to converge

Suppose we random walk greedily?

Walksat

Guess a random truth assignment

while (#unsat clauses > 0){pick an unsat clause

with probability p {

flip a variable that minimizes number of newly-unsatisfied clauses }

otherwise {

flip any variable in clause }

Walksat results Best known method for many problems

graph coloring (certain) planning problems hard random problems

For many (all?) other domains, can mix random walk & backtrack search run backtrack DPLL until time out restart with new random seed for choosing

branching variables

Stochastic Backtrack Search

Quasigroup Completion Demo

Parallel local search

Techniques considered so far only look at one point on the search space at a time…

Easy to run many copies in parallel with different random seeds Called portfolio approach Often gives super-linear speedup!

Run time distributions

Often there is a great variability in run time depending on random seeds…

As parallel processes are added, probability of getting a “lucky” short run can increase quickly!

Informed parallel search

Can we further improve performance by exchanging information between the parallel processes? Genetic algorithms Ant colony algorithms

Genetic algorithms Idea: the genetic code of an individual

is a point in the search space a gene = a dimension in the search space

Reproduction = local moves Asexual reproduction (mutation) = create a

child by a small change in the parent Sexual reproduction = create a child by

mixing genes of two parents

19

The GA Cycle of Reproduction

reproduction

population evaluation

modification

discard

deleted members

parents

children

modifiedchildren

evaluated children

20

A Simple Example

The Traveling Salesman Problem:

Find a tour of a given set of cities so that each city is visited only once the total distance traveled is minimized

21

Representation

Representation is an ordered list of city

numbers known as an order-based GA.

1) London 3) Dunedin 5) Beijing 7) Tokyo

2) Venice 4) Singapore 6) Phoenix 8) Victoria

CityList1 (3 5 7 2 1 6 4 8)

CityList2 (2 5 7 6 8 1 3 4)

22

Crossover

Crossover combines inversion and

recombination:

* *

Parent1 (3 5 7 2 1 6 4 8)

Parent2 (2 5 7 6 8 1 3 4)

Child (5 8 7 2 1 6 3 4)

This operator is called the Order1 crossover.

23

Mutation involves reordering of the list:

* *

Before: (5 8 7 2 1 6 3 4)

After: (5 8 6 2 1 7 3 4)

Mutation

24

TSP GA Demo http://cs.felk.cvut.cz/~xobitko/ga/tspexa

mple.html

25

TSP Example: 30 Cities

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

x

y

26

Solution i (Distance = 941)

TSP30 (Performance = 941)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

x

y

27

Solution j(Distance = 800)

44626967786462544250404038213567606040425099

TSP30 (Performance = 800)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

x

y

28

Solution k(Distance = 652)

TSP30 (Performance = 652)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

x

y

29

Best Solution (Distance = 420)

423835262135327

3846445860697678716967628494

TSP30 Solution (Performance = 420)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

x

y

30

Overview of Performance

TSP30 - Overview of Performance

0

200

400

600

800

1000

1200

1400

1600

1800

1 3 5 7 9 11

13

15

17

19

21

23

25

27

29

31

Generations (1000)

Dis

tan

ce

Best

Worst

Average

Practical Applications Design of small sorting circuits Optimization of code fragments

Used in Windows kernel??? Training neural networks

Can outperform backprop Airport scheduling (Generally) unanswered questions:

Would other local search techniques work just as well?

Is crossover really key?

Ant colony optimization Idea:

Each ant in a colony is local search process

Ants (processes) communicate by laying down phenoromes at visited states (“this place looked good to me”)

When deciding where to move, ants prefer (with some probability) to follow trail left by other ants

Phenorome eventually “evaporates”

Real ants

Why might it work?

More “intelligent” noise

Ant B @ t10

Ant A @ t3

gradient

gradient

not gradient, but a good alternative

Beyond phenomerones Combine local gradients to get more global

view

Ant B @ t10

Ant A @ t10

gradient

gradient

move by weighted sum of B’s own gradient and A’s gradient

Does it work?

Many successful applications in engineering & science

Appears to outperform single-thread local search procedures for quadratic programming

More work needed to determine exactly why metaphor works when it does!

Summary Local search is an important tool for

large, complex optimization and decision problems Ideas: Iterative repair Noise to escape local optima

Much work on parallel local search Portfolios Genetic algorithms Ant colony optimization

So….

End of the course. Is that all there is??? Where was the artificial

intelligence??!!

In my view… AI is the study of general problem-

solving strategies “Meta” algorithms 101 Life: one damned problem after

another Strategies evolve at many levels

Genetic – “real” evolution Individual human – logical thinking Society – spread of innovation

Unity

A relatively small number of concepts various state-space search strategies syntactic structure (both formal logic &

natural language) learning strategies – minimizing entropy,

error, complexity

arise over and over again in biological and artificial systems

Why? This unity of

thought biology computationis a cause for wonder …

Work in AI is one way of working on the ultimate questions Why? Why here? Why now? Why us?

As the Dali Lama once said…

“But if anything I have said is not helpful to you…

… then just forget about it!”