Chapter 4 Chapter 4

Informed Search and ExplorationInformed Search and Exploration

OutlineOutline Informed (Heuristic) search strategies

(Greedy) Best-first search A* search

(Admissible) Heuristic Functions Relaxed problem Subproblem

Local search algorithms Hill-climbing search Simulated anneal search Local beam search Genetic algorithms

Online search * Online local search learning in online search

Informed search strategiesInformed search strategies Informed search

uses problem-specific knowledge beyond the problem definition

finds solution more efficiently than the uninformed search

Best-first search uses an evaluation function f(n) for each node

e.g., Measures distance to the goal – lowest evaluation Implementation:

fringe is a queue sorted in increasing order of f-values. Can we really expand the best node first?

No! only the one that appears to be best based on f(n). heuristic function h(n)

estimated cost of the cheapest path from node n to a goal node Specific algorithms

greedy best-first search A* search

Greedy best-first searchGreedy best-first search expand the node that is closest to the goal : Straight line distance heuristic)()( nhnf SLD

Greedy best-first search exampleGreedy best-first search example

Properties of Greedy best-first Properties of Greedy best-first search search

Complete?

Optimal?

Time?

Space?

No

No – can get stuck in loops, e.g., Iasi –> Neamt –> Iasi –> Neamt

Yes – complete in finite states with repeated-state checking

)( mbO , but a good heuristic function can give dramatic improvement

)( mbO – keeps all nodes in memory

A* searchA* search evaluation function f(n) = g(n) + h(n)

g(n) = cost to reach the node h(n) = estimated cost to the goal from n f(n) = estimated total cost of path through n to the goal

an admissible (optimistic) heuristic never overestimates the cost to reach the goal estimates the cost of solving the problem is less than it actually is e.g., never overestimates the actual road distances

A* using Tree-Search is optimal if h(n) is admissible could get suboptimal solutions using Graph-Search

might discard the optimal path to a repeated state if it is not the first one generated

a simple solution is to discard the more expensive of any two paths found to the same node (extra memory)

)(nhSLD

: Straight line distance heuristic)(nhSLD

A* search exampleA* search example

Optimality of A*Optimality of A*)(),,()( nhnancnh Consistency (monotonicity)

n’ is any successor of n, general triangle inequality (n, n’, and the goal) consistent heuristic is also admissible

A* using Graph-Search is optimal if h(n) is consistent the values of f(n) along any path are nondecreasing

)()()( nhngnf

)(),,()( nhnancng

)()()( nfnhng

Properties of A*Properties of A* Suppose C* is the cost of the optimal solution path

A* expands all nodes with f(n) < C* A* might expand some of nodes with f(n) = C* on the “goal contour” A* will expand no nodes with f(n) > C*, which are pruned! Pruning: eliminating possibilities from consideration without examination

A* is optimally efficient for any given heuristic function no other optimal algorithm is guaranteed to expand fewer nodes than A* an algorithm might miss the optimal solution if it does not expand all nodes

with f(n) < C*

A* is complete

Time complexity exponential number of nodes within the goal contour

Space complexity keeps all generated nodes in memory runs out of space long before runs out of time

Memory-bounded heuristic searchMemory-bounded heuristic search Iterative-deepening A* (IDA*)

uses f-value (g + h) as the cutoff

Recursive best-first search (RBFS) replaces the f-value of each node along the path with the best f-value of its

children remembers the f-value of the best leaf in the “forgotten” subtree so that it can

reexpand it later if necessary is efficient than IDA* but generates excessive nodes changes mind: go back to pick up the second-best path due to the extension

(f-value increased) of current best path optimal if h(n) is admissible space complexity is O(bd) time complexity depends on the accuracy of h(n) and how often the current

best path is changed

Exponential time complexity of Both IDA* and RBFS cannot check repeated states other than those on the current path when search

on Graphs – Should have used more memory (to store the nodes visited)!

: Straight line distance heuristic)(nhSLD

RBFS exampleRBFS example

Memory-bounded heuristic search (cont’d)Memory-bounded heuristic search (cont’d)

SMA* – Simplified MA* (Memory-bounded A*) expands the best leaf node until memory is full then drops the worst leaf node – the one has the highest f-value regenerates the subtree only when all other paths have been shown to look

worse than the path it has forgotten complete and optimal if there is a solution reachable might be the best general-purpose algorithm for finding optimal solutions

If there is no way to balance the trade off between time an memory, drop the optimality requirement!

(Admissible) Heuristic Functions(Admissible) Heuristic Functions

h1?

h2?

)(1 nh = the number of misplaced tiles

)(2 nh = total Manhattan (city block) distance

= 7 tiles are out of position

= 4+0+3+3+1+0+2+1 = 14

Effect of heuristic accuracyEffect of heuristic accuracy Effective branching factor b*

total # of nodes generated by A* is N, the solution depth is d b* is b that a uniform tree of depth d containing N+1 nodes would have

well-designed heuristic would have a value close to 1 h2 is better than h1 based on the b*

Domination h2 dominates h1 if for any node n A* using h2 will never expand more nodes than A* using h1 every node n with will be expanded

the larger the better, as long as it does not overestimate!

dbbbN *2** ...11

)()( 12 nhnh

*)( Cnf *)()()( Cnhngnf )()( * ngCnh

)()()( *21 ngCnhnh

Inventing admissible heuristic functionsInventing admissible heuristic functions h1 and h2 are solutions to relaxed (simplified) version of the puzzle.

If the rules of the 8-puzze are relaxed so that a tie can move anywhere, then h1 gives the shortest solution

If the rules are relaxed so that a tile can move to any adjacent square, then h2 gives the shortest solution

Relaxed problem: A problem with fewer restrictions on the actions Admissible heuristics for the original problem can be derived from the

optimal (exact) solution to a relaxed problem Key point: the optimal solution cost of a relaxed problem is no greater than

the optimal solution cost of the original problem Which should we choose if none of the h1 … hm dominates any of the

others?We can have the best of all worlds, i.e., use whichever function is most accurate on the current node

Subproblem * Admissible heuristics for the original problem can also be derived from the

solution cost of the subproblem. Learning from experience *

)(),...,(max)( 1 nhnhnh m

Local search algorithms and optimizationLocal search algorithms and optimization

Systematic search algorithms to find (or given) the goal and to find the path to that goal

Local search algorithms the path to the goal is irrelevant, e.g., n-queens problem state space = set of “complete” configurations keep a single “current” state and try to improve it, e.g., move to its

neighbors Key advantages:

use very little (constant) memory find reasonable solutions in large or infinite (continuous) state spaces

(pure) Optimization problem: to find the best state (optimal configuration ) based on an objective

function, e.g. reproductive fitness – Darwinian, no goal test and path cost

Local search – example Local search – example

Local search – state space landscapeLocal search – state space landscape elevation = the value of the objective function or heuristic cost function

global minimum

heuristic cost function

A complete local search algorithm finds a solution if one exists A optimal algorithm finds a global minimum or maximum

moves in the direction of increasing value until a “peak” current node data structure only records the state and its objective function neither remember the history nor look beyond the immediate neighbors like climbing Mount Everest in thick fog with amnesia

Hill-climbing searchHill-climbing search

complete-state formulation for 8-queens successor function returns all possible states generated by moving a single queen

to another square in the same column (8 x 7 = 56 successors for each state) the heuristic cost function h is the number of pairs of queens that are attacking

each other

Hill-climbing search - exampleHill-climbing search - example

best moves reduce h = 17 to h = 12 local minimum with h = 1

Hill-climbing search – greedy local searchHill-climbing search – greedy local search Hill climbing, the greedy local search, often gets stuck

Local maxima: a peak that is higher than each of its neighboring states, but lower than the global maximum

Ridges: a sequence of local maxima that is difficult to navigate

Plateau: a flat area of the state space landscape a flat local maximum: no uphill exit exists a shoulder: possible to make progress

can only solve 14% of 8-queen instance but fast (4 steps to S and 3 to F)

Hill-climbing search – improvementHill-climbing search – improvement Allows sideways move: with hope that the plateau is a shoulder

could stuck in an infinite loop when it reaches a flat local maximum limits the number of consecutive sideways moves can solve 94% of 8-queen instances but slow (21 steps to S and 64 to F)

Variations stochastic hill climbing

chooses at random; probability of selection depends on the steepness first choice hill climbing

randomly generates successors to find a better one All the hill climbing algorithms discussed so far are incomplete

fail to find a goal when one exists because they get stuck on local maxima Random-restart hill climbing

conducts a series of hill-climbing searches; randomly generated initial states Have to give up the global optimality

landscape consists of a large amount of porcupines on a flat floor NP-hard problems

Simulated annealing searchSimulated annealing search

combine hill climbing (efficiency) with random walk (completeness)

annealing: harden metals by heating metals to a high temperature and gradually cooling them

getting a ping-pong ball into the deepest crevice in a humpy surface shake the surface to get the ball out of the local minima not too hard to dislodge it from the global minimum

simulated annealing: start by shaking hard (at a high temperature) and then gradually reduce the

intensity of the shaking (lower the temperature) escape the local minima by allowing some “bad” moves but gradually reduce their size and frequency

Simulated annealing search - ImplementationSimulated annealing search - Implementation

Always accept the good moves The probability to accept a bad move

decreases exponentially with the “badness” of the move decreases exponentially with the “temperature” T (decreasing)

finds a global optimum with probability approaching 1 if the schedule lowers T slowly enough

0E

0E

Local beam searchLocal beam search

Local beam search: keeps track of k states rather than just one generates all the successors of all k states selects the k best successors from the complete list and repeats quickly abandons unfruitful searches and moves to the space where the

most progress is being made – “Come over here, the grass is greener!”

lack of diversity among the k states

stochastic beam search: chooses k successors at random, with the probability of choosing a given successor having an increasing value

natural selection: the successors (offspring) if a state (organism) populate the next generation according to is value (fitness).

Genetic algorithmsGenetic algorithms Genetic Algorithms (GA): successor states are generated by combining

two parents states. population: s set of k randomly generated states each state, called individual, is represented as a string over a finite

alphabet, e.g. a string of 0s and 1s; 8-queens: 24 bits or 8 digits for their positions

fitness (evaluation) function: return higher values for better states,e.g., the number of nonattacking pairs of queens

randomly choosing two pairs for reproducing based on the probability; proportional to fitness score; not choosing the similar ones too early

Genetic algorithms (cont’d)Genetic algorithms (cont’d)

schema: a substring in which some of the positions can be left unspecified instances: strings that match the schema GA works best when schemas correspond to meaningful components of a

solution.

a crossover point is randomly chosen from the positions in the string larger steps in the state space early and smaller steps later

each location is subject to random mutation with a small independent probability