Principles*of*Ar.ficial*Intelligence* · PennsylvaniaState*University* **** College of Information...
Transcript of Principles*of*Ar.ficial*Intelligence* · PennsylvaniaState*University* **** College of Information...
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Ar.ficial Intelligence
Vasant Honavar Ar3ficial Intelligence Research Laboratory
College of Informa3on Sciences and Technology Bioinforma3cs and Genomics Graduate Program
The Huck Ins3tutes of the Life Sciences Pennsylvania State University
[email protected] hHp://vhonavar.ist.psu.edu
hHp://faculty.ist.psu.edu/vhonavar
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Summary of uninformed (blind) search algorithms Criterion Breadth
-First Depth-First
Iterative deepening
Bi-directional
Complete? YES NO YES YES*
Time bd+1 bm bd bd/2
Space bd+1 bm bd bd/2
Admissible?
Optimal?
YES NO
NO NO
YES YES
YES* NO
Assuming all arc costs are equal m – max depth of search d – depth of solu7on, b – finite branching factor * Assuming forward and backward search are BFS
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Finding an op3mal solu3on
• All operator applica3ons may not be equally expensive • Suppose we have a cost func3on c: S x O à ℜ+
• c (q,o,r) = cost of applying operator o in state q to reach state r • Path cost is typically assumed to be the sum of costs of operator
applica3ons along the path • An op3mal solu3on is one with the lowest cost path from the
specified start state s to a goal g ∈ G
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Finding op3mal (minimum cost) solu3ons
• Branch and bound search (BBS) with dynamic programming • Maintain a list of nodes sorted by cost g(n) of cheapest known par3al
paths to the respec3ve nodes • Terminate when a node picked from the open list happens to be a goal
node A
B
D
C
G E F
I H
1
3 8
2
31
2 1
((A,0))
((AB, 1) (AC, 2)
((AC, 2) (ABD, 4)(ABG,9))
((ACE, 3) (ABD, 4)(ACF, 5) (ABG,9))
((ACEI, 4) (ABD, 4)(ACEH, 5)(ACF, 5) (ABG,9))
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Finding op3mal solu3ons
• Branch and bound search (BBS) with dynamic programming • Maintain a list of nodes sorted by cost g(n) of cheapest known par3al paths to
the respec3ve nodes • Terminate when a node picked from the open list happens to be a goal node • Ques3ons:
– Is BBS complete? • Yes
– Is BBS admissible? • Yes
– Under the assump3on that each arc cost is bounded from below by some posi3ve constant δ
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Informed search
• Informed search uses problem-‐specific knowledge • Which search strategies?
– Best-‐first search and its variants • Heuris3c func3ons
– How to design them • Local search and op3miza3on
– Hill climbing, local beam search, gene3c algorithms,… • Local search in con3nuous spaces • Online search agents
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
A heuris3c func3on
• [dic3onary]“A rule of thumb, simplifica7on, or educated guess that reduces or limits the search for solu7ons in domains that are difficult and poorly understood.”
Heuris3cs, those rules of thumb, Oaen scorned as sloppy, dumb, Yet slowly commonsense become!
– Judea Pearl, in Heuris7cs
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Why we need heuris3cs • Combinatorial explosion • Uninformed search runs out of 3me, space, or both quickly • Solu3on – incorporate heuris3cs to guide search • Perfect heuris3c would guide search directly from start state
to a goal state yielding the least expensive solu3on – No need to search!
• In prac3ce, we have imperfect but useful heuris3cs • A heuris3c func3on h(n), roughly speaking, es3mates the cost
of comple3ng a par3al path (s..n) to obtain a solu3on, i.e., a path (s..n…g) where g is a goal state
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Why we need heuris3cs • Combinatorial explosion • Uninformed search runs out of 3me, space, or both quickly • Perfect ra3onality unaffordable • Can we come up with a compromise?
– Use a heuris3c func3on to guide choice of ac3ons • Suppose we could es3mate the cost of the cheapest solu3on
obtainable by expanding each node that could be chosen
A
s1 s2 s3
( )( )
( ) ( ) ( ) 6.1 0.2 8.00 ,
0 ,:
321 ===
=∈∀
≥∈∀
ℜ→ +
shshshghGgnhSn
Sh
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
A heuris3c func3on
In problems requiring the cheapest path from start state to a goal state
– h(n) = es3mated cost of the cheapest path from node n to a goal node
– h(g1)=h(g2)=0 s
n
g2
m
g1
h(m) h(n)
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Hill-‐climbing search
• Choose “locally best” moves, guided by the heuris3c func3on, with random choice to break 3es
• Terminates when a goal state is reached • Does not look beyond the immediate successors of the
current state in deciding which move to make • Essen3ally DFS, where at each step, successors are
ordered by heuris3c evalua3on • a.k.a. greedy local search
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Heuris3c func3on -‐ Example
• E.g for the 8-‐puzzle – Avg. solu3on cost is about 22 steps – Exhaus3ve search to depth 22 ≈ 3.1 x 1010 states – A good heuris3c func3on can reduce search
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Heuris3c func3ons
Two commonly used heuris3cs • h1 = the number of misplaced 3les (not coun3ng blank) rela3ve
to the goal (why?) h1(s)=8
• h2 = the sum of the distances of the 3les (not coun3ng blank) from their desired posi3ons (ManhaHan distance) (why?) h2(s)=3+1+2+2+2+3+3+2=18
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Drawbacks of hill-‐climbing search
• Ridge – sequence of local maxima difficult for greedy algorithms to navigate
• Plateau – an area of the state space where the evalua3on func3on is flat
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Hill-‐climbing varia3ons
• Stochas3c hill-‐climbing – Random selec3on among the uphill moves – The selec3on probability can vary with the steepness of the uphill move
• First-‐choice hill-‐climbing – stochas3c hill climbing by genera3ng successors randomly un3l a beHer one is found
• Random-‐restart hill-‐climbing – Tries to avoid geqng stuck in local maxima
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Best-‐first search
• General approach to informed search: – Best-‐first search: node is selected for expansion based on an evalua7on func7on f(n)
• Idea: evalua3on func3on measures es3mated cost of comple3ng the par3al solu3on (s..n) – Choose node which appears best
• Implementa3on: – Open-list is a queue sorted in decreasing order of desirability – Special cases: greedy search, A* search
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Examples of Best-‐first search
• Best first search – List of par3al paths is ordered by h values of the nodes
• A* search – BBS-‐like search with dynamic programming – Open list nodes ordered by
( ) ( ) ( )nhngnf +=
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
A* search
s
n
g2
m
g1
h(m)=6 h(n)=8
g(n) =2 g(m)=3
3 4
2
3 5 6 8
( ) ( ) ( )nhngnf +=
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Romania with step costs in km
• hSLD= straight-‐line distance heuris3c.
• hSLD cannot be computed from the problem descrip3on, requires addi3onal informa3on or knowledge on the part of the agent
• In this example f(n)=h(n) – Expand node that is closest to goal
= Greedy best-‐first search
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
A* search
• Best-‐known form of best-‐first search • Idea: avoid expanding paths that are already expensive • Evalua3on func3on f(n)=g(n) + h(n)
– g(n) the cost (so far) to reach the node. – h(n) es3mated cost to get from the node to the goal – f(n) es3mated total cost of path through n to goal
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
A* search
• A* search using an admissible heuris3c – A heuris3c is admissible if it never overes7mates the cost to reach the goal
– Op3mis3c
Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n 2. h(n) >= 0 so h(G)=0 for any goal G.
e.g. hSLD(n) never overes3mates the actual road distance
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Romania example
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Admissible heuris3cs • A heuris3c h(n) is admissible if for every node n,
h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n. • An admissible heuris3c never overes3mates the cost to reach the goal, i.e., it
is op3mis3c • Example: hSLD(n) (never overes3mates the actual road distance) • Theorem: If h(n) is admissible, A* using TREE-SEARCH is admissible • (provided each arc cost is bounded from below by a posi3ve constant δ) • Proof Sketch:
– Show that A* terminates with a solu3on – Show that A* terminates with an op3mal solu3on
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Completeness of A*: A* Terminates with a solu3on
• Assump3on: Cost of each arc is bounded from below by a posi3ve constant
• Any 3me a path gets extended, its cost increases by at least • There is a solu3on, i.e., a bounded cost path to a goal (by assump3on) • Termina3on condi3on: a path termina3ng in a goal is picked off the front of
the list of par3al paths L • There are two kinds of paths other than those that terminate in a goal:
– Paths that lead to dead ends – discovered and discarded (if their cost is lower than that of a path to a goal)
– Paths that are infinite (demoted behind a path leading to a goal because sooner or later their cost exceeds the cost of a path leading to a goal)
– See notes for a formal proof.
€
∀ni,n j ,c ni,n j( ) ≥ δ > 0
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Admissibility of A*: A* cannot terminate with a subop3mal goal
Suppose some subop3mal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an op3mal goal G.
f(G2) = g(G2) since h(G2) = 0 g(G2) > g(G) since G2 is subop3mal f(G) = g(G) since h(G) = 0 f(G2) > f(G) from above
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Admissibility of A*: A* cannot terminate with a subop3mal goal
Suppose some subop3mal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an op3mal goal G.
f(G2) > f(G) from above h(n) ≤ h*(n) since h is admissible g(n) + h(n) ≤ g(n) + h*(n) f(n) ≤ f(G) Hence f(G2) > f(n), and A* will never select G2 for expansion That is, A* tree search is admissible provided the heuris3c func3on
is admissible See notes for a more formal proof.
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
What if the search space is not a tree?
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Graph search • Previous proof breaks down • Possible solu3ons:
– Discard more expensive paths when there are mul3ple paths to a node • Adds messy book-‐keeping • Admissible if h is admissible
– Ensure that op3mal path to any repeated state is always first followed • Extra requirement on h(n): consistency
∀n,∀n ' where n ' is a child of n, h(n) ≤ h(n ')+ c(n,n ')
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Consistency If h is consistent, we have
f(n) is non-‐decreasing along any path à First goal selected for expansion must be an op3mal goal à A* using graph search is admissible provided the heuris3c used is
consistent
)(),()( mhmncnh +≤
)()()(
)(),()()()()(
nfnhng
mhmncngmhmgmf
>
++≥
++=
+=
δ
c ( n ; m ) c ( n ; m )
n
m
G
h ( n )
h ( m )
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Proper3es of A* search
• Complete?: YES – Since bands of increasing f are added – Unless there are infinitely many nodes with f < f(G)
• Not possible when the branching factor is finite and arc costs are bounded from below by δ
• Time complexity: – exponen3al in the length of the solu3on (in the worst case)
• Space complexity: – Need to maintain all generated nodes in memory – Space is a more serious problem than 3me
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Memory-‐bounded heuris3c search
• Some solu3ons to A*’s space problems – Itera3ve-‐deepening A* (IDA*)
• Cutoff informa3on is the f-‐cost (g+h) instead of depth • Can expand too many nodes when every node has a different f value
– (simple) Memory-‐bounded A* ((S)MA*) • Drop the worst-‐leaf node when memory is full
– Recursive best-‐first search (RBFS) • Recursive algorithm that aHempts to mimic standard best-‐first search with linear space
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
(simple) memory-‐bounded A*
• Use all available memory. – i.e. expand best leafs un3l available memory is full – When full, SMA* drops worst leaf node (highest f-‐value) – Backup the f-‐value of the forgoHen node to its parent
• What if all leafs have the same f-‐value? – Same node could be selected for expansion and dele3on – SMA* solves this by expanding newest best leaf and dele3ng oldest worst leaf.
• SMA* is complete if solu3on is reachable, admissible if op3mal solu3on is reachable within the available memory bound
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Recursive best-‐first search (RBFS)
• Similar to IDA* but keeps track of the f value of the best-‐alterna3ve path available from any ancestor of the current node – If current f values exceeds this alterna3ve f value then backtrack to the best alterna3ve path
– During backtracking (unwinding of recursion) change f value of each node to best f-‐value of its children
– Remembers the f value of the best leaf in the “forgoHen” sub-‐tree and hence can revisit that sub-‐tree, if warranted, at some point in the future
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Recursive best-‐first search func.on RECURSIVE-‐BEST-‐FIRST-‐SEARCH(problem) return a solu3on or failure
return RFBS(problem,MAKE-‐NODE(INITIAL-‐STATE[problem]),∞) func.on RFBS( problem, node, f_limit) return a solu3on or failure and a new f-‐
cost limit if GOAL-‐TEST[problem](STATE[node]) then return node successors ← EXPAND(node, problem) if successors is empty then return failure, ∞ for each s in successors do f [s] ← max(g(s) + h(s), f [node]) repeat best ← the lowest f-‐value node in successors if f [best] > f_limit then return failure, f [best] alterna7ve ← the second lowest f-‐value among successors result, f [best] ← RBFS(problem, best, min(f_limit, alterna7ve)) if result ≠ failure then return result
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Recursive best-‐first search: Example
• Path un3l Rumnicu Vilcea is already expanded • f-‐limit for every recursive call is shown above the node • f value is shown below the node • The path is followed un3l Pites3 which has a f value worse than
the f-‐limit.
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Recursive best-‐first search: Example
• Unwind recursion and store best f-‐value for current best leaf Pites3 result, f [best] ← RBFS(problem, best, min(f_limit, alterna7ve))
• best is now Fagaras. Call RBFS for new best – best value is now 450
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Recursive best-‐first search: example.
• Unwind recursion and store best f-‐value for current best leaf Fagaras
• best is now Rimnicu Viclea (again). Call RBFS for new best – Subtree is again expanded. – Best alterna7ve subtree is now through Timisoara.
• Solu3on is found because 447 > 417.
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
RBFS
• RBFS is a bit more efficient than IDA* – S3ll excessive node genera3on – Like A*, admissible if h(n) is admissible
• Space complexity is O(bd). • Time complexity difficult to characterize
– Depends on accuracy if h(n) and how oaen best path changes • IDA* and RBFS suffer from too li&le memory • Other variants of A* aim to address these limita3ons
Pennsylvania State University
College of Information Sciences and Technology Artificial Intelligence Research Laboratory
Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar
Learning to search beHer
• All previous algorithms use fixed strategies. • Agents can learn to improve their search by exploi3ng
experience gained during the search (e.g., by analyzing missteps) • We will see examples of this when we consider learning agents