CISC453 Winter 2010 Classical Planning AIMA3e Ch 10.

CISC453 Winter 2010

Classical PlanningAIMA3e Ch 10

Classical Planning

2

What is Planning?

generate sequences of actions to perform tasks and achieve objectives involving states, actions & goals

a search for a solution over an abstract space of plans the assumptions for classical planning problems

fully observable we see everything that matters

deterministic the effects of actions are known exactly

static no changes to environment other than those caused by agent

actions discrete

changes in time and space occur in quantum amounts single agent

no competition or cooperation to account for

Classical Planning

3

Background: Planning

real systems help humans in practical applications design and manufacturing environments military operations, games, space exploration

notes: the methods of Ch 10 (Classical Planning)

assume the classical environment assumptions in Ch 11 (Planning & Acting in the Real World)

we'll introduce methods to handle real-world situations where the classical assumptions may not hold

Classical Planning

4

Planning: Language

What is a good language? expressive enough to describe a wide variety of problems restrictive enough for efficient algorithms to operate on it planning algorithm should be able to take advantage

of the logical structure of the problem

historical AI planning languages STRIPS was used in classical planners

Stanford Research Institute Problem Solver ADL addresses expressive limitations of STRIPS

Action Description Language adds features not in STRIPS

negative literals, quantified variables, conditional effects, equality

we'll use one version of the now de facto standard PDDL

Classical Planning

5

Planning language

there's now wide agreement on some planning language basics

one key is adopting of a factored representation for states each state is represented as a collection of variables this contrasts with

the atomic representation typical of our previous search algorithms (recall the Romanian driving problem: state = city)

the language we'll use is a variant of PDDL Planning Domain Definition Language

to see its expressive power, recall propositional agent in the Wumpus World, which requires 4Tn2 actions to describe a movement of 1 square

PDDL captures this with a single Action Schema

Classical Planning

6

State & Action Representations

each state is represented as a conjunction of fluents these are ground, functionless atoms in addition, we'll use Database semantics 1. the Close World Assumption

fluents not explicitly mentioned are false 2. the Unique Names Assumption

different ground terms are different objects: Plane1, Plane2 this state representation allows alternative algorithms

it can be manipulated either by logical inference techniques or by set operations

actions are defined by a set of action schemas these implicitly define the ACTIONS(s) & RESULT(s, a)

functions required to apply search techniques

Classical Planning

7

Action Schemas

PDDL & the Frame Problem recall the representational issue of capturing what stays the

same given some action in PDDL we specify what changes, and if something is not

mentioned, it stays the same Action Schemas

are a lifted representation (recall Generalized Modus Ponens) lifts from propositional logic to a restricted subset of FOL each one stands for a set of variable-free actions each includes the schema name, list of variables used,

preconditions & effects we consider variables as universally quantified, choose values

to instantiate them PRECOND: defines states in which an action can be executed EFFECT: defines the result of executing the action

Classical Planning

8

Action Schemas

each represents a set of variable-free actions form: Action Schema = predicate + preconditions + effects example:

Action(Fly(p, from, to),PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬AT(p, from) At(p, to))

an action schema in which (p, from, to) need to be instantiated the action name and its parameter list its preconditions

a conjunction of function-free literals its effects

a conjunction of function-free literals

Classical Planning

9

Applying Action Schemas

action a is applicable in state s if its preconditions are satisfied in s

its preconditions are entailed by s given variables, there can be multiple applicable

instantiations for v variables in a domain with k unique object names, worst

case time to find applicable ground actions is O(vk) leads to 1 approach for solving PDDL planning problems

propositionalize by replacing action schemas with sets of ground actions & applying a propositional solver like SATPlan

impractical for large v & k

result of executing a in s is s' s' is derived from s & action a's EFFECTs remove -ve literal fluents (the delete list, DEL(a)) add +ve literals in EFFECTs (the add list, ADD(a)) RESULT(s, a) = (s - DEL(a)) ADD(a)

Classical Planning

10

Action Schemas

PDDL schemas 1. variables & ground terms

variables in effects must also be in precondition so matching to state s yields results with all variables bound i.e. that contain only ground terms

2. handling of time no explicit time terms, unlike axioms we saw for logical agents instead time is implicitly represented in PDDL schemas preconditions always refer to time: t effects always refer to time: t + 1

3. a set of schemas defines a planning domain a specific problem adds initial & goal states

Classical Planning

11

Initial States, Goals, Solutions

initial state conjunction of ground terms

goal conjunction of +ve & -ve literals both ground terms & those containing variables variables are treated as existentially quantified

solution a sequence of actions ending in s that entails the goal example: Plane(P1) At (P1, SFO) entails At(p, SFO) Plane (p)

defines planning as a search problem

Classical Planning

12

Some Language Semantics

How do actions affect states? an action is applicable in any state that satisfies its

precondition(s) applicability involves a substitution for the variables in the

PRECOND example

State:At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK)

Airport(SFO)

Satisfies PRECOND of Fly action:At(p, from) Plane(p) Airport(from) Airport(to)

With substitution ={p/P1, from/JFK, to/SFO}Thus the action is applicable.

Classical Planning

13

Language Semantics

result of executing action a in state s is the state s’ s’ is same as s except

any positive literal P in the effect of a is added to s’ any negative literal ¬P is removed from s’ example

Fly(p, from, to)EFFECT: ¬AT(p, from) At(p, to), ={p/P1, from/JFK, to/SFO}

s: At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)s': At(P1,SFO) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK)

Airport(SFO)

Classical Planning

14

Example: air cargo transport

simple actions to illustrate moving cargo between New York(JFK) & San Francisco(SFO)

Init(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO))

Goal(At(C1,JFK) At(C2,SFO))

Action(Load(c, p, a)PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: ¬At(c, a) In(c, p))

Action(Unload(c, p, a)PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: At(c, a) ¬In(c, p))

Action(Fly(p, from, to)PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬ At(p, from) At(p, to))

Classical Planning

15

Air Cargo Example

this planning domain involves 3 actions and 2 affects predicates, In(c, p) & At(x, a) properly maintaining At predicates could be an issue

where is cargo that is in a plane? without quantifiers, problem of expressing what happens to cargo

that is in a plane solve by treating At predicate as applying only to cargo after it is

unloaded, in effect not At anywhere when it is in a plane another issue is that this representation allows "empty"

actions that produce contradictory effects Fly(P1, JFK, JFK) yields At(P1, JFK) ¬At(P1, JFK) can ignore issues like this, rarely produce incorrect plans, or add inequality precondition: from to

Classical Planning

16

Air Cargo Example

the solution is pretty obviousInit(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2)

Plane(P1) Plane(P2) Airport(JFK) Airport(SFO))Goal(At(C1,JFK) At(C2,SFO))

Action(Load(c, p, a)PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: ¬At(c, a) In(c, p))

Action(Unload(c, p, a)PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)EFFECT: At(c, a) ¬In(c, p))

Action(Fly(p, from, to)PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)EFFECT: ¬ At(p, from) At(p, to))

[Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK), Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)]

Classical Planning

17

Example: flat tire problem

simple actions to illustrate changing a flat tire (in a bad neighbourhood)

Init(At(Flat, Axle) At(Spare, Trunk))Goal(At(Spare, Axle))

Action(Remove(Spare, Trunk)PRECOND: At(Spare, Trunk)EFFECT: ¬At(Spare, Trunk) At(Spare, Ground))

Action(Remove(Flat, Axle)PRECOND: At(Flat, Axle)EFFECT: ¬At(Flat, Axle) At(Flat, Ground))

Action(PutOn(Spare, Axle)PRECOND: At(Spare, Ground) ¬At(Flat, Axle)EFFECT: At(Spare, Axle) ¬At(Spare, Ground))

Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk) ¬ At(Flat, Ground) ¬ At(Flat, Axle) )

Classical Planning

18

Flat Tire example

highly simplified, abstracted version of the problem of constructing a plan to fix a flat tire 4 actions note capturing of "bad neighbourhood" as tires "disappearing"

if the car is left overnight the simple solution

[Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)]

Classical Planning

19

Example: Blocks World

simple actions to illustrate building a tower in Blocks World

Init(On(A, Table) On(B, Table) On(C,A) Block(A) Block(B) Block(C) Clear(B) Clear(C))

Goal(On(A,B) On(B,C))

Action(Move(b, x, y)PRECOND: On(b, x) Clear(b) Clear(y) Block(b) (b x) (b y) (x y) EFFECT: On(b, y) Clear(x) ¬ On(b, x) ¬ Clear(y))

Action(MoveToTable(b, x)PRECOND: On(b, x) Clear(b) Block(b) (b x) EFFECT: On(b, Table) Clear(x) ¬ On(b, x))

simple plan given the initial state note PRECOND of Move uses inequality to prevent empty moves Move(B,C,C)

Classical Planning

20

Blocks World example

illustration of the problem

Classical Planning

21

Blocks World example

problem domain: stack cube-shaped blocks that are on a table

Notes: only 1 block can fit on top of another moves can be onto another block or onto the table FOL would use quantifiers to express

there's no block on top of some other block without quantifiers, PDDL requires Clear(x) predicate

Move action schema must be complemented by MoveToTable to avoid errors with the Clear predicate

since Clear(Table) is always true residual problem if bind y to Table in Move(b, x, y)

search space grows, though answers still correct fix with a Block(m) predicate & adding Block(b) Block(y) to the

preconditions for Move

Classical Planning

22

Complexity & Classical Planning

first, we distinguish 2 versions of the planning problems PlanSAT:

is there a plan that solves the problem? Bounded PlanSAT:

is there a solution of k or fewer steps? this can be used to find optimal plans

so long as we don't allow functions the number of states is finite & both categories are decidable

if we allow functions, PlanSAT becomes semi-decidable (may not terminate on unsolvable problems) though Bounded PlanSAT remains decidable even with functions

23

Complexity & Classical Planning

sadly, in general, both problems are NP-hard putting restrictions on the expressiveness may help for example, disallowing -ve preconditions lets PlanSAT reduce

to P fortunately for planners, many useful problems are easier than

the worst case for blocks world & air cargo domains, Bounded PlanSAT is NP-

Complete, while PlanSAT is in P! so, optimal planning is hard, sometimes sub-optimal planning is

easy reminder of complexity category relationships

24

Planning with state-space search

given the problem formulation we can use search algorithms to generate plans

systematic heuristic search local search (while recording trail of states)

search can be in either the forward or the backward direction

recall the bi-directional example from search topic problems related to non-invertible successor functions

Progression planners do a forward state-space search consider the effects of all possible actions in a given state

Regression planners do a backward state-space search to achieve a goal

consider what must have been true in the previous state from effects through actions to preconditions

Classical Planning

25

Progression & Regression

partial samples: plane moving example

Classical Planning

26

Progression algorithm

formulation as a state-space search problem Initial state = initial state of the planning problem

literals not appearing are false Actions = apply those whose preconditions are satisfied

add positive effects, delete negative Goal test = does the state satisfy the goal Step cost = for simplicity, each action costs 1 recall, functions are not permitted in the representation

any graph search that is complete is a complete planning algorithm

for example, A*

Classical Planning

27

Progression algorithm

Progression approach while complete is inefficient

any applicable action is a possible next step typically its problems are related to branching factor

(1) irrelevant actions just because the preconditions of an action are satisfied does not mean that it should be done

(2) requirement for a good heuristic in order for improved efficiency of search

Classical Planning

28

Progression: issues

illustrate with the air cargo example assume 10 airports, 5 planes & 20 pieces of cargo at each problem: move all cargo at airport A to airport B solution is pretty simple, but uninformed search has serious

complexity difficulties just considering planes: 50 x 9 = 450 possible flights suppose all planes & packages were at 1 airport: 10450 possible

actions (200 x 50 + 50 x 9) if 2000 actions available on average, 200041 nodes in search

graph fortunately, good (domain-independent) heuristics are

possible, & can be derived automatically

Classical Planning

29

Regression algorithm

what about Regression planners issues:

How do we determine predecessors? as we saw for bi-directional search, the problem may not allow

predecessor determination (n-Queens) fortunately PDDL formulation facilitates it terminology: predecessor is g', action a, goal g g' = (g - Add(a)) Precond(a), since any effects added by a might

not have been true previously & preconditions must have been met or could not execute a

this process requires partially uninstantiated actions & states use standardized variable names (avoid clashes, retain generality

of variables) implicitly quantify over these so 1 description summarizes

possibility of using any appropriate ground term

Classical Planning

30

Regression: relevant actions

which action? forward uses applicable actions (their preconditions are met) backward uses relevant actions (they could be the last step

leading to current state)

relevant: at least 1 effect (+ve or -ve) must unify with a term in the (current) goal & there can't be any effect that negates a term in the goal

process: use the action formed by substituting the most general unifier into the standardized action schema

more formal definition of relevant actions if goal g contains literal gi & action schema A (standardized to

produce A') has effect literal ej' where Unify (gi, ej') = & a' = SUBST(, A') & there's no effect in a' that is the negation of a literal in g, then a' is a relevant action toward g

terminates when a predecessor is satisfied by the initial state

Classical Planning

31

Progression & Regression

status of the regression approach 1. there's a lower branching factor in the backward direction,

the result of only considering relevant actions 2. but each state is really a set of states

the state defined by true or false for the ground fluents, and the states not mentioned (see mid p 374)

this representation of stages in the backward search as sets of states complicates the process of developing heuristics

progression versus regression the greater availability of accurate heuristics for forward

search has resulted in it being used in many more planning systems

Classical Planning

32

Heuristics for planning

the complexity of planning problems means that we need good heuristics below: 2 general approaches to finding admissible heuristics

recall that an admissible h(n) (i.e. one that never overestimates cost) allows A* search for optimal solutions

recall from our (informed search) discussion of finding heuristics

as the optimal solution to a relaxed version of the problem from problem decomposition

if we can make the subgoal independence assumption we can approximate the cost of solving a conjunction of subgoals as the sum of the costs of solving the corresponding subproblems

(1) relaxation: search problem viewed as a graph allows (a) adding edges to make a path to a solution easier to find (b) merging states in an abstraction of the original problem so

searching is in a space with fewer nodes

Classical Planning

33

Heuristics by problem relaxation

(a) adding edges to the search space (1) ignore all preconditions

assume all actions are applicable in all states, so any goal fluent takes just a single step

apparently #steps ≈ #unsatisfied goals, but note: some actions may achieve > 1 goal, & others may undo goals ignoring the latter, h(n) = min # actions for which the union of their

effects satisfies the goal this is a version of the "set-cover problem", known to be NP-hard! a greedy algorithm yields set covering within log n of the true min (n

is # literals in goal), but is not guaranteed admissible

(2) ignore selected PRECONDs of actions as in the N-puzzle heuristics: misplaced tiles, manhattan distance the factored representation in action schemata for planning

problems allows automated derivation of such heuristics (3) ignore delete lists

if all goals & preconditions have only +ve literals (true for many problems anyway & others can be converted to this form)

Classical Planning

34


(a) adding edges to search space (3) ignore delete lists

with only +ve literals in all goals & preconditions then remove -ve literals from effects now steps are monotonic progress towards the goal w/o undoing but, it's still NP-hard to find an optimal solution, though it can be

approximated in polynomial time with hill climbing summary

adding edges relaxes a planning problem to yield heuristics but they're still expensive to calculate

(b) reduce nodes by abstraction to merge states return to the air cargo example

10 airports, 50 planes, 200 packages means the general case has 5010 x 20050+10 or 10155 states

a specific problem: say all pkgs are at 5 airports & all pkgs at any airport have 1 destination

35


(b) reduce nodes by abstraction to merge states in the air cargo example, in general, 10155 states

a specific problem has all pkgs at 5 airports & pkgs at any airport have 1 destination

an abstraction of this version omits At fluents except those re: 1 plane & 1 pkg at each of 5 airports, reducing # states to 510 x 55+10 or 1017 states

so a solution is shorter & an admissible h(n) & is extensible to the original problem by adding more Load & Unload actions

problem decomposition approaches for h(n)'s the relevant pattern is:

subdivide problem solve subproblems combine solutions the subgoal independence assumption

is useful in developing a heuristic as a sum of subgoal costs is too optimistic if there are -ve interactions between subplans is too pessimistic (inadmissible) when there are redundant actions

in subplans

Classical Planning

36

Heuristics by decomposition

problem decomposition approaches for h(n)'s notation: represent the goal as a set of fluents G, where

disjoint subsets of G are G1, …Gn, for which we find plans P1, … Pn & then use their costs to estimate cost for G

Cost(Pi) is an estimate, a possible heuristic

maxi Cost(Pi) is admissible, overly optimistic

sumi Cost(Pi) is not generally admissible unless all Gi & Gj are independent (Pi does not affect preconds or goals of Pj) in which case the sum is admissible & more accurate

37

Heuristics by decomposition

the challenge of abstraction is to find one that has a significantly lower total cost (defining the

abstraction + doing the abstracted search + mapping back to original problem) than the original problem

the pattern database techniques may be useful recall them from informed search discussion of heuristics amortize the cost of building the database over multiple problem

solution instances

the FF (FASTFORWARD) planning system is a hybrid it uses heuristics from a planning graph (our next discussion) it uses local search (storing plan steps) at a plateau or local maximum it does iterative deepening

systematic search for a better state or gives up & restarts

Classical Planning

38

Planning Graphs

Planning Graphs are an alternative intermediate representation/data structure

they are polynomial complexity approximations of full (exponential) trees of all states & actions

they can't answer the question: is G reachable? though they are correct when they say it's not reachable they provide an estimate of the number of steps to G any such estimate is optimistic, so is an admissible heuristic

planning graphs provide a possible basis for better search heuristics and they can also be used directly, for extracting a solution to a

planning problem, by applying the GRAPHPLAN algorithm

Classical Planning

39

Planning Graphs

a planning graph is a directed graph organized in levels the levels of nodes correspond to time steps in a plan

they consist of alternating S levels (in which the nodes are fluents that hold) & A levels (in which the nodes are actions that might be applicable)

level S0 is the initial state each level is a set of literals Si or a set of actions Ai

Si: all the literals that could be true at that time step depending on the actions executed at the previous steps

Ai: all actions that could have PRECONDs satisfied at that step depending on which of the literals actually hold

Classical Planning

40

Planning Graphs

explanation regarding could on the previous slide a planning graph only captures a restricted subset of the

possible -ve interactions so a literal might appear at a level earlier than it actually would

(if it would at all), though it never appears too late despite this error, the level j of the first appearance is a good

estimate of how difficult it is to achieve from the initial state refers to the approximate nature of the state/action lists in part, this allows efficient construction by recording a restricted

subset of possible -ve interactions among actions

note: planning graphs apply only for propositional problems action schemas for problems with variables can be

propositionalized their advantages may make this worthwhile, despite the

increased size of the problem description

41

Planning Graphs

our simple example: "have your cake & eat it too" the problem description

Init(Have(Cake))Goal(Have(Cake) Eaten(Cake))Action(Eat(Cake)

PRECOND: Have(Cake)EFFECT: ¬Have(Cake) Eaten(Cake))

Action(Bake(Cake)PRECOND: ¬ Have(Cake)EFFECT: Have(Cake))

the corresponding planning graph

42Planning Graph Cake Example

start at level S0, determine action level A0 & next level S1

A0: all actions whose preconditions are satisfied in the previous level

(initial state) actions are shown in rectangular boxes lines connect PRECONDs at S0 to EFFECTs at S1

also, for each literal in Si, there's a persistence action (square box) & line

to it in the next level Si+1

level A0 contains the actions that could occur conflicts between actions are represented by arcs: mutual exclusion or

mutex links


level S1 contains all the literals that could result from picking any subset of actions in A0

so S1 is a belief state consisting of the set of all possible states each is a subset of literals with no mutex links between members

conflicts between literals that cannot occur together are represented by the mutex links.

the level generation process is repeated eventually consecutive levels are identical: leveling off

Classical Planning


mutex relation holds between 2 actions at a level when 1. inconsistent effects

one action negates the effect of another 2. interference

an effect of one action negates a precondition of the other 3. competing needs

a precondition of one action is mutex with a precondition of the other

Classical Planning


mutex relation holds between 2 literals at a level when 1. one is the negation of the other 2. if each possible action pair that could achieve the literals is

mutex (Have(Cake) & Eaten(Cake) at S1) this is termed inconsistent support

Classical Planning

46

Planning Graphs & Heuristics

Planning Graphs construction has complexity polynomial in the size of the

planning problem given l literals, a actions, & a PG of n levels: O(n(a + l)2)

the completed PG provides information about the problem & candidate heuristics 1. a goal literal g that does not appear in the final level cannot be

achieved by any plan 2. the level cost, the level at which a goal literal first appears, is

useful as a cost estimate of achieving that goal literal note that level cost is admissible, though possibly inaccurate

since it counts levels, not actions we could find a better alternative level cost by using a serial

planning graph variation, restricted to one action per level mutex links between every pair of actions except persistence actions

Classical Planning

47


Planning Graph provides possible heuristics for the cost of a conjunction of goals 1. max-level: highest level of any conjunct in the goal

admissible, possibly not accurate 2. level sum: the sum of level costs of conjuncts in the goal

incorporates the subgoal independence assumption so may be inadmissible to degree the assumption does not hold

3. set-level: level where all goal conjuncts are present without mutex links

this one dominates the max level heuristic & is good where there are interactions among subplans

Classical Planning

48


a Planning Graph is a relaxed version of the problem if a goal literal g does not appear, no plan can achieve it, but

if it does appear, is not guaranteed to be achievable why?

the PG only captures pairwise conflicts & there could be higher order conflicts

likely not worth the computational expense of checking for them similar to Constraint Satisfaction Problems where arc consistency

was a valuable pruning tool 3-consistency or even higher order consistency would have made

finding solutions easier but was not worth the additional work an example where PG fails to indicate unsolvable problem

blocks world problem with goal of A on B, B on C, C on A any pair of subgoals are achievable, so no mutexes problem only fails at stage of searching the PG

Classical Planning

49

The GRAPHPLAN Algorithm

the GRAPHPLAN algorithm generates the Planning Graph & extracts a solution from it

function GRAPHPLAN(problem) return solution or failuregraph INITIAL-PLANNING-GRAPH(problem)goals CONJUNCTS(problem. GOAL)nogoods an empty hash tablefor tl = 0 to do

if goals all non-mutex in St of graph then

solution EXTRACT-SOLUTION(graph, goals,NUMLEVELS(graph), nogoods)

if solution failure then return solutionif graph and nogoods have both leveled off then return failuregraph EXPAND-GRAPH(graph, problem)

Classical Planning

50

Example: Spare Tire Problem

recall the spare tire problemInit(At(Flat, Axle) At(Spare, Trunk))

Goal(At(Spare, Axle))

Action(Remove(Spare, Trunk)PRECOND: At(Spare, Trunk)EFFECT: ¬At(Spare, Trunk) At(Spare, Ground))

Action(Remove(Flat, Axle)PRECOND: At(Flat, Axle)EFFECT: ¬At(Flat, Axle) At(Flat, Ground))

Action(PutOn(Spare, Axle)PRECOND: At(Spare, Ground) ¬At(Flat, Axle)EFFECT: At(Spare, Axle) ¬At(Spare, Ground))

Action(LeaveOvernightPRECOND:EFFECT: ¬ At(Spare, Ground) ¬ At(Spare, Axle) ¬ At(Spare, Trunk)

¬ At(Flat, Ground) ¬ At(Flat, Axle) )

Classical Planning

51GRAPHPLAN Spare Tire Example

Notes: this figure shows the complete Planning Graph for the problem arcs show mutex relations

but arcs between literals are omitted to avoid clutter it also omits unchanging +ve literals (for example, Tire(Spare))

& irrelevant -ve literals bold boxes & links indicate the solution plan

Classical Planning


S0 is initialized to 5 literals from the problem initial state and the CWA literals

no goal literal in S0 so EXPAND-GRAPH add actions those with preconditions satisfied in S0

also adds persistence actions for literals in S0

adds the effects at level S1, analyzes & adds mutex relations repeat until the goal is in level Si or failure


EXPAND-GRAPH adds constraints: mutex relations inconsistent effects (action x vs action y)

Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare, Ground)

interference (effect negates a precondition) Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND & ¬At(Flat,

Axle) as EFFECT competing needs (mutex preconditions)

PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle) inconsistent support (actions to produce literals are mutex)

in S2, At(Spare, Axle) & At(Flat, Axle)

Classical Planning

54

Reminder: GRAPHPLAN Algorithm

the GRAPHPLAN algorithm both generates the Planning Graph & extracts a solution from it

function GRAPHPLAN(problem) return solution or failuregraph INITIAL-PLANNING-GRAPH(problem)goals CONJUNCTS(problem. GOAL)nogoods an empty hash tablefor tl = 0 to do

if goals all non-mutex in St of graph then

solution EXTRACT-SOLUTION(graph, goals,NUMLEVELS(graph), nogoods)

if solution failure then return solutionif graph and nogoods have both leveled off then return failuregraph EXPAND-GRAPH(graph, problem)


in S2, the goal literals exist, & are not mutex just 1 goal literal so obviously not mutex with any other goal

since a solution may exist, EXTRACT-SOLUTION tries to find it

EXTRACT-SOLUTION may search backwards for a solution initial state: last level of the PG + goals of the planning problem actions from Si

select a set of conflict-free actions in Ai-1 with effects covering the goals conflict free = no 2 actions are mutex & no pair of preconditions are mutex

goal: reach a state at level S0 such that all goals are satisfied cost: 1 for each action

56

GRAPHPLAN Solutions

if EXTRACT-SOLUTION fails at that point it records (level, goals) as a "no-good" subsequent calls can fail immediately if they require the same

goals at that level

we already know planning problems are computationally hard require good heuristics

greedy search with level cost of literals as a heuristic works well

1. pick literal with highest level cost 2. to achieve it, pick actions with easier preconds

action with smallest sum (or max) of level costs for its preconds

Classical Planning

57

GRAPHPLAN Solutions

alternative to backward search for a solution EXTRACT-SOLUTION could formulate a Boolean CSP

variables are actions at each level values are Boolean: an action is either in or out of the plan constraints are mutex relations & the need to satisfy each goal &

precondition

58

GRAPHPLAN Termination

does GRAPHPLAN terminate? does it stop & return failure if there's no solution the general rationale is presented here for some proof details see the textbook & its Ghallab reference

background recall that levelled off means consecutive PG levels are

identical now note that a graph may level off before a solution can be

found, on a problem for which there is a solution example of 2 airports, 1 plane & n pieces of cargo at one of

them, plan required to move all n to the other airport but only 1 piece at a time fits in the plane

graph levels off at level 4, from which full solution can't be extracted (that would require 4n - 1 steps)

we need to take account of the no-goods (goals that were not achievable) as well

these may decline even after the graph levels off

59


does GRAPHPLAN terminate? given that no-goods might continue to decrease after the graph

levels off we now require a proof that both the graph & no-goods will

level off literals increase monotonically

once a literal appears, its persistence action causes it to stay actions increase monotonically

once a literal that is a precondition appears, the action stays mutexes decrease monotonically

though the graph simplifying conventions may not show it if 2 actions are mutex at Ai, they are also mutex at all previous levels

where they appear

60


does GRAPHPLAN terminate? no-goods decrease monotonically

if a set of goals is not achievable at level i, they are not achievable at any previous level

so literals & actions increase monotonically given a finite number of actions and literals, they will level off

and mutexes & no-goods decrease monotonically since there cannot be fewer than 0, they must level off

if the graph & no-goods have levelled off & a goal is missing or is mutex with another goal, the algorithm can stop & return failure

Classical Planning

61

Alternative Planning Approaches

the following slides outline brief discussions of 4 alternative approaches to solving planning problems 1. convert the PDDL description into a form solvable by a

propositional logic satisfiability algorithm: SATPlan 2. convert into a situational calculus representation solvable

by First-Order Logic inference techniques 3. represent & solve as a Constraint Satisfaction Problem 4. use a "plan refinement" approach to searching in a space of

partially ordered plans

Classical Planning

62

Planning with Propositional Logic

1. form a propositional logic satisfiability problem for submission to SATPlan iteratively tries plans of increasing length (increasing number

time steps) up to the Tmax parameter value, so finds the shortest plan, if one exists

the SATPlan algorithm finds models for a (very long) PL sentence that includes initial

state, goal, successor-state axioms, precondition axioms, & action-exclusion axioms

assigns true to the actions that are part of the correct plan & false to the others

if the planning is unsolvable the sentence will be unsatisfiable any model satisfying the sentence will be a valid plan an assignment that corresponds to an incorrect plan will not be a

model because of inconsistency with the assertion that the goal is true

Classical Planning

63


translation from PDDL to PL form for SATPlan the process involves 6 steps

step 1: propositionalize actions, replacing each schema with a set of ground actions (constants substituted for variables)

step 2: define an initial state asserting F0 for all fluents in the initial state, ¬F0 for all fluents not in the initial state

step 3: propositionalize the goal - for each variable replace literals containing it with a disjunction over constants

step 4: include successor-state axioms for each fluentFt+1 ActionCausesFt (Ft ¬ActionCausesNotFt), where

ActionCausesF is disjunction of all ground actions with F in their add List

ActionCausesNotF is disjunction of all ground actions with F in their delete list

air cargo example: axioms are required for each plane, airport and time step!

Classical Planning

64


the 6 steps from PDDL to SATPlan format step 5: add precondition axioms (for every ground action A,

add At PRE(A)t

step 6: add action exclusion axioms to say every action is distinct from every other action

submit to SATPlan

Classical Planning

65

The SATPLAN Algorithm

the algorithm note that in AIMA 3e, this is first presented in the earlier

section on propositional logic agents

function SATPLAN(init, transition, goal, Tmax) returns solution or failure

inputs: init, transition, goal form a description of the planning problem Tmax, an upper limit to the plan length

for t = 0 to Tmax do

cnf TRANSLATE-TO-SAT(init, transition, goal, t )model SAT-SOLVER(cnf)if model is not null then return EXTRACT-SOLUTION(model)

return failure

Classical Planning

66


some explanatory notes distinct propositions for assertions about each time step

superscripts denote the time step: At(P1,SFO)0 At(P2,JFK)0

PDDL includes the Closed World Assumption, so conversion involves the need to specify which propositions are not true

¬At(P1,JFK)0 ¬At(P2,SFO)0

unknown propositions are left unspecified goal is associated with some particular time-step: which one?

Classical Planning

67


some explanatory notes we can't know in advance how many steps are required to

achieve a goal so, to determine the time step T where the goal will be reached, the algorithm loops until success or some arbitrary time step limit

repeat start at T = 0

Assert At(P1,JFK)0 At(P2,SFO)0

on failure: try T = 1 Assert At(P1,JFK)1 At(P2,SFO)1

add terms for next time step

until a solution is found or some max path length Tmax is exceeded: setting Tmax assures termination

Classical Planning

68

Models

the pre-condition axioms avoid models with illegal actions

e.g. for T = 1 Fly(P1, SFO, JFK)0 Fly(P1, JFK, SFO)0 Fly(P2, JFK, SFO)0

the second action is infeasible, precluded if we include the appropriate pre-condition axiom: Fly(P1,JFK,SFO)0 At(P1,JFK)0

since At(P1, JFK)0 false in initial state, Fly(P1, JFK, SFO)0 false in any model

Classical Planning

69

Models

without action-exclusion axioms a model might include actions, for example having 1 plane fly

to two destinations in a time step Fly(P1, SFO, JFK)0 Fly(P2, JFK, SFO)0 Fly(P2, JFK, LAX)0

though the second P2 action is infeasible, it requires the added action-exclusion axioms to avoid such solutions

¬(Fly(P2,JFK,SFO)0 Fly(P2,JFK,LAX)0)

Classical Planning

70

Planning in First-Order Logic

alternative 2: convert to the Situation Calculus format of FOL & apply FOL inferencing recall Situation Calculus as a topic in the 352 Knowledge

Representation discussion of time & action PDDL omits quantifiers which limits expressiveness but helps

control complexity of algorithms applied to it PL representation also has limitations like the requirement

that a "timestamp" becomes part of each fluent Situation Calculus replaces explicit timestamps by using

situations that correspond to a sequence or history of actions 1. situations

initial state is a situation if s is a situation & a an action, RESULT(s, a) is also a situation situations correspond to sequences or history of actions 2 situations are the same only if their start & actions are the

same

71


convert to FOL Situation Calculus 2. fluents are functions or relations that vary over situations

syntax: situation is conventionally the last argument At(x, l, s) is a relational fluent, true when object x is at location l in

situation s & Location(x, s) is a functional fluent such that Location(x, s) = l in the same situation

3. a possibility axiom describes preconditions of each action (s) Poss(a, s) indicates when an action can be taken is a formula giving preconditions Alive(Agent, s) Have(Agent, Arrow, s) Poss(Shoot, s)

4. successor-state axioms for each fluent indicate what happens, depending on the action taken

here are the general form & an example from Wumpus WorldAction is possible (Fluent is true in result state Action's effect made it true VIt was true before & the action left it unchanged)Poss(a, s) (At(Agent, y, Result(a, s)) a = Go(x, y) V (At(Agent, y, s) Λ a Go(y, z)))

72


express problem using FOL Situation Calculus 5. unique action axioms required to allow deductions such as

a Go(y, z) for each pair of action names Ai & Aj there's an axiom to say they

are different: Ai(x, …) Aj(y, …)

plus, for each action name Aj an axiom indicates 2 uses are only equal if all arguments are equal

Ai (x1, …, xn) = Ai(y1, …, yn) x1 = y1 … xn = yn

6. the solution is a situation (action sequence) that satisfies the goal

unfortunately, no major planning systems use situation calculus basic efficiency issues of FOL inference compounded by lack of good heuristics for planning with SC

Classical Planning

73

Planning: Constraint Satisfaction

we noticed in discussions in 352 the similarities between Constraint Satisfaction Problems &

boolean satisfiability conversion to a CSP is similar to that of conversion to a

propositional logic satisfiability problem one difference is that it only requires a single variable Actiont

at each time step: its domain is the set of all possible actions thus we also don't need the action exclusion axioms

Classical Planning

74

Partial-order planning

plans may be developed through refinement of partially ordered plans approaches describe so far, including both Progression and

Regression planning, are totally ordered forms of plan search they cannot take advantage of problem decomposition decisions must be made on how to sequence the actions for all the

subproblems though some are independent (in the air cargo problem, loading cargo at 2 different airports)

a least commitment strategy would, where possible, delay choices during search

partially ordered planning implements this adds actions to plans without committing to an absolute time/step,

instead dealing mainly with relative constraints action A must precede action B

partially ordered plans are generated by a search in the space of plans rather than through the state space

so the planning operators are different from the real-world actions

Classical Planning

75

A plan to put your shoes on

an illustration from 2e notes

Goal(RightShoeOn LeftShoeOn)Init()

Action(RightShoe,PRECOND: RightSockOnEFFECT: RightShoeOn)

Action(RightSock,PRECOND: EFFECT: RightSockOn)

Action(LeftShoe,PRECOND: LeftSockOnEFFECT: LeftShoeOn)

Action(LeftSock,PRECOND: EFFECT: LeftSockOn)

Classical Planning

76

A plan to put your shoes on

a partially-ordered planner can place two actions into a plan without specifying which comes first sample POP and corresponding totally ordered plans

77

POP as a search problem

states are (mostly unfinished) plans the empty plan contains only the special start & finish actions

each state (plan) is a partially ordered structure it consists of a set of actions (the steps of the plan) and a set of

ordering constraints of the form Before(ai, aj) meaning that one action occurs before another

graphically, actions are shown as boxes, ordering constraints as arrows

the starting point is an empty plan consisting of the initial state & goal, with no actions between them

the search process adds an action to correct a flaw in the plan or if it can't, it backtracks

flaws keep a partial plan from being a solution as an example, the next slide shows POPs corresponding to the

"flat tire" planning problem, both the empty plan and a solution

78

POP for the flat tire problem

at the top the empty plan & below it, the solution plan Remove(spare, trunk), Remove(flat, axle) can be done in either

order, so long as both are done before PutOn(spare, axle)

Classical Planning

79

POP for the flat tire problem

search in plan space seeks to add to the plan to correct a flaw flaw: not achieving At(spare, axle) in the flat tire empty plan adding PutOn(spare, axle) fixes that, but introduces new flaws

in the form of its (unachieved) preconditions process repeats, backtracking as necessary each step makes the least commitment (ordering constraints

& variable bindings) in order to fix a flaw Remove(spare, trunk) must be before PutOn(spare, axle) but

no other ordering requirements

Classical Planning

80

Partial-Order Planning

was previously popular because of explicitly representing independent subproblems but it lacks an explicit representation of states in the state-

transition model & was superseded by progression planners with heuristics that allowed discovery of independent subproblems

nevertheless it's useful for tasks like operations scheduling, enhanced with domain specific heuristics

also appropriate where humans must understand the resulting plans

plan refinement approach is easier to understand & verify example: spacecraft/rover plans are generated by this approach

Classical Planning

81

Solving POP

POP in action the initial plan contains

only the Start & Finish steps Start step/action has the initial state description as its effect Finish step/action has the goal description as its precondition

the single ordering constraint Before (Start, Finish) there's a set of open preconditions: all those in Finish

the successor function picks an open precondition p on an action B and generates a

successor plan using an action A that achieves p then does goal test

any remaining open preconditions?

Classical Planning

82

Process summary

operators on partial plans may add link from an existing action to an open precondition add a step (a new action) to fulfill an open condition order one step wrt another to remove possible conflicts

plan generation involves gradually moving from incomplete/vague plans to complete/correct plans

plan construction may require backtracking if an open condition is unachievable or conflict is

unresolvable

83

Classical Planning Summary

approaches rely on techniques of search or logic search to assemble a plan or to constructively prove one exists

handling the inherent complexity of these problems is a key identifying independent subproblems provides potential for

exponential speedup, but full decomposability is inconsistent with -ve interactions between actions

heuristically guided search can gain efficiency even if subproblems are not completely independent

some problems allow efficient solution by ruling out -ve interactions serializable subgoals = there's a subgoal ordering for a solution

that avoids having to undo any subgoals tower problems in Blocks World are serializable if subgoals are

ordered bottom to top

Classical Planning

84


serializable subgoals tower problems in Blocks World are serializable if subgoals are

ordered bottom to top NASA spacecraft control commands are serializable

(unsurprising since the control systems are designed for easy control)

when problem is serializable, a planner can seriously prune the planning search

what's next? where does further progress come from? probably not simply factored representations & propositional

approaches combine efficient heuristics + first-order, hierarchical

representations

Classical Planning

85


PDDL (Planning Domain Definition Language) initial & goal states as conjunctions of literals actions as lists of preconditions & effects

search in state space may be forward (Progression) or backward (Regression) derive heuristics from subgoal independence assumption &

relaxed versions of the problems Planning Graphs

build from the initial state to the goal with alternating incremental sets of literals or actions representing possibilities for each time step, & encoding mutexes between inconsistent pairs of literals or actions

layers are supersets of literals/actions possible at that time step the basis of good heuristics for state space searches or can

yield a plan by application of the GRAPHPLAN algorithm

Classical Planning

86


possible alternative approaches FOL inference on a situation calculus knowledge base conversion to boolean satisfiability problem for SATPlan encoding as a Constraint Satisfaction Problem search backwards over a space of Partially Ordered Plans

what's best? there's no agreement on a single best representation or

algorithm we saw one hybrid that performs well a particular domain may provide a best fit to one particular

technique

CISC453 Winter 2010 Classical Planning AIMA3e Ch 10.

Documents

Transcript of CISC453 Winter 2010 Classical Planning AIMA3e Ch 10.