1

Chapter 15-16

Planning & Planning Methods

2

Planning Deciding upon a course of action before

acting. A plan is a representation of a course of

action. A finished plan is a linear or partially

ordered sequence of operators. Planning is a problem solving technique. Planning is reasoning about future events

in order to verify the existence of a reasonable series of actions to take in order to accomplish a goal.

3

Planning

There are three major benefits of planning:

1. Reducing search 2. Resolving goal conflicts 3. Providing a basis for error recovery.

4

Nonhierarchical Planning This is roughly the intuitive idea of

planning; it consists of finding a sequence of

operators to achieve each of the goals. The problem is that the planner does not

distinguish between important goals and less critical ones, and so it can waste considerable amounts of time finding solutions to non-critical parts of a plan, only to find it can't solve a critical part.

Examples include STRIPS, HACKER, WARPLAN.

5

Hierarchical Planning

Planning that use a hierarchy of abstractions of a plan to solve the problem.

The ground plan is the form that lists executable operations; levels above increase in abstractions (and simplicity).

The goal of hierarchical planners is to simplify the search and reasoning process by finding vague solutions at levels where the details are not computationally overwhelming, and then refine them.

6

Hierarchical Planning (II) Both non-hierarchical and hierarchical

planners generate hierarchical plans, but only hierarchical planners use a hierarchy representation.

There are two ways to form hierarchies. Plan abstraction means using higher-level

concepts as operators (e.g., in NOAH, NONLIN, SIPE). Space abstraction (e.g. in ABSTRIPS) forms equivalence classes of states by ignoring less-critical preconditions and subgoals.

7

Least Commitment Approach

A constructive approach in which no commitment is made unless it is necessary to do so.

Decisions are made only when the planner can determine they will not interfere with past or future decisions.

The goal is to avoid backtracking as much as possible.

This approach is used in nonlinear planners (e.g., NOAH, MOLGEN, SNLP).

8

Nonlinear Planning

Least commitment with respect to time. If the resulting plan is not linear, either

linearization for a single agent, or execution by multiple agents can be used.

Ordering decisions are only made when necessary.

Partial plans are partial orders on operators, which are left unordered until a conflict detected, at which point constraints are added.

9

Opportunistic Planning

Real-time planning in which operators are invoked when they are useful in the current state.

10

AI Planning

Introduction Logic-based planning Operator-based planning Planning Algorithms Case-based planning Reactive Approaches Scheduling vs. Planning Planning in AI Texts References & Resources

11

AI Planning Introduction

Planning = How do I get from here to there?

There have been various formulations that attempt to solve the planning problem:

12

Several Approaches

Logic-based approaches Operator-based approaches Time-based approaches Case-based approaches Constraint-based approaches Distributed Planning Reactive approaches

13

Logic-based Planning

Given the following: \alpha designating an initial state \Gamma designating a set of actions \rho designating a goal \Omega is a database of sentences

about the initial state

14

Logic-based Planning

Also sometimes categorized as change-based planning . This is best introduced by the following figure taken from Genesereth & Nilsson.

15

Logic-based Planning

The planner will try to generate a plan, \Gamma which,

when executed by the acting module or the executor when the system is in the state i satisfying the initial state description,

will result in the state g satisfying the goal state description.

16

This can then lead into a presentation/discussion of situation calculus .

Also, a good point to introduce various planning problems:

17

After you drive your 4-door car from point A to point Bwhat is its color? how many doors does it have? ...etc.

Frame Problem

18

Qualification Problem

If you turn the ignition key of your car the engine will start...

unless the battery is dead...

or it is out of gas...

or there is a potato blocking the tailpipe......etc.

19

Ramification Problem

If you drive your car from point A to point B, then as a resultyour car is now at B...so is its engine...its tires......etc.

20

Operator-based Planning Actions are represented as operators This approach, also called, the

STRIPS approach , utilizes various operator schema and plan representations .

The frame problem is solved by using the STRIPS assumption.

The major points to be presented in this context are:

21

Design of the operator schema Add-delete lists, procedural vs declarative representations (NONLIN vs NOAH), etc.

Design of plan representations Linear plans, non-linear plans, hierarchical (abstract) plans, partial-order plans, conditional plans, etc.

Planning Algorithms Planning as search, world-space vs. plan-space, partial-order planning, total-order planning, progression, goal-regression, etc. The computational complexity of planning.

Plan Critiquing Plan reformulation, repair, total-ordering, etc.

22

Planning Algorithms

Introduce planning as search . There are two approaches:

1. Searching a World Space

2. Searching a Plan Space

23

Searching a World Space

Each node in the graph denoted a state of the world.

Arcs in the graph correspond to the execution of a specific action.

The planning problem is to find a path from the initial state to the goal state.

There are two algorithms:

24

Algorithms 1 - Progression

An algorithm that searches for the goal state by searching through the states generated by actions that can be performed in the given state, starting from the initial state.

25

Algorithms 2 - Regression

An algorithm that searches backward from the goal state by finding actions whose effects satisfy one or more of the posted goals, and posting the chosen action's preconditions as goals ( goal regression).

26

Both the algorithms are sound (if a plan is returned, will it work?)

and complete (if a plan exists, does the algorithm

guarantees that it will find it?). In most situations regression is a

better strategy.

27

Second Approach

Searching a Plan Space Each node in the graph represents partial

plans. Arcs denote plan refinement operations.

One can search for a plan with a totally-ordered sequence of actions (total order planning),

or a plan with a partially ordered set of actions (partial order planning (POP)

or least commitment planning).

28

Partial Order Planning (POP):

A partial order plan has three components:

1. A set of actions: For example, {eat-breakfast, take-

shower, wake-up, go-to-work}.

29

2. A set of ordering constraints For example, {wake-up before eat-

breakfast, wake-up before take-shower, wake-up before go-to-work, take-shower before go-to-work}

30

3. A set of causal links: For example, wake-up ---awake---> eat-

breakfast is a link from the action, wake-up to the action eat-breakfast.

When the action wake-up is added to the plan, the above causal link is recorded, along with the ordering constraint [wake-up before eat-breakfast], because wake-up's effect that the individual is awake is a precondition of eat-breakfast.

Causal links help detect inconsistencies whenever a partial plan is refined.

31

Case-based Planning

Given a new problem, a goal, and a description of an initial state.

Look into the library of cases to recall a similar problem, with similar initial and goal states.

Modify the retrieved solution for the new problem.

The key is to find good similarity metrics.

32

Reactive Approaches Planning & Execution

Planners think , executors do . Predictability (thinking) vs. Reactivity

(doing) on-line vs. off-line planning

Classical planning is done off-line. The generated plan is then fed to the on-line execution module.

Closed vs. open-loopsReaction rules encode sense-act cycles.

33

Reactive Approaches – cont..

Triangle Tables Universal Plans Situated Automata Action Nets Reactive Action Packages Task Control Architectures

34

Scheduling vs. Planning

Planning is deciding what to do.

Scheduling is deciding when to do it.

35

36

Chapter 16 Contents (1)

STRIPS STRIPS Implementation Partial Order Planning The Principle of Least Commitment Propositional Planning SAT Planning

37

Chapter 16 Contents (2)

Planning Graphs GraphPlan ADL and PDDL Probabilistic Planning Dynamic World Planning Case-Based Planning Scheduling

38

STRIPS Fikes and Nilsson, 1971 Devised as an

improvement upon situation calculus and Green's formalism. STRIPS represents the world as a set of formulae in first-order logic.

Each state in the search space consists of a world model and set of goals to be achieved.

A resolution-based theorem prover used means-end analysis to search;

the theorem prover was also used to verify operator preconditions and establishing the validity of the goal formula.

A separate system, PLANEX, supervised execution.

39

STRIPS (1)

Stanford Research Institute Problem Solver. An operator based planning system. STRIPS uses wffs in FOPC to describe the

world. For example: STRIPS was designed to enable a planner to

devise plans for a robot which is to solve problems in the blocks world.

40

Interpretation of the Formula

If an object o, is in location x, where x is not the same location as y, then object o cannot be in location y.

NOT Operator

41

42

STRIPS (2) STRIPS defines operators as in the following rule

schemata: (Push(o, x, y) operator)Precondition: AT(r, x)

Λ AT(o, x)Delete: AT(r, x)

AT(o, x)Add: AT(r, y)

AT(o, y) The preconditions specify what must be true for the

operator to be applied. The delete and add lists specify the changes that will

take place after the operator is applied.

43

Interpretation of the Push(o, x, y)

To push object o from position x to position y, the robot and the object must both start out in position x.

As a result of this action, neither the robot nor the object will still be in position x;

Both will be in position y.

44

Operator Schema

The previous Push(o, x, y) operator defined an operator schema.

That also means that it defined a type of action, not an actual action.

A real action is an instance of the schema, such as:

Push(o1, (2,3), (1, 4)) is an instance of the schema Push(.. ) operator schema.

45

Goal State of STRIPS

The final element of STRIPS is the goal state,

Which is described by a wff, or a set of wffs.

Once the planner finds a way to reach the goal state, it has solved its problem and is ready to execute the plan.

46

STRIPS Implementation

STRIPS uses resolution and means-ends analysis to devise plans:

The goal is negated, and the rule schemata are instantiated with objects from the real world.

If the resolution fails, then the goal has been achieved.

Otherwise, a plan must be devised.

47

Example of STRIPS

The initial state of the world is shown bellow:

Our goal is to place block C on top of block A.

A

B

C

48

Example of STRIPS

We will use two predicates to describe the world:

On(x, y) – means that block x is on top of block y.

Clear(x) – means that block x has no block on top of it.

And we use t to represent the table. Therefore, On(a, t) – means that block a is

on top of the table

49

Therefore, our goal is: On(C, A) And, our start state is: On(A, t) On(B, t) On(C, t) Clear(B) Clear(A) Clear(t) In which Clear(t) is always true because we

assume that the table is large enough to hold at least three blocks at once.

50

One available operator schema MoveOnto(x, y)

It simply means “move object x from wherever it is, and place it on top of object y.”

The operator MoveOnto(x, y) then is defined as:

Precondition: On(x, z)ΛClear(x) ΛClear(y) Delete: On(x, y) Clear(y) Add: On(x, y) Clear(y)

51

An Additional Operator: MoveOntoTable(x)

Preconditions: On(x, y) Λ Clear(x) Delete: On(x, y) Add: On(x, t) Clear(y)

52

Build a Plan

There are a number of approaches can be used to build the plan:

Assume we adopt forward chaining We then need to construct a tree where the

root node represents the start state And other nodes represent other possible

states that can be obtained by applying operators.

53

Example

From the initial state, we can apply the following three operators:

MovOnto(A, B) MoveOnto(B, C) MoveOntoTable(B) Assume we choose to apply

MoveOntoTable(B) and it has the precondition:

On(B, y) Λ Clear(b), which is matched by instantiating y with C. which leads to:

54

Cont…from last slide Delete: On(B, C) Add: On(B, t) Clear(C) Our state description will be: On(A, t) On(B, t) On(C, t) Clear(B) Clear(A) Clear(C) Clear(t) and from this position, we can apply any

of the following operators:

55

MoveOnto(A, B) MoveOnto(A, C) MoveOnto(B, A) MoveOnto(B, C) MoveOnto(C, A) MoveOnto(C, B) By using the blind search, we could simply

try each of these and add a new node to the tree for each resulting state.

In fact, by applying MoveOnto(C, A), we have produced a sate that matches the goal state

Therefore, a suitable plan has been found.

56

Example of STRIPS

The initial state of the world is shown bellow:

Our goal is to place block C on top of block A.

A

B

C

57

The Sussman Anomaly

Sussman anomaly A famous instance of subgoal

interaction, where the solutions to two separate subgoals must be interleaved in order to solve them both.

No sequential ordering of the subgoal solutions will work.

58

The Sussman Anomaly

The initial state of the world is shown bellow:

Our goal now is to place block A on top of block B and place block C on top of block A.

A

B

C

59

The Sussman Anomaly

http://en.wikipedia.org/wiki/Sussman_Anomaly

60

Partial Order Planning

Consider the block problem bellow,

BA

DC

A

B

CD

61

Partial Order Planning (1)

A total order plan specifies the order in which all actions must be carried out.

A partial order plan can specify some actions in parallel – these actions can be carried out in any order relative to each other.

62

How do we fill the following flow-chart to realize the goal?

Start

Finish

63

Partial Order Planning (2)

A partial order plan can be implemented in one of several ways.

The partial order plan on the left is implemented in one of two ways, shown in the total order plans, center and right:

64

The Principle of Least Commitment

In building a plan there will be some variables and objects that can be ignored, as they are superfluous to the goal of the plan.

Some variables do not need to be instantiated – for example it is preferable, where possible, to use MoveOnto (a, y) than to use MoveOnto (a, b).

This is the Principle of Least Commitment.

65

Propositional Planning (1)

Any STRIPS plan can be expressed in propositional logic.

This will often involve increasing the number of variables.

66

Propositional Planning (2) Any STRIPS plan can be expressed in propositional

logic. This will often involve increasing the number of

variables. For example:Clear (x)On (x, y)

These predicates can be represented as propositions:

X1 is equivalent to Clear (A)

X2 is equivalent to Clear (B)

X3 is equivalent to On (A, B)

X4 is equivalent to On (B, A)

67

Propositional Planning (3) States can be represented as an assignment of

truth values to the propositions:

X1 Λ ¬X2 Λ X3 Λ ¬ X4

This state can be represented in STRIPS notation as:

Clear (A) Λ ¬Clear (B) Λ On (A, B) Λ ¬On (B, A) The following sentence represents all

states in which A is clear and B is not clear:

X1 Λ ¬X2

68

Propositional Planning (4) Actions can also be represented as the preconditions

and the results of the action. We use the notation ¬X1’ to indicate that X1’ is no

longer true after the action. Hence, an action might be:

X1 Λ X2 Λ ¬X3 Λ ¬X4 Λ X1’ Λ ¬X2’ Λ X3’ Λ ¬X4’ This action is MoveOnto (A, B). This is a simple example, but propositional planning

can lead to very complex expressions being used. The advantage of using propositional planning is that

automated systems can be built to manipulate the plans.

69

SAT Planning

The satisfiability problem, of determining whether a given propositional logic sentence is satisfiable or not, is NP-Complete.

A number of efficient methods have been developed for devising plans by determining the satisfiability of propositional logic expressions.

Methods are either systematic, which involve checking all possible assignments of truth values, or stochastic.

70

Planning Graphs (1)

Even-numbered levels represent states. Odd-numbered levels actions.

Level 0 contains the propositions that represent the start state. The arrows from level 0 to level 1 show how those propositions match the preconditions of the actions in level 1.

All possible actions are shown in the graph.

71

Planning Graphs (2)

The partial planning graph shown includes persistence actions (things which do not change) as lines with squares on.

The heavy black lines show mutexes: Two propositions joined by such a line are mutually

exclusive, and cannot both be used in the same plan.

72

Planning Graphs (3)

The planning graph for even a simple problem can be extremely complex.

By producing a complete planning graph for a problem, it can be determined whether a plan is possible, and the plan itself can also be derived.

Algorithms such as GraphPlan can be used to extract the plan.

73

GraphPlan

Problems are expressed in STRIPS notation. GraphPlan iteratively builds a planning

graph, starting from the initial state and working towards the goal state.

All applicable operators are applied at each level to produce the next level .

When the propositions necessary for the goal are included in the current level, and they are not mutex, a possible solution may have been reached.

74

ADL

Another method for representing planning problems.

ADL – Action Description Language: More expressive than STRIPS. Allows quantified expressions such as:

x.P(x) Λ ¬ Q(x) Preconditions can include disjunctions. Allows conditional effects – effects of actions

that are dependent on other factors.

75

Probabilistic Planning

Thus far we have assumed that all actions are deterministic.

In fact, some actions are non-deterministic – their effects can vary.

It is possible to extend situation calculus to deal with non-deterministic actions.

76

Dynamic World Planning

Our discussion so far has assumed the world is static.

In fact, the world is dynamic – things outside of the control of the planner change.

Execution monitoring is used to monitor the execution of plans: If something changes during execution, replanning

may be necessary. Another approach is conditional planning – this

includes every possible outcome in the plan.

77

Case-Based Planning

Case-based planning involves storing each plan that is devised.

Plans (and partial plans) can be re-used later to solve other, similar problems.

Example: CHEF. A system that is used to devise recipes for Chinese food given a set of ingredients.

If presented with a set of ingredients it has not seen before, it is able to use similar sets of ingredients it has seen before to devise a new recipe.

78

Scheduling

Scheduling is like planning, but also takes into account the length of time each action takes to execute.

Job-shop scheduling involves allocating machinery to a set of tasks.

The scheduler plans when each task will start, and how long it will take.

Scheduling can be treated as planning with constraints, where the constraints specify how long tasks will take.