Activity Planning

Tom Krenzke and Stephen Thrasher

Introduction

Planning involves generating a sequence of actions to achieve a particular goal. Search algorithms and constraint satisfaction problems can find such a sequence, but planning algorithms usually move a step farther and exploit the logical properties of planning problems to work much faster, making larger, more complex problems feasible. These properties can include the ability to decompose the problem into subgoals or the concept of working backwards from the goal state to avoid irrelevant actions. A good planning algorithm should mirror the completeness of the planning problem by either returning a plan or saying that no plan exists.

The planning example and API in this tutorial is Graphplan. Graphplan is a general purpose planning algorithm developed by Avrim Blum and Merrick Furst of Carnegie Mellon University. Graphplan's addition to the planning problem is its polynomial-size plan graph, a compact tree representation that incorporates many planning constraints into its structure. Graphplan is complete and optimal, finding the shortest possible plan or saying that no plan exists.

In a planning problem, the state of the world is encoded as a set of boolean variables called propositions. For compactness, a proposition is listed at a particular time step only if it is reachable from the propositions at the previous time step. These propositions can change between time steps by applying actions. Actions are operators that change the world's state. An action cannot be applied unless its preconditions are met. Preconditions are simply a list of propositions associated with an action that must be true before that particular action can be executed.

Often, as in this tutorial, planning problems have concrete limitations. Propositions must be fully observable, meaning there is no uncertainty about a proposition's truth. The actions of a planning problem must be deterministic and static. Deterministic means that the effects of each action are known with certainty. Thus, no action can have two different results with associated probabilities. Static means that the environment does not change outside of the applied actions. Also, there is no notion of time in this formulation, so the time it takes to execute an action should not be a major factor of the problem. Both propositions and actions must be finite and discrete.

Several other methods exist to solve planning problems. Notable methods are total-order and partial-order planners, which are implemented by the algorithms Prodigy and UCPOP,

respectively. In total-order planning, only one action is considered at each timestep. This avoids the problem of conflicting simultaneous actions, but its search space is large. Partial-order planning allows for simultaneous actions, which reduces the problem space size, but it requires extra bookkeeping. Graphplan outperforms these algorithms on several benchmark problems for any problem size. In fact, the larger the problem, the better Graphplan does in comparison. The efficiency of Graphplan comes from the planning graph structure and the method of searching for a plan.

Also, planning problems may be cast as propositional logic problems and solved using satisfiability algorithms, such as SATplan, which solve planning problems in a general logical manner. The disadvantage of this approach is that it requires a huge propositional knowledge base to be generated from the original planning problem. The advantage is that efficient solvers such as Blackbox [3] do exist which can handle these large problem domains. Graphplan can outperform Blackbox on some smaller problems, and Blackbox can greatly ourperform Graphplan on larger problems.

Problem Description

Motivation for Problem

Activity planning is useful in any case where a set of interdependent tasks must be completed efficiently. This is true in industry, where even small changes in production methods can save lots of money. Even class scheduling, where each class has prerequisites and exclusive time slots, can benefit from activity planning.

Autonomous systems that are capable of self-diagnosis, self-repair, planning, and re-planning are more robust in the presence of failure or unexpected situations. The Mars rovers and deep space probes must be able to decide a course of action without any input from a ground control because communication is not fast enough.

Formal Problem Statement

To illustrate the planning problem, consider baking an apple pie. The problem begins with a cold oven, uncut apples, and oven mitts on the counter. The planner must determine a sequence of actions to successfully construct an apple pie.

Inputs

Every activity planning problem has

● A set of known initial propositions,

● A set of actions, with defined preconditions and effects, and ● A set of desired goal propositions.

The problem can be represented in the STRIPS format. STRIPS stands for STanford Research Institute Problem Solver, which was developed for the Shakey robot project in 1971. In this format, propositions are represented by string literals. States only consist of positive literals, and every absent literal is assumed to be false. Sometimes, however, it is necessary to enumerate both the positive and negative literals for a condition. For example, the pie example below contains both mitts-off and mitts-on. This is because actions check for the presence of positive literals in their preconditions. In order to execute, the action put-on-mitts must find mitts-off, not just the absence of mitts-on. Actions are represented by a string literal, preconditions, and effects. The effects can be divided into added propositions and deleted propositions.

In the pie example, the initial conditions are oven-cold and mitts-off. The goal state is pie-served.

# Preconditions Action Add Effects Delete Effects1 oven-cold preheat-oven oven-hot oven-cold2 mitts-off put-on-mitts mitts-on mitts-off3 mitts-on, pie-baked pull-pie-from-oven pie-served, oven-cold oven-hot4 mitts-off chop-apples apples-chopped 5 apples-chopped, mitts-off mix-ingredients pie-prepared 6 pie-prepared, oven-hot bake-pie pie-baked

Outputs

The output of an activity planning algorithm is a sequence of actions or the declaration that no solution exists. If possible, some steps in the sequence will contain more than one action, as long as those actions can be executed in any order with no change in the result. For example, preheating the oven and chopping the apples can be done at the same step because their order does not matter. However, preheating the oven and baking the pie cannot be executed in the same step because baking requires that the oven to first be hot.

Specification of a Correct Solution

A correct solution is a sequence of actions to acheive the goal state in the fewest steps without any conflict between actions, or the declaration that no solution exists, if that is the case. A conflict between two actions is called a "mutual exclusion" or "mutex." There are three ways

that actions can conflict

● Conflicting preconditions. put-on-mitts requires mitts-off, whereas pull-pie-from-oven requires mitts-on.

● Conflicting effects. preheat-oven causes oven-hot, whereas pull-pie-from-oven causes oven-cold (presumably because you let the heat out when you open the door).

● Conflict between an effect and a precondition. put-on-mitts has the effect of mitts-on, whereas mix-ingredients requires mitts-off. Because actions must be interchangeable in time in order to be in the same timestep, they conflict.

Here is a table of the mutexes, where the numbers mean

1. Conflicting preconditions 2. Conflicting effects 3. Effect of one deletes precondition of another

preheat-oven

put-on-mitts

pull-pie-from-oven

chop-apples

mix-ingredients

bake-pie

preheat-oven X none 2 none none 1put-on-mitts X 1 3 3 nonepull-pie-from-oven

X 1 1,3 1,3

chop-apples X 3 nonemix-ingredients

X 3

bake-pie X

Java API

class Graphplan void setInitialPropositions(List prop) void setFinalPropositions(List prop) void setActionLibrary(List actions) List solve()

class Action

void setPreconditions(List precond); void setAddEffects(List propositions); void setRemoveEffects(List propositions);

Propositions are represented by Strings.

Example Run of API on Pie World

Instantiate the required objects.

Graphplan planner = new Graphplan();

ArrayList initialPropositions = new ArrayList();ArrayList finalPropositions = new ArrayList();ArrayList actionLibrary = new ArrayList();

Populate the initial and goal propositions lists.

initialPropositions.add("oven-cold");initialPropositions.add("mitts-off");

finalPropositions.add("pie-served");

Here is an example of how to create an action.

Action action = new Action("preheat-oven");ArrayList temp = new ArrayList();temp.add("oven-cold");action.setPreconditions(temp);temp = new ArrayList();temp.add("oven-hot");action.setAddEffects(temp);temp = new ArrayList();temp.add("oven-cold");action.setRemoveEffects(temp);

actionLibrary.add(action);

Once the initialization lists are made, they can be input into the planner, and the Graphplan problem can be solved.

planner.setInitialPropositions(initialPropositions);

planner.setFinalPropositions(finalPropositions);planner.setActionLibrary(actionLibrary);

List plan = planner.solve();

Method Description

Planning Graph Structure

Throughout this explanation, refer to the graphic below for a visual aid.

The power of Graphplan is its ability to efficiently encode the necessary problem information into a compact form. The planning graph starts with the initial conditions at step 1. These conditions determine what actions are possible at step 1, and the effects of these actions appear at step 2. In a planning problem, actions can only be executed if their preconditions are met. Accordingly, in a planning graph, an action will appear in the graph at a particular step when its preconditions exist at that step.

Besides a planning problem's declared action operators, there also exist "no-op" operators, meaning "no operation." There are as many no-op operators as there are propositions, i.e. oven-hot propagates from one time step to the next because no-op-oven-hot gives the option to not change it.

Like a partial-order planner, Graphplan allows multiple non-conflicting actions to be executed on the same step. If such groups of actions exists, putting them at the same step makes for fewer steps, a smaller graph, and less time to finding a solution.

Once a proposition appears in the graph, it is sustained for all subsequent steps by its no-op operator. Thus, propositions can only be added, and the number of propositions monotonically increases with time. As a result, the number of actions also monotonically increases because no preconditions are ever removed.

Graph Expansion

The first step of Graphplan is to construct the planning graph by expanding propositions through actions for as many levels as it needs. Given the propositions available at level i, all actions whose preconditions are met are inserted into the graph. The effects of every action are then carried or added to the next level of the graph. Expansion at every level includes the no-op action for every existing proposition. When Graphplan is started, it expands the graph to the first level where all the goal propositions appear, at which point search begins.

Mutual Exclusions

Two kinds of mutexes exist, those between actions and those between propositions. Examples of action mutexes are in an above section for Pie World. Action mutexes are relatively easily found by checking them pairwise for (1) precondition conflicts, (2) effect conflicts, and (3) conflicts between a precondition and an effect. Again, the third conflict is a problem because actions at the same step must be interchangeable.

Proposition mutexes are trickier. They result from action mutexes. Two propositions are considered mutually exclusive at some step if no sequence of non-conflicting actions exists that can generate them. To determine this might require a backward search through the graph to check all the possible ways of generating a pair of propositions. The number of proposition mutexes monotonically decreases with each step. This is because once a pair of propositions appears as non-mutex, no-op operators will preserve them as non-mutex.

Search Through the Plan Graph

If the final step does not include all the goal propositions, no solution can exist at that step, so Graphplan continues to expand the graph until all goal propositions appear. Once they appear,

if no mutexes exist between them, then a feasible solution exists, and if mutexes do exist, the graph may need to be expanded further.

To determine whether mutexes exist between the goal propositions, a search is necessary. One way to do this is with a backtrack search. Graphplan picks a set of non-mutex actions that generate the goal propositions. The set of preconditions to this set of actions is considered to be a subgoal. Graphplan then repeats this process for the subgoal, and its subgoals, and so forth. If at any point no set of non-mutex actions exists, Graphplan backtracks to the last place where possible action sets remain. It also records every set of subgoal propositions that had no path to the goal. This is called a memo. When searching, Graphplan tests every subgoal against its stored memos, so that if a match is found, it can backtrack without searching the rest of that branch.

If the algorithm reaches the initial conditions, the search is finished. If an exhaustive search reveals no sequence of non-mutex action sets, no solution exists at that level. At that point, Graphplan might expand the graph to the next level and restart the search. This search can be very time-consuming, exponential in the worst case, because adding a level multiplies the number of action sets that need checking.

Termination Conditions

When a solution is found, Graphplan ends its search and returns the shortest feasible sequence of actions required to reach the goal conditions. Graphplan also terminates when no solution exists. When the graph levels off, that is, no new actions and propositions are added between consecutive levels, Graphplan looks at the number of memos saved where the graph first levels off. If the number of memos at this level does not change as the graph is expanded further, then Graphplan terminates. The proof is available in [1].

For help visualizing the full Graphplan algorithm, view this animated GIF, which shows the expansion and search of the planning graph for the Pie World problem.

A Stroll Through Pie World

Following along with the algorithm pseudocode, the initial conditions of Pie World are mitts-off and oven-cold. These get inserted into level one of the graph. The variable numprops is two, and numMutexes is zero for level one.

program graphplan()level=1propositions[level] = initial conditionsnumprops[level] = numInitialConditionsnumMutexes[level] = 0do expandGraph() level = level + 1 if (goal props appear) [bts,numMutexes[level]] = backtracksearch() if (bts not null) return bts end if end ifwhile ((numprops[level] not equal numprops[level-1]) or (numMutexes[level] not equal numMutexes[level-1]))end graphplan()

At this point, the graph gets expanded. Since the preconditions for the actions preheat-oven, chop-apples, and put-on-mitts exist at level one, they go in the graph's action list for that level. The add effects of all these actions are then unioned with the propositions at the current level to create the proposition list for the next level. In Pie World, level two contains mitts-off, oven-cold, oven-hot, apples-chopped, and mitts-on.

procedure expandGraph() actionList(level) = all actions with preconditions met by current level propositions(level)=propositions(level-1)+effects(actionList(level))end expandGraph()

The outermost while loop in Graphplan continues until all the goal propositions appear in the proposition list. Then backtrack search begins to check for mutexes between the goal propositions. In Pie World, the only goal proposition is pie-served. The only action that results in this is pull-pie-from-oven. The preconditions are pie-baked and mitts-on. The search continues backwards to see if a mutex exists between these two subgoals. Continuing the search, only bake-pie causes pie-baked, and only put-on-mitts causes mitts-on. No action mutex exists, and the search continues for earlier levels.

procedure backtrackSearch() i=1; a_i={} actionListCopy(i) = actionList(i) while (1<=i<=n) x_i = selectValue() if (x_i==null) i=i-1 if (i==1), increment numMutexes[level] else i=i+1 actionListCopy(i) = actionList(i) end if end while if (i==0) return null else return a_i end ifend backtrackSearch()

procedure selectValue() while actionListCopy(i) is not empty select a non-mutex set of actions a from D'_i if (no such set exists), return a end while return nullend selectValue()

The solution to Pie World is Step 1: preheat-oven, chop-apples Step 2: mix-ingredients Step 3: bake-pie, putt-on-mitts Step 4: pull-pie-from-oven

Benchmark Testing

A bi-annual competition is held to determine the fastest activity planning solver. Each planner is run on benchmark problems of varying size and complexity. The competition is usually divided into deterministic vs. probabilistic categories. More information is available here. The benchmark problems are described in the PDDL language, which allows for compact representation of operators. Additionally, it allows for parameterization of operators. Our simple graph plan solver isn't able to handle parameterization, so every possible operator has to be enumerated. As such, it requires some work to get a standardized benchmark to run on the provided code. One of the smaller benchmark problems was implemented: Monkey World. The basic objective is to have a monkey get the bananas by moving needed items such as a box and knife between rooms. Blum and Furst used their orginal Graphplan code to solve this example for three different problem instances. The first case was a small example with only 4 rooms to move between. The second case involved 6 rooms and an extra goal proposition. The third case was the same size as the second, but the goal propositions were chosen so that the problem was infeasible. The runtimes for Graphplan on these three cases were 0.7s, 3.4s, and 2.8s respectively. When the problems were run on our implementation the first case took 3.7 seconds to solve, and other two did not produce a solution within 5 minutes so execution was terminated. The most likely cause of our relatively poor performance is the way memos are handled. For ease of implementation, a simple list structure was used to store and check for memos in our code. Far more efficient data structures, such as hash tables, are available which would certainly have decreased runtime.

One other 'standard' problem is the 2-rockets problem. The objective is to get cargo from their current location to some desired location. You only have 2 rockets with which to do this, and the ordering of the goal propositions can have an impact on the solution time. The problem was run for cargo load sizes of 2,3,4,5,6. The runtime for our code on each problem was 0.015s, 0.062s, 0.969s, 13.3s, and 195.1s respectively. Blum and Furst solved most of these problem instances in under a second. The results are presented in [1]. Our sub-par times are, again, probably due to the way in which memos are handled.

References

[1] Blum, Avrim L., and Merrick L. Furst. Fast planning through graph analysis. Artificial Intelligence, 90, 1997.

[2] Russell, Stuart, and Peter Norvig. Artificial Intelligence: A Modern Approach. 2nd ed. Prentice Hall, 2002.

[3] Kautz, Henry, and Bart Selman. Unifying SAT-Based and Graph-Based Planning.Workshop on Logic-Based Artificial Intelligence, Washington, DC, June 14-16, 1999.

[4] Weld, Daniel S. Recent Advances in AI Planning. AI Magazine, vol 20, num 2, pp 93-123, 1999.