Temporal Planning
description
Transcript of Temporal Planning
Temporal Planning
action models Using PDDL2.1 standard
how to model the search Progression; Regression; PO planning
how to extract good heuristics
Done
No Temporal Gap Classical + Scheduling
Forbidding temporal gap implies All effects at one time Before-conditions meet effects After-conditions meet effects
Unique transition per actionA [d] *
pre
eff
Essence of Temporal Planning
A *
B *
C*D*
Approaches for MTP In theory, pretty much every one of the approaches we
saw for classical planning can be (and have been) extended to MTP (with varying degrees of scalability)
There are some interesting tradeoffs PO planners are easiest to extend to support the concurrency
needed for durative actions Have harder time handling resources (because resource consumption
depends on exactly what actions occurred before this time point) Progression planners easiest to extend to support resource
consuming actions But harder time handling concurrency (need to consider “advancing
clock” as a separate option in addition to applying one of the actions)
Temporal Planning via Plan Space Planning
Instead of constraints w.r.t steps (s<s’ or P@s), we will have constraints on time points (t – t’ = 4 etc.).
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
at_fuse_box@G}
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Cross_cellar GI
At_fusebox
Have_light@<t1,t2>
t1
t2
t2-t1 =10t1 < tGtI < t1
Have_light@t1
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
at_fuse_box@G}
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Cross_cellar GI
At_fusebox
Have_light@<t1,t2>
t1
t2
t2-t1 =10t1 < tGtI < t1T4<tGT4-t3=15T3<t1T4<t3 V t1<t4
Have_light@t1
Burn_matcht3 t4
~have-light
The ~have_light effect at t4 can violate the <have_light, t3,t1> causal link! Resolve by Adding T4<t3 V t1<t4
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
at_fuse_box@G}
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Cross_cellar GI
At_fusebox
Have_light@<t1,t2>
t1
t2t2-t1 =10t1 < tGtI < t1t4<tGt4-t3=15t3<t1t4<t3 V t1<t4t3<t2t4<t3 V t2<t4
Have_light@t1
Burn_matcht3 t4
~have-light
To work on have_light@<t1,t2>, we can either --support the whole interval directly by adding a causal link <have-light, t3,<t1,t2>> --or first split <t1,t2> to two subintervals <t1,t’> <t’,t2> and work on supporting have-light on both intervals
Notice that zenoallows arbitraryslack betweenthe two actions
PO (Partial Order) Search
[Zeno; 1994]
Split theInterval intoMultiple overlappingintervals
Involves Posting temporal Constraints, andDurative goals
Involves LPsolving overLinear constraints(temporal constraintsAre linear too);Waits for nonlinear constraintsTo become linear.
More on Temporal planningby plan-space planners (Zeno)
The “accommodation” to complexity that Zeno makes by refusing to handle nonlinear constraints (waiting instead until they become linear) is sort of hilarious given it doesn’t care much about heuristic control otherwise Basically Zeno is trying to keep the “per-node” cost of the search down (and if
you do nonlinear constraint consistency check, even that is quite hard) Of course, we know now that there is no obvious reason to believe that reducing the
per-node cost will, ipso facto, also lead to reduction in overall search. The idea of “goal reduction” by splitting a temporal subgoal to multiple sub-
intervals is used only in Zeno, and helps it support a temporal goal over a long duration with multiple actions. Neat idea.
Zeno doesn’t have much of a problem handling arbitrary concurrency—since we are only posting constraints on temporal variables denoting the start points of the various actions. In particular, Zeno does not force either right or left alignment of actions.
In addition to Zeno, IxTeT is another influential metric temporal planner that uses plan-space planning idea.
Temporal Planning with Progression?
All the progression planners we looked at were able to produce sequential plans.
Can progression be used to produce concurrent plans? Sub Question: Can progression be used to produce
parallel plans?
Producing parallel plans with progression? The naïve idea is to project over subsets of non-interfering actions (rather than single
actions). Problem: Exponential branching factor
A better idea: Consider “fattening” as well as “lengthening” the current partial plan as two options. We start by representing the state of a partial plan prefix as [S, {A1…Ak}] where S is the
current state, and {A1..Ak} are the mutually non-interfering actions that we have already committed to applying at S.
Notice that this is just a generalization of the normal progression state, in which the action set {A1..Ak} will be a singleton
Given a state [S,{A1..Ak}] to expand, we have (backtrackable) choices: Fatten: Consider applying another action B in state S [One branch for each possible action B]
For this to be feasible, B should be applicable in Si and B should not be interfering with A1..Ak. The resulting state will be {S; {A1…Ak}+B}
Advance: Consider advancing the state. If S’ is the state resulting from application of {A1..Ak} to S, then generate the state {S’; {}}
Notice that Fattening is only done at the current state (once advancing is done, the current state changes. So
any new fattening will be done at the new state. Normal progression does an automatic advance after each fatten (which means you will have only
one action at each step)
Generating concurrent plans is similar to generating parallel plans…almost..
We will continue to consider fattening at the current state, and advancing to the next state
Several issues: Actions have durations. So the state information should
include the actions we committed to (but haven’t completed) How much do we advance?
Worst case: Advance time to the “next time point” (could be bad news if we have dense time!)
Clever idea: Advance time to the “next happening” –where the state changes
State-Space Search:Search is through time-stamped states
Search states should have information about -- what conditions hold at the current time slice (P,M below) -- what actions have we already committed to put into the plan (,Q below)
S=(P,M,,Q,t)
Set <pi,ti> of predicates pi and thetime of their last achievement ti < t.
Set of functions represent resource values.
Set of protectedpersistent conditions(could be binary or resource conds).
Event queue (contains resource as wellas binary fluent events).
Time stamp of S.
In the initial state, P,M, non-empty Q non-empty if we have exogenous events
Search Algorithm (cont.) Goal Satisfaction: S=(P,M,,Q,t) G if <pi,ti> G either:
<pi,tj> P, tj < ti and no event in Q deletes pi. e Q that adds pi at time te < ti.
Action Application: Action A is applicable in S if:
All instantaneous preconditions of A are satisfied by P and M.
A’s effects do not interfere with and Q. No event in Q interferes with persistent
preconditions of A. A does not lead to concurrent resource change
When A is applied to S: P is updated according to A’s instantaneous
effects. Persistent preconditions of A are put in Delayed effects of A are put in Q.
Flying
(in-city ?airplane ?city1)
(fuel ?airplane) > 0
(in-city ?airplane ?city1) (in-city ?airplane ?city2)
consume (fuel ?airplane)
Flying
(in-city ?airplane ?city1)
(fuel ?airplane) > 0
(in-city ?airplane ?city1) (in-city ?airplane ?city2)
consume (fuel ?airplane)
S=(P,M,,Q,t)
Search: Pick a state S from the queue. If S satisfies the goals, endElse non-deterministically do one of
--Advance the clock (by executing the earliest event in Qs
--Apply one of the applicable actions to S
[TLplan; Sapa; 2001—talk given 9/12/01]
Decision Epochs: Limiting the places where clock can be advanced
To support concurrency, we need to consider advancing the clock
How far to advance the clock? One popular strategy is to advance the clock to the
time of the next earliest event in the event queue; since this is the least advance needed to make changes to P and M of S.
At this point, all the events happening at that time point are transferred from Q to P and M (to signify that they have happened)Light-match
Cross-cellar
~have-light
1510
In the cellar plan above, the clock,If advanced, will be advanced to 15,Where an event (~have-light will occur)This means cross-cellar can either be doneAt 0 or 15 (and the latter makes no sense)
Cross-cellar
Interference
Clearly an overkill
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
Let current state S be P:{have_light@0; at_steps@0}; Q:{~have_light@15} t: 0(presumably after doing the light-candle action) Applying cross_cellar to this state gives
S’= P:{have_light@0; crossing@0}; :{have_light,<0,10>} Q:{at_fuse-box@10;~have_light@15} t: 0
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Light-match
Light-match
Cross-cellar
1510
Time-stamp
Short matches
No epoch available “middle of
nowhere” Decision Epoch
Planning is incomplete!
!!!
Wow!
Decision Epoch Planning: DEP Only start actions after events Choose
Start an action Advance epoch
Temporally Simple Complete, suboptimal
Temporally Expressive Incomplete, suboptimal
Salvaging DEP
A [3]21 GG
B [2]
2G
light-match [8]ML L
M
fix-fuse [4]L
F
Generalized DEP: DEP+ Also end actions after events Choose
Start an action End an action Advance epoch
Temporally Simple Complete, optimal
Temporally Expressive Incomplete, suboptimal
Salvaging DEP
A [3]21 GG
B [2]
2G
Wow! Temporally Simple
Classical + Scheduling
Winners incomplete for all Temporally Expressive
Languages
Most/all benchmarks are classical!
!!!
State of the Art: Incomplete or Slow Metric-FF, MIPS, SGPlan, SAPA,
TP4, TPG, HSP*, ... Guarantees only for temporally
simple languages Can solve some concurrent problems
Light-match, but not short-match Difficult to detect
ZENO, IxTeT, VHPOP, LPGP, ... Complete Slow
!!!
Interleaving-Space: TEMPO Delay dispatch decisions until afterwards Choose
Start an action End an action Make a scheduling decision
Solve temporal constraints
Temporally Simple Complete, Optimal
Temporally Expressive Complete, Optimal
Salvaging State-space Temporal Planning
light
fix
match
fuse
fix
fix light
fusefix light
matchfusefix light
Slides beyond this not explicitly discussed in the class
Regression Search is similar…
In the case of regression over durative actions too, the main generalization we need is differentiating the “advancement of clock” and “application of a relevant action”
Can use same state representation S=(P,M,,Q,t) with the semantics that P and M are binary and resource
subgoals needed at current time point Q are the subgoals needed at earlier
time points are subgoals to be protected over
specific intervals We can either add an action to support
something in P or Q, or push the clock backward before considering subgoals
If we push the clock backward, we push it to the time of the latest subgoal in Q
TP4 uses a slightly different representation (with State and Action information)
[TP4; 1999]
A2:X
A3:W
A1:Y
QRWXy
We can either workOn R at tinf or R and QAt tinf-D(A3)
To work on have_light@<t1,t2>, we can either --support the whole interval directly with one action --or first split <t1,t2> to two subintervals <t1,t’> <t’,t2> and work on supporting have-light on both intervals
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
Let current state S be P:{at_fuse_box@0} t: 0
Regressing cross_cellar over this state gives
S’= P:{}; :{have_light,< 0 , -10>} Q:{have_light@ -10;at_stairs@-10} t: 0
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Cross_cellar
Have_light
Notice that in contrast to progression,Regression will align the end points of Concurrent actions…(e.g. when we put inLight-match to support have-light)
(:durative-action cross_cellar:parameters ():duration (= ?duration 10):condition (and (at start have_light)
(over all have_light)(at start at_steps))
:effect (and (at start (not at_steps)) (at start crossing)(at end at_fuse_box)
)
S’= P:{}; :{have_light,< 0 , -10>} Q:{have_light@-10;at_stairs@-10} t: 0
If we now decide to support the subgoal in QUsing light-match
S’’=P:{} Q:{have-match@-15;at_stairs@-10} :{have_light,<0 , -10>} t: 0
(:durative-action burn_match:parameters ():duration (= ?duration 15):condition: (and (at start have_match)
(at start have_strikepad)):effect (and (at start have_light)
(at end (not have_light)))
)
Cross_cellar
Have_light
Notice that in contrast to progression,Regression will align the end points of Concurrent actions…(e.g. when we put inLight-match to support have-light)
Cross_cellar
Have_light
Light-match
Tradeoffs: Progression/Regression/PO Planning for metric/temporal planning
Compared to PO, both progression and regression do a less than complete job of handling concurrency (e.g. slacks may have to be handled through post-processing).
Progression planners have the advantage that the exact amount of a resource is known at any given state. So, complex resource constraints are easier to verify. PO (and to some extent regression), will have to verify this by posting and then verifying resource constraints.
Currently, SAPA (a progression planner) does better than TP4 (a regression planner). Both do oodles better than Zeno/IxTET. However TP4 could be possibly improved significantly by giving up the insistence
on admissible heuristics Zeno (and IxTET) could benefit by adapting ideas from RePOP.
When is Temporal Planning Really Temporal?
William CushingSubbarao Kambhampati
Special thanks to: J. Benton, Menkes van den
Briel
MausamDaniel Weld
Temporal Planning
Plan-space Extended planning graph Reduction to ILP State-space
Competition winners Reachability heuristics
Infinite number of time points Decision Epochs
Restrict start times to events
Introduction
name [duration]start-pre end-preover-pre
start-eff end-eff
light-match [8]ML L
M
fix-fuse [4]L
F
M - matchL - lightF - fuse
Troubling Questions What do/should the
IPCs measure?
Essence of Temporal Planning Required Concurrency Temporally Simple Temporally Expressive
Can Decision Epoch Planning be fixed?
No. But! DEP+
“Less” incomplete TEMPO
Reachability heuristics
Overview
≈ Classical ≈
Harder
Required Concurrency Temporally Simple Languages
Concurrency never necessary …but can be exploited for quality
Temporally Expressive Languages Can specify problems such that
concurrency is needed
Essence of Temporal Planning
Temporal Action Languages
eo,s,es,Lname [duration]
Start-pre End-preOver-pre
Start-eff End-eff
Essence of Temporal Planning
oeLname [duration]
Over-pre
End-eff
Temporal Action Languages Temporally Simple
Rescheduling is possible MIPS, SGPlan, LPG, …
Sequential planning is complete – “optimal” ? TGP, yes In general, yes
Temporally Expressive
Temporal Gap
A [d]s eo
s eeo,s,
es,L
Les Ls
eLs,e
Essence of Temporal Planning
(Minimal) Temporally Expressive Languages
Temporal Gap Before-condition and effect After-condition and effect Two effects
Temporally Simple No Temporal Gap
Essence of Temporal Planning
No Temporal Gap Classical + Scheduling
Forbidding temporal gap implies All effects at one time Before-conditions meet effects After-conditions meet effects
Unique transition per action
Theorem: Every concurrent plan is an O(n) rescheduling of a sequential plan And vice versa
A [d] *pre
eff
Essence of Temporal Planning
A *
B *
C*D*
Conclusions Required concurrency is the essence of temporal
planning Otherwise classical planner + O(n) scheduling
suffices Simple test for required concurrency: Temporal gap
Decision epoch planning is fundamentally incomplete But DEP+ may solve most real-world problems
Complete state-space temporal planning: TEMPO Allows leveraging of state-based reachability
heuristics !!!!!
Evaluating Temporal Planning Domains
William CushingSubbarao Kambhampati
Kartik Talamadupula
Daniel WeldMausam
Competition winners are incomplete
fix-fuse
light-match
L
FM
L -M^ -L
How incomplete? What should the IPC measure?
Epoch A time at which an event
happens Decision Epoch Planning
Only start actions after epochs
Temporal Planning
Required Concurrency
How deep is the problem?
Required Concurrency
Languages Incomplete for
temporally expressive languages
Complete for temporally simple languages
(Minimal) Temporally Expressive Languages
Temporal Gap Before-condition and effect After-condition and effect Two effects
Temporally Simple No Temporal Gap
Required Concurrency Inherently sequential is easy
Timestamps (with support for arithmetic) Loose integration with a PERT scheduler TGP, LPG-td, SGPlan, MIPS, …
Required concurrency is hard The plan space is larger The scheduling sub-problem is harder
Sub-problem optimality principle State of the art is VHPOP, LPGP, CRIKEY
TEMPO, reduction to CSP
The International Planning Competition
Benchmarks must not require (much) concurrency
How much? None at all
How do we show it? Use temporal gap?
Problem: “every” action has temporal gap
Solution: Decompile temporal gap (navigate ?rover ?alpha ?omega)
Pre: (at start (at ?rover ?alpha)) Eff: (and
(at start (not (at ?rover ?alpha))) (at end (at ?rover ?omega)))
(navigate ?rover ?alpha ?omega) (over all (=> (at ?rover) ?alpha ?
omega))
Causal Structure and Concurrency
A *
B
D *
C*
light-matchML L
M
fix-fuseL
F
light-match
fix-fuseA B C D
*
Inherently Sequential Inherently Concurrent
Technique: Start-time Sequentialization
light-matchML L
M
fix-fuseL
F
AL
BLL
AL
BLL
Do not want to enumerate plans! Nor every sequentialization!
Start-time sequentialization Fixed attempt Suffices for benchmarks (not necessary)
End-time sequentialization Critical-time sequentialization
Start times of containing actions in same order as all dependencies
Element Safety Y < X S(A(Y)) > S(A(X)) Threat-free
X supports Z, Y threatens Z
Interaction-free Z supports Y X threatens Y
Link-free Y supports X
AL
BL
AL
BL
AL
BL
Benchmarks never require concurrency
Durative change on m.v. fluents is safe Unbounded resources are safe
“The Perils of Discrete Resource Models” ICAPS workshop on IPC
A few special cases (at end (calibrated ?c ?r))
Document… http://rakaposhi.eas.asu.edu/is-benchmarks.html
Forthcoming
(:durative-action navigate:parameters (?x - rover ?y - waypoint ?z - waypoint):duration (= ?duration 5) :condition (and
;;(at start (at ?x ?y)) ;; MV Fluent ;;(at start (>= (energy ?x) 8)) ;; Resource Consumption (over all (can_traverse ?x ?y ?z)) (at start (available ?x)) (over all (visible ?y ?z)) )
:effect (and ;;(at start (decrease (energy ?x) 8)) ;; Resource Consumption (over all (consume (energy ?x) 8)) ;; Resource Consumption ;;(at start (not (at ?x ?y))) ;; MV Fluent ;;(at end (at ?x ?z)))) ;; MV Fluent (over all (-> (at ?x) ?y ?z)) ;; MV Fluent
))
;;(at ?x - rover ?y - waypoint)
(at ?x - rover ) - waypoint
Only RC due to Modeling Bugs 1: drop 1.1: drop 2.05: sample … (and (full ?s) (empty ?s))
1: recharge 1.1: recharge 1.2: recharge … (>= (energy ?x) (* k (capacity ?x)))
Syntactic Sugar for avoiding Errors Action drop (store)
full(store) == true at start full(store) := false at end
Should be at start empty(store) := true at end
Explicit resources amount(store) :consume 1 space(store) :produce 1
Explicit durative change + m.v. fluents amount(store) == full => empty
Temporal Machine Shop Benchmarks lack required concurrency
Real world lacks required concurrency? (:durative-action fire-kiln
:parameters (?k - kiln):duration (= ?duration 20):effect (and (over all (lend (firing ?k)))
(over all (–> (ready ?k) true false)) (:durative-action bake-ceramic
:parameters (?p - piece ?k - kiln):duration (= ?duration (bake-time ?p)):condition (and (over all (firing ?k))
(over all (shaped ?p))):effect (over all (–> (baked ?p) false true)))
Real world required concurrency (and
(lifted bowl-left) (lifted bowl-right))
Spray-oil (during milling) Heat-beaker (while adding
chemicals) Ventilate-room (while drying glue) …
Lessons for the Competition Competitors tune for the benchmarks
Most of the competitors simplify to TGP Either required concurrency is
important Benchmarks should test it
Or it isn’t Language should be inherently sequential
PDDL spec. highlights light-match RC occurs in the real world
Might need processes, continuous effects
Conclusion Required Concurrency separates easy and hard
temporal planning The easy case allows offloading to a scheduler
Still an intriguing problem Simplify the language – push the classical track
The hard case forces temporal reasoning by the planner Real world required concurrency is frequent
PDDL 2.1.3 was designed for required concurrency But the benchmarks fell through
Analysis of domains is hard Automatable? Embeddable within a search?
Domain modeling is very hard Durative change Resources
Evaluating Temporal Planning Domains
ICAPS 2007
When is Temporal Planning Really Temporal?IJCAI 2007
The Perils of Discrete Resource Models
ICAPS 2007, IPC workshop