Sheila Mcllraith: Of Programs, Plans, and Automata: 101 things you can do with AI planning

85
Of Programs, Plans, and Automata: 101 things you can do with AI planning Sheila A. McIlraith Joint work with Jorge Baier, Christian Fritz & Shirin Sohrabi University of Toronto Toronto, Canada Oxford University, February 24, 2015 Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 1 / 71

Transcript of Sheila Mcllraith: Of Programs, Plans, and Automata: 101 things you can do with AI planning

Of Programs, Plans, and Automata: 101 things you can do with AI planning

Sheila A. McIlraith

Joint work with

Jorge Baier, Christian Fritz & Shirin Sohrabi

University of Toronto

Toronto, Canada

Oxford University,

February 24, 2015

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 1 / 71

...some great graduate students and collaborators ...

Jorge Baier Shirin Sohrabi Christian Fritz Tran Cao Son

Meghyn Bienvenu Aws Albarghouthis Niloofar Razavi

Nathan Robinson

Yixiao Wang

Sotirios Liaskos

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 2 / 71

It’s been 50 years ...

2015 marks the 50th anniversary of the Shakey Project, which resultedin such innovations as A* search and STRIPS planning.

We’ve seen substantial advances in AI Automated Planning since thattime:

compact state and transition system encodings

advances in heuristic search and SAT

plans with hundreds of actions

... hundreds of objects

... 21000 states

plan computation ranging from milliseconds to a couple of hours

Top-performing planners exploit heuristic search, thoughplanning-tailored SAT-based planners are starting to close the gap.

Most of these advances have been in classical planning.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 3 / 71

It’s been 50 years ...

2015 marks the 50th anniversary of the Shakey Project, which resultedin such innovations as A* search and STRIPS planning.

We’ve seen substantial advances in AI Automated Planning since thattime:

compact state and transition system encodings

advances in heuristic search and SAT

plans with hundreds of actions

... hundreds of objects

... 21000 states

plan computation ranging from milliseconds to a couple of hours

Top-performing planners exploit heuristic search, thoughplanning-tailored SAT-based planners are starting to close the gap.

Most of these advances have been in classical planning.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 3 / 71

Automated Planning

Given (F ,O, I,G), where:

F , set of propositions

O, transitions system described in terms of a set of operators

E.g, pickup(X )Precondtion: handEmptyEffect: holding(X ), ¬handEmpty

I ∈ L(F), initial state description

G ∈ L(F), goal condition

Planning Problem:

Find an ordering of operators which if applied in I lead to a statesatisfying G

Complexity:

PSPACE-complete [Bylander, 1994]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 4 / 71

Automated Planning

Given (F ,O, I,G), where:

F , set of propositions

O, transitions system described in terms of a set of operators

E.g, pickup(X )Precondtion: handEmptyEffect: holding(X ), ¬handEmpty

I ∈ L(F), initial state description

G ∈ L(F), goal condition

Planning Problem:

Find an ordering of operators which if applied in I lead to a statesatisfying G

Complexity:

PSPACE-complete [Bylander, 1994]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 4 / 71

Automated Planning

Given (F ,O, I,G), where:

F , set of propositions

O, transitions system described in terms of a set of operators

E.g, pickup(X )Precondtion: handEmptyEffect: holding(X ), ¬handEmpty

I ∈ L(F), initial state description

G ∈ L(F), goal condition

Planning Problem:

Find an ordering of operators which if applied in I lead to a statesatisfying G

Complexity:

PSPACE-complete [Bylander, 1994]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 4 / 71

Classical Planning

Given:

Initial state (fully described)

Goal condition (condition about final state)

Operators (deterministic)

Task:

Find any sequential plan

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 5 / 71

Non-Classical Planning

Non-classical planning can take many forms:

Given:

Initial state (partially described)

Goal condition (temporally extended goal and/or preferences, ...)

Operators (non-deterministic actions, complex actions, ...)

Task:

Find a (high quality) plan

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 6 / 71

Claims

1st ClaimMany tasks that involve reasoning about dynamical systems can be castas non-classical planning problems (and indeed many compellingreal-world planning applications are non-classical).

2nd ClaimWe can exploit state-of-the-art classical planners to effectively solvenon-classical planning problems.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 7 / 71

Claims

1st ClaimMany tasks that involve reasoning about dynamical systems can be castas non-classical planning problems (and indeed many compellingreal-world planning applications are non-classical).

2nd ClaimWe can exploit state-of-the-art classical planners to effectively solvenon-classical planning problems.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 7 / 71

Claims

1st ClaimMany tasks that involve reasoning about dynamical systems can be castas non-classical planning problems (and indeed many compellingreal-world planning applications are non-classical).

2nd ClaimWe can exploit state-of-the-art classical planners to effectively solvenon-classical planning problems.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 7 / 71

The 1st Claim

Many tasks that involve reasoning about dynamical systems can be castas non-classical planning problems. E.g.,

(customized) web service composition

synthesis of business processes

verification & counter-example generation

diagnosis of dynamical systems

explanation generation

activity recognition, plan understanding

narrative generation

stakeholder goal modeling

test generation for concurrent programs

network topology transformation

...

and at least 91 other tasks!

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 8 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

Spoiler Alert

Key Enabler:It is the richness of the statement of the planning objective thatenables us to view 101 (and more) diverse endeavours as instances ofnon-classical planning problems.

These objectives can be specified via

Linear Temporal Logic

Automata

Regular Expressions in the form of (Con)Golog

Hierarchical Task Networks (HTN)

and as either temporally extended goals (aka domain controlknowledge) or as preferences.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 9 / 71

LTL in a Nutshell

A modal temporal logic with modalities referring to time. Interpretedover sequences of states (paths).

Syntax

if f ∈ F then f is a LTL formula

if φ and ψ are LTL formulae then so are ¬φ, φ ∨ ψ, ...

formulae constructed with the following temporal modalities:future: U(until), ©(next), �(always), ♦(eventually)past: S(since),•(yesterday), �(always past), �(eventually past)

E.g., Always have your keys and eventually be at home.�(have(keys)) ∧ ♦(at(home))

(e.g., [Pnueli, 1977])

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 10 / 71

Example - DiagnosisExample 1 - Diagnosis [Sohrabi, Baier, M, KR2010, AAAI11]

Observations

I started my car this morning; drove to work; on the way to work Ibought $5 worth of gas; I hit a pothole; the radio said it was -20Celsius; I parked outside. At noon, I picked up my bag from thetrunk of the car. At the end of the day, my car would not start. Ichecked the radio and it was still working.

What’s the explanation for my car not starting?

Battery died

Punctured gas tank, then ran out of gas

Starter motor broke

: 8[Sohrabi, Baier, M KR2010, AAAI11]Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 11 / 71

Diagnosis as Non-Classical Planning

Dynamical Diagnosis

• Initial State

• Goal State

• Transition System

obsnobs1

System Description

obs2 obs3 . . .

Observations(i.e., multiple partial states)

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 12 / 71

Diagnosis as Non-Classical Planning

Dynamical Diagnosis

• Initial State

• Goal State

• Transition System

obsnobs1a1 a2 a3 ak

Diagnosis = {Assumptions, {a1, a2, a3, …, ak}}

System Description

obs2 obs3 . . .

Observations(i.e., multiple partial states)

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 13 / 71

Example - Test Generation for Concurrent Programs

...

T1 T2 Tk serialization

[Razavi,Farzan,M,VVPS11,STTP14]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 14 / 71

Example - Synthesis of Business Processes

Process Order

PrePay

Account

Invoice Out Source

Manufacture

. . .

Fill Order Ship Goods Process Payment

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 15 / 71

Example 4 - Web Service CompositionWeb Services (WS)

Web Services are Web-accessible programs and devices.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 16 / 71

Automating Web Service Composition (WSC)

Problem: Automatically select and compose services to achieve a user’s objectives. E.g., Make the travel arrangements for my IJCAI 2015 conference

1

Approach: non-classical planning objective comprising: 1. A generic procedure or script to specify roughly how to

perform the task.

2. A set of hard constraints and preferences that customize and optimize for the individual user.

[M, Son, IEEE01, KR02]

[M, Son, IEEE01, KR2002]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 17 / 71

User PreferencesWhy is this Hard ?

E.g., preferences can be

temporally-extended

• I prefer to always pay with my American Express credit card.

conditional

• If my return flight departs before 9am, then I prefer to stay in a hotel located at the airport the night before departure.

interacting and/or mutually inconsistent

• I prefer to stay in one hotel for my entire trip.

• I prefer to stay at the conference hotel.

• I prefer to spend $100/night or less on my hotel room.

nested

• I prefer to always book my flight after my hotel reservation is confirmed, if booking is being done 2 weeks or more prior to the trip.

1. User preferences can be arbitrarily complex and mutually inconsistent.

[Sohrabi, M, ISWC06,10][Sohrabi, Baier, M, IJCAI09]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 18 / 71

Historically, planning with LTL, Golog, and HTNrequired special-purpose planners.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 19 / 71

The 2nd Claim

Question:

Can we leverage the benefits of classical approaches for thesecompelling non-classical planning problems?

Claim:

In many cases it is possible to reformulate a non-classicalplanning problem in order to apply directly or almost directlysuccessful classical approaches, yielding superior performance.

[Jorge Baier, Thesis 2010]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 20 / 71

We Address Compelling Non-Classical Problems

We have addressed the following problems:

Planning for LTL Temporally Extended Goals (TEGs)

Planning with LTL Temporally Extended Preferences (TEPs)

Planning with restricted RE’s – Golog-like Procedural Control

Planning with RE’s plus concurrency – ConGolog Procedural Control

Planning with Golog Sensing Procedures (not discussed)

Planning with Golog-like Procedural Preferences (not discussed)

Key Enabler: Translation to automata or petri-nets in a manner thatsupports the exploitation of heuristic search planners

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 21 / 71

Common Approach: Reformulation

Common Approach: Reformulation

Reformulated

Non−classicalplanning task

planning task

Reformulation

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 7 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 22 / 71

Common Approach: Reformulation

Common Approach: Reformulation

Reformulated

Handled by existing solver? Is not handled by existing solver?

Non−classicalplanning task

planning task

Reformulation

PlannerModified

PlannerExisting

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 7 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 22 / 71

The Rest of the Talk

Planning for LTL Temporally Extended Goals (TEGs)

Planning with LTL Temporally Extended Preferences (TEPs)

Planning with restricted RE’s – Golog-like Procedural Control

Planning with RE’s plus concurrency – ConGolog Procedural Control

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 23 / 71

Planning for LTL Temporally Extended Goals

Main Results:

1 First-order, finite logic to represent TEGs: f-FOLTL(Following [Cerrito, Mayer, Praud, LPAR99])

2 Method to compactly convert TEGs into final-state goals

Algorithm: f-FOLTL formula → parameterized automatonTheorem: Algorithm is correctTheorem: Translation worst-case exponentialProperty: Simplification and parameterization avoids worst-caseblowup in practice.Representation of the automaton in a planning problem

3 Experimental evaluation: showing orders-of-magnitudeimprovements over traditional approaches.

[Baier, M, AAAI2006]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 24 / 71

From f-FOLTL to Parameterized NFAFrom f-FOLTL to Parameterized NFA

Example PNFA for (∀x). (ECapital(x) ⊃ ♦at(x)) ∧ ♦�at(Paris).

ParisC2 NYCC1

at(x) at(Paris)NYC

(¬ECapital(x) ∨ at(x)) ∧ at(Paris)

⊤ ⊤

Plan: nil ;

Objects: C1, C2, Paris, NYC . ECapital = {C1,C2,Paris}Initial State: {at(NYC )}

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 9 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 25 / 71

From f-FOLTL to Parameterized NFAFrom f-FOLTL to Parameterized NFA

Example PNFA for (∀x). (ECapital(x) ⊃ ♦at(x)) ∧ ♦�at(Paris).

ParisC2 NYCC1

at(x) at(Paris)NYC

(¬ECapital(x) ∨ at(x)) ∧ at(Paris)

⊤ ⊤

Plan: nil ;

Objects: C1, C2, Paris, NYC . ECapital = {C1,C2,Paris}Initial State: {at(NYC )}

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 9 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 26 / 71

From f-FOLTL to Parameterized NFAFrom f-FOLTL to Parameterized NFA

Example PNFA for (∀x). (ECapital(x) ⊃ ♦at(x)) ∧ ♦�at(Paris).

ParisC2 NYCC1

at(x) at(Paris)C1

NYC

(¬ECapital(x) ∨ at(x)) ∧ at(Paris)

⊤ ⊤

Plan: FlyTo(C1);

Objects: C1, C2, Paris, NYC . ECapital = {C1,C2,Paris}Initial State: {at(NYC )}

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 9 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 27 / 71

From f-FOLTL to Parameterized NFAFrom f-FOLTL to Parameterized NFA

Example PNFA for (∀x). (ECapital(x) ⊃ ♦at(x)) ∧ ♦�at(Paris).

ParisC2 NYCC1

at(x) at(Paris)C1

C2 NYC

(¬ECapital(x) ∨ at(x)) ∧ at(Paris)

⊤ ⊤

Plan: FlyTo(C1);FlyTo(C2);

Objects: C1, C2, Paris, NYC . ECapital = {C1,C2,Paris}Initial State: {at(NYC )}

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 9 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 28 / 71

From f-FOLTL to Parameterized NFA (PNFA)From f-FOLTL to Parameterized NFA

Example PNFA for (∀x). (ECapital(x) ⊃ ♦at(x)) ∧ ♦�at(Paris).

ParisC2 NYCC1

at(x) at(Paris)C1

C2 NYCParis

(¬ECapital(x) ∨ at(x)) ∧ at(Paris)

C1C2 NYCParis

⊤ ⊤

Plan: FlyTo(C1);FlyTo(C2);FlyTo(Paris);

Objects: C1, C2, Paris, NYC . ECapital = {C1,C2,Paris}Initial State: {at(NYC )}

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 9 / 19

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 29 / 71

From PNFA to PDDL

Key Idea: Represent the PNFA within the planning domain.

We update original problem with:

New Predicates represent what objects are in what states

New Updates in Actions for the new predicates

New (Classical) Goal based on the accepting condition of theautomaton

Augmented Initial State

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 30 / 71

Experimental Evaluation

Our Reformulation + FFX [Thiebaux, Hoffman, Nebel, 2005]versus

TLPLAN [Bacchus & Kabanza, 1998]

... on standard benchmarks extended with TEGs.

Experimental Evaluation

Our Reformulation + FFX (Thiebaux, Hoffmann, & Nebel, 2005)versus

TLPlan (Bacchus & Kabanza, 1998)

... on standard benchmarks extended with TEGs.

Domain Speedup

< 2 timesfaster

2-10 timesfaster

1 order ofmagn.

2 orders ofmagn.

≥ 3 ordersof magn.

ZenoTravel 0 11% 22% 56% 11%

Logistics 6% 24% 24% 35% 12%

Robot 0 44% 33% 22% 0

Translation time never exceeds 15% of planning time.Ratio |Automatonϕ|/|ϕ| never exceeds 1.Solution lengths are comparable.

J. Baier: Effective Search Techniques for Non-Classical Planning via Reformulation 11 / 19

Translation time never exceeds 15% of planning time.

Ratio |Automatonϕ|/|ϕ| never exceeds 1.

Solution lengths are comparable.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 31 / 71

The Rest of the Talk

Planning for LTL Temporally Extended Goals (TEGs)

Planning with LTL Temporally Extended Preferences (TEPs)

Planning with restricted RE’s – Golog-like Procedural Control

Planning with RE’s plus concurrency – ConGolog Procedural Control

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 32 / 71

Planning with LTL Temporally Extended Preferences

Main Results:

1 Compilation of TEPs in PDDL3 to final-state preferences.

2 Branch-and-bound algorithm that finds plans of increasing quality

3 Suite of heuristics, based on existing heuristics

4 Formal analysis:

conditions for optimality and k-optimality.

5 Implemented system: HPlan-P (IPC Runner Up)

6 Experimental analysis that shows that:

Heuristic and pruning are key to good performanceOur planner is more consistent than the competition winner

[Baier, Bacchus, M, AIJ2009]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 33 / 71

But what about (action-centric) procedural control?

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 34 / 71

The Rest of the Talk

Planning for LTL Temporally Extended Goals (TEGs)

Planning with LTL Temporally Extended Preferences (TEPs)

Planning with restricted RE’s – Golog-like Procedural Control

Planning with RE’s plus concurrency – ConGolog Procedural Control

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 35 / 71

Planning with Regular Expressions+

In planning, experts often know how to solve a problem, e.g.:

“If not at point A, then drive the truck to point A; while thereare packages at point A, pick a package and load it onto thetruck; drive to point B; while there are packages on the truck,pick a package and unload it from the truck.”

We call this search control.

It can drastically increase planning efficiency.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 36 / 71

Golog

One language for specifying search control is Golog:

if not at(A) then driveTo(A);

while (∃pkg)at(pkg,A) do πpkg.load(pkg);

driveTo(B);

while (∃pkg)onTruck(pkg) do πpkg.unload(pkg)

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 37 / 71

Golog and ConGolog: Syntax

Golog:a primitive actionφ? test condition φ(δ1; δ2) sequenceif φ then δ1 else δ2 conditionalwhile φ do δ′ loops(δ1|δ2) non-deterministic choiceπv .δ non-deterministic choice of argumentδ∗ non-deterministic iteration{P1(~t1, δ1); . . . ;Pn(~tn, δn); δ} procedures

ConGolog: (in addition)(δ1 ‖ δ2) concurrent execution(δ1 〉〉 δ2) prioritized concurrencyδ‖ concurrent iteration

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 38 / 71

Golog and ConGolog: Syntax

Golog:a primitive actionφ? test condition φ(δ1; δ2) sequenceif φ then δ1 else δ2 conditionalwhile φ do δ′ loops(δ1|δ2) non-deterministic choiceπv .δ non-deterministic choice of argumentδ∗ non-deterministic iteration{P1(~t1, δ1); . . . ;Pn(~tn, δn); δ} procedures

ConGolog: (in addition)(δ1 ‖ δ2) concurrent execution(δ1 〉〉 δ2) prioritized concurrencyδ‖ concurrent iteration

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 38 / 71

Golog and ConGolog: Syntax

Golog:a primitive actionφ? test condition φ(δ1; δ2) sequenceif φ then δ1 else δ2 conditionalwhile φ do δ′ loops(δ1|δ2) non-deterministic choiceπv .δ non-deterministic choice of argumentδ∗ non-deterministic iteration{P1(~t1, δ1); . . . ;Pn(~tn, δn); δ} procedures

ConGolog: (in addition)(δ1 ‖ δ2) concurrent execution(δ1 〉〉 δ2) prioritized concurrencyδ‖ concurrent iteration

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 38 / 71

Golog and ConGolog: Syntax

Golog:a primitive actionφ? test condition φ(δ1; δ2) sequenceif φ then δ1 else δ2 conditionalwhile φ do δ′ loops(δ1|δ2) non-deterministic choiceπv .δ non-deterministic choice of argumentδ∗ non-deterministic iteration{P1(~t1, δ1); . . . ;Pn(~tn, δn); δ} procedures

ConGolog: (in addition)(δ1 ‖ δ2) concurrent execution(δ1 〉〉 δ2) prioritized concurrencyδ‖ concurrent iteration

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 38 / 71

ConGolog: Semantics

The semantics is defined in the situation calculus, a logical language forreasoning about action and change. Details are in [Reiter, 2001].

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 39 / 71

ApproachOverview + Contributions

Search ControlDomain Definition+Goal

Domain Definition+Goal

Objective: enable search control in state-of-the-art planners.

Approach: compile search control into domain definition.

Fritz, Baier, McIlraith: ConGolog, Sin Trans 5 / 25

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 40 / 71

Approach

Overview + Contributions

HTNs∗

ConGolog Basic Action Theories (BAT)

ConGolog with restrictions

Golog w/o proceduresPDDL

Objective: enable search control in state-of-the-art planners.

Approach: compile search control into domain definition.

Add procedures and concurrency: ConGolog → BATs.

Identify a subset of ConGolog that can be compiled to PDDL.

Can compile ∗simple HTNs into classical planning.

Fritz, Baier, McIlraith: ConGolog, Sin Trans 5 / 25Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 41 / 71

Planning with Golog-like Procedural Control

Compile Golog into PDDL using ε-PNFA, using additional book-keepingactions.

Golog in any PDDL Planner (ICAPS-07)

Golog program

while φ do [if ψ then a else b endif] endwhile

PNFA for the control

q1 q2

q3 q4

q5 q6

q7 q8test(φ) tes

t(ψ)

test(¬ψ)

a

b

noop

noop

test(¬φ)

noop

Automaton’s dynamic in PDDL

New predicate: (aut-state x)

New effects to update foraut-state

New goal/initial state

Experiments/Applicability shown in our ICAPS-07 paper

Baier et al. (UofT): Procedural Control and Preferences State of the Art Planners 11 / 21

[Baier, Fritz, M, ICAPS07]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 42 / 71

Planning with Golog-like Procedural Control

Main Results:

1 Golog-based procedural control language for Planning

PDDL-based semantics for the language

2 Compilation of Golog-like control into classical planning

Theorem: Translation is correctTheorem: Translation is polynomial

3 Experimental analysis showing that underconstrained controlprovides superior performance

[Baier, Fritz, M, ICAPS07]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 43 / 71

The Rest of the Talk

Planning for LTL Temporally Extended Goals (TEGs)

Planning with LTL Temporally Extended Preferences (TEPs)

Planning with restricted RE’s – Golog-like Procedural Control

Planning with RE’s plus concurrency – ConGolog Procedural Control

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 44 / 71

Extending the Compilation to ConGolog (Sometimes)

Next we added:

concurrency: being in more than one state simultaneously;

procedures: requires call stack.

Approach:

consider ConGolog program as a Petri Net,

represent this in basic action theory in the situation calculus.

identify a subset of ConGolog that can be compiled to PDDL

[Fritz, Baier, M, KR08] [Christian Fritz, Thesis 2009]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 45 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 46 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 47 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 48 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 49 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 50 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 51 / 71

Compilation: Approach

ConGolog:

(if business then bookHotel(Marriott) else bookHotel(DaysInn)

‖ bookFlight)

Petri Net:

1

2

3

4

5

6

7

8

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 52 / 71

Compilation: Approach

1

2

8

3

5

4

6

7

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Basic Action Theory: (excerpt)

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 53 / 71

Compilation: Approach

1

2

8

3

5

4

6

7

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Basic Action Theory: (excerpt)

Poss(spawn(t, 2, 8), s) ≡ state(t, s) = 1

state(0t, do(spawn(t, 2, 8), s)) = 2

state(1t, do(spawn(t, 2, 8), s)) = 8

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 54 / 71

Compilation: Approach

1

2

8

3

5

4

6

7

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Basic Action Theory: (excerpt)

Poss(spawn(t, 2, 8), s) ≡ state(t, s) = 1

state(0t, do(spawn(t, 2, 8), s)) = 2

state(1t, do(spawn(t, 2, 8), s)) = 8

Poss(test(t, 2, 3), s) ≡ state(t, s) = 2 ∧ business(s)

state(t, do(test(t, 2, 3), s)) = 3

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 55 / 71

Compilation: Approach

1

2

8

3

5

4

6

7

9

10

business

¬business

bookHotel(Marriott)

bookHotel(DaysInn)

bookFlight

noop

noop

spawn join

Basic Action Theory: (excerpt)

Poss(spawn(t, 2, 8), s) ≡ state(t, s) = 1

state(0t, do(spawn(t, 2, 8), s)) = 2

state(1t, do(spawn(t, 2, 8), s)) = 8

Poss(test(t, 2, 3), s) ≡ state(t, s) = 2 ∧ business(s)

state(t, do(test(t, 2, 3), s)) = 3

Poss(bookFlight(t), s) ≡ state(t, s) = 8 ∧ ΠbookFlight(s)

state(t, do(bookFlight(t), s)) = 9← state(t, s) = 8

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 56 / 71

Compilation: Theorems

Theorem (Correctness)

omitted.

Theorem (Succinct)

The new theory is of size O(n2), where n is the size of the program.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 57 / 71

Compilation: Challenges

Procedures:New fluent: stack(i , r , s): stack of “return addresses”.

Program variables:New fluent map(v , x , s): “variable v has value x”.

Concurrency:Actions get additional thread argument.New actions spawn and join.

Prioritized concurrency:New fluent Prio(t2, t2, s): “thread t1 has priority over t2”.

Synchronized if’s and while’s.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 58 / 71

Previous Experimental Results

Storage Domain: (log scale)

0.01

0.1

1

10

100

1000

4 6 8 10 12 14 16 18 20 22 24

seconds

testcase

original

dumb

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 59 / 71

HTNs → ConGolog

unordered: Move2(p1, p2, s, d)

Move1(p1, s, d) Move1(p2, s, d)

→ proc Move2(p1, p2, s, d) : Move1(p1, s, d) ‖ Move1(p2, s, d)

ordered: Move1(p, s, d)

load(p) drive(s, d) unload(p)

→ proc Move1(p, s, d) : load(p); drive(s, d); unload(p)

Several methods m1, . . . ,mn for decomposing a task Task :→ proc Task : m1| . . . |mn

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 60 / 71

Recap

AI planning can be leveraged to improve the generate:

plans with LTL Temporally Extended Goals

plans/Golog programs LTL Temporally Extended Preferences

Plans with Golog-like Procedural Control

...and other things not discussed.

We took a reformulation approach:

Its output is in most cases usable with a wide range of planners

It is composable

Sheds light on the construction of new heuristics

It is applicable to other types of reasoning

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 61 / 71

Recap (continued)

Formally analyzed the properties of translations/algorithms:

Theorems on the correctness of translations,

Size bounds,

Conditions for optimality,

...

Showed significant experimental advantages of the approach

Orders-of-magnitude over state of the art

Superior/competitive with state of the art

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 62 / 71

Recap (continued)

Contribution: A method for compiling ConGolog search control intodomain definitions (e.g. Basic Action Theories, PDDL).

Merits:

Planning:

Rich search control in any state-of-the-art planner:

ConGolog (with restrictions),

Simple HTNs.

KR:

Makes ConGolog first-order definable.

Avoids need for program reification.

Facilitates automated reasoning about programs.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 63 / 71

Take Home Message

Many tasks that involve reasoning about dynamical systemscan be cast as non-classical planning problems and solvedeffectively using state-of-the-art planning techniques.

A key enabler for at least 101 of these problems is therestatement of the objective in terms of LTL, RegularExpressions (Golog) and/or Automata.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 64 / 71

Related Work

LTL/Control Knowledge

LTL Search Control for Planning (Bacchus & Kabanza, AIJ 2000)

Compiling limited subset of LTL to PDDL (Rintanen 2000)

Compiling propositional LTL to PDDL (Cresswell & Codington, ECAI2004)

LTL Search Control for Planning (Doherty et al., 2000)

LTL Preferences (Son & Pontelli, LPNMR 2004, TPLP 2006)

Extension of PDDL to incorporate LTL (Gerevini, Haslum, Long, et al2005 & AIJ 2009)

Precondition Control (Gabaldon IJCAI 2003, KR 2004)

Finite LTL + Regular Expressions (De Giacomo & Vardi, IJCAI 2013)

...

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 65 / 71

Related Work (cont.)

PDDL3 Planners

HPlanP (Baier, Bacchus, M, 2006, AIJ 2009)

MIPS-XXL (Edelkamp et al, 2006)

MIPS-BDD (Edelkamp, 2006)

SGPlan (Hsu et al., 2007)

HPlan-P (Baier, Bacchus, M, 2006, AIJ 2009)

LPRPG-P - (Coles & Coles ICAPS 2011, & Benton ICAPS 2012)

... ?

Planning with Golog Search Control

Planning with Golog (Klassen, Eyerich, Lakemeyer & Nebel, IJCAI2007)

First-order characterization of Golog and ConGolog (Lin, KR 2014)

Nebel & Lakemeyer & Roeger work on Golog and planning (various)

...

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 66 / 71

My Personal Journey in Academia

Hon BSc (Pure & Applied Math - Prob & Stats) [U.Ottawa]

In my last 2 years of ugrad, worked for my Stats prof, at thecomputer help centre, and as a CS Teaching Assistant

Worked developing software for the oil & gas sector (1.5 yrs)

MSc (Computer Science) [U.Waterloo]

Worked for an applied research lab in Canada (3 years)

PhD (Computer Science) [U.Toronto], continuing p-t work at lab

Postdoc [joint Xerox PARC & Stanford]

(Senior) Research Scientist [Stanford]

Associate Professor → Professor → Vice-Chair → Professor[U.Toronto]

...and married with 2 children along the way!

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 67 / 71

Questions?

[Baier2010] [Fritz2009] [Baier and McIlraith2006] [Baier et al.2009]

[Sohrabi et al.2009] [Albarghouthi et al.2009] [Baier et al.2007]

[Sohrabi et al.2006] [Sohrabi et al.2010] [Baier and McIlraith2008]

[Razavi et al.2014] [Razavi et al.2011] [McIlraith et al.2001]

[Bienvenu et al.2011] [Baier et al.2008] [Pnueli1977] [Reiter2001]

[Sohrabi and McIlraith2010] [Bylander1994]

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 68 / 71

References I

Aws Albarghouthi, Jorge A. Baier, and Sheila A. McIlraith.

On the use of planning technology for verification.In Proceedings of the Validation and Verification of Planning and Sched. Systems Workshop (VVPS), Thessaloniki, Greece, 2009.

J. Baier and S. McIlraith.

Planning with first-order temporally extended goals using heuristic search.In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI06), pages 788–795, Boston, MA, July 2006.

Jorge A. Baier and Sheila A. McIlraith.

Planning with preferences.Artificial Intelligence Magazine, 29(4):25–36, 2008.

Jorge A. Baier, Christian Fritz, and Sheila A. McIlraith.

Exploiting procedural domain control knowledge in state-of-the-art planners.In Proceedings of the 17th International Conference on Automated Planning and Scheduling (ICAPS), pages 26–33, Providence,Rhode Island, USA, September 22 - 26 2007.

Jorge A. Baier, Christian Fritz, Meghyn Bienvenu, and Sheila McIlraith.

Beyond classical planning: Procedural control knowledge and preferences in state-of-the-art planners.In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), Nectar Track, pages 1509–1512, Chicago, Illinois,USA, July 13–17 2008.

Jorge A. Baier, Fahiem Bacchus, and Sheila A. McIlraith.

A heuristic search approach to planning with temporally extended preferences.Artificial Intelligence, 173(5-6):593–618, 2009.

Jorge A. Baier.

Effective Search Techniques for Non-Classical Planning via Reformulation.Ph.D. in Computer Science, University of Toronto, 2010.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 69 / 71

References II

Meghyn Bienvenu, Christian Fritz, and Sheila A. McIlraith.

Specifying and computing preferred plans.Artificial Intelligence, 175(7–8):1308–1345, 2011.

Tom Bylander.

The computational complexity of propositional STRIPS planning.Artificial Intelligence, 69(1-2):165–204, 1994.

Christian Fritz.

Monitoring the Generation and Execution of Optimal Plans.PhD thesis, University of Toronto, April 2009.

Sheila A. McIlraith, Tran Cao Son, and Honglei Zeng.

Semantic web services.IEEE Intelligent Systems. Special Issue on the Semantic Web, 16(2):46–53, March/April 2001.

Amir Pnueli.

The temporal logic of programs.In Proceedings of the 18th Annual Symposium on Foundations of Computer Science (FOCS), SFCS ’77, pages 46–57, 1977.

Niloofar Razavi, Azadeh Farzan, and Sheila A. McIlraith.

Predicting atomicity violations in concurrent programs via planning.In Proceedings of the ICAPS-11 Workshop on Verification and Validation of Planning and Scheduling Systems (VVPS), 2011.

Niloofar Razavi, Azadeh Farzan, and Sheila A. McIlraith.

Generating effective tests for concurrent programs via AI automated planning techniques.International Journal on Software Tools for Technology Transfer (STTT) (STTT), 16(1):49–65, 2014.

Raymond Reiter.

Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems.MIT Press, Cambridge, MA, 2001.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 70 / 71

References III

Shirin Sohrabi and Sheila A. McIlraith.

Preference-based web service composition: A middle ground between execution and search.In Proceedings of the 9th International Semantic Web Conference (ISWC-10), pages 713–729, Shanghai, China, November 2010.

Shirin Sohrabi, Natasha Prokoshyna, and Sheila. A. McIlraith.

Web service composition via generic procedures and customizing user preferences.In Proceedings of the 5th International Semantic Web Conference (ISWC), pages 597–611, Athens, Georgia, 2006.

Shirin Sohrabi, Jorge Baier, and Sheila A. McIlraith.

Htn planning with preferences.In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), pages 1790–1797, Pasadena,California, 2009.

Shirin Sohrabi, Jorge Baier, and Sheila A. McIlraith.

Diagnosis as planning revisited.In Proceedings of the 12th International Conference on Knowledge Representation and Reasoning (KR), Toronto, Canada, 2010.To appear.

Sheila A. McIlraith (UofT): Of Plans, Programs, and Automata 71 / 71