Continuous Planning and Execution

Dr. Karen Myers (PI)

Mr. David Blei

Mr. Thomas J. Lee

Dr. Charlie Ortiz

Dr. David E. Wilkins

Artificial Intelligence Center

SRI International

333 Ravenswood Avenue

Menlo Park, CA 94025

Project URL http://www.ai.sri.com/~cpef/

SRI International

SRI International

Domain Characteristics

• Tasks are complex and open-ended

• Operating environments are dynamic and possibly hostile

• Complete and accurate knowledge of the world can never be attained

• Full automation is neither possible nor desirable

Successful operation requires a mix of

» user involvement and control

» continuous planning

» rapid response to unexpected events

» dynamic adaptation of activities

SRI International

CPEF Foundations

• leverage several mature SRI technologies

– Procedural Reasoning System (PRS)

» Knowledge-based Reactive Control system

– SIPE-2: Hierarchical Task Network (HTN) planner

– Advisable Planner (AP)

– Multiagent Planning Architecture (MPA)

Functional integration: more than just interfaces ...

SRI International

Technical Thrusts

A. Flexible Process Management

– provide intelligent management of planning and execution that is responsive to the dynamics of the operating environment

B. Dynamic Plan Adaptation

– provide situation monitoring, execution monitoring, and plan repair that enable reactive, timely adaptation of plans

C. Robust Plans

– generate plans that are sensitive to the execution environment, knowledge limitations

D. User Guidance

– enable users to direct and manage key aspects of the planning and execution processes

SRI International

CPEF Architecture

Plan Manager

PRS

Planner

AP/SIPE

Simulator

PRS

Repair

Requests

Repair

Info

Plan

Requests

Plan Info

Situation

Updates

Execution

Status

Execution

Requests

Updates,

Guidance

Plan Server

PRS

Interface

PRS

Plan Repair

PRS

Notifications,

Requests

Plans

PlansPlans

MPA Messages

SRI International

Infrastructure: Multiagent Planning Architecture (MPA)

• Builds on components from the Multiagent Planning Architecture (MPA)

– Agent-based framework for addressing large-scale planning problems

– distributed operations

– modularity via plug-and-play paradigm

– Protocols related to plans and planning activities

» layered on top of KQML

– Plan Server for storage/retrieval of plans and plan fragments

• Extensions to MPA for plan execution

– class of Executor agents

– protocols for plan execution, repair, updates, advice

SRI International

Plan Manager: Responsibilities

Execution Tracking

– Supervise progress through execution of plans

Knowledge Management

– Situation monitoring

– Perform information-gathering tasks

Process Management

– Control generation of plans and options for outstanding tasks

– Provide timely response to user requests, unexpected events

» Reactive response to unexpected events

EX: downed pilot in Area of Operations

» Runtime adaptation of plans in response to failures, events

EX: pop-up targets, change in weather

SRI International

Plan Manager Design

• PRS reactive control technology

– Multi-threaded, highly responsive

– Mixture of goal-directed and event-directed activity

• Execution tracking via Flow Model

– Await outcomes in accordance with sequencing info in plans

Outcomes: success, failure, unknown, time-out

– More opportunistic models would be preferable

SRI International

PRS Reactive Control Architecture

Plan: partial order of goals

Controller: manages procedure application in accordance with Plan and New Events

Monitors: checking for critical events

Database: dynamically maintained knowledge of the real world

Procedures:

– goal refinement

– reactive responses

Controller

Plan Procedures

MonitorsDatabase

User

World

SRI International

Monitors

Event change in the world state

– action completes, new info, time passes

Response several possibilities:

1. Invoke standard operating procedures

2. Perform plan transformations (e.g., plan repair, plan extension)

3. Record changed world model

Monitor Classes

Failure Monitors respond to failures that occur during plan execution

Knowledge Monitors test for availability of info needed for decision-making

Assumption Monitors respond to situation changes that violate key plan assumptions

SRI International

Automated Monitor Generation

• Create assumption monitors with a range of possible responses from formal plan representation

– alerts, plan repairs, standard operating procedures

• Traversal of causal links within plan derivation structure to collect conditions/assumptions that are:

– Dynamic

– Not established by earlier actions in the plans (ie, in initial world)

– Declared as significant

» Certain violations can be disregarded until entry into a critical time window (Ex: weather)

PLAN = Actions + Monitors

SRI International

Plan Repair

• Perform minimal-perturbation replanning for impacted portions of current plan

– wedge beneath ‘failure nodes’

– minimal changes provide plan continuity, understandability

» can be computationally expensive

Planning-time

– Adaptation of plan in response to information updates

Asynchronous Execution-time

– Adaptation of active plan during execution

» world continues to change, unaffected actions are executed

– Plan Manager must synchronize new plan with continued progress along previous plan

SRI International

ACP Knowledge Bases

Support Packages

Targets

Strikes

Air Objectives

CTEM

Planner

Planner

• Strategy-to-Task refinement of selected Air objectives

• Final plans at the level of targets, CAPS (with several thousand nodes)

• Key Components

– strategic knowledge for objective refinement

– hierarchical target network models

– threat models

– geographic knowledge

– force and equipment knowledge

• Assumes key intel info: COGs, threats

Scope of TIE-97-1 Air Campaign Planning (ACP)

Knowledge Base

EXECUTOR

PLANNER

USER

t=0

Generate

Plans and

Monitors

Execution

Begins

Integrate

New Plan

Specify

Objectives,

Guidance

Request

Plan Repair

Repair

Plan

Repair

Plan

Report of

Downed

Pilot

Intel

Report

Dispatch

Rescue

Mission

Reports of

Unsuccessful

Missions

Critique

Plan

Situation Monitoring

Execution Tracking

Process Management

Request

Plan Repair

CPEF Demonstration Scenario: Air Campaign Planning

SRI International

CPEF Demo: Technical Highlights

• Rapid generation of alternative courses of action

– Strategy-to-task refinement of Air Superiority objectives

• Incremental generation of qualitatively different options by user (via advice)

• Application of automatically- and user-created monitors

• Realtime Execution Tracking in a simulated environment

• Asynchronous adaptation of activity in response to realtime monitoring of

– situation changes (Ex: Downed Pilot)

– plan execution results (Ex: failure to neutralize critical targets)

• Advised Plan Repair

SRI International

Conclusions

• CPEF Prototype demonstrates flexible process management for living plans in Air Campaigns

– Full Spectrum: plan generation, execution, repair

• Several Notable Technical Accomplishments

– Models for Plan Management, Execution Tracking

– Generalized Failure Models, Repair Methods

– Promising preliminary work on Open-ended Planning

• Major Contributor within the JFACC Program

Process Management

Plans

Air Campaigns

CPEF

SRI International

Backup Slides

SRI International

Execution Models

Direct Execution (Do it!)

– actions are dispatched directly by the system

» EX: controller for a mobile robot

Indirect Execution (Supervisory)

– plan is executed by diverse, distributed agents

– agents are pre-assigned execution tasks

– status of action execution is not directly available

– delays in redirecting agents that perform planned actions

– time lag on receipt of information about the world

SRI International

Generalized Failure Models

• Limited scope of current models:

Precondition Failure action precondition not satisfied

Action Failure intended effects not achieve

Maintenance Failure established condition no longer maintained

• New directions --- beyond plan dependency structures

Unattributable Failure no individual action has failed or assumption violated yet plan is deemed inadequate

» Ex: CA indicates failure to establish sufficiently strong breach of IADS

Aggregate Failure require collections of failures, possibly with key relationships among them (eg, A fails then B fails)

» Ex: key subset of a target network

SRI International

PRS Control Loop

Execution Cycle

1. New information arrives that updates facts and goals

2. Acts are triggered by new facts or goals

3. A triggered Act is intended

4. An intended Act is selected

5. That intention is activated

6. An action is performed

7. New facts or goals are posted

8. Intentions are updated

Goal2ACT8

sleeping

Fact1ACT2normal

Goal3ACT3

sleeping

Intention Graph

Cue: (TEST (overpressurized tank.1))

ACT2

Act Library

Act Execution

(overpressurized fuel-tank)

(ACHIEVE (position ox-valve closed))

New Facts & Goals

ExternalWorld

1

2

3

4

5

6

7

8

Cue: (ACHIEVE (position valve.1 closed))

ACT1Facts&

Goals

(ACHIEVE (position ox-valve closed))

ACT1current

SRI International

CPEF: Continuous Planning and Execution Framework

• process management technology for living plans

– plan creation, execution, repair

• vertical slice of the JFACC system

Process Management

Living Plans

Air Campaigns

Workflow Management

Plan Gen

Specialized Components

JFACC System

CPEF

Layered View of CPEF

SRI International

Process Management: Generality and Ubiquity

CPEF(JFACC, SUO)

Process Management

Process Management

Plans

Air Campaigns

Process Management

Info Needs

ISR

SWIM(AIM)

TRAC(CoABS)

Agents

SRI International

Accomplishments: Technical

• Process Management for plan generation, indirect execution, monitoring, repair

• Automated extraction of monitors from plans

• Generalized models of failure and execution monitoring

• Mixed-initiative options generation and plan repair (using advice)

• Preliminary models for open-ended planning

• “Towards a Framework for Continuous Planning and Execution”, AAAI 1998 Fall Symposium on Distributed, Continual Planning (Special Issue of AI Magazine)