Download - 9/10/98USC-ISI / ASTT IPR1 Flexible Group Behavior Randall Hill, USC-ISI Jonathan Gratch, USC-ISI ASTT Interim Progress Review September 10, 1998.

9/10/98 USC-ISI / ASTT IPR 1

Flexible Group Behavior

Randall Hill, USC-ISI

Jonathan Gratch, USC-ISI

ASTT Interim Progress Review

September 10, 1998


Hypotheses

• The key to flexible behavior is handling situation interrupts– Understand the nature of the situation and adjust behavior appropriately– Achieve goals in spite of unexpected obstacles

• Flexible group behavior requires the ability to:– Understand behavior of groups of other agents

• Maintain situation awareness of friendly and adversarial groups

• Recognize when a situation does not match expectations

– Plan a mission for groups against groups• Collaborate with peers and superiors

• Perform adversarial reasoning

– Execute mission plan in a coordinated manner• React to situation interrupts, as a team

• Repair plans, when necessary, and continue executing mission


Hypotheses (2)

• Flexible group behavior interleaves the processes of situation awareness, planning, and execution– Requires concurrent, continuous integration of each process

– Spans individuals, groups, and echelons

– Affects how plans are generated and repaired

– Need methods for integrating the component processes

• Group behavior requires a theory of multi-agent interaction– Provides a framework for understanding others’ behavior

– Addresses issues of authority & autonomy

– Enables collaborative & adversarial reasoning

– Provides protocol for coordination and communication


Towards Flexible Group Behavior

Understand

Execute

Plan Execute

Understand

Understand Plan Execute Battalion

Company

Learn

Entity


State of the Art

• Current research on flexible group behavior is fragmented

• Group Understanding & Execution– Teamwork model

• Developed for RWA-Soar and participated in STOW-97 ACTD– Attack Helicopter Company + Command entity

– Marine transport / Marine escort teams

• Executes plans with reactive behavior

• Understands own team’s roles, activities, and goals

• Provides coordination and communication protocol during execution

– Limitations of teamwork model• Does not understand other groups’ behavior

• Does not perform deliberative planning

• Does not reason about task dependencies when resolving conflicts


State of the Art

• AI planning systems– Hierarchical task network / partial order planners (IPEM,X11,DPLAN)

• Generate, execute, repair plans

• Example: Soar-CFOR RWA company command entity

• Not designed for multi-agent, collaborative planning

• Multi-agent planning– Joint Intentions / Shared Plans (Cohen & Levesque; Grosz & Kraus)

• Focuses on collaboration and reasoning about intentions

• Lacks situation awareness, execution, repair, authority, adversarial planning.

– Generalized Partial Global Planner (Lesser & Decker)• Focuses on coordination strategies and collaborative scheduling

• Lacks notion of authority, adversarial planning


Approach

• Build on Soar-CFOR model of planning, execution & repair– Need more flexibility in responding to situation interrupts

– Need to extend model to higher echelons

• Develop more situation awareness– Detailed awareness of commander’s intent

– Detailed awareness of other friendly activities

– Detailed awareness of enemy situation

– Constant tracking and updates of friendly and adversarial forces


Approach (2)

• Develop a more flexible approach to planning– Take advantage of enhanced situation awareness

– Enable multi-agent reasoning• Collaboration

• Authority

• Adversarial reasoning

– Implement as a meta-reasoning capability on top of standard planner

• Don’t re-invent technology– Uses a standard planning paradigm

– Builds on existing systems (RWA-Soar and Soar-CFOR)

– The meta-reasoning layer integrates fragments of related research


Technology and R&D

• Architecture

• Situation Awareness

• Planning

• Situation Interrupts and Planning

• Multi-agent Planning


System Architecture

Battalion Commander

Company ACommander

Company XCommander

Company A

PilotHelicopter

PilotHelicopter

PilotHelicopter

ModSAF

Company X

PilotHelicopter

PilotHelicopter

PilotHelicopter

….

….

Operations Order(plan)



Situation Report(understanding)



PerceptsActions PerceptsActions


Command Agent Architecture

• Understanding– Perceive and report groups of enemy entities

– Understand plans from superiors/subordinates

– Monitor execution of plans by subordinates

• Planning– Generate plans (collaboratively) at battalion and company levels

– Battalion commander resolves conflicts in company plans

• Execution– Battalion executes plans

– Repair plans when situation interrupt occurs


PlansPlans

Command Agent Architecture

Environment

Current Situation

DomainTheory

DomainTheory

Planner

Situation Awareness

Radio Vision

Radio Platform Commands

Expectations


Situation Awareness

• Hide information gathering details from Planner

• Derives consolidated picture of current situation from:– Radio reports (via 16 CCSIL message types)

• OpOrders, SitReps, Status Reps, Replacement Reqs, Flight Advisory, BDA, Request Passage Coordination, etc...

– Vehicle Sensors (via MITRE CFOR platform services)

– Expectations• expected enemy contact (derived from OpOrder)

• frequency of subordinate Status Reps

• Rule-based reasoning

• Can perform limited sensing actions– e.g.. Request situation reports


Situation Awareness Output

• List of facts currently true in the world– 16B11 at holding_area ha11

– 16B14 presumed dead

– Enemy ADA platoon threatening battle_position bp141

– Target in EA nelson has been attritted

– I’ve communicated order76 to 16C11

– I’ve received new orders from my commander

• Facts are echelon and unit type specific– Battalion tracks different information than company

– CSS unit tracks different information than RWA unit

– Determined by domain theory


Planner

• Implements basic command and control functions– Generates plans

– Controls execution and coordination of subordinates

– Recognizes Situation Interrupts and makes repairs

• GIVEN: – Domain theory (tasks, plan fragments, assets)

– Mission objectives, friendly/enemy plans (from OPORDER)

– Existing plans

– Current situation (from Situation Awareness)


What are Plans?

• Plans are partially ordered sequences of tasks

• Plans capture assumptions– Column movement assumes enemy contact unlikely

• Plans capture dependencies between task – Move_to_Holding_Area results in unit being at the HA, which is a

precondition to moving to the Battle_Position

– Enemy and Company must be at the Engagement_area at the same time

• Different plans associated with different groups– Battalion plan, company plan, enemy plans


Planning Basics

• Plan generation– Sketch basic structure via decomposition

– Fill in details with causal-link planning

• Plan execution– Explicitly initiate and terminate tasks

– Initiate tasks whose preconditions unify with the current world

– Terminate tasks whose effects unify with the current world


Example of Plan Generation

Destroyed(Enemy)

Attack(A, Enemy)

Move(A,BP) Engage(A,Enemy)

at(A,BP)at(A,FARP)

at(Enemy,EA)

at(A,BP) Destroyed(Enemy)

Destroyed(Enemy)

at(A,FARP)

at(Enemy,EA)

Current World

. . .

init


Situation Interrupts Happen!

• What if the world changes unexpectedly?– Agents exist for extended time periods

– Plans and goals may change over time

– Actions have duration and may fail

– Environment may change unexpectedly

– Other agent may take unanticipated action

• To the planner this means– Effect of an action in the plan is deleted even though no task deletes it– Effect of an action is added even though no task adds it

• Need method for handling situation interrupts


Example

destroyed(Enemy)

Attack(A, Enemy)

Move(A,BP) Engage(A,Enemy)

at(A,BP)at(A,FARP) at(A,BP) destroyed(Enemy)

destroyed(Enemy)

at(A,FARP)

at(Enemy,EA)

Current World

active(A)

….

init

Unexpected Event

-active(A)

active(A)active(A)


Reacting to Situation Interrupt

• Situation interrupt may force retraction of plan decisions– Example:

• Bn commander generates plan sending Company A to engage enemy tanks

• Company A is destroyed on way to the battle position

• Movement tasks of Company A are now invalid and must be retracted.

• Plan dependencies capture the ramifications of interrupts

• Repair plans in response to ramifications– Retract steps

– Add new steps

– Add additional constraints


Multi-Agent Planning

• Want to plan in context of other agents

• Want to explicitly reason about their plans

• Want to plan in a “socially appropriate” manner– Treat my friends with respect (collaborate or avoid negative interactions)

– Treat my enemies with contempt

– Understand what I am required to do

– Understand where I have discretion to make decision

– Understand who I have authority to command


Multi-Agent Planning

• Must represent goals and activities of other agents

• Must reason about interactions / conflicts across agents

• Cannot treat all plans equally– Must understand your relationship to other agents during planning

– Modulate behavior of planner based on these relationships


Interaction Example

Move(A,BP) Engage(A,Y)

Dead(Y)

Move(CSS,HQ)

at(CSS,HQ)at(CSS,FAA)

at(gas,FAA) at(gas,HQ)

at(A,BP)at(A,FAA) at(A,BP)

at(gas,FAA)

Init

ial S

tate

Combat Service Support Plan

Attack Helicopter Company Plan

resupplied(HQ)


Planning Stances

• Authoritative: modify/generate other agents’ plans – Tell CSS to abandon re-supply operation

• Subordinate: use other’s plan to defer to their actions– Find a way to work around re-supply activity

• do I have time to do the engagement first?

• Collaborative: use other’s plan to reach mutual solution – Consult to see if some other resource could achieve the resupply

• Adversarial: use other’s plan against them– Try to introduce threats into other agent’s plan


Coordination and Control

• Joint commitments– Commander develops and communicates a plan

– Commander must be committed to maintaining the plan • Don’t make capricious changes

– If plan must be changed, need to inform affected units

• Framing effects – Commander has one plan approved by his commander

– Commander given subsequent mission to plan

– Use the first plan to “frame” the second• Consider the first plan fixed

• Develop second plan in the context of the first


Approach

• Add domain-independent theory of multi-agent interaction– Represent stances, authority relationships, joint commitments

• Add multi-agent controller that modulates general planner

general purposeReasoner(Planner)

MultiAgentController

Meta PlansMeta Plans

Base PlansBase Plans

Meta Domain Theorydomain independent

Meta Domain Theorydomain independent

Base Domain Theory

Base Domain Theory


Plan Properties

• Associate properties with plans

• Multi-agent controller uses the properties as input

• Example: Modifiability– If a plan is unmodifiable, the planner cannot add or retract constraints to

the plan in response to flaws

– Unmodifiability is a component of modeling authority• my bosses plans should be unmodifiable by me


Interaction Example

Move(A,BP)

Move(CSS,HQ)

at(CSS,HQ)at(CSS,FAA)

at(gas,FAA) at(gas,HQ)

at(A,BP)at(A,FAA)

at(gas,FAA)

Init

ial S

tate Pla

nn

er

Retract

Retract

unmodifiable


Plan Properties

• Executability– Can I initiate tasks in this plan?

• Threat resolution– If a decision in one plan introduces a threat in another, is this ok?

• Role assignments– Tasks have an agent variable that planner can assign

• If I’m helpful I can add myself to tasks in your plan

• If I’m authoritative I can add you to tasks in my plan

• Commitment– Plans can be either conjectured or committed– Two conjectured plans can’t create flaws in each other

• Allows us to consider alternative courses of action

• Allows us to represent proposed changes to other plans


Meta Domain Theory

• Theory of authority– Battalion commander has authority over Company commander

– Therefore• Co cannot modify Bn’s plans; Bn can change Co’s

• Co’s plans must defer to Bn’s plans

• Bn can assign Co to tasks; Co can’t assign Bn to tasks


Meta Domain Theory (2)

• Military Decision Making Process (MDMP) modeled as meta domain theory

COADevelopment

COAExecution

Create a plan free of flawsDon’t execute it until approved

Run the plan till completionassuming there are no unpredicted flaws

COAAnalysis

Take adversarial stance toward own plans(war gaming)


• Meta tasks change the properties of other plans

• Executing meta-tasks changes state of multi-agent controller

Meta-tasks

COA Development

-Flawed(p)Goal(g)

-Modifiable(p)-Executable(p)

Plan-for(g, p)Situation Interrupt

Flawed(p) -CommonlyKnown(p)-Flawed(p) Modifiable(p)


Summary

• Reason about activities of other agents– Represent plans as subsets of planning decisions

• Tracks flaws across individuals’ activities– Uses standard threat detection

• Varies the way flaws are treated– Via plan properties

• Reasons about common knowledge– Also via plan properties

• Interleaves planning, execution, repair


Measures of Success

• Collective Measure:– Ability of a group of entities (RWA Battalion) to achieve mission

objectives in scenarios containing a wide range of situation interrupts

• Individual Measures– Scalability: size of groups that can act autonomously

– Flexibility: classes of situation interrupts handled by group behavior• effectiveness of situation awareness, planning, execution, plan repair

– Types of multi-agent reasoning integrated into framework• i.e., collaborative, adversarial, temporal, ...

– Breadth and depth of domain knowledge• e.g., # of tasks, echelon levels, functional categories (battlefield operating sys)

– ...


Evaluation Issues

• Need a systematic evaluation framework– Automatic scenario generator

– Library of situation interrupts

– How do we evaluate what happened?

– Can situations be partitioned into classes?


Significant Results to Date

• Implemented RWA battalion commander

• Added a combat service support company to battalion

• Improved situation awareness– Enhanced pilot’s perceptual capabilities to handle larger groups

– Implemented awareness of enemy situation at company/battalion

– Improved tracking of subordinates

• Reorganized company/battalion cdr’s domain theory– Task hierarchy now matches with Logicon documentation

• Made C2 reasoning more accessible to user– Visualization tools

– Built-in links from domain theory & plans to documentation

Scope of Task Coverage

ATKHB Attack MissionAchieve Tactical

Disposition

Reduce Enemy Posture

Achieve Culminating

Task

Consolidate

1-4-1101: Personnel (S1) planning (C2)1-4-1201: Intelligence (S2) planning (C2)1-4-1301: Operations (S3) planning (C2)1-4-1401: Logistics (S4) planning (C2)1-4-1302: Establish and maintain

tactical operations center (C2)1-4-1305: Coordinate maneuver with

CSS and rear ops (C2)---------------------------------------------------1-2-0320: Provide supply support (CSS)1-2-7723: Perform maintenance (CSS)1-2-7728: Process ammo and fuel (CSS)1-4-1103: Replacement operations (CSS)1-4-1402: Coordinate supply/equip. (CSS)1-4-1405: Plan and coordinate transport

assets (CSS)

Achieve Readiness

1-3:0001: Plan and organize move (Mnv)1-2-0101: Move to and occupy assembly

area (Mnv)1-4-1306: Establish and maintain tactical

command post (C2)1-2-7726: Conduct FARP operations (CSS)

Achieve Physical Posture

1-4-1305 (Section 6.1.2): Integrate fire support

Attack (METL task) 1-4-1206:

1-2-xxxx: Establish satellite comm. (C2)1-2-xxx0: Establish ground comm (C2)1-2-7509: Establish voice comm (C2)11-5-0104: Establish FM radio (C2)1-4-1001: Perform C2 operations (C2)1-4-1303: Control tactical operations (C2)------------------------------------------------------------1-4-1202: Implement security measures (Int)1-4-1203: Process intelligence information (Int)1-4-1311: Liaison operations (Int)------------------------------------------------------------1-4-1105: Provide admin services (CSS)1-2-7708: Provide food support (CSS)1-2-7710: Operate field mess (CSS)1-2-7720: Establish med support (CSS)1-2-7721: Conduct med activities (CSS)1-4-1102: Perform strength management (CSS)1-4-1104: Conduct casualty reporting (CSS)1-4-1308: Direct army airspace C2 (CSS)1-4-1310: Civil-military operations (CSS)1-4-1403: Monitor equipment readiness (CSS)1-4-1406: Provide logistic services (CSS)

Continuous Tasks

Legend

Implemented

Partially implemented

Desire to implement

Less relevant


Significant Results (2)

• Progressed toward theory for flexible group behavior

• Papers– “Metaplanning for Multiple Agents” in Proceedings of AIPS Plan

Execution Workshop, Pittsburgh, PA

– “Soar-RWA: Planning, Teamwork and Intelligent Behavior for Synthetic Rotary Wing Aircraft” in Proceedings of Seventh Conference on Computer Generated Forces and Behavioral Representation

– “Reasoning about Multiple Plans in Dynamic Multi-agent Domains,” AAAI Fall 1998 Symposium on Continuous Distributed Planning


Expected Results

• Domain Independent Technology– Theory of flexible group behavior covering:

• Situation awareness

• Planning

• Execution

– Integrated architecture for flexible group behavior

• Implementation of Technology– RWA Soar Attack Helicopter Battalion

• Evaluation of effectiveness of technology


Problem Areas

• Limiting factors– Change in personnel: Milind Tambe and Paul Rosenbloom

– Impact: • Re-focused approach to group understanding

• Transition to new PI (Randall Hill)

• Should not impact end result of project


Schedule

• Milestone 3: 9/98– Technology POP Demonstration 2

• RWA Attack Battalion

• Demonstrate Advanced Group Understanding– Scale up to larger groups of entities

• Demonstrate Advanced Group Planning– Group understanding (monitoring only)

– Collaborative planning (added)

• Demonstrate advanced group execution

– Deliver software and domain independent descriptions of new capabilities


Schedule (2)

• Milestone 4: 12/98– Design Review 2

• Approach to learning improved group models

• Approach to temporal planning


Schedule (3)

• Milestone 5: 9/99– Technology POP Demonstration 3

• RWA Attack Battalion

• Demonstrate advanced group understanding

• Demonstrate more advanced group planning– Temporal planning

– Group understanding: plan recognition

• Demonstrate advanced group execution– Commander utilizes teamwork model (scaled down)

• Demonstrate group learning– Improve group models through experience

– Deliver software and domain independent descriptions of new capabilities