Integrating a Distributed Agent-Based Simulation into an HLA Federation

Integrating a Distributed Agent-Integrating a Distributed Agent-Based Simulation into an HLA Based Simulation into an HLA

FederationFederation

Gary Kratkiewicz

Amelia Fedyk

Daniel Cerys

Purpose of this SessionPurpose of this Session

• Understand how to:– Use distributed multi-agent systems, with a

specific emphasis on the Cougaar architecture– Integrate a distributed multi-agent simulation

into an HLA federation– Design a full-scale Cougaar society for

integration with an HLA federation

OutlineOutline

• Motivation and Goals

• Distributed Agent Computing and Cougaar

• Integration Approach

• Lessons Learned

• Design considerations for Cougaar societies

• Conclusion

MotivationMotivation

• Military combat simulations lack sophisticated logistics components

• Logistics support is critical to combat operations

• Cougaar-based logistics simulations provide capabilities not available in current military simulation systems

• Roadblock: lack of interoperability

PurposePurpose

• Demonstrate the ability of Cougaar societies to interact with other simulations via HLA

• This would allow the following:– Credible proposals of simulation systems based

on Cougaar-HLA linkages– The addition of realistic simulation of logistics

to combat simulations

Technical Goals (1)Technical Goals (1)

• Demonstrate that a Cougaar-based society of agents can act as an HLA federate

• Show that Cougaar time mechanisms can integrate with HLA time synchronization

• Confirm that a Cougaar society can work with a variety of RTIs:– HLA 1.3 and HLA 1516– Fully asynchronous and partially asynchronous– Java and non-Java based

Technical Goals (2)Technical Goals (2)

• Develop techniques for creating or modifying Cougaar societies to be HLA federates

• Identify – Further areas of research – Additional functionality required in a full-scale

Cougaar HLA federate

OutlineOutline




• Lessons Learned


• Conclusion

AgentsAgents

• Agents are independent software entities that react to events and initiate actions by themselves

• Different agent-based architectures yield agents with varying capability levels

• Agents can have or define roles, tasks, beliefs, desires, or intentions

• Agents can be static or mobile

• Some agents or agent systems are considered intelligent

Distributed Agent ComputingDistributed Agent Computing

• A distributed system is a collection of separate processes or information systems that can act together as a single system

• Distributed agent computing involves agent-based systems that operate in a distributed manner

• Purpose:– Implement complex behavior – Model or simulate complex systems

CougaarCougaar

• Cognitive Agent Architecture • Java-based architecture• Rich design can model diverse objects and

interactions• Agents are Cougaar components with a

defined functionality and local memory store (Blackboard)

• The behavior of the agent emerges from the composite of plugins

CougaarCougaar

• Blackboard(BB) implements a publish/ subscribe API

• Plugins view objects on the BB by creating subscriptions

• Logic Providers are lightweight agent components responsible for messaging and BB modifications

• A collection of agents make up a cougar society

CougaarCougaar

PLATFORM OS

JAVA VMJAVA VM

COUGAAR NODE

Agent Binder

Agent Binder

PLATFORM SERVICES

COUGAAR NODE SERVICES

Agent Binder

AGENT.AGENTFRAMEWORKSVCS.

PluginBinder

PluginBinder

PluginBinder

PluginBinder

PlugInPlugIn

COUGAAR NODE

Agent Binder

COUGAAR NODE SERVICES

Agent BinderAgent Binder


PluginBinder

PluginBinder

PluginBinder

PluginBinder

Agent Binder


PluginBinder

PluginBinder

PluginBinder

PluginBinder

PlugInPlugIn

PlugInPlugIn

PlugIn

PlugIn

Plugin, Agent, and Node Structure of the Cougaar Architecture (K. Kleinmann et al, 2003)

UltraLogUltraLog

• 4 year project sponsored by DARPA

• Layer built on top of Cougaar Architecture

• Developed to model military logistics within a distributed multi-agent system

• Added Security, Robustness and Adaptivity to the cougar infrastructure.

• Test society models interaction between large set of military organizations.

UltraLogUltraLog

• Society runs in strictly planning mode(predictive solution) and execution mode(simulated solution)

• Solution includes detailed transportation and supply chain plan

• Society capable of garbage collection of completed/stale BB objects

FCS SupportabilityFCS Supportability

• Future Combat Systems (FCS) Supportability extends Ultralog Functionality

• Agents designed to reduce the logistics footprint

• Additional business logic added to plugins to model highly mobile, self-sufficient units.

• Plugins enhanced to accept 3rd party simulation data

OutlineOutline




• Lessons Learned


• Conclusion

HLA OverviewHLA Overview

Simulations

Runtime Infrastructure (RTI)

Data Collectors Interfaces to

Live PlayersSimulationsSimulationsSimulations

Data CollectorsData CollectorsData Collectors

Interfaces to Live Players



Interface

Federation and Society DesignFederation and Society Design

• Created small UltraLog logistics society– Pared down to model only bulk fuel consumption– 2 fuel-consuming organizations (agents)– 18 other organizations in supply chain and chain

of command

• Two society federates based on above:– High-fidelity logistics society for general

simulation– Demand generation society to model fuel

demand

Prototype SocietyPrototype Society

HNS

1-35-ARBN 1-6-INFBN 47-FSB

2-BDE-1-AD

123-M SB

DISCOM -1-AD

1-AD

900-POL-SUPPLYCO

316-POL-SUPPLYBN

7-CSG

3-SUPCOM -HQ

5-CORPS

110-POL-SUPPLYCO

6-TCB N

37-TRANSGP

21-TSC-HQ

USAEU R

USEUCOM

NC A

Non-Cougaar FederateNon-Cougaar Federate

• Performed verification with Java-based, non-agent, non-Cougaar federate

• Demand generation society to model fuel demand

RTI SelectionRTI Selection

• Tested prototypes with following HLA RTIs:– Pitch 1516 LE– Pitch 1.3 LE– DMSO NG 1.3

• Demonstrated the feasibility of using Cougaar with: – HLA 1.3 and IEEE 1516 RTIs– Fully asynchronous (Pitch) and partially

asynchronous (DMSO) RTIs– Java-based (Pitch) and non-Java (DMSO) RTIs

Simulation ModelsSimulation Models

• Cougaar time:– Continuously advances with real (wall clock) time

– Can be advanced ahead to some future time

• Mapping to simulation model:– Advance time with real time or ahead in equal steps

time-stepped simulation model

– Advance time in unequal steps event-driven model

• Most Cougaar simulations run for hours and simulate operations over months– Equal steps scaled real-time

– Unequal steps non-real-time

Time ManagementTime Management

• Cougaar time– Initialized to physical time– Advances at same rate as wall clock– Can be stepped to future point in time– Society should be in quiescence before stepping

• Cougaar and HLA time synchronization– None: not useful for simulation, OK for planning– Conservative: appropriate– Optimistic: not appropriate; Cougaar cannot roll back

time• OK since different federates can use different

synchronization methods

Iterative Development ApproachIterative Development Approach

1. Build a federation without HLA time management

• Two federates• Allowed us to get running quickly

2. Build a federation with HLA time management• Two federates• Added RTI time synchronization

3. Build a federation with a multi-node Cougaar society

• Three federates total• Two federates map to one society

Phase 2: HLA Time ManagementPhase 2: HLA Time Management

High - Fidelity Logistics Society

NCA

1 -35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Cougaar Node

Demand Generation Society

Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node

HLA RTI Supply Task Interaction

Time Management

Federate 1 Federate 2


NCA

1 -35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Cougaar Node


Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node


Time Management



NCA

1 - 35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Cougaar Node


NCA

1 - 35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Cougaar Node


Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node


Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node


Time Management



Time Management


Phase 3: Multi-Node SocietyPhase 3: Multi-Node Society

High - Fidelity Logistics Society Demand Generation Society


Time Management

NCA

1 - 35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node Cougaar Node


High - Fidelity Logistics Society Demand Generation Society


Time Management

NCA

1 - 35 - ARBN

Ambassador

Service

1 - 6 - INFBN

Advance

Time

Ambassador

Service

NCA

1 - 35 - ARBN

1 - 6 - INFBN

Advance

Time

Cougaar Node Cougaar Node


Ambassador

Service

Cougaar Node

Federate 2

Results of Integration ExperimentsResults of Integration Experiments

• Demonstrated interoperability between Cougaar and HLA RTIs in various combinations

• Demonstrated various operations:– Subscribing a Cougaar federate and plugins to specific HLA

interactions

– Interfacing a Cougaar society to the RTI via a single ambassador class

– Interfacing a Cougaar society to the RTI via a Cougaar service

– Synchronizing society time via the HLA mechanism using conservative synchronization

– Detecting tasks in one society, transferring the task info via HLA interactions, reconstituting the tasks in a second society

• Learned techniques and identified design considerations

OutlineOutline




• Lessons Learned


• Conclusion

Interfacing Between Society and RTIInterfacing Between Society and RTI

• Singleton ambassador class– Separate ambassador class per RTI

– Performs following actions:• Instantiates the RTI’s ambassador class

• Joins the federation execution (creating it first if necessary)

• Connects to interactions and objects

• Handles the interactions and object attributes from the federation

• Distributes them to the appropriate agent in the society

• Later moved ambassador class to Cougaar service• Separate class instantiated per Cougaar node

Time Synchronization: Phase 1Time Synchronization: Phase 1

• Used a servlet-based, user-directed time advancement mechanism– Allowed us to operate with a variety of

simulation models– Used for all phases

• Initially implemented without HLA time management– For development purposes only– Advanced time in high-fidelity society– Time advance passed to demand generation

society via interactions• Small society – always ready

Time Synchronization: Phase 2Time Synchronization: Phase 2

• Conservative time synchronization– Standard HLA mechanism– Modified version of the user-directed Cougaar

time advancement mechanism.

• Cougaar mechanism modified – Obtains permission from the RTI before

advancing time in the society– Required ambassador class, adding plugins to

handle time, and modifying the time advance servlet

Time Management DetailsTime Management Details1. User manually advances time via servlet

– Must manually determine quiescence

2. Servlet places time change request object on agent’s blackboard

3. Time plugin subscribes to object on blackboard– Reads it when it appears and calls ambassador service

4. Service checks for pending request; if not, sends request to RTI

5. RTI grants advance when appropriate6. Service calls callback method in plugin

– Changes society time– Stores new federation time

7. RTI delivers appropriate messages to society

Mapping Agents, Actions, and ObjectsMapping Agents, Actions, and Objects

• Cougaar societies contains many objects– Map these objects to HLA actions and objects carefully

to ensure acceptable performance– Data can be transferred in HLA via interactions (events)

and/or object attributes

• Mapping is heavily dependent on the society design– Cougaar agents or asset objects map to HLA objects – Cougaar events map to HLA interactions – Cougaar objects such as UltraLog tasks could be mapped

either way • Depends on how long they can live and whether their contents

change

OutlineOutline




• Lessons Learned


• Conclusion

Federate GranularityFederate Granularity

• Federate granularity: the level of mapping of federates to portions of a Cougaar society– Consider size and organization of the society, and design

of other (non-Cougaar) federates

• In our demonstration prototypes, each Cougaar node mapped to a federate.

• In a large complex society, a logical group of agents (covering multiple nodes) might constitute a single federate– Transportation organization, combat organization, etc.

• Single agents could map to individual federates– Make sure this makes sense

Society DesignSociety Design

• Selection of agents when federation contains multiple societies

• Functional split– Same agents, different functions

• Organizational split– Different agents in logical groupings

Time SynchronizationTime Synchronization

• Need automated time advance mechanism– Trigger time advance requests automatically

• Based on schedule or events

– Check for quiescence before making request

• Society time is always advancing– Time step needs to be large enough that society

time advances an insignificant amount between steps

Adaptive Society DesignAdaptive Society Design

• Prototype HLA ambassador class hardcodes HLA interactions, objects, attributes– Need to modify class when plugins change

• Better way: hardcode nothing in the ambassador class– Each plugin would

• Register with the ambassador class • Tell it which interactions and objects it was interested in

– The ambassador class would then • Dynamically subscribe to the requested interactions• Read the FOM file to get the attributes for each interaction• Dynamically get the attribute handles

• Leverage Cougaar object discovery to match up objects in federates with those in the society

Operation Independent of FederationOperation Independent of Federation

• For more flexibility and survivability, design your society to operate – Whether or not the appropriate federates are present – If not available, use default plugins

• Or, use the Cougaar Message Transport Service (MTS) and create an HLA-specific link protocol– Appropriate where a society

• May or may not be a member of a federation • Has a number of other communication channels at its disposal

– MTS would select communication method based on availability and prioritization of channels

– Allows interoperation with simulations that exist outside of both the Cougaar society and the HLA federation

OutlineOutline




• Lessons Learned


• Conclusion

Conclusion (1)Conclusion (1)

• Successfully Integrated a Cougaar-based agent society as a federate in V1.3 and 1516 HLA federations

• Integration leverages the benefits of both architectures– Distributed agent-based architecture– Standard simulation interoperability architecture

• Established how to – Interface the society and the HLA RTI– Synchronize society time with HLA RTI time– Map agents, objects, and actions in the society to HLA

objects and interactions

Conclusion (2)Conclusion (2)

• Examined integrating a full-size Cougaar society with an HLA federation

• Lessons learned can be applied to large-scale efforts– Such as integrating large Cougaar-based

logistics simulations with combat simulations– Such integrated simulations would provide

greater effectiveness than combat-only simulations

Integrating a Distributed Agent-Integrating a Distributed Agent-Based Simulation into an HLA Based Simulation into an HLA

FederationFederation

Gary Kratkiewicz

Amelia Fedyk

Daniel Cerys

Integrating a Distributed Agent-Based Simulation into an HLA Federation

Documents

Transcript of Integrating a Distributed Agent-Based Simulation into an HLA Federation