Download - Analyzing Air Combat Simulation Results with Dynamic Bayesian Networks

1

Helsinki University of Technology Systems Analysis Laboratory

Analyzing Air Combat Analyzing Air Combat Simulation Results with Simulation Results with

Dynamic Bayesian Networks Dynamic Bayesian Networks

Jirka Poropudas and Kai VirtanenSystems Analysis Laboratory

Helsinki University of TechnologyP.O. Box 1100, 02015 TKK, Finland

http://www.sal.tkk.fi/[email protected]


2Winter Simulation Conference, Washington D.C. 2007

OutlineOutline Air combat (AC) simulation Analysis of simulation results Modelling the progress of AC in time Dynamic Bayesian network (DBN) Modelling AC using DBN Summary



Analysis of AC Using SimulationAnalysis of AC Using Simulation

Most cost-efficient and flexible method Commonly used models based on

discrete event simulation

Objectives for AC simulation study: Acquire information on systems performance Compare tactics and hardware configurations Increase understanding of AC and its progress



Discrete Event AC SimulationDiscrete Event AC Simulation

Simulation input Aircraft and

hardware configurations

Tactics Decision making

parameters

Simulation output Number of kills

and losses Aircraft

trajectories AC events etc.

Decision making logic

Aircraft, weapons, and hardware models



Traditional Statistical Models Turn AC into a Static EventTraditional Statistical Models Turn AC into a Static Event

Simulation data has to be analyzed statistically Statistically reliable AC simulation may require tens of

thousands of simulation replications Descriptive statistics and empirical distributions for the

simulation output, e.g., kills and losses Regression models describe the dependence between

simulation input and output

These models do not show the progress of AC in timeor the effect of AC events on AC and its outcome



Overwhelming Amount of Simulation DataOverwhelming Amount of Simulation Data

Not possible, e.g., to watch animations and observe trends or phenomena in the simulated AC

How should the progress of AC be analyzed?

How different AC events affect the outcome of the AC?



Modelling the Progress of AC in TimeModelling the Progress of AC in Time State of AC

– Definition depends on, e.g., the goal of analysis and the simulation model properties

Outcome of AC– Measure for success in AC?

– Definition depends on, e.g., the goal of analysis

Dynamics of AC must be included– How does AC state change in time?

– How does a given AC state affect AC outcome?



Definition for the State of ACDefinition for the State of AC 1 vs. 1 AC, blue and red Bt and Rt are AC state

variables at time t State variable values “Phases” of simulated

pilots– Are a part of the decision

making model

– Determine behavior and phase transitions for individual pilots

– Answer the question ”What is the pilot doing at time t?”

Example of AC phases in X-Brawlersimulation model



Outcome of ACOutcome of AC Outcome Ot is described by a variable with four possible values

– Blue advantage: blue is alive, red is shot down

– Red advantage: blue is shot down, red is alive

– Mutual disadvantage: both sides have been shot down

– Neutral: Both sides are alive

Outcome at time t is a function of state variables Bt and Rt



Probability Distribution of Probability Distribution of AC State Changes in TimeAC State Changes in Time

State variables are random– Probability distribution estimated from

simulation data

Distributions change in time = Progress of AC

What-if analysis– Conditional distributions are estimated

– Estimation must be repeated for all analyzed cases, ineffective

Dynamic Bayesian Network



Dynamic Bayesian Network Model for ACDynamic Bayesian Network Model for AC Dynamic Bayesian network

– Nodes = variables

– Arcs = dependencies

Dependence between variables described by– Network structure

– Conditional probability tables

Time instant t presented by single time slice

Outcome Ot depends on Bt and Rt

time slice



Dynamic Bayesian Network Is Dynamic Bayesian Network Is Fitted to Simulation DataFitted to Simulation Data

Basic structure of DBN is assumed Additional arcs added to improve fit Probability tables estimated from

simulation data



Continuous probability curves estimated from simulation data

DBN model re-produces probabilities at discrete times

DBN gives compact and efficient model for the progress of AC

Progress of AC Tracked by DBNProgress of AC Tracked by DBN



DBN Enables Effective DBN Enables Effective What-If AnalysisWhat-If Analysis

Evidence on state of AC fed to DBN For example, blue is engaged within

visual range combat at time 125 s– How does this affect the progress of AC?

– Or the outcome of AC?

DBN allows fast and efficient updating of probability distributions– More efficient what-if analysis

No need for repeated re-screening simulation data



Future Development of Existing ModelsFuture Development of Existing Models

Other definitions for AC state, e.g., based on geometry and dynamics of AC

Extension to n vs. m scenarios

Optimized time discretization– In existing models time

instants have been distributed uniformly



SummarySummary Progress of simulated AC studied by estimating

time-varying probability distributions for AC state Probability distributions presented using

a Dynamic Bayesian network DBN model approximates the distribution of AC state

– Progress of AC

– Dependencies between state variables

– Dependence between AC events and outcome

DBN used for effective what-if analysis



References References

» Anon. 2002. The X-Brawler air combat simulator management summary. Vienna, VA, USA: L-3 Communications Analytics Corporation.

» Feuchter, C.A. 2000. Air force analyst’s handbook: on understanding the nature of analysis. Kirtland, NM. USA: Office of Aerospace Studies, Air Force Material Command.

» Jensen, F.V. 2001. Bayesian networks and decision graphs (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc.

» Law, A.M. and W.D. Kelton. 2000. Simulation modelling and analysis. New York, NY, USA: McGraw-Hill Higher Education.

» Poropudas, J. and K. Virtanen. 2006. Game Theoretic Analysis of Air Combat Simulation Model. In Proceedings of the 12th International Symposium of Dynamic Games and Applications. The International Society of Dynamic Games.

» Virtanen, K., T. Raivio, and R.P. Hämäläinen. 1999. Decision theoretical approach to pilot simulation. Journal of Aircraft 26 (4):632-641.