PGM risk Sensitivity Analysis

20 July 2008Henry Neimeier

OWS Threads

U S A R M Y

Air LayerRadioSATCOM Fiber GPS/GBS

National Assets Reachback

Fiber Node

Fiber Node

Ground Station

Theatre C2 Node

Tactical C2 Node Ground

Station

CSG C2 Node

PGM risk Player Interface: green/gray buttons inputs, red buttons output results

Portfolio

MOEs

Scenario Options PGM risk

Air Layer

Scenario Parameters

Stepwise Regression

Rank Correlation

Tornado Uncertain All Inputs

Elastic ity MOE % change

from 1% change In Input

Sensitivity Analysis

ModelDetails

(proportion) :Corrected Pk Calc

(minutes) :Path … Calc

Survivable Throughput Calc

(probability) :Path Survival Calc

(proportion) :Link Thruput Calc

Targets At Risk Calc

Select Alt… Edit Table

(seconds/link) :Service Time Edit Table

Link Survival Probability Edit Table

Priority Proportion Edit Table

Utilization Edit Table

Baulking Utilization 0.8

Nodes 300

Setup Proportion 0.05

Pclassify Calc

(minutes) :Revisit Calc

(km2) :AOIsc Edit Table

ISR Platform Select Edit Table

ISR Parameters Edit Table

(m) :Target Size Edit Table

(probability) :Escape Calc

(hz) :Theater Aerial … Calc

Relay Planes Calc

(minutes) :Enemy Respon… Edit Table

(km) :Move Distance Edit Table

(km/hr) :Speed Edit Table

(minutes) :Pause Edit Table

Proportion Update 0.5

Kills Per Sortie Edit Table

Loadout Edit Table

(Hz) :Bandwidth Edit Table

(knots) :Speed Edit Table

(hours) :Endure Edit Table

(ft) :Relay Altitude Edit Table

(nmi) :Base Range Edit Table

(dollars) :Cost Edit Table

Payload Lb Hrs/AC $ Calc

Average Pk Calc

Select Scenario MCO2

Select Threat Level Low Intensity

Select Path Cell overfly

(Regression Coe) :Pk … Result

(Regression Coe) :Avg Pk Reg Calc

(Regression Coe) :Path Survival Reg Calc

(Regression Coe) :Survivable Thruput Reg Calc

(Regression Coe) :Path Delay Reg Calc

(Rank Correlation) :Pk RC Calc

(Rank Correlation) :Avg Pk RC Calc

(Rank Correlation) :Path Survival RC Calc

(Rank Correlation) :Survivable Thruput RC Calc

(Rank Correlation) :Path Delay RC Calc

(kill probability) :Pk T Calc

(kill probability) :Avg Pk T Calc

(probability) :Path Survival T Calc

(proportion) :Survivable Thruput T Calc

(minutes) :Path Delay T Calc

(kill probability) :Pk T Calc

(probability) :Path Survival T Calc

(kill Probability) :Avg Pk T Calc

(proportion) :Survivable Thruput T Calc

(minutes) :Path Delay T Calc

(%) :Pk Elasticity Calc

(%) :Average Pk Elasticity Calc

(%) :Path Survival Elasticity Calc

(%) :Survivable Thruput Elasticity Calc

(%) :Path Delay Elasticity Calc

Select Alternatives Edit Table

(minutes) :Outage Time Edit Table

Model Details (Module InfluenceDiagram & Module

Summary)

Portfolio: Selects scenario options, calculates TLE, and sets uncertain input variable distributions.Target Movement: Calculates target escape probability from target movement statisticsWeapons: Assigns weapons to target classes to maximize effective kills persortie corrected for target location error TLE, survivable throughput, escape probabilityComm Net: Calculate path delay, path survival probability, and survivable throughputfor specified network parameters and threat levelKill Thread Delay: Set thread task, sense, fuse, plan, execute times.Compare total time to enemy response time to determine targets at risk.ISR: Assigns ISR platforms to scenario, calculates revisit time, & probability of target classificationTerrain Weather: Calculates ISR coverage multiplier to correct for scenario weather and terrain masking.

indiciesfunctions

TargetMovement

ISR

Weapons

Kill ThreadDelay

Air Layer

Comm Net

Portfolio

TerrainWeather

SensitivityAnalysis

Overview

Portfolio Module: Specifies uncertain input variable distribution and selects scenario options

10.8 0.90.75 0.85 0.950

1

0.10.20.30.40.50.60.70.80.9

LinkSur

Cu

mu

lati

ve

Pro

ba

bil

ity

Link UpgradeNo Yes

Input Chance VariableTriangular DistributionsLight Blue Ellipses

TLE*Drift Rate

Service Time

Node Degree

Traffic ExtentPower

Link SurvivalProbability

SelectAlternatives

CA

CS

Link Survival

COSTTotal

AlternativeCost

Node DegUncertainty

Term Acc

Link Surv

Traf Pwr

SelectScenario

Select ThreatLevel

Service Tm

CV Ar

CV Svtm

svtms

TerminalAccess

Select LinkUpgrade

Outage Time Outage Out Tm

TLE

TLE multiplier of kill probability

Kill Thread Delay

Processing

STS DelayExecute

ExecuteUncertainty

Revisit

EnemyResponse Tm

Remaining TimeDistribution

Total Task TimeDistribution

Sense

Plan &Deconflict

Task Time

Task TimeDistribution

Total TaskTime

RemainingTime

Task ISR

Fuse &Integrate

Targets AtRisk

Path Delay

ExecutePriority

Comm Delay

RedResponse Tm

Select ThreatLevel

Execute Time

Weapons

Average Pk

Kills PerSortie

Loadout

Cost

ExpectedKills

MaxExpected

Kills

Pk

PotentialPlatforms

PlatformSpeed

Execute Time standoff

Range BandDistsnce

speed

CAP

WeaponAssigned Kills/Sortie

type

FastestStrike

Platform

CAP Direct

GPS

TLE*Probability

UpdateEscape

SurvivableThroughput

TactomProportion

ExpectedKills TacticalTommahawk

Pk Avg

Range BandProportion

Target ClassProportion

total

total

Targets AtRisk

Classification

Pk Cor

Kill probability is product of following factors:TLE multiplier (Portfolio)Effective kill probability per sortie (Weapons)Probability of classification (Weapons)Proportion of time SA fixed (TacTom)Probability of escape (Target motion)Survivable throughput (Comm)Targets at risk (Kill Thread)

Expected Kills Per Sortie Weapons Input (nominal Unclassified)

Sensitivity Analysis• Model non linear in both functions and interactions (products) of factors

– Sensitivity results vary by input variables not selected– Operating point effects sensitivity– Global versus local techniques

• Analytica supports both deterministic (analytic) and simulation modes in same model (output display option)

– Deterministic (mid) mode-fast same answer each time (analytic)– Chance probability distribution – slower slightly different answer with different

random number seeds (simulation)• Selected techniques: Tornado Diagrams, Elasticity, Rank Correlation,

Stepwise Regression, Orthogonal Factorial Designs, Correlation & Partial correlation

• Note the PGM risk model was developed in Analytica that supports the above sensitivity analysis techniques directly

Reference: Sensitivity Analysis by A. Saltelli…

Y

X

Sensitivity Analysis ModulesSensitivity Analysis Tornado

Regression

Elasticity

Rank Correlation

Average PkCorrected Pk

Tornadoes ImportanceElasticity

Path Survival Path DelaySurvivableThroughput

StepwiseRegression

SensitivityAnalysisLibrary

RankCorrelation

Vars

Input Ranges

Average Pk

Avg Pk T

Average PkInput Ranges

Level

Pk T

Path Delay TPath SurvivalT

SurvivableThruput T

Corrected Pk

All PlayerInputs

All PlayerInputs

Average PkElasticity

Pk Elasticity

Path SurvivalElasticity

Path DelayElasticity

Chance PkInputs

SurvivableThruputElasticity

Pk Elasticity

Path SurvivalElasticity

SurvivableThruputElasticity

Path DelayElasticity

Average PkElasticity

Corrected PkTop

Pk Details

Average Pk

AvgPkDetails

AvgPkNumber

AvgPk InputDescriptor

Avg Pk Reg

Pk Number

Pk InputDescriptor

Pk Reg

Path SurvivalSurvivableThroughput

Surv ThruDetails

Surv ThruNumber

Surv ThruDescriptor

SurvivableThruput Reg

Path Delay

Delay Details

DedlayNumber

DelayDescriptor

Path DelayReg

SurvivalDetails

Delay Mode

SurvivalNumber

SurvivalDescriptor

Path SurvivalReg

SelectPriority

Corrected PkTop

Pk Details

Average Pk

AvgPkDetails

AvgPkNumber

AvgPk InputDescriptor

Avg Pk RC

Pk Number

Pk InputDescriptor

Pk RC

Path SurvivalSurvivableThroughput

Surv ThruDetails

Surv ThruNumber

Surv ThruDescriptor

SurvivableThruput RC

Path Delay

Delay Details

DedlayNumber

DelayDescriptor

Path DelayRC

SurvivalDetails

Delay Mode

SurvivalNumber

SurvivalDescriptor

Path SurvivalRC

SelectPriority

Tornado Diagrams• Select output variable (MOE)• Select Input variables and low, baseline, and high values for them• Calculate output variable values for low and high values of the

selected input variable with all other input values at their baseline values

• Repeat for each successive input variable• Sort the input variables in decreasing order of the high-low output

MOE values– Note changes in non selected input values can change order (non

monotonic tornado values)• Many real world processes are not controllable

– Weather, enemy actions, time delays..– Tornado process assumes total parameter control of all input variables

• Handles both fixed and chance variable distributions – Distribution parameter values (min, mode, max…) controllable

Example Tornado Results(MOE: kill probability, average kill probability, priority 1 path delay)

MCO2, All Options, Low Intensity, Cell Overfly, Link Upgrade

Explanation Of Example Tornado Plot

• Input variables in order of significance to medium ship kill probability included link survival probability, red response time, network utilization, baulking utilization

• Input variables were changed by +10% (high) and -10% (low) from the baseline case• Operating point:

– Threat level: low intensity– All options selected– The target class is medium ship– The target is in the 40-150 kilometer range band

• Baseline kill probability for the operating point is 37%• Raising the link survival probability by 10% improves the kill probability to 51%• Lowering the link survival probability by 10% reduces the kill probability to 23%• Increasing red response time by 10% increases kill probability to 41• Decreasing red response time by 10% reduces the kill probability to 33%• Increasing utilization by 10% reduces kill probability to 33%• Decreasing utilization by 10% increases kill probability to 41%• Other parameters have less impact• Non linear system: high and low bars different lengths• Other MOEs shown are: average kill probability, communication path delay in minutes, and

survivable throughput.