Cyber Test Analysis and Simulation Environment...

54
Army Department of Defense DASD - DT&E / TRMC CTEIP Air Force Cyber TASE Navy DISA UNCLASSIFIED//FOUO UNCLASSIFIED//FOUO Cyber Test Analysis and Simulation Environment (TASE) August 20, 2015 UNCLASSIFIED//FOR OFFICIAL USE ONLY Michael Winslow Rich Wride Donn Puckle Cyber TASE PM Cyber TASE DPM Cyber TASE Army PM SPAWARSYSCENPAC 96 TW/46 TS USAEPG/ATEC [email protected] [email protected] [email protected] 619-553-0341 850-882-0765 520-538-4830

Transcript of Cyber Test Analysis and Simulation Environment...

Page 1: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

UNCLASSIFIED//FOUO

Cyber Test Analysis and Simulation Environment (TASE)

August 20, 2015

UNCLASSIFIED//FOR OFFICIAL USE ONLY

Michael Winslow Rich Wride Donn Puckle

Cyber TASE PM Cyber TASE DPM Cyber TASE Army PM

SPAWARSYSCENPAC 96 TW/46 TS USAEPG/ATEC

[email protected] [email protected] [email protected]

619-553-0341 850-882-0765 520-538-4830

Page 2: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Reliance/CTEIP Program Process

Proposal Phase

• Defining Project Scope

Pre-Phase 0

• Risk ReductionActivities

• Develop AoA

Phase 0

• Requirements Developmentand Planning

• Develop program documents

Phase 1

• Concept Development and Preliminary Design

Phase 2

• System Development

1-2 years 0.5 – 1 year 1-2 years 3-4 years

1 – 2

years

1.5 – 3

years2.5 – 5

years

6.5 – 11

years

1-2 years

3.5 – 7

years

FY 11 FY 13 FY 14 FY 15 FY 16

Gather

Needs

Needs combined

with proposed

solutions

Core

requirements

set

KPPs & KSAs

developedRequirements

set / Solutions

selected

4 years

Page 3: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Cybersecurity DT Requirements

• Revision of DoDI 5000.02: Issued 6 Jan 2015 – New/better guidance for both developmental and operational testing of IT

• Revision of DoD 8500.01, Cybersecurity: 14 Mar

2014

– Expanded scope and specificity

• DoDI 8510.01 – Risk Management Framework (RMF)

for DoD IT: 14 Mar 2014 – Provides policy, clarity and guidance on the RMF and compliance

• Four Phased Cybersecurity DT&E Process: In Work– Incorporated into Defense Acquisition Guidebook Chapter 9

• OSD DOT&E- Procedures for Operational Test and

Evaluation of Cybersecurity in Acquisition Programs:

01 Aug 2014– Formalizes OT&E Phases

• Cybersecurity Implementation Guidebook for PMs– Address Cybersecurity T&E across the acquisition lifecycle

• Cybersecurity T&E Guidebook planned– To provide detailed Cybersecurity T&E guidance for DT/OT Community

Current DT&E Cybersecurity Guidance

Phase 1: Understand

Cybersecurity Requirements

Phase 2: Characterize Cyber Attack

Surface

Phase 3: Cooperative Vulnerability Identification

Phase 4: Adversarial

Cybersecurity DT&E

Understand Cybersecurity

requirements and develop

an approach for

cybersecurity T&E

Characterize the attack

surface; in the integrated

environment, determine

possible threat vectors.

Analyze and evaluate

potential vulnerabilities to

determine measures to

improve resilience.

Cybersecurity DT&E event in a

realistic mission environment,

with use of cyber range,

CNDSP, representative users

and Cybersecurity threat

representation.

MS B

TechnologyMaturation &

CDD Risk Reduction

SRR SFR PDR CDR

MS CATO

MS A

Engineering &ManufacturingDevelopment

TRR EventSVR DT&E

ASR

MaterielSolution AOA

DRAFT

AnalysisMDD CDD CPD

IATT

DT&E

Req Decision

Pre-EMD

DT&EAssess-

ment

Cyber TASE will aid in performing both the functional and Cybersecurity DT testing in EMD phase.

Page 4: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Cybersecurity OT Requirements

Phase 5: Cooperative Vulnerability and

Penetration Assessment

Phase 6: Adversarial Assessment

This phase assesses the ability of a unit equipped with a system to support its missions while withstanding validated and representative cyber threat activity.

Provide a comprehensive characterization of the cybersecurity status of a system in a fully operational context, and to substitute for reconnaissance activities in support of adversarial testing when necessary

O&SProduction andDeploymentCPD

Assess- OTRR IOT&Ement

Full Rate Production

Decision ReviewMS C

ATO

Cyber TASE will greatly aid in the analysis required for both Phases of theOSD-DoT&E Cybersecurity Test Memo levied upon acquisition programs.

Engineering &ManufacturingDevelopment

Memorandum from

Dr. J. Michael Gilmore (DOT&E)

Page 5: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Cybersecurity OT Requirements

Phase 5: Cooperative Vulnerability and

Penetration Assessment

Phase 6: Adversarial Assessment

This phase assesses the ability of a unit equipped with a system to support its missions while withstanding validated and representative cyber threat activity.

Provide a comprehensive characterization of the cybersecurity status of a system in a fully operational context, and to substitute for reconnaissance activities in support of adversarial testing when necessary

O&SProduction andDeploymentCPD

Assess- OTRR IOT&Ement

Full Rate Production

Decision ReviewMS C

ATO

Cyber TASE will greatly aid in the analysis required for both Phases of theOSD-DoT&E Cybersecurity Test Memo levied upon acquisition programs.

Engineering &ManufacturingDevelopment

Memorandum from

Dr. J. Michael Gilmore (DOT&E)

“All oversight systems capable of sending or receiving digital information are required to conduct cybersecurity testing. This includes uploading or downloading data by physical means such as Universal Serial Bus (USB) connections or removable data devices.”

Page 6: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Integrated Test Concept

Phase 1: Understand

Cybersecurity Requirements

Phase 2: Characterize Cyber Attack

Surface

Phase 3: Cooperative Vulnerability Identification

Phase 4: DT&E Cyber Aggressor

Team

Phase 5: Cooperative Vulnerability and

Penetration Assessment

Phase 6: Adversarial Assessment Cyber

Red Team

DT

OT

Cyber TASE

Page 7: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Cyber Testing Capability Gaps

• Two gaps addressed by Cyber TASE• Lack of effective IT and Data Link testing

instrumentation • Collect NW Data, Host Data, Application level Data, &

Truth Data

• Automated analysis of collected data to assess how C4I systems perform against an ongoing cyber attack

• Correlation of data gathered across Cyber stacks including Enterprise/Web Services

• Develop visualization capabilities

• Lack of a Live-Virtual-Constructive (L-V-C) environment capable• Mimicking large scale operational scenarios with

Cyber instrumentation

• High fidelity

• Ability to emulate cyber threat

7

• Gaps not addressed by Cyber TASE• PIT systems

• DCO systems

• OCO systems

Page 8: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

What will Cyber TASE Provide?

8

Planning

• Developing Metrics and Measures

• Designing and Planning

• Scheduling needed infrastructure

Environment

• Implementing Test Environment

• Physical/Virtualized stimulation

• Threat (provide path)

• Simulated stimulation

• Threat Emulation

• Implementing Contra-Technologies

Execution

• Integrating LVC environment

• Measuring and Monitoring

• Real-Time Analysis

• Real-Time Visualization

Evaluation

• Post-Test Analysis

• Formulation of Conclusions

• Evaluation of Test Sufficiency and Accuracy

Cyber TASE provides integrated instrumentation and Constructive simulation environment that improves the capabilities across the Cyber Test Workflow.

Page 9: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Operational View

9

Blue

Team

White Cell

SUT

SUT

SUT

SUT

Red

Team

Gray

Team

Page 10: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Army

GCCS/Army

Air Force

GCCS/AF

Shore

Use CaseSUT: GCCS/M on CANES

10

CANES

ADNS

Ship

ADNSCANES

Computing

GCCS/MWeb Server

GCCS/MDB Server

DISN

Users

Shore Computing

GCCS/MWeb Server

GCCS/MDB Server

Commander

Tactical Platform

COP Server Sensor

TDL

SATCOM

Navy DISA / Joint

DISN

Shore Computing

GCCS/JWeb Server

GCCS/JDB Server

Commander

JMETC 2.0

Page 11: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Army

GCCS/Army

Air Force

GCCS/AF

Shore

Use CaseData Manipulation Attack

11

CANES

ADNS

Ship

ADNSCANES

Computing

GCCS/MWeb Server

GCCS/MDB Server

DISN

Users

Shore Computing

GCCS/MWeb Server

GCCS/MDB Server

Commander

Tactical Platform

COP Server Sensor

TDL

SATCOM

Cyber Threat

Navy DISA / Joint

DISN

Shore Computing

GCCS/JWeb Server

GCCS/JDB Server

Commander

JMETC 2.0

Page 12: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Army

GCCS/Army

Air Force

GCCS/AF

Shore

Use CaseData Manipulation – Operator View

12

CANES

ADNS

Ship

ADNSCANES

Computing

GCCS/MWeb Server

GCCS/MDB Server

DISN

Users

Shore Computing

GCCS/MWeb Server

GCCS/MDB Server

Commander

Tactical Platform

COP Server Sensor

TDL

SATCOM

Cyber Threat

Navy DISA / Joint

DISN

Shore Computing

GCCS/JWeb Server

GCCS/JDB Server

Commander

JMETC 2.0

You You

Tactical Picture

Before Threat After Threat

Page 13: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Army

GCCS/Army

Air Force

GCCS/AF

Shore

Use CaseData Manipulation – Instrumentation After

13

CANES

ADNS

Ship

ADNSCANES

Computing

GCCS/MWeb Server

GCCS/MDB Server

DISN

Users

Shore Computing

GCCS/MWeb Server

GCCS/MDB Server

Commander

Tactical Platform

COP Server Sensor

TDL

SATCOM

Cyber Threat

Navy DISA / Joint

DISN

Shore Computing

GCCS/JWeb Server

GCCS/JDB Server

Commander

JMETC 2.0Analysis

Visualization

Warning:

CANES User getting corrupt COP Picture.

Sources of Event:

SQL Injection – Source GCCS/M DB Server observed

unusual sources of data. New network traffic confirmed

by network at Ingress point.

Page 14: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Live-Virtual-Constructive GapsConstructive Simulation

14

Service Lab (Live)

NCR (Live/Virtual)

Cyber TASE Simulation (Constructive)

Cyber TASE enhances constructive simulation capability to represent high-fidelity, large-scale operational

scenarios, not achievable in lab environments.

Page 15: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Cyber TASE Block Diagram

15

Network Data Collector

TDL Data Collector

Host Data Collector

Ground TruthData Collector

Cyber TASEAnalysis

Cyber TASEVisualization

SUT &Infrastructure

Cyber TASE Instrumentation

Cyber TASE Constructive Simulation

Users

Stealthnet

Page 16: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

End Vision: Cyber TASE

16

Distributed testing with the JMETC MILS NW

(JMN) / Joint IO Range

Cyber Aggressor

Team

White Cell

Sim Operational System

Environment -stimulation

SUT

SUT

SUT

SUT

Servers, Applications,Users, TDLs

Servers, Applications,Users, TDLs Servers,

Applications,Users, TDLs,

HITLs

Network Data Collector Agent

Host/TDL Data Collector Agent

Navy Army Air Force DISA

Correlation, Analysis, and Visualization

Simulation Env.(L-V-C)

Simulation Env.(L-V-C)

OFFLINE

Servers, Applications,Users, TDLs,

HITLs

Mission Threads

SUT RSDPs*

*Regional Service

Delivery Points

Page 17: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

What is JMETC?Enables Distributed Testing

17

Systems

Under

Test

Joint Operational Scenarios

Integrated

Test

Resources

JMETC

Infrastructure

ReuseRepository

Distributed TestSupport Tools

Data ManagementSolutions

JMETC

MILS

Network

Customer Support

Virtual

Prototype

Hardware

in the

Loop

Installed

Systems

Test

Facility

Range Environment

Generator

Threat

Systems

Cyber Aggressor

Team

Regional Service Delivery Point (RSDP)

Page 18: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Questions?

UNCLASSIFIED//FOR OFFICIAL USE ONLY

Page 19: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Backups

Page 20: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

What organization sponsors Cyber TASE?

• ASD-DT&E (Developmental Test and Evaluation)– Provides oversight over DT

• TRMC (Test Resource Management Center)– Strategic planning of Testing Ranges

– Reviews and certifies T&E Budgets

– Runs the Centralized T&E Improvement Program (CTEIP) – Cyber TASE

– Runs the T&E S&T Program - CTT

– Runs the Joint Mission Environment Test Capability (JMETC) Program - JMN

Secretary of Defense(SECDEF)

Under Secretary of Defense – Acquisition, Technology,

Logistics (USD-AT&L)

Assistant Secretary of Defense – Research and

Engineering(ASD-R&E)

Assistant Secretary of Defense – Developmental

Test and Evaluation(ASD-DT&E)

Test Resource Management Center

(TRMC)

Page 21: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Relationship within TRMC

21

TRMC

Service

Improvement &

Modernization/

Programs

Acquisition Programs /

Advanced Concept

Technology

Demonstrations

T&E Multi-Service /

Agency Capabilities

Risk mitigation needsTechnology shortfalls

Risk mitigation solutions

Advanced development

Requirements

Capabilities

DoD Corporate

Distributed Test

Capability

TRMCJointInvestmentPrograms

6.3 6.4 6.5

DoD Corporate

Cyber

George

Rumford

Chris Paust Chip

Ferguson

Derrick Hinton

DASD-DT&E

DT&E

Dr. C. David

Brown

Dr. C. David

Brown

TRL 3 - 6 TRL 7 - 9 TRL 9

Cyber TASE

CTTJMN

NCR

Page 22: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

AT&L, DT&E / TRMC Organization

Chris

Paust

Page 23: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

DOT&E, ECRE Active Requirements

• Purpose of ECRE ACTIVE is to:• Create a rapidly reconfigurable LVC simulated cyberspace environment / solution

to address various SUTs.

• Use ECRE ACTIVE to assess DOD C4I system compliance with Department of Defense Instruction (DODI) 8500.01, Cybersecurity, dated 14 March 2014.

• Use ECRE ACTIVE to assess DOD C4I system compliance with the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-53, Security and Privacy Controls for Federal Information Systems and Organizations, dated 30 April 2013.

• Ascertain the suitability, survivability, effectiveness and interoperability of C4I systems under operationally realistic adversarial cyber threats and heavy network traffic loads.

• Characterization of an SUTs attack surface by the execution of fuzzing conditions (e.g., providing invalid, unexpected, or random data to the inputs of a SUT)

Page 24: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

DOT&E, ECRE Active Solutions

• Enable Cyber Opposing Force (OPFOR) to inject enumerated operationally realistic adversarial cyberspace threats against SUTs

• Monitor and measure network cyberspace activity

• Measure C4I system effectiveness (detection, protection, interoperability with other C4I systems, and restoration processes) against cyber attacks in a burdened network

• Assess ECRE ACTIVE tools and C4I compliance with DOD cybersecurity policies per DODI 8500.01 or NIST SP 800-53

• Collect system and mission data and maintain a central repository

• Interpret collected data against assessment criteria to generate cyberspace vulnerability and C4I effectiveness and interoperability assessment reports.

• Develop and refine ECRE ACTIVE processes and measurement methodology.

Page 25: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

ECRE ACTIVE Operational View

Page 26: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Planning

26

Planning Environment Execution Analysis

Cyber TASE Unique Capabilities1. TASE will address basic Cyber Security Metrics development 2. Planning process used for data collection & analysis plan will be used to develop

capabilities needed

Determine system

requirementsDevelop Metrics

Develop objectives

Determinedistributed

environment needed

Develop data collection plan

Develop Analysis plan

Establish method to

reconstitute SUT*

CTT project

* System Under Test

Page 27: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Environment

27

Planning Environment Execution Analysis

Cyber TASE Unique Capabilities1. Enhance Existing Instrumentation

• Develop integrated data collection across NW, host, & SUT applications• Automated analysis of threat progress & effect on SUT• Configurable user friendly tool set – consistent user interface & simple installation • Based on distributed test concept

2. Visualization and Situation Awareness Tools

Network environment

Instrumentation / Detection

Attack methods Grey Network Data AnalysisUser

environment / interface

Distributed connections

CTT project Provided by JMETC

Page 28: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Execution

28

Planning Environment Execution Analysis

Cyber TASE Unique Capabilities1. Track threats through NW to application level2. Assess effect of threat on CIA* for SUT

Non-threat operation

AttackDetection /

Threat Identification

Blue Team Assessment -

Human Factors

Near-Real-Time Analyze

Data

Reset systems to known

configuration

Daily Reports Generated /

Feedback

Input from CTT project

* Confidentiality, Integrity, & Availability

Provide path

Page 29: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Analysis

29

Planning Environment Execution Analysis

Cyber TASE Unique Capabilities1. Correlate data – locally & distributed2. Automate analysis3. Visualize and provide situation awareness4. Provide report inputs for risks & deficiencies

Document Attacks used

Analyze DataThreat

Identification

System response to

threat

Operator response to

threat

Determine Risks

Reports Generated /

Feedback

Page 30: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

T&E Reliance Process

Proposal Submission

Concept Refinement / Briefing Phase

Technical Subject Matter

Experts (TSMEs)

Test Resource Activity Group

(TRAG)

Board of Directors

(Executive Secretariat

Staff) (BoD(ESS))

Board of Directors

(Executive Secretariat) (BoD(ES))

Project Initiation

Submit Proposal Online

Combine Proposal with Similar Joint

Efforts

TSMEs Review Proposals

Form Joint Proposal Team

400+

10

4

2Receive T&E

Executive Approval

Assign Service Lead

Brief TRMC / CTEIP Program

Enter Pre-Phase 0 (Risk

Reduction)

Proposals

Page 31: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Functional ArchitectureInstrumentation

31

Instrumentation

Network Data Collection

Collect IP Data

Collect TDL

Host Data Collection

Collect Systems Data

Collect Operator Data

Ground Truth Data Collection

Collect Red Team Ground Truth

Collect Operator Ground Truth

Correlation and Data Analysis

Conduct near-Real-Time Analysis

Conduct Post-Test Analysis

Near-Real-Time Visualization

Visualize System Data

Visualize Operator Data

Post-Test Visualization

Visualize System Data

Visualize Operator Data

Cyber TASE

• Provides integrated instrumentation for collecting, analyzing, and visualizing the test data across multiple layers/sources to understand the mission impacts in a Cyber contested environment.

• Provides constructive simulation to scale L-V-C environment so we can represent a full scaled operational environment and the impact of Cyber threats on conducting mission operations.

Page 32: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Functional ArchitectureConstructive Simulation

32

Constructive Simulation

(STEALTHNET)

Host Models

Vulnerabilities of Host

Host Topology Pallet

Network Models

Vulnerabilities of Network

Infrastructure

Network Topology Pallet

Threat ModelsInterface to Live/Virtual

Instrument Simulation

Visualize Data

Cyber TASE

• Provides integrated instrumentation for collecting, analyzing, and visualizing the test data across multiple layers/sources to understand the mission impacts in a Cyber contested environment.

• Provides constructive simulation to scale L-V-C environment so we can represent a full scaled operational environment and the impact of Cyber threats on conducting mission operations.

Page 33: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Technical Approach

Network Data Collector

TDL Data Collector

Host Data Collector

Red Team Data Collector

Operator Data Collector

StorageCyber TASE

Analysis

Cyber TASEVisualization

SUT &Infrastructure

Cyber TASE Instrumentation

Cyber TASE Constructive Simulation

Users

Stealthnet

Other user Desired

Analysis Tool

Page 34: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Live-Virtual-Constructive

Service Lab (Live)

NCR (Live/Virtual)

Cyber TASE Simulation (Constructive)

Cyber TASE enhances constructive simulation capability to represent high-fidelity, large-scale operational

scenarios, not achievable in lab environments.

Page 35: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Design and Development Plan

36

Phase Fiscal Year Activities

Phase 0 FY14 Requirements Development and Planning

Phase 1 FY15 Concept Development and Preliminary Design

Phase 2

FY16(EOC)

Deliver Toolset Enhancements and Basic Integration of tools, First Subset of Constructive Simulator Enhancements

FY17(IOC)

Deliver Correlation and Analysis Capability with additional integration, Second Subset of Constructive Simulator Enhancements

FY18(FOC)

Deliver Visualization Capability with final integration, Final Subset of Constructive Simulator Enhancements

Cyber TASE will deliver capabilities in 3 increments during FY16, FY17, and FY18. Demonstrations will be conducted with each delivery.

Page 36: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Deliverables

Instrumentation L-V-C Environment Demonstration

EOC12/2016

• Toolset enhancements and basic integration of tools• Common Storage DB• Architecture established• ICD developed

• Interface to SUT• Palette of Network Topologies• Automated host model configuration

• Procure Hardware for Navy lab• Software for C4I Cyber Labs• Limited Demonstration against GCCS/M 4.1 on CANES @ SPAWAR

IOC9/2017

• Correlation and analysis capability• Integrated Instrumentation Toolset

• Models of CND tools• Editable attacks models• Common attack library

• Procure Hardware for DISA JITC C4I Cyber Lab• Expanded Demonstration against GCCS/M 4.1 (@ SPAWAR) and JC2 (@ DISA)

FOC7/2018

• Cyber metric and visualization capability• Integrated Instrumentation Toolset

• Cyber event logging• Cyber metric computation• Cyber operating picture

• Procure Hardware for Army and Air Force C4I Cyber Labs• Full Demonstration with all Services

Capabilities developed incrementally enhancing fielded capabilities with provided processes, manuals, and training.

Page 37: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Capability Deployment

38

Capabilities by FOC(Could also be provided to

other DOD Cyber T&E Labs)

DeploymentDate

Network & Host Data Collection

Tactical Data Links (TDLs)

Collection

Analysis & Visualization

Constructive Simulator

SPAWAR SSC-PACSan Diego, CA

EOC12/2016

DISA JITCFt Huachuca, AZ

IOC9/2017

Air Force 46th TSEglin AFB, FL

FOC7/2018

Army EPGFt Huachuca, AZ

FOC7/2018

Nat’l Cyber RangeOrlando, FL

FOC7/2018

DoD IA/Cyber RangeQuantico, VA

FOC7/2018

Regional Service Delivery Points (RSDP)

FOC7/2018

= Capability to be deployed at site = Capability will not be deployed at site

due to lack of requirement.

Page 38: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key Performance Parameters (KPPs)

39

KPP Key Performance Parameter Threshold Objective

1 Data Collection: Cyber TASE shall provide configurable data collection

for networks, hosts, applications, and TDLs.

Manual data collection

configuration.

Automatic data collection

configuration.

2

Analysis: Cyber TASE shall provide automated near real-time and post-

test analysis that uses integrated and correlated test data to associate

test events with the configured metrics, threat actions, SUT

performance, and mission performance..

Pre-defined analysis

techniques with integration

and correlation of data from

devices and sensors.

Customizable analysis techniques

with configurable integration and

correlation of data from devices and

sensors.

3

Near Real-Time Visualization Environment: Cyber TASE shall provide a

near real-time visualization environment capable of user configurable

representations demonstrating the threat, its propagation through the

SUT, and its impact on mission performance.

Visualization of networks,

hosts, and threats as well as

the impact of the threats on

the SUT, mission

performance.

Visualization of applications,

network nodes, and threat

propagation.

4

Post-Test Visualization Environment: Cyber TASE shall provide

graphical representation of the results of statistical data from Post-Test

analysis of the threat, its propagation through the SUT, and its impact

on mission performance.

Visually represent analysis

results, and provide the

ability to playback the near

real-time visualization with

additional analytics.

Configurable post-test visualization

capability and visual representation

of the results of analysis on

applications and threat propagation

through the environment.

5

Constructive Model Scalability and Set-up Time: Cyber TASE shall

support timely configuration of a large number of nodes in order to

create an operationally relevant environment.

10,000 nodes in 1 week 1,000,000 nodes in 1 week

6

Constructive Vulnerability and Attack Models: Cyber TASE shall

provide pre-defined and user configurable vulnerability and attack

models representing threats in the constructive environment.

Attack and attack vector

models to exploit

vulnerability models based on

open source databases (i.e.

CVE/CWE).

Attack and attack vector models

based on real intelligence sources

and ability to attack the SUT from

the constructed model.

Page 39: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Overall (1 of 2)

40

Capabilities Threshold Objective

Usability. Cyber TASE shall ensure that the developed

capabilities provide the ease of use.

Semi-automated, moderately skilled and

specialized personnel required to install, configure

and use Cyber TASE capabilities [IAT (Information

Assurance Technical)-2, DoD 8570.01-M]

Highly automated, only limited skilled and

specialized personnel required to install,

configure and use Cyber TASE capabilities

and expose APIs for expert users

Consistency. Cyber TASE shall ensure that the developed

capabilities are uniform in nature.

consistent users experience across all

components.

consistent users experience across all

components.

Flexibility. Cyber TASE shall have a modular design for

reusability and future enhancementAbility to use components individually.

All components should be designed for

ease of reuse and be platform-independent

where possible

Access Control. Cyber TASE shall provide access control.Limited access control (compliance with DoDI

8500).Comprehensive user access control.

Reliability. Cyber TASE shall produce stable and

consistent results. Test-retest reliability assesses the

degree to which test results are consistent from one test

administration to another. Measurements are collected

from a single rater who uses the same methods or

instruments and the same testing conditions

Ability to result correlation coefficient of 0.999,

and a low mean standard deviation

Ability to result correlation coefficient of

0.99999, and a low mean standard

deviation

Scalability. Cyber TASE shall scale with SUT complexity,

data volume, and data rate.

Enhanced Testing instrumentation and LVC must

scale to 10,000 nodes with no loss of fidelity.

Enhanced Distributed Testing

instrumentation and LVC must scale with

no loss of fidelity.

Page 40: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Overall (2 of 2)

41

Capabilities Threshold Objective

Joint Test Interoperability. Cyber TASE shall allow

the data to be transferred across test

environments.

Sharing of data across JMETC. Sharing across any provided test network.

Safety. Cyber TASE will be designed with

characteristics eliminating or mitigating

environmental, safety, health risks.

The user shall have the ability to train, operate, maintain, and dispose of

the system in full compliance with applicable U.S., foreign, and

international environmental quality laws, regulations, executive orders,

international agreements, and DoD and DoN policies. The design,

production, operation, maintenance, and disposal or the system shall

eliminate, or minimize to the greatest extent possible, adverse

environmental quality impacts, environmental, safety, and occupational

health hazards, and ESOH risks.

The user shall have the ability to train, operate,

maintain, and dispose of the system in full

compliance with applicable U.S., foreign, and

international environmental quality laws,

regulations, executive orders, international

agreements, and DoD and DoN policies. The design,

production, operation, maintenance, and disposal or

the system shall eliminate, or minimize to the

greatest extent possible, adverse environmental

quality impacts, environmental, safety, and

occupational health hazards, and ESOH risks.

Training. Cyber TASE shall provide sufficient

training of the developed capabilities.

Training manual and personal instruction presentation

provided for basic usability of Cyber TASE capabilities.

Training course provided to provide

proficiency in Cyber TASE capabilities.

Logging. Cyber TASE shall provide logging of

events that occur within the Cyber TASE system.User access events, error messages, and system configuration. Log all events in Cyber TASE system.

Documentation. Cyber TASE shall provide

sufficient documentation of the developed

capabilities.

Generic configuration manual and users guide provided.

Critical APIs documented for service re-use. It shall include

how to implement security patches for the OS, applications,

database platforms (storage) or third-party software or

hardware. Include specifications of disposal or replacement.

Detailed configuration manual and user guide

provided. Thorough documentation of all

APIs documented for service re-use.

Page 41: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Instrumentation Generic

42

Capabilities Threshold Objective

Unified Report. Cyber TASE shall support the generation of

an integrated, unified report.

Generate Reports capable of using all the data

analysis available.

Generate Customizable, Stackable Reports of

Varying Levels of Detail.

Single Pane of Glass. Cyber TASE shall support the effective

viewing of the instrumentation user interfaces on available

display resources.

Centralized instrumentation user interfaces viewable

on a single pane of glass

Centralized instrumentation user interfaces

viewable across multiple screens.

Page 42: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Network Data Collector

43

Capabilities Threshold Objective

Network Data Collection. Cyber TASE shall monitor,

capture and store in real-time TDL data, network data, and

application data from various sensors, network devices and

network links.

Collect Application Layer, Presentation Layer, Session

Layer, Transport Layer, Network Layer and

Data Link layer data from all network devices and

links.

Collect Application Layer, Presentation Layer,

Session Layer, Transport Layer, Network

Layer and

Data Link layer data from all network devices

and links intelligently and provide for Physical

Layer Instrumentation via APIs.

Network Data Consolidation. Cyber TASE shall provide the

network data to the analysis module for both real-time and

post-test analysis.

Transfer pre-processed data (i.e. summaries) to the

analysis engine.

Transfer relevant, pre-processed data to the

analysis engine.

Network Data Formats. Cyber TASE shall collect data using

methods and data formats applicable to the network

devices and links.

The supported methods and formats shall include

packet capture, Simple Network Management

Protocol (SNMP), NetFlow, and network device logs.

The supported methods shall include

additional data formats.

Network Data Encryption. Cyber TASE shall be capable of

decrypting network packets when provided private keys.

Be capable of decrypting Secure Sockets Layer

(SSL)/Transport Layer Security (TLS) when provided

the private keys.

Be capable of decrypting SSL/TLS/Other when

provided the private keys.

Page 43: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Host Data Collection

44

Capabilities Threshold Objective

Host Data Collection. Cyber TASE shall monitor, capture

and store in real-time host data from various host devices.

Collect application view and application internal

data, application controller layer in the least

obtrusive way possible including windows Common

Language Runtime (CLR) and JRE, OS System Services

data including log data and Kernel data. Support

Windows XP, Vista, 7, 8, Windows Server 2003, 2008,

2012, Red Hat Enterprise Linux 5, Ubuntu 10.0.4,

Solaris 10, Mac OS X 10.7, VMWare ESX 4, HyperV,

Citrix XEN 3 or newer, Apache 2.0 or newer, IIS

(Internet Information Services) 6 or newer, Microsoft

Exchange Server 2008 or newer.

Collect application view and application

internal data, application controller layer in

the least obtrusive way possible including

windows CLR and JRE, OS System Services

data including log data, Kernel data,

Hardware Abstraction Layer (HAL) data,

support Firmware and Basic Input/Output

System (BIOS) Instrumentation for direct

Hardware (HW) access, and provide

Application Program Interface (API) for HW

Instrumentation via Joint Technical

Architecture Group (JTAG). Support Windows

NT, XP, Vista, 7, 8, Windows Server 2003,

2008, 2012, Red Hat Enterprise Linux 5,

Ubuntu 10.0.4, Solaris 10, Mac OS X 10.7,

VMWare ESX 4, HyperV, or Citrix XEN 3 or

newer, Apache 2.0 or newer, IIS 6 or newer,

Microsoft Exchange Server 2008 or newer.

Host Data Consolidation. Cyber TASE shall provide the data

to the analysis module for both real-time and post-test

analysis.

Transfer pre-processed data (i.e. summaries) to the

analysis engine.

Transfer relevant, pre-processed data to the

analysis engine.

Host Data Formats. Cyber TASE shall collect data using

methods and data formats applicable to the host OSs (both

x86 and x64) and applications.

Support Windows XP, Vista, 7, 8, Windows Server

2003, 2008, 2012, Red Hat Enterprise Linux 5,

Ubuntu 10.0.4, Solaris 10, Mac OS X 10.7, VMWare

ESX 4, HyperV, or Citrix XEN 3 or newer

Support Windows NT, XP, Vista, 7, 8,

Windows Server 2003, 2008, 2012, Red Hat

Enterprise Linux 5, Ubuntu 10.0.4, Solaris 10,

Mac OS X 10.7, VMWare ESX 4, HyperV, or

Citrix XEN 3 or newer

Page 44: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)TDL Data Collection

45

Capabilities Threshold Objective

Tactical Data Link Types. Cyber TASE shall collect data using

methods and data formats applicable to various TDL

sensors and device.

Support collection of Link 16 data.

Support collection of Advanced Tactical Data

Link (ATDL)-1, Link 11, Link 16, Tactical

Targeting Network Technology (TTNT),

Warfighter Information Network-Tactical

(WIN-T), and Link 22 data.

Tactical Data Link (TDL) Collection. Cyber TASE shall

monitor, capture and store in real-time TDL data from

various sensors and devices.

Collect link data and terminal data Collect all relevant data.

Tactical Data Link (TDL) Gateway/Translator/ Forwarders.

Cyber TASE shall monitor, capture and store in real-time

TDL data from various sensors and devices.

Collect link data and terminal data. Collect all relevant data.

Tactical Data Link Consolidation. Cyber TASE shall provide

the data to the analysis module for both real-time and post-

test analysis.

Transfer pre-processed data (i.e. summaries) to the

analysis engine.

Transfer relevant, pre-processed data to the

analysis engine.

Page 45: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Data Storage (1 of 2)

46

Capabilities Threshold Objective

Local Storage. Cyber TASE shall provide local storage at the

data collection points.

Temporary storage of Level 1 (raw) data for network,

host, and TDL collectors.

Storage of Level 1 (raw) data for the duration

of the test.

Distributed Storage. Cyber TASE shall provide distributed

storage for network, host, and TDL collection, analysis, and

visualization in an intelligent manner

Distributed Storage for the duration of the test,

including post-test analysis. Consistent with Service

storage policies.

Distributed Storage Long-term, Consistent

with Service storage policies.

Secure Storage Data. Cyber TASE shall provide

confidentiality and integrity for the stored data.

Protect data at rest in accordance with DoD

guidance.

Protect data both at rest and in-motion in

accordance with DoD guidance.

Storage Performance. Cyber TASE shall provide sufficient

data storage throughput.

Storage write rate matches collection rates. Storage

read rate exceeds collection rates.

Storage write rate matches collection rates.

Storage read rate exceeds collection rates.

Local Storage Availability. Cyber TASE shall provide

sufficient availability to meet data analysis and visualization

needs.

99.999% availability during test event. 99.99999% availability during test event.

Distributed Storage Availability. Cyber TASE shall provide

sufficient availability to meet data analysis and visualization

needs.

97% 98.5%

Storage Data Isolation. Cyber TASE shall isolate data by

test.Support multiple simultaneous test events. Support multiple simultaneous test events.

Page 46: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Data Storage (2 of 2)

47

Capabilities Threshold Objective

Storage Data Sanitization. Cyber TASE shall support the

ability to sanitize the data storage.

Sanitization of data in accordance with applicable

security standards within 24 hours.

Sanitization of data in accordance with

applicable security standards within 6 hours.

Storage Data Export. Cyber TASE shall provide a

mechanism to export the data from the storage.Configurable export from distributed data storage.

Configurable export from localized and

distributed data storage.

Storage Data Import. Cyber TASE shall provide a

mechanism to import data into the storage.

For limited, specific, pre-defined formats including

Comma Separated Values (CSV) files

eXtensible Markup Language (XML), other

formats

Centralized Control of Storage. Cyber TASE shall provide

mechanisms to perform storage management from a

centralized location.

Control data collection initiation and completion,

data transfer, and storage

import/export/sanitization.

Automation and scheduling of control data

collection initiation and completion, data

transfer, and storage

import/export/sanitization.

Page 47: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Analysis

48

Capabilities Threshold Objective

Data Aggregation. Cyber TASE shall enable data

aggregation across multiple data sources.Aggregate data from all devices and sensors.

Aggregate data from all devices and sensors

in a configurable, intelligent manner.

Analysis API. Cyber TASE shall provide an API to allow other

data types to be processed in the analysis and visualization

environment.

Provide pre-defined documented API Provide customizable documented API

Analyze Data from Multiple Sources. Cyber TASE shall be

capable of performing analysis on collected data.Correlation of the pre-processed data.

Multiple analysis techniques of all data

available in the data storage.

Analysis Algorithms. Cyber TASE shall provide analysis

algorithms and the ability to create and edit the algorithms.Pre-defined and customizable algorithm sets. Pre-defined and customizable algorithm sets.

Near Real-Time Performance. Cyber TASE shall provide

sufficient throughput of the analysis engine.

Analysis to process test data in near real-time to

support testing situational awareness.

Analysis to process the raw data in near real-

time.

Post-Test Performance. Cyber TASE shall provide sufficient

throughput of the analysis engine.

Complex post-test analysis to support formulating

conclusions from test data.

Complete set of post-test analysis tools

perform complex analysis.

Data Format Conversion. Cyber TASE shall provide the

ability to convert the collected data to common formats.Convert data to format necessary for processing. Convert all data to TENA-compliant format.

Define MoEs/MoPs. Cyber TASE shall allow users to define

their MoEs and MoPs to be scored for the SUT.

Allow users to define a limited set of critical MoPs

and MoEs.

Allow users to define a full set of integrated

MoPs and MoEs.

Page 48: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Visualization

49

Capabilities Threshold Objective

Visualization Display. Cyber TASE shall provide default

interactive visualization display formats and the ability to

create and edit the displays.

Pre-defined and customizable display formats.Pre-defined and customizable display

formats.

Visualization Sources. Cyber TASE shall provide the ability

for the user to select the sources of the visualization

engine.

Sources are data collection points, centralized

storage, and analytic output

Sources are data collection points, localized

storage, distributed storage, analytic output

Visualization Performance. Cyber TASE shall provide a

visualization engine with near real-time capability.

Visualization engine can display the analytic data in

near real-time.

Visualization engine can display the analytic

data or source data in near real-time.

Visualization Cyber Operating Picture. Cyber TASE shall

provide a visualization of an integrated cyber operating

picture.

Include mission timeline, critical mission events, and

critical metrics display.

Include mission timeline, all mission events,

and all metrics display.

Export Visualizations. Cyber TASE shall provide a

mechanism to export the visualizations.Export visualization reports. Export visualization data and reports.

Multiple Source Playback. Cyber TASE shall be able to

record and playback Cyber test events.Sources are screen captures, and critical event logs.

Sources are screen captures, critical event

logs, audio, and video, and analysis results.

Page 49: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Constructive Simulation (1 of 3)

50

Capabilities Threshold Objective

Constructive to Physical Interface. Cyber TASE shall

provide a framework to interface constructive simulator

with physical platforms.

Support IP based interfaces. Support IP and non-IP based interfaces.

Constructive Device Palette. Cyber TASE shall provide a

library of pre-defined network devices for use in the

constructive simulation environment.

Standard Commercially available IT devices. Standard Commercial and Military devices.

Constructive Platform Palette. Cyber TASE shall provide a

library of pre-defined network platforms in the constructive

simulation environment.

Air Force Base, Army Base, Army Remote Unit, Navy

Ground Station, Navy Afloat Platform, and Joint

Collaboration Sites

Air Force Base, Army Base, Army Remote

Unit, Navy Ground Station, Marine

Command, Navy Afloat Platform, and Joint

Collaboration Sites

Constructive Topology Import. Cyber TASE shall provide

the mechanism to import network topologies into the

constructive simulation environment.

Support Visio Import and configuration of network

elements i.e. fire walls, routers, switches, etc.

Support Visio Import, Network Configuration

Files, EMC Smarts Topology View

Constructive Statistics Collection. Cyber TASE shall allow

users to generate and export statistics collected from

sensors, network, TDL, and application

Pre-defined levels of statistics detailsCustomizable, Stackable Reports of varying

levels of statistics detail

Constructive Host Vulnerability Model Automation. Cyber

TASE shall provide an automated mechanism to configure

the vulnerabilities of a modeled host.

Support semi-automated import from host scanning

tool.

Support fully-automated import from host

scanning tool.

Constructive Computer Network Defense (CND) Device

Model. Cyber TASE shall provide a library of pre-defined

CND device models within the constructive simulation

environment.

Support model of Firewalls, Intrusion Prevention

System (IPS)/ Intrusion Detection System (IDS),

Network Admission Control (NAC)

Support model of Firewalls, IPS/IDS, Network

Admission Control (NAC), and other security

appliances.

Constructive Computer Network Defense (CND) Device

Configuration. Cyber TASE shall allow users to configure

the CND models to reflect the real world configurations

within the constructive simulation environment.

Customizable pre-defined configuration and

automated import of configuration files from

vendors like Cisco ACL, Sidewinder.

Customizable pre-defined configuration and

automated import of configuration files from

other common vendors.

Page 50: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Constructive Simulation (2 of 3)

51

Capabilities Threshold Objective

Constructive Attack Model Automation. Cyber TASE shall

provide a mechanism to automatically configure attack

models within the constructive simulation environment.

Support the ability to import attack models.Support fully-automated import from attack

tools.

Constructive Attack Model. Cyber TASE shall provide the

capability to model multi-hop sequences of attacks.Pre-defined attack sequence.

Statically and dynamically defined attack

sequences.

Constructive Attack Model Library. Cyber TASE shall

provide a library of pre-defined attack sequences.

Pre-defined chain of attacks and allow user to create

and save custom attack sequence models.

Pre-defined chain of attacks and allow user to

create and save custom attack sequence

models and automatically suggest new attack

sequences and chains

Constructive Configurable Logging. Cyber TASE shall allow

the user to enable or disable logging data and results within

the constructive simulation environment.

Enable/Disable log data and results across the

network and application layers.Enable/Disable log of all simulation events.

Constructive SUT Scoring. Cyber TASE shall allow the user

to define MoEs/MoPs associated with the SUT model within

the constructive simulation environment.

Produce a pre-defined set of critical MoPs and MoEs,

aided with manual analysis.

Produce pre-defined and SUT customizable

MOPs and MOEs.

Constructive Mission Scoring. Cyber TASE shall allow the

user to define metrics associated with mission scenario

events within the constructive simulation environment.

Produce a set of critical* metrics, aided with some

manual analysis and identify status of mission critical

messages

Produce a full set of integrated metrics.

Constructive Display. Cyber TASE shall provide the ability to

display metrics and associated thresholds within the

constructive simulation environment.

Display the pre-defined metrics necessary for

analysis.Display all of the metrics collected.

Constructive Cyber Operating Picture. Cyber TASE shall

provide an integrated cyber operating picture within the

constructive simulation environment.

Include mission timeline, critical mission events, and

critical metrics display

Include mission timeline, all mission events,

and all metrics display

Page 51: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Key System Attributes (KSAs)Constructive Simulation (3 of 3)

52

Capabilities Threshold Objective

Constructive Distributed Environment. Cyber TASE shall

provide the ability to execute in a distributed (either locally

or geographically) environment with multiple instances of

the constructive simulator.

Support 2 instances of constructive simulatorsSupport more than 2 instances of

constructive simulators

Constructive Hybrid Environment. Cyber TASE shall provide

the ability to execute in a federated environment consisting

of multiple Modeling and Simulation (M&S) systems.

Support 2 instances of M&S systems.Support more than 2 instances of M&S

systems.

Constructive APIs. Cyber TASE shall support a set of

predefined APIs to interface with other M&S systems.

Support DoD High Level Architecture (HLA) with 1

other M&S system.

Support DoD High Level Architecture (HLA)

with 2 or more other M&S systems.

Constructive Control Plane Protection. Cyber TASE shall

provide safety mechanisms to prevent a cyber attack’s

ability to extend onto the control plane within the

constructive simulation environment.

Control plane protection provided by VPN.Control plane protection provided by VPN

and other mechanisms.

Constructive Palette Modifications: Cyber TASE shall

support the ability for the user to edit palettes or the

objects therein.

All Palettes and Objects. All Palettes and Objects.

Constructive modeling of in-place threats: Cyber TASE

models shall support the ability to assess in-place threats.Payload effects Beaconing, Command and Control (C2)

Constructive remediation model automation: Cyber TASE

shall support the ability to model remediation and cleanup.

Models of the operational network must continue to

operate and pass data in a representative manner as

the remediation to remove a threat proceeds

Model of future networks

Page 52: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Program Schedule

FY13 FY14 FY15 FY16 FY17 FY18

Milestone

Enhanced Solution Phase

Phase 0

Requirements Development and

Planning

Phase I

Concept Development and Preliminary

Design

Phase II

System Development

ESP Phase 1 Phase 2

Requirements Development

Analysis of Alternatives

Cyber Alignment

Final Report

TCRD Acq Strat& Afford

PMR PDR IOC FOCCDR

Sustainment Strategy Development (LCSP)

Tools Trade Studies

Detailed System Design

Component Design

Collector & Simulation Development

Initial Delivery &Demonstration

Development of Use Cases

Development of TCRD

Development of Program Plan (PMP)

Trade Studies Bounding Analysis

Phase 0

EOC

System Design

SecondaryDelivery & Demo

FinalDelivery & Demo

VisualizationDevelopment

Analysis & Simulation Development

Page 53: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Management Structure

CTEIP(TRMC)

Chris Paust

Service Lead – Navy(OPNAV N84)

Deborah Giordano

Execution Oversight(NAVAIR 5.0C)Bruce Eanes

Cyber TASE(SSC Pacific)

Michael Winslow, PMRaheleh Dilmaghani, CHENG

Instrumentation

(Air Force)46 Test Squadron

L-V-C Environment

(Navy)SSC Pacific

Logistics & Demo

(Army and DISA)EPG and JITC

Blue and Gray Teams

(Joint)Army EPG, DISA JITC,

46 TS, SSC-PAC

Red Team

(Army)TSMO

White Cell

(Army and DISA)EPG and JITC

JMETCChip Fergusson

A.J. Pathmanathan

National Cyber RangePete Christensen

Threat System Management OfficeSkip Tornquist

OSD-DOT&ELCDR Chris Werber

Cliff Liang

Page 54: Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation

Army Department of Defense

DASD-DT&E / TRMC

CTEIP

Air Force

Cyber TASENavy

DISA

UNCLASSIFIED//FOUO

Phase 1 IPT Structure

55

Program Management IPT

Logistics IPT

Systems Engineering, Integration, & Test IPT

Instrumentation WIPT

Constructive SimWIPT