1

17th ITEA Engineering Workshop:

System-of-Systems in a 3rd Offset

Environment: The Future is Now

Mr. Derrick Hinton

Principal Deputy Director

Test Resource Management Center

January 25, 2017

2

• TRMC Overview

• TRMC Update

• FY2017 NDAA

• Tenets of the Third Offset Strategy

• Intellectual Capital and Innovation Investment

• DoD S&T Spending

• Use Case: Maritime Strike

• Goal

• Points of Contact

Topics of Discussion

3

TRMC Overview

4

TRMC Organization

Chief Financial Officer

DD, T&E Range Oversight

Agency RO

Army RO

AF RO

Navy RO

PM, CTEIP DD, Major Initiatives and Technical Analysis

Deputy EA for Cyber Test Ranges

PM, REP

Deputy PM, CTEIP

Range Director, NCR

Deputy Range Director, NCR

Chief Operating Officer

PM, T&E/S&T

Deputy PM, T&E/S&T

PM, JMETC

Director, TENA SDA

As of 23 Jan 2017

Principal Deputy, TRMC

Mr. Paul D. Mann (Acting)

Under Secretary of Defense for Acquisition, Technology & Logistics

Mr. James A. MacStravic (Acting)

Mr. G. Derrick Hinton (Acting)

DASD(DT&E) Dir, TRMC

5

TRMC Mission

TRMC Mission

T&E Infrastructure T&E Investments

MRTFB Planning, Assessment

and Oversight

Strategic Plan for DoD T&E

Resources

Updated every 2 years in coordination

with the Military Departments, Defense

Agencies, and specified OSD offices

T&E Budget Certification

Certify to the Secretary of Defense

adequacy of DoD T&E Budgets

Test & Evaluation/Science &

Technology (T&E/S&T) Program

Central Test & Evaluation

Investment Program (CTEIP)

T&E advanced technology development

T&E capability development

Joint Mission Environment Test

Capability (JMETC)

T&E distributed test integration

National Cyber Range (NCR)

Cyber T&E Environment

6

TRMC Update

7

1) Develop the Biennial Integrated Plan which includes:

a. Maintaining comprehensive list of test capabilities (DoD and non-DoD)

b. Organizing and managing designated test capabilities

c. Certify DoD cyber range investments

d. Perform assessments/analysis per the SD direction

2) Certify component cyber test infrastructure investments

3) Generate requirements and standards for cyber security test

infrastructure. (i.e. cyber event and threat data language)

4) Maintain a list of government and non-government cyber test ranges

5) Use the T&E Board of Directors (BoD) as an advisory board

Executive Agent (EA) for Cyber Test

Ranges Responsibilities

Key: Statute Policy

8

NCR Complex Concept

• FY17 (Current): 8 Test Beds, OPTEMPO of

70 events/year, Single Location with

distributed capability via JMETC MILS

Network (JMN) and Joint Information

Operations Range (JIOR)

• FY21 (Future): 40 Test Beds, ~400-500

events/year, multiple locations

seamlessly integrated via JMN & JIOR

VISION: Distributed

Access to

Persistent Cyber

T&E Capabilities

Mission: Improve the resiliency of our warfighters in the cyber-contested battlespace by conducting

testing and training events in operationally-representative cyberspace environments

NCR Complex

= NCR Flagship

= Existing NCR Network Connectivity= Potential/Planned Presence of NCR

Nationwide NCR Network

• Refinement of operating processes and improvements in automation tools

will promote significant efficiencies across the NCR Complex along with

seamless interoperability, leading to reduced operating costs.

• Augmenting existing capabilities to deploy ALL Current and Future Systems

to include avionics HW&SW, ICS/SCADA, and more.

• The NCR Complex will be adaptive, rapidly reconfigurable, modular, and

scalable to support all needs of the Services.

• The NCR Complex Flagship is in Orlando, FL.

9

JMETC SECRET Network (JSN) Site Map

Ft Hood (2): CTSF, TTEC

WPAFB: SIMAF

As of 7 Dec 2016

Leverages the SECRET Defense Research and Engineering Network (SDREN) for connectivity

Operates only at the SECRET classification

Continuous monitoring, troubleshooting, and optimization of the end-to-end network infrastructure

Capable of supporting numerous simultaneous test events

Functional JSN Locations: 45 (access to 76 labs/facilities)

Planned JSN Locations: 6

Connection Points to Other Networks: 5

Pax River:

(10) E2C, E2D, MCL, ATR

ACETEF, SAIL, MFS,

E-2C SIL, UASIL, P-8

EP-3

JMETC SYSCON

East Interconnect Rtr.

Aberdeen: ATC-STE, ARL

Langley TDLITC

JS J6 DDC4 C4AD

Wallops Island SCSC

Dahlgren:(3) IWSL, DOC, ICSTFRapid-SIL/C2

Dam Neck CDSA

Newport News NGC VASCIC

McLean MITRE NSEL

Redstone (12): SMDC

RTC: DTCC, DISTL, AvSTIL

AMRDEC SSDD

ARMDEC SED: Patriot,

THAAD, FAAD, GSIL, JLENS,

MUSE, C-RAM

Crane (2):

NSWC RDT&E, NTN Lab

Moorestown (2):

CSEDS, CPTS

Army

Air Force

Navy

Marines

Joint

Industry

GTRI

Ft Huachuca: (4)

JITC, JTDL, CMIS, JTRS

Metro Park JMETC / TENA SDA Lab

Camp Pendleton:MCTSSA

China Lake (3):F/A-18, IBAR,

TSPIL

Edwards (2):

Corona: NSWC

Point Loma :

SSC-PAC 59140

SSC-PAC CTB LMMT

Point Mugu (4):ITEC, EW CyCon,AEA, Sea Range

W. Interconnect Rtr.

Rancho Bernardo NGC Triton

412th EWG IFAST

Ridley

Dugway Proving Ground

Eglin AFB (4):AOC, DTF, GWEF, KHILS Melbourne

NGC JSTARS

Keyport NUWC

Rome NY AFRL

Newport NUWC

Tinker AFB AWACS

Greenville Rivet JointWSMR IRCC

Raytheon Tucson

Yuma PG

LMCO Global Vision Network

Pittsfield GD LCS

Arlington IDT

Manassas LM ASWMHPCC

PMRF: MHPCC Lab

Hawaii

Ft. Sill, SMART Lab

Port Hueneme: NSWC

10

JMETC MILS Network (JMN) Site MapFunctional JMN Locations: 24 (access to 42 labs/facilities)

Planned JMN Locations: 10

Army

Air Force

Navy

Marines

Joint

Industry

Agency

Leverages the Defense Research and Engineering Network (DREN) for transport

Accredited to operate up to and including TS//SCI

Managed by the JMN NOSC at Redstone Arsenal, AL & CDSA Damneck

Capable of concurrently supporting numerous Test and Training events at varying classifications

x11

x2

x5x7

As of 4 Jan 2017

11

Regional Service Delivery Points

(FY17)

As of 4 Jan 2017

12

12

3445

61 69 72 74 75 78 76

15

31 42

0

20

40

60

80

100

120

140

20

07

2008

2009

2010

2011

20

12

2013

2014

2015

2016

JMN Sites

JSN Sites

Increase in JMETC Sites

As of 4 Jan 2017

13

NCR Complex Concept

• FY17 (Current): 8 Test Beds, OPTEMPO of

70 events/year, Single Location with

distributed capability via JMETC MILS

Network (JMN) and Joint Information

Operations Range (JIOR)

• FY21 (Future): 40 Test Beds, ~400-500

events/year, multiple locations

seamlessly integrated via JMN & JIOR

VISION: Distributed

Access to

Persistent Cyber

T&E Capabilities

Mission: Improve the resiliency of our warfighters in the cyber-contested battlespace by conducting

testing and training events in operationally-representative cyberspace environments

NCR Complex

= NCR Flagship

= Existing NCR Network Connectivity= Potential/Planned Presence of NCR

Nationwide NCR Network

• Refinement of operating processes and improvements in automation tools

will promote significant efficiencies across the NCR Complex along with

seamless interoperability, leading to reduced operating costs.

• Augmenting existing capabilities to deploy ALL Current and Future Systems

to include avionics HW&SW, ICS/SCADA, and more.

• The NCR Complex will be adaptive, rapidly reconfigurable, modular, and

scalable to support all needs of the Services.

• The NCR Complex Flagship is in Orlando, FL.

14

FY2017 NDAA

15

FY2017 NDAA Language Affecting TRMC and DT&E

• Dismantles USD(AT&L) Acquisition, Technology, and Logistics

• Establishes in its place:

– USD(R&E) Research and Engineering *Responsibility for developmental activities and programs

– USD(A&S) Acquisition and Sustainment

• Abolishes statutory requirement for DT&E

• Retains statutory requirement for TRMC

– But Director, TRMC is no longer directed to be a 3-star equivalent

• Did not strike language prohibiting DOT&E from overseeing DT&E

activities

FY2017 NDAA

Senate Report on FY2017 NDAA

• Directs Secretary of Defense study “to review appropriate balance

between developmental and operational T&E activities and the

resources required to accomplish related activities within OSD”

– Interim Report 01 Mar 2017 / Final Report 01 Aug 2017 / Implementation ~ Feb 2018

16

FY2017 NDAA Takeaways

• USD(AT&L) will be reorganized into USD(R&E) and USD(A&S)

• FY2017 NDAA included language to specifically eliminate various political

appointee positions (to include Senate confirmed and President

confirmed)

• One of the positions eliminated was DASD(DT&E)

• There was no change to the overarching TRMC language

• FY2017 NDAA included language to specifically eliminate grade levels

• One of the grade levels eliminated was D, TRMC (no longer required to be

3-star or equivalent)

• Our interpretation is that all of these changes were made to provide a

“blank slate” for efforts to make Acquisition more efficient

• Any major reorganizations arising out of the “balancing” study directed

by the Senate Report on FY2017 NDAA will not be implemented until

February 2018

17

Tenets of the Third Offset Strategy

18

Previous Offset Strategies

• “First Offset Strategy”

– Emphasis on nuclear deterrence to avoid the large increase in

defense expenditures that would be necessary to conventionally

deter Warsaw Pact forces during the 1950s

• “Second Offset Strategy”

– Following the Vietnam War, U.S. tolerance for defense

expenditures plummeted while Warsaw Pact forces outnumbered

NATO forces by three to one in Europe

– DoD sought technology to “offset” the numerical advantages

by holding adversary forces at risk before they could bring larger

forces to bear

• Emphasized: Intelligence, Surveillance, and Reconnaissance (ISR)

platforms; Precision-Guided Weapons; Stealth; and the expansion of

space’s role in military communications and navigation

Technology Enables Strategy

19

Third Offset Strategy

• “Third Offset Strategy”

– To “offset” advances in Anti-Access/Area Denial systems

– Promising technology areas include: robotics and systems

autonomy, human systems, miniaturization, biotechnology,

advanced computing and big data, and advanced manufacturing

– Potential components include:

• Autonomous Learning Systems Making Time Critical Decisions

• Delegating decisions to machines in applications that require faster-

than-human reaction times, i.e. Cyber Defense, Missile Defense, EW

• Human-Machine Collaborative Decision Making

• Exploiting the advantages of both humans and machines for better

and faster decisions, i.e. humans providing strategic guidance

combined with the tactical acuity of a computer

20

Third Offset Strategy (Cont’d.)

– Potential components include (cont’d.):

• Machine-Assisted Human Operations

• Machines helping humans perform better in combat

• Advanced Manned/Unmanned Systems Operations

• Employing innovative cooperative operations between manned and

unmanned platforms, i.e.. “swarm operations”

• Network-Enabled, Autonomous Weapon Systems, Hardened to

Operate in a Future Cyber/EW Environment

• Enabling for cooperative weapon systems operations in

communications-denied environments

FY 2017 will be a year of considerable war-gaming and testing of

theories and operational concepts.

The strategy is constantly being updated.

21

DoD Needs to Develop New Ways to Project Power

• Improved Intelligence, Surveillance, &

Reconnaissance

• Electronic Attack / Electronic Protection

• Surface to Surface Ship Missiles

• Ballistic and Cruise Missile Defense

BAE Systems Sea Lightning EX system

Technologically advanced capabilities needed for the future

• Improved Long-Range Precision

Strike

• Cyber and Space Capabilities

• Undersea Warfare

• Advanced Air Defenses

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

22

Rise of the CommonsCyber, Electromagnetic Spectrum & Space

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

Military operations increasingly depend on being able to operate in

places “no one owns” – the Commons

23

Alt

itu

de

, k

Ft

Directed Energy

• Offensive HEL

Notional 6th Generation Aircraft &

Armaments Technology Drivers

Computer Processing

• Voluminous data processing

• Sophisticated algorithms

• Autonomy

Low Observable

•Special Coatings

•Smart Skins

•Radar Cross Section

Super Cruise• Propulsion system

• Performance and operability

• Accurate flight conditions

• Fuel schedule optimization (max range)

Notional Mission

Data Communications• Real-time Satcom

• Sensor connectivity

24

Intellectual Capital & Innovation Investment

25

Engineering’s Best and Brightest

• Visual representation of subject keywords of 2015-2016 accepted doctoral dissertations from the top 12 engineering graduate

schools, as ranked by U.S. News and World Report

• Data Source: ProQuest Dissertations & Theses Global Database // Word Cloud Generator: Tagul

26

Engineering’s Best and Brightest

• Visual representation of subject keywords of 2015-2016 accepted doctoral dissertations from the top 12 engineering graduate

schools, as ranked by U.S. News and World Report

• Data Source: ProQuest Dissertations & Theses Global Database // Word Cloud Generator: Tagul

• 20% related to optimization

• 12% related to autonomy

• 10% related to RF spectrum /

wireless / LTE

2015-2016 Accepted

Doctoral Dissertations

at Top 12 Engineering Schools

27

10 Grand Challenges Where Engineering Can Have an Impact

• Published in Stanford Engineering Future

• http://soefuture.stanford.edu/impact

Au

ton

om

y

Big Data / KM

Cybersecurity

Optimization

28

DoD S&T Spending

29

Defense Research and Engineering (R&E) Strategy

1. Mitigate current and anticipated threat

capabilities

- Cyber - Electronic Warfare

- Counter Space - Counter-WMD

- Missile Defense

2. Affordably enable new or extended

capabilities in existing military systems

- Systems Engineering - Modeling and Simulation

- Capability Prototyping - Developmental T&E

- Interoperability - Power & Energy

3. Create technology surprise through

science and engineering

- Autonomy - Data Analytics

- Human Systems - Hypersonics

- Quantum Systems - Basic Sciences

Technology Needs

• Cyber / Electronic Warfare

• Engineering / M & S

• Capability Prototyping

• Protection & Sustainment

• Advanced Machine Intelligence

• Anti-Access/Area Denial (A2/AD)

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

30

Examples of FY 2017 S&T Investments Aligned to Defense R&E Strategy

• Mitigate

– Counter Weapons of Mass Destruction (~$0.9B)

– Cyberspace and Space (~$1.0B)

– Electronic Warfare (~$0.4B)

• Surprise

– High-speed Strike Weapons (~$0.3B)

• Affordability

– Advanced Manufacturing (~$0.14B)

– Prototyping Efforts (~$0.3B)

~$3.0B in S&T activities that align with the 3 principles

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

31

PB 2016 FY 16 S&TBasic Research BA1 & BA2/BA3 by Community of Interest (COI)

Components

S&T = $12.3B

Note:

- “Non-COI” includes Battlespace

Environments, M&S

Technology, and other

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

$223M

32

DoD S&T Communities of InterestWith Sub-Areas

• Advanced Electronics– Advanced Electronics - Electronic Materials

– Advanced Electronics - Electronics Integration

– Advanced Electronics - EO/IR Components

– Advanced Electronics - Microelectronics & Nanoelectronics

– Advanced Electronics - RF Components (sensors related)

• Air Platforms– Air Platforms - Aircraft Propulsion, Power and Thermal

– Air Platforms - Fixed Wing Vehicles

– Air Platforms - High-Speed/Hypersonics

– Air Platforms - Rotary Wing Vehicles

• ASBREM– ASBREM - Medical Radiological Defense

– ASBREM - Biomedical Informatics / Health Information Systems & Technology

– ASBREM - Clinical and Rehabilitative Medicine

– ASBREM - Combat Casualty Care

– ASBREM - Medical Chem-Bio Defense

– ASBREM - Military Infectious Diseases

– ASBREM - Military Operational Medicine

• Autonomy– Autonomy - Human/Autonomous System Interaction and Collaboration

– Autonomy - Machine Perception, Reasoning and Intelligence

– Autonomy - Scalable Teaming of Autonomous Systems

– Autonomy - Testing, Evaluation, Verification and Validation

• C4I– C4I - Advanced Computing/Software Development

– C4I - Human Computer Interfaces for Decision Making

– C4I - Information Collection / Management

– C4I - Networks and Communications

– C4I - Synthesis / Analytics / Decision Tools

33

Autonomy – Test, EvaluationValidation and Verification (TEVV)

• Fundamental science of V&V for autonomy

• Reduce time & cost to field emerging tech

• Must improve safety and reliability for

landscape of autonomous systems RDT&E

• New methods to augment gap in exhaustive

testing

• Run time (operational) assurances

• formal and enhanced analysis techniques

• New Design of Experiments (DOE) methods

for autonomous systems

• Guaranteed trust in Human-Agent Teaming

• Assessing risk operation in complex,

contested, uncertain environments

• Early verification of coordinated actions by

multiple agents

Central Tech Challenge:

From algorithms up to scalable teams of

multiple agents--New V&V technologies needed

to enable complete system evaluation

V&V of

DesignNew T&E

Methods

34

Unmanned & Autonomous Systems Test (UAST)

Stress Testing of Autonomy Architecture (STAA)

Description: STAA is an automated tool used to

test the robustness of black-box UASs to

unexpected operating scenarios. It feeds a UAS

inputs that trigger software bugs in other systems

tested, along with a mix of good / bad data, and

automatically detects safety violations.

Current Status: Complete

Transition Partner: Army Aberdeen Test Center,

Army Autonomous Ground Resupply (AGR)

program

FY16 Accomplishments:

Developed improved testing processes and two-channel architecture for safe autonomy (2CASA) for AGR program at TARDEC

Developed anomaly-detection capabilities to detect UAS failures without a priori invariants (Aug/16)

Deliverables

Software Architecture and interface document,

reports, and briefings

Start / Finish: Sep 11 – Sep 16

Key Future Events / FY17 Plans

Transition and T&E/S&T Contract Closeout

Unmanned Systems Under Test by STAA

J1939

35

DoD S&T Spending SummaryWhere We Are Today

• FY17 S&T President Budget Request (PBR) is $12.5 billion, as

compared to FY 2016 PBR of $12.3 billion (FY16 appropriation

was $13.0 billion)

– S&T is 2.4% of DoD Topline

– S&T maintained 0% real growth FY16 to FY17 PBR

• Basic Research is funded at $2.1 billion, as compared to FY16

PBR of $2.1 billion (FY16 appropriation was $2.3 billion)

• Defense Advanced Research Projects Agency (DARPA) is funded

at $3.0 billion RDT&E to develop technologies for revolutionary,

high-payoff, military capabilities

• S&T funding for each Military Department is between $2.1 - $2.5

billion

• Funds aligned to support strategic guidance and S&T priorities

Taken from the Deputy Director, Plans & Programs (R&E)’s briefing to the NDIA S&T Conference (April 2016)

36

Use Case: Maritime Strike

37

Maritime Strike Operational T&E

Kill Web Performance

• Weapon-to-weapon,

sensor-to-weapon

• Collaborative behavior in

final targeting process

Heterogeneous Massed Salvos:

• Legacy and future weapons

• Kinetic and non-kinetic effects

• Multiple launch sites

Homogeneous Weapon Performance

• Single weapon

• Kill chain assessment – pilot-to-weapon

• Individual maritime target with marginal

realism

Multiple Naval Combatant Targets

• Sophisticated IADS

• Cyber Capability

• GPS jamming

Current

Future

Weapons Effects Performance

• Mission effectiveness

• Commander’s flexibility

• Increased effectiveness of

legacy systems

Legacy

Future

Future

Legacy

Future

Notional Mission

38

Goal

39

2016 Strategic Plan for DoD T&E Resources

Goal: Actionable

Strategic Plan to Guide

DoD T&E Spending

Source: Institute for Defense Analyses (IDA) Report

“Cost of Testing Analysis Origin, Description, Data Sources, Assumptions and

Limitations, and Results” June 2015

~$9B in FY16

40

Points of Contact

41

Points of Contact

http://www.acq.osd.mil/dte-trmc/

Mr. G. Derrick Hinton

DASD(DT&E) / Director, TRMC

(Acting)

george.d.hinton2.civ@mail.mil

(703) 697-3443

Mr. Paul Mann

Principal Deputy Director, TRMC

(Acting)

paul.d.mann2.civ@mail.mil

(571) 372-2761

Dr. J. Brian Hall

Principal Deputy Director, DT&E

jason.b.hall20.civ@mail.mil

(571) 372-2755