Post on 16-Dec-2018
© 2014 Carnegie Mellon University
Enabling Success via Applying Modern
Software Engineering Processes, Methods and
Technologies in the Rapid Acquisition of
Operational Capabilities
Dr. Kenneth E. Nidiffer
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213
703-908-1117
26 Annual Software Technology Conference 2014
Meeting Real World Challenges through Software Technology
29 March – 3 April 2013
Long Beach, California
Logogram: Symbol developed by OSD (DCMO) for the DoD IT Acquisition Reform Task Force
2
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
2
Why Are IT Software-Intensive Projects Hard to Manage and Lead?
1939’s Science Fiction World of 2000 Actual World of 2000
Software is the building block for modern society – in
fact, the world runs on software!
3
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Overview, Scope and Motivation The military will be smaller and leaner, but it will be agile, flexible, ready and technologically advanced*
• Perspective
• The Problem Space
• The Solution Space (Pre-Decisional)
• What Success Looks Like
Sources: Keynote Address, Mr. Alan R. Shaffer, A, SERC, Feb 2014
H.R.1232 - Federal Information Technology Acquisition Reform Act
(passed House – Congress*Government, 28 Feb 2014)
H.R.1232 (2014) - Federal Information
Technology Acquisition Reform Act*
Global Rapid Acquisition
of IT Operational
Capabilities
4
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
4
Perspective: IT Software Landscape
What are the opportunities?
+ + + + …
Transportation
Infrastructure
Healthcare
Infrastructure
Banking & Financial
Infrastructure
Energy & Utilities
Infrastructure
Communications
Infrastructure
Includes all:
• System of Systems
• Architecture
• Services
• Networked Hardware/ Platforms
• People who digitally connect to
cyberspace
Source: SEI
5
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Perspective: Improving Efficiency and Effectiveness in IT/Cyber Acquisitions in DoD
Source: Director, Command and Control, Programs & Policy (OSD) - Pre-Decisional
6
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Perspective: Fiscal IT Budget and Reliance
• The federal government reportedly plans to spend at least $82
billion on IT in fiscal year 2014.*
• Defense plans to spend over $39 billion—$5.5 billion on
classified systems, $9 billion on acquisitions, and $25 billion on
operations and maintenance.*
• Deep reliance on commercial infrastructure, services, and
products will grow and is a double-edged sword
Reference: Leveraging Best Practices and Reform Initiatives Can Help Defense Manage Major
Investments. GAO-14-400T: Published: Feb 26, 2014. Publicly Released: Feb 26, 2014.
7
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Perspective: A New Reality Global Dimensions Affect IT Science and Technology
Pace of Technology
Rise of the Commons
Expanding Global
Knowledge Base
Information Agility
Mass Collaboration
Economic and S&T Mega-
Trends
Technology Commercialization
Black Swan Syndrome
Source: Dr Reginald Brothers’ chart
Deputy Assistant Secretary of Defense for Research
8
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: Cyber Compared with Other Sciences
PHYSICAL SCIENCE BIOSCIENCE COMPUTER/SOFTWARE/CYBER
SCIENCE
Origins/History Begun in antiquity Begun in antiquity Mid-20th Century
Enduring Laws Laws are foundational to
furthering exploration in
the science
Laws are foundational to
furthering exploration in the
science
Only mathematical laws have proven
foundational to computation
Framework of
Scientific Study
Four main areas:
astronomy, physics,
chemistry, and earth
sciences
Science of dealing with
health maintenance and
disease
prevention/treatment
Several areas of study:
computer science, software/
systems engineering, IT, HCI,
social dynamics, AI
All nodes attached to/relying on
netted system
R&D and Launch
Cycle
10-20 years 10-20 years Significantly compressed; solution
time to market needs to happen
very quickly
Source: SEI
HCI: Human Computer Interaction; AI: Artificial intelligence
9
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: Developing the Workforce: Recruiting, Training, Education, Retention
The development of cybersecurity professionals is not keeping pace with
the exponential growth of cybersecurity challenges faced by the DoD
and all critical infrastructure sectors.*
Source: SEI
10
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Building out capabilities to manage large
information technology projects has been a
sore spot for the Air Force.
Specifically, the service has been challenged
with developing IT acquisition talent among its
ranks, adopting and maintaining processes that
foster best practices and aligning acquisition
and cybersecurity strategies.
Lt. Gen. Charles Davis, the military deputy in the office of the
secretary of the Air Force for acquisition
Problem Space: Human Capital*
Source: C4ISR & Networks, Feb 2014
11
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: DoD IT Acquisition Cycle-Time - 32
MAIS*
Initial
Operational
Capability Planning Phase
Analysis of
Alternatives
Economic
Analysis
Milestone B
MS C
40
48
5
Test
Build Phase
Development
Cycle-Time Driven by Processes Developed to Counter a Cold War Adversary In Industrial Age Society
43
91
*Source: Defense Science Board Report, March 2009
** Source: Dr. William Scherlis, CTO, CEO
A Modality of Warfare – Software is the Material**
12
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: The Call For Change
Acquisition
• Long acquisition cycle-times
• Successive layers … built over years
• Limited flexibility and agility
Requirements
• Understanding and prioritizing requirements
• Ineffective role and communications in acquisitions
Test/Evaluation
• Testing is integrated too late and serially
• Lack of automated testing
Funding & Governance
• Program-centric, not capability-centric
• Overlapping decision layers
(e.g., multiple review processes)
• Lack of customer-driven metrics
• Funding inflexibility & negative incentives
13
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: An Effective Process for Major Defense Systems – But Not Very Agile for IT Systems*
Source: Defense Acquisition University
* Major Defense Systems Life Cycle Management System has Been Updated
to Address IT System Acquisitions Among Other Changes
a
14
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: Software-Reliant Acquisitions Can Be Difficult to Manage
According to Fred Brooks* software projects are difficult because of
accidental and essential difficulties
• Accidental difficulties are caused by the current state of our
understanding
— of methods, tools, and techniques
— of the underlying technology base
• Essential difficulties are caused by the inherent nature of software
— invisibility - lack of physical properties
— conformity
— changeability
— complexity
Dr. Fred Brooks
* Source: The Mythical Man-Month by Fred Brooks, Addison Wesley, 1995
15
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Problem Space: Rate of Technology Development and Adoption Is Growing
15 UNCLASSIFIED 15
High
Low
1980 1985 1990 1995
Sophistication
Required of Actors
Declining
So
ph
isti
cati
on
cross site scripting
password guessing
self-replicating code
password cracking
exploiting known vulnerabilities
disabling audits
back doors
hijacking
sessions
sweepers
sniffers
packet spoofing
GUI
automated probes/scans
denial of
service
www attacks
“stealth” / advanced
scanning
techniques
burglaries
network mgmt. diagnostics
distributed
attack tools
Staging
sophisticated C2
…next?
Increased GIG Complexity
and dependence equates to
lower entry barriers and
potential for increased
number of malicious actors
Sophistication
Of Available Tools
Growing
Defensive measures are outpaced by the well resourced sophisticated threat . . .
Source: DoD
16
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Software Engineering Processes, Methods and Technologies – Partial List
•Interim DoD Instruction 5000.02,
Operation of the Defense Acquisition
System, Nov 2013
•IT Body of Knowledge (ITBOK)
•Software Extension to Project
Managers BOK (SWX PMBOK)
•Software Engineering Body of
Knowledge (SWEBOK)
•Helix - Investigating the DNA of the
Systems Engineering Workforce
•Risk Management Framework (RMF)
for DoD Information Technology (DoD
Directives 8500/8510)
•IT Box
•Program Protection Plan (PPP)
•Graduate Software Engineering
Reference Curriculum (GSwERC)
•Body of Knowledge and Curriculum to
Advance Systems Engineering
(BKCASE)
•Software Assurance Community of
Practice (SwA COP)
•Software Engineering Competency
Model (SECOM)
•SE Role-Based Competency
•Joint Competency Experience
Accelerator
•Skills for the Information Age (SFIA)
17
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: IT is Different from a Weapon System - and Critical to Enable a more Resilient Cyber Environment
•Weapon platform centric
•Military unique requirements
•Development of military-
unique, breakthrough
technologies
•Development cycle of decade
or more
•Production decisions for
unique HW
•Service lives extending into
decades
•Enterprise network
centric
•Adapt commercial
capabilities for military needs
•Leverage commercial
technologies
•Technology cycle 12-18 months
•Procure commodity HW
•Periodic technology refresh to
avoid obsolescence
Weapon Systems IT& Business Systems
DOD Instruction 5000.02
Provides Different Acquisition Processes Sources: IT Acquisition Reform Task Force/MITRE Corporation
18
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
18
Solution Space: DoDI 5000.02 - 26 Nov 2013
Source: Defense Acquisition University &
DEPSECDEF Interim Policy on 26 NOV
2013
19
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Value Recognition of Software Engineering: Top-Paying Majors for New College Graduates in 2012
http://www.naceweb.org/s01232013/top-majors-salary-survey.aspx
20
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Refocusing University Curriculums: Alignment of Software and Systems Engineering
System Design
System Analysis
Software (SW) Requirements Analysis
Architectural SW Design
SW Subsystem Testing
Code and Unit Test
Detailed SW Design
System Testing
System Integrated Testing
SW System Testing
SW Integration Testing
SW Engineering SW Engineering
SW Systems
Engineering
SW Systems Engr.
Systems Engr.
SW Systems Engr.
Systems Engr.
Systems
Engineering
Three OSD Initiatives: Graduate Software Engineering Reference Curriculum (GSwERC)
& Body of Knowledge and Curriculum to Advance Systems Engineering (BKCASE)
SW = Software
21
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Recognizing the Breath of IT: A New Reality – IT BOK
Scope - Dimensions of the IT Acquisition Space
Source: IEEE, 2014 IT BOK = IT Body of Knowledge
22
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Information Technology (IT) Box
Applications &
System Software
Development &
Acquisition
Requirements Organization &
Oversight
Capabilities
Required $
$
Hardware
Refresh &
System
Enhancements
& Integration
JROC
Approved
IS ICD*
• Information Systems Initial Capabilities Document (ICD)
• Requirements Definition Package Sources:
Katrina McFarland, DoD ASD, C4ISR, 28 Feb 2014
•CJCSI 3170.01H, 10 Jan 2012; JCIDS Manual, 19 Jan
2012, DAU
Joint Concepts
Capabilities Based
Assessment
Strategic Guidance
MS A/B
O&S Engineering Analysis/ Design
ICD
(NF) Rapid Delivery
Full Deployment Decisions
IOC
CD
Agile Development
CD CD CD CD CD CD CD CD
RDP*
23
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Software Project Management Software Extension to the Project Management BOK
Source: Software Extension to the PMBOK® Guide Fifth Edition
24
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Improvements in Human Capital
•Software Engineering
Competency Model (SECOM)
•SE Role-Based Competency
Helix - Investigating the DNA of the Systems Engineering Workforce
Skills for the
Information Age (SFIA)
25
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space Example: Focus on Software Assurance
Image Source: www. technobuffalo.com
The level of confidence that software
functions as intended (and only as
intended) and is free of vulnerabilities,
either intentionally or unintentionally
designed or inserted as part of the software
throughout the lifecycle*.
* Source: DoDI 5200.44 Protection of Mission Critical Systems to Achieve Trusted Systems and Networks
(TSN), November 5, 2012
26
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: Cybersecurity Policy Alignment
CNSSP 22
IA Risk Management Policy for
NSS
Knowledge Service
DoDI 8500.01 “ Cybersecurity”
IT Definitions
Security Controls Guidance
Enterprise Governance
DoDI 8510.01
“Risk Management Framework
for DoD IT”
NIST SP 800-39
Managing Information Security
Risk
NIST SP 800-37
Risk Management Framework
NIST SP 800-30
Risk Assessment
NIST SP 800-53
Cybersecurity Controls and
Enhancements
CNSSI 1253
Categorization
Baselines
NSS Assignment Values
CNSSI 1253A
Implementation and
Assessment Procedures
CNSS 4009
Information
Assurance/Cybersecurity
Definitions
NIST SP 800-53A
Cybersecurity Control
Assessment Procedures
NIST SP 800-137
Continuous Monitoring
NIST SP 800-60
Mapping Types of Information
to Security Categories
NIST SP 800-160 (DRAFT)
Security Engineering Guideline
DoD NSS NIST
26
27
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space: DoDI 8510.01* “Risk Management Framework for DoD IT” - Adopts NIST’s Risk Management Framework, Used by Civil and Intelligence Communities (* Target Publish Date: 2Q FY14)
27
28
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Real-time
Modification of
Systems
2: Methods of Secure
Systems Development
1: Foundations for Software
Assurance
3: SwA Management &
Operation
Tailored
Trustworthy
Spaces
“Science of
Security”
Software
Composability
Digital Curation and
Forensics
Modeling,
Simulation, Testing &
Certification
Architecture for
Secure Systems
Domain Specific
Assurance
Mitigations
SwA for Agile
Software
Methodologies
Metrics
Using Big Data
Analysis to
Advance
Software
Assurance
Techniques
SwA
Economic
Incentives
SwA Core
Competencies,
Education &
Training
SwA
Workforce
Development
SwA in Highly Parallel,
High-Performance
Computing Environments Security in Socio-
technical
Computing Security of Mobile
Applications &
Platforms
Designing Secure
Cyber-Physical
Systems Critical Infrastructure
Resiliency & Catastrophic
Recovery
Electronic
Effects in SwA
Effective
Acquisition Policy
& Guidance
Vulnerability
Prevention and
Detection Tools &
Techniques
Supply Chain
Visibility
Cultivating SwA
Maturity
5: Critical Infrastructure
4: Emerging & Disruptive
Technology
Scaling of
Assurance
Techniques
Solution Space: Defining Areas of Research – Focus on Software Assurance
Intrinsic Internet
Infrastructure
Security
29
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Solution Space Example: SEI’s Software Assurance Capabilities
Secure Coding
• Coding standards in Java, C,
and C++
• Source Code Analysis
Laboratory (SCALe) to test
software applications for
conformance
30
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
What Success Looks Like The military , although smaller and leaner, will be agile, flexible, ready and technologically advanced
Source: Director, Command and Control, Programs & Policy (OSD) – Pre-Decisional
32
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
Contact Information
Dr. Kenneth E. Nidiffer, Director of Strategic Plans for
Government Programs
Software Engineering Institute, Carnegie Mellon University
Office: + 1 703-908-1117
Fax: + 1 703-908-9317
Email: Nidiffer@sei.cmu.edu
33
Meeting Real World Challenges through
Software Technology
Dr. Kenneth E. Nidiffer
NO WARRANTY
THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE
MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY
MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO
ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR
PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM
USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY
WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT,
TRADEMARK, OR COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the
rights of the trademark holder.
Requests for permission to use or reproduce should be directed to the Software
Engineering Institute at permission@sei.cmu.edu.
This work was created in the performance of Federal Government Contract Number
FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software
Engineering Institute, a federally funded research and development center. The
Government of the United States has a royalty-free government-purpose license to use,
duplicate, or disclose the work, in whole or in part and in any manner, and to have or
permit others to do so, for government purposes pursuant to the copyright license under
the clause at 252.227-7013.