Copyright © 2008 by Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP. All rights reserved. 1...

Copyright © 2008 by Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP. All rights reserved.

1

Factors Related to the Implementation of ISSE: A Quality Perspective

Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP

Doctoral Candidate, Information AssuranceUniversity of Fairfax

Vienna, VA (USA)

Director, IS Risk ManagementThe Children’s Hospital of Philadelphia

Philadelphia, PA (USA)

INCOSE International Symposium 2008


2

ISSE in the Acquisition of Secure IT

• Introduction• Background• Research Proposal• Research Review and Synthesis• Preliminary Conclusions• Preliminary Recommendations


3

Introduction

• Technical Risk Impacts the Corporate “Bottom Line”– Individual Companies Estimate Annual Losses of up to

$5M (Deloitte, 2006)• Loss of Mobile Devices

– Identity Theft Considered “Number Two Hot-Button” and “Crime of the 21st Century”• Viruses/Worms• Phishing/Pharming• Spyware/Malware

– Increasing Cost of “Compliance”• Sarbanes-Oxley (SOX)• Federal Information Security Act (FISMA)• Health Insurance Portability and Accountability Act

(HIPAA)• Payment Card Industry (PCI)


4

Introduction

• Technical Risk Generally Attributed to System Design, Development, and/or Configuration– System Design & Development

• Incorrect System– Inadequate Understanding of User Needs– System Specification Doesn’t Reflect User Needs

• System Incorrect– System Not Designed to Specifications– System Not Built to the Design

– System Configuration• Unnecessary Services• Incorrect Settings• Inadequate Patching


5

Introduction

• Ensures Requirements are Addressed Early in (and throughout) the System Development Life Cycle (SDLC)

LIFE CYCLE COST

O & S COST

Operations &Support Phase

Production & DeploymentPhase

ConceptPhase Development

Phase

R & D COST

PROGRAMPECULIAR

R & D COST

INVESTMENT COST

ACQUISITION COST

COST

INCOSE SE Hdbk ver. 2a


6

Introduction

• Return on Investment (ROI) Related to Stage “Defect” is Identified and Subsequently Corrected

INCOSE SE Hdbk ver. 3.1


Background

7

• Federal Law• National Policy• DoD Policy, Regulations• Service Policy, Regulations


Background

8

• DoD Rules and Regulations– CJCSI 6510.01D, IA & Computer Network Defense (CND)

• Policy & guidance for IA and CND– IA architecture– C&A– MAC levels– Defense-in-Depth

• “Incorporate … IA … into [IS] during all phases of [the] system design life cycle including analysis, design, development, test and operation …”


Background

• DoD Rules and Regulations (Cont)– CJCSM 6510.01, Defense-in-Depth: IA and CND

• Guidance & procedures for implementing Defense-in-Depth strategy and standards

• Defense-in-Depth elements:

9


Background

10

• DoD Rules and Regulations (Cont)– CJCSM 6212.01D, Interoperability and Supportability of

IT and National Security Systems• NR-KPP mandatory

– IA compliance one of 4 NR-KPP requirements• Specifies use of Information Systems Security

Engineering (ISSE)– IA component of the GIG– Throughout system lifecycle


Background

11

• DoD Rules and Regulations (Cont)– DoDD 8320.2, Data Sharing in a Net-Centric DoD

• “… data assets shall have associated [IA] and security metadata, and an authoritative source for the data …”

– DoDD 8580.1, IA in the Defense Acquisition System• Describes required & recommended levels of IA

activities related to acquisition– Describes the IA strategy– Prescribes an IA strategy submission and review

process• Captures the acquisition-related IA policies of 8500.1

and 8500.2


Background

12


13

Background

• DoD Rules and Regulations– DoDI 8500.1, Information Assurance

• Establishes IA policy and assigns responsibilities• Requires a defense-in-depth approach to IA

– DoDI 8500.2, IA Implementation• Prescribes DoD IA Requirements via Minimum

Baseline IA Controls• Additional Controls Specified in Related Documentation


14

Background

• DoD Implementation– Controls-based C&A Since 2003 (DoDI 8500.2)– Security Technical Implementation Guides (STIGs)

predate DoDI 8500.2 – PMs Required to Perform Information Systems Security

Engineering (ISSE) (DoDD 8500.1, DoDI 8500.2, DoDI 8580.1)

• Ensure Security is Incorporated into Overall Design• Support Successful Certification of System Security

Design and Configuration


15

Background

• Continuing Perception Security is Not Adequately Addressed in System Design/Configuration– General increase in number and types of exploitable

vulnerabilities (Deloitte, 2006)– Excessive Technical Risk Often Identified During

Security/Certification Testing (Cline, 2007) – Need to Incorporate Security in Development Practices

(Mouratidis, 2007; Davis 2004)—Limited Practice, Limited Research (Siponen, 2007; Woon, 2007)

– Problems/Issues with Specification/Incorporation of Security Requirements (Mayer, 2005; Mead, 2005; Wilander, 2005)


16

Background

• Impact– Program Managers: Scope, Cost, & Schedule– Certifiers: Cost & Schedule– DAAs: Cost, Schedule, Acceptance of Excessive Residual

Risk– Users: Scope, Cost, Schedule, & Acceptance of Excessive

Residual Risk


17

Research Proposal

• Problem Statement• Opportunity for Research• Research Objective


18

Problem Statement

• Barriers to the successful implementation of ISSE in DoD acquisition organizations result in IS/IT with excessive residual security risk—as determined by the DAA based on the certifiers recommendation—that potentially impacts:– Project Scope (Requirements / Capabilities)– Project and/or System Cost– Project Schedule– Technical Security Risk


19

Opportunity for Research

• Information Assurance Technical Framework (IATF)– NSA Responsibility

– Provides:• Technical

guidance for protecting information and infrastructures

• Prescribes a Defense-in-Depth approach

• Defines a process for system development …


20


• Two Principal Disciplines in System Development* (Cline, 2007)– Program Management– Systems Engineering

• Intersection forms Technical Management

– aka Engineering Management

– aka Systems Engineering Management

*in addition to specialty engineering

ProgramManagement

SystemsEngineering

TechnicalManagement


21


• Two Principal Disciplines in System Security Risk Management (Cline, 2007)– Information Assurance– IT/IS Compliance

• Intersection forms Certification & Accreditation

IT/ISCompliance

InformationAssurance

Certification&

Accreditation


• Information Systems Security Engineering (ISSE) … Systems Development meets Security Risk Mgmt

22


ProgramManagement

SystemsEngineering

InformationAssurance

SecurityEngineering

IT/ISCompliance

Rules &Regulations

InformationSystemsSecurity

Engineering

TechnicalManagement

Certification &Accreditation

• Four Domains in the “ISSE Rose” (Cline, 2007)– Security Engineering– Technical Management– C&A– Rules & Regulations


23


• Information Systems Security Engineering (ISSE)– System development process described by the IATF– Tightly integrated into the systems engineering model

1. DiscoverInformationProtection

Needs2. DefineSystemSecurity

Requirements3. DesignSystemSecurity

Architecture4. Develop

DetailedSecurityDesign

User Involvement

6. AssessInformation Protection

Effectiveness

5. ImplementSystemSecurity



SE Activities ISSE Activities

Discover NeedsThe systems engineer helps

the customer understand and document the information management needs that support the business or mission. Statements about information needs may be captured in an information management model (IMM).

Discover Information Protection Needs

The ISSE helps the customer understand the information protection needs that support the mission or business. Statements about information protection needs may be captured in an Information Protection Policy (IPP).

24

Mission/Business Function

Information Management Functions




Define System RequirementsThe systems engineer allocates identified needs to systems. A

system context is developed to identify the system environment and to show the allocation of system functions to that environment. A preliminary system Concept of Operations (CONOPS) is written to describe operational aspects of the candidate system (or systems). Baseline requirements are established.

Define System Security RequirementsThe ISSE allocates information protection needs to systems. A

system security context, a preliminary system security CONOPS, and baseline security requirements are developed.

25




Design System ArchitectureThe systems engineer

performs functional analysis and allocation by analyzing candidate architectures, allocating requirements, and selecting mechanisms. The systems engineer identifies components or elements, allocates functions to those elements, and describes the relationships between the elements.

Design System Security Architecture

The ISSE works with the systems engineer in the areas of functional analysis and allocation by analyzing candidate architectures, allocating security services, and selecting security mechanisms. The ISSE identifies components or elements, allocates security functions to those elements, and describes the relationships between the elements.

26




Develop Detailed DesignThe systems engineer analyzes design constraints, analyzes

trade-offs, does detailed system design, and considers life-cycle support. The systems engineer traces all of the system requirements to the elements until all are addressed. The final detailed design results in component and interface specifications that provide sufficient information for acquisition when the system is implemented.

Develop Detailed Security DesignThe ISSE analyzes design constraints, analyzes trade-offs, does

detailed system and security design, and considers life-cycle support. The ISSE traces all of the system security requirements to the elements until all are addressed. The final detailed security design results in component and interface specifications that provide sufficient information for acquisition when the system is implemented.

27




Implement SystemThe systems engineer moves the

system from specifications to the tangible. The main activities are acquisition, integration, configuration, testing, documentation, and training. Components are tested and evaluated to ensure that they meet the specifications. After successful testing, the individual components—hardware, software, and firmware—are integrated, properly configured, and tested as a system.

Implement System SecurityThe ISSE participates in a

multidisciplinary examination of all system issues and provides inputs to C&A process activities, such as verification that the system as implemented protects against the threats identified in the original threat assessment; tracking of information protection assurance mechanisms related to system implementation and testing practices; and providing inputs to system life-cycle support plans, operational procedures, and maintenance training materials.

28




Assess EffectivenessThe results of each activity are evaluated to ensure that the

system will meet the users’ needs by performing the required functions to the required quality standard in the intended environment. The systems engineer examines how well the system meets the needs of the mission.

Assess Information Protection EffectivenessThe ISSE focuses on the effectiveness of the information

protection—whether the system can provide the confidentiality, integrity, availability, authentication and non-repudiation for the information it is processing that is required for mission success.

1. DiscoverInformationProtection

Needs

2. DefineSystemSecurity

Requirements

3. DesignSystemSecurity

Architecture4. Develop

DetailedSecurityDesign

User Involvement

6. AssessInformation Protection

Effectiveness

5. ImplementSystemSecurity

29



ISSE Activity Assess Information Protection Effectiveness Task

Discover Information Protection Needs • Present an overview of the process. • Summarize the information model• Describe threats to the mission or business through information attacks• Establish security services to counter those threats and identify their relative

importance to the customer.• Obtain customer agreement on the conclusions of this activity as a basis for

determining system security effectiveness.

Define System Security Requirements • Ensure that the selected solution set meets the mission or business security needs.

• Coordinate the system boundaries.• Present security context, security CONOPS, and system security

requirements to the customer and gain their concurrence.• Ensure that the projected security risks are acceptable to the customer.

Design System Security Architecture • Begin the formal risk analysis process to ensure that the selected security mechanisms provide the required security services and to explain to the customer how the security architecture meets the security requirements.

Develop Detailed Security Design • Review how well the selected security services and mechanisms counter the threats by performing an interdependency analysis to compare desired to effective security service strengths.

• Once completed, the risk assessment results, particularly any mitigation needs and residual risk, will be documented and shared with the customer to obtain their concurrence.

Implement System Security • The risk analysis will be conducted/updated.• Strategies will be developed for the mitigation of identified risks• Identify possible mission impacts and advise the customer and the

customer’s Certifiers and Accreditors.

30



• ISSE and C&A

31


32


• Little in the Formal Literature on ISSE Implementation– General Failure to Implement ISSE

• Davis (2004), Mouratidis (2007)– Literature Focused on Methods, Processes, Tools

• Application of Quality Construct to ISSE– Security is a Non-Functional or Quality Requirement– SSE-CMM is ISSE Maturity (Quality) Model

• Further Indicates Applicability of Quality Concepts• Indicates ISSE both Technical & Mgmt Innovation

• Allows Synthesis of Process & Factor Models / Frameworks (Adoption Literature)


33


• Construct Proposed by Sebastianelli & Tamimi (2004)– Barriers Based on Factor Analysis:

» Human Resources Development & Mgmt, Planning, Leadership, Resources, Customer Focus

– Difficult to Interpret from Survey Items• Construct modified by Cline (2007)

– Factors Based on a Synthesis of the Literature (Hill, 2006)» Training, Planning, Leadership, Resources, Culture

– Factors Formally Defined


34

Research Objective

• The research seeks to determine if the implementation of ISSE in support of acquired DoD IS/IT encounters the same impediments or barriers to implementation as traditional quality programs.


35

Research Review and Synthesis

• Approach to the Review• Empirical Research in Information Security and Assurance• Empirical Research in Relevant Disciplines


36

Approach to the Review

• Purpose of Methodical Review– Understand Research Topic and Justify the Research– Identify Research Problem and Gaps in the Literature– Provides Suitable Methodologies

• Relevance to Research Problem is Key Discriminator“Each literature piece should constantly be evaluated on how applicable it is to the proposed study. Thus, only the applicable literature articles that are relevant to build the theoretical foundations for the validity of the theories, constructs, and measures should be noted.” (Levy & Ellis, 2006, p. 188)

• ISSE is “an engineering process … that captures and refines information protection requirements [SRE] and ensures their integration into IT acquisition processes [implementation] through purposeful security design and configuration [development]”


37


• Primary Bodies of Knowledge

Quality

Management

Hypothesized relationship

Demonstrated relationship

SRE

Implementation

Development(Design &

Configuration)

Implementation

Capability Maturity

Factors

Non-Functional Requirements

ISSE Quality

Adoption Theory


38


• Based on IS/ISSE Research Models (Lowry et al., 2002; Bjorck, 2001)

Rigorous Relevant

Est

ab

lish

ed

Em

erg

en

t

ISSE

Quality

Quality Management

Development (Design & Configuration)

SRE

Ado

ptio

n T

heo

ryIS

SE

/ Q

ua

lity

Imp

lem

en

tatio

n


39


• InfoSec Research Model (Siponen & Oinas-Kukkonen, 2007)

– Levels of Abstraction

• Organizational, Conceptual, Technical

– Information Security Area / Topic / Contribution

• Access to IS

• Secure Communication

• Security Management

• Development


40


• Applicable Categories (Italicized Most Relevant to Research Problem)

– Access to IS / Conceptual (e.g., modeling of access control policy)

– Secure Communication / Conceptual (e.g., modeling in distributed systems)

– Security Management / Organizational (e.g., organizational ISSE implementation policies and guidelines)

– Development / Organizational (e.g., ISSE methods, processes)


41

Empirical Research in Information Security and Assurance

• ISSE Methodology• ISSE Implementation


42

ISSE Methodology

• Organizational-level Literature on Development of Secure IS/IT• Landwehr (1981)

– Assurance-based Security Design (circa 1972 - )• Development Using Formal Models (TCSEC, ITSEC, CC)

• Baskerville (1993)– Checklist-based Security Design (circa 1972 - )

• Design Based on Available (Known) Controls• Use of Risk in Cost-Benefit Analysis (Controls Selection)

– Mechanistic Engineering Security Design (circa 1981 - )• Design Based on Requirements• Continued Use of Risk in Cost-Benefit Analysis

– Formative Logical Security Design (circa 1988 - )• Design Based on Abstract Models (Functional/Organizational)• Limited or No Use of Risk in Cost-Benefit Analysis)


43

ISSE Methodology

• Weiss et al. (1996)– Discussed Integration of Engineering and Risk Management

• Straub & Welke (1998) – Discussed Systems Risk in Organizational Risk

Management– Security Design a Component of Systems Risk Assessment

• Chivers (2004)– Traditional Methods Ineffective for Distributed Systems– Integrated Risk/Engineering Security Design Methods (circa

1999 - )• Failure Mode Analysis, Fault Tree Analysis• Increased Complexity Degrades Abstraction


44

ISSE Methodology

• Chivers (2004) (Continued)– Need to Integrate Security Analysis in the Systems

Engineering Process– Most Relevant Work in Requirements Engineering (circa

1992 - )• Functional Requirements: Use Cases, Goals• Non-Functional Requirements: Patterns, Abuse Cases

& Misuse Cases, Anti-Goals• Need for Integration of Functional and Non-Functional

Requirements– Object-Oriented Patterns; Goal-Based Refinement– Most Promising Area is Goal-Based Refinement


45

ISSE Implementation

• Organizational-level Literature on Security Management (ISSE)• Marmor-Squires & Rougeau (1988) (cited in Ashby et al.,1989)

– Early Paper Addressing Security Engineering in Systems Engineering

– Did Not Address ISSE Implementation Issues• Froscher et al. (1990)

– Specific to C&A But Addressed Secure IS/IT Development– Recommended Certification Activity Early in Development– Recommended Specific Interactions: CA, DAA, PM, Developer– Did Not Address ISSE Implementation Issues

• Weiss et al. (1996)– Discussed Integration of Engineering & Security Risk Mgmt– Did Not Address ISSE Implementation Issues


46

ISSE Implementation

• Peter & Schleipfer (2004)– Recommended Addressing Security Engineering Early in

Development– Did Not Address ISSE Implementation Issues

• Davis (2004)– Advocated Security Engineering Throughout Development

Lifecycle– Did Not Address ISSE Implementation Issues


47

ISSE Implementation

• Mouratidis (2007)– Security Engineering and IS Engineering Communities

Work Separately• Little Integration of Security and IS Engineering

Principles/Practice– Research in Security Engineering Primarily Technical

Issues• Little Consideration of Social Aspects / Social Context

– Recommended New Discipline: Secure Information Systems Engineering (Note: No Significant Literature on DoD ISSE)

– Did Not Address ISSE Implementation Issues


48

Empirical Research in Relevant Disciplines

• Quality Management Models• Quality Implementation


49

Quality Management Models

• Seminal Authors (Hill, 2006)– Crosby– Deming– Feigenbaum– Ishikawa– Juran– Shewart– Taguchi

• Quality Management Models (Hill, 2006)– Quality Control– Quality Trilogy– Quality Assurance– Quality Management– TQM– Six-Sigma– CMM


50

Quality Implementation

Cline (2007)

Sebastianelli & Tamimi (2003)

Nwabueze (2001) 1

Deming (1987) 2

Amar & Zain (2002) 3

Juran (1993) 4

Williams et al. (2004) 5

Leonard & McAdam (2001) 6

Chin & Pun (2002) 7

Kwak & Anbari (2006) 8

Culture Lack of customer focus

No focus on customer satisfaction 5

No understanding of cultural change 1

Culture or relationships between departments 3

Resistance to change 6

No long-term management (mgmt) commitment due to rewards based on short-term successes 5

Inappropriate organizational culture 7, 8

Incompatibility of attitudes between management and workers 1

Attitudes towards quality 3

Employee ‘buy-in’ 4

Insufficient cooperation 7


51


Cline (2007)


Nwabueze (2001) 1

Deming (1987) 2


Juran (1993) 4



Chin & Pun (2002) 7


Leadership Inadequate human resources (HR) development and management

HR issues 3 No focus on employee satisfaction, innovation, or quality systems 5

Failure of management to recognize employee importance 7

Lack of leadership for quality

Lack of management commitment 1

Ineffective mgmt 1, 2

Lack of management commitment 3

Unstable management due to high turnover 3

Lack of long-term management commitment due to rewards based on short-term successes 5

Lack of management leadership 7

Lack of realism for what TQM can do 7

Planning Lack of quality planning

Poor planning 1

Lack of goals 1

Implementation without regard to context 1

Use of improper framework 1

Divide between strategic & operational goals/planning 6

Lack of strategic planning 8


52


Cline (2007)


Nwabueze (2001) 1

Deming (1987) 2


Juran (1993) 4



Chin & Pun (2002) 7


Resources Inadequate resources

Lack of resources 1 Insufficient raw materials 3

Lack of machines & equipment 3

Training Lack of training 1 Inadequate training 3 Inadequate training 7

Training in leadership & project mgmt 8


53

Preliminary Conclusions

• Limited Research on Implementation in the ISSE Literature (Mouratidis, 2007; Siponen, 2007)

• Applicability of quality concepts to ISSE (Cline, 2007)– Non-Functional Requirements (Chung, 1995)– Capability Maturity (SSE-CMM) (ISSEA, 2007)

• Factors Related to Quality Implementation (Hill, 2006)• Factors Related to the Implementation of ISSE by DoD

Acquisition Agencies are Similar to the Implementation of Traditional Quality Management Models (Cline, 2007)– Culture– Leadership– Planning– Resources– Training


54

Preliminary Recommendations

• In Addition to Mandating Use of ISSE in System Acquisition, the DoD Should Provide a Level-of-Effort Similar to the Implementation of Traditional Quality Programs

• Recommendations Specific to Each Barrier:– Culture—Stress the Importance of ISSE through Policy;

Provide Penalties for Non-Compliance and Enforce the Penalties

– Leadership—Ensure Leadership/Management Understand their Compliance Requirements and the Penalties for Non-Compliance


55


• Recommendations Specific to Each Barrier (cont):– Planning—

• Formally Incorporate ISSE in Systems Engineering Plans, Processes, Procedures, Standards and Guidance

• Fully Integrate ISSE in All Engineering & Program Reviews • Implement the SSE-CMM as part of CMMI in Acq Agencies

– Resources—• Mandate ISSEs for Acquisition Programs• Program/Budget Needed Tools (e.g., Mgmt and Assessment)

– Training• Require ISSE Awareness Training for Systems Engineers

and PMs• Require Properly Trained and Certified ISSEs


56


• Senior Management Responsible for Fielding Secure IS/IT (e.g., DAA, CA, PM, & User Rep) Should:– Receive Briefings on the Role of ISSE– Apply Lessons Learned From Prior Quality Model

Implementations to ISSE– Be Tracked to Determine

• Rate & Level of Adoption• Subsequent Impact on Residual Risk

– Technical Vulnerabilities Associated with System Development and Configuration


57

Recommended Reading

• DS Herrmann (2002). A practical guide to security engineering and information assurance. Boca Raton, FL: Auerbach Publications.


58

Questions

… that haven’t already been asked?

Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAPUniversity of Fairfax

[email protected] Children’s Hospital of Philadelphia

[email protected]


59

Additional Information


60

Justification for Research

• Answering the research questions will allow DoD acquisition agencies to:– Address the most critical issues facing program and

project managers – Provide targeted application of scarce resources (dollars

and personnel)– Improve overall system security– Reduce risk to project scope, cost and schedule


61

Theoretical Foundation

• Extensive Literature on ISSE Methodology• Little or No Relevant Literature on ISSE Implementation

– No Construct for Barriers to ISSE Implementation• Applicability of Quality Concepts to ISSE

– Nonfunctional Requirements (Engineering)– Capability Maturity Models (Management)

• Quality Construct – Barriers to Implementation– Sebastianelli and Tamimi (2003)

• Exploratory Quantitative Research– Correlational Survey Study

• Barriers Based on Interpretation of Optimal Factor Analysis– Cline (2007)

• Barriers Based on Synthesis of Relevant Literature


62

Research Methodology

• Theoretical Framework• Research Design Approach• Context of Study• Feasibility Analysis and Design Selection


63

Theoretical Framework

• Research Hypotheses• Operational Definition of Variables• Rival Hypotheses• Plausibility of Rival Hypotheses


64

Research Questions

• Question 1: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s culture?

• Question 2: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s leadership?

• Question 3: Is there a relationship between the successful implementation of ISSE and the acquisition organization’s planning?

• Question 4: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s resources?

• Question 5: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s training?


65

Research Hypotheses

H1

H10 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s culture.

H11 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s culture.

H2

H20 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s leadership.

H21 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s

leadership.

H3

H30 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s planning.


planning.

H4

H40 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s resources.


resources.

H5

H50 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s training.

H51 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s training.


66

Operational Definition of Variables

• Independent Variable: ISSE implementation is defined as the implementation of ISSE within a DoD IS/IT acquisition organization, where ISSE is defined as “an engineering process … that captures and refines information protection requirements and ensures their integration into IT acquisition processes through purposeful security design and configuration” (DoD, 2006, p. 19).

• Dependent Variable 1: Culture is defined as “a set of shared assumptions, values, and behaviors that characterize the functioning of an organization” (Schwalbe, 2006, p. Glossary 8).

• Dependent Variable 2: Leadership is defined as the actions of an individual, usually in a formal position of authority in an organization (management), “who focuses on long-term goals and big-picture objectives, while inspiring people to reach those goals” (Schwalbe, 2006, p. Glossary 6).


67

Operational Definition of Variables

• Dependent Variable 3: Planning is defined as the activities and processes used to devise and maintain a workable scheme to ensure organizational needs are met (adapted from Schwalbe, 2006, p. Glossary 8).

• Dependent Variable 4: Resources are defined as “skilled human resources (specific disciplines either individually or in crews or teams), equipment, services, supplies, commodities, materiel, budgets, or funds” (Program Management Institute, 2004, p. 372) but not the management of personnel as defined by leadership.

• Dependent Variable 5: Training is defined as “the level of learning required to adequately perform the responsibilities designated … and accomplish the mission” (DAU, 2005, p. B 170).


68

Rival Hypotheses

• Use of integrated commercial off-the-shelf (COTS) or non-developmental items (NDI)– Burbank & Kasch (2004); Chung & Cooper (2002); Steves

(1997)• Use of a rapid acquisition process


69

Plausibility of Rival Hypotheses

• Both type of development (NDI vice developmental) and type of acquisition (rapid vice traditional) will be accounted for and controlled as mediating variables.


70

Research Design Approach

• Post-Positivistic with Hermeneutic (Constructivist) Elements– Applied– Explanatory (Confirmatory?)– Quantitative– Correlational (Predictive?)– Survey


71

Context of Study

• Setting• Population• Limitations• Sample Design and Selection


72

Setting

• Product Group Twelve (PG-12), Marine Corps Systems Command– Multiple government and contractor facilities and

workspaces


73

Population

• Theoretical Study Population– All military, government civilian and civilian contractor

acquisition and engineering personnel supporting DoD IS/IT acquisition programs on behalf of a DoD acquisition agency

• Target Study Population– Restricted to Marine Corps Systems Command


74

Limitations

• Facility Access– Duty Days, Duty Hours– Formal Approval and/or Escort Required

• Personnel Access– Not-to-Interfere Basis

• Information– Restrictions on Use of Classified and SBU


75

Sample Design and Selection

• Random Selection Infeasible– Research Setting Restricted– Small Target Study Population

• Non-Random Alternative– Grounded Theory of Generalized Causal Inference

(Shadish et al., 2002) – Purposive Sample of a Typical Instance (Theoretical Study

Population)• Marine Corps Systems Command• Product Group Twelve


76

Sample Design and Selection

• Principles of Generalization (Framework for Validity Claims)– Surface Similarity

• Categorization of Phenomena Based on Similar Characteristics (Homogeneity)

– Ruling Out Irrelevancies• Ignoring Characteristics of Phenomena Not Relevant to

Categorization (Heterogeneity)– Making Discriminations

• Discarding Phenomena That Do Not Fit Categorization– Interpolation and Extrapolation

• Generalizing Between or Beyond Sampled Values– Causal Explanation

• Attributing Causation Based on Theorized Structural Similarity


77

Feasibility Analysis and Design Selection

• Assumptions• Limitations• Delimitations• Design Approaches• Resources


78

Assumptions

• Additional technical risk discovered during IV&V is due to a failure of one or more aspects of the ISSE implementation

• Theoretical framework provided by identified barriers to quality implementations is applicable to ISSE

• Target study sample is assumed extensible to the target and theoretical study populations

• Perceptions of surveyed personnel are a valid indicator of the variables measured


79

Limitations

• Threats to Validity– External Validity of the Purposive Sample

• Addressed Though Five Principles of Generalization– Construct Validity of Barriers to Implementation

• Addressed Through Analytical Comparison with Original Construct (Factor Definitions / Item Loadings)

– Internal Validity due to Construct Modifications• Addressed Through Grounded Interpretation of Factors (Factor

Definitions / Item Loadings) – Statistical Conclusion Validity

• Addressed Through Use of Generally Accepted Statistical Procedures and Statistical Software Package

• Threats to Predictive Analysis– Non-Experimental Design; Small Sample Size– Addressed Through Full Disclosure of the Issues


80

Delimitations

• Research Restricted to:– DoD IS/IT– DoD Acquisition Organizations


81

Design Approach

• Operational Constraints– Limited Invasiveness / Interference– Suggests Non-Experimental Design

• Site Constraints– Small Population– Limited Sample Size– Suggests Non-Random Selection– Suggests Purposive Sample

• Foundational Constraints– Partial Replication of Prior Study– Suggests Exploratory or Confirmatory Research– Requires Correlational Design– Requires Survey Study (Approach to Data Collection)


82

Resources

• Limited Requirements– General Computing and Printing Resources– Statistical Software Package– Web Hosting (Survey Instrument)– Direct (e.g., Books, Literature Sources)– Indirect (e.g., Communications, Travel)

• All Costs Born by the Author


83

Research Design Specification

• RDS Completed and Approved• Study has IRB Approval; Permission to Collect Data Received• Next Steps …

– Survey Instrument Completed / Hosted on SurveyMonkey– Data Collection to Start as Early as First Week in June

2008


84

Backup Slides


Clinger-Cohen Act

• Clinger-Cohen Act (CCA)– IA-related requirements:

• IT Registration– DoD IT Registry– DoN IT Registration Database

http://www.don-imit.navy.mil/cca/Registration/registration.asp

– USMC IT Registration Database ??http://www.marcorsyscom.usmc.mil/sites/ia/documents/IA%20Form.html

• IA Strategy– Stand-alone document

» Written at program inception» More comprehensive for higher levels of assurance

– Supplemental/supporting documents:» ISP (Policies, Standards, & Architecture)» SSAA (Certification & Accreditation)

85


National Policy, Regulations

• National Security Directive 42 (NSD-42)– Established the National Security

Telecommunications and Information Security Committee (NSTISSC)

• Chaired by the DoD• Provides security guidance for National Security

Systems (NSS)• Evaluates the status of security for NSS

Note: NSTISSC is now the Committee on National Security Systems, CNSS)

86


FISMA

• Federal Information Security Management Act (FISMA)– Federal Agencies must:

• Inventory systems (IT Database)• Identify & provide appropriate security

protections (IA Controls / C&A)• Institute an INFOSEC program

– NIST provides security standards & guidelines

• DoD provides own standards & guidance– OMB oversees FISMA (and CCA)

compliance• Authorizes annual program reviews• Non-compliance can result in severe delays

or elimination of funding!

87


FISMA

• FISMA-compliance– IA Controls

• DODI 8500.2– C&A

• DoDI 5200.40 & DoD 8510.1-M– INFOSEC Program

• DoDD 8500.1 & DoDI 8500.2– Reporting

• Handled by MCSC IA Division

88


OMB Circular A-130

• Office of Management and Budget (OMB) Circular A-130, Appendix III – Provides security guidance and

defines responsibilities– Defines the use of “adequate

security”• “Commensurate with risk and

magnitude of harm”• Implements risk management vice

risk avoidance


NSTISSP 11

• National Security Telecommunications and Information Systems Security Policy (NSTISSP) No. 11 – “National Policy Governing the Acquisition

of IA and IA-Enabled IT”• Replaced “Orange Book”

– Requires use of National Information Assurance Program (NIAP)-certified products

• http://niap.nist.gov/cc-scheme/vpl/vpl_type.html• http://niap.nist.gov/cc-scheme/in_evaluation.html

– Based on the Common Criteria (CC)– Products evaluated against:

• Protection Profile (PP)• Evaluation Assurance Level (EAL)

90


NSTISSI 1000

• National Security Telecommunications and Information Systems Security Instruction (NSTISSI) No. 1000– “National IA C&A Program (NIACAP)”– Standardizes C&A for Federal Agencies– Implements C&A using FISMA guidelines– Basis for revision of current DoD C&A process

91


Key Performance Parameters

• KPPs establish requirements (including security) specific to a particular system– Objective: Preferred capability– Threshold: Minimum capability

• Failure to meet a threshold may result in a reevaluation, re-assessment or termination of the program, or a modification of the production increments

• Example: Time to restore full functionality into a degraded or corrupted system– Objective: 5 minutes– Threshold: 15 minutes

92


DoD 5000.2-R SE Process

93


IEEE Std 1220-1998 SE Process

94


95

Relevant Terms

• Accreditation – Formal declaration by a DAA that an IS is approved to operate at an acceptable level of risk, based on the implementation of an approved set of technical, managerial, and procedural safeguards.

• Acquisition – The conceptualization, initiation, design, development, test, contracting, production, deployment, logistics support, modification, and disposal of systems to satisfy DoD needs intended for use in, or in support of, military missions.

• Authority to Operate (ATO) – The authorization granted by a DAA for a DoD IS to process, store, or transmit information.

• Certification – A comprehensive validation of actual IA capabilities and services of a DoD IS to establish compliance with assigned IA Controls.


96

Relevant Terms

• Certifier – Individual responsible for making a technical judgment of the system’s compliance with stated requirements.

• Certification Determination – A certifier’s validation of the systems compliance with IA controls, identifying and assessing the risks with operating the system, and the cost to correct or mitigate the IA security weakness.

• Designated Accrediting Authority (DAA) – Official with the authority to formally assume responsibility for operating a system at an acceptable level of risk.

• DoD Information System (IS) – Set of information resources organized for the collection, storage, processing, maintenance, use, sharing , dissemination, disposition, display, or transmission of information.


97

Relevant Terms

• Information Assurance Control – An objective IA condition of integrity, availability, or confidentiality achieved through the application of specific safeguards or through the regulation of specific activities.

• Independent Verification and Validation (IV&V) – An independent review … [of a system] performed by an organization that is technically, managerially, and financially independent of the development organization.

• Information Systems Security Engineering – An engineering process that captures and refines information protection requirements and ensures their integration into [IS] acquisition processes through purposeful security design and configuration.

• Residual Risk – Risk due to a partial or unsatisfactory implementation of assigned IA controls.


98

Relevant Terms

• Risk – Possibility that a particular threat will adversely impact an IS by exploiting a particular vulnerability.

• Technical Controls – Security controls (i.e., safeguards or countermeasures) for an IS that are primarily implemented and executed by the IS through mechanisms contained in the hardware, software, or firmware components supporting the system.

• Test & Evaluation – Process by which a system or components are exercised and results analyzed to provide performance-related information.

• Threat – Any circumstance or event with the potential to adversely impact an IS through unauthorized access, destruction, disclosure, modification of data, and/or denial of service.


99

Relevant Terms

• Validation – Activity applied throughout the system lifecycle, to confirm or establish by testing, evaluation, examination, investigation, or competent evidence that a DoD information system’s assigned IA Controls are implemented correctly and are effective in their application.

• Vulnerability – Weakness in an IS, system security procedures, internal controls, or implementation that could be exploited.

Copyright © 2008 by Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP. All rights reserved. 1...

Documents

Transcript of Copyright © 2008 by Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP. All rights reserved. 1...