1

Derivation and Application of a Cost Model for Systems Engineering Cost Estimation(Derivación y Aplicación de un Modelo de Estimación de Costos para la Ingeniería de Sistemas)

Prof. Ricardo ValerdiSystems & Industrial EngineeringUniversity of Arizona 9 Enero 2017

Academia de IngenieríaPalacio de Minería

México DF

2

Principle #1

A solution that is too expensive, is not a solution.

Principle #2

Cost of products are a function of complexity and size.

Project Management Triangle

3

Performance

CostSchedule

4

5Nota: Producto Bruto Interno en Mexico para proyectos militares = 0.7%http://data.worldbank.org/indicator/MS.MIL.XPND.GD.ZS

6

7

8

How is Systems Engineering Defined?

• Acquisition and Supply – Supply Process– Acquisition Process

• Technical Management– Planning Process– Assessment Process– Control Process

• System Design– Requirements Definition Process– Solution Definition Process

• Product Realization– Implementation Process– Transition to Use Process

• Technical Evaluation– Systems Analysis Process

– Requirements Validation Process– System Verification Process– End Products Validation Process

EIA/ANSI 632, Processes for Engineering a System, 1999.

9

COSYSMO Data Sources (2002-present)Boeing Integrated Defense Systems (Seal Beach, CA)

Raytheon Intelligence & Information Systems (Garland, TX)Missile Systems (Tucson, AZ)

Northrop Grumman Mission Systems (Redondo Beach, CA)

Lockheed Martin Transportation & Security Solutions (Rockville, MD)Integrated Systems & Solutions (Valley Forge, PA)Systems Integration (Owego, NY)Aeronautics (Marietta, GA)Maritime Systems & Sensors (Manassas, VA; Baltimore, MD; Syracuse, NY)

General Dynamics Maritime Digital Systems/AIS (Pittsfield, MA)Surveillance & Reconnaissance Systems/AIS (Bloomington, MN)

BAE Systems National Security Solutions/ISS (San Diego, CA)

Information & Electronic Warfare Systems (Nashua, NH)

SAIC Army Transformation (Orlando, FL)

Integrated Data Solutions & Analysis (McLean, VA)

L-3 Communications Greenville, TX

11

COSYSMO Scope• Addresses first four phases of the

system engineering lifecycle (per ISO/IEC 15288)

• Considers standard Systems Engineering Work Breakdown Structure tasks (per EIA/ANSI 632)

Conceptualize DevelopOper Test & Eval

Transition to Operation

Operate, Maintain, or Enhance

Replace orDismantle

EIA/ANSI 632, Processes for Engineering a System, 1999.

ISO/IEC 15288, System Life Cycle Processes, 2008

12

COSYSMO

SizeDrivers

EffortMultipliers

Effort

Calibration

# Requirements# Interfaces# Scenarios# Algorithms

+3 Adj. Factors

- Application factors-8 factors

- Team factors-6 factors

COSYSMO Operational Concept

Valerdi (2005)

13

Software Cost Estimating Relationship

cSaMM e ⋅⋅=

cKDSIMM ⋅⋅= 05.1)(4.2

Boehm, B. W., Software Engineering Economics, Prentice Hall, 1981.

MM = Man monthsa = calibration constantS = size driverE = scale factorc = cost driver(s)KDSI = thousands of delivered source instructions

14

COSYSMO Model Form

∏∑=

⋅

Φ+Φ+Φ⋅=

14

1,,,,,, )(

jj

E

kkdkdknknkekeNS EMwwwAPM

Where: PMNS = effort in Person Months (Nominal Schedule)

A = calibration constant derived from historical project data k = {REQ, IF, ALG, SCN}wx = weight for “easy”, “nominal”, or “difficult” size driver

= quantity of “k” size driverE = represents diseconomies of scaleEM = effort multiplier for the jth cost driver. The geometric product results in an

overall effort adjustment factor to the nominal effort.

xΦ

Valerdi (2005, 2008)

15

UNDERSTANDING FACTORS– Requirements understanding – Architecture understanding– Stakeholder team cohesion – Personnel experience/continuity

COMPLEXITY FACTORS– Level of service requirements– Technology Risk– # of Recursive Levels in the Design– Documentation Match to Life Cycle Needs

OPERATIONS FACTORS– # and Diversity of Installations/Platforms– Migration complexity

PEOPLE FACTORS– Personnel/team capability – Process capability

ENVIRONMENT FACTORS– Multisite coordination – Tool support

Cost Driver Clusters

Valerdi (2005)

16

Stakeholder team cohesion Represents a multi-attribute parameter which includes leadership, shared vision, diversity of stakeholders, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in stakeholder community of the end users, customers, implementers, and development team.

1.5 1.22 1.00 0.81 0.65

Viewpoint Very Low Low Nominal High Very High

Culture Stakeholders with diverse expertise, task nature, language, culture, infrastructure Highly heterogeneous stakeholder communities

Heterogeneous stakeholder communitySome similarities in language and culture

Shared project culture

Strong team cohesion and project cultureMultiple similarities in language and expertise

Virtually homogeneous stakeholder communitiesInstitutionalized project culture

Compatibility Highly conflicting organizational objectives

Converging organizational objectives

Compatible organizational objectives

Clear roles & responsibilities

Strong mutual advantage to collaboration

Familiarity and trust

Lack of trust Willing to collaborate, little experience

Some familiarity and trust

Extensive successful collaboration

Very high level of familiarity and trust

Valerdi (2005)

Technology RiskThe maturity, readiness, and obsolescence of the technology being implemented. Immature or obsolescent technology will require more Systems Engineering effort.

Viewpoint Very Low Low Nominal High Very High

Lack of Maturity

Technology proven and widely used throughout industry

Proven through actual use and ready for widespread adoption

Proven on pilot projects and ready to roll-out for production jobs

Ready for pilot use Still in the laboratory

Lack of Readiness

Mission proven (TRL 9)

Concept qualified (TRL 8)

Concept has been demonstrated (TRL 7)

Proof of concept validated (TRL 5 & 6)

Concept defined (TRL 3 & 4)

Obsolescence

- Technology is the state-of-the-practice- Emerging technology could compete in future

- Technology is stale- New and better technology is on the horizon in the near-term

- Technology is outdated and use should be avoided in new systems- Spare parts supply is scarce

17

Valerdi (2005)

Migration complexity This cost driver rates the extent to which the legacy system affects the migration complexity, if any. Legacy system components, databases, workflows, environments, etc., may affect the new system implementation due to new technology introductions, planned upgrades, increased performance, business process reengineering, etc.

Viewpoint Nominal High Very High Extra High

Legacy contractor

Self; legacy system is well documented. Original team largely available

Self; original development team not available; most documentation available

Different contractor; limited documentation

Original contractor out of business; no documentation available

Effect of legacy system on new system

Everything is new; legacy system is completely replaced or non-existent

Migration is restricted to integration only

Migration is related to integration and development

Migration is related to integration, development, architecture and design

18

Valerdi (2005)

19

Cost Drivers Ordered by Effort Multiplier Ratio (EMR)

Valerdi (2005)

Benefits of Local Calibration

Before local calibration

After local calibration

Systems Engineering Effort (SE Hours)

System Size (eReq)

Systems Engineering Effort (SE Hours)

System Size (eReq)

20

Wang, G., Valerdi, R. and Fortune, J., “Reuse in Systems Engineering,” IEEE Systems Journal, 4(3), 376-384, 2010.

21

Prediction AccuracyPRED(30)

PRED(25)

PRED(20)

PRED(30) = 100% PRED(25) = 57%

Valerdi (2005)

Academic prototype

Commercial Implementations

Proprietary Implementations

COSYSMO-R

SECOST

SEEMaP

ImpactAcademic Curricula

Intelligence CommunitySheppard Mullin, LLC

Policy & Contracts

Model

∏∑=

⋅

Φ+Φ+Φ⋅=

14

1,,,,,, )(

jj

E

kkdkdknknkekeNS EMwwwAPM

COSYSMO

23

20June 2004

Example:Company “Lockheed Grumman” is building a system that has:

COSYSMO

SizeDrivers

EffortMultipliers

36 Person Months of systemsengineeringeffort

Calibration

100 easy, 50 nominal, 75 difficult requirements2 easy, 3 difficult interfaces4 easy algorithms5 nominal operational scenarios

High requirements undHigh tech riskHigh process capability

192

24

ContactRicardo Valerdi

rvalerdi@arizona.edu

http://rvalerdi.faculty.arizona.edu/