Post on 05-Jan-2016
description
University of Southern California
Center for Systems and Software Engineering
COSATMO, COCOMO III, and COSYSMO:
Developing Next-GenerationCost Models
Jim Alstad, PhD Candidate
USC Center for Systems and Software Engineering
CS 510 Lecture
September 17, 2014
09/16 1
University of Southern California
Center for Systems and Software Engineering
Next-Gen Cost Models
09/16 2
• Learning objectives for CS 510 lecture:– To provide a quick view of how an estimating model is
developed– To describe some of the current research in cost model
development
University of Southern California
Center for Systems and Software Engineering
Next-Gen Cost Models Agenda
09/16 3
Agenda:•Learning goals for 510 lecture•A brief history of the COCOMO family of estimating models•COCOMO III•COSYSMO 3.0•COSATMO
University of Southern California
Center for Systems and Software Engineering
COCOMO II and Some Derivatives
• 1995: one-size-fits-all model for 21st century software
• 1999: poor fit for schedule-optimized projects; CORADMO
• 2000: poor fit for COTS-intensive projects: COCOTS
• 2003: need model for product line investment: COPLIMO
• 2003: poor fit for agile projects: Agile COCOMO II (partial)
• 2012: poor fit for incremental development: COINCOMO
09/16 4
University of Southern California
Center for Systems and Software Engineering
COQUALMO1998
COCOMO 811981
COPROMO1998
COSoSIMO2007
Legend:Model has been calibrated with historical project data and expert (Delphi) data
Model is derived from COCOMO IIModel has been calibrated with expert (Delphi) data
COCOTS2000
COSYSMO2005
CORADMO1999,2012
iDAVE2004
COPLIMO2003
COPSEMO1998
COCOMO II2000
DBA COCOMO2004
COINCOMO2004,2012
COSECMO 2004
Software Cost Models
Software Extensions
Other IndependentEstimation Models
Dates indicate the time that the first paper was published for the model
COTIPMO2011
AGILE C II2003
COCOMO Family of Cost Models
09/16 5
University of Southern California
Center for Systems and Software Engineering
Determine Model Needs
Step 1
USC-CSSE Modeling Methodology
Analyze existing literature
Step 2Perform Behavioral analyses
Step 3Define relative significance,data, ratingsStep 4
Perform expert-judgment Delphi assessment, formulate a priori modelStep 5
Gather project data
Step 6
Determine Bayesian A-Posteriori modelStep 7 Gather more data;
refine model
Step 8
- concurrency and feedback implied
09/16 6
University of Southern California
Center for Systems and Software Engineering
Next-Gen Cost Models Agenda
09/16 7
Agenda:•Learning goals for 510 lecture•A brief history of the COCOMO family of estimating models•COCOMO III•COSYSMO 3.0•COSATMO
Key slides provided by Barry Boehm, Vu Nguyen, and Brad Clark (CSSE 2014 Annual Research Review)
University of Southern California
Center for Systems and Software Engineering
COCOMO II Data by 5-Year Periods
09/16 8
COCOMO II Calibration
Add in new data when calibrating
University of Southern California
Center for Systems and Software Engineering
COCOMO II Data: Productivity Trends
09/16 9COCOMO II Calibration
Calibrate to new productivity values
University of Southern California
Center for Systems and Software Engineering
Application Experience (APEX)Rating Trends
09/16 10
COCOMO II Calibration
Very High is>= 6 years
Reduce VH (etc) definition to cover same % of projects
University of Southern California
Center for Systems and Software Engineering
Super-Domains and AFCAA Productivity Types
09/1611
Super Domain Productivity Types
Real-Time
(RT)
1 Sensor Control and Signal Processing
2 Vehicle Control
3 Vehicle Payload
4 Real Time Embedded-Other
Engineering(ENG)
5 Mission Processing
6 Executive
7 Automation and Process Control
8 Scientific Systems
9 Telecommunications
Mission Support
(MS)
10 Planning Systems
11 Training
12 Software Tools
13 Test Software
Automated Information System
(AIS)
14 Intelligence and Information Systems
Software Services
Software Applications
University of Southern California
Center for Systems and Software Engineering
More on Application Domains• COCOMO III plans to identify a set of application
domains, and identify and calibrate a submodel to each domain
• Allows user to identify domain, and get a group of accordingly-set cost driver values off the shelf
09/16 12
University of Southern California
Center for Systems and Software Engineering
COCOMO III Directions• CSSE’s April Annual Research Review discussions
set these directions for COCOMO development:– 149 new data points have been added since the 161 data
points used in COCOMO II.2000.• Add these data points
– The average level of productivity appears to have increased• Therefore recalibrate the model’s parameters
– The average level of several cost drivers appears to have changed
• Consider changing the definitions of the high/medium/low levels for these cost drivers
– More data points might add to variability in the data• Consider adding more cost drivers to improve the fit
– Simplifying the model would make it easier to use• Define (calibrated) submodels for certain important application
domains that provide off-the-shelf values for some cost drivers• Planned domains: Real-time, Engineering & Scientific,
Command and Control, Automated Information Processing09/16 13
University of Southern California
Center for Systems and Software Engineering
Next-Gen Cost Models Agenda
09/16 14
Agenda:•Learning goals for 510 lecture•A brief history of the COCOMO family of estimating models•COCOMO III•COSYSMO 3.0•COSATMO
University of Southern California
Center for Systems and Software Engineering
What Is Systems Engineering?
09/16 15
• Some clauses of a definition:– Considers the system as a whole– Starts from stakeholder needs, results in a technical
approach meeting those needs• Frequently develops system requirements from system needs• Technical approach is commonly stated as the system
architecture– Systems engineers typically need to have familiarity with all
the specific kinds of engineering involved in their type of system
– Systems engineers can usually be found working on systems that are large, critical, and/or important
– Participation in identifying all major risks and reducing/managing them (ICSM)
– Typically most system engineering is done in the Valuation phase, with major work also being done in the Exploration and Foundation phases (ICSM)
– See also ICSM book section 0.2.1• The cost of system engineering needs an estimation
model
University of Southern California
Center for Systems and Software Engineering
History of COSYSMO Models
09/16 16
COSYSMO 1.0Valerdi, 2005
• Identifies form of model• Identifies basic cost drivers• Identifies Size measure
Req’ts VolatilePena, 2012
• Adds scale factor based on requirements volatility
With ReuseFortune, 2009
• Adds weights to Size elements, reducing net Size in the presence of reuse
For ReuseWang et al, 2014
• Adds weights to Size elements, reducing net Size when artifacts are only partially completed
Sys of SysLane et al, 2014
• Adds effort multiplier when in the presence of system-of-systems
COSYSMO 3.0Alstad, 2015?
• Combines features of previous models
University of Southern California
Center for Systems and Software Engineering
Basic COSYSMO (1/2)• COSYSMO [2] starts by computing the “size” of a
system engineering project, in units of eReq (“equivalent nominal requirements”)
• These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios.
• Each artifact is evaluated as being easy, nominal, or difficult.
• Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:
09/16 17
Artifact Type Easy Nominal Difficult
System Req’ts 0.5 1.0 5.0
System Interfaces 1.1 2.8 6.3
System Algs 2.2 4.1 11.5
Op Scenarios 6.2 14.4 30.0
University of Southern California
Center for Systems and Software Engineering
Basic COSYSMO (2/2)
• Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant.
• This results in the following equation for a COSYSMO estimate of effort in person-months:
09/16 18
University of Southern California
Center for Systems and Software Engineering
What Is Systems Engineeringfor Reuse?
• Systems Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states:– Conceptualized for Reuse (e.g., Concept of Operations
document)– Designed for Reuse (e.g., component detailed design)– Constructed for Reuse (e.g., integrated component)– Validated for Reuse (e.g., validated component)
09/16 19
University of Southern California
Center for Systems and Software Engineering
COSYSMO For Reuse• A SEFR estimate adjusts each artifact’s size
contribution by considering its SEFR state according to this table:
09/16 20
SEFR State (Degree of Development) SEFR State Factor
Conceptualized for Reuse 36.98%
Designed for Reuse 58.02%
Constructed for Reuse 79.15%
Validated for Reuse 94.74%
University of Southern California
Center for Systems and Software Engineering
What Is Systems Engineeringwith Reuse?
• Systems Engineering with Reuse is project development, with reusable artifacts being brought into the product– A special case: zero reusable artifacts
• Each reusable artifact is included in one of these SEWR states of maturity:– New (i.e., not reused)– Re-implemented (through requirements & architecture)– Adapted (through detailed design)– Adopted (through implementation)– Managed (through system verification & validation)
09/16 21
University of Southern California
Center for Systems and Software Engineering
COSYSMO with Reuse• A SEWR estimate adjusts each artifact’s size
contribution by considering its SEWR state according to this table:
09/16 22
SEWR State (Maturity) SEWR State Factor
New 100.00%
Re-Implemented 66.73%
Adapted 56.27%
Adopted 38.80%
Managed 21.70%
University of Southern California
Center for Systems and Software Engineering
Next-Gen Cost Models Agenda
09/16 23
Agenda:•Learning goals for 510 lecture•A brief history of the COCOMO family of estimating models•COCOMO III•COSYSMO 3.0•COSATMO
University of Southern California
Center for Systems and Software Engineering
The Problem
09/16 24
• How much will the total system cost?
• Is one phase being optimized while increasing total cost?
• Is the system affordable?
• Does the acquisition comply with the Better Buying Power intiatives (DoD)?
University of Southern California
Center for Systems and Software Engineering
The Solution
09/16 25
Example acquisition process (DoDI 5000.02)
COSATMO assists acquirers and developers during these phases (highest payoff during early phases)
COSATMO estimates the cost for these phases
University of Southern California
Center for Systems and Software Engineering
COSATMO Objective
09/16 26
• Context:– Current and future trends create challenges for full-system
cost estimation• Emergent requirements, rapid change, net-centric systems of
systems, COTS, clouds, apps, widgets, high assurance with agility, multi-mission systems
– Current development practices can minimize cost of one phase, such as development, while raising full-system cost
• The COSATMO project is developing a modern full-system cost model (first space systems, then other DoD domains)– “Constructive SATellite cost MOdel”– Current estimating models focus on one aspect, such as
system engineering– COSATMO will enable:
• System-level trades to be handled within a single model• Easy customer evaluation of full-system cost• Modern technologies to be covered
University of Southern California
Center for Systems and Software Engineering
Segments of Satellite System Cost
• Total satellite system cost [tied to slide 25 phases] = System engineering cost [EMD]+Satellite software cost [EMD]+Satellite vehicle hardware development [EMD] and production [Prod] cost+Launch cost [Deploy]+Initial ground software cost [EMD]+Initial ground custom equipment cost [EMD]+Initial ground facility (buildings, communications, computers, COTS software) cost [EMD]+Operation & support cost [Deploy, O&S]
• Updated at GSAW (Feb 2014)• Model as sum of submodels is new structure in
COCOMO family09/16 27
University of Southern California
Center for Systems and Software Engineering
COSATMO Segment Tentative Models
• System engineering: COSYSMO, perhaps with add-ons• Satellite vehicle hardware development and production: Current
Aerospace hardware cost model(s); exploring extensions of COSYSMO for hardware cost estimation
• Satellite vehicle, ground system software development: COCOMO II, COCOTS, perhaps with add-ons
• Launch model: similarity model, based on vehicle mass, size, orbit
• Ground system equipment, supplies: construction, unit-cost, services cost models
• Operation & support: labor-grade-based cost models, software maintenance models
09/16 28
University of Southern California
Center for Systems and Software Engineering
Key Overall Satellite SystemCost Drivers
• Most Important:– Complexity, Architecture Understanding, Mass, Payload TRL
level/Technology Risk, and Requirements Understanding.
• Important:– Reliability, Pointing Accuracy, Number of Deployables, Number of
Key Sponsors, Data Rate, and Security Requirements for Communications.
• Determined at COCOMO Forum (Oct 2013)
09/16 29
University of Southern California
Center for Systems and Software Engineering
Ground System Segment Development (1/2)
• Determined at GSAW (Feb 2014)• Ground system-wide cost drivers
– Most important: Accreditation (information assurance, etc), Required security
– Also important: # satellites*
• Initial software cost drivers– Required data throughput– Generally handled by COCOMO II, COCOTS, COPLIMO
09/16 30
*Indicates a size measure
University of Southern California
Center for Systems and Software Engineering
Ground System Segment Development (2/2)
• Ground custom equipment cost drivers– Most important: Amount of new development required, # of
custom equipment sites*, Required site availability & reliability, Required site security
– Also important: # driving requirements*
• Ground facility cost drivers– Most important: # facilities*, location of facilities (especially
US vs foreign), # ground RF terminals*– Also important: Facility “reuse”
• Operation and support cost drivers– Most important: # years of operation*, # FTE staff (with
labor mix)*– Also important: Size of software maintained*, Leased line
cost*, level of automation
09/16 31*Indicates a size measure
University of Southern California
Center for Systems and Software Engineering
COSATMO as a Research Umbrella• General direction:
– Develop a full-coverage satellite system cost estimating model
– Generalize that to additional applications
09/16 32
Specific current research initiatives:
– COSYSMO 3.0
– COCOMO III
Research vehicles:
– My thesis
– Other theses
– Other research
University of Southern California
Center for Systems and Software Engineering
Bibliography1. “A Generalized Systems Engineering Reuse Framework and its Cost
Estimating Relationship”, Gan Wang, Garry J Roedler, Mauricio Pena, and Ricardo Valerdi, submitted for publication.
2. “The Constructive Systems Engineering Cost Model (COSYSMO)”, Ricardo Valerdi (PhD Dissertation), 2005.
3. “Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0)”, Jared Fortune (PhD Dissertation), 2009.
4. “Quantifying the Impact of Requirements Volatility on Systems Engineering Effort”, Mauricio Pena (PhD Dissertation), 2012.
5. “Life Cycle Cost Modeling and Risk Assessment for 21st Century Enterprises”, Barry Boehm, Jo Ann Lane, Supannika Koolmanojwong, Richard Turner (presentation), April 29, 2014.
6. "System Interoperability Influence on System of Systems Engineering Effort", Jo Ann Lane, Ricardo Valerdi, unpublished.
7. “COSYSMO Extension as a Proxy Systems Cost Estimation” (presentation), Reggie Cole, Garry Roedler, October 23, 2013.
09/16 33