COSATMO/COSYSMO Workshop
-
Upload
demetrius-martinez -
Category
Documents
-
view
58 -
download
2
description
Transcript of COSATMO/COSYSMO Workshop
University of Southern California
Center for Systems and Software Engineering
COSATMO/COSYSMO Workshop
Jim Alstad, USC-CSSE
Gan Wang, BAE Systems
Garry J Roedler, Lockheed Martin
April 30, 2014
04/29 1
University of Southern California
Center for Systems and Software Engineering
Workshop Agenda
04/29 2
• Reminder: general directions for COSYSMO 3.0 development
• Delphi for SEFR and SEWR category parameter values– Overview of SEFR and SEWR– Instructions for Delphi questionnaires
• Discussion of extending COSYSMO to estimate development costs
University of Southern California
Center for Systems and Software Engineering
COSATMO/COSYSMO 3.0 Direction• Several factors affecting the COSYSMO cost model
have been shown to be valuable in increasing estimation accuracy:– Reuse (simple model—SEWR)– Reuse (with SEFR)– Requirements volatility (SERV)The rating scales for these could be integrated into a comprehensive COSYSMO model.– Which should provide more accurate estimates than any of
these alone• Add additional data points exhibiting a range of
values for SEWR, SEFR, SERV• Fit a COSYSMO III model to the overall dataset
– Add variables and/or subset the data as needed
04/28 3
University of Southern California
Center for Systems and Software Engineering
COSATMO/COSYSMO Extension to Total Development Costs
• Explore a model for total development cost based primarily on the COSYSMO parameters (Roedler)– Can such a model be improved by dividing development
cost into three parts: system engineering, hardware engineering, software engineering? (Alstad)
04/28 4
University of Southern California
Center for Systems and Software Engineering
Workshop Agenda
04/29 5
• Reminder: general directions for COSYSMO 3.0 development
• Delphi for SEFR and SEWR category parameter values– Overview of SEFR and SEWR– Instructions for Delphi questionnaires
• Discussion of extending COSYSMO to estimate development costs
University of Southern California
Center for Systems and Software Engineering
What is this Delphi all about?• A Generalized Reuse Framework
implemented in COSYSMO– To assess not only the effort to leverage but
also the effort to invest in a development
project
• The goal is an industry-wide model for
design sensitivity analysis and product
line investment decisions– To bring back to organization to further
tailor or use
• Invite your participation to calibrate the
model coefficients, through a Delphi
• The Delphi process today…6
University of Southern California
Center for Systems and Software Engineering
System Engineering For Reuse (SEFR)
• Producer’s View• Production of
reusable resources
System Engineering With
Reuse (SEWR)• Consumer’s View• Consumption of
reusable resources
Two Fundamental Reuse Processes
7
University of Southern California
Center for Systems and Software Engineering
CONOPS – Reuse in a Project
% E
ffort
# of Articles in the Product Line
1 2 3 4
DFR
DWRDWR
DFR
DWR
DFR
DWR
DFR
100
0
Total Effort
Investments in Development for Reuse (DFR) are leveraged to reduce Product Line Cost
8
Total Project Effort = SEWR Effort + SEFR Effort
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:System Engineering for Reuse
• System Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states:– Conceptualized for Reuse (e.g., Concept of Operations
document)– Designed for Reuse (e.g., component detailed design)– Constructed for Reuse (e.g., integrated component)– Validated for Reuse (e.g., validated component)
04/29 9
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:System Engineering for Reuse
04/29 10
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:System Engineering with Reuse
• System Engineering with Reuse is project System Engineering, with reusable artifacts being brought into the product– A special case: zero reusable artifacts
• Each reusable artifact is included in one of these SEWR states of maturity:– New (i.e., not reused)– Re-implemented (through requirements & architecture)– Adapted (through detailed design)– Adopted (through implementation)– Managed (through system verification & validation)
04/29 11
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:System Engineering with Reuse
04/29 12
University of Southern California
Center for Systems and Software Engineering
COSYSMO – Context
• COSYSMO– COnstructive SYStems Engineering Cost
MOdel – Parametric Estimate of the Systems
Engineering Effort– Covers full systems engineering lifecycle– Originally developed by Dr. Ricardo Valerdi
and Dr. Barry Boehm at USC
• Inception of COSYSMO 1.0– Valerdi, R., The Constructive Systems Engineering Cost Model (COSYSMO),
PhD Dissertation, University of Southern California, May 2005.
• Introduced the Reuse Model Extension to COSYSMO 2.0– Wang, G., Valerdi, R., Ankrum, A., Millar, C., and Roedler, G., “COSYSMO
Reuse Extension,” Proceedings of the 18th INCOSE International
Symposium, June 2008.– Fortune, J. Estimating Systems Engineering Reuse with the Constructive
Systems Engineering Cost Model (COSYSMO 2.0). Ph.D. Dissertation.
University of Southern California. December 2009– Wang, G., Valerdi, R., Fortune, J., “Reuse in Systems Engineering,” IEEE
System Journal, v4, No.3, 2010.
• Marching to COSYSMO 3.0 (work in progress…)– Fortune, J. and Valerdi, R., “Considerations for Successful Reuse in Systems
Engineering,” AIAA Space 2008, San Diego, CA, September 2008.– Wang, G. and Rice, J., “Considerations for a Generalized Reuse Framework
for System Development,” Proceedings of the 21st INCOSE International
Symposium, June 2011.– Peña, M. Quantifying the Impact of Requirements Volatility on Systems
Engineering Effort. Ph.D. Dissertation. University of Southern California.
August 2012.– Fortune, J. and Valerdi, R., “A Framework for Systems Engineering Reuse,”
Systems Engineering, 16(2), 2013.
13
University of Southern California
Center for Systems and Software Engineering
COSYSMO Overview (1/2)• COSYSMO starts by computing the “size” of a
system engineering project, in units of eReq (“equivalent nominal requirements”)
• These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios.
• Each artifact is evaluated as being easy, nominal, or difficult.
• Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:
04/29 14
Artifact Type Easy Nominal Difficult
System Req’ts 0.5 1.0 5.0
System Interfaces 1.1 2.8 6.3
System Algs 2.2 4.1 11.5
Op Scenarios 6.2 14.4 30.0
University of Southern California
Center for Systems and Software Engineering
COSYSMO Overview (2/2)
• Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant.
• This results in the following equation for a COSYSMO estimate of effort in person-months:
04/29 15
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:SEFR Model Equations
• A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table (from the first Delphi):
04/29 16
SEFR State (Degree of Development) SEFR State Factor
Conceptualized for Reuse 36.98%
Designed for Reuse 58.02%
Constructed for Reuse 79.15%
Validated for Reuse 94.74%
University of Southern California
Center for Systems and Software Engineering
Generalized Reuse Framework:SEWR Model Equations
• A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table (from the first Delphi):
04/29 17
SEWR State (Maturity) SEWR State Factor
New 100.00%
Re-Implemented 66.73%
Adapted 56.27%
Adopted 38.80%
Managed 21.70%
University of Southern California
Center for Systems and Software Engineering
The Exercise – Mapping Activities to Reuse Categories
• Format of Questionnaire Spreadsheet: – 1st tab: definition of reuse
categories– 2nd tab: DWR questionnaire– 3rd tab: DFR questionnaire– 4th tab: definition of EIA-632
activities
• You do:– Fill out the cells intersecting EIA-
632 activities (rows) and the reuse categories (columns)
– Enter the percentage you believe that particular activity applies or is relevant to a particular reuse category
– Fill in a few words of your rationale for each selection
18
Reusable Artifacts Feeding into the DWR Process
==>
Syste
m C
once
pt
Syste
m D
efini
tion
Realiz
ed S
yste
m
Deplo
yed
Syste
m
Syste
m D
efini
tion
Realiz
ed S
yste
m
Deplo
yed
Syste
m
Realiz
ed S
yste
m
Deplo
yed
Syste
m
Realiz
ed S
yste
m
Deplo
yed
Syste
m
Realiz
ed S
yste
m
Deplo
yed
Syste
m
Deplo
yed
Syste
m
New Modified (Implemented) Adapted Deleted Adopted (Integrated)EIA/ANSI 632 Process EIA/ANSI 632 Task Rationale Rationale Rationale Rationale Rationale Managed Rationale
1. Product Supply 100% 100% 100% 100%
2. Product Acquisition 100% 100% 100% 100%
3. Supplier Performance 100% 100% 100% 100%
4. Process Implementation Strategy 100% 100% 100% 100%
5. Technical Effort Definition 100% 100% 100% 100%
6. Schedule and Organization 100% 100% 100% 100%
7. Technical Plans 100% 100% 100% 100%
8. Work Directives 100% 100% 100% 100%
9. Progress Against Plans and Schedules 100% 100% 100% 100%
10. Progress Against Requirements 100% 100% 100% 100%
11. Technical Reviews 100% 100% 100% 100%
12. Outcomes Management 100% 100% 100% 100%
13. Information Dissemination 100% 100% 100% 100%
14. Acquirer Requirements 100% 100% 100% 100%
15. Other Stakeholder Requirements 100% 100% 100% 100%
16. System Technical Requirements 100% 100% 100% 100%
17. Logical Solution Representations 100% 100% 100% 100%
18. Physical Solution Representations 100% 100% 100% 100%
19. Specified Requirements 100% 100% 100% 100%
20. Implementation 100% 100% 100% 100%
21. Transition to Use 100% 100% 100% 100%
22. Effectiveness Analysis 100% 100% 100% 100%
23. Tradeoff Analysis 100% 100% 100% 100%
24. Risk Analysis 100% 100% 100% 100%
25. Requirements Statements Validation 100% 100% 100% 100%
26. Acquirer Requirements Validation 100% 100% 100% 100%
27. Other Stakeholder Requirements Validation 100% 100% 100% 100%
28. System Technical Requirements Validation 100% 100% 100% 100%
29. Logical Solution Representations Validation 100% 100% 100% 100%
30. Design Solution Verification 100% 100% 100% 100%
31. End Product Verification 100% 100% 100% 100%
32. Enabling Product Readiness 100% 100% 100% 100%
33. End Products Validation 100% 100% 100% 100%
Technical Evaluation
New
Acquisition and Supply
Technical Management
System Design
Modified (Implemented) Adapted Deleted Adopted (Integrated)
Product Realization
X
University of Southern California
Center for Systems and Software Engineering
Helpful Hints• When evaluate the questionnaire, frame your thinking around the
COSYSMO construct, i.e., a group of– Requirements
– Interfaces
– Algorithms
– Scenarios
• Based on your overall professional experience, not a
single or a particular program experience– Seek nominal behavior, biased toward desirable/more
successful reuse situations• The handouts have the definitions of reuse categories and EIA-632
activities
• For definition of general (engineering) terms, use INCOSE SE Handbook
19
University of Southern California
Center for Systems and Software Engineering
Questions and Comments on Delphi
Contacts:
Jim AlstadUSC CSSE
Los Angeles, [email protected]
Gan WangBAE Systems
Reston, [email protected]
Garry J. RoedlerLockheed MartinPhiladelphia, PA
20
University of Southern California
Center for Systems and Software Engineering
Workshop Agenda
04/29 21
• Reminder: general directions for COSYSMO 3.0 development
• Delphi for SEFR and SEWR category parameter values– Overview of SEFR and SEWR– Instructions for Delphi questionnaires
• Discussion of extending COSYSMO to estimate development costs
University of Southern California
Center for Systems and Software Engineering
Extending COSYSMO To Estimate Development Cost
• The general goal: Use a model that is very similar to COSYSMO to estimate development cost
• There is some evidence that system engineering cost has some correlation with development cost
• Where to go from here?– Especially, who has data from actual projects that they are
willing to contribute?
04/29 22