1 Results of Reuse Survey Jared Fortune, USC Ricardo Valerdi, MIT Gan Wang, BAE COSYSMO Workshop @...
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of 1 Results of Reuse Survey Jared Fortune, USC Ricardo Valerdi, MIT Gan Wang, BAE COSYSMO Workshop @...
1
University of Southern California Center for Systems and Software Engineering
Results of Reuse Survey
Jared Fortune, USC
Ricardo Valerdi, MIT
Gan Wang, BAE
COSYSMO Workshop @ COCOMO Forum 2008
Los Angeles, CA
2
University of Southern California Center for Systems and Software Engineering
Outline
• Research Background
• State of the Practice Survey
• Results
• Implications for COSYSMO 2.0
3
University of Southern California Center for Systems and Software Engineering
COSYSMO Reuse Development Timeline
COSYSMO 2.02005 2006 2007 2008
COSYSMO 1.0 Published [1]
Reuse Identified as a Critical Improvement [2]
Extensions for COSYSMO to Represent Reuse Published [3]
Preliminary Categories of Reuse Identified [4]
COSYSMO-R Developed At Lockheed Martin [5]
Reuse Definitions Identified [6]
BAE Pilot Test Shows Promising Results [8]
Other Reuse Considerations Identified [10]
Survey Results on State of the Practice [11]
Reuse Continuum Identified;“Bottoms-Up”Approach Proposed [7]
“Bottoms-Up”Results from BAE Presented [9]
2009
[1] Valerdi. COSYSMO. Ph.D. Dissertation,2005.[2] Valerdi. COSYSMO Workshop. USC ARR, 2006.[3] Valerdi, Gaffney, Roedler, Rieff. COSYSMO Extensions. COCOMO Forum, 2006.[4] Valerdi. COSYSMO Working Group. PSM Workshop 2006. [5] Gaffney. COSYSMO-R, 2007.
[6] Valerdi. COSYSMO Working Group. PSM Workshop, 2007.[7] Valerdi, Wang, Roedler, Rieff, Fortune. COSYSMO Reuse Extension. COCOMO Forum, 2007.[8] Wang. COSYSMO Reuse. COCOMO Forum, 2007.[9] Wang, Valerdi, Fortune. COSYSMO Reuse Extension. IEEE, 2008.[10] Fortune, Valerdi. Reuse Considerations. AIAA Space, 2008.[11] Fortune, Valerdi, Wang. State of the Practice. COCOMO Forum, 2008
4
University of Southern California Center for Systems and Software Engineering
COSYSMO 2.0 Development
• Literature review helped formulate survey questions• Survey results guided proposed COSYSMO 2.0
revisions
Reuse Considerations
Literature Review
Reuse Observations
Industry Survey
Revised Drivers
COSYSMO 2.0
5
University of Southern California Center for Systems and Software Engineering
State of the Practice SurveyHow does industry handle reuse?
COSYSMO 2.0 Reuse Survey
Reuse Survey Responders
BAE Systems
General Dynamics
Lockheed MartinOrbital Sciences
RaytheonReynolds, Smith, and Hills
Eight responses, representing eleven subject matter experts
6
University of Southern California Center for Systems and Software Engineering
Survey Results (1)
…it varies
Selected responses• “No formal definitions”• “Use of all or part of systems engineering work products”• “Use of design, pattern, template, handbook, or other engineering
effort that shifts the way engineering is done”• “Use of assets developed or acquired in response to requirements
for one application, in whole or in part to satisfy requirements for another application”
How does your organization define reuse?
Use of existing systems engineering products in a new application
7
University of Southern California Center for Systems and Software Engineering
Survey Results (2)
Frequency of Reuse of Systems Engineering Artifacts
Requirements
Other
Architecture/Design Models
Test Data/Procedures
COTS Products
Documentation/Templates
Never Occasionally Always
General
Specific
What are the systems engineering artifacts your organization reuses and how frequently are they reused?
8
University of Southern California Center for Systems and Software Engineering
Survey Results (3)
Artifacts Cited as Most Effective at Providing a Benefit When Reused
Requirements45%
Test Data11%
Documentation22%
Other22%
Which of the artifacts listed above is the most effective at providing a net benefit when reused?
Requirements are the home run of reuse
9
University of Southern California Center for Systems and Software Engineering
Survey Results (4)
Extent of Reuse of Systems Engineering Artifacts
Transition to Operations
Test and Evaluation
Develop
Conceptualize
Unaware Ad Hoc Planned
To what extent does the reuse of systems engineering artifacts occur?
Similar to distribution of systems engineering effort
10
University of Southern California Center for Systems and Software Engineering
Survey Results (5)
What are the reasons for reuse successes?– Artifact reused with minimum or no change– Product lines with significant similarities– Requirements management Utilization of personnel with experience on the project that
developed artifact
What are the reasons for reuse failures?– Underestimated modification required for reuse– New requirements placed on a modified product– Customer didn’t modify expectation of risk Lack of knowledge/understanding
11
University of Southern California Center for Systems and Software Engineering
Survey Results (6)
Promoted Benefits for Systems Engineering Reuse
Schedule21%
Cost29%
Risk19%
Quality15%
Performance16%
What are the most frequently promoted benefits as justification for systems engineering reuse?
Cost benefits implied in others?
12
University of Southern California Center for Systems and Software Engineering
Survey Results (7)
Frequency of Systems Engineering Reuse Mentioned in an RFP
Seldom50%
Occasionally25%
Always13%
Never13%
How frequently is systems engineering reuse mentioned in an RFP for a new system?
Few instances of systems developed without reuse
13
University of Southern California Center for Systems and Software Engineering
Survey Results (8)
Scaling of Expected Reuse Savings
Linear25%
Other13%
Non-linear62%
How do the expected savings from reusing systems engineering artifacts scale?
Other is a combination of linear and non-linear
Consensus was non-linearly decreasing as the number of interfaces grows
14
University of Southern California Center for Systems and Software Engineering
Survey Results (9)
Expected Effort from Reusing a Systems Engineering Artifact
Deleted vs. Managed
Adopted vs. Managed
Adopted vs. Deleted
Modified vs. Managed
Modified vs. Deleted
Modified vs. Adopted
New vs. Managed
New vs. Deleted
New vs. Adopted
New vs. Modified
Less Than About the Same More Than
Evaluate the expected effort from utilizing a systems engineering artifact classified in the first category, compared to the second.
15
University of Southern California Center for Systems and Software Engineering
Implications to COSYSMO 2.0
• Effects of systems engineering reuse are more than what is captured in the size drivers– Survey results identify personnel, processes, and platform
factors
• Reuse needs to be accounted for in both the size and cost drivers– Size drivers: previously proposed reuse extensions (Valerdi,
Gaffney, Wang)– Cost drivers: newly proposed additional cost drivers (Fortune)
16
University of Southern California Center for Systems and Software Engineering
Proposed Size Driver Extensions
Modified
Adopted
New 1.0
0
DeletedDeleted
Managed
Reu
se w
eig
ht
Reuse category
New: Artifacts that are completely new
Modified: Artifacts that are inherited, but are tailored
Adopted: Artifacts that are incorporated unmodified, also known as “black box” reuse
Deleted: Artifacts that are removed from a system
Managed: Artifacts that are incorporated unmodified and untested
Based on survey results
17
University of Southern California Center for Systems and Software Engineering
Proposed Additional Cost Drivers
OrganizationProcesses to capture or implement the reuse of artifacts; repeatable
Domain Applicability
Overlap between the original domain of the artifact and the domain the artifact is being reused within
Technology Comprehension
Availability of documentation or other non-personnel related knowledge assets that provide for or improve the understanding of the technology being addressed in the reused artifact
Reuse Understanding
Completely familiar
Systems engineer directly assisted in the development of the artifact for the original system; continual experience with the artifact; first-hand knowledge of the heritage system is available
Mostly familiarSystems engineer participated in the development of the artifact for the original system; infrequent experience with the artifact
Somewhat familiar
Systems engineer has some familiarity with the artifact and the original system which it was derived from; no first-hand knowledge of the heritage system
Mostly unfamiliar
Systems engineer has experience with similar artifacts but not the current one being reused; limited knowledge of the heritage system
Completely unfamiliar
Systems engineer has no previous experience with the artifact or the system which the artifact was derived from; completely unknown
Artifact Unfamiliarity
Reuse Understanding Artifact Unfamiliarity