Pittsburgh, PA 15213-3890 “They Keep Moving the Cheese”
Transcript of Pittsburgh, PA 15213-3890 “They Keep Moving the Cheese”
Sponsored by the U.S. Department of Defense© 2003 by Carnegie Mellon University
page 1
Pittsburgh, PA 15213-3890
26 January 2003
“They Keep Moving the Cheese”
A Framework for EvolutionaryAcquisition of Large Software IntensiveSystems
Cecilia AlbertLisa Brownsword
Report Documentation Page Form ApprovedOMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.
1. REPORT DATE 26 JAN 2003 2. REPORT TYPE
3. DATES COVERED 00-00-2003 to 00-00-2003
4. TITLE AND SUBTITLE ’They Keep Moving the Cheese’ A Framework for EvolutionaryAcquisition of Large Software Intensive Systems
5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Carnegie Mellon University,Software Engineering Institute,Pittsburgh,PA,15213
8. PERFORMING ORGANIZATIONREPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as
Report (SAR)
18. NUMBEROF PAGES
24
19a. NAME OFRESPONSIBLE PERSON
a. REPORT unclassified
b. ABSTRACT unclassified
c. THIS PAGE unclassified
Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
© 2003 by Carnegie Mellon University page 226 January 2003
Who Moved My Cheese?
Reprinted through the courtesy of CIO© 2002 CXO Media Inc.
© 2003 by Carnegie Mellon University page 326 January 2003
A Story…Program goal: provide a tool for strategic, operational, and tacticalplanners from all services and defense agencies to support joint andcoalition engagements and peace keeping efforts• Run on existing enterprise backbone (managed by another agency)• Interface with multiple existing and developing systems• Operate across multiple security levels
6 increments delivered across 6-7years• First release in 18-24 months
• Automate manual process• Client-server architecture• Support 2-3 day planning cycle
Program Start (late ’90s)
• Increment 1 is obsolete• Struggling to build/field increment 2• Users have built “interim” solutions• Future is uncertain
• New planning processes• Web-based architecture• Dynamic planning cycles• Collaborative planning
2003
© 2003 by Carnegie Mellon University page 426 January 2003
Size Matters!
0%+36+500>$10M
8%+24+250$6M-$10M
15%1840$3M-$6M
25%1225$1.5M-$3M
33%912$750K-$1.5M
55%66< $750K
SuccessRate
Time(mos)
PeopleProject Size
Source:The Standish Group, 1999
© 2003 by Carnegie Mellon University page 526 January 2003
Definitions
A software-intensive system is one that• Relies on software to provide core/priority mission
function(s)
A large software-intensive system is one whose software• Takes longer than 6 months to implement• Takes more than 6 people to implement• Takes more that $750,000 to implement
and/or• Is comprised of multiple interrelated systems or
independently developed components implemented insoftware (system of systems, family of systems, etc)
© 2003 by Carnegie Mellon University page 626 January 2003
Outline
Change Happens
Adapting to Change
Be Ready to Change Quickly
© 2003 by Carnegie Mellon University page 726 January 2003
Change Happens
Large software-intensive systems change at a rate fasterthan the full system capability can be implemented – andthey change during development and operation
Sources of change• Enterprise priorities shift• Business or operational needs change• New technologies introduce new opportunities• COTS products add and delete key features• Participants rotate• …
© 2003 by Carnegie Mellon University page 826 January 2003
Adapt to Change
Evolutionary AcquisitionDelivers capability in increments,recognizing, up front, the need forfuture capability improvements
• Success of the strategy depends onthe consistent and continuousdefinition of requirements andmaturation of technologies that leadto disciplined development andproduction of systems that provideincreasing capability towards amaterial concept.
Spiral DevelopmentA desired capability is identified butthe end-state requirements are notknown at program initiation
• Those requirements are refinedthrough demonstration and riskmanagement; there is continuoususer feedback; and each incrementprovides the user the best possiblecapability. The requirements forfuture increments depend onfeedback from users and technologymaturation.
* The Operation of the Defense Acquisition System, 30 Oct 02
DoD 5000* provides mechanisms for coping with change
© 2003 by Carnegie Mellon University page 926 January 2003
Lessons Learned
Going after “low hanging fruit” in the absence of anoverarching architecture and coherent plan results inincompatible and stove-piped solutions
System requirements defined without sufficient insight intowhat can be realistically built, results in systems that cannotbe built
There are no “silver bullets” that avoid disciplined systemand software engineering (doing the right engineeringcorrectly)
© 2003 by Carnegie Mellon University page 1026 January 2003
Be Ready To Change QuicklyConsciously apply spiral development practices at 2 (ormore) discrete levels – with continuous interaction betweenthe levels
• Program or system level- Evolve definition and implementation plan for system
end-state- Define and spawn increments of useful capability that
will build to full system functionality and performance
• Project or increment level- Define and implement plan for delivering the defined
increment in the context of the system end-state
© 2003 by Carnegie Mellon University page 1126 January 2003
Disciplined Spiral Development
• Continuously determine acompatible and feasible set of:
business processes, requirements, plans, architecture,COTS products and other components
• Enterprise business objectives drive solution definition
• Risk considerations drive degree of detail
• Executable representations demonstrate current understandingand agreements
Spiral development facilitates evolving a viable solution –at both system and increment levels
Executable
Time
© 2003 by Carnegie Mellon University page 1226 January 2003
Phases Bounded by Anchor Points
Simultaneous Definition
and Tradeoffs
LifeCycleObjectives
LifeCycleArchitecture
InitialOperationalCapability
… … … …Plan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutablePlan Plan Plan Plan iteration
GatherGatherGatherGatherinformation
AssessAssessAssessAssessiteration
RefineRefineRefineRefineinto harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
AssembleAssembleAssembleAssembleexecutable
Converging
decisions
Scope Design Build Field
Multiple iterations per phase
© 2003 by Carnegie Mellon University page 1326 January 2003
Disciplines* Extend Across Phases
Analysis & design
Test
Requirements
Implementation
Project management
Market research
Business modeling
*adapted from Kruchten; shows partial set of disciplines
Scope Design Build Field
© 2003 by Carnegie Mellon University page 1426 January 2003
Keep a Long View in Systems Planning
Reprinted through the courtesy of CIO © 2002 CXO Media Inc.
© 2003 by Carnegie Mellon University page 1526 January 2003
A B C D
Current state
Increment 1
Increment 2
Future state
Z vision
Evolving System Definition
D1
Z1
New technologyModified environment
Changed mission
reassess and replanB1
actual
© 2003 by Carnegie Mellon University page 1626 January 2003
Take a Short View on Increment Planning
Allows a stable development environment – if a short timeframe(6-18 months)
Allows focused discovery, experimenting, and learning on amanageable scale to find optimum way to understand and meetuser needs
Reprinted through the courtesy of CIO© 2002 CXO Media Inc.
© 2003 by Carnegie Mellon University page 1726 January 2003
Increment Activity MappingScope Design Build Field
Define feasible scope
Survey/try components
Agree to business changes
Refine, experiment,& select solution
Try/select components
Prototype business changes
Implement selectedsolution
Apply/track components
Prepare to change business processes
Rollout and support solution
Use/track components
Change business processes
Establish project plan
Develop business case
Outline candidatearchitectures
Study COTS market;screen candidates
Prepare demos ofcandidate solutions
Identify key risks
Determine businesschanges
Update project plan
Define, baseline anddemonstrate solution
Evaluate COTS productsand components
Stabilize requirements andarchitecture
Develop plan to managebusiness process change
Update project plan
Build production qualitysolution for beta test
Continue market/COTSsurveys and evaluation
Prepare end users forinitial fielding
Complete rollout
Fix bugs, adjustfeatures, make minorenhancements
Achieve user satisfaction/ self-supportability
Continue market/COTSsurveys and evaluation
Support solution untilretirement
6 to 18 months
© 2003 by Carnegie Mellon University page 1826 January 2003
Leverage Feedback between Long- andShort-Term
Maintain long-term strategy(system level) aligned withenterprise improvement
Make short-termimplementation decisions(increment level) aligned withlong-term strategy
Use knowledge gained in short-term increments to evolve long-term strategy
Reprinted through the courtesy of CIO © 2002 CXO Media Inc.
Anticipate continuous change
© 2003 by Carnegie Mellon University page 1926 January 2003
planning andenactment
evaluation in context of systemscenarios
evaluatedincrementscenarios
initial scope System Level
Increment Level
refined scope,requirement,architectureadjustments
analyzed in context of incrementlevel scenarios
updatedrequirements,
architecture
Plan and Manage Efficient Feedback
Decisions take place simultaneously at both levels –one informs the other
planning andenactment
© 2003 by Carnegie Mellon University page 2026 January 2003
Managing Continuous EvolutionSystem level
Scope Design
• Businessmodel
• Scope• Constraints• Market study
• Critical use cases atsystem level (what)
• Architecture (how)• Available and
projected technology
LCO LCA
Scope Design
• Businessmodel
• Scope• Constraints• Market study
• Critical use cases atsystem level (what)
• Architecture (how)• Available and
projected technology
LCO LCA
…
Scope Design Build Field
Increment #1
LCO LCA IOC6-18 months
© 2003 by Carnegie Mellon University page 2126 January 2003
Scenarios of Multiple Increments
…
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Severalincrements fordifferent areasof systemcapabilityrunningconcurrently
System level
Scope Design
LCO LCA
System level
Scope Design
LCO LCA
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Scope Design Build Field
Increment #n
LCO LCA IOC6-18 months
Several increments for same area ofsystem capability where successivegenerations provide greater capability
© 2003 by Carnegie Mellon University page 2226 January 2003
The Handwriting on the WallChange Happens
Adapt To Change Quickly
• Anticipate Change
• Monitor Change
• Change
• Enjoy Change!
Be Ready To Change QuicklyAnd Enjoy It Again
Reprinted through the courtesy of CIO © 2002 CXO Media Inc.
© 2003 by Carnegie Mellon University page 2326 January 2003
Contact Information
Lisa Brownsword Cecilia [email protected] [email protected]
© 2003 by Carnegie Mellon University page 2426 January 2003
Lisa Brownsword is a senior memberof the technical staff in the Commercial-off-the-shelf- (COTS)-Based Systems(CBS) Initiative at the SoftwareEngineering Institute (SEI). Beforejoining the SEI, Lisa was on staff atComputer Sciences Corporation insupport of NASA/Goddard’s SoftwareEngineering Lab. Prior to that, she wasemployed at Rational SoftwareCorporation providing consulting tomanagers and technical practitioners inthe use of and transition to softwareengineering practices, includingarchitecture-centered development,product lines, object technology, Ada,and CASE.
Cecilia Albert is a senior member ofthe technical staff in the Commercial-off-the-shelf- (COTS)-Based Systems(CBS) Initiative at the SoftwareEngineering Institute (SEI). Beforejoining the SEI, Cecilia was in the AirForce where she served in a variety ofinformation technologies relatedpositions including: developing majorsoftware programs for simulation,command and control, and missionprocessing of national satellite systems;teaching acquisition and leading anindustry study on telecommunicationsand information systems at theIndustrial College of the Armed Forces;and managing the archive anddissemination programs at the NationalImagery and Mapping Agency.