Lean & Agile · Lean & Agile Systems Engineering for Systems of Systems Dr. David F. Rico, PMP, CSM...

Lean & AgileSystems Engineeringfor Systems of Systems

Dr. David F. Rico, PMP, CSMWebsite: http://davidfrico.com

LinkedIn: http://www.linkedin.com/in/davidfricoFacebook: http://www.facebook.com/profile.php?id=1540017424

2

Agenda

IntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

3

AuthorDoD contractor with 25+ years of IT experienceB.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.Large gov’t projects in U.S., Far/Mid-East, & Europe

Published six books & numerous journal articlesExpertise in metrics, models, & cost engineeringAdjunct at George Washington, UMUC, & ArgosySix Sigma, CMMI, ISO 9001, DoDAF & DoD 5000Agile Program Management & Lean Development

4

Purpose of BriefingProvide an overview of traditional, lean, and agile systems engineering concepts:

Define systems engineering, its purpose, and identify major approaches to traditional systems developmentIdentify the strengths and weaknesses of traditional systems engineering for today’s ever changing worldDiscuss lean and agile systems engineering as a means of managing ever increasing system complexityIntroduce mechanisms for scaling lean and agile systems engineering for larger systems of systemsExamine iterative testing techniques within agile systems engineering for verification and validation

5

AgendaIntroduction

Systems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

6

What is Systems Engineering?Sys-tem (sĭs-'təm): Interacting, interrelated, interdependent elements; A complex whole

Interdisciplinary approach and means to enable the realization of successful systems [INCOSE]Interdisciplinary tasks required to transformcustomer needs into a system solution [IEEE]Interdisciplinary approach for transforming a set of customer needs into a product solution [CMMI]Interdisciplinary approach for translating mission needs into operational systems [DoD 5000]Interdisciplinary processes spanning the conceptionof ideas through the retirement of a system [ISO]

7

Purpose of Systems EngineeringManage increasing system complexity (1950s)Optimize [sub]system performance (1960s)Improve system cost and quality (1970s)

Eisner, H. (2002). Essentials of project and systems engineering management. New York, NY: John Wiley & Sons.Blanchard, B. S., & Fabrycky, W. J. (2006). Systems engineering and analysis. Upper Saddle River, NJ: Pearson Prentice-Hall.

8

MIL-STD-1521BCreated by U.S. Air Force in 1976Framework for system and software reviewsStandardized milestone reviews and technical audits

U.S. Department of Defense. (1985). Military standard: Technical reviews and audits for systems, equipments, and computer software (MIL-STD-1521B). Washington, DC: Air Force Systems Command (AFSC).

Note: ActualTime Framesof Activities

Must BeTailored to

Each Program

Test

DraftSystem

RequirementsSpecification

SystemRequirementsSpecification

HWCIDevelopmentSpecification

SoftwareTop Level

DesignDocument

SoftwareListing

HWCIProduct

Specification

SoftwareProduct

Specification

IRSDBDD

DraftHWCIProduct

Specification

Sourceand

ObjectCode

SystemRequirements

Review

SystemDesignReview Software

SpecificationReview

PreliminaryDesignReviewCSCI

PreliminaryDesignReviewHWCI

CriticalDesignReviewCSCI

CriticalDesignReviewHWCI

TestReadiness

Review

FunctionalConfiguration

AuditHWCI

PhysicalConfiguration

AuditHWCI

FormalQualification

ReviewHWCI

FunctionalConfiguration

AuditCSCI

PhysicalConfiguration

AuditCSCI

SystemFunctional

ConfigurationAudit

SystemPhysical

ConfigurationAudit

SystemFormal

QualificationReview

Prepare Testand

EvaluationMaster Plan

PrepareSoftware

TestPlan

PrepareHWCITestPlan

PrepareSoftware

TestDescription

PrepareHWCITest

Procedures

PrepareSoftware

TestProcedures

PerformHWCI

SubsystemTests

PerformSoftware

Tests

PrepareSoftware

TestReports

PerformSystemTests

PrepareSystem

TestReports

ProgramActivity

ConceptExploration

Phase

Demonstrationand Validation

PhaseFull Scale Development Phase

TechnicalReviews

andAudits

Specifica-tions and

otherProducts

SystemFunctional

Identification

AllocatedIdentification

Developmental Configuration(HWCI Only) Product

IdentificationSystem

FunctionalBaseline

AllocatedBaseline

ProductBaseline

SoftwareRequirementsSpecification

IRDSoftwareDetailedDesign

Document

9

MIL-STD-498Created by U.S. Navy in 1994Consolidated multiple U.S. DoD standardsSoftware process and documentation standard

U.S. Department of Defense. (1994). Military standard: Software development and documentation (MIL-STD-498). Arlington, VA: Space and Naval Warfare Center (SPAWAR).

10

ISO-15288Created by ISO/IEC around 2002Standardization of international practicesMeant for complex, computer-based systems

International Organization for Standardization/International Electrotechnical Commission. (2002). Standard for systems engineering: System life cycle processes (ISO/IEC 15288). Geneva, Switzerland: Author.

11

CMMICreated by the SEI in 2002Merger of SW-CMM, SA-CMM, IPD-CMM, etc.Used for systems engineering process improvement

CMMI Product Team. (2006). CMMI for development version 1.2 (CMU/SEI-2006-TR-008). Pittsburg, PA: Software Engineering Institute.

12

DoD Acquisition LifecycleCreated by the U.S. DoD around 2003Latest evolution of acquisition best practicesMeant for large-scale, multi-billion weapon systems

DAU. (2009). Integrated defense acquisition, technology, and logistics life cycle management framework. Retrieved October 9, 2009, from https://acc.dau.mil/ifc

13

Systems Engineering BenefitsStudy funded by Australian defense instituteAlmost 44 programs studied from 2001 to 2004Systems engineering minimizes schedule overruns

Honour, Eric C. (2009). Demographics in measuring systems engineering return on investment (SE-ROI). Proceedings of the Joint 19th Annual International Symposium of INCOSE/Third Asia-Pacific Conference on Systems Engineering (INCOSE/APCOSE 2009), Singapore.

14

Systems Engineering StudiesU.S. Air Force Center for Systems EngineeringCase studies of 9 major U.S.Air Force ProgramsPrograms had significant cost and technical issues

Air Force Institute of Technology (AFIT). (2009). Systems engineering case studies. Retrieved October 19, 2009, from http://www.afit.edu/cse/cases.cfm

PROGRAM NAME EST. COST EST. UNIT ACT. COST ACT. UNIT OVERRUN OTHER ISSUES

A-10 Thunderbolt $2,547,600,000 439 $5,511,490,000 360 21,005% Wing Reliability

B-2 Spirit $58,200,000,000 132 $45,300,000,000 20 57,949% Low Observability

C-5A Galaxy $3,413,200,000 120 $4,426,400,000 81 7,585% Maintenance

F-111 Aardvark $2,171,590,000 547 $8,652,000,000 547 298% Structural

RQ-4A Global Hawk $2,904,600,000 51 $3,816,700,000 51 31% Quality

GPS Navstar $814,400,000 28 $8,650,000,000 14 31,764% Sofware Reliability

HST Hubble $651,200,000 1 $1,458,900,000 1 124% Schedule, Mirror

KC-10 Extender $1,055,000,000 20 $1,090,600,000 20 3% Reliability

LGM-118A Peacekeeper $16,600,000,000 21 $16,600,000,000 21 0% Range

$9,817,510,000 151 $10,611,787,778 124 13,196% $85,655,686

15

AgendaIntroductionSystems Engineering

Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

16

What is a Challenge?Chal-lenge (chăl-'ənj): Contest, competition, fight, defy, confront, or dispute; To question

21st century systems are more software-intensiveand highly-complex with numerous invisible partsTechnology is evolving at an exponential rate of change which severely limits the planning horizonGlobal competitiveness has intensified and new military threats are rapidly emerging all of the timeCustomers have unpredictable needs and necessitate decision-making flexibility throughout the programToday’s post-industrial information age knowledge workers need new systems engineering approaches

17

Information AgeU.S. is no longer an industrial-age nationU.S. part of a group of post-industrial countriesU.S. consists of information-age knowledge workers

Bell, D. (1999). The coming of post industrial society. New York, NY: Basic Books.

0%

20%

40%

60%

80%

100%

1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990

Perc

ent o

f Eco

nom

y

Information

Service

Industry

Agriculture

18

System Complexity is Growing21st century systems are becoming more complexNumber of physical parts are becoming smallerNano-circuitry and software hide complexity

Moody, J. A., et al. (1997). Metrics and case studies for evaluating engineering designs. Upper Saddle River, NJ: Prentice-Hall.

19

Software CenturyNumber of software-intensive systems is growingYearly software industry revenue exceeds $3 trillionPoor software quality costing trillions in lost revenues

Dvorak, D. L. (2009). NASA study on flight software complexity. Pasadena, CA: Jet Propulsion Laboratory (JPL).

Software in Military Aircraft

8% 10%20%

35%45%

65%

80%

0%

20%

40%

60%

80%

100%

1960F-4

1964A-7

1970F-111

1975F-15

1982F-16

1990B-2

2000F-22

Perc

ent o

f Fun

ctio

nalit

y

20

Exponential Rate of ChangeTechnology evolving at an ever increasing rateNano-scale computers will become the norm soonTechnological breakthroughs may climax in 25 years

Kurzweil, R. (2005). The singularity is near: When humans transcend biology. New York, NY: Penguin Group.

21

Crossing the ChasmNew technology spreads very slowlyThere are a few innovators and early adoptersYears and decades for most to adopt new technology

Moore, G. A. (1991). Crossing the chasm: Marketing and selling technology to mainstream customers. New York, NY: Harper Business.

22

Coping With Big ChangesHumans can’t cope with large technological changeChanges may be resisted for a long time (years)Big projects plunge organizations into chaos

Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).

23

Global Market CompetitionGlobalization has intensified market competitionDomestic competition is no longer the major threatThe trade deficit with the Far East is growing bigger

24

Cyber Threats are GrowingCyber threats have increased 10-fold in last decade70% of cyber incidents perpetrated by U.S. citizensCyber threats coming from Far East less than 3%

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 20030

500

1000

1500

2000

2500

3000

3500

4000

4500

Incidents

Vulnerablities

25

Complex Systems are UnstableLarge systems experience big downstream changesProject plans designed to cope with small changesSystems engineering not well-suited for changes

Jones, C. (1995). Patterns of software system failure and success. Boston, MA: International Thompson Computer Press.

Basic Assembly 575JCL 400Macro Assembly 400C 225Cobol 74 (Cobol I) 220FORTRAN 210Cobol 85 (Cobol II) 175Pascal 160PL/1 126RPG I 120RPG II/III 110Natural 100C++ 80Java 80dBase III 60Focus 60Clipper 60Oracle 60Sybase 60dBase IV 55Perl 50JavaScript 50VBScript 50Shell Script 50SAS 50APL 50

26

High Project Failure RatesFailed and challenged projects hover around 70%High failure rate due to inability to cope with changeBig projects exacerbate challenge and failure potential

Johnson, J., et al. (2009). Chaos summary 2009. Boston, MA: Standish Group International.

27

AgendaIntroductionSystems EngineeringSystems Engineering Challenges

Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

28

What is Lean?Lean (lēn): Thin, slim, slender, narrow, adequate, or just-enough; Without waste

A customer-driven systems engineering process that delivers the maximum amount of business valueAn economical systems engineering way of planning and managing the development of complex systemsA systems engineering process that is free of excess waste, capacity, and non-value adding activitiesJust-enough, just-in-time, and right-sized systems engineering processes, documentation, and toolsA systems engineering approach that is adaptable to change in customer needs and market conditions

29

Lean ThinkingTerm coined by John Krafcik of MIT in 1988Taiichi Ohno of Toyota is credited with its ideasToyota Production System was adapted from Ford

Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.

30

Lean Six SigmaCreated in late 1990s by Allied Signal and MaytagCombination of Six Sigma and Lean ThinkingFocuses on eliminating waste vs. variation

George, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.

31

Lean DevelopmentLean product development emerged in the 1980sAdaptation of Toyota Production System (TPS)“Toyota [New] Product Development System”

Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.

Lean Development

Frequent set-up changes

Lean Manufacturing

Short manufacturing throughput time

Reduced work-in-process inventory between manufacturing steps

Frequent transfer of small batches of parts between manufacturing steps

Reduced inventory requires slack resources and more information flow between steps

Adaptability to changes in volume, product mix, and product design

Broad task assignments for production workers gives higher productivity

Focus on quick problem solving and continuous process improvement

Simultaneous improvement in quality, delivery time, and manufacturing productivity

Frequent product changes

Short development time

Reduced information inventory between development steps

Frequent transfer of preliminary informationbetween development steps

Reduced development time requires slack resources and information flow between stages

Adaptability to changes in product design, schedule, and cost targets

Broad task assignments for engineers (developers) gives higher productivity

Focus on frequent incremental innovation and continuous product and process improvement

Simultaneous improvement in quality, development time, and development productivity

32

Lean Systems EngineeringOrigin in MIT Lean Aerospace Initiative in 1992Lean Systems Engineering WG formed in 2006Lean Enablers for Systems Engineering in 2009

INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg

33

Lean+ 10XCreated by Charles Toups of Boeing in 2008In-use by P-8A Poseidon and AEW&C SystemAdaptation of lean thinking for non-manufacturing

Brabant, E. M. (2009). Simple as. Retrieved October 20, 2009, from http://www.boeing.com/news/frontiers/i_ids01.pdf

34

Lean Engineering BenefitsMIT has studied dozens of systems for last 15 yearsThey applied criteria to determine if they were leanNumerous programs, past, present, and future

Murman, E., et al. (2002). Lean enterprise value: Insights from MIT's lean aerospace initiative. New York, NY: Palgrave.

35

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems Engineering

Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

36

What is Agility?A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and ease of movement; To be very nimble

The ability to create and respond to change in order to profit in a turbulent global business environmentThe ability to quickly reprioritize use of resources when requirements, technology, and knowledge shiftA very fast response to sudden market changes and emerging threats by intensive customer interactionUse of evolutionary, incremental, and iterativedelivery to converge on an optimal customer solutionMaximizing the business value with right-sized, just-enough, and just-in-time processes and documentation

37

What are Agile Methods?‘Adaptable’ software development methodologies‘Human-centric’ method for creating business value‘Alternative’ to large document-based methodologies

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

38

Crystal MethodsCreated by Alistair Cockburn in 1991Has 14 practices, 10 roles, and 25 productsScalable family of techniques for critical systems

Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.

39

ScrumCreated by Jeff Sutherland at Easel in 1993Has 5 practices, 3 roles, 5 products, rules, etc.Uses EVM to burn down backlog in 30-day iterations

Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.

40

Dynamic Systems Develop.Created by group of British firms in 199315 practices, 12 roles, and 23 work productsNon-proprietary RAD approach from early 1990s

Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.

41

Feature Driven DevelopmentCreated by Jeff De Luca at Nebulon in 1997Has 8 practices, 14 roles, and 16 work productsUses object-oriented design and code inspections

Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.

42

Extreme ProgrammingCreated by Kent Beck at Chrysler in 1998Has 28 practices, 7 roles, and 7 work productsPopularized pair programming and test-driven dev.

Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.

43

Side-Effects of Agile MethodsEnable us to cross-the-chasm sooner or earlierReduce chaos associated with large-scale changeReduce or divide the risk of change into small pieces

Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).

44

Essence of Agile MethodsHigh degree of customer & developer interactionHighly-skilled teams producing frequent iterationsRight-sized, just-enough, and just-in-time process

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

45

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems Engineering

Agile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

46

What is a Practice?Prac-tice (prăk-'tĭs): Action, tool, technique, or work instruction; Step-by-step procedure

A set of one or more systems engineering techniquesto accomplish a specific action or desired outcomeStandard or semi-formal best practices or rules-of-thumb that are proven to be effective or efficientA suite of manual or automated tools or instruments that are useful for system design and developmentAn array of optional elements that may be employed on an as-needed basis, i.e., right tool at the right timeValue-adding action that may significantly enhance productivity, quality, or other key performance metric

47

Release PlanningCreated by Kent Beck at Chrysler in 1998Project plan with a 30-60-90-day timing horizonDisciplined and adaptable project management F/W

Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.

48

Onsite CustomersTerm coined by Kent Beck in 1999Customer who sits with developers full-timeFast and efficient way to capture customer needs

Tabaka, J. (2006). Collaboration explained: Facilitation skills for software project leaders. Upper Saddle River, NJ: Addison Wesley.

49

User StoriesTerm coined by Kent Beck in 1999Functions or features of value to customersHighly adaptable requirements engineering process

Cohn, M. (2004). User stories applied: For agile software development. Boston, MA: Addison-Wesley.

50

Pair ProgrammingTerm coined by Jim Coplien in 1995Consists of two side-by-side programmersHighly-effective group problem-solving technique

Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.

51

RefactoringTerm coined by William Opdyke in 1990Process of frequently rewriting source codeImproves readability, maintainability, and quality

Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.

52

Test-Driven DevelopmentTerm coined by Kent Beck in 2003Consists of writing all tests before codingEnsures all source code is verified and validated

Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.

53

Continuous IntegrationTerm coined by Martin Fowler in 1998Process of automated build/regression testingEvaluates impact of changes against entire system

Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.

BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Compile Source Code

Run Unit Tests

Run Coverage TestsStatic Code Analysis

Build Database

Generate Help FilesArchive and Deploy

54

Agile DocumentationMyth that voluminous documentation is neededMyth that agile methods do not use documentationRight-sized, just-in-time, and just enough documents

Rueping, A. (2003). Agile documentation: A pattern guide to producing lightweight documents for software projects. West Sussex, England: John Wiley & Sons.

Contracts

Document Type

Project Plans

Requirements

Architecture

Design

Coding

Tests

User guides

Quality Assurance

Agile Documentation

Performance-based, time-and-materials, level-of-effort

Release plans, iteration plans, story boards, agile repositories

User stories, wire frames, use cases, paper prototypes

Metaphors, spikes, system modeling language, DoDAF

Wire frames, design patterns, unified modeling language

Code patterns, program design language, coding comments

Unit, component, integration, system, and acceptance tests

XML documents, online help, Wikis, FAQs, video and audio clips

Performance, reliability, code structure analysis, and test reports

55

Which Practices Are In-UseSurveys of agile practices are conducted annuallyRelease planning is the most often used practiceContinuous integration is also a major practice

Version One. (2008). The state of agile development: Third annual survey. Alpharetta, GA: Author.

86%

75%

72%

65%

62%

60%

59%

59%

52%

49%

44%

35%

34%

31%

31%

0% 20% 40% 60% 80% 100%

Iteration planning

Daily standups

Release planning

Continuous integration

Automated builds

Burndown

Retrospectives

Refactoring

Velocity

Test-driven development

Open work area

Digital taskboard

On-site customer

Pair programming

Information radiators

56

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering Practices

Agile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

57

What is Scalability?Scal-a-ble (skāl-'ə-bəl): To expand, grow, stretch, raise, or intensify; Increase in size

A systems engineering process that applies to projects of varying size, scope, magnitude, and complexityA product development process that is tailorable to the type, kind, and class of product under developmentA process that works well on range of products, from small to large programs involving systems of systemsAn approach that enables control of time, cost, scope, quality, and performance regardless of program typeSystems engineering processes designed to maximize business value under a wide variety of constraints

58

Multi-Level TeamsEnables projects to plan for the future and presentDecomposes capabilities into implementable piecesUnclogs the drainpipes to let the execution flow freely

Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.

59

Multi-Level BacklogEnables multiple levels of abstraction to co-existAllows customers and developers to communicateMakes optimum use of everyone’s time and resources


Feature SetRelated user stories that are grouped togetherAlso called a Theme, i.e., implemented as entityComprises 6 to 30 days worth of work

CapabilityHigh-level business or product functionAlso called an Epic, i.e., multiple feature setsComprises 18-90 days worth of work

User StorySimple requirement written by customer or userA small unit of functionality having business valueComprises 2 to 10 days worth of work

60

Multi-Level PlanningEnables multiple level enterprise plans to co-existAllows stakeholders to build viewpoint-specific plansEnsures capabilities are delivered at regular intervals

Release PlanFeature set focused, subsystem architectureStrategy, objectives, and backlog6 to 12 weeks

Product RoadmapCapability focused, enterprise architecture needsVision, objectives, backlog18 to 36 weeks

Iteration PlanUser story focused, component-level architectureImplementation plan, objectives, and backlog2 to 4 weeks


61

Multi-Level CoordinationEnables lean and agile methods to scale-upAllows enterprises to create large-scale programsUnleashes optimum productivity and overall control

Schwaber, K. (2004). Agile project management with scrum. Redmond, WA: Microsoft Press.

62

Multi-Level GovernanceEnables enterprises to achieve functional needsAllows programs to coordinate functional activitiesEnsures optimal technical performance is achieved

Ambler, S. W. (2009). Scaling agile software development through lean governance. Proceedings of the 2009 ICSE Workshop on Software Development Governance, Vancouver, Canada, 1-2.

Q

I

R

W

T

A

S

C

D

R

R

R

R

R

R

R

R

R

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

T

T

T

T

T

T

T

T

T

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

S

S

S

S

S

S

S

S

S

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

R T S

63

Multi-Level PMOEnables enterprises to optimize project performanceAllows enterprises to control and monitor programsEnsures projects are operating at peak capability

Augustine, S., & Cuellar, R. (2006). The lean-agile PMO: Using lean thinking to accelerate agile project delivery. Agile Project Management Executive Report, 7(10), 1-28.

64

Multi-Level AutomationEnables enterprises to be flexible but disciplinedAllows enterprises to distribute project work teamsEnsures distributed project teams are collaborating

Project Management

RequirementsDOORSRequisite ProSLATE

DesignRhapsodyTelelogic System ArchitectRational System Architect

CodingEclipseVisual StudioSun Studio

TestingJUnitNUnitXunitCPPUnit

GtestFitFitnesseSelenium

Quality AssuranceCheckStylePMDEMMAJdependCoberturaGcov

Configuration MgtSubversion (SVN)Concurrent Versions Sys.ClearCase

Build AutomationAntNAntMavenMake

Continuous Integ.Cruise ControlHudsonBuildBot

CollaborationWebExSkypeMeetMeWimba

WikiMediaWikiTracWikiPhpWiki

DocumentationNDocJavadocDoxygeniText

Version OneRallyScrum WorksVSTS

Agile TeamAgile EnterpriseScope ManagerStory Studio

XP Plan ItIterateXP TrackerAgilo

XP CGIXP WebXplannerIce Scrum

Project CardsTarget ProcessXtreme PlannerTeam System

CommunityEnterpriseMingleHansoft

65

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering Scaling

Agile Systems Engineering TestingAgile Systems Engineering ValueSummary

66

What is Integration?In-te-gra-tion (ĭn-'tĭ-grā-shən): To add, group, mix, or assemble; Act of combining

A critical verification and validation step in the systems engineering process for a complex new systemA process of testing and evaluating units, components, subsystems, systems, and systems of systemsA key best practice that enables suppliers to deliver operational systems to customers early and oftenA automated systems development process that lowers the risks of developing large-scale complex systemsA lean and efficient process that maximizes business value by eliminating waste from traditional testing

67

What is Agile Testing?Traditional testing is a late, manual processAgile testing is an early and automated processThe goal of agile testing is to deliver early and often

Traditional Testing

Combining source files

Combining software and environment

Combining software and data

Combining software and tests

Combining developers

Agile Testing

Code is frequently checked in

Code is automatically retrieved

Compilation is done automatically

Tests are done automatically

Code reports are generated

Developers get instant feedback

Code is automatically deployed or packaged for delivery

Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.

68

Agile Testing ProcessDevelopers check-in changes as they occurServer detects all changes and initiates testingServer compiles, tests, analyzes, builds, and deploys


BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Compile Source Code

Run Unit Tests

Run Coverage TestsStatic Code Analysis

Build Database

Generate Help FilesArchive and Deploy

69

Agile Testing TechnologiesThere are literally hundreds of agile testing toolsThere are tools for building, testing, and deployingIntegration tools monitor repositories and initiate tests

Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.

70

Agile Testing StatisticsFewer builds leave in higher bug countsA high number of builds eliminates the defectsGoal is to have as many, early builds as possible

Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.

71

Scaling Agile TestingAgile testing slows down with very large systemsSlow testing slows integration and increases bugsAgile testing can speed back up with proper attention

Kokko, H. (2009). Increase productivity with large scale CI: Reduce feedback cycle from weeks to 100 minutes. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.

Wide Impact Tuning

Fast builds – less changes – more green

Parallelization of test runs

ClearCase to subversion

Pre-installing as much as possible

Removal of randomness

Compilation in memory

Installation starting parallel with system build

Focused Impact Tuning

More memory and CPUs

Parallelize builds

Replace 3rd party test libraries

Reduce/remove timeouts in tests

Select different tests

Refactor code & components

Tune the network & software

Tune the database

72

Agile Testing CostsMost agile testing tools are “free” open sourceA build server is no more than a commodity PCLow overhead for new and subsequent setup time

Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.

73

Agile Testing BenefitsReduces the cost-of-change by 10 timesFrequent builds dramatically lower defect levelsEnables early “what-if” tests as well as late changes

Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.

when integration/regression testing at each code check-in

Major municipal gas utility

Large retail web site

Global supplier of healthcare equipment

More features and higher quality

Added new functionality 2 weeks before ship

“Oozing Confidence”

74

Agile Testing is NextGenManual testing is CMMI Capability Level 0 or 1Agile testing is a CMMI Capability Level 5 practiceIt is planned, defined, measured, and it’s optimizing

Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.

75

Agile Testing Side-EffectsEliminates big-bang integration in the 11th hourCreates a repeatable and reliable testing processEvaluates system-wide changes throughout project


76

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering Testing

Agile Systems Engineering ValueSummary

77

What is Business Value?Val-ue (văl-'yōō): An amount, quantity, rate, magnitude, or desirability; Economic worth

An economic estimation of the tangible worth of the organizational assets such as buildings and equipmentAn appraisal of intangible assets such as knowledge, experience, skills, patents, processes, and methodsA technique for evaluating the costs and benefits of investments in a business, operations, or personnelThe economic impact of deploying a new product development approach such as systems engineeringThe total life cycle costs of institutionalizing lean and agile systems engineering techniques in an enterprise

78

Agile (138 pt.) and traditional methods (99 pt.)Agile methods fare better in all benefits categoriesAgile methods 359% better than traditional methods

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.

Studies of Agile Methods

79

Productivity of Agile MethodsPP productivity 32X more than trad. methodsScrum productivity 5X more than trad. methodsAgile methods productivity 20X more than traditional

Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.

33.4044

29.28

21.2374

16.1575

5.4436

1.06190

8

16

24

32

40

PP TDD Agile XP Scrum CMMI®

Software Method

Pro

duct

ivit

y (L

OC

/Hou

r)

80

Quality of Agile MethodsXP quality 13X better than trad. methodsScrum quality 3X better than trad. methodsAgile methods quality 5X better than traditional

0.7466

1.7972 2.155 2.355

3.945

9.667

0

2.2

4.4

6.6

8.8

11

XP Agile TDD PP Scrum CMMI®

Software Method

Qua

lity

(Def

ects

/KLO

C)


81

Costs of Agile MethodsXP costs 8X less than traditional methodsScrum costs 2X less than traditional methodsAgile methods cost 5X less than traditional methods

$136,551$226,807 $249,653 $265,436

$578,202

$1,108,233

$0

$325,000

$650,000

$975,000

$1,300,000


Software Method

Cos

ts


82

Benefits of Agile MethodsXP benefits 1.5X more than traditional methodsScrum benefits 1.3X more than traditional methodsAgile methods benefits 1.4X more than trad. methods

$4,372,282$4,282,026 $4,259,180 $4,243,397

$3,930,631

$3,023,064

$2,750,000

$3,237,500

$3,725,000

$4,212,500

$4,700,000


Software Method

Ben

efit

s


83

ROI of Agile MethodsXP ROI 18X more than traditional methodsScrum ROI 3.4X more than traditional methodsAgile methods ROI 10X more than trad. methods

3,102%

1,788%1,606% 1,499%

580%

173%

0%

925%

1,850%

2,775%

3,700%


Software Method

Ret

urn

on In

vest

men

t


84

NPV of Agile MethodsXP NPV 2.4X more than traditional methodsScrum NPV 1.9X more than traditional methodsAgile methods NPV 2.3X more than trad. methods

$3,649,388$3,480,979 $3,438,351 $3,408,902

$2,825,313

$1,509,424

$1,000,000

$1,787,500

$2,575,000

$3,362,500

$4,150,000


Software Method

Net

Pre

sent

Val

ue


85

Real Options of Agile MethodsXP ROA 1.6X more than traditional methodsScrum ROA 1.4X more than traditional methodsAgile methods ROA 1.6X more than trad. methods

$4,265,936$4,105,884 $4,066,678 $4,040,377

$3,608,772

$2,633,052

$2,200,000

$2,825,000

$3,450,000

$4,075,000

$4,700,000


Software Method

Rea

l Opt

ions


86

AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering Value

Summary

87

SummaryAgility is the evolution of management thoughtConfluence of traditional and non-traditional ideasImprove performance by over an order-of-magnitude

Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing.

Systems engineering approaches

...

New product development approaches

Expertly designed to be fast and efficient

Intentionally lean and free of waste (muda)

Systematic highly-disciplined approaches

Capable of producing high quality systems

Right-sized, just-enough, and just-in-time tools

Intended to maximize business value for customers

88

New Book on Agile MethodsGuide to Agile Methods for business leadersCommunicates business value of Agile MethodsRosetta stone to Agile Methods for traditional folks

http://davidfrico.com/agile-book.htm (Description)http://www.amazon.com/dp/1604270314 (Amazon)

Table of Contents 1. Introduction to Agile Methods 2. Values of Agile Methods 3. History of Agile Methods 4. Antecedents of Agile Methods 5. Types of Agile Methods 6. Practices of Agile Methods 7. Agile Project Management 8. Agile Software Engineering 9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Surveys of Agile Methods14. Costs-Benefits of Agile Methods15. ROI Metrics of Agile Methods16. Measures of Agile Methods17. Costs of Agile Methods18. Benefits of Agile Methods19. ROI of Agile Methods20. NPV of Agile Methods21. Real Options of Agile Methods22. Business Value of Agile Methods23. Agile vs. Traditional Methods24. Future of Agile Methods

Lean & Agile · Lean & Agile Systems Engineering for Systems of Systems Dr. David F. Rico, PMP, CSM...

Documents

Transcript of Lean & Agile · Lean & Agile Systems Engineering for Systems of Systems Dr. David F. Rico, PMP, CSM...