3030

Software Engineering Software Engineering andand

Best PracticesBest Practices

Sources: Various. Sources: Various. Rational Software Corporation slides, Rational Software Corporation slides,

OOSE textbook slides, Per Kroll talk, How to Fail OOSE textbook slides, Per Kroll talk, How to Fail with the RUP article, textbookswith the RUP article, textbooks

Most slides have been modified considerablyMost slides have been modified considerably

30 2

Fundamental Terms / Fundamental Terms / ConceptsConcepts

►Science and EngineeringScience and Engineering Discover Discover

►Relationships that exist but are not foundRelationships that exist but are not found►Formulas; chemical composition, d=r*t; calories in Formulas; chemical composition, d=r*t; calories in

fats, carbohydrates, proteins; experimentation;fats, carbohydrates, proteins; experimentation;►Astrophysics – origins of the universeAstrophysics – origins of the universe

BuildBuild►Apply principles of science and mathematics to real Apply principles of science and mathematics to real

needs, commodities, structures, products, etc.needs, commodities, structures, products, etc.►Software Engineering; Software Software Engineering; Software

DevelopmentDevelopment

30 3

Fundamental Concepts / Terms Fundamental Concepts / Terms (2)(2)

►Software Engineering; Software Software Engineering; Software DevelopmentDevelopment

► Job positions: Job positions: Software developerSoftware developer ProgrammerProgrammer Software engineerSoftware engineer Analyst / ProgrammerAnalyst / Programmer Senior … what have you…Senior … what have you…

30 4

What is Software What is Software Engineering?Engineering?

►The process of solving customers’ problems The process of solving customers’ problems by the systematic development and by the systematic development and evolution of large, high-quality software evolution of large, high-quality software systems within cost, time and other systems within cost, time and other constraintsconstraints

►Note:Note: Process, systematic (not ad hoc), evolutionary… Process, systematic (not ad hoc), evolutionary… Constraints: high quality, cost, time, meets Constraints: high quality, cost, time, meets

user requirementsuser requirements

30 5

Analysis of the Definition:Analysis of the Definition:► Systematic development and evolutionSystematic development and evolution

An engineering process involves applying An engineering process involves applying well understood techniqueswell understood techniques in a in a organizedorganized and and disciplineddisciplined way way

Many Many well-accepted practices have been formally standardizedwell-accepted practices have been formally standardized► e.g. by the IEEE or ISOe.g. by the IEEE or ISO

Most development work is Most development work is evolutionaryevolutionary

► Large, high quality software systemsLarge, high quality software systems Software engineering techniques are needed because large systems Software engineering techniques are needed because large systems

cannot be completely understoodcannot be completely understood by one person by one person TeamworkTeamwork and co-ordination are required and co-ordination are required Key challenge: Dividing up the work and ensuring that the parts of the Key challenge: Dividing up the work and ensuring that the parts of the

system work properly togethersystem work properly together The end-product that is produced must be of sufficient qualityThe end-product that is produced must be of sufficient quality

► Cost, time and other constraintsCost, time and other constraints Finite resourcesFinite resources The benefit must outweigh the costThe benefit must outweigh the cost Others are competing to do the job cheaper and fasterOthers are competing to do the job cheaper and faster Inaccurate estimates of cost and time have caused many project Inaccurate estimates of cost and time have caused many project

failuresfailures

30 6

Comments:Comments:► $250 billion annually in US.$250 billion annually in US.► Over 175,000 projects!Over 175,000 projects!► Complexity, size, distribution, importance push Complexity, size, distribution, importance push

our limits.our limits.► Business pushes these limits:Business pushes these limits:

Great demands for Great demands for rapid development and deploymentrapid development and deployment► Incredible pressure: develop systems that are:Incredible pressure: develop systems that are:

On time, On time, Within budget, Within budget, Meets the users’ requirementsMeets the users’ requirements

► Figures in the late 90s indicated that at mostFigures in the late 90s indicated that at most 70% of projects completed70% of projects completed Over 50% ran over twice the intended budgetOver 50% ran over twice the intended budget $81 billion dollars spent in cancelled projects!!$81 billion dollars spent in cancelled projects!!

► Getting better, but we need better tools and Getting better, but we need better tools and techniques! techniques!

30 7

What Happens in PracticeWhat Happens in PracticeSequential activities: (Traditional ‘Waterfall’ Process)Requirements Design Code Integration Test

Late DesignBreakage

100%

Project Schedule

Deve

lopm

ent P

rogr

ess

(% c

oded

)

OriginalTarget Date

IntegrationBegins

Risk inadequately addressedProcess not receptive to ChangeProblems not really ‘seen’ until near delivery date!Until then, ‘all is well…’Big Bang approach – full deliveryLong development cycles…Little user involvement, etc. etc…

30 8

Symptoms of Software Symptoms of Software Development ProblemsDevelopment Problems

► Inaccurate understandingInaccurate understanding of end-user needs of end-user needs► Inability to deal with Inability to deal with changing requirementschanging requirements► Modules that don’t fit together (integration)Modules that don’t fit together (integration)► Software that’s hard to maintain or extend (brittle)Software that’s hard to maintain or extend (brittle)► Late discovery of Late discovery of serious project flawsserious project flaws (integration) (integration)► Poor software qualityPoor software quality (architecture, risks (architecture, risks

unanticipated…)unanticipated…)► Process Process not responsive to Change (Gantt Charts…)not responsive to Change (Gantt Charts…)► Unacceptable software performance Unacceptable software performance ► Team members in each other’s way, unable to Team members in each other’s way, unable to

reconstruct who changed what, when, where, why reconstruct who changed what, when, where, why (software architecture, …(software architecture, …

► ……and we could go on and on…and we could go on and on…

30 9

► We need a We need a processprocess that that Will serve as a framework for large scale and small Will serve as a framework for large scale and small

projectsprojects Adaptive – embraces ‘change!’Adaptive – embraces ‘change!’

► Opportunity for Opportunity for improvementimprovement not identification of not identification of failurefailure!! Iterative (small, incremental ‘deliverables’)Iterative (small, incremental ‘deliverables’) Risk-driven (identify / resolve risks up front)Risk-driven (identify / resolve risks up front) Flexible, customizable process (not a burden; Flexible, customizable process (not a burden;

adaptive to projects)adaptive to projects) Architecture-centric (breaks components into Architecture-centric (breaks components into

‘layers’ or ‘layers’ or common areas of responsibility…) common areas of responsibility…) HeavyHeavy user involvement user involvement

► Identify best ways of doing things – a better Identify best ways of doing things – a better process – acknowledged by world leaders…process – acknowledged by world leaders…

Need a Better Hammer!

30 10

Develop IterativelyDevelop Iteratively

Control ChangesControl Changes

Use Use ComponentComponent

ArchitecturesArchitecturesManage Manage

RequirementsRequirementsModel Model

VisuallyVisually VerifyVerify

QualityQuality

Best Practices of Software Best Practices of Software EngineeringEngineering

Know these!

30 11

Symptomsend-user needschanging requirementsmodules don’t fithard to maintainlate discoverypoor qualitypoor performancecolliding developers build-and-release

Root Causesinsufficient requirementsambiguous communicationsbrittle architectures overwhelming complexityundetected inconsistencies poor testingsubjective assessmentwaterfall development uncontrolled changeinsufficient automation

Best Practicesdevelop iteratively manage requirementsuse component architecturesmodel the software visuallyverify qualitycontrol changes

Addressing Root Causes Addressing Root Causes Eliminates the SymptomsEliminates the Symptoms

Symptoms of problems can be traced to having Root Causes.Best Practices are ‘practices’ designed to address the root causes of software problems.

30 12

Practice 1: Develop Software Practice 1: Develop Software Iteratively Iteratively

Develop IterativelyDevelop Iteratively

Control Changes

Use Component

ArchitecturesManage

RequirementsModel

Visually Verify

Quality

Considered by many practitioners to be the most significant of the six

30 13

Practice 1: Develop Software Practice 1: Develop Software IterativelyIteratively

►Until recently, developed under assumption - Until recently, developed under assumption - most requirements can be identified up front.most requirements can be identified up front.

►The research deconstructing this myth The research deconstructing this myth includes work by Capers Jones. (See next includes work by Capers Jones. (See next slide) In this very large study of 6,700 slide) In this very large study of 6,700 projects, projects, creepingcreeping requirements requirements — those not — those not anticipated near the start—are a anticipated near the start—are a very very significant fact of software development lifesignificant fact of software development life, , ranging from around 25% on average projects ranging from around 25% on average projects up to 50% on larger ones.up to 50% on larger ones.

30 14

Look up a definition of ‘Function Points.’

30 15

Interestingly, Interestingly, ► An initial design will An initial design will likely be flawedlikely be flawed with respect to its with respect to its

key requirements. Requirements rarely key requirements. Requirements rarely fully knownfully known up up front!front!

► Late-phase discovery of design defectsLate-phase discovery of design defects results in costly results in costly over-runs and/or project cancellation over-runs and/or project cancellation Oftentimes requirements change – even during implementation! Oftentimes requirements change – even during implementation!

► While large projects are more prone to cost overruns, While large projects are more prone to cost overruns, medium-size/small projects are vulnerable to cancellation. medium-size/small projects are vulnerable to cancellation.

► The key reasons continue to be The key reasons continue to be poor project planning and management, poor project planning and management, shortage of technical and project management expertise, shortage of technical and project management expertise, lack of technology infrastructure, lack of technology infrastructure, disinterested senior management, and disinterested senior management, and inappropriate project teams.”inappropriate project teams.”

30 16

Waterfall Delays RisksWaterfall Delays Risks

R

I

S

K

T I M E

IntegrationSystem

Test

Code

Design

Requirements

Waterfall risk

Walker Royce, 1995

30 17

Iterative DevelopmentIterative Development

Earliest iterations address greatest risks • Each iteration produces an executable release• Each iteration includes integration, test, and assessment!• Objective Milestones: short-term focus; short term successes!

Iteration 1 Iteration 2 Iteration 3

30 18

Accelerate Risk ReductionAccelerate Risk Reduction

Iterative

T I M E

Iteration Iteration Iteration Iteration Iteration

Risk reduction

R

I

S

KWaterfall risk

Walker Royce, 1995

30 19

Iterative Development CharacteristicsIterative Development Characteristics► Critical risks are resolved before Critical risks are resolved before

making large investmentsmaking large investments ► Initial iterations enable early user Initial iterations enable early user

feedbackfeedback Easy to resolve problems early. Easy to resolve problems early. Encourages user feedback in meaningful waysEncourages user feedback in meaningful ways

► Testing and integration are continuousTesting and integration are continuous – assures successful integration (parts all – assures successful integration (parts all fit) fit) Continuous testing.Continuous testing.

► Objective milestones provide short-term Objective milestones provide short-term focusfocus

► Progress measured by assessing Progress measured by assessing implementationsimplementations

► Partial implementations can be Partial implementations can be deployeddeployed Waterfall method – no deliveryWaterfall method – no delivery Incremental development? May be some Incremental development? May be some

great values in delivering key parts of great values in delivering key parts of application. Critical components delivered application. Critical components delivered first?first?

► No big-bang approach!No big-bang approach!

30 20

UP Lifecycle Graph – Showing UP Lifecycle Graph – Showing IterationsIterations

In an In an iteration, , you may walk you may walk through all through all disciplinesdisciplines

CONTENT

STRUCTURE

T I M E

STUDY THIS!!!

30 21

Executable Releases

Unified Process Iterations and Unified Process Iterations and PhasesPhases

An iteration is a distinct sequence of activitieswith an established plan and evaluation criteria,

resulting in an ‘executable release.’

(There is a lot of very important ‘key’ terminology used here…(cycle, iteration, phase, milestones, core disciplines, content of iterations, etc….)

PreliminaryIteration

Architect.Iteration

Architect.Iteration

Devel. Iteration

Devel. Iteration

Devel. Iteration

TransitionIteration

TransitionIteration

Elaboration Construction TransitionInception

30 22

Enables and encouragesEnables and encourages user user feedbackfeedbackSerious Serious misunderstandingsmisunderstandings evident early in the life cycleevident early in the life cycle

Development focuses on Development focuses on critical issues – break it critical issues – break it down!down!Objective assessment thru Objective assessment thru testing and assessment testing and assessment

Inconsistencies detected Inconsistencies detected earlyearly

Testing starts earlier – Testing starts earlier – continuous!continuous!

Risks identified and Risks identified and addressed early - via addressed early - via plannedplanned iterations! iterations!

Problems Addressed by Iterative Problems Addressed by Iterative DevelopmentDevelopment

Root CausesRoot Causes SolutionsSolutions Insufficient Insufficient

requirementsrequirements Ambiguous Ambiguous

communicationscommunications Brittle architecturesBrittle architectures Overwhelming Overwhelming

complexitycomplexity Subjective assessmentSubjective assessment Undetected Undetected

inconsistenciesinconsistencies Poor testingPoor testing Waterfall developmentWaterfall development Uncontrolled changeUncontrolled change Insufficient automationInsufficient automation

30 23

No Free LunchNo Free Lunch - Traps - Traps Abound…Abound…

► Major impacts on Project Managers, though….Major impacts on Project Managers, though….► Trap: When the initial risks are mitigated, new ones emergeTrap: When the initial risks are mitigated, new ones emerge Do not do just the easy stuff, to look good.Do not do just the easy stuff, to look good. Keep re-planning based on all new information.Keep re-planning based on all new information.► Trap: Remember ‘some’ Trap: Remember ‘some’ ReworkRework enables you to enhance your enables you to enhance your

solutionsolution Accommodate change Accommodate change earlyearly in the project in the project► Trap: Iterative development Trap: Iterative development does does notnot mean mean never to commit to a never to commit to a

solutionsolution

► Monitor ‘scrap and rework’Monitor ‘scrap and rework’► Trap: Must Control “requirement creep, ” however… SomeTrap: Must Control “requirement creep, ” however… Some clients will now naturally recognize many ‘musts…’clients will now naturally recognize many ‘musts…’

30 24

Many Traps in Iterative Many Traps in Iterative DevelopmentDevelopment

Here is another trap: Too long initial iterationHere is another trap: Too long initial iteration

► Winning is fun. Winning teams work better than Winning is fun. Winning teams work better than loosing teamsloosing teams

► BetterBetter to have a short initial iteration, than one too to have a short initial iteration, than one too longlong Cut scope if necessary (much more later)Cut scope if necessary (much more later)

► Avoid ‘analysis-paralysis’ by Avoid ‘analysis-paralysis’ by time-boxing;time-boxing; you can you can enhance in later iterations (more later)enhance in later iterations (more later)

► Establish an Establish an even rhythmeven rhythm for project (at least w/i a for project (at least w/i a phase)phase)

► Focus on Focus on resultsresults and and deliverablesdeliverables, not activities, not activities

30 25

Iterations Are Time-boxedIterations Are Time-boxed►Work is undertaken within an Work is undertaken within an iterationiteration..►The iteration plan The iteration plan definesdefines the the artifactsartifacts to be to be delivered, delivered, rolesroles and and activitiesactivities..

►An iteration is clearly An iteration is clearly measurablemeasurable..►Iterations are Iterations are risk-drivenrisk-driven►Iterations are Iterations are plannedplanned..►Iterations are Iterations are assessedassessed!!►Generally, Generally, initialinitial iterations (in Construction) iterations (in Construction) based on based on high risk and core functionalities!high risk and core functionalities!

30 26

The Iteration Plan Defines….The Iteration Plan Defines….TheThe deliverablesdeliverables for for that iteration.that iteration.

The The to do listto do list for the for the team membersteam members

artifacts

30 27

Problem: Problem: FixedFixed Plans Produced Upfront – Plans Produced Upfront –

Not Real Practical!Not Real Practical!►Yet, senior management wants firm, fixed plans!Yet, senior management wants firm, fixed plans!

Part of their culture / upbringing/ experiencePart of their culture / upbringing/ experience Necessary for ‘planning’ budgeting, etc. of resources, projects…. BUT:Necessary for ‘planning’ budgeting, etc. of resources, projects…. BUT:

►TrapTrap: Fine-grained planning : Fine-grained planning from start to end?from start to end? Takes too much timeTakes too much time Frustrating as change occurs (and it Frustrating as change occurs (and it will)will), if plans too fine-grained., if plans too fine-grained.

►Know that: Know that: Projects typically have some degree of Projects typically have some degree of uncertaintyuncertainty

►This makes This makes detaileddetailed plans for the plans for the entireentire project meaningless project meaningless

►Does not mean that we should not planDoes not mean that we should not plan

30 28

Solution: Solution: Plan With Evolving Levels of Plan With Evolving Levels of

DetailDetail

Current Iteration

Next Iteration

Phases and major milestones What and whenIterations for each phase Number of iterations Objectives and Duration

One For Entire Project

Fine-grained Plans: Iteration Plans

Coarse-grained Plan: Software Development Plan

Iterative Development does not mean less work and shorter schedule It is about greater predictability

30 29

Progress is made against Progress is made against MILESTONESMILESTONES

►In the Unified Process:In the Unified Process: Each phase is defined by a Each phase is defined by a milestonemilestone.. Progress is made by passing Progress is made by passing milestonesmilestones.. Milestones measure Milestones measure successsuccess

►PhasesPhases - NOT TIMEBOXED. - NOT TIMEBOXED.►IterationsIterations ARE TIMEBOXED. ARE TIMEBOXED.

Inception Elaboration Construction Transition

Major Milestones

30 30

SummarySummary►Much more about iteration and Much more about iteration and

iteration planning later in the course…iteration planning later in the course…

►You will see some of these again – You will see some of these again – and, more importantly, and, more importantly, useuse this this information in your own iteration information in your own iteration planning.planning.