30Software Engineering andBest PracticesSources: Various. Rational Software Corporation slides, OOSE textbook slides, Per Kroll talk, How to Fail with the RUP article, textbooksMost slides have been modified considerably

30*

30*Fundamental Terms / ConceptsScience and Engineering

Discover Relationships that exist but are not foundFormulas; chemical composition, d=r*t; calories in fats, carbohydrates, proteins; experimentation;Astrophysics origins of the universeBuildApply principles of science and mathematics to real needs, commodities, structures, products, etc.Software Engineering; Software Development

30*

30*Fundamental Concepts / Terms (2)Software Engineering; Software DevelopmentJob positions:

Software developerProgrammerSoftware engineerAnalyst / ProgrammerSenior what have you

30*

30*What is Software Engineering?The process of solving customers problems by the systematic development and evolution of large, high-quality software systems within cost, time and other constraints

Note:

Process, systematic (not ad hoc), evolutionary Constraints: high quality, cost, time, meets user requirements

30*

30*Analysis of the Definition:Systematic development and evolution

An engineering process involves applying well understood techniques in a organized and disciplined wayMany well-accepted practices have been formally standardizede.g. by the IEEE or ISO Most development work is evolutionary

Large, high quality software systems

Software engineering techniques are needed because large systems cannot be completely understood by one personTeamwork and co-ordination are requiredKey challenge: Dividing up the work and ensuring that the parts of the system work properly togetherThe end-product that is produced must be of sufficient quality

Cost, time and other constraints

Finite resourcesThe benefit must outweigh the costOthers are competing to do the job cheaper and fasterInaccurate estimates of cost and time have caused many project failures

30*

30*Comments:$250 billion annually in US.Over 175,000 projects!Complexity, size, distribution, importance push our limits.Business pushes these limits:

Great demands for rapid development and deployment Incredible pressure: develop systems that are:

On time, Within budget, Meets the users requirementsFigures in the late 90s indicated that at most

70% of projects completedOver 50% ran over twice the intended budget$81 billion dollars spent in cancelled projects!!Getting better, but we need better tools and techniques!

30*

30*What Happens in PracticeSequential activities: (Traditional Waterfall Process)Requirements Design Code Integration Test

100%Project ScheduleDevelopment Progress(% coded)OriginalTarget DateRisk inadequately addressedProcess not receptive to ChangeProblems not really seen until near delivery date!Until then, all is wellBig Bang approach full deliveryLong development cyclesLittle user involvement, etc. etc

30*

30*Symptoms of Software Development ProblemsInaccurate understanding of end-user needsInability to deal with changing requirementsModules that dont fit together (integration)Software thats hard to maintain or extend (brittle)Late discovery of serious project flaws (integration)Poor software quality (architecture, risks unanticipated)Process not responsive to Change (Gantt Charts)Unacceptable software performance Team members in each others way, unable to reconstruct who changed what, when, where, why (software architecture, and we could go on and on

30*These symptoms are readily observable in the real world. Just like in the medical world, it is not sufficient to treat the symptoms. One must address the root causes (as shown on the next page).An example of treating the symptoms is addressing the second bullet by insisting that no changes to requirements be accepted after a certain date. This is unrealistic. Change will happen to the business environment regardless, and the result may be that users reject the system that is built since it has not adapted to their changing needs.Encourage student interaction on this slide. One possibility is to ask students for symptoms of problems they have observed, before you show the slide. You could then compare the students list to this list. Or you could show the list first and ask for additional symptoms that students have observed.

30*We need a process that

Will serve as a framework for large scale and small projects Adaptive embraces change!Opportunity for improvement not identification of failure!Iterative (small, incremental deliverables)Risk-driven (identify / resolve risks up front)Flexible, customizable process (not a burden; adaptive to projects)Architecture-centric (breaks components into layers or common areas of responsibility)Heavy user involvement

Identify best ways of doing things a better process acknowledged by world leaders

Need a Better Hammer!

30*

30*

Develop Iteratively

Control Changes

Use ComponentArchitectures

Manage Requirements

Model Visually

VerifyQualityBest Practices of Software EngineeringKnow these!

30*

30*Symptomsend-user needschanging requirementsmodules dont fithard to maintainlate discoverypoor qualitypoor performancecolliding developers build-and-releaseRoot Causesinsufficient requirementsambiguous communicationsbrittle architectures overwhelming complexityundetected inconsistencies poor testingsubjective assessmentwaterfall development uncontrolled changeinsufficient automationBest Practicesdevelop iteratively manage requirementsuse component architecturesmodel the software visuallyverify qualitycontrol changes

Addressing Root Causes Eliminates the SymptomsSymptoms of problems can be traced to having Root Causes.Best Practices are practices designed to address the root causes of software problems.

30*

We dont expect students to read this chart. It is an abstract view that shows the many to many relationships among the symptoms and root causes, and between the root causes and practices. It is meant to indicate that we have done the mapping, so they dont have to.Animation notes:This animation is automatic. It shows the tracing from symptoms to root causes to the best practices we will discuss.

30*Practice 1: Develop Software Iteratively

Control Changes

Use ComponentArchitectures

Manage Requirements

Model Visually

VerifyQualityConsidered by many practitioners to be the most significant of the six

30*

We need to keep the concept of iterative development at a very high level in this module. It is easy to get bogged down in the many ramifications of iterative development.Try just to get the concept across here. If they would like to have more information, they can take the Rational Unified Process Overview course. Project managers might be interested in the Unified Software Project Management course.

30*Practice 1: Develop Software IterativelyUntil recently, developed under assumption - most requirements can be identified up front.

The research deconstructing this myth includes work by Capers Jones. (See next slide) In this very large study of 6,700 projects, creeping requirements those not anticipated near the startare a very significant fact of software development life, ranging from around 25% on average projects up to 50% on larger ones.

30*

30* Look up a definition of Function Points.

30*

30*Interestingly, An initial design will likely be flawed with respect to its key requirements. Requirements rarely fully known up front!Late-phase discovery of design defects results in costly over-runs and/or project cancellation

Oftentimes requirements change even during implementation!

While large projects are more prone to cost overruns, medium-size/small projects are vulnerable to cancellation. The key reasons continue to be

poor project planning and management, shortage of technical and project management expertise, lack of technology infrastructure, disinterested senior management, and inappropriate project teams.

30*

30*Waterfall Delays Risks

R

I

S

KT I M EIntegrationSystem TestCodeDesignRequirements Waterfall riskWalker Royce, 1995

30*

30*Iterative Development Earliest iterations address greatest risks Each iteration produces an executable releaseEach iteration includes integration, test, and assessment!Objective Milestones: short-term focus; short term successes!

Iteration 3

30*

30*Accelerate Risk ReductionIterativeT I M E Iteration Iteration Iteration Iteration IterationRisk reductionR

I

S

KWaterfall riskWalker Royce, 1995

30*

30*Iterative Development CharacteristicsCritical risks are resolved before making large investments Initial iterations enable early user feedbackEasy to resolve problems early. Encourages user feedback in meaningful waysTesting and integration are continuous assures successful integration (parts all fit) Continuous testing.Objective milestones provide short-term focusProgress measured by assessing implementationsPartial implementations can be deployedWaterfall method no deliveryIncremental development? May be some great values in delivering key parts of application. Critical components delivered first?No big-bang approach!

30*

30*

UP Lifecycle Graph Showing Iterations

In an iteration, you may walk through all disciplines

CONTENT

STRUCTURET I M ESTUDY THIS!!!

30*

30*Executable ReleasesUnified Process Iterations and PhasesAn iteration is a distinct sequence of activitieswith an established plan and evaluation criteria,resulting in an executable release.

(There is a lot of very important key terminology used here(cycle, iteration, phase, milestones, core disciplines, content of iterations, etc.)

30*

30*Enables and encourages user feedbackSerious misunderstandings evident early in the life cycleDevelopment focuses on critical issues break it down!Objective assessment thru testing and assessment Inconsistencies detected earlyTesting starts earlier continuous!Risks identified and addressed early - via planned iterations!Problems Addressed by Iterative DevelopmentRoot CausesSolutionsInsufficient requirementsAmbiguous communicationsBrittle architecturesOverwhelming complexitySubjective assessment Undetected inconsistenciesPoor testingWaterfall developmentUncontrolled changeInsufficient automation

30*

30*No Free Lunch- Traps AboundMajor impacts on Project Managers, though.

Trap: When the initial risks are mitigated, new ones emerge

Do not do just the easy stuff, to look good. Keep re-planning based on all new information.Trap: Remember some Rework enables you to enhance your solution

Accommodate change early in the projectTrap: Iterative development does not mean never to commit to a solution

Monitor scrap and reworkTrap: Must Control requirement creep, however Some

clients will now naturally recognize many musts

30*

30*Many Traps in Iterative DevelopmentHere is another trap: Too long initial iteration

Winning is fun. Winning teams work better than loosing teams

Better to have a short initial iteration, than one too long

Cut scope if necessary (much more later)

Avoid analysis-paralysis by time-boxing; you can enhance in later iterations (more later)

Establish an even rhythm for project (at least w/i a phase)

Focus on results and deliverables, not activities

30*

30*Iterations Are Time-boxedWork is undertaken within an iteration.The iteration plan defines the artifacts to be delivered, roles and activities.An iteration is clearly measurable.Iterations are risk-drivenIterations are planned.Iterations are assessed!Generally, initial iterations (in Construction) based on high risk and core functionalities!

30*This presentation does not attempt to describe in detail the iterative software development process. Good references on iterative development are contained within the Rational Unified Process itself.Suffice it to say that RUP is an incremental process whereby the overall project is broken down into phases and iterations. The important aspects of iterative development from a plan perspective are:The iteration plan provides a detailed description of the upcoming phase of work. It defines the roles, activities and artifacts delivered in that iteration.It has a very clear set of measurement criteria which can be assessed at the end (and during) the iteration. An iteration should deliver an executable piece of software that is demonstratable and testable against the projects requirements and use cases.An iteration should be time boxed-meaning that it should have a definite duration; Beginning and end.Iterations are the mechanism that the project manager uses to mitigate risk. Therefore, each iteration should reduce the overall risk to the project.

30*The Iteration Plan Defines.The deliverables for that iteration.The to do list for the team members

artifacts

30

*Once you have defined your coarse grained project plan, then an iteration plan is developed at a more detailed level prior to each and every iteration. In short, iteration plans are the roadmap for each iteration. Each phase has a series of milestones associated with it. Each iteration in a phase moves the project towards these milestones. An iteration has a clear objective defined and associated with that objective are the activities and deliverables required to accomplish that objective.

30*Problem: Fixed Plans Produced Upfront Not Real Practical!Yet, senior management wants firm, fixed plans!

Part of their culture / upbringing/ experienceNecessary for planning budgeting, etc. of resources, projects. BUT:

Trap: Fine-grained planning from start to end?

Takes too much timeFrustrating as change occurs (and it will), if plans too fine-grained.

Know that: Projects typically have some degree of uncertainty This makes detailed plans for the entire project meaningless

Does not mean that we should not plan

30*

30*Solution: Plan With Evolving Levels of DetailCurrent IterationNext IterationPhases and major milestones What and whenIterations for each phase Number of iterations Objectives and Duration

One For Entire ProjectFine-grained Plans: Iteration PlansCoarse-grained Plan: Software Development Plan

Iterative Development does not mean less work and shorter scheduleIt is about greater predictability

30*

30*Progress is made against MILESTONESIn the Unified Process:

Each phase is defined by a milestone.Progress is made by passing milestones.Milestones measure success

Phases - NOT TIMEBOXED.Iterations ARE TIMEBOXED.

Inception

30*Secondly, RUP projects measure progress against defined milestones.

Each phase is defined by a milestone; These milestones are defined by the project plan (as opposed to the iteration plan).Progress on you project is made by passing these milestones. Not passing your defined milestones is also a problem.Unlike your iterations, phases are not time boxed; There are not defined constraints on time for your four phases.In the end, milestones measure success! The milestones for a phase provide a focus for the iterations. The milestone helps to plan the objective of the iteration.

For example during the inception phase, one of the phase milestones is the need to understand scope. An iteration within that phase would be structured around the need to understand the scope of the system. The iteration(s) would provide the management framework for the team to explore the system boundary, implications of a possible solution and the size of that solution. The number of iterations would depend on how difficult it will be to define the scope of the project. If the scope was very hard to understand, or it could be grouped into easily defined pieces, then you may need more than one iteration. One iteration for each piece. If, as is normally the case, there is one clear piece of work to understand the scope, then one iteration would be appropriate. Judging the size and number of iterations for a project is described later in this presentation.

30*SummaryMuch more about iteration and iteration planning later in the course

You will see some of these again and, more importantly, use this information in your own iteration planning.

30*

*

*

*

*

*

*

*

*These symptoms are readily observable in the real world. Just like in the medical world, it is not sufficient to treat the symptoms. One must address the root causes (as shown on the next page).An example of treating the symptoms is addressing the second bullet by insisting that no changes to requirements be accepted after a certain date. This is unrealistic. Change will happen to the business environment regardless, and the result may be that users reject the system that is built since it has not adapted to their changing needs.Encourage student interaction on this slide. One possibility is to ask students for symptoms of problems they have observed, before you show the slide. You could then compare the students list to this list. Or you could show the list first and ask for additional symptoms that students have observed.

*

*

*

We dont expect students to read this chart. It is an abstract view that shows the many to many relationships among the symptoms and root causes, and between the root causes and practices. It is meant to indicate that we have done the mapping, so they dont have to.Animation notes:This animation is automatic. It shows the tracing from symptoms to root causes to the best practices we will discuss.*

We need to keep the concept of iterative development at a very high level in this module. It is easy to get bogged down in the many ramifications of iterative development.Try just to get the concept across here. If they would like to have more information, they can take the Rational Unified Process Overview course. Project managers might be interested in the Unified Software Project Management course.*

*

*

*

*

*

*

*

*

*

*

*

*This presentation does not attempt to describe in detail the iterative software development process. Good references on iterative development are contained within the Rational Unified Process itself.Suffice it to say that RUP is an incremental process whereby the overall project is broken down into phases and iterations. The important aspects of iterative development from a plan perspective are:The iteration plan provides a detailed description of the upcoming phase of work. It defines the roles, activities and artifacts delivered in that iteration.It has a very clear set of measurement criteria which can be assessed at the end (and during) the iteration. An iteration should deliver an executable piece of software that is demonstratable and testable against the projects requirements and use cases.An iteration should be time boxed-meaning that it should have a definite duration; Beginning and end.Iterations are the mechanism that the project manager uses to mitigate risk. Therefore, each iteration should reduce the overall risk to the project.

*Once you have defined your coarse grained project plan, then an iteration plan is developed at a more detailed level prior to each and every iteration. In short, iteration plans are the roadmap for each iteration. Each phase has a series of milestones associated with it. Each iteration in a phase moves the project towards these milestones. An iteration has a clear objective defined and associated with that objective are the activities and deliverables required to accomplish that objective. *

*

*Secondly, RUP projects measure progress against defined milestones.

Each phase is defined by a milestone; These milestones are defined by the project plan (as opposed to the iteration plan).Progress on you project is made by passing these milestones. Not passing your defined milestones is also a problem.Unlike your iterations, phases are not time boxed; There are not defined constraints on time for your four phases.In the end, milestones measure success! The milestones for a phase provide a focus for the iterations. The milestone helps to plan the objective of the iteration.

For example during the inception phase, one of the phase milestones is the need to understand scope. An iteration within that phase would be structured around the need to understand the scope of the system. The iteration(s) would provide the management framework for the team to explore the system boundary, implications of a possible solution and the size of that solution. The number of iterations would depend on how difficult it will be to define the scope of the project. If the scope was very hard to understand, or it could be grouped into easily defined pieces, then you may need more than one iteration. One iteration for each piece. If, as is normally the case, there is one clear piece of work to understand the scope, then one iteration would be appropriate. Judging the size and number of iterations for a project is described later in this presentation.

*