Overcoming Project Estimation Madness

Joel SemeniukCEO, Imaginet ResourcesMicrosoft Regional DirectorMicrosoft MVP – Team SystemDPR205

What Is an Estimate?

Estimate <> Target or Commitment

Estimate

• Prediction

Target

• Desire

Commitment

• Promise

Estimate <> Planning

Estimation• Analytical Unbiased

Process• Goal = Accuracy

Planning• Biased Goal-Seeking

Process• Goal = Seek Result

Plans Need Good Estimates

Estimation

Scheduling

Identifying Critical Path

Breaking Down WorkPrioritization

Defining Iterations

Common ScenarioHow long do you think this project

should take? We need to have it ready in 3 months with these

features….

My estimates say we can do this in 5 months

5 Months? Didn’t you hear me? I need to have this in 3 months for

the trade show!

The manager wasn’t asking for an estimate but a plan to solve a problem…

Tip: When asked for an estimate – determine if it should be an estimate or a plan to hit a target

Single Point Estimates

Rarely is a single point estimate an estimateEstimates should include some level of probability

Tip: When you see a single point estimate – question to see if this is really a target

Tip: When you see a single point estimate – you should question the probability

Tip: Don’t use a planning tool to make estimates (e.g., MS Project)

Using MS Project and Single Point Estimates

ID Task Name Work Baseline Variance Actual Remaining % W. Comp.

0 New Product 14,272 hrs 0 hrs 14,272 hrs 0 hrs 14,272 hrs 0%1 New Product Development Template14,272 hrs 0 hrs 14,272 hrs 0 hrs 14,272 hrs 0%2 Initial New Product Screening Stage72 hrs 0 hrs 72 hrs 0 hrs 72 hrs 0%3 New product opportunity identified0 hrs 0 hrs 0 hrs 0 hrs 0 hrs 0%4 Describe new product idea (1-page written disclosure)16 hrs 0 hrs 16 hrs 0 hrs 16 hrs 0%5 Gather information required for go/no-go decision48 hrs 0 hrs 48 hrs 0 hrs 48 hrs 0%6 Convene opportunity of screening committee (decision to pursue idea or not)8 hrs 0 hrs 8 hrs 0 hrs 8 hrs 0%7 Decision point - go/no-go to preliminary investigation0 hrs 0 hrs 0 hrs 0 hrs 0 hrs 0%

Really?

Estimation ChallengesProvide a 90% Confident Estimate for the following:

How many people are at this conference? What is the size of the largest whale ever discovered? How many bugs will you have in your next software project? What is the distance between Mars and the Sun? What is the surface temperature of the sun? What is the total volume of the Great Lakes?

Question: What makes you think you are 90% confident?Question: Did you use an artificially narrow range?Question: Where does the pressure to use a narrow range come from?

Two Important Laws of Nature

Parkinson’s LawIf you give a person 5 days to deliver a task that could be delivered in 4, the person will find something to do with that extra day

The Student SyndromeIf given too much time, people procrastinate until late in the project, at which point they rush to complete the work, and likely won’t finish on time

These are two common arguments against Overestimation

Uncertainty and Software Estimation

Not all requirements are known until development time in most cases

Narrowing the Cone

Tip: The cone does not narrow itself; it narrows when you continually make decisions that narrow variability

Sources of Error1. Chaotic Development Process2. Unstable Requirements3. Omitted Activities4. Unfounded Optimism5. Subjectivity6. Off-the-Cuff Estimates

Things that Influence Estimates

Project SizeType of Software Being DevelopedPersonnel FactorsProgramming Languagesand LOTS of others such as:

• Complexity• Constraints• Turnover• Experience• Tools• Ego

• Team Cohesion• Process maturity• Architecture Risk• Technology Maturity• I’m sure you can name 10 more…

Estimation PracticesCount, Compute, Judge

Calibration and Historical Data

Expert Judgment

Decomposition and Recomposition

Estimation by Analogy

Proxy-Based Estimates

Expert Judgment in Groups

Use of Multiple Approaches

Count, Compute, Judge

Count, Compute, Judge

If you can count something – count it Use “expert” judgment when there’s nothing to countWhat to count?

Something correlated with size of softwareSomething we can count sooner rather than laterSomething that will produce a statistically meaningful averageSomething that you understandSomething you can count with minimal effort

Count, Compute, Judge

Examples of things you can count…

List of things More things

Requirements Web Pages

Features Bugs

Use cases Reports

Stories Dialog boxes

Function Points Database Tables

Change Requests Classes

Tests Cucumbers

Calibration and Historical Data

Helps to transform Count into EstimateBest way to predict today’s weather is to look at yesterday’sWays to calibrate:

Industry DataHistorical DataProject Data

Industry Data

Historic Project Data Example

Team Size Average Stories Delivered per Sprint

1 5

2–3 12

4–5 22

6–7 31

8 No Data

Tip: Using Time and “Actual Effort” as historical data is difficult

Current Project Data

Use Data from the Current Project to Predict the Rest of the Project

Data for Sprint 1

27 story points (sp) delivered

12 staff weeks (sw)

3 calendar weeks (cw)

Preliminary Calibration

Effort = 27sp / 12sw = 2.25 sp per sw

Schedule = 27sp / 3w = 9 sp per cw

Remainder of Project

Effort = 2.25 sp per staff week (spsw)

Schedule = 9 sp per calendar week (spcw)

Project size = 180 sp

Preliminary Whole-Project Estimate

Effort = 180sp/2.25 spsw = 80 staff weeks

Schedule = 180sp / 9 spcw = 20 calendar weeks

Individual Expert JudgmentTwo Types

Intuitive Expert Judgment (Spidey Senses) – not very accurateStructured Expert Judgment – almost as accurate as model-based estimates

Who Estimates?People who are doing the work – most accurate

GranularityEnd up with estimates that are around ¼–1 day large

Use RangesBest Case, Most Likely Case, Worst Case

Use FormulasOptimistic PERT = [BestCase+(4XMostLikely)+WorstCase]/6Pessimistic PERT = [BestCase+(3xMostLikely)+(2xWorstCase)]/6

Decomposition and RecompositionAKA – Bottom UpThe Law of Large Numbers

Single Large Estimates are usually Largely wrong in a single direction (+/-)Breaking into smaller pieces – some are above, some are lower

The error at the detail level balances out the error on the whole

How Small to Break Things Down?You need 5–10 smaller things to start seeing benefits

Use Common Work Breakdown to Avoid Missing Activities

Category Create/Do Plan Manage Review Rework DefectIntegration X X X X

Documentation X X X X X

Etc X X X

Big Problem with Bottom UP

Aggregating Estimates Doesn’t Work WellUse complex standard deviation formulas to aggregate when you have more than 10 tasksThis can get VERY complicated if you want to be accurateTake a statistics course for more information

Estimation by Analogy

Check for congruent assumptions

Create an effort based on the size of the new project compared to old

Build up estimate of new project as a % of the old project size

Compare size of new project piece by piece

Get details of previous project decomposed by feature area, work breakdown or some other scheme

Proxy-based Estimates

Find a Proxy that correlates to what you are estimating that is easier to estimate/count

E.g., Estimation of # of Test Cases is correlated to # of Requirements

Some use Proxies to estimate lines of codeWhat do lines of code really give you?

Some great techniques Story PointsT-Shirt Sizing

Good for estimating test cases, defects, documentation, etc.

Story Points

Assign a size to each story you are building

Story Point Scale Specific Points to the Scale

Powers of 2 1, 2, 4, 8, 16, 32

Fibonacci 1, 2, 3, 5, 8, 13, 21

Story Points ExampleStory Points

Story 1 2

Story 2 1

Story 3 4

Story 4 8

…

Story 60 2

Total 180

Need to Calibrate an IterationData for Iteration 1

27 Story Points Delivered

12 Staff weeks

3 Calendar Weeks

Calibration:

Effort = 27 story points / 12 staff weeks = 2.25 story points/staff week

Schedule = 27 story points / 3 calendar weeks = 9 story points/calendar week

Projecting Further

Assumptions from Calibration

Effort = 2.25 story points/staff week

Schedule = 9 story points/calendar week

Project size = 180 story points

Preliminary Whole-Project Estimate

Effort = 180 story points / 2.25 story points/staff week = 80 staff weeks

Schedule = 180 story points / 9 story points/calendar week = 20 calendar weeks

T-Shirt Sizing

How do Sales people and Delivery people work together?

Sales people need to know relative size to help determine priorityEstimator won’t estimate until further decomposedEarly in the Cone of Uncertainty

Remember – goal isn’t accuracy, but accurate enough to support control and decisionsCan use T-shirt sizing to help

T-Shirt SizingFeature Business Value Dev Cost

Feature A Large Small

Feature B Small Large

Feature C Large Large

Feature D Medium Medium

Using for PrioritizationBusiness Value Extra Large Large Medium Small

Extra Large 0 4 6 7

Large -4 0 2 3

Medium -6 -2 0 1

Small -7 -3 -1 -

Feature Bus. Value Dev Cost Net Value

Feature A L S 3

Feature B L M 2

Feature C L L 0

Feature D M M 0

Expert Judgment in Groups

Group reviews can be funHave each team estimate pieces individually, compare resultsDon’t average estimatesArrive at consensus

TechniquesWideband Delphi Technique (quite formal)Estimation Poker (quite fun)

Mixing Things Up

E.g., Expert Proxy Judgment in Groups

Quick Demos

Feature Prioritization Worksheet – T-Shirt Sizing (kinda)Feature Point WorksheetMulti-Point WorksheetEstimation Poker

Some Parting Thoughts

Keep Estimation SimpleTime Boxing is your friendDo things that minimize uncertaintyDon’t be afraid to re-estimate and thus, re-planEstimation via consensus is harder than you thinkSometimes you have to commit; I’m sorryBe careful when recording historic data – time-based data is all a lie and lines of code are meaningless when technology changesFind a method that works for you – then continually improve

Reference

JoelS@Imaginets.comquestion & answer

www.microsoft.com/teched

Sessions On-Demand & Community

http://microsoft.com/technet

Resources for IT Professionals

http://microsoft.com/msdn

Resources for Developers

www.microsoft.com/learningMicrosoft Certification and Training Resources

www.microsoft.com/learning

Microsoft Certification & Training Resources

Resources

Complete an evaluation on CommNet and enter to win!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.