Smart Metrics

Agile On The Beach September 2011 � Smart Metrics

1Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved

September 15th, 2011

Agile On The Beach

Mike Griffiths, PMPLeading Answers Inc.

Smart Agile Metrics

www.LeadingAnswers.com

Presenter Background� Project Manager and Trainer

� >25 years IT experience on utilities, defense, & finance� 10 years PMO Agile-to-Traditional Integration

� Agile Project Management� Helped create Agile method DSDM in 1994� 17 years agile project experience (DSDM, XP, Scrum, FDD)� Board director of Agile Alliance and APLN� Author, trainer, and presenter Agile Conference 2001-9

� Traditional Project Management� PMP, PRINCE2 certifications� PMBOK v3 and v5 contributor and reviewer� Trainer for PMI SeminarsWorld 2005-2011� Presenter PMI Global Congress 2004-2011� Steering Committee PMI-ACP Exam



� 1 <Star>� 2� 3� 4� 5

Agenda

4) Measuring Up

3) �Design Factory� Metrics

2) The Hawthorne Effect

1) Measurement Troubles

6) Smart Metric examples

5) Leading vs. Lagging Metrics

Metrics are like fire�



MeasurementNot all observations are useful: � The sun rises up in the sky in the morning and then falls down again at night� Planets revolve around the earth� Stars come out at night� Heavier objects fall faster than lighter objects

�There are so many possible measures in a software process that some random selection of metrics will not likely turn up something of value� � Watts Humphrey

Not all useful things are easily observable: � Spouse�s mood � Sponsor Confidence� Team Commitment

"Not everything that can be counted counts, and not everything that counts can be counted� � Albert Einstein.

The Hawthorne Effect

You will influence what you measure...



Metrics Should be:1. Simple, self-generating2. Relevant to the end-goal3. Ideally, leading, future-focused

Design Factory Metrics

Traditional Metrics ExaminedDesirable Characteristics:ü The Hawthorne Effect is positiveü Simple, self generatingü Relevant to the end-goalü Leading, future-focused

Traditional Metrics:� Lines of Code Written � poor, does not reward

simplification, leads to code bloat

� Function Points Delivered � poor, effort to generate, not relevant to the end-goal of project

� Hours Worked � poor, leads to long hours, burn-out, defects, consumed budgets



Smarter Project Metrics

Since you will influence what you measure�

Choose Smarter Metrics:� Features Accepted� Sponsor Confidence� User Satisfaction� Defect Cycle Times

Smarter Metrics� Features Accepted

Not StartedIn ProgressAccepted

Scope (points) IPS Project Progress Spend

0

500

1000

1500

2000

2500

3000

1/10/0

6

1/12/0

6

1/2/07

1/4/07

1/6/07

1/8/07

1/10/0

7

1/12/0

71/2

/081/4

/081/6

/081/8

/08

1/10/0

8

1/12/0

8

1/2/09

1/4/09

1/6/09

1/8/09

1/10/0

9

1/12/0

9

Scope BuiltAccepted



Smarter Metrics� Features Accepted

050

100150200250300350400450

Jan Feb Mar Apr May Jun Jul Aug

Time

Feat

ures

Not StartedIn ProgressCompleted

Not StartedIn ProgressAccepted

Feature Based Reporting Example 2: Parking lot Diagrams

Feature Set Status: Enter Order Details

(15)

24%Percentage Complete:

Not StartedWork in ProgressCompletedAttention (i.e. behind)

Percentage Complete Bar

Sep 2011Target Completion Month:Not Complete Complete

FB Chief Programmers Initials

Feature Set Name

Number of Features in the Feature Set

The Feature Set called Enter Order Details has 15 features and is currently 24% complete. It is due to be completed by September 2011



Create New Order

(5)

100%

CaptureCustomer

Details(9)

75%

Enter OrderDetails

(15)

24%

ProcessPayment

(11)

Create NewCustomer

(8)

55%

AmendCustomer

Details(6)

20%

ArchiveCustomer

(4)

StockSearch

(6)

95%

ItemDetails

(12)

75%

Order Processing

Customer Management

Inventory Management

Jul 2011

Sep 2011

Sep 2011

Oct 2011

Aug 2011

Sep 2011

Sep 2011Oct 2011

Nov 2011

CM DH LF RS NC

AW

KB

SW SW

Feature Based Reporting

Smarter MetricsUser Satisfaction



Smarter Metrics� Sponsor Confidence

Smarter Metrics� Defect Cycle Time



� �You get what you measure�

� �You get only what you measure, nothing else�

� �You tend to loose the things that you can�t measure: insight, collaboration, creativity�

� Nucor Steel - From startup in 1970 to North America�s largest and most profitable steel company

� Incentive pay based on productivity

� Measure one level above where expected / directly controllable

Measuring Up

� Defects could be traced back to individual developers, but may be environmental

� Rolling up to an entire team and get testers involved earlier to provide more timely and valuable feedback to developers

�Instead of making sure that people are measured within their span of control. It is more effective to measure people one level above their span of control. This is the best way to encourage teamwork, collaboration, and global, rather than local optimization� � Mary Poppendieck

Measuring Up in Software



� For an accountant, a perfect view of the past might be useful

� For a project manager, a perfect or even an imperfect view of the future is far more useful

Leading vs. Lagging Metrics

Pay less attention to Lagging Metrics� Actual values

Pay more attention to Leading Metrics� Trends� Impacts of projections

Trends

CR = ChangeRequests

Clarif. = Clarifications

Observations Mar 3 Mar 10 Mar 17 Mar 24 Mar 31Defects Opened 5 25 30 20 10Defects Closed 1 16 35 22 15Defects Remaining 4 13 8 6 1CR Opened 0 18 20 23 12CR Closed 0 11 21 16 9CR Remaining 0 7 6 13 16Clarif. Opened 9 12 14 8 2Clarif. Closed 6 14 11 9 5Clarif. Remaining 3 1 4 3 0Total Observations 7 21 18 22 17

Observation Trends

02468

1012141618

Mar 3 Mar 10 Mar 17 Mar 24 Mar 31

Defects RemainingCR RemainingClarif. Remaining



Trends - Risks ReducedJan Feb Mar Apr

ID Risk Short Name Imp Prob Sev Imp Prob Sev Imp Prob Sev Imp Prob Sev1 JDBC driver performance 3 2 6 3 0 0 3 0 0 3 0 02 Calling Oracle Stored Procs via web service 2 2 4 2 0 0 2 0 0 2 0 03 Remote app distribution to PDA's 3 2 6 3 1 3 3 0 0 3 0 04 Oracle Warehouse Builder stability 2 2 4 2 3 6 2 2 4 2 0 05 Source system availability 2 1 2 2 1 2 2 0 0 2 0 06 Access to user community 2 1 2 2 2 4 2 2 4 2 1 27 Availability of Architect 2 2 4 2 3 6 2 2 4 2 0 08 Server Upgrade necessary 1 2 2 1 1 1 1 0 0 1 0 09 Oracle Handheld Warehouse Browser Launch 3 1 3 3 1 3 3 3 9 3 1 310 PST changes for BC 0 0 0 0 0 0 2 2 4 2 1 2

33 25 25 7

Project Risk Profile

0

5

10

15

20

25

30

35

Jan Feb Mar Apr

PST changes for BCOracle Handheld Warehouse Browser LaunchServer Upgrade necessaryAvailability of ArchitectAccess to user communitySource system availabilityOracle Warehouse Builder stability Remote app distribution to PDA'sCalling Oracle Stored Procs via web serviceJDBC driver performance

Analyst UI Designer Code & UT Sys. Test

45 30 40 35

� Identifying constraints� Buffering� Managing flow

Cycle Times



Cycle Times

Construction Cumulative Flow

020406080

100120140

20/02

/2004

27/02

/2004

05/03

/2004

12/03

/2004

19/03

/2004

26/03

/2004

02/04

/2004

09/04

/2004

16/04

/2004

23/04

/2004

30/04

/2004

07/05

/2004

14/05

/2004

21/05

/2004

28/05

/2004

Total FeaturesAnalysisUI DesignCode & UTSys Test

} Widening Area

Bottleneck Activity = �UI Design� (activity below widening area)

Little�s Law

Little�s Law:Cycle times are proportional to queue lengths.

(We can predict completion times based on queue size)



Task Boards & Kanban

Summary

1. Use metrics wisely

2. Leverage the Hawthorne Effect

3. Measure Up

4. Favour Leading Metrics

More info and slides: www.LeadingAnswers.comEmail: [email protected]

Smart Metrics

Technology

Transcript of Smart Metrics