Project Monitoring and Control with EVA and Burn Graphs

Guy Davis

Kendra Hamilton

Ed Dantsiguer

Agenda

Origins of EVA EVA Explained EVA Examples Shortcomings of EVA Agile side of EVA Burn Graphs Tools Discussion

Overview of EVA

“The essence of [Earned Value Management Systems] is that some level of detail appropriate for the degree of technical, schedule, and cost risk or uncertainty associated with the program, a target value ….is established.” Paul Solomon 2002.

History of EVA

PERT/Cost (1963) DoD Cost/Schedule Control Systems Criteria

(C/SCSC) (1967). Government Performance and Results Act (1993),

Federal Acquisition Streamlining Act, Title V (1994), Clinger-Cohen Act (1996)

EIA-748-1998 “Earned Value Management System” Office of Management and Budget (Circular A-11, Part

7) (2003)

Motivation for EVA

Integrates work, cost, and schedule metrics. Early warning signal. Driving by looking in the front windshield

instead of the rear view mirror. Statistical projection.

U.S. Government requirement.

EVA Explained 1/3

EVA concentrates on project management and control Requires a number of tasks to be

performed before utilizing EVA: Work Breakdown Structure

(WBS) Creation of detailed plan (critical

path plan) Includes all activities that have to

be performed, their durations, costs and relative contributions to the overall deliverable

EarnedValue

Cost

Schedule Technical Performance

EVA Explained 2/3

EVA Explained 3/3

EVA is used to determine current project performance and estimate/forecast future project performance

Based on 3 data points: Budgeted Cost of Work Performed (BCWP) Actual Cost of Work Performed (ACWP) Budgeted Cost of Work Scheduled (BCWS)

Budgeted Cost of Work Performed

Planned (budgeted/estimated) cost of work that has been completed until this point

Answers: “How much was performed work supposed to cost?” Based on features/activities completed and

the budgeted amount for these features/activities in the original project plan

Actual Cost of Work Performed

Actual cost of work that has been completed until this point

Answers: “What was the actual cost of work actually performed?” Based on features/activities completed and

cost of these features/activities in real life

Budgeted Cost of Work Scheduled

Planned (budgeted/estimated) cost of work that was supposed to be completed

Answers: “How much work should have been done and how much was it meant to cost?” Based on features/activities planned/scheduled and

the budgeted amount for these features/activities in the original project plan

Budget at Completion (BAC) is the total funds allocated (budgeted) for this project to complete

Derived Metrics

Schedule Variance (SV) SV = BCWP – BCWS Compares what is done with what was supposed to be

done SV < 0 project is behind schedule

Cost Variance (CV) CV = BCWP – ACWP Compares actual project cost with budgeted project

costs CV < 0 project is over budget

Schedule/Cost Performance Index

Schedule Performance Index (SPI) SPI = BCWP/BCWS SPI < 1 project is behind schedule

Cost Performance Index (CPI) CPI = BCWP/ACWP CPI < 1 project is over budget

Cost Schedule Index (CSI) CSI = CPI * SPI CSI < 1 project is not tracking to plan

The further away CSI is from 1, the less likely is successful project recovery

Using EVA Metrics in Project Control 1/2 Each individual EVA metric is not greatly

useful on its own Metrics need to be considered as a group Ex: Just because a project has a CSI of 1

does not imply that the project is doing well – it may be well ahead of schedule while also being well ahead of its budget

Using EVA Metrics in Project Control 2/2 ACWP metric can be used to project future activity

costs/durations This is called Estimate To Completion (ETC) The end of the projected ETC curve is

the Estimate At Completion (EAC) estimated schedule and cost required to complete the

project based on current productivity and spending Comparison between EAC and BAC shows is the

project is likely to be on schedule and/or on budget Variance at Completion (VAC) schedule difference

between BAC and EAC

Recommended Performance Metric Values

Performance Metric

Green

Yellow

Red

SPI 0.95 and greater

0.90 and greater

Less than 0.90

CPI 0.95 and greater

0.90 and greater

Less than 0.90

VAC 0.05 and lesser

0.10 and lesser

Greater than 0.10

Per the “U.S. Marine Corps Acquisition Procedures Handbook,” June 1997

EVA Task Types

Discrete Effort Activities with start and end time that result in

deliverables Apportioned Effort

Effort required to support discrete effort tasks (Ex: inspections, quality control)

Proportional to the type/size of discrete effort tasks that they support

Level of Effort Overhead activities with no concrete deliverables (Ex:

management and administrative activities)

Crediting Earned Value

Discrete Effort Credited upon completion with actual cost and duration

tracked Apportioned Effort

Credited upon completion of related discrete effort tasks

Level of Effort Credited according to plan (regardless of actual cost

and duration)

Crediting Earned Value Methods

Milestone Events Weighted Milestone Gates Percentage Complete Fixed Formula Level of Effort Percentage Complete and Milestone Gates

EVA Example 1/2

Planned/Scheduled Data: Duration of 10 months Includes 10 features with multiple tasks Budget of $100 million

Actual After 6 Weeks: 55% of the work has been completed $85 million has been spent

EVA Example 2/2Earned Value Analysis

0

20

40

60

80

100

120

140

160

180

0 2 4 6 8 10 12

Time, Working Weeks

Cum

ulat

ive

Spen

ding

BCWS

BCWP

ACWP

ETC

Date of progress measurement

Schedule variance Cost Variance

Time variance

BAC

EAC

VAC

Cost Overrun

Success Factors for EVA

Quality of the baseline; need to include all details.

Take action early based on performance indicators.

“I hate everything that merely instructs me without augmenting or directly invigorating my activity” Goethe

Shortcomings of EVA

Based on past performance; assumes constant rate of spend and value creation

Assumes a direct relationship between time and cost. Value measured in technical components, not

expected business value. Project must be fully defined at outset; the devil is in

the details. Time required for measuring project’s progress

EVA Tools

Excel Welcom “Cobra” http://www.welcom.com/ Schedulemaker http://www.schedulemaker.com Planisware “OPX2” http://www.planisware.com/ RiskTrak http://www.risktrak.com/index.htm Winsight http://www.cs-solutions.com/ Primavera Systems http://www.primavera.com/

EVA compared to Agile

Full project view vs. Iterative view Tasks fully defined vs. Changing requirements Attempts to forecast future vs. Determination

of next iteration EVA is not suitable for truly “Agile” projects

EVA in an Agile/Iterative Project

Approach 1: Stories = BCWS Tasks = BCWS in more detail Assignments = ACWP Velocity = BCWP Testable requirements = 0% or 100% BCWP

EVA in an Agile/Iterative Project

Approach 2: Do EVA on individual iterations

Approach 3: Generate micro estimations for current iteration

and macro estimations for future iterations

Burn Graphs

Origins of Burn Graphs Cumulative flow diagrams from lean production

Goal: to provide a succinct view of progress. Allows project sponsors to steer the project. Allows scrum master report visually to

stakeholders. Allows the team to gain experience estimating

by getting direct feedback. (Empowering)

Burn-Down Graphs

Shows remaining estimated effort on item Usually in ideal engineering time (IET hours)

Calculated for any level of task abstraction

Scrum Backlog Graphs Product Backlog Graph

High-level view of overall project progress. Completion date: work left versus resources

available. Quantitative tool for making trade-offs.

Sprint Backlog Graph Detailed view of a single sprint's progress. Sprint signatures: compare current with past

results. First notice of schedule slips seen here.

Not always pretty...

Burn-up Graphs

Shows progress on completion of item Usually displays percentage complete Just the inverse of a burn-down graph

Cumulative Flow Diagrams

Tracks number of features (and status) over time Better for reporting than:

% complete graph of feature milestone percentages Features completed over time

Overview of Burn Graphs

Benefits Easy to compile/track Feedback to the team

and status to the customer

Highlights: Schedule slips Scope creep

Drawbacks Metric choice is key Over-simplification?

Hides dependencies Should be able to

zoom to see levels Traditionalists will resist

implementation?

Burn Graph Tools Open-source

Outreach Project Tool (OPT) XPlanner

Commercial Version One MS Excel

References Anderson, David J. “Using Cumulative Flow Diagrams with FDD”. Feature Driven Development. 2003.

http://www.featuredrivendevelopment.com/node/view/515 Anderson, David J. Agile Management for Software Engineering. Prentice Hall. 2003 Alleman, Glen B., Henderson, Michael, “Making Agile Development Work in a Government Contracting

Environment” Proceedings of the Agile Development Conference, IEEE, 2003. Fleming, Quentin W., Koppelman, Joel M., “Earned Value Project Management: A Powerful Tool for Software

Projects”, Crosstalk, July 1998. Hayes, Heather, “Using Earned-Value Analysis to Better Manage Projects”, Pharmaceutical Technology,

February 2002. Howes, Rodney “Improving the Performance of Earned Value Analysis as a Construction Project Management

Tool”, Engineering, Construction and Architectural Management, 2000. Schwaber, Ken and Mike Beedle. Agile Software Development with Scrum. Prentice Hall. Upper Saddle River,

NJ. 2002 Solomon, Paul J., “Practical Software Measurement, Performance-Based Earned Value,” Crosstalk,

September, 2002.

Discussion Points

Can EVA be applied to agile projects?

What metrics would you use for burn graphs? Would you adopt burn graphs at your

organization?