"PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007...

"PMI" is a registered trade and service mark of the Project Management Institute, Inc.

© 2007 Permission is granted to PMI for Congress attendee use only

Why use a hammer when you need a wrench: Results-based monitoring and evaluation of projects

Joy Gumz, CPA, CISA, PMP

Project Auditors LLC





Objectives

• Explain differences among– Monitoring– Evaluating– Managing projects

• Clarify what results-based management is• Describe tools and approaches that can be

used for monitoring and evaluating • Provide some tools!



Format

• Case Studies • Terms• Process of Results Based Monitoring• A Sampling of Tools• Q & A

• Please: set your handy to “off” or “vibrate”!



About the session attendees

• Does anyone here perform monitoring and evaluation (M&E) of projects?

• What are your goals on attending this session?

• Is anyone here considering performing M&E in the future?



Terms

• Managing

• Evaluating

• Results-based monitoring



Implications

• Monitoring and evaluating projects is a different skill set

• Results-based monitoring is a new way to look at a project

• Understanding differences between roles is critical



Scotland - The National Stadium

• Budget: £58million

• 6 years

• Status– In service

• Problem:– Cost overrun and

funding gap



U.K. Courts – IT System

• £148m over 11 years

• £319m over 14 years

• £232 million

• 8 ½ years

• Status– Implementation Phase

• “the shoddiest ever”



U.S. Project Aspire – IT

…

• US$88+ million

• 3 ½ years

• Status– Under evaluation



U.S. People First: Florida

• US$350+million

• 9 years

• Status– Under evaluation



U.S. Government: Deepwater

• US$24 billion

• 25 years

• Status– “Moving forward”:

Primary contractor removed from program manager role



Deepwater - 8 Ships – $100M



Imagine you are building a vacation house in Spain…but you live in London

A Project Monitor could• Evaluate change orders• Conduct onsite visits• Provide proactive problem

identification• Review the schedule periodically• Evaluate monthly progress• Verify periodic progress payments are

in order• Assess the critical path• Take photos of progress• Be your advocate



Implementation monitoring and results based monitoring compared

• What does RBM do?• Why is it different from Earned Value?

IMPLEMENTATION MONITORING

RESULTS-BASED MONITORING

ACTIVITIES

Steps taken to implement

project

INPUTS

Resources, money,

staff

OUTPUT

What the project

produces

OUTCOME/ IMPACT

Change in the “real world”



M&E process

Looking at the effect of project activities and deliverables



Tools: A Sampling

• Standards– Swiss Evaluation Society– 20 page PDF

• www.seval.ch



Tool: Logical Framework

Agreed upon framework of how results will be assessed

Created collaboratively



Tool: Reporting Schedule

Agreed upon schedule of reporting

Monitor is to verify periodically the reporting is useful



Tools: Opportunities

• Canada site lists– Evaluation contracts open to tender offer– Opportunities for collaboration– Career employment opportunities

• www.evaluationcanada.ca– Contracts, collaboration and jobs



Tools: Networking

Publish your profile as an evaluator known in the evaluation community

• Search systematically – Evaluators– Commissioners of evaluations– Research partners

• See who is involved in evaluation• Over 180 names• http://www.seval.ch/en/evaluator/index.cfm



Success in Scotland Park and apartment complex

King’s Gait - Glasgow



Nine Step Process

1.Conduct an inventory2. Agree on what to monitor3. Gather baseline data on non-deliverable indicators4. Build the monitoring system 5. During project gather data on project/product deliverables6. Analyze and report monitoring information7. Verify quality of monitoring process8. Use the data 9. Reflect



Organisations are looking for results

• Drive to improve governance and increase accountability– A project audit is a recognized QA tool

– But it can be like hitting the project with a hammer!

when what you need is a wrench



Why Results Based Monitoring?

• Results Based Monitoring provides– Framework to avoid the confrontation that can

result from an audit– Demonstrated success in construction– When organisation needs assurance – risk

reduction – Especially useful when specialty expertise is

needed



What does it take to be successful?

• Clearly defined roles• Clear and verifiable objectives and indictors• An organisation ready for bad news as well as good• Subject matter expertise is vital - RBM is hands-on • Understand influence

• Make the project manager see the benefit



Implications for Project Managers

• Know what M&E using RBM is

• Know when it might be beneficial

• Understand steps for successful RBM

• Possible area of work for PMPs – Subject matter expertise in project area critical



A Summary

• RBM is a tool to improve project success

• Used by government, NGOs and public/private companies

• Requires – Definition in advance of what results will look like

– Monitoring to see whether those results are achieved



Evaluation Societies in Europe• Austria – 16 organisations

• Denmark – 170 members

• Finland - Not available

• France – 250 members

• Germany - 460 members

• Italy - 340 members

• Spain -150 members

• Sweden - 200 members

• Switzerland - 300 members

• UK - 150 members

• UN - 36 UN agencies

• EU-Europe - Not available



Questions?



Contact Information• Joy Gumz / Project Auditors LLC

• Phone: +001 949 452 0578

• Email: [email protected]

• Session #IND01



"PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007...

Documents

Transcript of "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007...