Copyright 2004 ROI Institute, Inc. how me the money! Moving from Impact to ROI Patti Phillips, Ph.D....

Copyright 2004 ROI Institute, Inc.

how me the money!how me the money!

Moving from Impact to ROIMoving from Impact to ROI

Patti Phillips, Ph.D.

[email protected]


In this session participants will:

• Identify program benefits

• Convert benefits to monetary value

• Tabulate fully-loaded costs

• Identify intangible benefits

• Calculate an ROI


ROI is reported as

BCR = Program Benefits

Program Costs

ROI = Net Program Benefits

Program CostsX 100


ROI is reported in the context of five levels of evaluation.

Level

1. Reaction & Planned Action

2. Learning

3. Application

4. Impact

5. Return on Investment

Measurement Focus

Measures participant satisfaction with the program and captures planned actions

Measures changes in knowledge, skills, and attitudes

Measures changes in on-the-job behavior

Measures changes in critical outcomes

Compares program benefits to the costs


Moving from Impact to ROI requires five steps.

Level 4

Level 5

1. Identify program benefits

2. Convert benefits to monetary value

3. Tabulate program costs

4. Identify intangible benefits

5. Compare monetary benefits to program costs


Level 4 1.Identify program benefits


Level 4 measures are defined as

… the consequence of doing something different as a result of the

program


Poll Question

A. Decrease error rates

B. Increase use of disciplinary discussion skills

C. Initiate at least three cost reduction projects

D. Achieve a 2:1 benefit to cost ratio

E. Improve teamwork

Which of the following measures represent Level 4 Impact measures?


A. Decrease error rates

E. Improve teamwork

Which of the following measures represent Level 4 Impact measures?


Hard Data

versus

Soft Data

Level 4 measures include two type of data.


Decrease error rates

versus

Improve teamwork



Tangible

versus

Intangible



All data can be converted.

• Productivity

• Quality

• Time

• Cost Savings

• Revenue

Convert data to cost savings

Sales converted to profit margin


Poll QuestionWhich one of the following is the most

credible item?

A. Vulcan Materials Company produced 195 million tons of crushed stone during 2001

Source: Annual Report

B. Wachovia Bank receives a 932% ROI in a training program for relationship managers.

Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 81.

C. Verizon receives a -85% ROI in a telephonic customer service skills program.

Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 131.


Credibility of data is influenced by

• Reputation of the source– Source of data– Source of study

• Bias– Researcher bias– Audience bias

• Methodology– Assumptions made in the analysis– Type of data– Scope of analysis– Realism of the outcome data


Data are converted by• Converting output to contribution - standard value• Converting the cost of quality - standard value• Converting employee’s time – standard value• Using historical costs• Using internal and external experts• Using data from external databases• Linking with other measures• Using participants’ estimates• Using supervisors’ and managers’ estimates• Using staff estimates


Historical Costs and Expert Input are Used to Convert Data

Actual Costs Legal Fees, Settlements,from Records Losses, Material,

Direct Expenses

Additional Estimated EEO/AA Staff Time, Costs from Staff Management Time

The Cost of A Sexual Harassment Complaint

35Complain

ts

$852,000Annually

Cost per Complaint = $24,343

$852,000

35


External Databases are Used to Convert Data

Cost of one turnover*

Middle Manager $70,000 annual salary

Cost of turnover 150%

Total cost of turnover $105,000

* External data – value obtained from industry related study


Supervisor Estimates Used to Convert Absenteeism

Unit of Measure: AbsenteeismData Collection Technique: Focus GroupsSource of Data: Supervisors

1. What happens when someone is unexpectedly absent?2. In monetary terms, how much does it cost you when a

person is unexpectedly absent for one day?1. What is your basis for this estimate?

3. On a scale of 0% - 100% (0% = no confidence; 100% = certainty) how confident are you in your estimated value?


Supervisor Estimates Used to Convert Absenteeism

Supervisor Est. Per Day Cost

% Confidence Adjusted Per Day

Cost

1 $1,000 70% $700

2 $1,500 65% $975

3 $2,300 50% $1,150

4 $2,000 60% $1,200

5 $1,600 80% $1,280

$5,305

Average adjusted per day cost of 1 absence $1,061


Estimated Value of Absenteeism

$0

$500

$1,000

$1,500

$2,000

$2,500

1 2 3 4 5

Supervisors

Est

imat

ed V

alue

s

Estimated Value Adjusted Value


To Convert or Not Convert

Use the 4-Part Test Is there a standard value? Is there a method to get

there? Can we get there with

minimum resources? Can we convince our

executive in 2 minutes that the value is credible?


Data conversion requires 5 steps:

1. Focus on a unit of measure2. Determine the value (V) of each unit3. Calculate the improvement in the

measure (∆P)4. Determine the annual improvement

in the measure (A∆P)5. Calculate the total monetary value of

the improvement (A∆P x V)


Grievances Converted by Internal Experts

Step 1: 1 Grievance

Step 2: V = $6,500

Step 3: ∆P = 7 out of 10 grievances prevented per month due to program

Step 4: A∆P =

Step 5: A∆P x V =


Grievances Converted by Internal Experts

Step 1: 1 Grievance

Step 2: V = $6,500

Step 3: ∆P = 7 out of 10 grievances prevented per month due to program

Step 4: A∆P =

Step 5: A∆P x V = 84 x $6,500 = $546,000

7 x 12 = 84


A BOperating CostsSupport Costs

Administrative CostsParticipant Compensation and Facility CostsClassroom Costs

C DProgram Development CostsAdministrative CostsClassroom CostsParticipant Costs

Analysis CostsDevelopment CostsDelivery CostsOverhead CostsEvaluation Costs

Poll QuestionWhich cost categories represent the fully-loaded costs of a program?


Recommended Cost Items• Analysis Costs (Prorated)• Development Costs (Prorated)• Program Materials• Instructor/Facilitator Costs• Facilities Costs• Travel/Lodging/Meals• Participant Salaries and Benefits• Administrative/Overhead Costs • Evaluation Costs


Potential Intangible Benefits

• Increased Job Satisfaction

• Increased Organizational Commitment

• Improved Teamwork

• Improved Customer Service

• Reduced Complaints

• Reduced Conflicts

• Reduced Stress


ROI is reported one of two ways:

BCR = Program Benefits

Program Costs

ROI = Net Program Benefits

Program CostsX 100


Healthcare Inc.

• Program Costs– Fully loaded to include needs assessment, development,

coordination, participant salaries and benefits, and evaluation– Total Costs = $277,987

• Monetary Benefits– Complaint Reduction

• Value of one internal complaint = $24,343• Annual improvement related to program – 14.8 complaints

prevented

– Turnover Reduction• Value of one turnover statistic = $20,887• Annual improvement related to program = 136 turnovers prevented

Your assignment: Calculate the BCR and the ROI


Healthcare Inc.

• Program Costs = $277,987• Monetary Benefits

– Complaint Reduction• Value of one internal complaint = $24,343• Annual improvement related to program – 14.8 complaints

prevented• 14.8 X $24,343 = $360,276

– Turnover Reduction• Value of one turnover statistic = $20,887• Annual improvement related to program = 136 turnovers

prevented• 136 X $20,887 = $2,840,632


Healthcare, Inc.

Program Benefits =

Program Costs =

BCR = =

ROI = x 100 =

$3,200,908

$277,98711.51

$2,922,921

$277,9871,051.5%

$360,276 + $2,840,632

$277,987


What is a good ROI?

• Set the value as with other investments, e.g. 15%

• Set slightly above other investments, e.g. 25%

• Set at break even - 0%

• Set at client expectations


During this session we

Defined program benefitsConverted benefits to monetary valueIdentified fully-loaded costsIdentified intangible benefitsCalculated an ROI


Remember

Data ConversionWhen in doubt, leave it out!

Use the most conservative alternative

Use the most credible data course

Use the four-part test! Use only first-year

benefits for short-term programs

Cost TabulationWhen in doubt, leave it in!

Tabulate the fully-loaded costs

Develop a consistent cost profile

Use the most conservative alternative

Use the most credible data source


Go forth and….Go forth and….

how them the how them the money!money!

t

Copyright 2004 ROI Institute, Inc. how me the money! Moving from Impact to ROI Patti Phillips, Ph.D....

Documents

Transcript of Copyright 2004 ROI Institute, Inc. how me the money! Moving from Impact to ROI Patti Phillips, Ph.D....