Implementing & Sustaining Implementing & Sustaining ROIROI

Patti Phillips, Ph.D.

patti@roiinstitute.net

Poll QuestionPoll QuestionBUSINESS 2.0 BUSINESS 2.0 Dream Team*Dream Team*

CEOCOOCFOCMOCIO

SVP – Industrial DesignSVP – R&DSVP – Online SalesSVP - Procurement

*BUSINESS 2.0 July 2004

Who’s missing?a. No one

b. SVP – Corporate Services

c. Chief Learning Officer

d. Michael Jordan

Sample of Sample of Dream Team Dream Team AccomplishmentsAccomplishments

• Saved $1.8 billion in 18 months and increased sales from $10 million to more than $85 million in five years

• Increased shareholder base from less than 15 percent institutional ownership to more than 90 percent

• Increased number of new patents• Reduced 4 payroll systems to 1• Reduced 11 billing systems to 2 in 24 months

Meet Donald HodgesMeet Donald Hodges•President, GlobalCom University

•15 years experience in adult learning

•Handpicked by previous CEO

•Wants to impress new CEO with program results

Donald’s results as reported to new CEO Developed 10 new programs

Offered 1,120 hours of training Had 1,500 employees attend training

Received on average 4.5 out of 5 on the program satisfaction rating

Donald’s results as reported to new CEO Developed 10 new programs

Offered 1,120 hours of training Had 1,500 employees attend training

Received on average 4.5 out of 5 on the program satisfaction rating

Poll QuestionPoll Question

How likely is Don to be recruited for a position on the Dream Team?

a. No way.

b. Possibly, with a little improvement.

c. Absolutely!

11 ObjectivObjectiv

ee

To help you beginToday we have

1. Effectively communicating results on a routine basis

2. Developing an ROI implementation strategy

3. Determining the ROI on the ROI

What do we mean by What do we mean by resultsresults??

Level

1. Reaction & Planned Action

2. Learning

3. Application

4. Impact

5. Return on Investment

Measurement Focus

Measures participant satisfaction with the program and captures planned actions

Measures changes in knowledge, skills, and attitudes

Measures changes in on-the-job behavior

Measures changes in business measures

Compares program benefits to the costs

Eco

nom

icS

yste

mIn

divi

dual

ROI MethodologyROI MethodologyAdapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) Adapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) The Human Resources Scorecard: Measuring the Return on InvestmentThe Human Resources Scorecard: Measuring the Return on Investment. Boston, MA: . Boston, MA: Butterworth-HeinemannButterworth-Heinemann

Develop EvaluationPlans and

Baseline Data

Develop EvaluationPlans and

Baseline Data

Stage 1Evaluation Planning

Stage 2 Data Collection

Collect DataDuring SolutionImplementation

Collect DataDuring SolutionImplementation

Collect DataAfter Solution

Implementation

Collect DataAfter Solution

Implementation

DevelopObjectivesOf Solution

DevelopObjectivesOf Solution

Level 1

Level 2

Level 3

Level 4

TabulateCosts

Of Solution

TabulateCosts

Of Solution

Isolate theEffects ofSolution

Isolate theEffects ofSolution

Convert Data to Monetary

Value

Convert Data to Monetary

Value

Calculatethe Return On

Investment

Calculatethe Return On

Investment

Implement Communication

Process

Implement Communication

Process

Stage 3 Data Analysis

Identify Intangibles

Identify Intangibles

Stage 4 Communicate

Results

Level 5

Intangible Benefits

Reaction & Planned Action

Learning

Application & Implementation

Impact

ROI

Isolate the Effects of the Program

Report the complete story of success.

Intangible Benefits

Developing a communication Developing a communication strategy requires us to answer strategy requires us to answer

three questions.three questions.

1. What do we want?

2. Who can give it to us?

3. How best can we ask?

Common Target AudiencesCommon Target Audiences

What do we want? Who can give it to us?Secure approval for program Client, top executives

Gain support for the program Immediate managers, team leaders

Build credibility for the training staff Top executives

Enhance reinforcement of the program Immediate managers

Enhance results of future programs Participants

Show complete results of the program Key client team

Stimulate interest in training programs Top executives

Demonstrate accountability for client

expenditures All employees

Market future training programs Prospective clients

How best can we ask?How best can we ask?

• Ongoing Summaries

• Complete ROI Impact Study Report

• Executive Summary

• One-page Report

• Training Scorecard

How? What? Who? When?

Ongoing Summary Brief report of program success

•Training staff

•Evaluation team

•Participants

One week after program (L1 & L2)

One week after data are collected (L3 & L4)

Complete Report Details of evaluation project

•Training staff

•Evaluation team

•Client

•Senior Managers*

30 days after data are collected

Executive Summary Brief overview of evaluation project

•Training staff

•Evaluation team

•Client

•Participants

•Senior Managers*

30 days after data are collected

One-Page Report Micro-level scorecard of program success

•Training staff

•Evaluation team

•Senior Managers*

•Supervisors

30 days after data are collected

(Use with caution!)

Complete ReportComplete Report

• General Information• Methodology for Impact Study• Data Analysis• Costs• Results• Barriers and Enablers• Conclusions and Recommendations• Exhibits

When communicating with senior managers consider

two questions:

Do they believe

you

Can they

take it

Communication ProgressionCommunication Progression

First 2 ROI

Studies

3-5 ROI Studies

6 Plus ROI

Studies

Detailed Study

Executive Summary

One Page Summary

Meeting

No Meeting

No Meeting

Training ScorecardTraining Scorecard

• Provides macro-level perspective of success

• Serves as a brief report versus detailed study

• Shows connection of training’s contribution to business objectives

• Integrates various types of data• Demonstrates alignment between

programs, strategic objectives, and operating goals

Training scorecards include Training scorecards include seven categories of data.seven categories of data.

• Indicators• Reaction and Planned Action• Learning• Application• Business Impact• ROI• Intangibles

Developing an ROI strategy Developing an ROI strategy requires us to answer four requires us to answer four

questions.questions.1. Do we need it?

a. Our assessmentb. Our clients’ assessment

2. What are our desired outcomes?a. What are the consequences of our implementing an ROI strategy?b. What are the negative consequences if we do not implement an

ROI strategy?

3. Why?a. What purpose will implementation success serve us?b. What purpose will implementation success serve our team?c. What purpose will implementation success serve the organization?

4. How can we get there?a. What are the perceived barriers?b. How can we overcome them?c. What is one action we can take today to move forward?

Are you a candidate for ROI? Are you a candidate for ROI? Sample Assessment QuestionsSample Assessment Questions

• Is your organization large with a variety of programs?

• Is your organization undergoing significant change?

• Is there pressure from senior management to measure results?

• Is the image of your training function less than desirable?

• Is there an increased focus on linking training to strategy?

For complete assessment go to Tools & Templates at www.roiinstitute.net.For complete assessment go to Tools & Templates at www.roiinstitute.net.

Potential Outcomes of ROI Potential Outcomes of ROI ImplementationImplementation

• Eliminate unnecessary programs• Increase (or maintain) training budget• Improve image with line supervisors• Reallocate resources to allow for staff

development• Reallocate resources to build evaluation

infrastructure• Expand implementation of successful

programs

If you implement an ROI strategy, what outcomes do you expect?

What are the consequences of your implementing this strategy?

If you do not implement this strategy, what are the negative consequences?

Exercise 1

Purpose drives the initiative to Purpose drives the initiative to achieve outcomes. achieve outcomes.

Implementation of our evaluation strategy gives us the information we need to help develop our people, improve our processes, and contribute to the bottom line – literally!

We ensure that taxpayer dollars are allocated to the programs that directly support the agency mission.

What purpose will achieving the ROI strategy outcomes

serve you, your team, and your organization?

Exercise 2

Poll QuestionPoll Question

Which of the following represents the biggest barrier to your implementing ROI?

1.Confidence to implement ROI2.Management support3.Cost4.Time5.Resources (people, technology)

Cost-Saving Approaches to ROI

• Plan for evaluation early in the process

• Build evaluation into the training process

• Share the responsibilities for evaluation

• Require participants to conduct major steps

• Use short-cut methods for major steps

• Use sampling to select the most appropriate

programs for ROI analysis

• Use estimates in the collection and analysis of

data

• Develop internal capability to implement the

ROI process

• Utilize web-based software to reduce time

• Streamline the reporting process

Cost-Saving Approaches to ROI

Actions to make ROI work Actions to make ROI work includeinclude

• Planning and discipline– Establish goals and

targets– Assign responsibilities

• Improve needs analysis process

• Implement cost-savings approaches

• Communicate progress• Engage Management

– In Programs– In Evaluation

• Develop staff skills– ROI Networks– Workshops– ROI Certification– Competency groups– Read books/case

studies/articles– Develop a case study– Teach the process to others

What is one action you can take today to move forward with ROI implementation?

Exercise 3

So what is the ROI on ROI?So what is the ROI on ROI?

Maximum Use

$ High ROI $

Success with ROI implementation is related to the extent to which results are used for process improvement!

Success with ROI implementation is related to the extent to which results are used for process improvement!

Individual Perspective (Levels 1 and 2)

•Are facilitation scores improving?

•Are programs perceived as more relevant and important?

•Do content and program design better reflect adult learning principles?

•Is there evidence that learning is taking place at a higher rate?

System Perspective (Level 3)

•Do supervisors better support training?

•Are systems in place to better support the learning transfer?

•Is frequency of use of knowledge and skills increasing?

•Are there fewer barriers to learning transfer?

When developing the ROI on When developing the ROI on the ROI consider results from the ROI consider results from

all perspectives!all perspectives!

When developing the ROI on When developing the ROI on the ROI consider results from the ROI consider results from

all perspectives!all perspectives!Economic Perspective (Level 4 and 5)

• How many unnecessary programs have we eliminated?

• Have we decreased the cost of training while improving the quality of our programs?

• How much has our budget increased?

• When comparing cost savings and budget increases to the cost of ROI implementation, do we see a positive economic return?

Join the Join the Dream TeamDream Team

CEOCOOCFOCMOCIO

SVP – Industrial DesignSVP – R&DSVP – Online SalesSVP - Procurement