ROI Toolkit
-
Upload
philipkock -
Category
Documents
-
view
223 -
download
0
Transcript of ROI Toolkit
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 1/75
ROI of Human Capital Interventions
A Practice Toolkit
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 2/75
Description Icon
Relevant quote
Concept/Framework
Tools, Techniques
Template
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 3/75
Step 1
Presenting Problem to Real Problem
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 4/75
Step 1. Presenting Problem to Real Problem
The first task of the practitioner is almost always analysis
Starts with a“presenting”
problem – not a
“hunch” but a
problem … as it
presents itself
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 5/75
Step 1. Presenting Problem to Real Problem
Example…
Let's take an
example – higher
staff turnover for
Graduate
Engineers (let say
compared to an
industry standard)
The first question should not be, what skills and knowledge are required. It should be which
metric(s) are affected negatively.
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 6/75
Step 1. Presenting Problem to Real Problem
Example…
Unit cost perreplacement
Cost of 'covering' avacant position
(Calculated costs of other employees
'filling in' while theposition is vacant)
Cost to fill a vacantposition
On boarding &Orientation cost
Cost of productivityramp-up
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 7/75
Step 1. Presenting Problem to Real Problem
1.1 Calculating benchmark employee cost
Nr Metric Element Description Example
1. Departing employee annual base
salary
N/a 100,000
2. Calculated annual benefits cost Estimated at 20% of
base salary
20,000
3. Calculated monthly salary + benefits 10,000
4. Calculated daily salary + benefits Based on 230 (8hr)
working days
522
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 8/75
Step 1. Presenting Problem to Real Problem
1.2 Cost of 'covering' a vacant position (Calculated costs of other employees 'filling
in' while the position is vacant)
Nr Metric Element Description Example
1. Number of days until the vacant
position is filled
N/a 90
2. Calculated daily cost of 'covering' a
vacant position
33% of departing
employee's daily
salary + benefits
522/3
3. Total cost to 'cover' vacant position 15644
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 9/75
Step 1. Presenting Problem to Real Problem
1.3 Cost to fill a vacant position
Nr Metric Element Description Example
1. HR/Hiring manager's annual salary N/a 200,000
2. Calculated HR/Hiring manager's hourly rate Based on 230 (8hr) working
days and 20% fringe rate
130
3. Cost of advertising (online and/or print) Enter cost 3000
4. Cost of resume screening HR/Hire manager hours *hourly rate
50*130=6500
5. Cost of interviews (telephone screening, 1st and
2nd)
HR/Hire manager hours *
hourly rate
50*130=6500
6. Cost of behavioural and skills assessments Enter cost 3000
7. Cost of background checks (criminal, credit,
reference, education)
Enter cost 4000
8. Cost of travel/moving expenses (if applicable) Enter cost 5000
9. Total cost to fill a vacant position: 28,000
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 10/75
Step 1. Presenting Problem to Real Problem
1.4 On boarding & Orientation cost
Nr Metric Element Description Example
1. Trainer/Manager annual salary: N/a 100,000
2. Calculated trainer/manager daily rate Based on 230 (8hr) working
days and 20% fringe rate
522
3. Total training days Enter number of days 10
4. Total on boarding and orientation cost N/a 5220
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 11/75
Step 1. Presenting Problem to Real Problem
1.5 Cost of productivity ramp-up (During the first 3 months, an average new
employee performs at 50% productivity of a tenured top performing employ
Nr Metric Element Description Example
1. Daily employee cost (salary + benefits) N/a 522
2. Number of working days during first 3 months: Enter number of days (avg.
58 days
50
3. Cost of productivity ramp-up N/a 26100
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 12/75
Step 1. Presenting Problem to Real Problem
1.6 Unit cost for filling a vacancy
R 74964
Two important learning points:
I. Know your metrics
II. Negotiate, and even request help from your client to determine the right metrics. They are
closer to the business and at the same time you obtain by in for the intervention.
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 13/75
Step 1. Presenting Problem to Real Problem
1.7 Total cost of replacing graduate engineers
R 74964 x 6
= R 449784
Let's assume that we had to replace five graduate engineers the
past six months
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 14/75
Analysis andDefinition of
Business Need
Identification ofPerformance
Need
Competence GapSolution
GenerationIntervention
Strategy
PresentingProblem/
Opportunity
Non TrainingSolution
Step 1. Presenting Problem to Real Problem
Now we can move on to analysis
We know now which
metrics are affected
Consultant speciality
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 15/75
Step 1. Presenting Problem to Real Problem
The basic tasks in analysis
4. Perspective and informationfor decision making
3. Dataanalysis
2. Dataconsolidation
1. Datagathering
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 16/75
Step 1. Presenting Problem to Real Problem
Two types of data
Data
Quantitative
Qualitative
” More about context, how
people experience in their
own words”.
”The truth is in a number” –
however does not say much
about context
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 17/75
Step 1. Presenting Problem to Real Problem
Data gathering
I. Questionnaires
II. Surveys
III. Tests
IV. Interviews
V. Focus groupsVI. Observation
VII. Performance records
VIII.Knowledge and skills testing
IX. Program follow up
X. Project assignments
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 18/75
Step 1. Presenting Problem to Real Problem
Data gathering
I. Target population/participants;
II. Supervisors of target
population/participants;
III. Subordinates of target
population/participants;
IV. Peer Group;V. HRD Staff;
VI. Documentation:
• Organizational Performance
Records:
• Disciplinary records;
• Safety records;• Scorecards
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 19/75
Step 1. Presenting Problem to Real Problem
Contextual factors to consider when selecting data collection methods
I. Time for participants to respond;
II. Costs;
III. Amount of disruption of normal
activities;IV. What is the level of certainty that
you will achieve with the result;
V. Practicality;
VI. Culture/Philosophy – example the
10% syndrome.
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 20/75
Step 1. Presenting Problem to Real Problem
Requirements for effective data collection instruments and techniques
I. Construct validity ( methods,
techniques and test instruments
should be evaluated in terms of its
theoretical grounding);II. Predictive validity (methods,
techniques and test instruments
should be evaluated regularly in
terms of its ability to predict what it
purport to predict);
III. Face validity – for line managementvery important!
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 21/75
Step 1. Presenting Problem to Real Problem
Requirements for effective data collection instruments and techniques
Reliability:
I. Produce consistent results
over time, that differentiateseffectively, and are
independent of the
assessor/data collector, or
context.
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 22/75
Step 1. Presenting Problem to Real Problem
Data consolidation
Data Consolidation Techniques areWide and Varied
Descriptive statistics
Qualitative description, using organizational
and other models and concepts
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 23/75
Step 1. Presenting Problem to Real Problem
Data analysis
Data Analysis Varies According toSituation
Descriptive and inferential statistics
Qualitative decisions about the situation,
based on decision making and techniques
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 24/75
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 25/75
Brief Exposition of Problem
Our analysis shows that the HR and line
managers involved in selection decisions do
interviews in a haphazard and nonscientific
fashion
Solution
Following discussions with consultants, HR
experts and line all agreed we need to
implement competency based interviewing
Step 1. Presenting Problem to Real Problem
Lets assumethis based
on the core
example
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 26/75
The training department is instructed to
develop an appropriate program and put all
line and support staff through training – they
however indicate that a change in results is
required quickly and that ROI for the trainingneed to be justified soon after
implementation…
Step 1. Presenting Problem to Real Problem
Lets assume this
based on the core
example
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 27/75
Presenting Problem to Real Problem
So we have addressed this even before we asked any questions about training – and
now we can move on to the next step
Starts with a“presenting”
problem – not a
“hunch” but a
problem … as it
presents itself
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 28/75
Step 2a
Planning the Intervention (Starting With the End
In Mind)
St 2 Pl i th I t ti (With ROI i Mi d)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 29/75
Step 2a: Planning the Intervention (With ROI in Mind)
When we plan an intervention – considering that we want to measure ROI- we really reverse
engineer (or start with the end in mind) – We use the Kirkpatrick Model to achieve this
4. Impact
What is the business impact we need to achieve (refer step 1where we got a firm grip on the metric that must change)
3. Transfer
What is the work place behaviour that must change
2. Learning
What knowledge, skills and attitudes need to be acquired
1. Reaction
What is the required reaction of the learner to theintervention
Step 2a Planning the Inter ention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 30/75
Step 2a: Planning the Intervention (With ROI in Mind)
Back to our case study to show how we plan “backwards” – remember these objectives and
measures for clarification, based on our core example/case study (re graduate engineers)
Kirkpatrick Level Objectives Measures
Impact Train target population (define) in behaviour based
interviewing techniques, with the aim to:
a. Improve predictive validity of selection decisions
b. Driving down cost of filling vacancies for graduate
engineers
25% reduction in total cost of filling
vacancies for graduate engineers
Transfer Correct application of behaviour based interviewing
techniques by target populations during selection
interviewing
Expert observation with the aid of
checklists show that all involved in
selection interviews use the correct
interviewing structure (agenda) and
questioning techniques at least 85%
of the time. Observation is
conducted for two months following
the initial training and corrective
coaching is implemented where
required, following interviews.
Step 2a: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 31/75
Step 2a: Planning the Intervention (With ROI in Mind)
Back to our case study to show how we plan “backwards” – remember these objectives and
measures a merely examples
Kirkpatrick Level Objectives Measures
Learning Using Blooms Taxonomy of Learning Objectives here are
a few examples of the objectives that will be outlined
here:
a. The learner should demonstrate that they can
analyse a CV, with the aim to develop behaviour
based questions for an interview;b. The learner should demonstrate the ability to
develop interview questions according to the
behavioural interview format
c. Etc. etc…
100% compliance with the Bloom
Taxonomy, alignment with SAQA Unit
Standards and also SAQA guidelines
and requirements for assessment
and moderation;
Learners score 75% on theoreticalpaper to pass and 80% on practical
simulation interview.
Remember this is asnapshot – there is a
whole learning map and
course development
process underlying
these objectives
Step 2a: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 32/75
Step 2a: Planning the Intervention (With ROI in Mind)
Back to our case study to show how we plan “backwards” – remember these objectives and
measures a merely examples
Kirkpatrick Level Objectives Measures
Reaction The reaction to the learning material, environment,
perception of preparation, facilitator style and
Using a Likert type scale participants
rate the following on average no lessthan “agree”:
a. Preparation
b. Facilitator style
c. Learning environment
d. Participation
e. General expectations met
Step 2a: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 33/75
Step 2a: Planning the Intervention (With ROI in Mind)
What have we achieved by planning the intervention this way?
4. Impact
What is the business impact we need to achieve (refer step1 where we got a firm grip on the metric that must change)
3. Transfer
What is the work place behaviour that must change
2. Learning
What knowledge, skills and attitudes need to be acquired
1. Reaction
What is the required reaction of the learner to theintervention
A p a t h –
t r a i l l i n k i n g t h e
b u s i n
e s s n e e d w i t h t h e
i n t e r v e n t i o n
Results
Results
Results
Results
ROI ?
Step 2a: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 34/75
Step 2a: Planning the Intervention (With ROI in Mind)
We can also go some steps further and plan where we will get the information to evaluate
the intervention and how…
Kirkpatrick Level Objectives Measures Where Will We Find
Information For
Evaluation
In What Format
Impact See previous slides See previous
slides
HR Budget vis Actual
Staff Turn Over Records
Business Report
HR Report
Transfer See previous slides See previous
slides
Audit/Observation results Possibly checklists
Learning See previous slides See previous
slides
Test scores
Practical simulation
scores
Tests papers and
answer sheets
Simulation checklists
and grading papers
Reaction See previous slides See previousslides Survey results Possibly tabulatedand presented in
graphs
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 35/75
Step 2b
Planning the Intervention (Planning Beyond
Training – With ROI In Mind)
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 36/75
Step 2b: Planning the Intervention (With ROI in Mind)
Usually once we have done the planning and developed the training intervention we are
ready to implement…
a. With ROI in mind our planningneeds to go some steps further…
b. We need to determine whattechnique(s) we will use to showthat the intervention actually hadan impact
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 37/75
Step 2b: Planning the Intervention (With ROI in Mind)
We need to determine what technique(s) we will use to show that the intervention actually
had an impact – but our dilemma is this…
External Factors
Management
Attention
Incentives
Systems/Procedures
Changes
Here is our training
program
AND YES THERE IS
IMPROVEMENTAFTER
PROGRAM
IS IT THETRAININGPROGRAM?
OR SOMETHING
ELSE?
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 38/75
Step 2b: Planning the Intervention (With ROI in Mind)
We have to isolate the effects of our intervention!
I. Isolating the results/effect of our intervention is the most difficult and also most
important element of building the case for value (ROI);
II. Because of this, value at the higher levels is seldom determined;
III. However, the credibility not only of your study to show value, but also the training
function is based on this element.
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 39/75
p g ( )
We have to isolate the effects of our intervention!
I. The complexity of isolating the effect, is a function of the variables that are in play
at any moment in time in an organization;
II. The analyst should however always try to prove a causal relationship to an
acceptable degree of accuracy;
III. Acknowledging the forces that influence individual, group and business
performance is the first step in this process.
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 40/75
p g ( )
We have to isolate the effects of our intervention!
Remember that we are trying to isolate the effect of out intervention to an acceptable
degree of accuracy – this makes very important to involve and agree with the client:
a. What metrics should change as result of the intervention (see step 1)
b. What isolation technique will be used (because they differ in terms of accuracy)
Let’s consider the isolation techniques…
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 41/75
p g ( )
Let's look at the techniques!
a. Trend line analysis of performance
data;
b. Control groups;
c. Participant’s estimation of impact;
d. Supervisor’s estimation of impact;
e. Management’s estimation of impact;
f. Use of previous studies
g. Subordinate’s report of other factors;
h. Estimating the impact of other factors;
i. Use of customer input.
With these
techniques we
try to isolate the
effects of our
intervention
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 42/75
Let's look at the techniques!
a. Trend line analysis of performance
data;
b. Control groups;
c. Participant’s estimation of impact;
d. Supervisor’s estimation of impact;
e. Management’s estimation of impact;
f. Use of previous studies
g. Subordinate’s report of other factors;
h. Estimating the impact of other factors;
i. Use of customer input.
With these
techniques we
try to isolate the
effects of other
interventions
and then declare
the rest as due
to our effort
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 43/75
Trend Line Analysis
I. Draw a trend-line using previous
performance data as a base;
II. When intervention is conducted,
actual performance is plotted and compared to the trend line;
III. Improvement in actual performance
over what the trend line predicted
can then be reasonably attributed to
the intervention.
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 44/75
Advantages Disadvantages
I. Show quick results
II. Easy to calculate
III. Easy to understand
IV. Inexpensive
V. Not difficult to organize (basically just
monitor changes in metrics)
VI. A good first order analysis
I. Uses only one factor to isolate the effect
of the intervention (time- before and
after implementation)
II. Thus less accurate and client could be
sceptical to accept only an analysis of
the trend line.
Trend Line Analysis- Advantages and Disadvantages
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 45/75
Example- Not from our central case study…
AveragePost Program 35
AveragePre Program 55
TIME
C O M P L A I N T S
SexualHarassment
Prevention Program O N D J F M A M J J A S O N D J F M A M J J
A S O
Effect of the
intervention
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 46/75
Control Groups Technique
I. Similar to classical research the control group method, involves an experimental group and a
control group;
II. The so called experimental group will be the target of our intervention;
III. The control group would be a group that is paired with the experimental group, but would
not undergo the intervention;
IV. Important things to remember with this method are:
a. Paring need to be thought about carefully so as not to dilute the impact of the
intervention;
b. Although not as rigid as classical research key factors that could dilute the impact or
assessment of the impact of the intervention need to be controlled.
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 47/75
Advantages DisadvantagesI. More accurate than most techniques
II. If the pairing and control of variables are
agreed with the client the outcome has high
level of credibility
III. With this control groups, given the level of
accuracy it is possible to benchmark the ROI
of certain interventions given a
predetermined set of variables
I. More complicated
II. Require more planning, negotiation,
control and monitoring
III. As result of above many practitioners areunwilling to use this technique
IV. Requires absolute agreement with the
client environment that variables will be
kept constant for the duration of the ROI
evaluation.
Control Groups- Advantages and Disadvantages
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 48/75
Schematic Explanation of control groups design
Control
Group
Experimental
GroupM1 Intervention
M2
M2
M1
*M = measurement
1. Pairing 2.
Control
Variables
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 49/75
Looking at pairing and control of variables
Select your
experimental and control groupcarefully
Note:
I. Too small then meaningful pairing is not
possible – too big then you are not going
to be able to control the variables during
and after the implementation of your
intervention;II. Remember we go for reasonable
accuracy therefore you pair the majorelements and also control the majorvariables!!
III. The above makes it very important toagree upfront with your clientenvironment what how pairing will be
done and which variables will becontrolled.
PAIRING
VARIABLES
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 50/75
So what do you typically pair?
Negotiate creafully
S i z e
N a t u r e o f w
o r k
L e a d e r s h i p s t y p e
P e r f o r m a n c e
l e v e l
G e n e r a l d e m o g r a p h y
Pair for (Primarily)
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 51/75
So what do you typically try to control?
Negotiate creafully
S i z e
N a t u r e o f w
o r k
L e a d e r s h i p s t
y p e
P e r f o r m a n c e l e v e l
G e n e r a l d e m o g
r a p h y
Changes in:
Negotiate creafully
s t r u c t u r e
F u n c t i o n
R e c o g n i t i o n a n d
r e w a r d
T u r n o v e r
S y s t e m s a n d p r o c e d u r e s
Changes in:
+
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 52/75
Remember your decision about pairing and control of variables….
= (f) (your knowledge and
judgement) + agreement withthe client environment +
practicability
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 53/75
Estimation of impact by third parties ( this technique can include an estimation of the
perceived impact of the intervention, by participants, participants supervisors, participants
managers)
This technique provides participants with a questionnaire, and based on the
improvement ask them to respond to the following:
I. What percent of this improvement can be attributed to the application of
skills/techniques/knowledge gained in the training program?
II. What is the basis for this estimation?
III. What confidence do you have in this estimate, expressed as a percentage?
IV. What other factors contributed to this improvement in performance?
V. What other individuals or groups could estimate this percentage or determine
the amount?
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 54/75
Estimation of impact by third parties – how do you increase the confidence level for this
technique?
I. Individuals who do not respond to the questionnaire or provide non- usable data
on the questionnaire are assumed to report no improvement/impact;
II. Extreme data and unrealistic claims are omitted from the analysis;
III. Only annualized values are used, it is assumed that there are no benefits fromthe program after the first year of implementation;
IV. The confidence level, expressed as a percent, is multiplied by the improvement
value to reduce the amount of the improvement by the potential error;
V. If value is expressed it is also factored by the confidence value.
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 55/75
Advantages Disadvantages
I. Easy to implement;
II. Based on the assumption that participants
are capable of determining or estimating
impact;
III. Although an estimate, this value will usually
have considerable credibility with
management because participants are at the
centre of the change or improvement;
IV. High face validity.
I. Less accurate than control groups
II. Based on perceptions – could be
regarded as Return on Expectation
rather than ROI
III. Could become victim of “group think”
Estimation Of Impact By Third Parties - Advantages And Disadvantages
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 56/75
Example from an actual coaching intervention- Not from our central case study…
All indicators
were pre
agreed with the
client
Participantsand
management
agreed which
the respective
groups will
rate
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 57/75
Step 2b: Planning the Intervention (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 58/75
Note – because the techniques related to isolating the impact of other factors are
not used often it is only referred to in this toolkit…
a. Trend line analysis of performance
data;
b. Control groups;
c. Participant’s estimation of impact;
d. Supervisor’s estimation of impact;
e. Management’s estimation of impact;
f. Use of previous studies
g. Subordinate’s report of other factors;
h. Estimating the impact of other factors;
i. Use of customer input.
With these
techniques we
try to isolate the
effects of other
interventions
and then declare
the rest as dueto our effort
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 59/75
Step 3
Implementation and Data Gathering (With ROI in
Mind)
Implementation and Data Gathering (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 60/75
Referring back to our original example (turnover in graduate engineers) – let us assume the
following:
Mine A
Control
Correctpairing
Correctcontrol of
variables
Agreementon metrics
Mine B
Experiential
Correctpairing
Correctcontrol of
variables
Agreementon metrics
We opted for a control group
evaluation method
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 61/75
Implementation and Data Gathering (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 62/75
Impact
Transfer
-Evaluation completed
Learning -Evaluation completed
Reaction – Evaluation
completed
Referring back to our original example (turnover in graduate engineers) – let us assume the
following:
In the control group the
need for replacement of
graduate engineers also
down by 10%...possibly
because of anticipation
of changes in HR
approach
Implementation and Data Gathering (With ROI in Mind)
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 63/75
Impact
Transfer
-Evaluation completed
Learning -Evaluation completed
Reaction – Evaluation
completed
Referring back to our original example (turnover in graduate engineers) – let us assume the
following:
In the control group the
need for replacement of
graduate engineers also
down by 10%...possibly
because of anticipation
of changes in HR
approach
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 64/75
So because to the method we used we can assign the
change in te metric to our intervention, mainly because we
controlled major variables. We do this as follows…
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 65/75
Step 4
Converting the Change As Result of the
Intervention to Value and Calculating ROI
Step 4.a Converting the Change As Result of the
Intervention to Value
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 66/75
Referring back to our original example (turnover in graduate engineers)…this is how it is
dome…
Identify the unit of
improvement
In our case study:
Number of
replacements of
graduate engineers in
the past six months
Value per unit =
R 74964 (see slide 12
AND 13
Determine the
performance levelchange
Original replacements =
6Six months after the
intervention = 3
Improvement of 50%
Original cost of
replacementR 449784
Our latest cost of
replacement following
intervention
R 224892
Calculate theperformance
improvement
50% saving R 224892
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 67/75
Step 4.a Converting the Change As Result of the
Intervention to Value
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 68/75
Always try to link to a metric –
alternatively use the method of
Estimation Of Impact By Third Parties – See slide 55 and 56 (this method is also
referred to as Return on Expectation)
Step 4.a Converting the Change As Result of the
Intervention to Value
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 69/75
See whether one can link tangible values to these?
OUTPUT
Units Produced
Tons Manufactured
Items Assembled
Money Collected
Items Sold
Forms Processed
Loans Approved
Inventory Turnover
Patients Visited
Applications Processed
Tasks Completed
Output Per HourProductivity
New Accounts Generated
COSTS
Budget Variances
Unit Costs
Cost By Account
Variable Costs
Fixed Costs
Overhead Cost
Operating CostsNumber of Cost Reductions
Project Cost Savings
Accident Costs
Sales Expense
TIME
Equipment Downtime
Overtime
Time to Project Completion
Processing Time
Break in Time for New Employees
Learning Time
Meeting Schedules
Repair Time
Work Stoppages
Order Response
Late Reporting
Lost Time DaysQUALITY
Scrap
Waste
Rejects
Error Rates
Rework
Shortages
Product Defects
Deviation From Standard
Product FailuresInventory Adjustments
Time Card Corrections
Percent of Tasks Completed Properly
Number of Accidents
Step 4.a Converting the Change As Result of the
Intervention to Value
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 70/75
And these….
WORK HABITS
Absenteeism
Tardiness
Visits to the Dispensary
First Aid Treatments
Violations of Safety Rules
Number of Communication Break-downs
Excessive Breaks
Follow-Up
WORK CLIMATE/SATISFACTION
Number of Grievances
Number of Discrimination Charges
Employee Complaints
Job Satisfaction
Employee Turnover
Litigation
Organization Commitment
Employee Loyalty
Increased Confidence
CUSTOMER SERVICE
Customer Complaints
Customer Satisfaction
Customer Dissatisfaction
Customer Impressions
Customer Loyalty
Customer Retention
Customer Value
Lost Customers
EMPLOYEE DEVELOPMENT/ADVANCEMENT
Number of PromotionsNumber of Pay Increases
Number of Learning Programs Attended
Requests for Transfer
Performance Appraisal Ratings
Increases in Job Effectiveness
INITIATIVE/INNOVATION
Implementation of New Ideas
Successful Completion of Projects
Number of Suggestions Implemented
Setting Goals and Objectives
New Products and Services Developed
New Patents and Copyrights
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 71/75
Step 4.a Converting the Change As Result of the
Intervention to Value
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 72/75
In our case example we did determine the value of
the change and can therefore now move on to
calculate cost benefit and therefore also ROI
Step 4.b Calculating ROI
First thing to do now is to determine the cost of the intervention – again back to our main
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 73/75
First thing to do now is to determine the cost of the intervention – again back to our main
example …
Cost element Notes ZAR value (for the sake of our example)
Development Costs If you use more than one trainer or one
trainer on more than one project – pro-rate
R50,000
Program Materials R10,000
Instructor/Facilitator Costs If you use more than one trainer or one
trainer on more than one project – pro-rate
R10,000
Facilities Costs R10,000
Travel and lodging R10,000
Salaries and time loss of participants Will need to consult HR and line and pro-rate R50,000
Administration costs R5,000
Evaluation costs R 5,000
Total cost of the intervention R150,000
Step 4.b Calculating ROI
First thing to do now is to determine the cost of the intervention – again back to our main
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 74/75
First thing to do now is to determine the cost of the intervention again back to our main
example …
Benefit/Cost Ratio =Net Program Benefits (R224892- R150,000)
Program Costs (R150,000)
Ratio = 0.499928
Step 4.b Calculating ROI
First thing to do now is to determine the cost of the intervention – again back to our main
8/4/2019 ROI Toolkit
http://slidepdf.com/reader/full/roi-toolkit 75/75
Net Program Benefits (R224892- R150,000)
Program Costs (R150,000)
First thing to do now is to determine the cost of the intervention again back to our main
example …
Benefits ROI =X 100
1
Benefit ROI = 50% thus for
every R1 I spent on this
program you got your money
back through saving as well
as an additional 50 cents.