L+D ROI

Tony Wiggins, Principal Human Resource Tony Wiggins, Principal Human Resource Officer (Learning and Development)Officer (Learning and Development)

Strengthening your position as a people oriented employer through

learning and development

Queensland Building Services Authority (BSA) Queensland Building Services Authority (BSA)


Nature of Queensland Building Services Authority

plays crucial role in the residential building and construction industry in Queensland

provides remedies for defective work, support, education and advice for both community and industry stakeholders

led significant changes in the regulation of the building industry both in Queensland and Australia over the past 3 years


People and Location

230 employees Head Office – South Brisbane 7 Regional Offices Learning and Development function located in Human

Resource Services team located in Head Office.


Stimulus for Change

• Out of fit being reactive • Industry driven• Divided leadership team


Engaging community + building industry

Platform for change underpinned by systems theory approach1. Commitment to total quality management2. Customer value3. Leadership

Solution = Integrated Systems Framework (ISF) 2002

4. Score card approach (Kaplan and Norton, 1992) 2004


Integrated Systems FrameworkIntegrated Systems Framework

BUSINESS APPROACH

Corporate DirectionExecutive

Quality Philosophy

Drive

Customer Value

Framework

Guided Self Assessment

People

Quality Council

Well directed learning and development

Improvement TeamsCustomer Satisfaction

Quality Assurance Certification

Team Approach


Integrated Systems FrameworkIntegrated Systems Framework

• Philosophy of a Smart State workforce Philosophy of a Smart State workforce • Platform for learning delivery - Platform for learning delivery - Learning and Development

Initiative (LDI). • Aligned to supporting individual and organisational Aligned to supporting individual and organisational

performance performance • Learning seen as ‘silent’ driver for change Learning seen as ‘silent’ driver for change • Strengthen our position as a people-oriented employer Strengthen our position as a people-oriented employer


Learning and Development InitiativeLearning and Development Initiative

Leadership (Building Organisational Capability)

Relationship Management (Improving and Responding to Current and Future Relationships)

People Development (Developing, Valuing and Involving Staff)


Stimulus for Change


BenchmarkingBenchmarkingKirkpatrick’s Model was utilised as platform to define parameters at 3distinct levels – Benchmarking Research

evidence based research at Princess Alexandra Hospital, Brisbane

Jack Phillips and Ron Stone model focusing on Return on Investment strategies at Level 5 evaluation

Completion of Masters level studies in the area of organisational development and training


To measure the success of this lifelong learning and its impact on stakeholders’ satisfaction in the community and building industry, a research based evaluation model has been developed with strategic links to business outcomes.


Achieving a ROI ModelAchieving a ROI Model

• Kirkpatrick’s Four Level Evaluation Model • Extrapolated to a fifth level - Phillips and Stone • Consistent valid methodology - identified and developed

performance indicators reflecting BSA commitment to community and industry stakeholders

• Clear demonstration of impact on business outcomes - customer satisfaction.


44

ROI model• Research based • Masters • Kirkpatrick Model• Extrapolated to work Phillips and Stone


Levels of EvaluationPercentage of Programs Evaluated

Research Model BSA Model

Kirkpatrick

Level 1 Reaction and Planned Action

100% 100%

Level 2 Learning 75-90% 75%

Level 3 Job Applications 40-70% 50%

Level 4 Business Impact 0-20% 20%

Phillips and Stone

Level 5 Return on Investment 0-20% 20%

Levels of EvaluationLevels of Evaluation


Data Collection Methodology

Level 1 Course Evaluation Level 1 Course Evaluation Emphasis on relevancy of knowledge and skills Emphasis on relevancy of knowledge and skills

transference back to the workplace.transference back to the workplace.

Level 2 Pre and Post Assessment AnalysisLevel 2 Pre and Post Assessment Analysis Changed attitudes, improved knowledge and increased skillChanged attitudes, improved knowledge and increased skill

Level 3 Transfer of Learning Level 3 Transfer of Learning Change in behaviour - testing the participants’ capabilities Change in behaviour - testing the participants’ capabilities

to perform the learned skills back in the workplaceto perform the learned skills back in the workplace


Developing the ROI Formula Adapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) Adapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) The Human Resources Scorecard: Measuring the Return on InvestmentThe Human Resources Scorecard: Measuring the Return on Investment. Boston, . Boston,

MA: Butterworth-HeinemannMA: Butterworth-Heinemann

Develop EvaluationPlans and

Baseline Data

Develop EvaluationPlans and

Baseline Data

Stage 1Evaluation Planning

Stage 2 Data Collection

Collect DataDuring SolutionImplementation

Collect DataDuring SolutionImplementation

Collect DataAfter Solution

Implementation

Collect DataAfter Solution

Implementation

DevelopObjectivesOf Solution

DevelopObjectivesOf Solution

Level 1

Level 2

Level 3

Level 4


TabulateCosts

Of Solution

TabulateCosts

Of Solution

Isolate theEffects ofSolution

Isolate theEffects ofSolution

Convert Data to Monetary

Value

Convert Data to Monetary

Value

Calculatethe Return On

Investment

Calculatethe Return On

Investment

Implement Communication

Process

Implement Communication

Process

Stage 3 Data Analysis

Identify Intangibles

Identify Intangibles

Stage 4 Communicate Results

Level 5

Intangible Benefits


Data Collection Methodology

Level 4 and Level 5 EvaluationLevel 4 and Level 5 Evaluation Business impact - building relationships the community and Business impact - building relationships the community and

industry stakeholdersindustry stakeholders Tracking improvements in strategic courses such Tracking improvements in strategic courses such

leadership development program and Creating Business leadership development program and Creating Business Relationship program Relationship program


Developing the ROI Formula

ROI formula developed form the planned evaluation framework- strategies based upon research by Phillip and Stone

Stage 1 Determining the current market return on cashinvested

Stage 2 Identifying organisational factors impacting on the learningoutcomes

Stage 3 Outcome and Baseline Data Questionnaire developed .


LDI scorecards include seven categories of data.

• Indicators• Reaction and Planned Action• Learning• Application• Business Impact• ROI• Intangibles


LDI scorecards

Level

1. Reaction & Planned Action

2. Learning

3. Application

4. Impact

5. Return on Investment

Measurement Focus

Measures participant satisfaction with the program and captures planned actions

Measures changes in knowledge, skills, and attitudes

Measures changes in on-the-job behavior

Measures changes in business measures

Compares program benefits to the costs

Econ

omic

Syst

emIn

divid

ual


Current Results

• Solid evidence of change• Number of scientific requirements satisfied - credible• Significant correlation - learning when expertly designed

and delivered reflected in positive business outcomes• estimated ROI results obtained varies• Example - Mediation Skills Program - 2 years - generated

significant financial returns of $1.89 million


The FutureThe Future

• ROI – sixth level – measure intangibles• ROI – Team Building Interventions• Validating model through further research

L+D ROI

Documents

Transcript of L+D ROI