Click icon to add picture

QUALITY ASSURING TECHNICAL -VOCATIONAL EDUCATION & TRAINING

DEMONSTRATING RETURN ON INVESTMENT (ROI)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

PRESENTATION OVERVIEW

DefinitionsKey FeaturesGlobalizationHistory of ROICalculating ROIIndicators of ROI

ROI in TVETOther ROI

IndicatorsEvaluation MethodsBenefits of ROISummaryConclusion

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

PRINCIPAL OBJECTIVES• Greater Awareness of ROI in TVET• ROI as strategic tool for HCD• ROI Influences resource allocation• Measures & Indicators of ROI • Evaluation Methodologies of ROI• Benefits of Evaluating ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHAT IS QUALITY ASSURANCE? “Programme for the Systematic

Monitoring and Evaluation of the various aspects of a Project, Service, or Facility to Ensure that Standards of Quality are met”

Merriam – Webster on-line Dictionary

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHAT IS QUALITY ASSURANCE ? “Part of Quality Management,

providing Confidence that Quality requirements (needed or expectations that are stated, generally implied, or obligatory) will be Fulfilled”

International Standards Organization (ISO)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY QUALITY ASSURE ?• Build & Maintain Organizational

Brand, Integrity and Reputation• Minimize mismatch & LM distortions• Create Competitive Advantage• Provide Superior Utility Value • Gain/retain Customer Affirmation

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

KEY ELEMENTS OF QA SYSTEM• Planning – goal setting • Implementation – guiding SOPs • Evaluation – monitoring

mechanism• Review – analyse/compare results (EQAVET)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVETGlobalization defines 21st century econ. Skills = currency of LM transactions, certification => instrument of trade, to navigate LM. Productivity determined by competence of employees as well as standardization of production systems.

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVET• In final analysis, businesses

compete for markets in the international arena

• In this environment (NWO), labour becomes internationalized through standardization and portability

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVET Against this background, modern

and efficient TVET Quality Management System becomes an imperative, especially within context of Resource Constraints, Programme Relevance and International Competitiveness

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

QUALITY ASSURING TVET

Competitive Forces Shape Strategy:• Structures, Systems & Processes • Facilities, Standards & Procedures• Monitoring & Evaluation • Technology - Leadership & People • Value Proposition

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

TRAINING & ORG PERFORMANCE Why invest in TVET? Correlation

training performance?• Body of literature that links training

with organizational performance• ILO links skills development (TVET)

with organizational productivity

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

TRAINING & ORG PERFORMANCE• Through training HR is

transformed into HC => greater value-added

• Firms invest in HR, create distinctive competence and build org capacity - even without knowing the ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MEASURING ROI Calculating ROI ranges from a simple

ratio: [Benefits/Costs] x 100 = %ROI To more complex means such as; Pay

Back - months/years before benefits match costs incurred

Note: shorter the pay back period, more feasible the investment

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MEASURING ROI• Sophisticated models - Computer

applications, where I/O variables are processed and results generated

• ROI when indirect variables are less quantifiable => challenging exercise

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI (1) Productivity Improvement:• Improved work methods =>

reduced human effort• Higher skill levels => faster work,

reduced Cycle Time• Greater intrinsic motivation =>

improved morale, greater output

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI (2) Labour Savings:• Less duplication of effort• Fewer mistakes => less rework =>

higher quality => greater productivity• Faster access to information• Critical differentiator, provides

competitive advantage

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI (3) Other Cost Savings:• Fewer machine breakdowns => lower

maintenance costs => greater profit• Lower Staff T/O => lower HR cost• Fewer customer complaints, greater

loyalty => higher retention rate

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATING ROI IN TVET• ROI in TVET means; measuring

all economic returns generated from investing in training

• Comparing results with the true cost of training programme

• Determine average annual ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET ?• Evaluating ROI is not an audit of

training, neither cost saving• Rather an objective evaluation and

re-engineering of training programmes

• Tool to ensure that future training is targeted and effective

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET?

Rationalization of Scarce Resourc

es

Provides Scientifi

c Evidenc

e of Outcom

es

Facilitates Career Advance

ment

Facilitates Comparative Analysis

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET?• Like other investments, QA TVET

consumes resources (cost), it also produces benefits

• Reasonable rate of return (for risk) is therefore an expectation

• Probably more talked about, than actually done!

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

HISTORY OF ROI IN TRAINING Kirkpatrick’s 4 - level model1. Reaction – the degree of favourable

reaction to training intervention2. Learning – degree to which intended

competence (KSA) is developed after training intervention

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

HISTORY OF ROI IN TRAINING3. Behaviour – degree of

application, post training intervention4. Results – degree to which

targeted outcomes are achieved (ROI)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EQAVET INDICATORS OF ROI• Programme Participation Rate• Completion Rate (Certification)• Employment Rate (skills utilization)• Productivity Improvement• Unemployment Rate (decline)• Organizational Competitiveness

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

OTHER INDICATORS OF ROI Individual level:• Enhanced Reflective Capacity• Critical Thinking & Analytical Skills• Problem Solving Capability• Continuous Improvement• Autonomous & Accountability

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATION METHODOLOGIES• Several methods of evaluating

ROI• Method used depends on

objectives of ROI evaluation, examples:

• Control Group, Modelling, and Estimating

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

CONTROL GROUPS• High validity in terms of isolating

effects of training programme• One group participates, other does

not (similar in all other respects)• Random selection techniques• Disadvantage - some staff do not get

benefit of the training programme

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MODELLING• Analytical & mathematical, non-

linear relationships• Several variables influence results,

calls for sophisticated stats models• Appropriate data & models

available, => accurate predictor of performance variables

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

ESTIMATION• Participants state how much of

their improvement due to training prog.

• Adjusted for level of confidence (60% x 70% con = 42%)

• Validate – supervisors & peers asked

to weight estimates = > 360 -degree

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATION CRITERIA• Evaluation criteria classified as

either hard or soft data• Hard: quantitative - easily

convertible into monetary terms• Soft: qualitative - from HR training

database, not easily convertible $$

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EXAMPLE - EVALUATION CRITERIA H A R D D A T A

Production outputUnits sold Products rejectedScrap produced Labour productivityNumber of

accidents

S O F T D A T A

Work ethicsAbsenteeismWork climate Social TensionWork attitudeStaff

development

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

BENEFITS OF EVALUATING ROI• Validates training as a business tool for

improving performance & profit• Justifies training costs incurred by HR• Improvement in training prog design• Rational selection of training delivery

methodologies

SUMMARY & CONCLUSION• Globalization drives International

Competitiveness• TVET’S Relevance linked to QA• Demonstrating ROI Strategic Tool• Comparative Analysis • Capacity Building & HCD

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

ACKNOWLEDGEMENTS• European Quality Assurance in

Vocational Education & Training (EQAVET)

• International Labour Organization (ILO)

• Naish, Richard: ROI – Building Business case for people development

• Kirkpatrick, Donald: Evaluating Training Prog. THANKS FOR YOUR ATTENTION !!