Anything, which Anything, which is not is not

evaluated, is evaluated, is not worth doingnot worth doing

Arpita SharmaArpita Sharma

Contents---Contents---

• TrainingTraining• EvaluationEvaluation• Training EvaluationTraining Evaluation• Types of Training EvaluationTypes of Training Evaluation• Qualitative and Quantitative Qualitative and Quantitative

methods of Evaluationmethods of Evaluation

TrainingTraining

Training consists largely of well organized opportunities for participants to Training consists largely of well organized opportunities for participants to acquire necessary understanding and skill.acquire necessary understanding and skill.

Lynton and Pareek Lynton and Pareek (1967)(1967)

Training as an act of increasing knowledge and skills of an employee.Training as an act of increasing knowledge and skills of an employee.

Flippo (1966)Flippo (1966)

Training as activities aimed to provide attitude, knowledge and skills Training as activities aimed to provide attitude, knowledge and skills required for employment in a particular occupation for exercising a required for employment in a particular occupation for exercising a function in any field of study. function in any field of study.

(ILO (ILO 1986)1986)

Characteristics of Characteristics of trainingtraining

• Aims to improve the jobAims to improve the job• Better productivityBetter productivity• Emphasis on applicationEmphasis on application• Involve learningInvolve learning• Learners are adults from diverse Learners are adults from diverse

backgroundsbackgrounds• Shorter durationShorter duration• Improve the performance of peopleImprove the performance of people

TrainingTraining

Confidence

CapacitiesSkills

Knowledge

EvaluationEvaluationEvaluation is systematic Evaluation is systematic

determination of determination of merit, worth, and merit, worth, and significance of something or significance of something or someone using criteriasomeone using criteria against a against a set of standards.set of standards.

It is a It is a processprocess of establishing a of establishing a worth of something.worth of something.

The The ‘worth’,‘worth’, which means the which means the value, value, merit or excellencemerit or excellence of the thing of the thing

Contii….Contii….

Evaluation is a Evaluation is a continuous effortcontinuous effort to to judge a judge a programme, activity or programme, activity or thingthing..

Is a Is a systematic instructional systematic instructional processprocess based on based on empirical dataempirical data..

It measure the extent of It measure the extent of achievement achievement of objectivesof objectives as well as as well as relevance relevance and quality of efforts made.and quality of efforts made.

Training evaluation

• Training evaluation ensures that whether candidates Training evaluation ensures that whether candidates are able to are able to implement their learningimplement their learning in their in their respective workplacesrespective workplaces, or to the , or to the regular work regular work routinesroutines..

• Addresses how one determines whether the Addresses how one determines whether the goals or goals or objectivesobjectives were met and what were met and what impact the trainingimpact the training had had on on actual performanceactual performance on the on the job or in the community.job or in the community.

Training Evaluation helps Training Evaluation helps ….….

• Evaluation helps to Evaluation helps to measure outcomemeasure outcome of training.of training.

• It helps to know both It helps to know both strengths and strengths and weaknessesweaknesses

• It helps in creating a It helps in creating a sense of sense of accountabilityaccountability among staff. among staff.

• It creates a It creates a culture of continuous culture of continuous improvementimprovement..

Purposes of Training EvaluationPurposes of Training Evaluation

Purposes of Training Purposes of Training EvaluationEvaluation

• FeedbackFeedback• ResearchResearch• Power gamesPower games• InterventionIntervention

Process of Training Process of Training EvaluationEvaluation

Process of Training Process of Training EvaluationEvaluation

• Before TrainingBefore Training • During TrainingDuring Training • After TrainingAfter Training

Models of Training Models of Training EvaluationEvaluation

• Objective oriented training Objective oriented training EvaluationEvaluation

• Decision oriented training Decision oriented training EvaluationEvaluation

• Transaction training EvaluationTransaction training Evaluation• Adversary training EvaluationAdversary training Evaluation• Illuminative training EvaluationIlluminative training Evaluation

Contii---Contii---1) Objective oriented Training Evaluation1) Objective oriented Training Evaluation• Tyler Tyler • Bench markBench mark

2) Decision oriented Training Evaluation2) Decision oriented Training Evaluation• Daniel StufflebeamDaniel Stufflebeam• Context- To help the objectives.Context- To help the objectives.• Input- To help shape the proposalsInput- To help shape the proposals• Process- To guide implementationProcess- To guide implementation• Product- To guide decision- makingProduct- To guide decision- making

3) Transaction Training 3) Transaction Training EvaluationEvaluation

• Robert Stake (1967)Robert Stake (1967)• Full description and judgment Full description and judgment

of programmeof programme

1.1. AntecedentAntecedent

2.2. TransactionsTransactions

3.3. OutcomeOutcome

Contii----Contii----

4) Adversary Training Evaluation4) Adversary Training EvaluationInformation for decision making

5) Illuminative Training Evaluation5) Illuminative Training Evaluation• Massachusetts(1969)• Instructional System- Formalized

plan and standard• Learning Milieu- Social,

Psychological

S.M.Israr

What can be evaluatedWhat can be evaluated

Remember 3 PsRemember 3 Ps

•The PlanThe Plan•The ProcessThe Process•The ProductThe Product

Evaluating the Training Evaluating the Training ProgrammeProgramme

• To To improve the programmeimprove the programme as a as a wholewhole

• To To improve major componentsimprove major components of of the training programmethe training programme

• To To increase the effectivenessincrease the effectiveness of of specific modules or sessionsspecific modules or sessions

Evaluating the Training Evaluating the Training ProcessProcess

• To improve the To improve the training climatetraining climate• To improve the To improve the training training

methodologiesmethodologies• To improve To improve effectiveness of the effectiveness of the

trainerstrainers team. team.

Evaluating the Training Evaluating the Training Outcome/ImpactOutcome/Impact

• To To increase learningincrease learning by individual by individual participantsparticipants

• To To increase use of learningincrease use of learning in in work performancework performance

• To contribute data to To contribute data to organizational effectivenessorganizational effectiveness

• To help the To help the organizational changeorganizational change

• To To maximize cost effectivenessmaximize cost effectiveness of of trainingtraining

• To To ensure post training supportensure post training support at workat work

• To identify To identify helping and hindering helping and hindering factorsfactors

Evaluating Post Training Evaluating Post Training FactorsFactors

Typology of Training Evaluation

• According to level of Training Evaluation

• According to stages of Training Process

According to level of Training Evaluation

1) Reaction Level2) Learning Level3) Job behavior Level

According to stages of training Process

• Evaluation for planning• Process Evaluation• Terminal Evaluation

Types of Training evaluationTypes of Training evaluation

Formative Evaluation Formative Evaluation

• Takes place Takes place prior to or duringprior to or during instruction.instruction.

• It provides It provides ongoing feedbackongoing feedback to the to the curriculum designers and curriculum designers and developersdevelopers to ensure that to ensure that what is what is being createdbeing created really meets the really meets the needs needs of the intended audienceof the intended audience..

• Used to Used to improve the performanceimprove the performance..

Types of Formative Types of Formative EvaluationEvaluation

• Needs assessmentNeeds assessment• Evaluability assessmentEvaluability assessment• Implementation evaluationImplementation evaluation• Process evaluationProcess evaluation

Process EvaluationProcess Evaluation• Is conducted Is conducted during the during the

implementation phaseimplementation phase of training. of training.

• This includes This includes giving and receiving giving and receiving verbal feedbackverbal feedback. .

• It focuses on It focuses on procedures and actionsprocedures and actions being used to produce results. being used to produce results.

Training Need Training Need AssessmentAssessment

• Training need refer to the Training need refer to the gaps between gaps between desired and expected levels of desired and expected levels of performance.performance.

• Training need refer to those Training need refer to those needs that needs that concern performance of employees. concern performance of employees.

• Thus an employee needs training if Thus an employee needs training if there is gap in his there is gap in his attitude, knowledge, attitude, knowledge, understanding or skills related with understanding or skills related with work.work.

Performance AppraisalPerformance Appraisal

• It can be useful means of It can be useful means of establishing establishing effectively and relatively quickly the effectively and relatively quickly the training needstraining needs of individuals of individuals

• It is the process of It is the process of obtaining, analyzing, obtaining, analyzing, and recording informationand recording information about the about the relative relative worth worth of an employee to the of an employee to the organization organization

Need to Appraise PerformanceNeed to Appraise Performance• To assess the To assess the ability of the individualability of the individual in in

order to make the best use of his talent in the order to make the best use of his talent in the present jobpresent job

Task AnalysisTask Analysis

• Task comprises of a number of steps Task comprises of a number of steps or components.or components.

• Analyses each taskAnalyses each task• Breakdown each task into its Breakdown each task into its

constituents elementsconstituents elements• Assessing each taskAssessing each task

Gap analysisGap analysis

• Gap between what prospective Gap between what prospective trainees know and what they need to trainees know and what they need to know to function effectively in their know to function effectively in their job.job.

• What is and what should beWhat is and what should be

Summative EvaluationSummative Evaluation

• Summative evaluation is a method of Summative evaluation is a method of judging the worth of a programjudging the worth of a program at at the end of the program activities. The the end of the program activities. The focus is on the focus is on the outcomeoutcome ((Bhola 1990).).

• Effort are use information about Effort are use information about students’ feeling after a set of students’ feeling after a set of instructional activities has occurred.instructional activities has occurred.

Outcome EvaluationOutcome Evaluation

• It determines whether or not the desired It determines whether or not the desired results of applying new skills were achieved in results of applying new skills were achieved in the the short-term.short-term.

• "What happened to the knowledge, attitudes, "What happened to the knowledge, attitudes, and behaviors of the intended population?" and behaviors of the intended population?"

• Outcomes refer to Outcomes refer to changes in behaviorchanges in behavior• Outcome evaluation data is intended to Outcome evaluation data is intended to

measure what training participants were able measure what training participants were able to to do at the end of training and what they do at the end of training and what they actually did back on the jobactually did back on the job or in their or in their communitycommunity as a as a result of the trainingresult of the training. .

Impact EvaluationImpact Evaluation• Changes brought out due to Changes brought out due to application of application of

learninglearning at work can be measured only after at work can be measured only after lapse of some timelapse of some time after training. after training.

• Impact Evaluation takes even Impact Evaluation takes even longer than longer than outcomeoutcome evaluation and you may never know evaluation and you may never know for sure that your project helped bring about for sure that your project helped bring about the change.the change.

• Impact evaluation is meant to assess the Impact evaluation is meant to assess the extent to which extent to which what was learnedwhat was learned is making is making a difference at the a difference at the community level, or community level, or targeted groups, or beneficiaries of the targeted groups, or beneficiaries of the interventionintervention

Evaluation MethodsEvaluation Methods

• Qualitative (e.g., case studies, focus Qualitative (e.g., case studies, focus groups)groups)

• Quantitative (e.g., surveys, Quantitative (e.g., surveys, experiments). experiments).

Qualitative MethodsQualitative Methods

• Based on observationBased on observation• DescriptiveDescriptive• Easily doneEasily done• interactiveinteractive• Generating new product Generating new product

ideas/concepts ideas/concepts • Identifying key product attributes or Identifying key product attributes or

attitudes using focus groupsattitudes using focus groups

Quantitative evaluationQuantitative evaluation

• Based on dataBased on data• Precise and unbiasedPrecise and unbiased• Trained staff to collect, analyze and Trained staff to collect, analyze and

reportreport• Formal and totally structuredFormal and totally structured• Comparison of perceptions of Comparison of perceptions of

usefulness of trainingusefulness of training

Interview Schedule Interview Schedule

• Structured interviewStructured interview• Unstructured interviewUnstructured interview• Semi-Structured interviewSemi-Structured interview• Individual or Group interviewIndividual or Group interview

ObservationObservation• A basic tool of information gatheringA basic tool of information gathering• The act of apprehending things and The act of apprehending things and

events their attributesevents their attributes• Concrete relationshipConcrete relationship• Direct awareness of our own mental Direct awareness of our own mental

awarenessawareness• An act of recognizing and noting An act of recognizing and noting

some fact or occurrencesome fact or occurrence

Case StudyCase Study

• An An intense, qualitative and in-depthintense, qualitative and in-depth study of a study of a social unitsocial unit to gain to gain holistic holistic understandingunderstanding of the of the interrelationship interrelationship of various factorof various factor..

• Intensive investigationIntensive investigation• It may It may analyze a person, family or an analyze a person, family or an

institution, a cultural group or even institution, a cultural group or even entire communityentire community..

• Full analysis of a Full analysis of a limited member of limited member of eventsevents or or conditions and their conditions and their interrelationshipinterrelationship..

Techniques of EvaluationTechniques of EvaluationKnowledge TestKnowledge Test• Go through the content and Go through the content and

objectives of the courseobjectives of the course• Review the course materialsReview the course materials• Identify the area and sub areaIdentify the area and sub area• Decide on the weightageDecide on the weightage• Formulate different questionsFormulate different questions

Skill TestSkill Test• List sequence of the steps to be followed in List sequence of the steps to be followed in

performanceperformance• State activities in sequenceState activities in sequence• Make sure that all the observable activities Make sure that all the observable activities

necessary for performing, task is stated.necessary for performing, task is stated.• Decide on standards of performance-what Decide on standards of performance-what

would be excellent, good, fair or poor.would be excellent, good, fair or poor.• Determine scoring procedure which is Determine scoring procedure which is

quick and convenient.quick and convenient.

Evaluation WheelEvaluation Wheel

• Interactive techniqueInteractive technique

1)1) Trainer give an introduction about Trainer give an introduction about the training programme. He invites the training programme. He invites participants to list criteria on which participants to list criteria on which training should be evaluated.training should be evaluated.

2)2) Each of participant is given a piece Each of participant is given a piece of prepared marker pen. He has to of prepared marker pen. He has to draw a circle.draw a circle.

Conti----Conti----

3) He has to draw spokes from one 3) He has to draw spokes from one side of the wheel to another crossing side of the wheel to another crossing the radius.the radius.

4) Each participant has to rate each 4) Each participant has to rate each aspect 1-10.aspect 1-10.

5) Evaluation wheels can be pasted on 5) Evaluation wheels can be pasted on the wall and photographed for visual the wall and photographed for visual documentation.documentation.

Guided Discussion/Art from Guided Discussion/Art from ReflectionReflection

• Context or background narrationContext or background narration• Objective levelObjective level• Subjective levelSubjective level• Interpretative levelInterpretative level• Decision levelDecision level

Training Performance indicators

1) Training Participation Index (TPI) Ta/Ts x 100Ta= No. of trainees attending a training

courseTs= No. of trainees slotted for the course2) Training Utility Index (TUI)Te/Ta x 100Te= No. of trainees who found the

training course useful on evaluation

Contii--

• Training Effectiveness Index (TPI)Te/Ts x 100Te= No. of trainees who found the

training course useful on evaluationTEI=TPI X TUI

Training Cost indicators

1) Expenditure on training per extension personnel

=Total training expenditure/Total no. of extension personnel

2) Expenditure on training per Trainees

=Total expenditure on training/Total no. of extension trainees

Other extension cost Indicators

3)Training cost per training day= Total training cost/Total no. of training

day4)Training cost per extension trainees=Total training cost/Total no. of extension

trainees5)Training cost per extension personnel=Total no. of extension trainees/ total no.

of extension personnel

S.M.Israr

Benefits of EvaluationBenefits of Evaluation

• Improved quality of training activitiesImproved quality of training activities• Improved ability of the trainers to relate inputs Improved ability of the trainers to relate inputs

to outputsto outputs• Better discrimination of training activities Better discrimination of training activities

between those that are worthy of support and between those that are worthy of support and those that should be droppedthose that should be dropped

• Better integration of training offered and on-the Better integration of training offered and on-the job developmentjob development

• Better co-operation between trainers and line-Better co-operation between trainers and line-managers in the development of staffmanagers in the development of staff

• Evidence of the contribution that training and Evidence of the contribution that training and development are making to the organizationdevelopment are making to the organization

Limitations of Limitations of EvaluationsEvaluations

• It requires time resources and It requires time resources and continuous efforts.continuous efforts.

• It demands organized commitment.It demands organized commitment.• It creates threat among workers.It creates threat among workers.

A study on effectiveness of A study on effectiveness of trainings organized by trainings organized by

SAMETI- U.K.SAMETI- U.K.

• To measure the effectiveness of To measure the effectiveness of training programme.training programme.

• Measurement of effectiveness Measurement of effectiveness

1- Learning1- Learning

2-Reaction2-Reaction

Contii----Contii----

Their pre and post training Their pre and post training knowledge score indicates that knowledge score indicates that during pre training most of the during pre training most of the trainees had low level of knowledge. trainees had low level of knowledge. In post training session a high In post training session a high majority (70.58%) of the majority (70.58%) of the respondents had medium knowledge respondents had medium knowledge level whereas (8.83%) had gain high level whereas (8.83%) had gain high level of knowledge.level of knowledge.

Review of LiteratureReview of Literature

Pablico Pablico et al.et al. (1984) (1984) found that found that after post training written tests, in after post training written tests, in general the criticalgeneral the critical knowledge knowledge, , practical skillspractical skills and extension and extension communication techniquescommunication techniques of of trainees increased as a result of trainees increased as a result of attending training.attending training.

Contii---Contii---

Anupriya (1987)Anupriya (1987) reported that reported that proper selection of trainees and proper selection of trainees and trainers with regards to training trainers with regards to training course, better use of course, better use of communication materials, easy communication materials, easy availability of physical facilities and availability of physical facilities and more application oriented lecture more application oriented lecture adhered properly to the theme of adhered properly to the theme of coursecourse were lead to more effective were lead to more effective training programmes.training programmes.

Contii---Contii---

Choudhary (2000) Choudhary (2000) reported reported that knowledge was increased that knowledge was increased after training session in all age after training session in all age group, but lower age group group, but lower age group showed higher percentage of showed higher percentage of increase knowledge. increase knowledge.

Contii---Contii---

Sharma Sharma et al.et al. (2003) (2003) found that before found that before acquiring the training acquiring the training 93.88 % extension 93.88 % extension officer had low to medium level of officer had low to medium level of knowledgeknowledge about plant protection about plant protection measure. Only measure. Only 6.12% respondents had 6.12% respondents had high level knowledge.high level knowledge. However, after However, after completion of training programme completion of training programme 57.14 57.14 % respondents had high knowledge% respondents had high knowledge and only and only 12.24% respondents had low 12.24% respondents had low level of knowledge.level of knowledge.

Contii---Contii---

Sharma (2003)Sharma (2003) reported that reported that all the all the trainees were fully satisfied with course trainees were fully satisfied with course contents, message delivery and human contents, message delivery and human relation aspect of training.relation aspect of training. More than 80% More than 80% trainees were trainees were fully satisfied with the fully satisfied with the organization of courseorganization of course while 12.5% trainees while 12.5% trainees were partially satisfied. were partially satisfied. 80% trainees were 80% trainees were fully satisfied with the physical facilitiesfully satisfied with the physical facilities available,available, while 50% trainees were dissatisfied. while 50% trainees were dissatisfied. All the trainees were dissatisfied with the All the trainees were dissatisfied with the boarding and lodging facilities available at boarding and lodging facilities available at training centretraining centre

““Use evaluation to Use evaluation to improve situations but improve situations but

not simply to prove not simply to prove performance”performance”