Measuring Learning OutcomesMeasuring Learning OutcomesEvaluationEvaluation

ObjectivesObjectives::By the end of the lecture, the student will be By the end of the lecture, the student will be

able to:able to: Define evaluation in educationDefine evaluation in education State purposes of evaluation (why we evaluate?)State purposes of evaluation (why we evaluate?) List five basic components of evaluationList five basic components of evaluation State what we want to evaluate?State what we want to evaluate? Differentiate between evaluation and Differentiate between evaluation and

assessmentassessment List steps of student evaluationList steps of student evaluation Identify different types of evaluationIdentify different types of evaluation Select appropriate evaluation Methodologies Select appropriate evaluation Methodologies

according according educational objectives and domains educational objectives and domains to be evaluatedto be evaluated

Mention qualities of testMention qualities of test

IntroductionIntroduction

D-Implement Educational Program

G-Improvement the Educational Program

A-Needs Assessment

F-Feedback to 1- Learner

2- Teacher3-Organization

E-Evaluate the EducationalProgram

B-Educational objectives1- Cognitive

2- Psychomotor3-Attitude

C-Plan and Design Educational Program

1-Content 2-Method 3- Material

4-Evaluation methodology5-Environment

Definition of evaluationDefinition of evaluation

•Evaluation is defined as a systematic Evaluation is defined as a systematic process by which the process by which the worthworth or or valuevalue of of somethingsomething is is judged.judged.

Evaluation in education can be defined Evaluation in education can be defined as a continuous process based upon as a continuous process based upon criteria which cooperatively developed criteria which cooperatively developed (Table 7.1) and (Table 7.1) and concerned with concerned with measurement ofmeasurement of : :The performance of learners.The performance of learners.The effectiveness of teachersThe effectiveness of teachersThe quality of the programThe quality of the program

EvaluationEvaluation . . ...... . . . . . . ...... . . . .of whom?of whom? of what?of what?

StudentsStudentsTeachersTeachersProgrammes and courses Programmes and courses

in relation to what?in relation to what?

In relation to educational objectives. In relation to educational objectives.

Purposes of evaluation (why we evaluate?)Purposes of evaluation (why we evaluate?)

Incentive to Learn (motivation)Incentive to Learn (motivation)Feedback to studentFeedback to studentModification of Learning ActivitiesModification of Learning ActivitiesSelection of studentsSelection of studentsSuccess or failureSuccess or failureFeedback to teacherFeedback to teacherSchool public relationsSchool public relationsProtection of society Protection of society (certification of (certification of competence)competence)

components of evaluationcomponents of evaluationFive basic components of evaluation Five basic components of evaluation : : (1)(1) audience, audience, (2)(2) purpose, purpose, (3)(3) questions questions (4)(4) scope, and scope, and (5)(5) resources resources To determine these components, ask the following questions:To determine these components, ask the following questions:

For what For what audienceaudience is the evaluation being is the evaluation being conducted?conducted?

For what For what purposepurpose is the evaluation being is the evaluation being conducted?conducted?

What What questionsquestions will be asked in the will be asked in the evaluation?evaluation?

What is the What is the scopescope or the evaluation? or the evaluation? What What resourcesresources are available to conduct the are available to conduct the

evaluation?evaluation?

Different between evaluation &AssessmentDifferent between evaluation &Assessment

While assessment and evaluation are While assessment and evaluation are highly interrelated and are often used highly interrelated and are often used interchangeably as terms, interchangeably as terms, they are not they are not synonymous.synonymous. The process of The process of assessment is to gather, summarize, assessment is to gather, summarize, interpret, and use data interpret, and use data to decide a to decide a direction for actiondirection for action.. The process of The process of evaluation is to gather, summarize, evaluation is to gather, summarize, interpret, and use data interpret, and use data to determine the to determine the extent to which an action was extent to which an action was successful.successful.

Cont. Cont. Different between evaluation &AssessmentDifferent between evaluation &Assessment

The The primary differencesprimary differences between these two terms between these two terms are those of are those of timing and timing and purposepurpose.. For example, an For example, an education program begins with an assessmenteducation program begins with an assessment of learners' needsof learners' needs. . From the perspective of From the perspective of systems theory, systems theory, assessment dataassessment data might be called might be called the"inputthe"input."." While the program While the program is is being being conductedconducted,, periodic evaluationperiodic evaluation lets the educator lets the educator knowknow whether the program and learners are whether the program and learners are proceed as plannedproceed as planned. . After program completionAfter program completion evaluation identifies whether and to what evaluation identifies whether and to what extent the identified needs were met.extent the identified needs were met.

steps of student evaluationsteps of student evaluation

Four steps of student evaluationFour steps of student evaluationThe criteria (acceptable level of performance) of the The criteria (acceptable level of performance) of the

educational objectiveseducational objectives

Development and use of measuring instrumentsDevelopment and use of measuring instruments

Interpretation of measurement dataInterpretation of measurement data

FormulationFormulation of judgments and taking of appropriate of judgments and taking of appropriate actionaction

Types of EvaluationTypes of Evaluation

Five basic types of evaluation includeFive basic types of evaluation include: : 1.1. process, process, 2.2. content, content, 3.3. outcome, outcome, 4.4. impact, and impact, and 55.. program. program. These types ranged from the These types ranged from the simplesimple ( (process process

evaluationevaluation) to the ) to the complexcomplex ( (impact impact evaluation). evaluation).

Types of EvaluationTypes of Evaluation

Types of EvaluationTypes of Evaluation1. Process (Formative) Evaluation1. Process (Formative) Evaluation The purpose of this evaluation is to The purpose of this evaluation is to make adjustments make adjustments in an educational activityin an educational activity as soon as they are neededas soon as they are needed, , whether those adjustments be in personnel, materials, whether those adjustments be in personnel, materials, facilities learning objectives, or even attitude facilities learning objectives, or even attitude Adjustments may need to be Adjustments may need to be made after one class or made after one class or session before the next is taughtsession before the next is taught , so process evaluation , so process evaluation occurs occurs more frequently during and throughout every more frequently during and throughout every learning experience than any other type.learning experience than any other type. This ongoing evaluation prevent problems before This ongoing evaluation prevent problems before they occur or identify problems as they arisethey occur or identify problems as they arise

Types of EvaluationTypes of Evaluation2. Content Evaluation2. Content EvaluationThe purpose of content evaluation The purpose of content evaluation is to determine is to determine whether learners have acquired the knowledge or skills whether learners have acquired the knowledge or skills taught during the learning experience,taught during the learning experience, as taking place as taking place immediatelyimmediately after the learning experience to answer the after the learning experience to answer the question "question "To what degree did learners achieve specified To what degree did learners achieve specified objectives?"objectives?"Examples: Examples: - Asking a patient to give a return demonstration - Asking a patient to give a return demonstration immediately after observing itimmediately after observing it - Asking participants to complete a cognitive test - Asking participants to complete a cognitive test at the at the completion of a 1-day seminar. completion of a 1-day seminar. Content evaluation as the level is in Content evaluation as the level is in between process and between process and outcomeoutcome evaluation levels. In other words, content evaluation levels. In other words, content evaluation can be considered as evaluation can be considered as focusing on how the focusing on how the teaching-learning process affected teaching-learning process affected immediateimmediate, , short-term short-term outcomesoutcomes

Types of EvaluationTypes of Evaluation3. 3. Outcome (Summative) EvaluationOutcome (Summative) EvaluationThe purpose of outcome or summative evaluation The purpose of outcome or summative evaluation is to is to determine the effects or outcomes of teaching effortsdetermine the effects or outcomes of teaching efforts. Its . Its intent is intent is to sum what happened as a result of educationto sum what happened as a result of education. . Outcome evaluation occurs after teaching has been Outcome evaluation occurs after teaching has been completed or after a program has been carried out, it is completed or after a program has been carried out, it is focusing on measuring long-term change that persists focusing on measuring long-term change that persists after the learning experienceafter the learning experience..Guiding questions in out- come Guiding questions in out- come evaluation include the following:evaluation include the following:•• Was teaching appropriate?Was teaching appropriate?•• Did the individual(s) learn?Did the individual(s) learn?•• Were behavioural objectives met? Were behavioural objectives met?

Types of EvaluationTypes of Evaluation4. Impact Evaluation4. Impact EvaluationThe purpose of impact evaluation is to:- The purpose of impact evaluation is to:- Determine the relative Determine the relative effects of educationeffects of education on on the the institutioninstitution or the community. obtain or the community. obtain information that information that will help decide whetherwill help decide whether conducting an educational activity conducting an educational activity is worth its is worth its cost. cost. The scope of impact evaluation is broader, The scope of impact evaluation is broader, more more complex, and usually complex, and usually more long termmore long term than that than that of of process, content, or outcome process, content, or outcome evaluationevaluation Outcome evaluation would Outcome evaluation would focus onfocus on course course objectiveobjective,, whereas impact whereas impact evaluation wouldevaluation would focus onfocus on course goal course goal..

Types of EvaluationTypes of Evaluation

5. Program Evaluation5. Program EvaluationThe purpose of total program evaluation The purpose of total program evaluation is to is to determine the extent to which all activities for determine the extent to which all activities for an entire program over a specified period of an entire program over a specified period of time meet or exceed goals originally time meet or exceed goals originally established.established.Example Guiding question appropriate for a Example Guiding question appropriate for a total program evaluation from this total program evaluation from this perspective might be "perspective might be "To what extent did To what extent did programs undertaken by faculty members of programs undertaken by faculty members of the nursing college academic dethe nursing college academic de partments partments during the year accomplish annual goals during the year accomplish annual goals established by the departments?" established by the departments?"

Types of EvaluationTypes of Evaluation

Cont . Cont . Program EvaluationProgram Evaluation

The scope of program evaluation is broad, The scope of program evaluation is broad, generally focusing on generally focusing on overall goalsoverall goals rather rather than on than on specific objectivesspecific objectives..It is It is encompassing all aspects of educational encompassing all aspects of educational activityactivity (e.g., process, content, outcome, (e.g., process, content, outcome, impact) with input from all the participants impact) with input from all the participants (e.g., learn(e.g., learn ers, teachers, institutional ers, teachers, institutional representatives, comrepresentatives, com munity representatives). munity representatives). The time period over which data are The time period over which data are collected may collected may extend from several months to extend from several months to 1 or more years1 or more years, depending on the time frame , depending on the time frame established for meeting the goals to be established for meeting the goals to be evaluated.evaluated.

Evaluation MethodologiesEvaluation Methodologies Methods of evaluation must be selected according to educational objectives and Methods of evaluation must be selected according to educational objectives and

domains to be evaluated . Below are different evaluation methods for domains to be evaluated . Below are different evaluation methods for each educational domains .each educational domains . Cognitive DomainCognitive Domain

Written testWritten test Objective type ( Matching , Multiple choice , True & false , Short Objective type ( Matching , Multiple choice , True & false , Short

answers)answers) Subjective type (Long answer , Simulation)Subjective type (Long answer , Simulation)

Oral testOral test Observational rating scaleObservational rating scale QuestionnaireQuestionnaire

Attitude DomainAttitude Domain Direct observationDirect observation Rating scaleRating scale ChecklistChecklist

Psychomotor DomainPsychomotor Domain Direct observationDirect observation Practical testsPractical tests In real SituationIn real Situation In simulation conditionIn simulation condition

Qualities of a testQualities of a test

Directly related to educational objectivesDirectly related to educational objectives

Realistic & practicalRealistic & practical

Concerned with important & useful Concerned with important & useful

mattersmatters

Comprehensive but briefComprehensive but brief

Precise & clearPrecise & clear