Utilizing Kirkpatrick’s “Levels of Evaluation”

Why Evaluate?

Evaluation is the engine the powers the vehicles of innovation and progress

Without evaluating the effectiveness of a product or a program improvements will never be made

Utilizing Kirkpatrick’s “four levels of evaluation” best ensures an instructor is meeting the educational needs of his or her students in an object-driven, constructivist classroom.

Setting

6th Grade Social Studies Classroom in Virginia

Constructivist Classroom

Objective-driven : VA DOE Standards of Learning

United States History to 1865

American Indians to the Civil War

Level 1 - Reaction

Reaction measures how those who participated in the program react to it

The instruction needs to be interesting and relevant so the learner will have a favorable reaction to it

Kirkpatrick believes positive reaction may not single-handedly ensure learning, but negative reaction almost certainly reduces the possibility of it occurring.

Classroom Example

Standard USI.9b

“the student will demonstrate knowledge of the causes, major events, and effects of the Civil War by explaining how the issues of states’ rights and slavery increased sectional tensions” (Virginia Department of Education, 2008)

Compromises on slavery in the western states

Negative Reaction

Too much info in too little space

Picture is faded and difficult to interpret

When a discussion question is posed there is not enough room for the learner to write his or her response

Positive Reaction

Considers good message design principles

Simplistic and structured

An adequate amount of information is presented

A later activity could focus on the map visualization

Level 2 - Learning

Learning is defined as the extent to which participants change attitudes, improve knowledge, and/or increase skill as a result of instruction

Lessons can focus on knowledge, skills, attitudes, or a combination of each.  In a school setting a combination of the three is the focus for most lesson plans.

Starting Point - Preassessments Taken before each

new unit of study

Not graded

Use of ActivExpressions

Results can be stored for future comparisons

Along the Way – Formative Assessment

“Checkmarks”

Immediate feedback

Homework, classroom activities, class discussions

At the End – Achievement Tests The test should cover the concepts

presented to the students

The test should be consistent with the standards set forth by the State Department of Education

Comparison of preassessments and achievement tests will show if learning took place

Level 3 - Behavior

Behavior is defined as the extent to which change in behavior has occurred because the participant during instruction

Difficult to measure behavior

The learning shown in the classroom does not guarantee the student has mastered the knowledge, skill, or attitude.

What Can the Teacher Do? In order to create an ideal learning

environment a teacher needs to create either an encouraging environment or a requiring environment.   An encouraging environment is the teacher

encouraging the student to use new knowledge, skills, and attitudes in the classroom. 

A requiring environment is the teacher monitoring and requiring the use of new knowledge, skills, and attitudes in the classroom

Classroom Example

Standard USI.8a

states “the student will demonstrate knowledge of westward expansion and reform in America from 1801 to 1861 by describing territorial expansion and how it affected the political map of the United States, with emphasis on the Louisiana Purchase, the Lewis and Clark expedition, and the acquisitions of Florida, Texas, Oregon, and California” (Virginia Department Of Education, 2008)

End of the Unit Project showing change in behavior

Western Territories Project

Western Territories Project

Western Territories Project

Level 4 - Results

Results is defined as the final results that occurred because the student attended instruction

Best evaluated by the experience of a student from one grade to the next

Many extraneous factors other than training can influence results outcomes so it’s difficult to validate the relationship between training and results

Student participation, parental expectations, and resource availability among other things

Results

The results level in a school setting is creating a better student who uses the knowledge, skills, and attitudes he or she has learned during instruction to be a productive member of society

The easiest way to measure results is whether or not a student graduates from high school or a higher level of education

Determining whether or not a student has become a productive member of society is so much harder to evaluate

Critiques of Kirkpatrick

Kaufman and Keller’s Expansion

Conclusion

Kirkpatrick’s “four levels of evaluation” best ensure an instructor is meeting the educational needs of his or her students in an object-driven, constructivist classroom.

If followed effectively each level will force the qualified instructor to reflect and determine what changes could lead to better performance by students at different stages of learning.

Experience can play an important role in the effectiveness of evaluation.

The most important attribute for an effective instructor is the ability to realize that evaluation of instruction has the biggest effect on the progress and success of the educational system.