Using RUBRICS to Assess Program Learning Outcomes By Dr. Ibrahim Al-Jabri Director, Program...

Using RUBRICS to Assess Using RUBRICS to Assess Program Learning OutcomesProgram Learning Outcomes

ByByDr. Ibrahim Al-JabriDr. Ibrahim Al-Jabri

Director, Program Assessment CenterDirector, Program Assessment Center

April 16, 2007April 16, 2007

OUTLINEOUTLINE

• Questions??• Assessment Cycle• Direct & Indirect Measures• What is rubric?• Type of rubrics• Constructing Rubrics• Using Rubrics• Advantages of Rubric• Examples

Questions??Questions??

• What do we want students learn?

• Do students have the opportunity to learn it?

• How do we know that they learned it?

• What do we do with that information?

Assessment CycleAssessment Cycle

Write expected outcomes/objectives

Effect improvements through actions

View results Assess performance against criteria

Ensure students have opportunity to learn

Direct and Indirect Measures of Direct and Indirect Measures of Student LearningStudent Learning

• Direct measures:– “… tangible, visible, self-explanatory evidence

of exactly what students have and haven’t learned” (Suskie, 2004, p. 95).

• Indirect measures:– “… signs of what students are learning, but

the evidence of what they are learning is less clear and less convincing” (Suskie, 2004, p. 95).

Direct and Indirect Measures of Direct and Indirect Measures of Student LearningStudent Learning

• Direct– Tests– Rubrics– Portfolios– Capstone projects– Field supervisor

ratings– Employer ratings

• Indirect– Course grades– Surveys– Focus Groups– Student self ratings– Course evaluations– Graduate school

admissions

One Useful Direct Measure of One Useful Direct Measure of Student Learning in Courses Student Learning in Courses

or Programs is Rubricor Programs is Rubric

What is Rubric?What is Rubric?

Rubrics offer the Instructor an opportunity to assess the student's understanding of a scientific topic by levels of performance on certain criteria

Type of rubricsType of rubrics

Rubrics may be used “holistically” or “analytically”…

• “Holistic” Rubric:

– The entire response is evaluated and scored as a single performance category

• “Analytical” Rubric:

– The response is evaluated with multiple descriptive criteria for multiple performance categories

Template for analytic rubrics

Beginning1

Developing2

Accomplished3

Exemplary4

Score

Criteria #1

Description reflecting beginning level of performance

Description reflecting movement toward mastery level of performance

Description reflecting achievement of mastery level of performance

Description reflecting highest level of performance

Criteria #2





Criteria #3





Proficient- 3 points

The student's project has a hypothesis, a procedure, collected data, and analyzed results. The project is thorough and the findings are in agreement with the data collected. There are minor inaccuracies that do not affect the quality of the project.

Adequate- 2 points

The student's project may have a hypothesis, a procedure, collected data, and analyzed results. The project is not as thorough as it could be; there are a few overlooked areas. The project has a few inaccuracies that affect the quality of the project.

Limited

- 1 point

The student's project may have a hypothesis, a procedure, collected data, and analyzed results. The project has several inaccuracies that affect the quality of the project.

Example for holistic rubrics

Constructing a RubricConstructing a Rubric

– Know the goals for instruction- what are the learning outcomes?

– Decide on the structure of the rubric- holistic or analytical--what fits best for the task?

– Determine the levels of performance- are there levels of performance specific to each criteria?

– Share the rubric with your students-students should have an opportunity to see, discuss or even design the rubric.

What are the steps in creating a rubric?What are the steps in creating a rubric?

1.Define the focus / topic / task / learning objective

2.List the criteria for assessing the performance• 3 - 8 characteristics• Clearly written with action verbs

3.Write descriptions for the levels of performance• Usually 3 - 5 levels• Define top level first, then bottom level, and finally middle

level(s)

4.Test rubric

Using RubricsUsing Rubrics

Useful in other ways:• Self-assessment• Peer evaluations• Analysis of electronic

discussion threads• Employer ratings of

recent graduates

• Program assessment, giving direct evidence of student learning – most common use

• Embedded course assignments

• Capstone experiences• Field experiences• Portfolios

Using rubrics in assessmentUsing rubrics in assessment

• Formative assessment: Rubrics can be used as ongoing ways of measuring student progress. This feedback provides direction for further teaching.

• Summative assessment: Rubrics can be used to measure achievement of specific objectives, competencies, or learning goals.

Advantages of RubricsAdvantages of Rubrics

• Help students know what to expect

• Provide detailed feedback

• Help students see improvement over time

• Offer guidance for continuous improvement

• Align teaching and learning to standards

ExamplesExamples

Oral Presentation Rubric

Criteria 0 1 2 3

Eye ContactDoes not attempt to look at audience at all, reads notes the entire time

Only focuses attention to one particular part of the class, does not scan audience

Occasionally looks at someone or some groups during presentation

Constantly looks at someone or some groups at all times

Facial Expressions

Has either a deadpan expression of shows a conflicting expression during entire presentation

Occasionally displays both a deadpan and conflicting expression during presentation

Occasionally demonstrates either a deadpan OR conflicting expression during presentation

Gives audience clues to what the content of speech is about; Appropriate expression, never notice a deadpan or conflicting expression

EnthusiasmShows absolutely no interest in topic presented

Shows some negativity toward topic presented

Occasionally shows positive feelings about topic

Demonstrates a strong positive feeling about topic during entire presentation

Visual AidPoor, distracts audience and is hard to read

Adds nothing to presentation

Thoughts articulated clearly, but not engaging

Visual aid enhances presentation, all thoughts articulated and keeps interest

Completeness of Content

One or more points left out

Majority of points glossed over

Majority of points covered in depth, some points glossed over

Thoroughly explains all points

Professionalism of Presentation

Audience has difficulty hearing, confusing

Thoughts don't flow, not clear, does not engage audience

Thoughts articulated clearly, though does not engage audience

Presentation is organized and the interest level of the audience is maintained

Thank YouThank You

Using RUBRICS to Assess Program Learning Outcomes By Dr. Ibrahim Al-Jabri Director, Program...

Documents

Transcript of Using RUBRICS to Assess Program Learning Outcomes By Dr. Ibrahim Al-Jabri Director, Program...