Transformation:The Rubric Paradigm

Gary BrownThe Center for Teaching, Learning & Technology

Copyright Statement

• Copyright Gary Brown, 2002. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

We should not expect the guidance for change of this magnitude—in institutional culture and values—to come from the faculty ranks. After all, faculty are deeply rooted in the traditional values of higher education. Fundamentally, this is a leadership issue.                          --Carole Barone, Vice President

Educause

Student Enrollments in WSU Online Learning Spaces

0

4000

8000

12000

16000

20000

1998 1999 2000 2001 2002

The Goals, Activities, and Practices

• Invite faculty to join in the process of formulating the assessment.

• A series of two short, online surveys designed to provide formative assessment: one instructor survey and one student surveys.

The Assessment Gold Standard

Participants Who Used Data to Inform Change

563

10655 16 2

0

300

600

Invited faculty Faculty who tookinstructor survey

Faculty w/ studentresponses

Faculty w/ >50%student response

rate

Faculty who usedthe data

Transformation?

• Rubrics—a road map AND a compass• Help faculty grade student performance• Help students understand expectations• Provide measures of growth• Help inform policy

What is a rubric?

The Rubric Paradigm

Critical Thinking and Measures of Growth

Scant _____________________________ Substantial

Identifies and summarizes the problem/question at issue (and/or the source's position).

Does not identify and summarize the problem, is confused or identifies a different and inappropriate problem.

Does not identify or is confused by the issue, or represents the issue inaccurately.

Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem, and identifies them clearly, addressing their relationships to each other.

 

Identifies not only the basics of the issue, but recognizes

nuances of the issue.

7 Dimensions of Critical Thinking1. Identifies and summarizes the problem/question at issue (and/or

the source's position).

2. Identifies and presents the STUDENT’S OWN perspective and position as it is important to the analysis of the issue.

3. Identifies and considers OTHER salient perspectives and positions that are important to the analysis of the issue.

4. Identifies and assesses the key assumptions.

5. Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue.

6. Identifies and considers the influence of the context * on the issue.

7. Identifies and assesses conclusions, implications and consequences.

Critical Thinking:one course/two semesters

0

1

2

3

4

5

6

Prob

lem

Stu

dent

View

Oth

er V

iew

s

Cont

ext

Evid

ence

Ass

umpt

ions

Impl

icat

ions

Ave

rage

Entomology--spring 99 Entomology--spring 00

One Particular Finding

• The faculty questionnaire revealed a singular focus on grading over fostering critical thinking for broader life-long learning.

• If we want our constituencies to value what we do, we might consider how to make what we do more valuable….

Dimensions of Transformation

1. Purpose2. Data 3. Application4. Dissemination

Premise—”High Standards” is not the same as standardization

Criteria for Prioritizing Scoring Form

Rater: Project:

* Faculty * Designer

* Student * Assessment Specialist

* Community Colleague * Administrator

* Other _________

Rating

Assessment Purpose

Data & Data Acquisition

Application

Dissemination

Bonus—Faculty Rewarded for Assessment

Total Score

Comments:

The Scoring Form

U#1 U#2 U#3 U#4 U#5 DimensionAverage

Purpose 3 3.1 3.6 4 3 3.34Data 2.4 3.5 3.8 3.5 3 3.24Application 2.5 3.4 3.7 3.5 4.5 3.52Dissemination 3.3 3 3.5 2 4.5 3.26University Average

2.8 3.3 3.7 3.3 3.8 3.343.38

Pilot Findings

Pilot Findings

0

1

2

3

4

5

6

U # 1

U # 2

U # 3

U # 4

U # 5

Reviewers’ SuggestionsU#1

• Include faculty more in this process; audience is clearly the upper echelon of the institution and future employers.

Reviewers’ Suggestions U#2

• Not many rewards for assessment.

• No feedback to faculty and students.

Reviewers’ Suggestions U#3

• Assesses key concern: learning to learn.

• No plan to apply or fund results of findings. Faculty support neglected.

Reviewers’ Suggestions U#4

• Assumes that tech alone causes transformation

Reviewers’ Suggestions U#5

• Good start but…