Effective grading (and teaching) through the
use of rubrics & use of rubrics for
assessment
Sarah Murnen
Kenyon College
Personal Background Although a psychologist, no
particular background in this topic As Kenyon’s “Assessment
Coordinator” from 2002-2006 I learned that: Some faculty resent the assessment
process and see it as extra burden Departments with faculty who feel
burdened by assessment don’t get much out of it
Departments that find a way to integrate assessment with what they already do benefit from the process – they discuss what students are learning and change their teaching methods, assignments, and sometimes their entire curricula to help students learn
Benefits of Ohio-5 Project
To help improve student learning (and faculty teaching) by encouraging the use of rubrics (Many other benefits to the use of rubrics)
To specifically encourage a focus on creativity and critical thinking, important aspects of a liberal education
To use rubrics for assessing critical thinking and creativity
To show faculty how all of the work they put into grading can be used for assessment purposes
Background Information Presentation by Dr. Douglas Eder on
“Primary Trait Analysis” Presentation by Barbara Walvoord
on the use of rubrics for effective grading
Walvoord, B. E., P& Anderson, J. A. (1998). Effective Grading: A tool for learning and assessment. Jossey-Bass: San Francisco, CA.
Stevens & Levi (2005). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Stylus Publishing: Sterling, Virginia.
Our primary concern is student learning
Good practice in undergraduate education (Walvoord, 1998, p. 15): Encourages student-faculty contact Encourages cooperation among students Encourages active learning Gives prompt feedback Emphasizes the time that students devote
to the task Communicates high expectations Respects diverse talents and ways of
learning
Walvoord asks (1998, p. 15):
“How many of these principles of good practice in some way involve the grading system in your class, the tests and assignments on which that system is based, and your ways of communicating with students about their work and their grades?”
Walvoord’s Argument for Establishing Clear Criteria and Standards for Grading
Saves time in grading process Allows you to make the process consistent and fair Helps you explain to students what you expect Shows you what to teach Identifies essential relationship between discipline
information and processes Help students evaluate their own and each other’s
work Saves you from having to explain your criteria to
students after they have handed in their work, as a way of justifying the grades they are contesting
Helps student peers give each other constructive feedback on plans and drafts
Helps team teachers or teaching assistants grade student papers consistently
Helps teachers of sequenced courses communicate with each other about standards and criteria
Form the basis for departmental or institutional assessment
How to Establish Clear Criteria: Primary Trait Analysis Developed to score essays on the
National Assessment of Educational Progress by Lloyd-Jones in1977
Creates a scoring rubric that can be used to assess any student performance
Is assignment specific Can be used for grading For this project, we will develop
rubrics using PTA to measure the critical thinking and creativity we are encouraging in our students
Why take the time to do PTA for grading? Makes grading more consistent
and fair Saves time in the grading
process once the rubric is developed
Can diagnose students’ strengths and weaknesses very specifically in order to teach more effectively
Can track changes in student performance
Process of PTA (Eder)
• Identify the “primary traits” – essential or central components of the discipline to be learned by the student
• Build a scale for scoring the student’s performance on the trait
• Evaluate the student’s performance against those criteria
Key Stages in Constructing a Rubric (Stevens & Levi)1. Reflecting -
• What do we want from our students?• Why did we create the assignment?• What happened the last time we used the
assignment? • What are our expectations for the assignment?
2. Listing – Focus on particular details of the assignment and what specific learning objectives we hope to see completed (Sometimes helps to imagine the best and the worst performance on the assignment)
3. Grouping and Labeling the goals together - Organize the results of our reflections in Stages 1 and 2, grouping similar expectations together in what will probably become the dimensions of the rubric
4. Application – Transfer groupings to a rubric grid
Example: Research Article Critique1. Reflecting -
• Want students to learn to be critical evaluators of psychological research
• The research article critique should start them thinking critically, will follow up with class discussion on each critique, and use multiple critiques throughout the semester to help develop their skills
• I expect students will move from description to analysis
2. Listing – • I have seen students move through
Bloom’s taxonomy in a semester using this assignment: Most can show “knowledge” and “comprehension” at the beginning of the course, move to “application,” “analysis,” “synthesis,” and “evaluation” (hopefully). I want to try to capture this process.
3. Grouping and Labeling the goals together – • see the questions that follow
4. Application – Transfer groupings to a rubric grid• See the rubric that follows
Research Article Critique Assignment For each article analysis you are to answer
the following questions: What is the primary question posed by the
study? Is there a hypothesis stated? If so, what is it? What is the theoretical explanation for the
proposed hypothesis? Briefly describe the way the independent and
dependent variable(s) were manipulated or measured
How do the results of the study affect the originally posed hypothesis (or purpose of study)?
Two strengths of the study? Two weaknesses of the study? What is a logical extension of the study?
Briefly describe a study you could conduct to extend the research
Sample Rubric – Designed for AssessmentACC = Accomplished, AVG = average, DEV = developing, BEG = beginning
Understands primary purpose of research
Clearly labels the independent and dependent variable without
extraneous material provided and shows understanding of
theoretical basis of study
ACC AVG DEV BEG
Describes measurement issues clearly
Hones in on how the variables are manipulated/measured without
providing extraneous material
ACC AVG DEV BEG
Understands significance and implication of the results
E.g., does research represent a methodological improvement, novel hypothesis, important test of a theory, etc.
ACC AVG DEV BEG
Chooses relevant and important strengths of study
Discusses strengths that are very relevant to judging the internal validity of the study (or possibly external validity)
ACC AVG DEV BEG
Chooses relevant and important weaknesses of study
Discusses weaknesses that are very relevant to judging the internal validity of the study (or possibly external validity)
ACC AVG DEV BEG
Proposed study an important extension of the research
E.g., methodological improvement, advancement of theory, tests an alternative explanation, etc.
ACC AVG DEV BEG
Creativity
e.g., Unusual association of ideas in responses, proposed study is novel and represents a significant advancement
ACC AVG DEV BEG
Rubric Used for Grading – weight each component
5% Understands primary purpose of research
Clearly labels the independent and dependent variable without
extraneous material provided and shows understanding of
theoretical basis of study
ACC AVG DEV BEG
5% Describes measurement issues clearly
Hones in on how the variables are manipulated/measured without
providing extraneous material
ACC AVG DEV BEG
10% Understands significance and implication of the results
E.g., does research represent a methodological improvement, novel hypothesis, important test of a theory, etc.
ACC AVG DEV BEG
25% Chooses relevant and important strengths of study
Discusses strengths that are very relevant to judging the internal validity of the study (or possibly external validity)
ACC AVG DEV BEG
25% Chooses relevant and important weaknesses of study
Discusses weaknesses that are very relevant to judging the internal validity of the study (or possibly external validity)
ACC AVG DEV BEG
25% Proposed study an important extension of the research
E.g., methodological improvement, advancement of theory, tests an alternative explanation, etc.
ACC AVG DEV BEG
5% Creativity
e.g., Unusual association of ideas in responses, proposed study is novel and represents a significant advancement
ACC AVG DEV BEG
Describing the level of performance on the rubric Anywhere from 2 to 5+ levels of performance
on each trait Terms used to describe level of performance
might be positive and active to encourage motivation in students (suggestions from Stevens & Levi): Mastery, partial mastery, progressing,
emerging high level, middle level, beginning level Sophisticated, competent, partly competent,
not yet competent Exemplary, proficient, marginal, unacceptable Advanced, intermediate high, intermediate,
novice Distinguished, proficient, intermediate, novice Accomplished, average, developing,
beginning
Using Rubrics for Assessment
We all need to attend to the issue of assessment, and Walvoord argues that we can make use of what we already do in the grading process
American Association for Higher Education’s “Principles of Good Practice for Assessing Student Learning”
Answer questions that people care about
Lead directly to improvement in teaching and learning
Be embedded in the context of learning
Take place repeatedly over time
The rubrics we use for grading can be used for assessment
“If we as faculty do not make our learning goals, tests, criteria, and standards explicit and understandable to legislatures, boards, accrediting agencies, and other audiences in ways that meet their needs and concerns, we face the very real possibility that some of the control we currently exercise in the classrooms will be taken away from us. We must deal with assessment, but we need not construct a parallel assessment structure that ignores the assessment we already conduct.” (Walvoord, 1998, p. 5).
How to turn this into assessment
Paper 1 Paper 2 Paper 3 TotalProf ACreative1 4 1 2 7Creative2 4 1 2 7Critical1 3 1 3 7Critical2 2 1 3 6 Prof BCreative1 3 2 2 7Creative2 4 1 3 8Critical1 3 1 3 7Critical2 2 1 3 6
Prof CCreative1 4 1 2 7Creative2 3 2 3 8Critical1 2 2 3 7Critical2 2 1 3 6
Prof DCreative1 4 2 3 9Creative2 4 2 3 9Critical1 3 2 3 8Critical2 3 2 4 9
Total 50 23 45
How have faculty been influenced by this process? “I was able to clarify my expectations for the course by
articulating the purpose behind each graded exercise. This helped dramatically in easing anxieties about performance and final grades. Students seemed to appreciate understanding the logic behind the assignments and seeing that each focused on building and assessing a particular skill, rather than merely providing me with another grade.”
“Best of all, I’m looking forward to sharing the process of developing PTA-based rubrics with my students. Specifically, students will work in groups to identify their own PTA-based critical thinking rubrics before they embark on writing a particular essay. I can hardly think of a better way to develop the students’ meta-cognition., i.e., how they think about their own thinking!”
“Yes, we’ve been disappointed with some aspects of student performance (for example synthetic ability) and using the rubric has helped us communicate better with students, and we’re thinking about ways to improve courses to help teach the skills needed to succeed on this senior exercise objective.”
“I have learned a great deal, and I think that my teaching has been positively influenced by this important work. I can also say that my research and scholarship have been impacted by these foci, as well.”
Go to Website for Rubric Samples:
http://www.wooster.edu/teagle/default.html
Top Related