Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

33
4th Annual Conference No Educator Left Behind: Equipping Adjunct Faculty with Knowledge and Skills Rubric’s Cube: Complimenting, Critiquing, and Challenging Student Work

description

Presented at the No Educator Left Behind Conference 2009

Transcript of Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Page 1: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

4th Annual Conference No Educator Left Behind:  Equipping Adjunct Faculty with Knowledge and Skills

 

Rubric’s Cube:

Complimenting, Critiquing, and Challenging Student Work

Page 2: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Objectives

Gain an understanding of the role of grading rubrics in:

• Clarifying teacher expectations• Evaluating student learning• Providing detailed feedback• Improving instruction• Performing outcome assessment

Evaluate the strengths and weaknesses of grading rubrics as a student assessment tool

Integrate the principles of primary trait analysis (PTA) into designing a rubric for an assignment or performance skill

Take the first step toward using a grading rubric for an assignment, project, or exam

Page 3: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Considerations For Effective Rubric Design

•does its job (well written)

•uses precise criteria and descriptors (well chosen)

•is student-friendly (distinguish between clear and ambiguous grading criteria)

Page 4: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Why do we Grade?• Grading

– The process used by faculty to assess student learning via assignments and exams, including:Relating test items or assignments to learning objectivesEstablishing criteria/standards

Helping students acquire needed knowledge/skillsFacilitating student motivationGiving feedback about performanceCommunicating about what has been learned Using results to influence teaching and curriculum

((BE Walvoord, VJ Anderson, 1998. BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for Effective grading: A tool for learning and assessmentlearning and assessment. Jossey-Bass: San Francisco). Jossey-Bass: San Francisco)

Page 5: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Grading Challenge # 214:

Student’s Perceptions

Page 6: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Myths About Grading

• Grading:– Is a necessary EVIL– Must be objective– Should be easy– Should not take or detract from teaching– Uses knowledge of experts to evaluate novices– Is assessment only, doesn’t impact learning– With clear expectations, means you are “spoon-

feeding”– Cannot be used in program assessment

Page 7: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Expert Input

Experts agree:– Rubrics are hard to design.– Rubrics are time-consuming to design.– “A rubric is only as useful as it is good. Using a bad rubric

is a waste of time…”--Michael Simkins in “Designing Great Rubrics”

Experts disagree:– how to design a “good” rubric

Bottom line: Is it working for you and for your students?

Page 8: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Principles of Effective Grading1. Appreciate the complexity of grading2. Substitute judgment for objectivity3. Distribute time effectively 4. Be open to change5. Listen & observe6. Communicate & collaborate with students7. Integrate grading with other key processes8. Seize the teachable moment (feedback)9. Make student learning the primary goal10. Be a facilitator FIRST, gatekeeper LAST11. Encourage learning-centered motivation12. Emphasize student involvement

(Walvoord & Anderson, 1998)(Walvoord & Anderson, 1998)

Page 9: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Student Assessment Methods• Multiple choice examination• Short answer questions on tests• Essay questions• Annotated bibliographies• Literature reviews• Case Studies• Oral Examinations• Practical/Performance Examinations• Clinical Assessments• Journals• Portfolios• Lab Conclusions

(LC Jacobs, CI Chase, 1992. Developing and using tests effectively. Jossey-Bass: San Francisco)

Student-Constructed Responses

Page 10: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Methods of Grading Student-Constructed Responses

Method Advantage Disadvantage

Norm Referenced: categorize work into A, B, C, D

Little work up front, recognizes faculty as ‘expert’, flexible

-Potential for bias

-Less opportunity for learning, vague

Checklist: list of criteria to include (introduction, research question…)

Assignment directions match checklist, not difficult to prepare

-Usually lacks descriptions

-Lists + traits or behaviors, no negative

Grading Rubric: criterion referenced, describe performance expectations & weighting

Explicit expectations, better feedback, greater inter-rater reliability, links to performance

-Takes time to construct, may need to modify after 1st run

(Blackinton, 2008)

Page 11: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

What is a Grading Rubric?• Method of “articulating expectations for an assignment

by listing the criteria, or what counts, & describing levels of quality from excellent to poor”1

• Type of assessment that specifies gradations of quality from excellent to poor2

• A criterion-referenced method of grading using highly specific grading criteria that are linked to objectives

1HG Andrade, Y Du (2005). Students perspectives on rubric-referenced assessment. Practical Assessment, Research & Evaluation, Vol 10 (3).

2HG Andrade (2005). Teaching with rubrics: The good, the bad, and the ugly. CollegeTeaching, 53 (1)

Page 12: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

What is a Grading Rubric?

Performance Level 3

Performance Level 2

Performance Level 1

Dimension 1 3 pts: describe 2 pts 1 pt

Dimension 2 4 pts 2.5 pts 1 pt

Dimension 3 3 pts 2 pts 1 pt

Dimension 4 2 pts 1 pts 0 pts

Gradation: excellent-poorCategories

important to the teacher/class

Weighted PointsDimensions also called criteria(Blackinton, 2008)

Page 13: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Sample Rubric Format

Criteria 1 Criteria 2 Criteria 3

Level 1 3 pts: describe 2 pts 1 pt

Level 2 4 pts 2.5 pts 1 pt

Level 3 3 pts 2 pts 1 pt

Level 4 2 pts 1 pts 0 pts

(Blackinton, 2008)

Page 14: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

How do Students Perceive Rubrics?

• Andrad & Yu, 2005– Investigated how students use grading rubrics

• Focus group / qualitative design

– Students reported that they used rubrics to• Help them determine faculty expectations• To plan an approach to assignment• Check/revise work before handing in• Help reflect on their learning-see strengths/weaknesses clearly

– Perceived results of rubric use• Better, fairer grades• Improvements in quality across classes• Less anxiety

Page 15: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Using Primary Trait Analysis To Develop a Grading Rubric

Rationale– Analyze traits / characteristics of student learning

and then clearly articulate them, leading to:• Assignment specific

• Explicit criteria

• Adds objectivity & consistency to holistic scoring

• Lets students know in ADVANCE how will be graded

((Walvoord &Anderson, 1998)Walvoord &Anderson, 1998)

((R Lloyd-Jones, C. Cooper & L. Odell (Eds), 1977. R Lloyd-Jones, C. Cooper & L. Odell (Eds), 1977. Primary Trait Scoring in Primary Trait Scoring in Evaluating Writing: Describing, measuring, judgingEvaluating Writing: Describing, measuring, judging. Urbana IL: National . Urbana IL: National Council of Teachers Council of Teachers

Page 16: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Constructing a Primary Trait Analysis Scale1

1. Choose assignment/test that matches course objective

2. Identify all traits that will count for scoring in assessment 1. Body Language vs Thoroughness vs Accuracy

3. Build scale for scoring performance, gradations of skill1. Scale usually ranges from 2-5 points

2. Include what should be demonstrated and what should be avoided

3. Build a range that discriminates A from A- from B+

4. Run scale by colleague, graduate, teaching assistant

5. Weight items: content > spelling; accuracy > efficiency, etc

4. Evaluate performance against criteria1. Try scale with sample & revise as needed

((BE Walvoord, VJ Anderson, 1998. BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for learning Effective grading: A tool for learning and assessmentand assessment. Jossey-Bass: San Francisco). Jossey-Bass: San Francisco)

Page 17: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #1

• Don’t use generic or “canned” rubrics without careful consideration of their quality and appropriateness for your project.

• These are your students, not someone else’s.

• Your students have received your instruction.

Page 18: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #2

• Avoid dysfunctional detail.– “…in most instances, lengthy rubrics probably

can be reduced to succinct…more useful versions for classroom instruction. Such abbreviated rubrics can still capture the key evaluative criteria needed to judge students’ responses. Lengthy rubrics, in contrast, will gather dust” (Benjamin 23).

--Includes wordiness, jargon, negativity

Page 19: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #3

• Limit the number of criteria– Well…– Don’t combine independent criteria.

• “very clear” and “very organized” (may be clear but not organized or vice versa).

Page 20: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #4

• Use key, teachable “criteria” (What counts)– Don’t vaguely define levels of quality.

– Concrete versus abstract• “poorly organized” (Organization: sharply focused thesis,

topic sentences clearly connected to thesis, logical ordering of paragraphs, conclusion ends with clincher)

• “inventive” “creative” “imaginative” UNLESS…

Key Question to ask yourself: What does it look like?

Page 21: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #5

• Use measurable criteria.--Specify what quality or absence looks like

vs. comparatives (“not as thorough as”)

or value language (“excellent content”)

Page 22: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #6

• Aim for an even number of levels– Create continuum between least and most– Define poles and work inward– List skills and traits consistently across levels

Page 23: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #7

• Include students in creating or adapting rubrics• Consider using “I” in the descriptors

• I followed precisely—consistently—inconsistently—MLA documentation format.

• I did not follow MLA documentation format.

---Highlight the impact of the performance

--Was the paper persuasive or problem solved? (Note importance of PURPOSE)

--What are the traits of effective persuasion?

--Be sure that the descriptor is not the criterion and vice versa

Page 24: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #8

• Motivate students to use rubric.

Instructional rubric (“Buy one, get one…”)

“At their very best, rubrics are also teaching tools that support student learning…” (Andrade 13).

• Do they understand the criteria and descriptors? How do you know?

• When do you give the rubric to your students?

Page 25: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Rubric Tip #9

• Provide models of the different performance levels.

Page 26: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

The Assignment Sheet

• Don’t forget the importance of the assignment sheet

• Connection to rubric (Use same language!)– The lawyers in your class

“But the rubric doesn’t say that…”Project/paper/presentation must meet all requirements of assignment– Due date and late penalty– Format requirements– Non-negotiables

• Skills and reasonable expectations

Page 27: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Don’t Forget the Check-in Stage

• Use your rubric as a formative assessment to give students feedback about how they are doing.– Isolate a particularly challenging aspect– Have student isolate an area of difficulty– Center revision instruction around rubric

Page 28: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Applying the Rubric

• Distribute ( or post) rubric to students in advance– Have conversations about expectations

• Ask students to attach rubric to assignment– Helps students pay attention!

• Use rubric to grade– Match written comments to phrases in rubric

• Revise after use– Make changes soon after grading for next time

• Answer Curriculum Questions– Did the students learn? To what degree were objectives met?– If not, was it the teaching? The assignment? Background skills?– What, if any, changes should be made in the class or curriculum?

Rubric?

Page 29: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

CautionDon’t let the rubric stand alone:ALWAYS, ALWAYS provide specific

“Comments” on your rubric and/or on the student product itself.

Page 30: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Advantages of Grading Rubrics• Save time in grading process• Makes process of grading reliable/fair• Clarifies expectations for students • Reinforces key concepts - help faculty relate to objectives• Students are participants as expectations are known• Student peer review• Works well if team-teaching• Share across courses or over curriculum• Basis for departmental/program assessment Which of these advantages pertain to your teaching in the health

professions?

(Walvoord & Anderson, 1998)(Walvoord & Anderson, 1998)

Page 31: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

CAUTION: Common Rubric Pitfalls

• Rubric does not correspond with class or program outcomes– Example: Entire rubric focused on writing quality not content

• Scale does not have enough gradations or levels– Not distinguishing the A’s from the B+’s

• All traits are given equal weight regardless of complexity– Grammar = Content

• Too broad, not enough content described– Words like ‘breadth’ and ‘depth’ used in lieu of specifics– Students still not sure, other grading faculty still not clear

• Too long/too complicated– Faculty + students get lost in the rubric

Page 32: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

A Rubric … … is a guide for the evaluation of student work that defines a facilitator’s expectations and identifies grading criteria point by point.

… provides a clear set of criteria for judging students’ work by specifying factors on which the facilitator will grade the student thereby helping the facilitator define expectations ad prompting the student to focus on specific points.

… takes extensive thought and planning to be effective and that facilitators need to be lucid in their explanations of grading standards and be sure of the assignment’s objective to create rubrics that are pedagogically sound.

Page 33: Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

Classroom Assessment Technique Application

• Write down ONE thing you will commit to incorporate regarding the use of grading rubrics

• Write down ONE unanswered question