Grant Wiggins on Grading - Authentic Education · 4 Grant Wiggins on Grading 10 Criteria for any...

1

Grant Wiggins on Grading

1

GradesGrades

2

Provocations...• What is a grade’s purpose? What follows for what it

should represent, regardless of tradition?• Should we - would we - give grades if we didn’t have to?• Is it acceptable for teachers at the same grade and

teaching the same course to view the same or similarstudent work differently? What follows for gradingpolicy and practice?

• Why calculate the ‘mean’ when it penalizes progress andover-rewards inconsistency?

• Why isn’t every public school giving grades against statestandards if those standards are obligatory?

• Why do parents like grades? Why is this interest sound,even if teachers don’t like giving them?

3

Looking closely at habit• Always tricky: habit runs so deep, we

rationalize it without realizing it• We ask that you work hard to keep an

open mind and resist the ‘Yes, but...”reaction that is inevitable

2


4

Essential Qs of Grading• Why grade? Why not?• Audience: For whom are we grading?• Purpose: What is the primary purpose of a

grade? (What if our purposes conflict?)• What should we grade? What shouldn’t we

grade?• How should a grade be determined?• How consistent should grades and grading

policy be (across time and across teachers)?• How is quality control in grading best

achieved?

5

An indication of unthinkingand harmful habit

• I received the following email: “I was meeting with our high school Advanced

Placement Teachers, who were expressingconcerns about our open enrollment processand the high failure rate. One math teachersaid that while a particular student was nowmaking grades in the 80's, she had made a 12on an initial test, “so there is no way she's goingto make a passing grade for the first nineweeks.”

6

Other countries• In Belgium, France, Morocco, Portugal, Peru,Venezuela, Iran and Tunisia a 20-pointgrading scale is used, in which 20 is thehighest grade and 0 is the lowest.

• The "passing" grade is usually 10• Grades of 10-11 is "adequate".• Grades of 12 or 13 are "passable"(better than

adequate)• Grades of 14 to 15 are "good" (better than "passable")• Grades of 16 to 17 are regarded as excellent and

outstanding, respectively. From this point on, you havetruly mastered the course.

• Grades of 18 to 19 are nearing perfection.• Grades of 20 are just perfect.

3


7

Denmark• The current Danish scale is called the 13-scale and consists of

10 grades ranging from 00 to 13, with 00 being the worst.• 00 completely unacceptable performance.• 03 very hesitant, very insufficient and unsatisfactory

performance.• 5 hesitant and not satisfactory performance.• 6 just acceptable performance.• 7 mediocre performance, slightly below average.• 8 average performance.• 9 good performance, a little above average.• 10 excellent but not particularly independent performance.• 11independent and excellent performance.• 13 exceptionally independent and excellent performance.

8

Where is it written thatwe must, as teachers,…• Give only one (aggregate )grade to each

student in our class, even if the transcriptonly permits one grade?

• Calculate grades using the mean score (asopposed to, say, the median or the trend)?

• Grade each student in a diverse classroomagainst the same standard?

• Keep different weighting and gradingvariables constant all year?

• Assess and grade all students at the samepoint in time (thereby making it impossible todo a thorough and timely job of grading)?

9

Judging grading policiesand practices

• No progress without sound criteria: We need criteria by which we can

evaluate grading policies and practices ina pedagogically wise and defensiblemanner

Only agreed-upon criteria can free usfrom the tyranny of bad habits that wetry to rationalize

4


10

Criteria for any gradingsystem to meet:

• Honest feedback about one’s standing• Fair to each student and other students• Transparent and without mystery• Credible to clients & constituencies• Valid assessment against key long-term

learning goals• Useful (actionable) and user-friendly

information about performance and howto improve

• Pedagogically wise - it sends the rightmessage and gets the incentives right forlearners

11

Exercise: So, whatfollows?

• In small groups, we’ll analyze somepractical implications for individualand school-wide grading, given eachcriterion: Fair Honest Transparent Credible Useful & User-Friendly Pedagogically wise

12

Symbols and theirmeaning

• A big problem may not be “grades” but asingle grade, hiding vast differences inperformance, habits, and attitudes! Why do we only put single grades on report

cards and transcripts when disaggregated datawould greatly help the reader understand whatthe learner has accomplished and who thelearner is as a learner?

5


14

Break out the IndependentVariables in Grading

• Validity, transparency, and usefulnessrequire knowing at least these 3 elements,broken out:

What is my level of achievement? What has been my progress (against

standards)? How are my work habits and attitudes?

15

Independent Variables inGrading

• Most Grading Systems conceal more thanthey reveal!

Achievement (against standards) Sub-achievement (discrete competencies

making up overall achievement) Progress over time (against standards) Habits and attitudes (effort, open to

learning, etc.)

6


16

Problems with the(Single) Grade

• Different Work, Same Grade Average Achievement, Great Progress

vs. High Achievement, No progress Different sub-scores on multiple

rubrics, SAME AVERAGE Wild swings vs. consistency - SAME

AVERAGE

17

sub-grades vital in asubject

• “Yes, I know I got a ‘B’ in LanguageArts, but break it down for me” by – Grades against each state standard Control over varied genres of reading Control over varied genres of writing Each criterion used in rubrics related to

my writing The quality of my participation

18

Some implications,explored in this conference

• Thus, we are not against grades! We are againstdumb grading systems!

Conventional summary grades,calculated by computing thearithmetical ‘mean’, areindefensible (despite thelongstanding habit of doing so)

Failure to give grades against statestandards is irresponsible

Providing only a single grade isunhelpful

7


19

Beyond the Mean: ithides vital information

• The “average” (the mean) hides or under-rewards:

Trend of work over time - progress Consistency of work quality Key feedback on independent variables

making up the grade Degree of true mastery against an

objective standard

20

Beyond the mean: OtherMethods of “Averaging”

• Beyond the mean: Reliable pattern (median, mode) Consistency (range, Standard Deviation.) Throw out the highest and lowest scores

(Olympics) Factor in degree of difficulty of the tasks

(as in music, diving, gymnastics)

21

Beyond the ‘average’:Other Methods for

Determining Grades• think of diving, karate, chess:

Total score over time (e.g. Olympics) Final mastery level (chess, karate,

ACTFL) Measure of progress against standards

over time (pre/post, the value-added)

8


22

Varying the weightingand honoring difference

• Why penalize early effort or prior level? Don’t grade (or average in) all formative work INITIALLY grade Effort and Progress more than

level of achievement in the first month(s),gradually making level of achievement moresignificant

Have LEVELS in your class, via pre-testing; make itlike skiing - Novice/Intermediate/Expert - wherethat designation is given along with the grade - Awork for a novice, but C work for Expert

Emphasize some rubrics more than others initially,then give grades against all rubrics by the end

23

Provide sub-grades for betterfeedback

• A standards-based report card: Grade control over core genres in

Language Arts, state standards in mathand science

Distinguish between mastery of contentvs. mastery of processes

24

Feedback: on which keytasks?

• Key to reform: to view assessment in termsof longer-term learning goals, not justgrades on quizzes of recent content;

• Feedback against – Key competencies/exit standards Key transfer tasks Key habits of mind Key long-term inquiries (essential questions)

9


25

Longitudinal Progressvia Rubrics

• Rubrics to track progress over timeagainst standards On a novice-expert continuum in addition

to rubrics for judging increasingsophistication of ideas and processes• cf. Rubrics in ACTFL, American Literacy

Profiles, Meisels’ Work Sampling System,chess/karate/bridge etc.

26

Longitudinal Progressvia Rubrics

• Too Many Rubrics are Task-Specific A reporting and grading system should

align, for tracking progress over time Rubrics should be more generalized and

linked to exit-level standards• Too many grading systems and rubrics

conflate descriptive LEVEL withjudgment about WORK QUALITY(regardless of level)

27

ACTFL Example• Novice-Low: Oral production consists of isolated words

and perhaps a few high-frequency phrases. Essentiallyno functional communicative ability.

• Novice-Mid: Oral production continues to consist ofisolated words and learned phrases within verypredictable areas of need, although quantity isincreased. Vocabulary is sufficient only for handlingsimple, elementary needs and expressing basiccourtesies. Utterances rarely consist of more than two orthree words and show frequent long pauses andrepetition of interlocutor's words…. Some Novice-Midspeakers will be understood only with great difficulty.

10


28

ACTFL Example• Novice-High Able to satisfy partially the

requirements of basic communicative exchangesby relying heavily on learned utterances butoccasionally expanding these through simple re-combinations of their elements…. Shows signs ofspontaneity although this falls short of realautonomy of expression…. Vocabulary centers onareas such as basic objects, places, and mostcommon kinship terms. Pronunciation may still bestrongly influenced by first language. Errors arefrequent and, in spite of repetition, some Novice-High speakers will have difficulty beingunderstood even by sympathetic interlocutors.

29

ACTFL• Advanced Able to satisfy the requirements of

everyday situations and routine school and workrequirements…. Can narrate and describe withsome details, linking sentences together smoothly.Can communicate facts and talk casually abouttopics of current public and personal interest, usinggeneral vocabulary. Shortcomings can often besmoothed over by communicative strategies, such aspause fillers…. Circumlocution which arises fromvocabulary or syntactic limitations very often is quitesuccessful, though some groping for words may stillbe evident. The Advanced-level speaker can beunderstood without difficulty by nativeinterlocutors.

30

UK natl curriculum:science

• Level 4 Pupils recognize that scientific ideas are based on evidence. Intheir own investigative work, they decide on an appropriate approachfor example, using a fair test to answer a question. Where appropriate,they describe, or show in the way they perform their task, how to varyone factor while keeping others the same. Where appropriate, theymake predictions. They select information from sources provided forthem. They select suitable equipment and make a series of observationsand measurements that are adequate for the task. They record theirobservations, comparisons and measurements using tables and barcharts. They begin to plot points to form simple graphs, and use thesegraphs to point out and interpret patterns in their data. They begin torelate their conclusions to these patterns and to scientific knowledge andunderstanding, and to communicate them with appropriate scientificlanguage. They suggest improvements in their work, giving reasons.

11


31

From UK natl. curriculum• Level 8 Pupils give examples of scientific explanations or models that

have had to be changed in the light of additional scientific evidence. Theyevaluate and synthesize data from a range of sources. They recognize thatinvestigating different kinds of scientific questions requires differentstrategies, and use scientific knowledge and understanding to select anappropriate strategy in their own work. They decide which observationsare relevant in qualitative work and include suitable detail in theirrecords. They decide the level of precision needed in comparisons ormeasurements, and collect data enabling them to test relationshipsbetween variables. They identify and begin to explain anomalousobservations and measurements and allow for these when they drawgraphs. They use scientific knowledge and understanding to drawconclusions from their evidence. They consider graphs and tables ofresults critically. They communicate findings and arguments usingappropriate scientific language and conventions, showing awareness of arange of views.

32

Reading LevelsAssessment

• Lexile Scores• Degrees of Reading Power• Meisel’s Work Sampling System• American Literacy Profiles• UK Reading rubrics

33

12


34

NO justification for fittinggrades to a bell curve

• True both mathematically and pedagogically Mathematics: the point of a curve is to show the

normal distribution of a vast number ofelements, likely to be ‘naturally’ distributed -e.g. the cholesterol numbers for an entirepopulation

Education: We aim for performanceimprovement across the board; a normal curve isa sign of failure in a small class

• Note: so-called “curving” of a set of test grades is different, andmay reflect a wise adjustment based on the hunch that the rawgrades are not reliable

35

Bloom et al:Grading on a Curve

• Grading on a curve - unjustified: “There is nothing sacred about the

normal curve. It is the distribution mostappropriate to chance and randomactivity. Education is a purposefulactivity, and we seek to have thestudents learn what we have to teach…”

36

Bloom et al:Grading on a Curve

“If we are effective in our instruction, thedistribution of achievement should be verydifferent from the normal curve. In fact,we may insist that our efforts have beenunsuccessful to the extent that [grades]approximate the normal distribution.”• Evaluation to Improve Learning, p. 53

13


37

Why fitting to a curve iswrong - and wrong-headed• More generally:

Fitting grades to a normal curve is toexaggerate student differences and reportagainst norms not standards.

We should seek the opposite: let the gradesfall where they may, against worthystandards.

Standards and expectations are lowered ifwe fail to give disinterested grades againstclear criteria and models

38

Example of “Mike” -a high-mobility student

Year Location Grade

1989 Salem C+ 1990 Vero Beach C -1991 Bakersfield C1992 Albuquerque A1992 San Antonio A +1992# Los Angeles C-1993 Los Angeles B+

# 1st year of mainstreaming

39

“Mike” (continued)

Year Location Score

1989 Salem 2681990 Vero Beach 2501991 Bakersfield 2771992 Albuquerque 3411992 San Antonio 3711992# Los Angeles 2321993 Los Angeles 318

14


40

41

42

15


43

Grading Criteria,Grading Criteria,explainedexplained

44

Key tension: Honest andFair

It is difficult but essential to be both; mostreports err on one side or the other

Honest: a dispassionate account of studentstrengths and weaknesses against standards andnormed expectationsFair: mindful of extenuating circumstances,personal strengths and challenges, local norms,and reasonable expectations of that individual

45

Fair• Honors idiosyncracies and appropriate

extenuating circumstances Don’t compare apples and oranges:

• Don’t confuse “behind” with “differentdevelopmental path”

• Don’t emphasize narrow and arbitrary test types• Be explicit about special-student status

16


46

Fair• We work consciously and carefully to

eliminate bias and capriciousness Bloom et al.: “Unfortunately, extraneous

factors, such as the student’s speech,sociability, personality can unconsciouslyinfluence a teacher in grading”

47

Honest• Faithfully describes absolute levels ofperformance Honest reports do not flinch from stating the

level of performance against standards, andmaking predictions about likely exit status

Dishonest to report only the strengths - cf.some narrative and portfolio systems

48

Credible• Means that it is trustworthy and

believable information for ALLconstituencies - implying: disinterested scoring and grading on a

regular basis external validation in terms of tasks,

criteria, standards

17


49

To be credible a grade(judgment) must be... Based on multiple & varied assessment, and

sensitive to MI and learning styles Comparable grades/marks Given by someone who consistently monitors

based on instruction Measuring achievement toward standards Inter-rater reliability Measured over time More than one person has input

50

To be credible a grade(judgment) must ...

Be specific about what the child knows and is able to do Not confuse what a child does with what a child can do

(“potential” separated from “achievement”) Knowing the criteria and standards upon which the grade

was given Discrepant data not swept under the rug! Fair opportunities to learn Observable, measurable data is central to a judgment

51

Credibility: QualityControl in Grading

• Grades are too often unreliable orinvalid (and inconsistent acrossteachers) Agreement on “anchors” and grading

criteria needed Grading policies and oversight needed Need to link grades to external standards

18


52

Useful• Refers to the reader’s ability to use

and profit from the report Links back to clarity about purpose(s) and

audience(s) Not likely to be useful if fails to assess

and evaluate strengths and weaknesses

53

User-Friendly• Refers to intelligibility and ease of

access of information Many reforms of grading and reporting

have not been user-friendly, though highlyvalued by teachers

Few parents understand why letter gradesare inadequate and are suspicious ofreforms

54

User-friendly (feasible)• Must be manageable for teachers

We need to rethink our use of timeto ensure rich, timely, and accurateinformation• Why do we unthinkingly report for ALL students

on the same day? Why not stagger report cardsby sections of the alphabet to lighten the loadand ensure better reports?

19


55

Toward CredibleGrading & Scoring

• Setting standards locally requires: on-going faculty group scoring of student

work system-wide rubrics and 'anchors' for

shared standards on-going study of performance results

leading to recommendations validating local grading standards

56

Credible

Valid, supported analysis, using astandard approachA trustworthy source, an expertjudgmentBelievable, plausible, based on acareful consideration

57

Achievement, defined

Performance against academic/intellectual standards

Measure of “pure” performance level,without extenuating factors or normsExemplars, criteria and specifications areexplicit and deliberately referenced

20


58

Progress, defined

Performance measured over timeagainst explicit standards

Requires a longitudinal assessmentsystem, on a novice-expert continuum

We need to ensure there is both pre-testingand post testingLongitudinal rubrics

59

Purposes: Assess vs. Evaluate• Assess: comment on and analyze work

against valid criteria - so as to givevalue-neutral feedback & guidance

• Evaluate: summarize implications ofresults, by assigning a value to theoverall performance, against somestandard(s) or expectation(s)

6060

“Objective” &“Subjective”

• Human judgments can and must be as“objective” as possible! Objective: disinterested, valid, and reliable

judging against standards Subjective (in the bad sense): Biased or

capricious judgment Subjective (in the good sense): A human

judgment based on a variety of information, notmechanically computed

• Beware of defining machine-grading as “objective” andhuman scoring as “subjective”

21


61

“Objective” &“Subjective”

• Thus, to say that local grades are “sosubjective” is misleading The issue is sound professional judgment We need more disinterested judging locally to

make scores and grades defensible - objective• E.g. IB, Advanced Placement, group scoring of writing or

music against standards and rubrics, etc.

62

A Mantra:• Give fewer grades and more feedback

We tend to give too much evaluation, toosoon

We tend to “grade” without helping thestudent really grasp the feedback thegrade is based on

Giving better feedback enables thestudent to take greater control of givingthemselves (or seeking) advice, sooner

63

Excellent feedback• The most common answers to our

exercise on good feedback: Timely user-friendly - in approach and amount Descriptive & specific re: performance Consistent Expert Accurate Honest, yet constructive Derived from concrete standards On-going

22


64

for furtherinformation...Contact me:

[email protected] grading and reporting bulletinboards at authenticeducation. orgwww.bigideas.org Our monthly resourceon a theme related to teaching andassessing

The archive contains an issue devoted tograding (winter 2005)

Grant Wiggins on Grading - Authentic Education · 4 Grant Wiggins on Grading 10 Criteria for any...

Documents

Transcript of Grant Wiggins on Grading - Authentic Education · 4 Grant Wiggins on Grading 10 Criteria for any...