SST PresentationSST Presentation

Assessment and Assessment and Evaluation Evaluation

Oct 17, 2005Oct 17, 2005

tom.macartney @ ocdsb.catom.macartney @ ocdsb.ca

KWL KWL - Assessment and - Assessment and

EvaluationEvaluation• What do I Know?What do I Know?

• What do I Want to know?What do I Want to know?

• What have I Learned?What have I Learned?

The greatest adjustment The greatest adjustment teachers have to make teachers have to make

related to the new Ontario related to the new Ontario Curriculum is the way they Curriculum is the way they

assess, evaluate and assess, evaluate and report on student learning.report on student learning.

The adjustment requires a The adjustment requires a shift from a focus on shift from a focus on

teaching all to a focus on teaching all to a focus on the learning of each.the learning of each.

In medicine, the cardinal rule In medicine, the cardinal rule isis……

Do no harm.Do no harm.

Louis Pasteur - 1865Louis Pasteur - 1865

"This water, this sponge, this lint with which you wash or cover a wound, may deposit germs which have the power of multiplying rapidly within the tissue....If I had the honor of being a surgeon....not only would I use none but perfectly clean instruments, but I would clean my hands with the greatest care...I would use only lint, bandages and sponges previously exposed to a temperature of 1300 to 1500 degrees.”

1875 - 1900…1875 - 1900…

In education the In education the cardinal rule is…cardinal rule is…

Do not destroy hopeDo not destroy hope..

ResearchResearch

“ “The role of research is The role of research is to attach numbers to the to attach numbers to the obvious, to make it obvious, to make it undeniable.” undeniable.”

- anonymous- anonymous

Brain Research, Cognitive Brain Research, Cognitive Psychology, Education collidePsychology, Education collide

Today, there are Today, there are more scientists more scientists devoted to research devoted to research on the mind and brain on the mind and brain than any other than any other medical-research medical-research field. field.

Brain Compatible LearningBrain Compatible LearningAttention / MotivationAttention / Motivation

To learn well we believe people To learn well we believe people need:need:

• Physical needs satisfiedPhysical needs satisfied• Challenge / Mystery / NoveltyChallenge / Mystery / Novelty• Success Success • Connections / Perceived NeedConnections / Perceived Need• Fit to learning styleFit to learning style

Brain Compatible LearningBrain Compatible LearningMotivationMotivation

Three types of factors that Three types of factors that influence a student’s level of influence a student’s level of motivation:motivation:

1.1. Personal beliefsPersonal beliefs

2.2. School practices and policiesSchool practices and policies

3.3. Classroom EnvironmentClassroom Environment

Personal beliefsPersonal beliefs• Student personal beliefsStudent personal beliefs• Low self efficacyLow self efficacy• Lack of relevanceLack of relevance• Fear of failureFear of failure• Peer concernPeer concern• Learning problemsLearning problems• Lack of challengeLack of challenge• Desire for attentionDesire for attention• Emotional distressEmotional distress• Expression of angerExpression of anger

School practices and School practices and policies that inhibit policies that inhibit

motivationmotivation1.1. Culture based on competition - the Culture based on competition - the

labour of less talented students labour of less talented students seldom acknowledged ; grades do seldom acknowledged ; grades do not inspire effortnot inspire effort

2.2. Culture of low expectations for low Culture of low expectations for low achieving studentsachieving students

3.3. Student peer culture that Student peer culture that discourages effortdiscourages effort

Classroom EnvironmentClassroom Environment Classroom practices that undermine Classroom practices that undermine

student motivation:student motivation:

1.1. Tying success to ability rather than effortTying success to ability rather than effort

2.2. Promoting a competitive environmentPromoting a competitive environment

3.3. Lack of routines and proceduresLack of routines and procedures

4.4. Negative approach to classroom Negative approach to classroom managementmanagement

5.5. One size fits all tasksOne size fits all tasks

6.6. Public punishmentPublic punishment

7.7. Slow pacingSlow pacing

8.8. Uninspiring instructional practicesUninspiring instructional practices

Implications for SchoolsImplications for Schools

• Develop a culture of high expectations Develop a culture of high expectations where staff truly believe all students where staff truly believe all students can and will achieve.can and will achieve.

• Learner-centred classrooms designed Learner-centred classrooms designed to maximize opportunities for all to maximize opportunities for all students to become engaged.students to become engaged.

W. E. DemingW. E. Deming- Total Quality Improvement- Total Quality Improvement

Individuals have an Individuals have an intrinsic drive to learn intrinsic drive to learn and do well and they do and do well and they do not want to fail.not want to fail.

Individuals are motivated Individuals are motivated by improvement no by improvement no matter how small.matter how small.

Cairine Wilson 2004/05

141175

252

0

50

100

150

200

250

300

Lost Credits 50's-60's 70's-90's

Mark Ranges

# o

f S

tud

ents

24.8% 30.8% 44.4%

Deming principle:Deming principle:Deep LearningDeep Learning

Key points: Key points:

1.1. Drive out fear! Drive out fear!

(TPA, grading system, class (TPA, grading system, class rules…)rules…)

2.2. Develop systems for collecting and Develop systems for collecting and analysing data.analysing data.

3.3. Ensure all levels of system are Ensure all levels of system are expected to learn from expected to learn from experiences and data.experiences and data.

Four Examples to pay attention to…

1.

2.

3.

4.

High Levels of Learning for All Students

Raise the Bar

Close the Gap

Raise the Bar

Close the Gap

Raise the Bar

Close the Gap

Where’s the Bar?Where’s the Bar?

• Level 3 is the Provincial Standard.Level 3 is the Provincial Standard.• A student achieving at this level is A student achieving at this level is

well prepared for work in the next well prepared for work in the next grade or the next course.grade or the next course.

Where’s the Bar?Where’s the Bar?

• Level 4 means the student is Level 4 means the student is demonstrating a very high to demonstrating a very high to outstanding achievement of the outstanding achievement of the specified expectations and a specified expectations and a greater command of the requisite greater command of the requisite knowledge and skills than a knowledge and skills than a student achieving at a level 3.student achieving at a level 3.

Vygotskian Perspective: Vygotskian Perspective: Teacher/Student InteractionsTeacher/Student Interactions

Student Responsibility

TeacherResponsibility

Joint Responsibility

Self Responsibility

Adapted from Wilhelm, Baker, and Dube, 2001

Zone of Actual Development

Zone of ProximalDevelopment

New Zone of Actual Development

What the student can do on her own unassisted

Assistance provided by more capable others: teacher or peer or environment: classroom structures and activities

Transition from other assistance to self-assistance

Internalization, automatization

Assistance provided by the self

““Backwards Design” PlanningBackwards Design” Planning

Stage 1: Identify targeted understandings

Stage 2: Determine appropriate assessment of those understandings

Stage 3: Plan learning experiences and instruction that make such understanding possible

Wiggins and McTighe, Understanding by Design

Stage 1: What do we want students to learn?

Stage 2: How will we know they have learned it?

Stage 3: What time and support can we offer

students who are not learning?

Professional Learning Professional Learning CommunitiesCommunities

Dufour and Eaker, Professional Learning Communities

Identifying Curriculum Identifying Curriculum PrioritiesPriorities

Worth being familiar with

Important to Know and Do

“Enduring” Understanding

Curriculum Design Curriculum Design is Keyis Key

• Making an assessment planMaking an assessment plan• Target-method matchTarget-method match

– 1. Summative tasks of the course / grade1. Summative tasks of the course / grade– 2. Summative tasks of each unit / strand2. Summative tasks of each unit / strand– 3. Enabling tasks within each unit / strand3. Enabling tasks within each unit / strand

• Evaluation toolsEvaluation tools– Tools must match performanceTools must match performance

• Marking scheme, rubric, checklist, rating scale, Marking scheme, rubric, checklist, rating scale, anecdotal recordanecdotal record

Making an Assessment PlanMaking an Assessment Plan

1. Enduring UnderstandingsEssential QuestionsKey Knowledge and Skills• lasting value, key concept, focusing theme, the overarching principle

3. Develop evaluation tasks tools• Authentic performance task(s) (Say, Write Do) that develop knowledge/skills• Appropriate tools (marking scheme, rubric, checklist, rating scale, anecdotal record) that are shared with all before task begins• Balance of achievement chart categories

•(K/U, A, T, C)• Feedback (effective questioning; timely, specific, frequent, feedback; self / peer assessment; formative use of summative tasks)

2. Develop Summative Tasks• Exam• Performance Task(s)• Tools• shared with students at

beginning of course

4. Plan learning experiences and instructionW= where is unit / course goingH= how will design hold student interestE= equip students, explore issues, experience key ideasR= opportunities for students to rethink / revise understandingE= allow students to evaluate their work and its implicationsT= tailored to the needs, interests and abilities of learnersO= organized to maximize initial and sustained engagement

““Backwards Design” PlanningBackwards Design” Planning

To ensure high levels of engagement for all students…

1)Worry a lot about the assessment before thinking through the lesson plans.

2)Worry a lot about what the standards or desired results imply for assessment and learning.

3)Focus on the important Big Ideas to uncover student misunderstandings rather than cover material.Wiggins and McTighe, Understanding by Design

Assessment QualityStandards

FairDefensible

Valid Reliable

Targets!Targets!

““Students Students can hit any can hit any target they target they can see and can see and holds still for holds still for them.”them.”

Rick Stiggins

The Achievement Chart-targets for students

Communication“say” “write”

Thinking - creative / critical

inquiry decision makinggoal setting problem solving

analysis evaluationApplication

“do”making connections

Foundational Knowledge / Understanding Learning Skills

Evaluation ToolsEvaluation Tools

MethodsMethods

Pen and PaperPen and Paper

Performance Performance assessmentassessment

Personal Personal communicationcommunication

ToolsTools

Marking schemeMarking scheme

RubricRubric

ChecklistChecklist

Rating scaleRating scale

Anecdotal recordsAnecdotal records

The mustsThe musts1.1. Target – Method – Tool matchTarget – Method – Tool match

Targets

•Knowledge / •Understanding

•Thinking

•Communication

•Application

Developing Effective ToolsDeveloping Effective Tools 3 3 Key Questions Key Questions What is ...?What is ...?

Worth being familiar with

Important to Know and Do

“Enduring” Understanding

Assessment Typestraditional quizzes and tests• paper/pencil• multiple choice• constructed responseperformance tasks and projects• open-ended• complex• authentic• learning in context• student choice

Evaluation ToolsEvaluation Tools

The mustsThe musts

22.. Clear criteria for feedback and Clear criteria for feedback and guidanceguidance Students must learn how to self Students must learn how to self

and peer assessand peer assess Legitimately takes the burden of Legitimately takes the burden of

feedback off the teacherfeedback off the teacher What did you do well?What did you do well? What do you need to do to improve?What do you need to do to improve? Levels 4, 3, Not yetLevels 4, 3, Not yet

Effective Feedback Effective Feedback for Learningfor Learning

• Positive firstPositive first• Specific (not ambiguous)Specific (not ambiguous)• ConstructiveConstructive• Connected to clear criteriaConnected to clear criteria• Timely (self and peer)Timely (self and peer)• Helps identify next stepsHelps identify next steps• Followed throughFollowed through

Evaluation ToolsEvaluation ToolsThe mustsThe musts

3.3. Clear purpose – need to be clear when Clear purpose – need to be clear when student work is not included in the student work is not included in the gradegrade Is it diagnostic?Is it diagnostic?

Used to assess where students areUsed to assess where students are

Is it formative?Is it formative? Used to guide improvementUsed to guide improvement

Is it summative?Is it summative? Used to record achievement for reporting Used to record achievement for reporting

purposespurposes

Evaluation ToolsEvaluation Tools

The mustsThe musts

4.4. Student involvement in processStudent involvement in process Choice of demonstrations of Choice of demonstrations of

learning – this way or that waylearning – this way or that way Design of evaluation tools - rubricsDesign of evaluation tools - rubrics Record keeping – self reflections of Record keeping – self reflections of

learning, portfolios of worklearning, portfolios of work Communication – student-led Communication – student-led

parent-teacher conferencesparent-teacher conferences

Evaluation ToolsEvaluation Tools

The mustsThe musts

55.. FairnessFairness Not all kids are the same!Not all kids are the same!

Different types of learners require Different types of learners require different kinds of evidencedifferent kinds of evidence

Need enough evidence to “convict Need enough evidence to “convict them of learning.”them of learning.” The more inconsistent the demonstrations The more inconsistent the demonstrations

of learning, the more evidence you need.of learning, the more evidence you need.

““Working Inside the Black Working Inside the Black Box”Box”

•FeedbackFeedback•QuestioningQuestioning•Self and Peer AssessmentsSelf and Peer Assessments•Use of Summative Tasks Use of Summative Tasks

FormativelyFormatively

Pause to reflectPause to reflect

A diagnostic test at the beginning of the A diagnostic test at the beginning of the unit shows students with a wide range of unit shows students with a wide range of ability and understanding.ability and understanding.

• To learn well we believe, students need :To learn well we believe, students need :– ChallengeChallenge– SuccessSuccess– ConnectionConnection– FitFit

• How do we differentiate the curriculum for How do we differentiate the curriculum for all students for them to experience all students for them to experience success?success?

(differentiation includes accommodations and modifications)(differentiation includes accommodations and modifications)

Learners differ in:Learners differ in:

• How they learnHow they learn• What they care about learningWhat they care about learning• What they bring to the learning What they bring to the learning

experienceexperience• Speed of learningSpeed of learning

Clarifying DifferentiationClarifying Differentiation

• Expectations (content standards – Expectations (content standards – THE WHAT) are fixedTHE WHAT) are fixed

• Assessment tasks must match Assessment tasks must match content standards.content standards.

• All kids should have the opportunity All kids should have the opportunity to work with the BIG IDEAS.to work with the BIG IDEAS.

Differentiation flowDifferentiation flow

• Performance standards (THE HOW) Performance standards (THE HOW) can be altered by :can be altered by :

– How students How students accessaccess content content– The types of student The types of student activitiesactivities– Student Student demonstrationsdemonstrations of learning of learning

““Human beings differ in their talents;Human beings differ in their talents;

to teach them you have to start where to teach them you have to start where they are.”they are.” -Confucius-Confucius

Teaching for Teaching for UnderstandingUnderstanding

““Teaching for Teaching for understanding requires the understanding requires the

student to rethink student to rethink something they already something they already

know.”know.”

Grant WigginsGrant Wiggins

““Many kids have Many kids have abdicated the abdicated the responsibility of responsibility of thinking about thinking about anything in school.”anything in school.”

Cris TovaniCris Tovani

““When the teacher gives us When the teacher gives us all the answers, it makes all the answers, it makes me feel sleepy.”me feel sleepy.”

John Macartney, age 7John Macartney, age 7

Knowledge

Application

Rigour

Relevance

EvaluationEvaluation

SynthesisSynthesis

AnalysisAnalysis

ApplicationApplication

ComprehensionComprehension

MemorizeMemorize

Thinking

- W. Daggett

Knowledge in

one disciplin

e

Real world

predict

ableRea

l wor

ld

unpredict

able

Apply in

disciplin

e

Apply acr

oss

disciplin

es

Knowledge

Application

Integration

Acquisition Application

Adaptation

bits of

knowledge

solve new problems,

design solutions

automaticity to

solve problemscomplex thinking

Thinking

Knowledge

Application

Thinking

Application

Thinking

Knowledge

SAY

COMMUNICATION

WRIT

E DO

A Case Study - CLU 3M A Case Study - CLU 3M 5 Strands5 StrandsEnduring Understandings:Enduring Understandings:

Legal issues affect lives.Legal issues affect lives.Analyzing legal issues invokes argument.Analyzing legal issues invokes argument.Justice systems are important in civilized Justice systems are important in civilized democracies.democracies.

Essential Questions:Essential Questions:Are laws necessary?Are laws necessary?Do we need a judicial system?Do we need a judicial system?Is the judicial system fair to all?Is the judicial system fair to all?Do jails make sense in a civilized society?Do jails make sense in a civilized society?Should young offenders go to jail?Should young offenders go to jail?If we increased the number of police officers, If we increased the number of police officers, courts and jails would society become more courts and jails would society become more civilized?civilized?

A Case Study - CLU 3M A Case Study - CLU 3M

o 30% Summative30% Summative

o Case AnalysisCase Analysiso Legal Research Paper Legal Research Paper o Mock TrialMock Trial o DebateDebate

o Exam??Exam??

A Case Study - CLU 3MA Case Study - CLU 3M- page 224 - page 224

• Strand: Criminal Law and ProceduresStrand: Criminal Law and Procedures• 4 Overall expectations4 Overall expectations

– 5 Sub-strands5 Sub-strands• What is a crime?What is a crime?• Pre-trial proceduresPre-trial procedures• Trial ProceduresTrial Procedures• SentencingSentencing• Criminal Law and Young PeopleCriminal Law and Young People

Crimes123

Pre-trial123

Trial Procedures1234

YCJA123

Sentencing1234

Map of Court

Criticalthinking

Mini role playCourt players

AssignmentCase to Court

Strand: Methods of Legal Inquiry and Communication

M O C KT R I A L

CASE STUDIES DEBATEK - T

K - A K - A K - TT - CK

C - A - T

K - A - T - C

CLU 3M Strand: CLU 3M Strand:

Criminal Law and Criminal Law and ProceduresProcedures

Knowledge

Application

Thinking

M O C KT R I A L

Map of Court

Criticalthinking

•Mini role play -Court players

•Assignment -Case to Court

Evaluation

Synthesis

Analysis

Application

Comprehension

Memorize

CASE STUDIES

DEBATE

CURRENT EVENT

Knowledge in

one disciplin

e

Real world

predict

ableRea

l wor

ld

unpredict

able

Apply in

disciplin

e

Apply acr

oss

disciplin

es

Pause to reflectPause to reflect

How can the Mock Trial task be How can the Mock Trial task be differentiated so that students differentiated so that students invest the most that they can to invest the most that they can to grow the most that they can to grow the most that they can to include:include:– ESL students?ESL students?– LD students?LD students?– Gifted students?Gifted students?

(differentiation includes accommodations and modifications)(differentiation includes accommodations and modifications)

DifferentiationDifferentiation• ESLESL

– Court visitCourt visit– Lower level reading materialLower level reading material– Graphic organizersGraphic organizers– Visual images (TV, movie scenes, CD ROMs)Visual images (TV, movie scenes, CD ROMs)– Government websites for informationGovernment websites for information– Connections to home countryConnections to home country– Translation of assignmentTranslation of assignment– Write / video for someone at home (different Write / video for someone at home (different

target audience)target audience)– ……

DifferentiationDifferentiation• LDLD

– Court visitCourt visit– Lower level reading materialLower level reading material– Graphic organizersGraphic organizers– Visual images (TV, movie scenes, CD ROMs)Visual images (TV, movie scenes, CD ROMs)– Government websites for informationGovernment websites for information– Peer supportPeer support– Structured worksheets that chunk materialStructured worksheets that chunk material– Write / video for someone in grade 6 (different Write / video for someone in grade 6 (different

target audience)target audience)– Provide only part of the rubric at a timeProvide only part of the rubric at a time– ……

DifferentiationDifferentiation• GiftedGifted

– Meet with lawyer, law professorMeet with lawyer, law professor– Higher level reading materialHigher level reading material– Case analysisCase analysis– Twin scenariosTwin scenarios– Visual images (TV, movie scenes, CD ROMs)Visual images (TV, movie scenes, CD ROMs)– Government websites for informationGovernment websites for information– Peer mentoring (learn one - teach one)Peer mentoring (learn one - teach one)– More difficult casesMore difficult cases– Analyse a case for a lawyer (different target Analyse a case for a lawyer (different target

audience)audience)– Write a newspaper account of the trial.Write a newspaper account of the trial.

DifferentiationDifferentiation

““When curriculum is rich and When curriculum is rich and focused, you can differentiate it in focused, you can differentiate it in your sleep.”your sleep.”

-Carol Ann Tomlinson-Carol Ann Tomlinson

Lesson Plans Lesson Plans as Motivational Plansas Motivational Plans

Lesson plans must first Lesson plans must first be motivational plans!be motivational plans!

Building Capacity

Professional Development• Professional reading

• Book clubs• Workshops• Conferences• Professional memberships

Teacher Performance Appraisal• Opportunity for professional growth• Teacher - Administrator dialogue• Annual Learning Plan• Focus on learning behaviour of students

Standards of Practiceof the Teaching Profession• Commitment to student learning• Professional knowledge• Teaching Practice• Leadership and Community• Ongoing Professional Learning

Professional Communities of Practice• Common planning time• Common student performance tasks• Common assessment tools• System of Interventions for students who are not learning

Covey’s Seventh Habit …

Covey’s Seventh Habit …

Sharpen the Saw!

Task DescriptionTask Description

• Grade 11 students will teach one Grade 11 students will teach one overall expectation to a grade 10 overall expectation to a grade 10 student in a given time (90 – 120 student in a given time (90 – 120 minutes) and create a portfolio minutes) and create a portfolio that displays their planning, that displays their planning, implementation and reflection of implementation and reflection of the process.the process.

Portfolio ItemsPortfolio Items1.1. The overall expectation with translation to The overall expectation with translation to

student’s own language.student’s own language.2.2. The link of the grade 11 expectation to The link of the grade 11 expectation to

expectations learned in earlier grade(s).expectations learned in earlier grade(s).3.3. Connections/ Applications of expectation in the Connections/ Applications of expectation in the

real world (authentic use of the expectation or real world (authentic use of the expectation or concept) find adult who uses knowledge or skill concept) find adult who uses knowledge or skill and if possible interview them. Include Q&A in and if possible interview them. Include Q&A in portfolio.portfolio.

4.4. Planned learning experiences for the event: Planned learning experiences for the event: (lecture notes; use of models; sample (lecture notes; use of models; sample problems/tasks with solutions; simulations; use of problems/tasks with solutions; simulations; use of manipulatives or technology)manipulatives or technology)

5.5. Feedback statements that support learning Feedback statements that support learning (strengths, areas for improvement, next steps)(strengths, areas for improvement, next steps)

Portfolio ItemsPortfolio Items6.6. Anticipated student misunderstandings and Anticipated student misunderstandings and

strategies to intervene.strategies to intervene.7.7. A summative test/task for grade 10 student A summative test/task for grade 10 student

(with marking scheme or rubric) that will check (with marking scheme or rubric) that will check for grade 10 student understanding.for grade 10 student understanding.

8.8. Grade 10 student rating scale of grade 11 Grade 10 student rating scale of grade 11 student’s performance (preparation, student’s performance (preparation, communication skills (oral and written), support communication skills (oral and written), support for learning (feedback, use of for learning (feedback, use of models/manipulatives/technology), models/manipulatives/technology), understanding of expectation)understanding of expectation)

9.9. Grade 11 student self-reflection Grade 11 student self-reflection (metacognition). Includes: rating statements, (metacognition). Includes: rating statements, answers to leading questions, )answers to leading questions, )

Knowledge

Application

Thinking

1. Translate expectation

2. Link to expectation in earlier grade

Evaluation

Synthesis

Analysis

Application

Comprehension

Awareness

Knowledge in

one disciplin

e

Real world

predict

ableRea

l wor

ld

unpredict

able

Apply in

disciplin

e

Apply acr

oss

disciplin

es

3. Connect to real world

5. Feedback statements

6. Anticipate student misunderstandings4. Plan learning

experiences

7. Lesson delivery

9. Summative task/test

10. Student self reflection

8. Monitor learning (feedback/intervention)

““Never doubt that a small Never doubt that a small group of thoughtful and group of thoughtful and committed citizens can committed citizens can change the world. In change the world. In fact, it has never fact, it has never happened any other happened any other way.”way.”

Margaret MeadMargaret Mead

““Something has Something has to be done and to be done and it is absolutely it is absolutely pathetic that pathetic that we are the we are the ones who have ones who have to do it.”to do it.”

Jerry GarciaJerry Garcia