F RENCH TO E NGLISH T RANSLATION Jennifer Valcke [email protected].
1 Innovation in Assessment and Evaluation Prof. dr. Martin Valcke mvalcke/CV/CVMVA.htm Ghent...
-
Upload
stanley-marsh -
Category
Documents
-
view
215 -
download
0
Transcript of 1 Innovation in Assessment and Evaluation Prof. dr. Martin Valcke mvalcke/CV/CVMVA.htm Ghent...
1
Innovation in Assessment and
Evaluation
Prof. dr. Martin Valckehttp://allserv.ugent.be/~mvalcke/CV/CVMVA.htm
Ghent University
Maputo July 2011
Structure
• Advance organizer
• Traditional approaches
• Trends in assessment & evaluation
• Assumptions in relation to these trends
• Research
• Practices
• Conclusions2
Advance organizer
5
Vorm/ manier van overbrengenVorm/ manier van overbrengen3. oogcontact3. oogcontact
Bert probeert het anders:
Denk…
Transmission of your message3. Eye Contact
6
Kan ik het wel leren?Kan ik het wel leren?
Iedereen kan het leren
door ‘oefening’ + ‘feedback’
Luister maar eens naar de Getuigenis van Bert (student handelswetenschappen)(linkermuisklik in beeld)
1. Geloof in verandering!
2. Doe het ‘stap voor stap’ : 1 of 2 aandachtspunten tegelijk
3. Reflecteer zoals Bert: denk na over sterke punten en over veranderingspunten
Wat kunnen we leren van Bert? Denk even na en klik pas voor de antwoorden!
Can I learn this?
Critical issues
• Validity of evaluation approach in view of assessment of skills and complex knowledge
• Fant et al., (1985)• Rating scales, daily logs, anecdotal records, behavior
coding, and self-assessment for evaluating student teachers.• Oral examinations, portfolio assessment, central assessment
centres, 360° assessment
– …
10
Recent developments
Individual learner
Group learner
External institution
Teachers Expert eacher
Assessment systemInstitutional level
Recent developments
• Stronger focus on “consequential validity”of measurement (Gielen, Dochy & Dierick, 2003)
• Stronger emphasis on feedback value of assessment
• What is the “learning potential” of the assessment approach
12
Recent developments
• Stiggins (1987): performance assessment
• Performance assessment is expected to be geared in a better way to assess complex behavior in medical, legal, engineering, … and educational contexts (Sluijsmans, et.al., 2004).
13
Self- and peer assessment
• Learn about your own learning process.
• Schmitz (1994): “assessment-as-learning”.
• ~ self corrective feedback
17
• See experiential learning cycle of Kolb.
• Boekaerts (1991) self evaluation as a competency.
• Development of metacognitive knowledge and skills (see Brown, Bull & Pendlebury, 1998, p.181).
• Freeman & Lewis (1998, p.56-59): developing pro-active learners
18
Learning to evaluate
• Develop checklists
• Give criteria
• Ask to look for quality indicators.
• Analysis of examples good and less good practices: develop a quality “nose”
22
Learning to evaluate
• Freeman & Lewis (1998, p.127) :• Learner develops list of criteria.• Pairs of learners compare listed criteria.• Pairs develop a criterion checklist.• Individual application of checklist.• Use of checklist to evalute work of other learner.• Individual reworks his/her work.• Final result checkeed by teacher and result compared to learner
evaluation.• Pairs recheck their work on the base of teacher feedback.
23
Learning to evaluate
• Peer evaluation is not the same as Peer grading
• Final score is given by teacher
• Part of score could build on accuracy of self/peer evaluation and self-correction
24
Rubrics
• Rubrics focus on the relationship between competencies, criteria, and indicators and are organized along mastery levels (Morgan, 1999).
27
Rubrics• Rubric: scoring tool for a qualitative assessment of the
quality level of an authentic or complex activity– A rubric builds on criteria, enriched with a scale to indicate a
mastery level.
– For each level, standards are indicated that reflect this level.
– A rubric dictates both teacher and student what is concretely expected.
– Rubrics are used for “high stake assessment” and “formative assessment” (Arter & McTighe, 2001; Busching, 1998; Perlman, 2003).
• Rubrics focus on the relationship between competencies, criteria, are organized along mastery levels (Morgan, 1999).
29
Rubrics: indicator-based assessment
• Assessment objective– Criteria
• Enriched with indicators in terms of observable behavior
• Limited number of indicators
30
Critical thinking rubric
31
http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html
Assumptions about rubrics
• Larger consistency in scores (reliability).
• More valid assessment of complex behavior.
• Positive impact on related learning process.
32
Critical issues• Adoption of this assessment approach is marred by
teacher beliefs about nature of evaluation (see e.g., Chong, Wong, & Lang, 2004);
• Also student beliefs (Joram & Gabriele, 1998)• Validity of the criteria and indicators (Linn, 1990), • Reliability of performance evaluation, e.g., when
multiple evaluators assess and score performance (Flowers & Hancock, 2003).
33
Research about rubrics use• Review article 75 studies about rubrics usage• Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability,
validity and educational consequences. Educational Research Review, 2, 130–144.– (1) the reliable scoring of performance assessments can be enhanced by the
use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training;
– (2) rubrics do not facilitate valid judgment of performance assessments per se. However, valid assessment could be facilitated by using a more comprehensive framework of validity;
– (3) rubrics seem to have the potential of promoting learning and/or improve instruction. The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.
34
Conditions effective usage
• Check frame of reference for the rubric: tasks, objectives
• Train the users
• Use multiple assessors: interrater usage
• Developed by teacher and/or students!
35
Development of rubrics
• Choose quality criteria: 4 to 15 statements describing the nature of a criterion
• Determine bandwidth to judge differences in quality related to the criterion: e.g., 0-5 or qualitative descriptors
• Elaborate descriptors for each bandwidth level: concrete operational terms
• Start from available student work!
36
Rubrics• Rubric: scoringstool voor een kwalitatieve inschatting van het niveau
van een authentieke of complexe activiteit.– Een rubriek bouwt verder op criteria die verrijkt worden met een schaal
waarop beheersingsniveaus zijn aangegeven
– Per beheersingsniveau worden standaarden aangegeven die die niveau weerspiegelen.
– Een rubric geeft zowel voor de lesgever als de student aan wat concreet verwacht wordt.
– Rubrics worden voor “high stake assessment” gebruikt en voor “formatieve toetsing” (in functie van leren).(Arter & McTighe, 2001; Busching, 1998; Perlman, 2003).
• Rubrics focus on the relationship between competencies-criteria, and indicators and are organized along mastery levels (Morgan, 1999).
40
Performance assessment
• Rubrics focus on the relationship between competencies-criteria, and indicators and are organized along mastery levels (Morgan, 1999).
41
Aanpak ontwikkeling rubric
• Kies criteria voor verwacht gedrag– 4 tot 15 statements die criterium beschrijven
• Bepaal bandbreedte die verschil in bereiken criterium weergeven– Bijv. 0-5 of kwalitatieve termen
• Werk een beschrijving uit voor elke waarde in de bandbreedte– Concreet observeerbare/vaststelbare kwalificaties
42
More information?• Overview of tools, examples, theory, bacdkground, research:
http://school.discoveryeducation.com/schrockguide/assess.html
• Critical thinking rubrics: http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html
• Rubric generators: http://www.teach-nology.com/web_tools/rubrics/
• Interesting rubric sites: http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm
• Rubric APA research paper: http://web.njit.edu/~ronkowit/teaching/rubrics/samples/rubric_apa_research.pdf
• K12 examples: http://school.discoveryeducation.com/schrockguide/assess.html
• General intro and overview:http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm
45
Statements about evaluation
• Learners should be trained to develop themselves such rubrics.
• Staff should collaborate in developing formal assessment and summative assessment rubrics
• Rubrics will help staff to be more concrete as to their teaching and learning focus
47